ifference between revisions of "EMO:Input Data"

From EMO Wiki
Jump to navigation Jump to search
 
(35 intermediate revisions by 2 users not shown)
Line 6: Line 6:
 
*'''''Demand data
 
*'''''Demand data
 
*'''''Transmission data
 
*'''''Transmission data
 +
*'''''Reserves data
 
*'''''Static (user-defined) data'''''
 
*'''''Static (user-defined) data'''''
  
Line 15: Line 16:
  
 
=== Loading Generation Data ===
 
=== Loading Generation Data ===
−
Historical Generation data can be loaded from an online database (see '''''Connecting to a Networked Database'''''). Historical or user-created Generation data can be loaded into '''''Powerflow Case Definitions''''' from file as a '''''generation profile'''''.
+
Historical Generation data can be loaded from an online database (see '''''[[EMO:Connecting_to_a_Database_Server|Connecting to a Networked Database]]'''''). Historical or user-created Generation data can be loaded into '''''Powerflow Case Definitions''''' from file as a '''''generation profile'''''.
  
 
==Price Data==
 
==Price Data==
Line 21: Line 22:
  
 
=== Loading Historical Price Data ===
 
=== Loading Historical Price Data ===
−
Historical Price data can only be loaded into ''EMarketOffer'' through direct connection to an online database (see '''''Connecting to a Networked Database''''').
+
Historical Price data can only be loaded into ''EMarketOffer'' through direct connection to an online database (see '''''[[EMO:Connecting_to_a_Database_Server|Connecting to a Networked Database]]''''').
  
 
== Offers Data ==
 
== Offers Data ==
 
Historical Offers data is used:
 
Historical Offers data is used:
−
*As information-only in the '''''Units Information Grid'''' for the '''''Market Data Source'''''
+
*As information-only in the '''''[[EMO:Units Information Grid|Units Information Grid]]''''' for the '''''Market Data Source'''''
 
*As input into the seven-day '''''Offers Profile''''' in '''''forecast Definitions'''''
 
*As input into the seven-day '''''Offers Profile''''' in '''''forecast Definitions'''''
 
*As input into '''''Scenario Data Sources''''' based on historical Trading Periods (when available)
 
*As input into '''''Scenario Data Sources''''' based on historical Trading Periods (when available)
Line 39: Line 40:
 
*Offers profile files (Forecast Definitions only)
 
*Offers profile files (Forecast Definitions only)
  
−
Pre-formatted Offers files must use the same formatting used by the COMIT service for providing Offers data and be named in the following manner:
+
Pre-formatted Offers files must use the same formatting used by the market for providing Offers data and be named in the following manner:
  
 
:'''Offersyyyymmdd.csv
 
:'''Offersyyyymmdd.csv
Line 49: Line 50:
 
:''''dd'''' is the numerical 2 digit day (e.g. 08)
 
:''''dd'''' is the numerical 2 digit day (e.g. 08)
  
−
You can download Offers files directly from COMIT and use them without further editing (once renamed).
+
You can download Offers files directly from http://www.electricityinfo.co.nz and use them without further editing (once renamed).
  
 
----
 
----
Line 60: Line 61:
 
*Restrict the number of offering Generation Units to less than those actually offering on the day
 
*Restrict the number of offering Generation Units to less than those actually offering on the day
  
−
The formatting for the Offers files can be found in the '''[[EMO:Appendix|Appendix]].'''
+
The formatting for the Offers files can be found in '''''[[EMO:Offers File Format|Offers File Format]].'''''
  
−
Offers profile files can also be used. An Offers profile file uses a proprietary format designed to capture a complete seven-day profile suitable for importing into a Forecast Definition. The format of this file can be found in the '''[[EMO:Appendix|Appendix]].'''
+
Offers profile files can also be used. An Offers profile file uses a proprietary format designed to capture a complete seven-day profile suitable for importing into a Forecast Definition. The format of this file can be found in '''''[[EMO:Offers Profile File Format|Offers Profile File Format]].'''''
  
 
===Using Offers with Forecast Definitions===
 
===Using Offers with Forecast Definitions===
Line 68: Line 69:
  
 
The following points should be noted:
 
The following points should be noted:
−
*Where a ''''Forecast Effective Date Range''''' extends beyond seven days, Offers from the loaded profile will be re-used (by corresponding day of the week) from day eight onward
+
*Where a '''''Forecast Effective Date Range''''' extends beyond seven days, Offers from the loaded profile will be re-used (by corresponding day of the week) from day eight onward
 
*Where Offers for fewer than seven days are loaded into a Forecast Definition , any periods in the Effective Date Range not covered by the loaded Offers will fail to dispatch
 
*Where Offers for fewer than seven days are loaded into a Forecast Definition , any periods in the Effective Date Range not covered by the loaded Offers will fail to dispatch
 
*New Offers can be loaded over the top of existing Offers. examples of how this might be useful include:
 
*New Offers can be loaded over the top of existing Offers. examples of how this might be useful include:
Line 77: Line 78:
 
== Demand Data ==
 
== Demand Data ==
 
Historical Demand data is used:
 
Historical Demand data is used:
−
*As input into calculating the Powerflows in the '''''Market Data Source'''''
+
*As input into calculating the Powerflows in the '''''[[EMO:Data_Sources#Market_Mode|Market Data Source]]'''''
−
*As input into the seven-day Demand profile for '''''Forecast Definitions''''' and '''''Powerflow Cases'''''
+
*As input into the seven-day Demand profile for '''''[[EMO:The Forecast Definition Window|Forecast Definitions]]''''' and '''''[[EMO:Powerflow_Analysis|Powerflow Cases]]'''''
  
 
===Which Demand data to use with ''EMarketOffer?''===
 
===Which Demand data to use with ''EMarketOffer?''===
−
For best results ''EMarketOffer's'' database connection should be configured to use reconciled SPD Daily Demand data (available to subscribed COMIT users) as this most closely matches the historical Offers used in final dispatch.  However, ''EMarketOffer'' can be configured to work with SCADA or other Demand data.  Where SCADA data is used the net final Load at each Market Node should also be used.
+
For best results ''EMarketOffer's'' database connection should be configured to use reconciled SPD Daily Demand data (available to subscribed WITS users) as this most closely matches the historical Offers used in final dispatch.  However, ''EMarketOffer'' can be configured to work with SCADA or other Demand data.  Where SCADA data is used the net final Load at each Market Node should also be used.
  
 
===Loading Historical Demand Data===
 
===Loading Historical Demand Data===
Line 90: Line 91:
  
 
===Pre-formatted files===
 
===Pre-formatted files===
−
Pre-formatted Demand files must comply with ''EMarketOffer's'' Demand file format (see '''[[EMO:Appendix|Appendix''') and named in the following manner:
+
Pre-formatted Demand files must comply with ''EMarketOffer's'' Demand file format (see '''''[[EMO:Demand File Format|Demand File Format]]''''') and named in the following manner:
  
 
:'''Loadyyyymmdd.csv'''
 
:'''Loadyyyymmdd.csv'''
Line 109: Line 110:
 
*Restrict the number of Demand Nodes to less than those actually drawing down power on the day
 
*Restrict the number of Demand Nodes to less than those actually drawing down power on the day
  
−
The formatting for the Demand files can be found in the [[EMO:Appendix|Appendix]]
+
The formatting for the Demand files can be found in '''''[[EMO:Demand File Format|Demand File Formats]]'''''.
  
 
===Demand Profile Files===
 
===Demand Profile Files===
−
A Demand profile file uses a proprietary format designed to capture a complete seven-day profile suitable for importing into a Forecast Definition or Powerflow Case.  The format of this file can be found in the '''[[EMO:Appendix|Appendix]].'''
+
A Demand profile file uses a proprietary format designed to capture a complete seven-day profile suitable for importing into a Forecast Definition or Powerflow Case.  The format of this file can be found in '''''[[EMO:Demand Profile File Format|Demand Profile File Formats]]'''''.
  
 
===Using Demand with Forecast Definitions and Powerflow Cases===
 
===Using Demand with Forecast Definitions and Powerflow Cases===
Line 125: Line 126:
  
 
== Transmission System Data ==
 
== Transmission System Data ==
−
Transmission System data supplied by the System Operator through the EM6 and COMIT information services is used to:
+
''EMarketOffer'' can create a set of grid data (circuit details, line re-ratings and constraints) at run time from one of four sources:
−
*Calculate Powerflows for the '''''Market Data Source'''''
+
#Historical transmission system data files provided by Energy Link
−
*Define the Transmission System configuration in Forecast and '''''Powerflow Case Definitions'''''
+
#Transmission system files sourced through the EM6 web site operated by Transpower division EMS
 +
#In-house databases
 +
#The files available each business day from the Energy Link web site (for grids from 22-Jul-09 onward)
  
−
''EMarketOffer'' requires the following Transmission System data:
+
'''Of the sources above, 1 - 3 are legacy data sources and we have stopped supporting them as the data available from EM6 is incomplete for the purposes required in ''EMarketOffer'', and because we have gone to great effort to provide daily grid and reserves files that include all data reuired to solve the market by half hour.  Only the daily updated grid files from 22-Jul-09 are now supported (though the other types of existing grid data can still be used in the meantime).  The daily updated files also include full reserves data.'''
−
*Network configuration (default values for base grid)
 
−
*Circuit electrical properties
 
−
*Circuit Re-ratings
 
−
*Group Constraint descriptions
 
  
−
By preference ''EMarketOffer'' automatically loads Transmission System data from any files found in the '''''EM6/COMIT sub-folders''''' on start-up (unless the '''Use Grid File Inputs box''' in the '''''Source Data Administration Window''''' is checked). A full 'map' of the historical and expected network is created, allowing the exact configuration of the Transmission System to be determined for any Trading Period covered by the file data. Where Transmission System data must be obtained from an online database, the 'map' is built up period by period in response to requests made through the Presentation Interface.
+
On startup ''EMarketOffer'' checks the Energy Link web site for the new grid and reserves data and automatically downloads files if you are not up to date.
  
−
===Base Transmission System===
+
The '''Connect Grid Data Tables''' check box in the '''''Database Connection Settings''''' window can be checked if you wish to use grid data from your own database, subject to the following:
−
The Branch_Data file (obtainable as a downloaded from the EM6 service) defines the properties of all Circuits (including Summer and Winter ratings) and how they are connected. Both historical and expected changes to these properties are stored in the file. Files downloaded from EM6 should be placed in the ''Grid\EM6\BaseGrid'' sub-folder named according the the following convention (the same as used by EM6):
+
----
 +
'''WARNING!''' ''EMarketOffer'' will ignore any database links for grid data if grid data can be obtained from files.  To use grid data from your own database you must delete any grid data files that are downloaded by ''EMarketOffer'' on startup.
 +
----
 +
 
 +
----
 +
'''DISCLAIMER''':  Energy Link takes a great deal of care in extracting grid and reserves data from the SPD daily final pricing case data files provided on the Electricity Authority's web site.  However, we are totally unsupported in this endeavour and cannot in anyway guarantee the accuracy of the data we provide to ''EMarketOffer'' users. We provide the data in good faith and will not be held liable for the use of the data in ''EMarketOffer'' or in any other application, for the accuracy of the data, or for any special or consequential damages or losses resulting in any way whatsoever from the use of the data.
 +
----
 +
 
 +
The following descriptions of grid data relate only to the data which is updated on Energy Link's web site each business day.  For assistance with earlier versions of grid data (for all periods up to and including 15-Jul-09) you need to contact Energy Link or refer to an earlier ''EmarketOffer'' User Manual.
  
−
:'''Branch_Data_ddmmyyyy.csv'''
 
  
−
Where:
+
The SPD daily final pricing case data files are parsed by Energy Link each bussiness day to extract grid and reserves data.  Grid data is stored in files with names:
−
*''''Branch_Data'''' is the text 'Branch_Data'
+
*BaseGrid_yyyymmdd_yyyymmdd.csv contain the base data for circuits, transformers and other key grid elements by half hour
−
*''''ddmmyyyy'''' is the date the file was downloaded from EM6
+
*Equations_yyyymmdd_yyyymmdd.csv contain the equation constraints by half hour
 +
*Reratings_yyyymmdd_yyyymmdd.csv contain circuit and transformer outages and reratings by half hour
  
−
On start-up ''EMarketOffer'' uses the Branch_Data file with the latest date. Where no Branch_Data file exists, ''EMarketOffer'' attempts to access data through a database connection. Branch_Data files will always be used preferentially to source Transmission System data regardless of whether a current database connection for the data has been defined or not.
+
In each case, the two dates show the range of dates covered by each file (from date first, to date second). The files are added to day by day but do not contain more than one year of data each.
  
−
''EMarketOffer'' will fail to operate if no Transmission System data can be found from any source at startup.
+
In the same folder as the files above, you will also find VLRs_yyyymmdd_yyyymmdd.csv files which contain the VLR ratings for a small number of circuits which are currently being trialled:  this data comes from Transpower's web site and are checked periodically.
  
−
----
+
== Reserves Data ==
−
NOTE: You will need to download new branch data files from EM6 from time to time. These should be named according to the above convention and placed in the ''EM6\BaseGrid'' sub-folder.  It is not necessary to remove previous Branch_Data files as only data from the file with the latest date will be used.
+
Reserves data is collected from the same source as the grid data, and is stored in two sets of files:
−
----
+
*Reserves_yyyymmdd_yyyymmdd.csv
 +
*RiskSetters_yyyymmdd_yyyymmdd.csv
  
−
'''WARNING!''' Files provided by the system operator no longer accurately reflect historical system changes prior to 2002.  If you wish to model the pre-2002 system you should use Transpower branch data files that were created close to the time you are interested in analysing. ''This will require removing Branch_Date files with later create dates from the EM6\BaseGrid sub-folder.''
+
In this case the files store reserves data for up to one month at a time.
  
−
===Security Constraints===
+
== Static (User-Defined) Data ==
−
Files, in CSV format, of historical and expected Security Constraints are provided through both the EM6 and COMIT free to air services. The following table summarises the files that can be processed by ''EMarketOffer:''
+
''EMarketOffer'' uses static data to calculate or organise the dynamic data produced by the Run (see the Circuit entity example below). Static data tends to remain unchanged between the Trading Periods contained in a Run,
  
 
{|class="wikitable"
 
{|class="wikitable"
−
! Source !! Data Content !! EMO Location !! EMO File Name !! Comment
+
! Entity !! Description !! Static Data !! Dynamic Data
 
|-
 
|-
 +
! Circuit
 
|
 
|
−
:EM6
+
:A branch (Circuit) conducting power on the NZ Transmission System
 
|
 
|
−
:Security Constraints
+
*Name
 +
*From Node
 +
*To Node
 +
*Contributes to HVDC (Yes/No)
 +
*Resistance
 +
*Reactance
 +
*Capacity (Summer/Winter)
 
|
 
|
−
:Input\Grid\EM6\Reratings
+
*Powerflow
−
|
+
*Price Separation
−
:branch_constraints_ddmmyyyy_ddmmyyyy.csv
+
*Losses
−
|
+
*Loading
−
:Contains all security constraints (circuit re-ratings) in effect on any Trading Period with the date range contained in the file name (naming as used by EM6).
 
−
|-
 
−
|
 
−
:EM6
 
−
|
 
−
:Security Constraints
 
−
|
 
−
:Input\Grid\EM6\Equations
 
−
|
 
−
:constraints_ddmmyyyy_ddmmyyyy.csv
 
−
|
 
−
:Contains all security constraints (equations) in effect on any Trading Period with the date range contained in the file name (naming as used by EM6).
 
−
|-
 
−
|
 
−
:EM6
 
−
|
 
−
:Security Constraints
 
−
|
 
−
:Input\Grid\EM6\Constraints
 
−
|
 
−
:constraints_change_ddmmyyyy_ddmmyyyy.csv
 
−
|
 
−
:Contains all security constraints in effect on any Trading Period with the date range contained in the file name (naming as used by EM6).
 
−
|-
 
−
|
 
−
:COMIT
 
−
|
 
−
:Security Constraints
 
−
|
 
−
:Input\Grid\COMIT
 
−
|
 
−
:constraints_change_ddmmyyyy_ddmmyyyy.csv
 
−
|
 
−
:Contains all security constraints in effect on any Trading Period with the date range contained in the file name (naming as used by EM6).
 
−
|-
 
−
|
 
−
:Ellsoft
 
−
|
 
−
:Security Constraints
 
−
|
 
−
:Input\Grid\Ellsoft
 
−
|
 
−
:ellsoft_constraints_ddmmyyyy_ddmmyyyy.csv
 
−
|
 
−
:Contains an Ellsoft generated file containing all security constraints in effect on any Trading Period with the date range contained in the file name (naming as used by EM6).
 
 
|}
 
|}
  
−
'''Warning!''' Ellsoft provides Constraint files to cover the historical period from the beginning of the market to a date subsequent to the mass circuit renaming exercise conducted by Transpower on the 8th December 2007.  The file has been constructed using historical TPIX and recent EM6 data files and for the period 1 Jan 2007 to 31 May 2008 has used the maximum capacities from arcflow data to construct realistic branch re-ratings. This file is intended to overcome inconsistencies in constraint source data. It is provided without guarantee of accuracy; you are advised to independently confirm any data found in this file.
+
Modelled entities are defined by static data. Some entities are global, i.e. common for all Definitions, and their associated static data must be pre-defined (e.g. '''''Companies''''' and '''''Regions''''') Most modelled entities, and their static data, are created for each new Definition (e.g. Generators or Nodes).
  
 +
----
 +
NOTE: To avoid time-consuming effort re-creating data in Definitions, Modelled entities, with their static data descriptions, can be imported into a Definition using ''EMarketOffer's'', '''''data loading functions''''', once loaded these entities can be modified, added to or removed with the Definition editor.
 +
----
  
−
===How ''EMarketOffer'' processes Transmission System data===
+
Functionality to maintain global static data can be accessed through the '''''[[EMO:The Edit Menu|Edit Menu]]''''' in the Main Window.
−
When constructing its internal Transmission System database, ''EMarketOffer'' first analyses the base grid and creates a period-by-period 'map' of the system which includes the tracking of all changes in rating, electrical properties and overall configuration.  Secondly the Branch re-ratings are applied 'over the top' of the initial map.  Finally the Group Constraints are examined and a parallel map of these Constraints is created.
 
  
−
Two rules are observed when loading Constraint data from file:
+
Static data in Definitions can be accessed through the '''''[[EMO:The Forecast Definition Window|Forecast Definition Window]]''''' or the '''''[[EMO:The Powerflow Case Definition Window|Powerflow Case Definition Window]]'''''.
  
−
#Data is loaded from folders in the following preference order:
 
−
##Input\Grid\Ellsoft
 
−
##Input\Grid\EM6\Constraints
 
−
##Input\Grid\COMIT
 
−
##Input\Grid\EM6\Equations and Input\Grid\EM6\Reratings
 
−
#Where files in different folders contain data for overlapping time periods, data will be taken from the file with the latest data, and from the first date in that file (see examples below)
 
  
−
[[File:Transmission system data.PNG|500px|thumb|centre|Figure 1 - Constraints File Handling Example]]
+
[[Main Page|Home]]

Latest revision as of 10:50, 10 February 2014

The input data types processed by EMarketOffer:

  • Generation Data
  • Price data
  • Offers data
  • Demand data
  • Transmission data
  • Reserves data
  • Static (user-defined) data

Generation Data

Generation data is used:

  • As information-only in theUnits Information Grid for the Market Data Source
  • To calculate Powerflows for the Market Data Source
  • To calculate Powerflows for the Powerflow Case Data Source

Loading Generation Data

Historical Generation data can be loaded from an online database (see Connecting to a Networked Database). Historical or user-created Generation data can be loaded into Powerflow Case Definitions from file as a generation profile.

Price Data

Historical Price data is used as information-only in the Units Information Grid for the Market Data Source

Loading Historical Price Data

Historical Price data can only be loaded into EMarketOffer through direct connection to an online database (see Connecting to a Networked Database).

Offers Data

Historical Offers data is used:

  • As information-only in the Units Information Grid for the Market Data Source
  • As input into the seven-day Offers Profile in forecast Definitions
  • As input into Scenario Data Sources based on historical Trading Periods (when available)

NOTE: EMarketOffer does NOT use Offers to calculate Powerflows for historical Trading Periods.

Loading Historical Offers Data

Historical Offers can be loaded either through:

  • direct connection to an online database
  • from pre-formatted Offers files
  • Offers profile files (Forecast Definitions only)

Pre-formatted Offers files must use the same formatting used by the market for providing Offers data and be named in the following manner:

Offersyyyymmdd.csv

Where:

'Offers' is the text 'Offers'
'yyyy' is the numerical 4 digit year (e.g. 2012)
'mm' is the numerical 2 digit month (e.g. 06)
'dd' is the numerical 2 digit day (e.g. 08)

You can download Offers files directly from http://www.electricityinfo.co.nz and use them without further editing (once renamed).

NOTE: Date format is important - the date in the file name is used to allocate offers data to specific week days. One Offers files cannot contain data for periods from two different calendar days.

Offers files are processed row by row without regard to overall integrity. This allows you to edit the raw Offers files prior to loading to:

  • Alter Offer Prices and Quantities
  • Restrict the number of offered periods to less than a full day
  • Restrict the number of offering Generation Units to less than those actually offering on the day

The formatting for the Offers files can be found in Offers File Format.

Offers profile files can also be used. An Offers profile file uses a proprietary format designed to capture a complete seven-day profile suitable for importing into a Forecast Definition. The format of this file can be found in Offers Profile File Format.

Using Offers with Forecast Definitions

EMarketOffer's Forecast Definitions are based on a weekly Offers Profile, organised by day of week (as opposed to date). Offers for no more than seven consecutive days can be loaded into a Forecast Definition.

The following points should be noted:

  • Where a Forecast Effective Date Range extends beyond seven days, Offers from the loaded profile will be re-used (by corresponding day of the week) from day eight onward
  • Where Offers for fewer than seven days are loaded into a Forecast Definition , any periods in the Effective Date Range not covered by the loaded Offers will fail to dispatch
  • New Offers can be loaded over the top of existing Offers. examples of how this might be useful include:
    • producing modified historical offers by loading a week profile from the database and then loading an edited and/or partial Offers file containing different data
    • Modifying a single generator's offering behaviour by loading an edited file containing Offers for only the Generation Unit and specific Trading Periods of interest leaving all previously loaded Offers (for this or any other generator) unchanged.
  • Offers files are loaded consecutively by date from earliest to latest. Loading more than seven-days of Offers data from file will cause over-writing of the weekly Offer profile. for example if ten days of offers are loaded from file the first three days will be lost.

Demand Data

Historical Demand data is used:

Which Demand data to use with EMarketOffer?

For best results EMarketOffer's database connection should be configured to use reconciled SPD Daily Demand data (available to subscribed WITS users) as this most closely matches the historical Offers used in final dispatch. However, EMarketOffer can be configured to work with SCADA or other Demand data. Where SCADA data is used the net final Load at each Market Node should also be used.

Loading Historical Demand Data

Historical Demand can be loaded into EMarketOffer either through:

  • direct connection to an online database,
  • from pre-formatted Demand files
  • from demand profile files

Pre-formatted files

Pre-formatted Demand files must comply with EMarketOffer's Demand file format (see Demand File Format) and named in the following manner:

Loadyyyymmdd.csv

Where:

'Load' is the text 'Load'
'yyyy' is the numerical 4 digit year (e.g. 2012)
'mm' is the numerical 2 digit month (e.g. 06)
'dd' is the numerical 2 digit day (e.g. 08)

NOTE: The date format is important. EMarketOffer uses the date in the file name to allocate the Demand data to specific days. One demand file cannot contain data for periods from more than one calendar day.

EMarketOffer processes the Demand files row by row and does not check overall integrity. This allows you to edit the raw files prior to loading into EMarketOffer to:

  • Alter Demand Quantities
  • Restrict the number of Demand periods to less than a full day
  • Restrict the number of Demand Nodes to less than those actually drawing down power on the day

The formatting for the Demand files can be found in Demand File Formats.

Demand Profile Files

A Demand profile file uses a proprietary format designed to capture a complete seven-day profile suitable for importing into a Forecast Definition or Powerflow Case. The format of this file can be found in Demand Profile File Formats.

Using Demand with Forecast Definitions and Powerflow Cases

EMarketOffer's Forecast and Powerflow Case Definitions are based on a weekly Demand profile, by day of week. Demand figures for no more than seven consecutive days can be loaded into a Definition.

The following points should be noted:

  • Where a Forecast or Powerflow Case Effective Date Range extends beyond seven days EMarketOffer will re-use the loaded Demand by day of the week from the profile for all weeks in the Effective Date Range.
  • Where Demand for fewer than seven days is loaded into a Definition, any periods in the Effective Date Range not covered by the loaded Demand will fail to dispatch
  • New Demand data can be loaded over the top of existing data. For example it is possible to load a full week profile from the database and then load a partial Demand file that has been edited to contain Demand for a single Node for a specific number of periods on a specific day. In this case only those periods for the selected Node will be overwritten with new data, all other Demand figures will remain unchanged.
  • Demand files are loaded consecutively by date from earliest to latest. Loading more than seven days of data from file will cause over-writing of the weekly Demand profile. For example if ten days of Demand figures are loaded from file the first three days will be lost.
  • When loading Demand data from database it is possible to specify a date range covering several weeks of Demand data. In this case an averaged Demand profile is created by day-of-the-week and Trading Period. It is not possible to create averaged Demand profiles by loading from file.

Transmission System Data

EMarketOffer can create a set of grid data (circuit details, line re-ratings and constraints) at run time from one of four sources:

  1. Historical transmission system data files provided by Energy Link
  2. Transmission system files sourced through the EM6 web site operated by Transpower division EMS
  3. In-house databases
  4. The files available each business day from the Energy Link web site (for grids from 22-Jul-09 onward)

Of the sources above, 1 - 3 are legacy data sources and we have stopped supporting them as the data available from EM6 is incomplete for the purposes required in EMarketOffer, and because we have gone to great effort to provide daily grid and reserves files that include all data reuired to solve the market by half hour. Only the daily updated grid files from 22-Jul-09 are now supported (though the other types of existing grid data can still be used in the meantime). The daily updated files also include full reserves data.

On startup EMarketOffer checks the Energy Link web site for the new grid and reserves data and automatically downloads files if you are not up to date.

The Connect Grid Data Tables check box in the Database Connection Settings window can be checked if you wish to use grid data from your own database, subject to the following:

WARNING! EMarketOffer will ignore any database links for grid data if grid data can be obtained from files. To use grid data from your own database you must delete any grid data files that are downloaded by EMarketOffer on startup.

DISCLAIMER: Energy Link takes a great deal of care in extracting grid and reserves data from the SPD daily final pricing case data files provided on the Electricity Authority's web site. However, we are totally unsupported in this endeavour and cannot in anyway guarantee the accuracy of the data we provide to EMarketOffer users. We provide the data in good faith and will not be held liable for the use of the data in EMarketOffer or in any other application, for the accuracy of the data, or for any special or consequential damages or losses resulting in any way whatsoever from the use of the data.

The following descriptions of grid data relate only to the data which is updated on Energy Link's web site each business day. For assistance with earlier versions of grid data (for all periods up to and including 15-Jul-09) you need to contact Energy Link or refer to an earlier EmarketOffer User Manual.


The SPD daily final pricing case data files are parsed by Energy Link each bussiness day to extract grid and reserves data. Grid data is stored in files with names:

  • BaseGrid_yyyymmdd_yyyymmdd.csv contain the base data for circuits, transformers and other key grid elements by half hour
  • Equations_yyyymmdd_yyyymmdd.csv contain the equation constraints by half hour
  • Reratings_yyyymmdd_yyyymmdd.csv contain circuit and transformer outages and reratings by half hour

In each case, the two dates show the range of dates covered by each file (from date first, to date second). The files are added to day by day but do not contain more than one year of data each.

In the same folder as the files above, you will also find VLRs_yyyymmdd_yyyymmdd.csv files which contain the VLR ratings for a small number of circuits which are currently being trialled: this data comes from Transpower's web site and are checked periodically.

Reserves Data

Reserves data is collected from the same source as the grid data, and is stored in two sets of files:

  • Reserves_yyyymmdd_yyyymmdd.csv
  • RiskSetters_yyyymmdd_yyyymmdd.csv

In this case the files store reserves data for up to one month at a time.

Static (User-Defined) Data

EMarketOffer uses static data to calculate or organise the dynamic data produced by the Run (see the Circuit entity example below). Static data tends to remain unchanged between the Trading Periods contained in a Run,

Entity Description Static Data Dynamic Data
Circuit
A branch (Circuit) conducting power on the NZ Transmission System
  • Name
  • From Node
  • To Node
  • Contributes to HVDC (Yes/No)
  • Resistance
  • Reactance
  • Capacity (Summer/Winter)
  • Powerflow
  • Price Separation
  • Losses
  • Loading

Modelled entities are defined by static data. Some entities are global, i.e. common for all Definitions, and their associated static data must be pre-defined (e.g. Companies and Regions) Most modelled entities, and their static data, are created for each new Definition (e.g. Generators or Nodes).

NOTE: To avoid time-consuming effort re-creating data in Definitions, Modelled entities, with their static data descriptions, can be imported into a Definition using EMarketOffer's, data loading functions, once loaded these entities can be modified, added to or removed with the Definition editor.

Functionality to maintain global static data can be accessed through the Edit Menu in the Main Window.

Static data in Definitions can be accessed through the Forecast Definition Window or the Powerflow Case Definition Window.


Home

Retrieved from "http://emo.energylink.co.nz/index.php?title=EMO:Input_Data&oldid=5928"