Output Specs
An Output Specification (OS) Resource allows you to define customised reports to be generated from run output data. Reports generated using OS Resources are text based in a comma separated (CSV) form and the resulting files can be opened in Excel or a number of other editors. OS reports can be run as part of a Report Batch and automatically generated on the completion of the run, or they may be generated at any time from run results. Any element shown in the results window can be exported via the Output Spec.
The Output Specs window has the following components:
- The 'OS Details Box:' Used to identify (Name) and describe (Description) the OS Resource.
- Two 'Formatting Check Boxes:' Repeat Column headings and Repeat Row headings - which determine whether the parent column and row headings appear once per column or row (Boxes unchecked), or for every column and row of the columns they apply to (Boxes checked) respectively.
- The 'OS Partition Box:' Specifies how the results data will be summarised and presented.
Note: OS Reports can be produced from previously generated data from the Results Window or Results Menu.
The Output Specs Details Box
Output Spec Details Box Fields
| Name | Output Spec Resource name - used to reference the OS in other Resources. |
| Description | Free text field for supplementary information.' |
| Prefilters | Defines a Prefilter that restricts the results being used to generate the report. (described below) |
Prefilter
There are two types of prefilter available: a date range prefilter, and version prefilters. You can run more than one prefilter by separating each specification with a semi-colon.
Date Prefilter
Format: Date:<date from>-<date to> (dates in d MMM yyyy format)
E.g. Date:1 Mar 2030 - 1 Mar 2035
This restricts the data being processed to the date range given
Version Prefilter
Format: Version:<volatility variable name>=<values list>
E.g. Version:InflowYear=1947,1980,2005-2010
This restricts the data being processed to versions where the volatility value is in the list of values given. The list of values are specified in the same way as they are in the Volatility Matrix.
Formatting Check Boxes
The Formatting Check Boxes are used to control the output of headers in columns and rows in the output. If the OS Resource is used when both boxes are left unchecked, then a heading only appears once in the output. For example a parent row heading will only appear with the first child row heading as in the table below:
| Parent Row Heading 1 | Child Row Heading 1 |
| Child Row Heading 2 | |
| Child Row Heading 3 | |
| Parent Row Heading 2 | Child Row Heading 1 |
| Child Row Heading 2 | |
| Child Row Heading 3 |
Running the same OS with the 'Repeat Row Headings' box checked would produce a heading configuration as below:
| Parent Row Heading 1 | Child Row Heading 1 |
| Parent Row Heading 1 | Child Row Heading 2 |
| Parent Row Heading 1 | Child Row Heading 3 |
| Parent Row Heading 2 | Child Row Heading 1 |
| Parent Row Heading 2 | Child Row Heading 2 |
| Parent Row Heading 2 | Child Row Heading 3 |
Note: Repeating the headers is useful for automatic post-processing and when viewing large output files with multiple row partitions.
Output Specification Partitions
To create an Output Spec, file, columns and rows need to be defined. This is done by progressively dividing the data using 'Partitions'. Each partition creates a number of divisions to which data is directed according to some criterion. The partition is specified in two parts <Partition Type>:<Partition specification>, where the specification is generally a comma delimited list. The results of an Output Spec report is one or more CSV style files. Partitions direct data either to separate output files or to separate columns or rows within the file (similar to the way a pivot table is displayed, or a crosstab query).
Partition Types fall into the following categories:
| Partitions Type | Purpose | Specification |
| Entity | Describes type of data in the results set that is to be processed with the OS. | Either 'All', or a list of entity names |
| Function | Describes the processing to be carried out on the data defined by an Entity partition. | One or more Output Spec functions |
| Version | Allows Output to be broken down by versions in the Volatility Matrix. | 'All' |
| Date/Time | Allows data to be aggregated by specified time periods. | 'All' |
In the example to the right, the file is defined by 'Year' so and output file will be created for each calendar Year.
The 'Column definition' is for the Average Nodal Price at the BEN and HAY Nodes by each inflow year. The :2 limits the results to two decimal places.
The 'Row Definition' is defined as MonthOfYear/Version:All which will display Average monthly Price for each inflow year
As a result, the final output will be a file for each calendar year, and the Average monthly price at Benmore and Haywards for each inflow year.
Function Partition
| Syntax | Example |
| Function:<list of output spec functions> | Function:Ave(Generator.Generation), Max(Node.Price) |
- Ave() - Average value, weighted by time.
- Tot() - Total value, summed by hour.
- Max() - Maximum value in a sample.
- Min() - Minimum value in a sample.
- First() - The first (earliest) value in a sample.
- Last() - The last (latest) value in a sample.
- Count() - The number of values in a sample.
- Percentile.<n>() - The nth percentile of values in the sample. Warning: this function may use a lot of memory and should be used with care, especially for large forecasts.
The First and Last functions would typically be used to get the earliest and latest (in modelled time) value in a sample. These functions do not have as much meaning if there are multiple records for each time period, i.e. if the Version or the Entity partition is not applied. For example if you were to use the Last(Generator.Generation) function over all generators then only the latest generation for the last generator (alphabetically) would be returned.
Volatility Matrix Partitions:
| Syntax | Example |
| Version:All, Version(<volatility matrix variable>):All | Version(InflowYear):All |
- Version - partitions all the simulation versions.
- Version(<volatility matrix variable>) - partitions the simulation versions by the given Volatility Matrix variable.
Entity Partition:
| Syntax | Example |
| <Entity Type>:<All or List> | Generator:Huntly, Waikato |
- <Entity Type Name> - partitions by the entities belonging to the named entity type. If 'All' is specified then a division is created for every entity, alternatively a list of individual entities can be specified, e.g. 'Node:BEN,HAY,OTA' will create a division for each of the three named nodes.
Date/Time Partitions:
| Syntax | Example |
| <date/time division>:All (except for 'Date' partition) | Week:All |
- hh - half hourly
- 1h, 2h, 3h, 4h, 6h, 8h - one, two, three, four, six and eight hourly
- DN - Day Night (12pm-8am, 8am-12pm)
- DN6 - Day Night (12pm-6am, 6am-12pm)
- WO - 2 divisions: Weekday, Other day
- WODN - 4 divisions: Week day, Other day, Week night, Other night (Day, Night as in DN)
- WODN6 - 4 divisions: Week day, Other day, Week night, Other night (Day, Night as in DN6)
- DOW - 7 Divisions - day of week
- MonthOfYear - 12 Divisions - one for each month in the year
- Day - A division for each day in the forecast
- Week - A division for each week in the forecast
- Month - A division for each month in the forecast
- Year - A division for each year in the forecast
- Year(<MMM>) - A division for each year in the forecast, where the year is deemed to start at the given month. For example: 'Year(Apr):All' will create divisions based on 1 Aoril - 31 March years
- Date - A division by user entered dates. For example 'Date:1 Apr 2015, 1 Nov 2015, 1 Jan 2016' will create four divisions separated by the three dates specified
Using The Output Specification Window
To create a new OS open Output Specs by clicking on the OutputSpec tab at the bottom left of the main page. Press <insert> to open a clean OS Window. Fill out the fields in the box at the top of the new window by pressing space to activate the cell and then typing the required information.
To create the first Column or first Row Partition highlight the Column or Row Partition heading by clicking on it and then pressing <Ctrl><Insert> to create a new blank Column or Row partition.
To create subsequent Column or Row Partitions right click an existing Column or Row Partition and select the required item from the drop down menu:
- Sibling Partitions can be created with the Insert Before option.
- Parent Partitions can be created (making a child of the highlighted partition) with the Insert Parent option.
- Child Partitions can be created (making a parent of the highlighted partition) with the Insert Child option.
To enter Partition details press the <space> key to activate the cell and start entering text.
If EMarket cannot validate the entry the text will simply revert to what was in the division beforehand. An entry may not be validated because:
- incorrect syntax
- invalid data-type or time-type
Warning: The specifications are case sensitive, and for function names, data types and time types, name completion is provided.