Improvements to Water Values
Aims
Water Value Series
Currently a run is restricted to the use of a single annual water value profile. This means sequences where the water values change significantly require multiple runs to model. I can see two ways to deal with this.
The first method is to produce annual water values starting at the end of a run, and then projecting these values backwards over the run. This method would produce a single dated water value sequence that is valid over the length of a run regardless of its length.
The second method would use annual water values always, but to allow them to be assessed starting at any date in the run. This method would produce a series of annual water value profiles. The run would then switch between profiles as it progressed.
The second method may allow more flexibility for user judgement on when profiles are likely to be reassessed. However, I have a preference for the first method as it is simpler, more widely understood from a theoretical viewpoint, and would not produce artefacts from switching from one profile to the next. This paper briefly describes water values calculated using the first method which is based around Stochastic Dynamic Programming (SDP) techniques.
Multi-Dimensional Water Values
The current system of controlling water release in EMarket operates each hydro system only with regard to its own stored energy. This method is clearly unrealistic in some situations, despite the fact that it manages total storage in the country fairly reasonably, with Waitaki dominating the control of marginal storage release. However, the other systems tend to be operated more loosely than would be expected in the real market as they are unable to anticipate system-wide shortage or excess.
With five major controlled reservoirs and two uncontrolled, the levels of which all potentially affect water values at any reservoir, a comprehensive multi-dimensional analysis of water value is quickly overloaded with complexity, inevitably resulting in excessive computation time for often marginal gain.
I propose a compromise consisting of a two dimensional profile for each reservoir, where two values determining water values are the local storage and the total of all other storage in the country. A water value profile could consist of, for each hydro system, $/MWh values for a set of storage values, e.g. 20 levels from empty to full for the local reservoir, 5 levels for the other national storage levels, giving 100 values in total. Water value at any level would be interpolated between these points
SDP can then be used to determine the two dimensional matrices for each hydro-system. It is still possible to use 1-in-N security factors to drive the cost of shortage, but this would probably need to be done by iterating the SDP and leading to excessive run times, so the design outlined in this paper uses another approach.
Retail/Hedge Commitment
Hydro operators will inevitably offset their generation earnings risks with wholesale purchase agreements and hedges. The effect of these pressures can be allowed for in water value modelling, but care needs to be taken that this is done in a balanced manner. In as far as retail and hedge commitment is required by the water value algorithm one average commitment power value (MW) per reservoir is all that is required for each weekly time step and scenario (A scenario is assumed to be one inflow sequence, but in could in theory extend to include multi-dimensional scenarios, for example, an inflow-demand scenario. But typically, runs and their respective WVs will be set up with approximately 80 historical inflow scenarios).
EMarket currently allows for the input of company portfolio data. This information could be used to generate commitment values, but there are two difficulties with this approach:
- extra care is required of the user to enter reasonable portfolio figures;
- where two or more reservoirs are owned by the same entity, the commitment should be shared – which would require a complication of the algorithm - or divided, which requires some assumptions about the reservoirs.
I don’t think co-optimising Manapouri and Waitaki (because they are both owned by Meridian) would impact greatly on the outcome, and to do so would add considerable complication to the algorithm.
Instead I suggest that a certain level of retail commitment is assumed, based on the total rated output of the hydro system, and then profiled using the average demand profile. While this may be far from accurate as an estimate of commitment, it is an easily generated and robust value that can be used at least to complement the response of hydro management to the provision of retail demand. The extent to which inaccuracies in this approach are unacceptably high will need to be tested.
Other Operating Considerations
Some reservoirs have strategic values that fall outside the profit maximisation objective of the normal water value relationships. Most importantly, low storage levels can be seen as undesirable without consideration of the lack of future revenue they represent. This is because a greater level of management, both operationally and politically, is required when security of supply is perceived to be threatened. Examples could include the analog of the Waitaki storage buffer currently used in EMarket, or perhaps the cost of risk aversion which acts to keep storage higher going into winter due to the risk of low winter inflows.
In order to include these considerations in water value calculation an additional marginal value of storage can be added. For example, if storage below 600GWh was considered undesirable to the extent that every GWh below 600 incurred a cost of $50 per week this can be expressed in the water values by adding a $50 * 0.168 = $8.4 cost per MWh to the water values after each weekly back projection (The figure of 0.168 is obtained by dividing 168 hours in a week by 1,000 MWh).
Another example is the mitigation of flood protection costs. This consideration may put a negative premium on high storage level, reducing their marginal value, possibly even below zero.
These ‘strategic’ costs may vary with the time of year as their nature can be a matter of perception and policy. In general the addition of costs such as these to the water value algorithm provides a flexible and fairly meaningful way of adapting the algorithm to mimic real life policies.
I propose that a number of additional marginal costs can be specified for each reservoir, expressed as a single $/MWh figure, a lower or higher limit to apply to the cost and a week from/week to range to apply the cost to.
Smaller Reservoirs
The current water value algorithm uses deterministic water values to evaluate smaller reservoirs. This will change with the new approach as these reservoirs need risk aware management to behave realistically. Accordingly, all reservoirs will be treated the same in the water value SDP algorithm.
Changing from the Current System
Matching Water Values to Simulation Runs
The current style of water valuation links contours to offer bands and uses this information to support two features. The first of these features is the ‘Link WVs to Offer Prices’ which responds to changes in the price of any generation tranche by adjusting the price of the related water value contour. This feature is apparently pretty much unused by modellers currently. It also suffers from problems when adjusted prices cause contours to swap.
The second feature is the ‘Adjust WVs by Location’ which also implements a price adjustment to contours, but does it by taking the location factor between the contour and its related offer into account. This feature is currently being made use of extensively and is essential given that water values are calculated on a ‘flat’ pricing grid whereas simulation runs use full nodal pricing. The water value adjusters, therefore, ensure that hydro offers adjust to match changing patterns of power flow across the grid.
Both contour adjustment features will have to be dropped when using a water value profile based on price alone, but use of the location factor adjustment suggests that some incorporation of location based pricing might be desirable.
It is possible that some form of locational pricing could to be incorporated into the water value generation, but this will potentially slow the model down significantly so an efficient scheme for applying the dispatch model will need to be found. A method with potential is to make location factor adjustments to water values which a re referenced to the demand-weighted average price (DWAP).
Performance
The current water valuation is very efficient with CPU usage and a new algorithm will inevitably be slower, but I hope to keep the time required to less than 20x the current time. As a highly fast and flexible modelling tool, we certainly do not want EMarket to suffer the same fate as the South American SDDP (which is used to a surprising extent in New Zealand), and given the small fraction of time that each multi-inflow run spends on calculating the current water values, even a 20x slowdown is not anticipated to degrade overall performance significantly.
Impact on Modellers
Input Parameters
I propose that when the new water value methodology is introduced the old method will be available for as long as is necessary for users to adapt their methods to the new method.
The old inputs that will be retired are:
- Link WVs to Offer Prices
- Major hydro Security factors
- Waitaki buffer, spill and reduction ratios
- Contour Separation (currently obsolete)
New Inputs are:
- Storage level additional costs/penalties, by Reservoir (replaces, for example, the Waitaki Low Buffer feature)
- Purchase commitment factor, either for all systems or for each system
- Market Power factor (used for the retail/hedge commitments), either for all systems or for each system, and possibly initially as hidden default market power factor
- Shortage Cost, possibly as a default system-wide shortage cost, say 100000 $/MWh.
The most important input parameters that do not translate from an old run one using the new water valuation calculation are those concerning spill and shortage buffers. These roughly correspond to the new storage level costs but no direct translation is possible.
I propose that the storage level costs, purchase commitment factors and market power factors be defined as part of the hydro system generator. A default value for purchase commitment and market power would be entered into all existing and new systems, with no storage level costs. This done, any existing run will be ready to be used to generate the new water values.
Water Value Profiles
The new water value profiles consist of a (W + 52) * N * M set of values for each hydro system, where W is the number of week in the run and N and M are the number of storage divisions for local and other storage. The profile is also given a start date. Unlike the previous profiles, the new profiles are fixed to a certain date, have an end date and have limited applicability beyond these dates. The profiles will be saved in the data directory as CSV files, allowing them to be processed or edited.
Viewing the water value profiles will need a somewhat different viewer to the one currently in use in EMarket (The current viewer makes use of the 1-1 relationship between offer bands and water value contours, a technique originally developed for use in ECNZ’s SPECTRA model and adapted into EMarket). There are no explicit price contours in the new format, but the viewer could show contours for a standard set of price values, e.g 10 dollar contours up to 100, 20 dollar contours up to 200, 50 up to 500 and so forth. The contours for any one value of ‘other storage’ can be view at a time.
Performance
As mentioned before, water valuation will definitely take longer under the new algorithm. I will aim to keep the total time for calculation to not much more than 10 minutes on a good machine. To put this in context, a dual core 3 GHz PC running Windows Vista calculates water values now in about 25 seconds and takes 1 hour 15 minutes to complete an all-inflows day-night run with reserves modelling turned on, but excluding generating reports. If the new water values take 10 times as long as the current water values then this adds 3.5 minutes to the run and 20 times longer adds 7.5 minutes.
There is a lot of uncertainty in my ability to include all the desired features of the algorithm described below and meet this performance constraint, so the design will necessarily be flexible for now.
Comparison to Current Water Values
The proposed new SDP-based water values will produce simulated behavior different to that observed in EMarket at present. There are three key expected benefits:
- More realistic management of individual reservoirs - reservoirs will be managed by the model taking the country-wide storage directly into consideration and so will not be subject to unanticipated shortage and surplus of supply.
This should result in less full and empty events in reservoirs than currently occurs, e.g. at present Taupo and Hawea tend to empty completely during a very dry winter. This in turn will result in corresponding changes in the output of the large marginal thermal stations, e.g. Huntly may tend to run harder earlier going into a dry winter. - Enhanced ability to model management policy – adjustments to hydro system policy based on purchase commitment and shortage or spill aversion should allow reservoirs to mimic some of these strategic objectives in a meaningful way.
- Better management of smaller reservoirs – use of stochastic water values for smaller reservoirs should result in more realistic management of storage.