Difference between revisions of "EMK:Multi Dimensional Water Values"

From EMK Wiki
Jump to navigation Jump to search
 
(43 intermediate revisions by the same user not shown)
Line 143: Line 143:
 
===Water Values and Marginal Benefit===
 
===Water Values and Marginal Benefit===
 
Water values are generated using a form of marginal analysis, the basis of which is the relationship (Eq. 1):
 
Water values are generated using a form of marginal analysis, the basis of which is the relationship (Eq. 1):
[[File:Eq 1.jpg|500px|centre|Equation 1]]
+
[[File:Eq 1.jpg|600px|centre|Equation 1]]
where ∆ is a small increment of storage and m<sup>+</sup><sub>t</sub>(s) is the market state at time t, given that reservoir r starts with a storage level of s. In other words, the water value is equal to the expected marginal benefit of extra storage in reservoir r (see Appendix 1: Relationship between Water Value and Benefit).
+
where ∆ is a small increment of storage and '''m'''<sup>+</sup><sub>t</sub>(s) is the market state at time t, given that reservoir r starts with a storage level of s. In other words, the water value is equal to the expected marginal benefit of extra storage in reservoir r (see Appendix 1: Relationship between Water Value and Benefit).
 +
 
 +
In practice the sets market states '''m'''<sup>+</sup><sub>t</sub>(s) and '''m'''<sup>+</sup><sub>t</sub>(s+&Delta;) will tend to converge, so that
 +
:Eq. 2.  '''''m'''<sup>+</sup><sub>t</sub>(s) = '''m'''<sup>+</sup><sub>t</sub>(s+&Delta;) : t &ge; t<sub>c</sub>''
 +
 
 +
where t<sub>c</sub> is the point of convergence. This means that marginal benefit only needs to be measured up to this point of convergence as it will be zero from that point on. This process is illustrated in Figure 7.
 +
[[File:MDWV Figure 7a.jpg|400px|thumb|centre|Figure 7 - Conversion of Projected Storage Trajectories]]
 +
 
 +
Figure 7 shows trajectories originating from storage points s and s + ∆ and converging when the higher trajectory is forced to spill when storage is full.  Of course the market statuses contain many other factors, including the storages of all other reservoirs, so this is a very simplified diagram.  However it does illustrate the factors involved in calculating marginal benefit.  These factors are:
 +
#To a certain degree the only difference between '''m'''<sup>+</sup><sub>t</sub>(s) and '''m'''<sup>+</sup><sub>t</sub>(s + ∆) is the difference in storage at reservoir r, and this difference may not change greatly until storage is constrained at full or empty;
 +
#Release, and therefore generation, of parallel storage paths in reservoir r will be the same, so the benefit from generation will be equal until convergence;
 +
#When storage is constrained at full, and spill occurs, the difference in benefit from generation will be zero;
 +
#When storage is constrained at empty, the difference in benefit from generation will be the energy price in the step where convergence occurs.
 +
 
 +
Subject to the reservations listed below then, it appears marginal benefit can be calculated by finding the point of convergence and measuring it just at that point.  If it is at a full point the marginal benefit is zero; if it is at an empty point marginal benefit will equate mostly to the energy price at the time of convergence.
 +
 
 +
Some reservations to consider when applying this method are:
 +
#There may be some additional penalty required for spilling water under some circumstances, to allow for the environmental, commercial or reputational risks involved, for example;
 +
#Some levels of storage may be deemed undesirable in themselves, for reasons such as flood risk control, dust problems, loss of head or reputational risks;
 +
#Trajectories will generally converge at a certain rate even when storage in not constrained, because marginally higher levels tend to make slightly cheaper energy offers;
 +
#The marginal benefit from generation also includes its effect on energy price in general and the benefit or loss incurred by the rest of the operator’s portfolio by this change in price.
 +
 
 +
These reservations are left out of the description for now and dealt with in the Appendices.  The primary method used here for finding marginal benefit, and therefore water value, is projection to a storage constraint.
 +
 
 +
===Calculation of Water Values===
 +
There is a circular set of relationships being used here, which implies an iterative method of solution.  The circularity arises as follows:
 +
#For marginal benefit of storage to be calculated, market projections and storage trajectories need to be generated;
 +
#For market projections and storage trajectories to be generated, the water value function W needs to be determined;
 +
#For the water value function W to be determined, marginal benefit of storage must be calculated.
 +
 
 +
If a water value function is optimal, then the marginal benefit of storage generated for any storage configuration must be equal to the water values for that configuration.  Conversely, if a water value function is found that produces marginal benefits that are the same as the water value then a solution has been found that is at least ‘locally optimal’.  The assumption is made here that such a solution is also fully optimal.  This assumption is supported by observing that convergence to the same value occurs when the iteration is started from a variety of seed values.
 +
 
 +
The method for iteration and convergence of water values is roughly:
 +
#From the current water values, produce a set of marginal benefits for given points in the storage configuration;
 +
#Combine the current water values with the calculated marginal benefits by making a weighted average of the two;
 +
#Continue steps 1 and 2, while steadily reducing the weighting given to the calculated values;
 +
#Check for convergence.
 +
 
 +
===The Water Value Function: Two Dimensional Evaluation===
 +
In EMarket the water values and marginal benefit are calculated on a two dimensional storage grid for each time step (two dimensions of storage and one of time gives three dimensions (3D) in total), although there is no conceptual reason that further dimensions cannot be added.  The final water value function is an interpolation of the values on this grid.  The two dimensions are the storage level in the reservoir being evaluated and the total storage in all other reservoirs.  For example, when evaluating marginal benefit for the Waikato, which has roughly 600 GWh of storage, evaluations might be made for each 50 GWh interval of storage in Waikato, and for when other storage is 0, 20%, 40%, 60% 80% and 100% full, making in total 12 x 6 = 72 points in each time period.
 +
 
 +
===Stochastic Method: Scenarios===
 +
This water valuation method is stochastic in the sense that ‘expected’ marginal benefit is used to assess water value.  The stochastic nature of the trajectories needed to evaluate marginal benefit is modelled by using a set of ‘scenarios’ which should be representative of the total set of likely outcomes.  Currently these scenarios consist of a set of historic inflow sequences, inflows being the major source of variability in the New Zealand electricity market, however demand and outage variations can be added to this set if needed, subject only to any restrictions on the length of time available to calculate water values.
 +
 
 +
The subscript ''k'' is used here to denote scenarios.  Using the terminology outlined in the section 4 2.3, each scenario consists of a set of '''m<sup>F</sup>''' values, defined for each step.  The full scenario set could be indicated as '''m<sup>F</sup>'''<sub>k,i</sub> which represents a set of fixed market values for each scenario and time step.
 +
 
 +
Figure 8 illustrates the determination of expected marginal benefit using scenarios.  In this case, four scenarios are used, three of which converge at full storage.  The other path (scenario 1) converges where storage is empty, which contributes a marginal benefit associated with constrained generation.  In this case, three of the four scenarios will contribute a zero marginal benefit, so the expected marginal benefit is one quarter of the marginal benefit of scenario 1.
 +
[[File:MDWV Figure 8.jpg|400px|thumb|centre|Figure 8 - Projection of Multiple Scenarios]]
 +
 
 +
===Computation of Expected Benefit===
 +
While Figure 8 illustrates the principle of calculating expected marginal benefit, it would be computationally inefficient to project paths in this way when recalculating expected benefit at each storage point.  Instead, a back projection method is used so that when the marginal benefits of each scenario have been calculated for one time step, these values are then used to calculate the marginal benefit of each scenario for the previous time step.
 +
 
 +
This method can be expressed using the following definitions
 +
{|class="wikitable"
 +
|-
 +
| N || The number of scenarios being modeled
 +
|-
 +
| '''m<sup>F</sup><sub>k</sub>'''(t) || Fixed market status given by scenario k for time t
 +
|-
 +
| V(s, t, W) || Calculated expected marginal benefit for a storage configuration s, at time t, using the water value function W
 +
|-
 +
| V<sub>k</sub>(s, t, W) || Calculated marginal benefit for a storage configuration s, at time t, using the water value function W and the scenario k.
 +
|}
 +
 
 +
The expected marginal benefit is simply an average of marginal benefit for all scenarios (assuming here that all scenarios are equally possible):
 +
 
 +
Eq. 3. [[File:Eq 3.jpg|300px|centre]]
 +
 
 +
The marginal benefit for each scenario is derived from the marginal benefit for the scenario in the next time step, where storage is unconstrained:
 +
 
 +
Eq. 4. [[File:Eq 4.jpg|700px|centre]]
 +
 
 +
and is determined by the marginal benefit at convergence where storage is constrained (Eq. 5,6):
 +
[[File:Eq 5 6.jpg|400px|centre]]
 +
 
 +
where V<sup>G</sup> represents the marginal benefit of generation calculated for the market state calculated in the step transformation.
 +
 
 +
Figure 9 illustrates the nature of the V<sub>k</sub> values, in one direction only:
 +
[[File:MDWV Figure 9.jpg|400px|thumb|centre|Figure 9 - Generation of V<sub>k</sub> values for a Single Scenario]]
 +
 
 +
The arrows indicate how storage evolves from period to period using the scenario.  The shaded regions converge at similar times and as such will have similar values.  For example the entire white region will have a V<sub>k</sub> value of 0.
 +
 
 +
To summarise, the marginal benefit is calculated for individual scenarios by projection, using the current water values.  When this is done the marginal benefit values are combined to give a new estimate of water value.
 +
 
 +
In practice marginal projection is done on a grid, so V<sub>k</sub> is only calculated for certain storage levels.  This means that a certain amount of smearing out of the marginal values occurs due to interpolation between these points.  Testing of the algorithm by using more sample points indicates that this artefact does not significantly affect the result beyond a certain number of sample points, so the use of a grid of points appears to be appropriate.
 +
 
 +
An alternative method is to track the boundary levels at each time period, between which storage converges under different conditions.  This method was tried initially, but it is complicated and difficult to test.  The use of a grid of storage points gives an almost identical result to the boundary method and is only slightly slower to run.  The advantages of simplicity make the grid method the method of choice here.
 +
 
 +
The use of extended projection by scenario has been found to produce meaningful adjustments to the water values when compared to the results using an uncorrelated stepwise back projection method.  Water values are increased where the risk of extended dry periods occurs and applying these water values reduces the occurrence of both shortage and spill events when compared to the uncorrelated case.  The addition of corrections for correlation can adjust the results of ordinary back projection to give reasonable results, as we understand is done in SPECTRA.  However, there is no guarantee that these corrections will appropriately model over the long term the overall statistical properties of the scenario distributions, so the elimination of this correction step is a significant and useful improvement.
 +
 
 +
===Appendix 1: Relationship between Water Value and Benefit===
 +
This appendix examines the assumptions behind the relationship given in section 4.4.
 +
 
 +
The assumption is made that hydro offers are generated (using the o<sup>H</sup> function) such that the value  ''w<sub>r</sub> s'<sub>r</sub> + B<sub>r</sub> ('''m'''<sup>+</sup>)''  will be maximised.  That is to say that, in a single step, the benefits measured by the benefit function are balanced against the loss or gain of stored water and the water value attributed to that water.  This value can be called ‘net benefit’ and is denoted by the variable v.
 +
 
 +
Eq. 7.  [[File:Eq 7.jpg|250px|centre]]
 +
 
 +
This optimisation can also be expressed as a function of total release from the reservoir.  Here the total release (denoted here by the variable y) is treated as the sole decision variable
 +
 
 +
Eq. 8.  [[File:Eq 8.jpg|290px|centre]]
 +
 
 +
Optimisation of v then implies that where the release decision variable is not constrained then
 +
 
 +
Eq. 9.  [[File:Eq 9.jpg|340px|centre]]
 +
 
 +
When storage is unconstrained the resulting storage ''s''' will be directly and inversely dependent on release, so:
 +
 
 +
Eq. 10. [[File:Eq 10.jpg|180px|centre]]
 +
 
 +
Replacing this relationship in Eq. 9 gives:
 +
 
 +
Eq. 11. [[File:Eq 11.jpg|220px|centre]]
 +
 
 +
The expression on the right of the equation can be summed up as ‘the marginal benefit of release’ and will be expressed as B’(y) here.  In fact Eq. 11 summarises the intuitively rather obvious relationship otherwise stated as ‘given optimal, unconstrained, release, water value will be equal to the benefit of additional release’.
 +
 
 +
In other terms, the expressions above demonstrate that if the operator has a choice between releasing y and releasing a marginal amount more, y + ∆, then the difference in immediate benefit between these choices is equal to w<sub>r</sub> ∆, where release is optimal (Since the most obvious benefit of release is the sale of electricity, B’(y) can be equated to nodal price and the marginal efficiency of the release y).
 +
 
 +
The definition of our primary objective is that expected overall benefit is being optimised.  This implies that an incremental change in release (where this decision variable is unconstrained) should result in neither an increase or decrease in expected overall benefit.  This means the change in immediate benefit expressed in Eq. 11 must be matched by an equal and opposite change in expected future benefit when y is incrementally adjusted:
 +
 
 +
Eq. 12. [[File:Eq 12.jpg|420px|centre]]
 +
 
 +
Where E<sub>r</sub> ('''m'''<sup>+</sup>(y)) is the expected future benefit from the resulting market conditions.  This function will practically depend only on the resulting storage levels, of which only the local level is directly (and inversely) affected by the release value y.  So:
 +
 
 +
Eq. 13. [[File:Eq 13.jpg|320px|centre]]
 +
 
 +
Combining Eq. 13 with Eq. 12 gives:
 +
 
 +
Eq. 14. [[File:Eq 14.jpg|180px|centre]]
 +
 
 +
Which summarises the relationship: '''''water value is equal to the marginal change in expected benefit over change in storage.'''''
 +
 
 +
===Appendix 2: Alterations and Corrections to the Projection Method===
 +
As mentioned in section 4.4 the method of projecting marginal benefit back from the point of convergence needs to be done with some extra considerations in mind.  Currently ''EMarket'' implements the spill penalty (4 10.1) and storage buffers (4 10.4), with the other considerations left as possible future enhancements.
 +
 
 +
====Marginal Benefit Adjustment at Storage Constrained Convergence====
 +
Where storage is full it can be assumed that marginal extra storage would not affect generation, so the marginal benefit derived from generation and energy prices will be zero.  However, there may be a cost associated with the spillage of water, and it is straight-forward to incorporate this cost by adding a penalty to the marginal benefit, resulting in a negative marginal benefit:  this is referred to here as a Spill Penalty.
 +
 
 +
Where storage is empty, marginal extra storage directly affects generation, indicating a marginal benefit equal to the energy price.  Additional generation will also affect nodal price, the marginal benefit of which is zero if generation and retail load and hedge cover are perfectly balanced.  Calculating the marginal benefit to the total portfolio of the operator is a complicated business involving assessment of the price impact of generation and the portfolio balance.  Some approximations can be made such as assuming a portfolio size that is commensurate with average expected generation and using a rule of thumb for estimating the relationship between marginal price and generation (which is normally far from simple).
 +
 
 +
====Gradated Storage Constrained Convergence====
 +
While reservoirs are modelled here as being forced to spill beyond a certain level, in reality this constraint is rarely as sudden as this implies.  Usually there is a gradual ramping up of forced release over a given level of high storage.  This can be modelled by combining a zero marginal benefit with the projected marginal benefit in progressive amounts as the storage exceeds this level.
 +
 
 +
====Marginal Benefit from Unconstrained Convergence====
 +
It was noted in section 4.4 that the incremental difference in water value incurred by marginal extra storage will generally cause paths to converge even when storage is not constrained.  In fact due to the non-continuous nature of the market there is nothing to stop trajectories from crossing.  If they converge in this way it is because the slight difference in offer price has caused a difference in generation.  This effect will depend on:
 +
#The gradient of water value function against storage;
 +
#The likelihood that offers based on the water value will marginally displace other offers in the market.
 +
 
 +
This effect can be approximated by assuming discretionary offers are made at water value and estimating the probability of item above 2 multiplied by the value in item 1.  This value will give the rate of convergence between the trajectories.  When convergence occurs it implies the difference in offers between the two marginally different cases has resulted in a difference in dispatch between them.  This in turn implies that the offer prices are on the price margin. There is a loose implication that marginal offers are made with a price that reflects water value (at least on average), so when convergence occurs we could assume that energy price will on the whole be the same as the current water value.  Using this assumption the expected marginal benefit of this convergence will be estimated as the current water value.  This implies a combination of the current water value with the marginal benefit being projected, at the rate of the expected convergence, altering Eq. 4 to:
 +
 
 +
Eq. 15. [[File:Eq 15.jpg|600px|centre]]
 +
 
 +
where α is the estimated rate of convergence.
 +
 
 +
====Marginal Benefit of Storage - Buffers====
 +
This consideration takes into account any other factor that may make marginal storage desirable or undesirable.  For example, flood protection may demand that areas of storage are avoided, or conversely the unpleasant environmental effects of a low lake at certain times of year may force the operator to try and avoid letting the lake get too low.
 +
 
 +
A method for modelling these ‘undesirable’ storage areas is to assume a cost associated with the time spent in these areas and the amount of intrusion into the area.  Figure 10 illustrates this cost:
 +
[[File:MDWV Figure 10.jpg|400px|thumb|centre|Figure 10 - Cost of Incursion into Undesirable Storage Levels]]
 +
 
 +
In the trajectory shown the total cost associated with the incursion into undesirable storage levels will be proportional to the shaded areas.  The rate at which this cost is measured would be in $/(time x Storage Units).
 +
 
 +
 
 +
[[EMK:Water Values Bulletins | Back to Water Values Technical Bulletins]]

Latest revision as of 09:47, 27 November 2012

Scenario Projection Modelling for Stochastic Water Valuation in EMarket

Disclaimer

Reasonable care has been taken to ensure that the information in this paper is up to date at the time of issue. Potential users of EMarket should, however, ensure that they evaluate EMarket and this paper through an appropriate evaluation process in consultation with Energy Link. The authors are also reliant on certain information external to EMarket and Energy Link, for which no liability or responsibility can be accepted.

Introduction

This technical bulletin is intended to provide users and interested parties with a detailed explanation of how EMarket’s multi-dimensional (MD) water values are calculated and applied. EMarket was designed as a market simulation model rather than as an operational model and the algorithm developed by Energy Link for use in EMarket was designed to give a high degree of accuracy at high speed for this purpose. Speed of operation is a strength of EMarket and ensures that users can turn new or modified simulations around quickly, achieving high levels of productivity.

EMarket4 replaced EMarket3 which used two-dimensional (2D) water values. A 2D water value for reservoir A is calculated with reference to time of year and to storage currently in reservoir A. MD water values in EMarket are calculated with reference to time of year, storage in reservoir A and storage in all other reservoirs (i.e. they are 3D water values). MD water values therefore take account of total storage, and recognise the fact, for example, that when storage in reservoir A is low, future market outcomes may be different depending on whether total storage is either high or low. The behaviour of MD water values relative to 2D water values is briefly explained in: Water Values and Hydro Offers.

EMarket is now set up to allow the user to switch between MD and 2D water values, ensuring backward compatibility.

The water values have also been rewritten in EMarket to make future enhancements easier to implement. For example, water values in Pukaki could be enhanced to take account of time of year, storage in Pukaki, storage in Tekapo, and all other storage (4D water values). Or water values could take account of contract position, market power, or uncertainty in demand, for example.

This document is divided into three parts. Section 3 outlines the scenario projection method and its application in EMarket. Section 4 describes the mathematic principles behind the method is a more thorough manner.

Summary

When water values have been calculated using the projection method described in this bulletin, and applied to the operation of hydro plant, they should be valid in the following sense:

Given that the scenarios modeled are considered to be representative of expected market conditions, and that the inputs used in the water value function are representative of the values considered when evaluating water value, then the marginal water value produced by this method will reflect the expected marginal value of that water.

MD water values currently use time of year, local storage, and total of all other storage to calculate and to look up water values in each major reservoir. This method is an improvement over 2D water values which made strong assumptions about how storage evolved across all reservoirs. These assumptions have been relaxed in MD water values so at each time step storage in other reservoirs can take on values other than those assumed in 2D water values. As a result, MD water values produce model outputs which are more consistent with actual market outcomes.

An important feature of the EMarket MD method is that it is able to allow for serial correlations in market scenarios, regardless of the form that such correlations might take. This avoids the need for the statistically based corrections applied in some other models. Serial correlations are not only important for individual reservoirs, but their impact on the market will also be affected by correlation of inflows between different catchments. Other statistical properties such as wind flow correlation can also be captured provided realistic sets of scenario can be generated.

Scenario Projection Modelling in Practice

This section aims to give a general overview of how scenario projection is applied to water valuation.

General Principles

The underlying meaning of water value relates to the decision to use water for generation at any given time. This decision involves comparing the immediate benefit accrued by using the water with the benefit of holding it for future use. Water value can be defined as expected future benefit that a small, or marginal, amount of extra storage would confer in this situation. Determining this value is a stochastic process because of the unpredictable elements affecting future operation, especially the variability of reservoir inflows.

The expected benefit of a marginal amount of storage is normally only easily determined in situations where reservoir storage is constrained. When the reservoir has reached its upper limit and storage cannot be reduced operationally, this marginal value is nominally zero – the extra water confers no benefit. When the reservoir reaches empty, and operational guidelines force all available water to be used, the marginal value is equal to the immediate value of using the water.

When the reservoir is unconstrained the marginal value is harder to determine. As well as requiring a good model of the stochastic elements of the market, a number of subtle assumptions need to be made about the relationship between estimated water value and hydro system operation.

The usual methods of solution for this problem all involve iteration. The reason for this iteration is a circular relationship between a chosen set of water value guidelines and the expected benefit that results from using those guidelines. In a very simple form this relationship could be summarised as:

  • Water value guidelines too low → Water use and Generation Higher → Greater likelihood of shortage → Higher expected marginal benefit.
  • Water value guidelines too high → Water use and Generation lower → Lower likelihood of shortage → Lower expected marginal benefit.

So, in a very simple sense, expected marginal benefit is an inverse function of water value. The purpose of iteration is to find the point where the values are equal. Of course the water value guidelines and expected marginal benefit are not single values, but functions of storage levels and time. The method of iteration is in principle the same, only significantly more complicated, as an iterative method for solving a single dimensional function such as the bisection or Newton-Raphson method.

Given that iteration is to be used to find optimal water values, the next important step to decide is how expected marginal benefit can be calculated from a set of water value guidelines. This is covered in the next section.

Calculation of Expected Marginal Benefit

The stochastic nature of the hydro system’s operating environment is modeled by generating a finite set of equally likely future situations from any given starting situation. The marginal benefit of storage can then be calculated by taking the average of the marginal benefit values in all the future situations. This method creates a number of future problems from a single current problem and usually requires some sort of ‘back projection’, that is an initial guess at a set of values in the distant future and the repeated calculation of values in discrete time steps that work back from that time. For a cyclic solution, such as one for seasonal hydro operation, this method can be viewed as an iterative one that converges after a certain number of iterations, rather than one that returns from a far distant date.

One of the primary problems of back projection lies in its step-wise nature. It can be very difficult to model features of the environment that extend beyond the period of one step. One particular bug bear of water value modelling is the serial correlation of inflows: it is unrealistic to regard them as independent from one step to the next. Simple analysis of inflow data shows that inflows can actually correlate over long periods, for example during an event resulting from a medium term climatic event.

Figure 1 - Back Projection and Scenario Projection

The method used here was developed with the aim of encapsulating the challenging statistical nature of inflow modelling. With this aim in mind a significant shift has been made in the philosophy behind the back projection method. Marginal values are thought of as being the result of evaluating marginal benefit along a projected path for as long as marginal benefit would be realised along that path. Conceptually this involves following paths until they reach a point of storage constraint. The paths follow the progression of storage as it would occur for the scenario with the water value guidelines in place. In Figure 1 the water value at the point O at time ti is evaluated using four scenarios. If back projection were used the resulting value would be determined from the four values at time ti + 1. With scenario projection the value results from the zero values where two paths reach full storage and the shortage values where two paths meet empty.

These scenario projections are never actually performed, but the same result is achieved by using back projection and adding the simple expedient of calculating values separately for every scenario. In the diagram the expected marginal benefit calculated at point A is back projected from the point B only (because this point results from applying the same scenario) and likewise comes in turn from points C and D. The marginal benefit at D results from a projection that arrives at full, so all the values A to D are zero. Bear in mind here that these values are the marginal benefits for only one of the scenarios. Only when it is time to calculate overall expected marginal benefit are the values from the scenarios combined to give one figure for each point in the storage-time space.

The marginal value of water at either full or empty is of fundamental importance in calculating water values. Consider a “thought experiment” concerning a hypothetical reservoir of infinite storage. In this case there is no full and no empty, and so the future value of water is irrelevant. Put another way, there is no trade-off to be made between releasing now or later, as there will always be more than enough water to release any time in the future. Hence, in this case water values would be irrelevant and the reservoir owner would offer and generate in a way which simply maximised the generation revenue at each time step.

But real reservoirs are not finite, and do have upper and lower storage constraints: it is the presence of these constraints that requires the present and future value of water to be traded-off, and which in turn creates water values as the opportunity cost of the reservoir being constrained in future.

Extra Dimensions

Figure 2 - Scenario Projection with two Storage Dimensions

Figure 1 illustrates projection of a scenario where one storage volume is traced over time, but the same concept can be applied where more than one storage value is traced. This means water value profiles can be generated that return water values given the time and multiple measures of the storage situation. The default method chosen for EMarket uses two storage values to determine water value at any time: the storage in the reservoir for which water values are being generated and the total (energy) storage in all other reservoirs in the market. This means projection is done over a three dimensional space, or 3D water values.

Figure 2 illustrates how the scenario projection in Figure 1 might appear with an extra storage dimension.

While 3D water values are used in EMarket (time, local storage, other storage) the process of calculating water values will be illustrated below using only two dimensions (time, local storage) to simplify the visualisation of the process.

The Calculation Process

The following steps illustrate the marginal value calculation process, starting with a set of water values.

Figure 3 - Starting Water Values

For each scenario the progression of storage from each of a set of sample points is calculated by simulating the market starting at that point. For example, if the point represented a local storage level of X and another storage level of Y at time T, the reservoirs are set to these levels (with other storage will be proportionally allocated). Water values calculated are for this configuration using the current water value profile, with offers generated given the time and the calculated water values, and dispatch calculated accordingly. The resulting storage configuration for the following step is then recorded:

Figure 4 - Storage Changes at Each Step

The expected value (for this scenario only) is calculated by assigning a value to all the constrained paths (shown as thick arrows) and back projecting all the others. In the figure below, the regions between the thick black lines will have similar values because they are back projected from areas with similar storage-constrained marginal values.

Figure 5 - Similar Back Projected Scenarios

All the scenario marginal value sets are averaged to produce an overall expected marginal value set, as illustrated in Figure 6 below.

Figure 6 - Averaging of Marginal Value Sets

This combined set of expected marginal values is then used to adjust the water values by calculating a weighted average of the two values, after which the process is repeated. The values are found to converge reliably by using a weight for the new marginal values that diminishes at each step.

For simplicity, the examples above show water values as a function of time and a single storage value (as in EMarket3), but the process is extended in EMarket gives water value so that water values are a function of time, storage at the reservoir in question and total storage in all other reservoirs. The water values for all large reservoirs need to be calculated at the same time, as the water value profile for each reservoir effects the operation of all other reservoirs. This means the marginal expected benefit is calculated for all reservoirs and then all water value profiles are updated in each step.

Operating Considerations and Adjustments to Marginal Values

The process of calculating expected marginal water value needs to be adjusted in some cases for operating considerations other than the simple storage constraints at empty and full. This is done in EMarket using three adjustment factors: spill penalty; high storage buffer; and low storage buffer.

Spill Penalty

Spill penalty aims to factor in the undesirability (for example, there may be an indirect cost associated with spill) of spillage by allowing the user to assign a negative benefit to the amount of water being spilled in any situation. When marginal benefit is calculated this negative value is given to projections that result in spill. The spill penalty also sets a minimum marginal value for water at any level, since the option to spill this water is assumed to always exist.

High Storage Buffer

This value represents an aversion to allowing storage to exceed a given level, for any reason. This could be for flood management or other resource consent related reasons. A penalty factor is subtracted from marginal value for each time step the projection passes above this level. The penalty is measured in dollars per week per unit of storage exceeding the buffer level in MWh. In order that this value remains fairly constant it is input as a multiple of the expected average price (which is determined by running simulations during the water valuation process). The high storage buffer may result in negative marginal benefit but this is still limited by the spill penalty value.

Low Storage Buffer

This value represents an aversion to allowing storage to fall below a given level. A penalty factor is added to the marginal value for each time period the projection passes below this level. As with the high storage buffer, the penalty is given as a multiple of the expected average price.

Principles of the Scenario Projection Method

Introduction

This section aims to describe in general terms the scenario projection method for determining water value guidelines for hydro generators in a spot electricity market, outlining the mathematical principles being applied and how they relate to best practice by hydro generators.

Expressions and Functional Relationships

Below are some of the fundamental expressions and relationship expressed with the symbols that will be used throughout this section.

Market Evolution

Implicit in the following definitions is an electricity market model that evolves in a series of steps. This could represent the modelling of every dispatch and pricing period in the actual market, but is more likely to be one that models coarser steps such as day-night or week long steps. The step length is undefined here in order to represent all possible models of this type.

Conventions

Symbols defined here can represent single variables or vectors in the natural sense (a subscript indicates a vector index), groups of values with no particular indexing (such as all current market conditions) or functions. The single or vector values will be given lower case variable names, the functions upper case names and the value groups are indicated by bold type.

Definitions

ti Time, indexed by step, where ti+1 is the time of the step following ti.
mF Fixed market status. This is the market status not including decision variables – includes date, time, demand, transmission grid, inflows and non-hydro offers, but does not include reservoir storages, water values or hydro offers. ‘Fixed’ refers to the fact that these aspects of the market will be constant throughout the calculations.
Sr Storage by reservoir (r), measured in terms of stored energy
Wr Water value by reservoir
OH Hydro offers
m Complete market status, as at the beginning of a step, including storages and hydro offers
m+ Market step status, includes the prices and dispatched generation for a time step and the market status that results at the end of that time step
m+=D(m) Step transformation: given a market status m this returns the market step status m+ that results for the following step where the market variables have evolved as expected over that period. This is a fixed input to the valuation process and includes pricing, dispatch and hydro operation considerations.
w = W(s, t) Water value function: returns water values for each reservoir given the storages at all reservoirs (s) and the date and time (t). The determination of the function Wr is the aim of the water valuation process.
oH=OH(w,s,mF) Hydro offers function: returns hydro offers given water values, storages and market conditions (inflows in particular). The OH function has a fixed form throughout the water valuation process.
m=M(mF,s,W) Market status as a function of fixed market status, storages and the water value function. Given that OH function is fixed for the valuation process, the function M is also fixed.
s'=S(mF,s,W) Storage step transformation: returns the storages at all reservoirs after an increment of time given the fixed market status, current storages and the water value function. Given that the step transformation is fixed, this function is also fixed for the valuation process.
sMAXr Maximum storage of reservoir r

Definition of the Optimisation Problem

In this determination of water values the objective is to find the water values for each reservoir that will generate the greatest benefit for the operator of that reservoir. Primarily this benefit will be measured by income from generation, but other factors such as retail commitments and operational and compliance costs can also be factored in. Where possible the description of this water valuation process will refer to ‘benefit’ in the most general terms, to allow for these factors to be considered. The time over which the benefit is measured is not directly defined, but can be considered as an arbitrarily long time period.

Water Values and Marginal Benefit

Water values are generated using a form of marginal analysis, the basis of which is the relationship (Eq. 1):

Equation 1

where ∆ is a small increment of storage and m+t(s) is the market state at time t, given that reservoir r starts with a storage level of s. In other words, the water value is equal to the expected marginal benefit of extra storage in reservoir r (see Appendix 1: Relationship between Water Value and Benefit).

In practice the sets market states m+t(s) and m+t(s+Δ) will tend to converge, so that

Eq. 2. m+t(s) = m+t(s+Δ) : t ≥ tc

where tc is the point of convergence. This means that marginal benefit only needs to be measured up to this point of convergence as it will be zero from that point on. This process is illustrated in Figure 7.

Figure 7 - Conversion of Projected Storage Trajectories

Figure 7 shows trajectories originating from storage points s and s + ∆ and converging when the higher trajectory is forced to spill when storage is full. Of course the market statuses contain many other factors, including the storages of all other reservoirs, so this is a very simplified diagram. However it does illustrate the factors involved in calculating marginal benefit. These factors are:

  1. To a certain degree the only difference between m+t(s) and m+t(s + ∆) is the difference in storage at reservoir r, and this difference may not change greatly until storage is constrained at full or empty;
  2. Release, and therefore generation, of parallel storage paths in reservoir r will be the same, so the benefit from generation will be equal until convergence;
  3. When storage is constrained at full, and spill occurs, the difference in benefit from generation will be zero;
  4. When storage is constrained at empty, the difference in benefit from generation will be the energy price in the step where convergence occurs.

Subject to the reservations listed below then, it appears marginal benefit can be calculated by finding the point of convergence and measuring it just at that point. If it is at a full point the marginal benefit is zero; if it is at an empty point marginal benefit will equate mostly to the energy price at the time of convergence.

Some reservations to consider when applying this method are:

  1. There may be some additional penalty required for spilling water under some circumstances, to allow for the environmental, commercial or reputational risks involved, for example;
  2. Some levels of storage may be deemed undesirable in themselves, for reasons such as flood risk control, dust problems, loss of head or reputational risks;
  3. Trajectories will generally converge at a certain rate even when storage in not constrained, because marginally higher levels tend to make slightly cheaper energy offers;
  4. The marginal benefit from generation also includes its effect on energy price in general and the benefit or loss incurred by the rest of the operator’s portfolio by this change in price.

These reservations are left out of the description for now and dealt with in the Appendices. The primary method used here for finding marginal benefit, and therefore water value, is projection to a storage constraint.

Calculation of Water Values

There is a circular set of relationships being used here, which implies an iterative method of solution. The circularity arises as follows:

  1. For marginal benefit of storage to be calculated, market projections and storage trajectories need to be generated;
  2. For market projections and storage trajectories to be generated, the water value function W needs to be determined;
  3. For the water value function W to be determined, marginal benefit of storage must be calculated.

If a water value function is optimal, then the marginal benefit of storage generated for any storage configuration must be equal to the water values for that configuration. Conversely, if a water value function is found that produces marginal benefits that are the same as the water value then a solution has been found that is at least ‘locally optimal’. The assumption is made here that such a solution is also fully optimal. This assumption is supported by observing that convergence to the same value occurs when the iteration is started from a variety of seed values.

The method for iteration and convergence of water values is roughly:

  1. From the current water values, produce a set of marginal benefits for given points in the storage configuration;
  2. Combine the current water values with the calculated marginal benefits by making a weighted average of the two;
  3. Continue steps 1 and 2, while steadily reducing the weighting given to the calculated values;
  4. Check for convergence.

The Water Value Function: Two Dimensional Evaluation

In EMarket the water values and marginal benefit are calculated on a two dimensional storage grid for each time step (two dimensions of storage and one of time gives three dimensions (3D) in total), although there is no conceptual reason that further dimensions cannot be added. The final water value function is an interpolation of the values on this grid. The two dimensions are the storage level in the reservoir being evaluated and the total storage in all other reservoirs. For example, when evaluating marginal benefit for the Waikato, which has roughly 600 GWh of storage, evaluations might be made for each 50 GWh interval of storage in Waikato, and for when other storage is 0, 20%, 40%, 60% 80% and 100% full, making in total 12 x 6 = 72 points in each time period.

Stochastic Method: Scenarios

This water valuation method is stochastic in the sense that ‘expected’ marginal benefit is used to assess water value. The stochastic nature of the trajectories needed to evaluate marginal benefit is modelled by using a set of ‘scenarios’ which should be representative of the total set of likely outcomes. Currently these scenarios consist of a set of historic inflow sequences, inflows being the major source of variability in the New Zealand electricity market, however demand and outage variations can be added to this set if needed, subject only to any restrictions on the length of time available to calculate water values.

The subscript k is used here to denote scenarios. Using the terminology outlined in the section 4 2.3, each scenario consists of a set of mF values, defined for each step. The full scenario set could be indicated as mFk,i which represents a set of fixed market values for each scenario and time step.

Figure 8 illustrates the determination of expected marginal benefit using scenarios. In this case, four scenarios are used, three of which converge at full storage. The other path (scenario 1) converges where storage is empty, which contributes a marginal benefit associated with constrained generation. In this case, three of the four scenarios will contribute a zero marginal benefit, so the expected marginal benefit is one quarter of the marginal benefit of scenario 1.

Figure 8 - Projection of Multiple Scenarios

Computation of Expected Benefit

While Figure 8 illustrates the principle of calculating expected marginal benefit, it would be computationally inefficient to project paths in this way when recalculating expected benefit at each storage point. Instead, a back projection method is used so that when the marginal benefits of each scenario have been calculated for one time step, these values are then used to calculate the marginal benefit of each scenario for the previous time step.

This method can be expressed using the following definitions

N The number of scenarios being modeled
mFk(t) Fixed market status given by scenario k for time t
V(s, t, W) Calculated expected marginal benefit for a storage configuration s, at time t, using the water value function W
Vk(s, t, W) Calculated marginal benefit for a storage configuration s, at time t, using the water value function W and the scenario k.

The expected marginal benefit is simply an average of marginal benefit for all scenarios (assuming here that all scenarios are equally possible):

Eq. 3.

Eq 3.jpg

The marginal benefit for each scenario is derived from the marginal benefit for the scenario in the next time step, where storage is unconstrained:

Eq. 4.

Eq 4.jpg

and is determined by the marginal benefit at convergence where storage is constrained (Eq. 5,6):

Eq 5 6.jpg

where VG represents the marginal benefit of generation calculated for the market state calculated in the step transformation.

Figure 9 illustrates the nature of the Vk values, in one direction only:

Figure 9 - Generation of Vk values for a Single Scenario

The arrows indicate how storage evolves from period to period using the scenario. The shaded regions converge at similar times and as such will have similar values. For example the entire white region will have a Vk value of 0.

To summarise, the marginal benefit is calculated for individual scenarios by projection, using the current water values. When this is done the marginal benefit values are combined to give a new estimate of water value.

In practice marginal projection is done on a grid, so Vk is only calculated for certain storage levels. This means that a certain amount of smearing out of the marginal values occurs due to interpolation between these points. Testing of the algorithm by using more sample points indicates that this artefact does not significantly affect the result beyond a certain number of sample points, so the use of a grid of points appears to be appropriate.

An alternative method is to track the boundary levels at each time period, between which storage converges under different conditions. This method was tried initially, but it is complicated and difficult to test. The use of a grid of storage points gives an almost identical result to the boundary method and is only slightly slower to run. The advantages of simplicity make the grid method the method of choice here.

The use of extended projection by scenario has been found to produce meaningful adjustments to the water values when compared to the results using an uncorrelated stepwise back projection method. Water values are increased where the risk of extended dry periods occurs and applying these water values reduces the occurrence of both shortage and spill events when compared to the uncorrelated case. The addition of corrections for correlation can adjust the results of ordinary back projection to give reasonable results, as we understand is done in SPECTRA. However, there is no guarantee that these corrections will appropriately model over the long term the overall statistical properties of the scenario distributions, so the elimination of this correction step is a significant and useful improvement.

Appendix 1: Relationship between Water Value and Benefit

This appendix examines the assumptions behind the relationship given in section 4.4.

The assumption is made that hydro offers are generated (using the oH function) such that the value wr s'r + Br (m+) will be maximised. That is to say that, in a single step, the benefits measured by the benefit function are balanced against the loss or gain of stored water and the water value attributed to that water. This value can be called ‘net benefit’ and is denoted by the variable v.

Eq. 7.

Eq 7.jpg

This optimisation can also be expressed as a function of total release from the reservoir. Here the total release (denoted here by the variable y) is treated as the sole decision variable

Eq. 8.

Eq 8.jpg

Optimisation of v then implies that where the release decision variable is not constrained then

Eq. 9.

Eq 9.jpg

When storage is unconstrained the resulting storage s' will be directly and inversely dependent on release, so:

Eq. 10.

Eq 10.jpg

Replacing this relationship in Eq. 9 gives:

Eq. 11.

Eq 11.jpg

The expression on the right of the equation can be summed up as ‘the marginal benefit of release’ and will be expressed as B’(y) here. In fact Eq. 11 summarises the intuitively rather obvious relationship otherwise stated as ‘given optimal, unconstrained, release, water value will be equal to the benefit of additional release’.

In other terms, the expressions above demonstrate that if the operator has a choice between releasing y and releasing a marginal amount more, y + ∆, then the difference in immediate benefit between these choices is equal to wr ∆, where release is optimal (Since the most obvious benefit of release is the sale of electricity, B’(y) can be equated to nodal price and the marginal efficiency of the release y).

The definition of our primary objective is that expected overall benefit is being optimised. This implies that an incremental change in release (where this decision variable is unconstrained) should result in neither an increase or decrease in expected overall benefit. This means the change in immediate benefit expressed in Eq. 11 must be matched by an equal and opposite change in expected future benefit when y is incrementally adjusted:

Eq. 12.

Eq 12.jpg

Where Er (m+(y)) is the expected future benefit from the resulting market conditions. This function will practically depend only on the resulting storage levels, of which only the local level is directly (and inversely) affected by the release value y. So:

Eq. 13.

Eq 13.jpg

Combining Eq. 13 with Eq. 12 gives:

Eq. 14.

Eq 14.jpg

Which summarises the relationship: water value is equal to the marginal change in expected benefit over change in storage.

Appendix 2: Alterations and Corrections to the Projection Method

As mentioned in section 4.4 the method of projecting marginal benefit back from the point of convergence needs to be done with some extra considerations in mind. Currently EMarket implements the spill penalty (4 10.1) and storage buffers (4 10.4), with the other considerations left as possible future enhancements.

Marginal Benefit Adjustment at Storage Constrained Convergence

Where storage is full it can be assumed that marginal extra storage would not affect generation, so the marginal benefit derived from generation and energy prices will be zero. However, there may be a cost associated with the spillage of water, and it is straight-forward to incorporate this cost by adding a penalty to the marginal benefit, resulting in a negative marginal benefit: this is referred to here as a Spill Penalty.

Where storage is empty, marginal extra storage directly affects generation, indicating a marginal benefit equal to the energy price. Additional generation will also affect nodal price, the marginal benefit of which is zero if generation and retail load and hedge cover are perfectly balanced. Calculating the marginal benefit to the total portfolio of the operator is a complicated business involving assessment of the price impact of generation and the portfolio balance. Some approximations can be made such as assuming a portfolio size that is commensurate with average expected generation and using a rule of thumb for estimating the relationship between marginal price and generation (which is normally far from simple).

Gradated Storage Constrained Convergence

While reservoirs are modelled here as being forced to spill beyond a certain level, in reality this constraint is rarely as sudden as this implies. Usually there is a gradual ramping up of forced release over a given level of high storage. This can be modelled by combining a zero marginal benefit with the projected marginal benefit in progressive amounts as the storage exceeds this level.

Marginal Benefit from Unconstrained Convergence

It was noted in section 4.4 that the incremental difference in water value incurred by marginal extra storage will generally cause paths to converge even when storage is not constrained. In fact due to the non-continuous nature of the market there is nothing to stop trajectories from crossing. If they converge in this way it is because the slight difference in offer price has caused a difference in generation. This effect will depend on:

  1. The gradient of water value function against storage;
  2. The likelihood that offers based on the water value will marginally displace other offers in the market.

This effect can be approximated by assuming discretionary offers are made at water value and estimating the probability of item above 2 multiplied by the value in item 1. This value will give the rate of convergence between the trajectories. When convergence occurs it implies the difference in offers between the two marginally different cases has resulted in a difference in dispatch between them. This in turn implies that the offer prices are on the price margin. There is a loose implication that marginal offers are made with a price that reflects water value (at least on average), so when convergence occurs we could assume that energy price will on the whole be the same as the current water value. Using this assumption the expected marginal benefit of this convergence will be estimated as the current water value. This implies a combination of the current water value with the marginal benefit being projected, at the rate of the expected convergence, altering Eq. 4 to:

Eq. 15.

Eq 15.jpg

where α is the estimated rate of convergence.

Marginal Benefit of Storage - Buffers

This consideration takes into account any other factor that may make marginal storage desirable or undesirable. For example, flood protection may demand that areas of storage are avoided, or conversely the unpleasant environmental effects of a low lake at certain times of year may force the operator to try and avoid letting the lake get too low.

A method for modelling these ‘undesirable’ storage areas is to assume a cost associated with the time spent in these areas and the amount of intrusion into the area. Figure 10 illustrates this cost:

Figure 10 - Cost of Incursion into Undesirable Storage Levels

In the trajectory shown the total cost associated with the incursion into undesirable storage levels will be proportional to the shaded areas. The rate at which this cost is measured would be in $/(time x Storage Units).


Back to Water Values Technical Bulletins