Short Term River Chain Optimisation

Disclaimer

Reasonable care has been taken to ensure that the information in this paper is up to date at the time of issue. Potential users of EMarket should, however, ensure that they evaluate EMarket and this paper through an appropriate evaluation process in consultation with Energy Link. The authors are also reliant on certain information external to EMarket and Energy Link, for which no liability or responsibility can be accepted.

Introduction

This technical bulletin is intended to provide users and interested parties with a detailed explanation of how EMarket’s Short Term River Chain Optimisation (STRCO) module works. EMarket was originally designed to model medium and long-term hydro generation optimisation. This meant that hydro offers were based on medium to long-run marginal costs and did not vary in the short term. The aim of STRCO is to improve the realism of results from EMarket by ensuring that generator behaviour is consistent with the detailed day to day operation of generators in the market. STRCO is not the default for EMarket’s large hydro systems so must be enabled by the user as required.

Overview

Modelling in EMarket is also usually done with steps that aggregate a large number of trading periods together. Whilst changes in load through the day can be modelled, generator offers remain constant. In modelling the market behaviour over the course of a day it is necessary to determine how generators may adjust for intra-day fluctuations in demand and prices.

The benefits of determining short term behaviour extend beyond the ability to model in detail the daily behaviour of the market, since short term fluctuations in offers will have a gross effect on the overall behaviour. An example of this is the meeting of minimum flow requirements on the outflow of a large hydro system. Here the water throughput that is required for the minimum flow at night can be used for generation during the day, accumulating at the lowest reservoir which will empty over the night periods. It is difficult to determine the overall long term effect of minimum flow requirements on energy offered without understanding the short term effects in some detail.

In principle, short term modelling for hydros could include:

all the hydrological or regulatory constraints on the system;
the effect of generation on energy price, and price on profit;
provision of instantaneous reserves.

In EMarket hydro systems can be enabled to optimise using the STRCO module. The optimisation horizon is one week at a resolution of 4 hours.

River Chain Hydro Modelling, the DP approach

The problem to be solved for a major hydro is the optimal pattern of generation in the short term when the configuration of storage within the system can vary considerably over a short period. Dynamic programming (DP) provides a robust algorithm for determining the basis of this behaviour.

Figure 1

Optimal behaviour is defined by maximising gross profit, taking into account the entire portfolio of generation assets, retail and hedge contracts, and marginal costs, of the company owning the system. The behaviour at any time is determined by the time of the day and the state variables – the most important of which represent the storage within the system. When the DP is run it becomes possible to calculate the marginal value of all the state variables, and offering behaviour at each step can be determined by optimising against these marginal values.

For example, Figure 1 above shows a hydro system with a large storage lake, A, at the head and two smaller lakes in a chain below. Three dams, a, b and c, convert the flows between the lakes into electricity at a rate of 1 MW/cumec. The storage in each lake is measured in cumec-days (CMD).

When the DP is run it should be possible to assign a marginal value to the storage at A, B and C, given the current storage levels and the time. This allows a marginal cost figure to be allocated to the generation at a, b and c. If these values are 2,520 $/CMD, 1,440 $/CMD and 480 $/CMD, respectively, then generation at a has a marginal cost of:

[(2520 – 1440) $/CMD] / [24 h/Day] × [1 MW/cumec] = $45/MWh.

Similarly the costs of generation at b and c are $40/MWh and $20/MWh respectively.

It might then be possible to offer in three bands of generation at 45, 40 and 20 $/MWh and with the maximum MW output of a, b and c.

But the marginal costs of generation do not completely determine offers for large systems. The offers have to allow for the fact that they apply over a fixed period of time, and storage or flow constraints must be obeyed over that time. For example, if generating at full output will empty one of the reservoirs before a trading period is finished, then it is not possible to offer in all generation. If the total system output can have an impact on prices, this effect needs to be taken into account and it is also necessary to know the load commitment and hedges of the company as a whole.

Hydro System Specification

The following describes the most important features of a hydro system with respect to the modelling EMarket does in STRCO. The description has two main features:

channels, through which water is moved, which include stations, canals or spillways, and
nodes, which are the departure or destination points for channels.

Nodes may represent reservoirs or may simply be the point where a number of channels meet.

The system configuration is specified as follows:

N	Number of channels
M	Number of nodes
e_i	MW/cumecs for channel i = 1 to N
Max_i, Min_i	Maximum, minimum flow though the channel, i = 1 to N
Nto_i, Nfrom_i	The nodes which are connected by the channel, i = 1 to N
S_j	Available storage at a node, j = 1 to M

The state of the system at time t is specified as follows:

s_j	Current storage at a node, j = 1 to M

The behaviour of the system at time t is specified as follows:

I_jt	Uncontrolled inflows to nodes for time t, j = 1 to M
F_it	Flow (cumecs) through a channel (hydro station or spillway) at time t, i = 1 to N

For example, a specification for the system in Figure 1 is:

N = 3; M = 4; e_i = 1, 1, 1; Nto_i = 2,3,4; Nfrom_i = 1,2,3; Max_i = 200,300,350;

Min_i = 0,0,50; S_j = 9000, 32, 120

The DP Process

Adding to the definitions given above:

MWV_j,t - the marginal value of storage at node j at time t

The result of the DP will be a multi-dimensional function

MWV_j,t = WVF_j,t (S_1t, S_2t,…, S_Mt)

The following describes the dynamic programming analysis using a week's worth of forecast data. The dynamic program requires the following data:

A price forecast – half hourly market prices for the week;
Market sensitivity forecast – this would take the form of a sensitivity figure given by (change in price)/(change in generation), that estimates the amount prices will fall as more generation is put online;
Load forecast, for company retail load only;
Hedges for the week.

The DP starts with a finite set of seed MWV's for all reservoirs. These are assumed to be the water values at the end of the week, given for a number of storage configurations. Then, for the preceding time period, water values are calculated on the assumption that the seed water values are correct. This is done by backward projection, roughly as follows. Taking a storage configuration at time t and assuming a set of MWV's to be correct for this configuration, an optimal production schedule can be calculated. This production schedule will result in a change in storage, which is subtracted from the storage configuration at time t to give a storage configuration at time t-1. The resulting storage configuration at time t-1 is then assigned the same set of MWV's.

This is repeated back through the week to the beginning. The solution will converge, so for any seed values, the resulting MWV's for the first time period should be similar. The process can then be repeated, using the MWV's for the first period as a seed for the following iteration. The resulting MWV's, once convergence has been achieved, will be optimal for the week's operation – with the weak assumption that the following week will have roughly the same conditions. The final result says that MWV should be constant throughout the week.

Optimal Production

At each step of the DP it is necessary to calculate optimal production for a given set of MWV's. Also, when the MWV's are calculated they will be used to construct offers for the market model. As is outlined above, it is fairly straightforward to convert MWV's into marginal costs for generation, but this does not provide us with a complete solution. It is necessary to take into account all the constraints on the system and the dynamics of the market. The following method takes account of most constraints and uses a straightforward estimate of profit maximisation.

First we define total gross profit for one period:

GP_t = P_t G_t + (RP - P_t) Lt + (HP_t – P_t)HQ_t – GC_t

Where P_t is energy price at time t, RP is the retail price for which energy is being sold, HP_t is the hedge price and HQ_t is the hedge quantity. GC_t is the cost of generation, which will be defined by the water values (since the marginal cost of hydro generation is virtually nil, marginal cost must be based on the MWV which is an opportunity cost giving the expected future value of water in storage.) Market price and generation are correlated giving the effect on the price of generation. Using a market sensitivity estimate calculated from the simulated offer stack within EMarket:

P_t = P⁰t - MS_tG_t

and removing constant factors, the maximise gross profit problem becomes:

max {(P⁰t - MS_tG_t) (G_t - L_t - HQ_t) - GC_t }

Generation, G_t can be given as a function of the flows, F_it:

and GC_t is given by the MWV's

, where j1 = Nfrom_i and j2 = Nto_i

The maximum gross profit problem then becomes:

The constraints on the problem are the flow constraints

Min_i < F_it < Max_i

and end storage constraints (T = the length of the time period):

0 < s_jt+1 < S_j , where

This expresses optimal production as the maximum of a quadratic function, with linear constraints. Quadratic programming (QP) can solve this class of problem.

If company optimisation (Company optimisation can be enabled for a company modelled in EMarket. When enabled, the simulated company optimises gross profit given its portfolio of generation assets, retail and hedge contracts, and marginal costs) is ignored then linear programming (LP) can be used to find optimal production given market price and the MWV's.

Backward projection in the DP process can be achieved by calculating optimal production at each step. This describes an optimal system response when prices and market sensitivity are known in advance, and as such is a deterministic solution.

Optimal Operation

This problem describes the best response to a given dispatch, given the discretion allowed in the New Zealand market by the block dispatch rules for large hydro systems (Block dispatch may be approved for hydro stations along a linked hydro system. The generator offers by station and the dispatch instruction arrive by station, but the generator may spread the total dispatched megawatts for the hydro systems any way they please along their linked stations). The load and market sensitivity are no longer a concern, as production and prices are already set. The maximisation of gross profit equates to the minimisation of costs:

Subject to a given total dispatch (D) being met:

and the flow and storage constraints:

This is a linear optimisation problem, which can be solved by linear programming (LP).

It should be noted here that for the optimal production and optimal operation formulations given above the MWV's are held to be static, where in fact they are a function of storage and time. For solutions that hold over long time periods, as in the WODN or DN mode modelling in EMarket, the variability of MWV has to be accounted for.

Optimal Offers

The section above describes optimal production for a given price, load and market sensitivity. For the purposes of constructing offers, however, we need to consider how the system should respond to a range of prices. At best, the offers made can only be a good approximation of an optimal set of offers, since the division of production into discrete price blocks cannot match a continuous marginal production curve. A relationship between price and market power has to be estimated too. A number of methods might be used. One method would be to:

Choose a number of sample prices.
For each sample price estimate market sensitivity using forecast market sensitivity and price and making the ratio of price to market sensitivity the same for all prices. For example, if forecast price = $40.00 and forecast market sensitivity = $0.02/ MW, then a sample price of $20.00 would be matched with a market sensitivity figure of $0.01/ MW.
Find optimal production for each of the sample prices.
Offer the extra production brought on by each sample price at that price.

This scheme is straightforward enough, but choosing the prices in stage 1 might be difficult. If too many prices are chosen the algorithm may run too slowly, if too few are chosen then the offers may be far from optimal.

Company Optimisation ( Company Optimisation in EMarket is a method by which a company-wide optimisation of offers can be achieved once the marginal generation costs are known. To make use of this method it is required that the hydro system provides a production-cost profile, in the form of a series of offers at marginal cost.

Company Optimisation in STRCO

In the process of deciding optimal offers, estimates or forecasts of market sensitivity are used. As it is used in the equations above, market sensitivity is itself an estimate of the negative impact on prices of adding extra generation to the market.

Company Optimisation for Optimum Production

In order to estimate the optimum production for a system in the DP analysis, an approximation of market sensitivity is used. This can be derived from the supply stack as follows:

Find the market price that would have occurred if no generation were offered from the system. Call this value P⁰.
Find the market price that would have occurred if the total generating capacity of the system was offered at $0. Call this value P¹.
Estimate the generation-price relationship as P = P⁰ – CO*G, where CO = P⁰ – P¹/ G^TOT. G^TOT is the total generating capacity of the system.

The linear approximation may not be accurate for prices between P⁰ and P¹. However, it does allow the use of a QP algorithm to estimate optimum production.

Company Optimisation for Optimum Offers

Company optimisation can be done on the offers made by the hydro system. The offers made will be based on marginal costs. The optimisation is done by a company-wide procedure.

Implementation in EMarket

Company optimisation for optimum production and optimum offers are implemented within EMarket as an independent code module serving the main hydro objects.

Data Requirements

The module is provided with the following information:

The hydro system configuration, as specified above;
Initial short term price and supply curve forecasts;
Forecast of company load and hedge cover.

The DP is re-run on a once a week where price and supply curve forecasts can be obtained from the previous week's results. A week is run with estimated water values before the forecast is started, to provide an initial price forecast.

Company load is defined within the company screen as a percentage of load at given nodes. Hedge cover is also defined in the company screen.

Speed

The DP is the most computationally intensive part of the short term optimisation process. At the beginning of each week a DP water value evaluation will be run for each main hydro that is being optimised. The complexity of the DP increases dramatically with the number of reservoirs being controlled. A method for reducing the complexity of the problem by controlling combined reservoirs has been designed and is described in FD-SimplifiedRiverChains.doc.

The process of offering also uses heavy computation, and results in more offers than were previously made by the main hydro systems. As a whole, using STRCO means EMarket will run significantly slower than with it turned off. It is generally advisable to enable STRCO on only those hydro systems for which it makes a significant difference, Waikato being the most likely in this regard.

Seasonal Water Value Calculation

The seasonal water value calculation models many years of inflows several times in order to do a stochastic analysis of water values for the four main hydros (see Modelling Major Hydro Systems), for which the modelling is simplified for the necessity of speed.

The only change at this point to the seasonal MWVs calculated earlier is in the calculation of the main hydro offering parameters. This is done as follows:

The minimum and maximum sustainable generation is evaluated for each main hydro;
The minimum is added to the must-run on the stack, the rest is offered in at water value in the seasonal MWV simulations.

Summary

Using the methods above, hydro systems can be optimised so that they behave consistently with hydrological constraints and profit maximisation over the short term – as opposed to seasonal water values which are based on optimisation over a year.

Aspects of the day-to-day operation can be specified by the user. For instance, the daily fluctuation of storage in one or more reservoirs could be defined, leaving EMarket to optimise the operation of the other reservoirs.

The operation of one or more stations can be specified by controlling the maximum and minimum flow through that station. These can also be specified through the Schedule if required.

As the DP mechanism takes a significant amount of time to run, STRCO is optional. If optimisation is turned off zero water values are used for all the small reservoirs in the system – seasonal water values are used for the large reservoirs. This, in effect, ignores the value of short term storage.

Back to Technical Bulletins