Optimal investment by large consumers in an electricity market with generator market power

The cost of electricity is a crucial input factor for many energy intensive industries and services. As a consequence, investment decisions on desirable locations for these enterprises have always looked carefully at the prospects for sustained low prices. Often, this has meant proximity to resources. Thus, we have seen aluminum factories located in countries with plentiful hydro, e.g. Norway, Canada or New Zealand, or surplus nuclear power, e.g. France, or geothermal springs, e.g. Iceland. Going forward we may see more data centers in these locations as well perhaps as new hydrogen hydrolysers located close to plentiful renewable energy facilities. This is quite natural and intuitive. Thus, Michielsen (2013) observed that energy prices are more important than capital and skilled labor for the location of manufacturing industries in the US, whilst (Panhans and Hanley 2017) reports similar findings for the EU. Furthermore, at a local level, transmission constraints and market structure have similar influences. Thus, location-specific pricing has been implemented in some electricity markets in part to incentivize energy intensive users to be attracted to the less constrained locations in the supply network. However, price considerations do not only depend upon the local proximity of resources. They also depend upon the market structure in the generation sector and the market power of generators. Evidently, the reason regulators sometimes attempt to limit the exercise of generator market power is to benefit consumers (Cretì and Fontini 2019). Indeed, active lobbying by large industrial consumers has often been the prompt for regulators to intervene. Thus, in considering the prospects for sustained low prices, infrastructure and market structure will also be important factors for large consumers and retailers, especially if they doubt the strength of market regulation to act strongly and in a timely manner. In this context, whilst there has been an extensive literature on imperfect competition through the exercise of market power by generators (Hesamzadeh and Biggar 2012; Downward et al. 2010; Murphy and Smeers 2010), there has surprisingly been very little formal analysis on how the prospects of generator market power may influence the investment \(<1>\) behavior of large consumers (Shin and Tunca 2010). Intuitively it is self-evident that a wholesale market with high prices will be unattractive to consumers, and vice versa. However, a market in which generators have not been raising prices may appear benign, but it may be easily disturbed by the introduction of new consumer investment. The demand and supply balance could thereby be altered so that generators acquire more ability to influence market prices because of the investments on the demand side.

Whilst the electricity supply chain can present a particularly acute manifestation of this effect, in fundamental terms this is a general property of supply chains in which products are traded through intermediate markets which have the potential for producer market concentrations to emerge endogenously. Examples in manufacturing are where assembly facilities may be attracted to locate in areas with ready access to components or in food processing where processing factories are close to the agricultural supplies or in services where the locations already have the skilled professionals. All these investments could influence the market prices for the supply of resources which attracted them in the first place. In this general context, various strategies for upstream producers to acquire market power through vertical and horizontal investments have been extensively researched by economists for many years [\(<1>\) e.g. Laffont and Tirole (1993)], as well as models for investment under uncertainty having been a classic problem within operations research and real options [\(<1>\) e.g. Dixit et al. (1994)]. In fundamental terms, the principle of changes in demand influencing price is elementary. The subtle point is that demand-side investment may be attracted because of low prices, but in doing so may increase the prices. Furthermore, the development of practical capacity planning methodologies for a demand-side investor to model its anticipated endogenous effect on the upstream market power of producers remains apparently under-researched. Appropriate formulations in which the demand functions depend upon prospective demand-side investments are analytically challenging but, nevertheless, represent an important consideration in practice. We seek to pursue this requirement and in particular address the following conjectures:

1. 1.

   A substantial demand investor will face higher prices in a market with imperfect competition than would have been apparent before the investment.

2. 2.

   It is possible that a market without substantial market power will have substantial market power because it has attracted demand investment.

3. 3.

   A demand investor will find it optimal to invest less than it would have intended if an analysis of its effect on generator behavior is considered.

\(<9>\) In practical world the generator side investment and operations are heavily dependent on the long term demand forecasts. This is evident from various national grid strategies like (AEMO, Infrastructure Sweden), and (Power India). However, the least cost plan as anticipated might not always happen due to various reasons. One reason being influence of market power in the system (CAISO), which is studied in this paper. Market power influences could hike the prices in the system and this might slow down the demand side investments (IEA). The consumer could anticipate the market power influences due to historical precedence set by generator or a future-looking analysis and could either reduce or delay demand side investments (Global). At this point the investments made by generators could be at risk as the reduction in demand side investments will play into their forecasted market volumes. The demand investment and strategic operation of generators happen at different time scales. The demand investor will always look at historical strategic behavior of generators (lagged signal) and make an anticipating price forecast (leading signal) to decide on optimal demand investments. Note that time scales at which the demand investment and strategic operation decision are made are different. The demand investment preludes the future strategic operation of the generators and therefore a Stackelberg model is adopted in this paper with consumer as the leader. The leading price forecast signal embodies all possible threats and anticipations about the future grid that demand investor considers. The anticipating threat has influenced and deterred demand side investment in real world as seen in (Global). This is the motivation of modeling the demand investor as a leader (speculative of the grid operation in future) in the Stackelberg game. The literature on generation investment considering market operation models the investment and operation at two different levels like (Wogrin et al. 2011) and (Wogrin et al. 2012), and one might also draw analogy from there to the two-level model approach taken in this paper between demand side investment and strategic operation.

We provide theoretical insights to the above questions. Then, we develop a detailed model to represent how in practice a large demand-side investor can anticipate the price formation it may induce in the market model and thereby determine its optimal investment. This leads to a Bayesian Nash market model of imperfect competition, formulated as a quad-level program. A methodological contribution is in the reduction of this to an equivalent bilevel mixed-integer linear program for computational tractability. This, in turn, determines the optimal investment of an energy intensive commercial consumer.

The context of this research is therefore in general terms within the themes of investment and imperfect competition in supply chains. We look at the electricity supply chain case in detail, not only because energy supply chains are, in themselves, crucial to economies and society, but the nature of the product and its delivery through a network make it particularly susceptible to this general problem. Traditionally, demand has been modeled as a passive consumer in power system models. The social surplus maximization based models started treating the active participation of consumer demands in operational power system models. To the best of the authors’ knowledge this is a first attempt in analyzing the participation and investment by consumers in a long-term investment model in the context of imperfect competition. The paper shows that the imperfect competition equilibrium can be impacted by the congestion levels in the system. This in-turn translates into a relation between the congestion levels and demand-side consumer investment levels.

Whilst research on the behavior of generators with market power is very extensive, our perspective starts from an observation (Biggar 2009) that the conventional economic analysis tends to assume that the demand-side is essentially passive, that customer behavior is summarized in the demand curve. However, in the presence of market power, downstream customers are concerned with the risk that, once they have made the investment, market power by generators may lead to an increase in price with a consequent loss in the value of that investment. Consequently, these customers may be deterred from making the necessary investment in the first place. Thus, while the short-term demand response of consumers with sunk investments may be quite inelastic, in the longer term, their investments may be very price sensitive. As an example, the Tasmanian Electricity Supply Industry Expert Panel references just these kinds of effects in its final report.^{Footnote 1} They write:

The Panel has concluded that robust, long-term competition in the retail market requires the participation of large nationally-based retailers, such as AGL, Origin, TRUenergy and Alinta, along with smaller niche retailers. However, these retailers have indicated they are unlikely to enter Tasmania in the absence of more competitive retail and wholesale market structures. They are clearly unwilling to enter if it means relying on the benign future strategy and conduct of Hydro Tasmania which would be capable of unilaterally stranding their entry investments if it chose to do so. In summary, the Panel’s view is that the development of a competitive retail market in Tasmania is constrained by the market structure in the wholesale energy market. This market structure means that there is significant additional risk, and potentially cost, for retailers to enter and operate in Tasmania compared to market opportunities elsewhere.

In this quote, the expert panel emphasizes a possible deterrent role to the entry of the larger mass-market retailers, but it also recognizes that similar arguments apply to electricity consumers. Latent market power can harm the long-term development of competition in the wholesale and retail markets. The presence of a generator with latent market power may act as a disincentive to investment by large consumers, retailers, or even other generators. This may occur as follows: Consumers and retailers may be concerned that if they invest in a market in which generators have latent market power, generators will exercise that power once the consumer or retailer enters and its investments are sunk. Whether potential new entrants’ concerns are real or perceived matters little, to the extent those concerns deter entry into the market, it will result in long term damage to the market’s competitive processes.

It is evident that this perspective is fundamental to the decisions of regulators and policymakers, to the extent that their primary economic concern is not the minimisation of deadweight loss, but rather a desire to protect the sunk investments by customers. Thus, in the UK, the regulator states that its primary duty is to protect the interests of existing and future consumers.^{Footnote 2}

Looking at the background research on this theme, as it related to electricity, He et al. (2015) and Bragança and Daglish (2017) observed that demand investment has to be modeled as a strategic expansionary move for a profit maximizing retailer or large consumer, appealing to the economic principle that market structure determines market conduct and ultimately prices [\(<1>\) e.g. Fudenberg and Tirole (1989)]. Similarly, Poletti et al. (2015), Rassenti et al. (2003) and Nasser (1997) suggest that strategic demand investments can lead to new market power scenarios. The research contribution of our work is not primarily about market power but the potential for market power to emerge is a crucial ingredient. Thus, more widely there is substantial research on the investment impacts of the exercise of market power by generators related to network and generation assets [\(<1>\) e.g. De Vries (2005), Tohidi et al. (2016), and Agency (2003)] and, in particular, somewhat related to the theme of our work, Murphy and Smeers (2010), noted that if high prices stimulate new generation investment and these are due to the exercise of market power by incumbent generators, then any future generation investment will not generate the predicted revenues based on existing prices, since the capacity investments will alter the market structure.

Turning to modeling market power in electricity markets more generally, it can be studied by computing the non-cooperative equilibrium in wholesale market operation and comparing it to the cooperative efficient market case. To this end, Pereira et al. (2005), Barroso et al. (2006), Hu and Ralph (2007), Hesamzadeh and Biggar (2012) and Steeger and Rebennack (2015) propose Mixed Integer Linear Programs (MILPs) to calculate the market equilibrium in electricity wholesale markets. Reference Hesamzadeh and Biggar (2012) extends the computation of equilibrium in electricity markets to analyze the effect of horizontal mergers between the Gencos on the market prices and equilibrium. Authors in Hesamzadeh and Biggar (2012); Hobbs et al. (2000) and Hu and Ralph (2007) discuss the market equilibrium under oligopolistic conditions while assuming that the market participants have complete information about their rivals. However, the assumption of complete information may not be valid in practice. Thus, for example, Tongia et al. (2017) show that there is a lack of complete information about the market participants in the Indian and Australian markets. Reference Léautier (2019) shows that the market equilibrium may be affected by incomplete information and, as a consequence, Léautier (2001), Li and Shahidehpour (2005) and Acemoglu et al. (2017), Moiseeva and Hesamzadeh (2017), propose the computation of Bayesian Nash equilibria.

The quad-level mathematical model presented in the paper is fairly new and a few works in literature have attempted similar quad-level models. Reference Ramyar and Chen (2020) presents a Leader-follower equilibria for power markets in presence of prosumers. It presents a Mathematical Program with Equilibrium Constraints (MPEC) with Wolfe’s duality to find the Stackelberg equilibria, and concluded that for a prosumer it was always beneficial to be in a Stackelberg Game as a leader than a simultaneous competition game. In this paper the consumer is modeled as a Stackelberg leader to influence the best demand side investment strategy. Reference Bjørndal et al. (2023) presents a quad-level Stackelberg game to analyze the market power exercised by a monopolistic energy storage system operator on short term markets (day-ahead market and real-time balancing market). Authors in Shivaie et al. (2020) present a quad-level vulnerability-constrained model for coordination of generation and transmission investments. In the power sector literature such quad-level problems are under researched and are difficult to solve. In this context, the models proposed in the current paper are one of the early works to analyze the effect of market equilibrium on consumer investment behavior.

The current paper therefore develops its new analysis as follows. Section 3\(<1>\) introduces the assumptions and the preliminary concepts needed for the theoretical results in Sect. 4 and for the quad-level mathematical program in Sect. 5. A computable algorithm for solving the required bilevel MILP is presented in Sect. 5. The computational results using IEEE 14-node system are presented in Sect. 7. Section 8 concludes.

In this section, we discuss the following:

1. 1.

   Underlying assumptions for the model presented and their rationale \(<1>\).

2. 2.

   An introduction to Demand Side Investment problem and its modeling premise.

3. 3.

   Market Clearing Model—Objective and constraints

4. 4.

   The final quad-level problem set up and interactions between: Demand Side Investment Problem, Market Clearing model and Strategic Generator models.

3.1 Assumptions

Assumption 1

The consumers buy electricity and enjoy a utility w.r.t. demand consumption \(d_s\) for all scenarios \(s \in {\mathcal {S}}\). This utility function is approximated by a linear function (Fig. 1a green curve)

where D is the demand. The marginal utility (Fig. 1b green curve) is then simply given by

The generator \(n \in {\mathcal {N}}\) serves the demand in the system incurring a cost for production. This cost function is approximated by a linear function (Fig. 1a red curve) in the generation levels \(g_{n,s}\)

with installed generation capacity G. The marginal cost (Fig. 1b red curve), is then given by

a Utility and Cost functions, b Marginal utility and Marginal cost functions (colour figure online)

Full size image

Assumption 2

The generators and demands submit their supply and demand functions (which are not necessarily their true marginal cost and utility functions) to a market operator, respectively. Based on the received functions, the market operator dispatches the generators and loads, while maximizing the social benefit. This step is called market clearing and since it leads to physical dispatch we refer to it as the spot market (SM). The ISO solves an optimization problem and publishes the market prices \(\mu _s\) and the volumes cleared for generation \(g_{n,s}\) and consumption \(d_s\).

Further, we assume that

This is a reasonable assumption because if \(U_s<C_{n,s}\), then zero generation and zero consumption is optimal in the SM.

In this paper, we compare the case of perfect competition (PC) and imperfect competition (IC).

Assumption 3

(Léautier 2019; Nasser 1997) In PC, the generators submit their true cost functions to the ISO in the SM (Steeger et al. 2014). In contrast, in IC the generators can submit a “modified" cost function, called price offer (PO), with the aim to maximize their profits by influencing the SM prices. A regulatory body imposes a market-power mitigation policy on the generators, enforcing the slope of the offer function \(b_{n} \in [{\underline{B}},{\overline{B}}]\) for \(n \in {\mathcal {N}}\).

Following Assumption 2, we assume that

This assumption ensures that the regulatory body has protected the consumers to some extent from being exploited even if the market becomes extremely monopolistic in nature. We further assume that \(C_{n,s} \le {\overline{B}}\) for all \(n\in {\mathcal {N}}\) and \(s\in {\mathcal {S}}\).

Assumption 4

(Schöne 2009) For ease of discussions, we assume w.l.o.g. that all generators have equal generation capacities G.

Assumption 5

The generation capacities G is public information, being disclosed for their grid connection agreements, whilst the cost functions of the generators remain private. It is assumed that the system has enough generation resources to support any future load increment over the planning horizon. In case of transmission congestion, we assume that all demand can be served.

Assumption 6

(Biggar and Hesamzadeh 2014) In anticipation of the utility to be acquired over a planning horizon T, the consumer makes investments to create the load. The consumer incurs a cost for these sunk investments I(D). The investment cost function \(I(\cdot )\), is a monotonically increasing and differentiable convex function in D.

Assumption 7

(Nasser 1997) Typically in SMs, the generators can submit both price and volume offers, but the modeling issue is whether competition is predominantly in the price or quantity offers. We assume that the competition is predominantly in price offers. This leads to a Bertrand game between the generators with imperfect information (Assumption 5). The unknown true costs of competing generators are approximated by a probability distribution.

3.2 Problem setup \(<3>\)

The consumer needs to decide on the demand investment D. The corresponding optimization problem (DSI) is given by (4). In order to obtain an optimal D, the consumer needs to know the electricity prices \(\mu _s\). This information is provided by the ISO, solving (SM)\(_s\) for all scenarios \(s \in {\mathcal {S}}\). \(<2>\) The scenario index s represents the uncertainty in the grid. The probability of each scenario s is represented by \(\rho _s\). The ISO itself needs to know the bids \(b_{n,s}\) of all generators \(n \in {\mathcal {N}}\) for all scenarios \(s \in {\mathcal {S}}\).

In PC, according to Assumption 3, all generators \(n \in {\mathcal {N}}\) submit their true cost \(C_{n,s}\) for all scenarios \(s \in {\mathcal {S}}\). Thus, the demand investment problem becomes a two-level leader-follower Stackelberg game, cf. Sect. 3.3 and Fig. 2. In IC, the electricity price bids are made by the N strategic generators \(n \in {\mathcal {N}}\), who play a Nash game against each other. The Nash equilibrium is obtained by a bilevel optimization problem, leading to a quad-level optimization problem when combined with the leader-follower Stackelberg game. This is illustrated in Fig. 2.

In the following paragraph we describe the framework of the quad-level problem set up. Any multi-level (quad) optimization problem is a series of nested optimization problems (a four level nested problem is called quad-level optimization problem). This would usually take the form (\(f_i\) and \(x_i\) correspond to the ith level objective and decision variable respectively):

As mentioned previously, the DSI problem is where consumer decided on demand investment D. As in Fig. 2, this forms the first level optimization problem. The D is passed on to a second level problem (SM), we note via the arrow bypass in Fig. 2, that this stage is not directly influenced by D. Rather D influences the third and fourth level problem (strategic generator’s profit maximization and generator’s view of spot market based on rival’s true cost assumptions). It is important to note here that Level III and Level IV problems together form a Bayesian Nash Equilibrium problem between strategic generators as we will see ahead in this paper. D influences the price offers that is discovered in the Bayesian Nash Equilibrium (Level II and IV), which influences prices discovered in Spot Market (Level II), and this in-turn again feedback influence on DSI (Level I) (Table 1).

Quad-level optimization framework

Full size image

3.3 Demand side investments

We formulate the Demand Side Investment (DSI) problem as an anticipatory investment decision, in which the consumer invests to install new load D. The consumer then pays a price \(\mu _s\) when consuming electricity \(d_s\) for demand scenario \(s \in {\mathcal {S}}\). Price \(\mu _s\) is determined by the ISO. We discuss the corresponding optimization problem for the ISO in Sect. 3.4.

This leads to a leader-follower Stackelberg game. The pro-active consumer is a leader who decides the demand investments (thus dictating D). This D becomes an input for the SM stage (follower). At the SM stage, the ISO dispatches the generators based on their POs. The follower stage is an horizontal equilibrium of all generators (based on POs) and the SM dispatch by ISO. As D changes, the price estimates from the SM also change and with that also the horizontal PO equilibrium among the generators. At a complete equilibrium, the leader-follower Stackelberg game (between the consumer and ISO) and simultaneous move game (between strategic generators submitting their POs to ISO) are in equilibrium. This is illustrated in Fig. 2.

For given prices \(\mu _s\) for all scenarios \(s \in {\mathcal {S}}\), the consumer solves the following convex optimization problem:

The DSI objective (4a) is the expected value of the net benefit over T periods from consumption across all stochastic scenarios, with the sunk investment cost of the consumption resource excluded. This is the net long term expected benefit for the consumer due to the DSI level D. The constraint (4b) restricts the consumption level with the lower and upper bounds (where the upper bound is dictated by variable D). \(C_s(b_{n,s},d_s) = \mu _s\) represents the marginal cost from SM stage of the ISO. Unless one of the cost setting generators is also capacity constrained in the SM, one can assume that in the close neighbourhood of \(d_s\): \(<2>\) \(\frac{\partial C_s}{\partial d_{s}} = 0\) (MOSEK 2020). We can therefore replace \(C_s(b_{n,s},d_s^*)\) with \(C_s^*\) at optimality.

The following proposition characterizes an optimal investment level.

Proposition 1

An optimal demand side investment levels D of (4a)–(4b) is proportional to the expected gap between marginal demand utility and market price, specifically,

Proof

See “Appendix 1” \(\square \)

Proposition 1 is important for us because it provides a characterization of the optimal investment level D for changes in prices \(C_s^*\). Specifically, if \(C_s^*\) increases for all \(s \in {\mathcal {S}}\) due to market power exploitation, then at optimality, \(I^{\prime}(D)\) decreases. With that, the optimal investment level D decreases due to Assumption 6.

3.4 Market clearing

In this subsection we focus on the creation of \(C_{s}(b_{n,s},d_s)\). Together with the consumer’s installed load capacity D and the POs \(b_{n,s}\) of the generators \(n \in {\mathcal {N}}\) and scenarios \(s \in {\mathcal {S}}\), the ISO solves the following dispatch problem as a linear optimization (LO) problem for all scenarios \(s \in {\mathcal {S}}\):

The objective function (6a) maximizes the utility minus the sum of generation cost, as a function of the price bids \(b_{n,s}\), whilst the constraint (6b) balances demand and generation for the given scenario s. The transmission constraints are modeled through a ‘DC-Power Flow’ model in constraints (6c). Lower and upper bounds on demand and generation are modeled by constraints (6d) and (6e), respectively. The variables associated with each constraint are the corresponding dual variables of the LO problem. For example, \(\mu _s\) is the spot market price paid by the consumer and paid to the generator(s). We choose a simple transmission model to demonstrate the market power effects on demand investment. A more accurate model would involve the non-convex AC power flow constraints (Frank and Rebennack 2016) which is out-of-the scope of this study.

It is important to show that the price signals and consumption levels in SM (6a)–(6e) are equal to the variables in DSI (4a)–(4b). If this equality is not ensured, then the DSI will have a false estimate of the anticipatory utility it would acquire.

Proposition 2

\(\mu _s\) and \(d_s\) values in SM (6a)–(6e) and in DSI (4a)–(4b) are equal.

Proof

See “Appendix 2”. \(\square \)

\(<3>\) It is important to note that due to Proposition 2, the calculation of \(d_s\) can be determined either via SM or DSI (with a sharing of variable between the two problems). This observation is used in two different ways in this paper:

• In proofs related to Sect. 4, we use SM (also referred to as optimal dispatch later) to determine the \(d_s\).

• In formulations related to Sect. 5, we use DSI to determine the \(d_s\).

Therefore Proposition 2 is the key to linking the analytical section of this paper with the numerical formulation sections.

In this section, we discuss the following:

1. 1.

   Analytic closed form solution to market prices in no congestion case.

2. 2.

   Analytic closed form solution to price offers by strategic generators in no congestion case (monopoly, duopoly and generalized to oligopoly)—{Perfect and Imperfect Competition}.

3. 3.

   Analytic closed form solution to market prices in 3-node congested case.

4. 4.

   Analytic closed form solution to price offers by strategic generators in congestion case (duopoly)—{Perfect and Imperfect Competition}.

4.1 No transmission network constraints

We consider three different cases: (1) One-generator system, (2) Two-generator system, and 3) N-generator system. For the general N-generator system (with \(N>2\)), we restrict our discussions to the case where \(D \le G\).

In the case of no transmission network constraints, (6c) are not binding in \(\text {(SM)}_s\). We refer to this LO problem as \(\text {(SM)}_s^\textrm{NT}\). This has the implication that \(d_s = D\) in any optimal solution of \(\text {(SM)}_s^\textrm{NT}\) for any scenario \(s \in {\mathbb {S}}\), due to Assumptions 3 and 5. We can consequently fix the consumption level \(d_s=D\) for the discussion around the no transmission case.

An optimal SM price, in case of no transmission restrictions, is given by

Corollary 3

An optimal dual variable \(\mu _s\), associated with constraint (6b) of \(\text {(SM)}_s^\textrm{NT}\), for scenario \(s \in {\mathcal {S}}\) is given by

for the one-generator system and

for the two-generator system and

for the N-generator system for \(0< D \le G\).

We provide a formal proof in the “Appendix 3 and 4”. Equations (7a)–(7c) are intuitive, as the SM price is the shadow price associated with constraint (6b), which is (loosely speaking) the reduction in the objective function value when the right-hand-side of (6b) is increased by 1 unit. The results of Corollary 3 are further illustrated in Fig. 3. The price \(\mu _s^\textrm{PC}\) is the y-axis projection of the equilibrium point (marked by \(\times \)).

a 1-Generator system, b 2-Generator system \(0\le D<G\), c 2-Generator system \(G\le D<2G\), d N-Generator system

Full size image

4.1.1 Perfect competition—the base case

In PC, as mentioned in Assumption 3, the strategic generators submit a PO equal to their true marginal cost \(C_s\). From Corollary 3, we thus obtain for scenario \(s \in {\mathcal {S}}\) for the 1-generator system

for the 2-generator system

and for the N-generator system

for the case that \(D \le G\).

4.1.2 Imperfect competition with market power

In IC, the optimal PO of the generators can vary in the range \([{\underline{B}},{\overline{B}}]\) with \(0<{\underline{B}}<{\overline{B}}<U\) (cf. Assumption 3). According to Assumption 5, the true cost of rival generators, \(C_{i,s}\) with \(i \in {\mathcal {N}} {\setminus } \{n\}\), is unknown to generator n. Following Li and Shahidehpour (2005), generator n estimates the joint probability distribution of all other generators’ cost functions. This estimate is a market-wide, rational expectation, given by \(F(\cdot )\). We do not consider strategic generators with heterogeneous beliefs, with \(F_n()\) corresponding to the \(n^{th}\) strategic generator. The rational expectations distribution \(F(\cdot )\) is discussed and used in a similar context by Nasser (1997). “Appendix 5 and 6” reiterate important properties of \(F(\cdot )\) as discussed in Nasser (1997). Now we discuss the price offer strategy.

Proposition 4

\(<4>\) The optimal PO in oligopoly, for \(0\le D\le G\)

The proofs for special monoploy and duopoly cases are given in the “Appendix 7 and 8”. The arguments for the oligopoly case are similar to “Appendix 8”. The proofs are adapted from Nasser (1997). The inequality \(b_{n,s} \le {\overline{B}}\) follows from “Appendix 9”. The integral terms in Proposition 4 reveal the effect of imperfect competition. This shows that the price offers will be above the true cost of generators and as discussed in “Appendix 9” are strictly greater than zero if F(.) has a positive mass distribution. The corresponding SM prices, \(\mu _s^\textrm{IC}\), are then obtained by combining Proposition 4 with Corollary 3.

4.2 With transmission network constraints

We now consider the case of transmission congestion for the following two topologies, with two generators each:

1. 1.

   Two-node system (with one line),

2. 2.

   Three-node system (with three lines).

The two systems are shown in Fig. 4.

a Two-node system, b three-node system

Full size image

For the two-node system, congestion occurs for \(K<\min \{D,G\}\) and \(b_{1,s} < b_{2,s}\) for scenario \(s \in {\mathcal {S}}\). For simplicity, we assume \(K < D \le G\). In the three-node system, the grid is congested if \(K<\frac{D}{3}\) (Biggar and Hesamzadeh 2014; Nasser 1997). We also assume that \(D \le G\). This leads to the following

Proposition 5

For scenario \(s \in {\mathcal {S}}\), the locational marginal price is given by

for a two-node system with transmission line capacity \(K<D \le G\), and

in a three-node system with transmission line capacity \(3K<D \le G\).

See online “Appendix 10 and 11” for the proof.

4.2.1 Perfect competition—the base case

As in the case without transmission constraints, the generators bid at their marginal (true) cost \(C_{n,s}\). Together with Proposition 5, this leads the SM prices \(\mu _s^{PC}\) with congestion.

4.2.2 Imperfect competition with market power

Note that in the two-node system, the SM price is dictated by the PO of generator two (cf. Proposition 5). Because this is a strategic bidding problem, generator two faces a trade-off between a high price at low dispatch quantity (\(D-K\)) and a low price at high dispatch quantity (D). This trade-off is the reason why a PO of \({\overline{B}}\) is not optimal, in general.

Proposition 6

The optimal PO

1. 1.

   in the two-node system duopoly by generator two is for \(K < D \le G\)

   $$\begin{aligned} b_{2,s}=C_{2,s}+ \int _{C_{2,s}}^{{\overline{B}}}{\frac{D-KF(t)}{D-KF(C_{2,s})}dt}. \end{aligned}$$
2. 2.

   in the three-node system duopoly for \(3K<D \le G\) and for \(n=1,2\) is

   $$\begin{aligned} b_{n,s}=C_{n,s}+ \int _{C_{n,s}}^{{\overline{B}}}{\frac{q-3KF(t)}{q-3KF(C_{n,s})}dt}, \end{aligned}$$

         where \(q=\frac{D+3K}{2}\).

We refer to the “Appendix 12 and 13” for the proof.

4.3 Market power negatively affects demand investment

We are now ready to derive our main result. Due to Proposition 1 and equation (5), an increase in the SM price \(\mu _s\) leads to a decrease in optimal demand investment D. This increase in the SM price in case of IC is stated in \(<1>\) the following Lemma.

Lemma 7

The SM price, \(\mu _s^\textrm{IC}\), in presence of imperfect competition exceeds the SM price, \(\mu _s^\textrm{PC}\), in presence of perfect competition, i.e.,

for the one-, two- and N-generator system (with \(D \le G\)) without transmission and, in case of congestion, for the two-node system (\(K<D \le G\)) and three-node system (\(3K<D \le G\))).

The proof is given in “Appendix 14 and 15”.

With that, we obtain the main theoretical result of this paper:

Theorem 8

The optimal demand investment, \(D^\textrm{IC}\), in presence of imperfect competition is lower than the optimal demand investment, \(D^\textrm{PC}\), in presence of perfect competition, i.e.,

for the one-, two- and N-generator system (with \(D \le G\)) without transmission and, in case of congestion, for the two-node system (\(K<D \le G\)) and three-node system (\(3K<D \le G\))) when considering a convexity of demand investment costs. \(<8>\)

For the proof, we refer to “Appendix 16”.

Note that Theorem 8 implies that the presence of market power alone leads to a decrease in demand investment. The crucial implication is that the strategic generators do not even have to exercises market power in order for the decrease in demand investment to occur. The threat alone has this impact on demand investment.

In this section we discuss the following:

1. 1.

   Bilevel problem of a strategic generators to find optimal price offers (Level III and IV)

2. 2.

   Bayesian Nash Equilibrium build up from the bilevel problems and its MILP reformulations (Level III and IV)

3. 3.

   Demand Side investment problem and interaction with BNE (Level I and II—with BNE) and its mixed integer bilevel reformulation

In the table below we enumerate key variables, parameters, and indices that can guide as a quick reference. These are further explained in context ahead in detail (Table 2).

5.1 Strategic price offer of a single generator

A single strategic generator \(n \in {\mathcal {N}}\) solves the following bilevel optimization problem \(\{BLP\}_{s,n}\) for a given scenario \(s \in {\mathcal {S}}\):

As discussed in Sect. 4.1.2, the unknown true cost of rival generators, \(C_{i,s}\) with \(i \in {\mathcal {N}} \setminus \{n\}\), is approximated by the distribution function \(F(\cdot )\). We assume that there are Z discrete value \(z \in {\mathcal {Z}}_n\) with cost \({\hat{b}}_{n,s,z}\) having probability \(\rho _z\). Then, generator \(n \in {\mathcal {N}}\) maximizes its expected profits over all rival’s price estimates as the expected value of the difference of the spot market price, \({\hat{\lambda }}_{s,n,z}\), and its own true generation cost, \(C_{n,s}\), multiplied by the dispatched quantity, \(g_{n,s,z}\), as formalized in (11a). The spot market price, \({\hat{\lambda }}_{s,n,z}\), depends on the bidding price \(b_{n,s}\) and rivals’ bidding prices \({\hat{b}}_{i,s,z}\), given in constraints (11b). The limits of the bidding quantity are given by constraint (11c), reflecting Assumption 3. The SM problem, for the bid \(b_{n,s}\) and all discrete values with index \(z \in {\mathcal {Z}}\) approximating the rivals’ cost function, is given by the second-level optimization problem (11d)–(11h) which then yields the SM price needed for the first-level problem. Note that the second level problem decomposes into Z independent problems.

Because the second-level problem (11d)–(11h) is a linear optimization problem for given \(b_{n,s}\), the KKT-conditions are sufficient and necessary for optimality. Thus, we can re-write \(\{BLP\}_{s,n}\) equivalently as the following continuous non-linear and non-convex optimization problem \(\{BP\}_{s,n}\) with the collection of decision variables \(\Omega ^\textrm{BP}_{s,n} =\) \(\big \{ b_{n,s}, g_{i,s,z},\) \(\lambda _{s,z},\) \( {\overline{\mu }}_{l,s,z},\) \({\underline{\mu }}_{l,s,z},\) \(\nu _{i,s,z}, \omega _{i,s,z} \; | \; i\in I{\setminus }\{ n \}, \ l\in L, \ z \in Z_n \big \}\).

The non-convex, non-linear problem is a reformulation of (11) where the lower-optimization problem is written as the KKT conditions with strong duality. The only non-linear terms are the bilinear terms \(b_{n,s} g_{n,s,z}\). They appear both in the objective function and the strong duality equation. Because both variables \(b_{n,s}\) and \(g_{n,s,z}\) are continuous, its multiplication leads to a non-convex expression which cannot be equivalently reformulated. Therefore, we choose to discretize the PO variables, because the product of a discrete variable with a continuous variable can be equivalently re-formulated used big-\({\mathcal {M}}\) constructs, as both variables \(b_{n,s}\) and \(g_{n,s,z}\) are bounded.

In formulation \(\{BP\}_{s,n}\), the rivals’ PO is fixed. In this case, the terms \({\hat{b}}_{i,s,z}g_{i,s,z}\) are linear. However, we make the rivals’ PO a variable later. Therefore, we discuss the discretization \(b_{n,s} \approx \alpha \sum _{k\in K} \gamma _{k,n,s}\) and \( {{\hat{b}}_{i,s,z} \approx \alpha \sum _{k\in K} {\hat{\gamma }}_{k,i,s,z}}\) together, where \(\alpha = \frac{{\overline{B}}}{|K|}\) and \(\gamma \in \{0,1\}\) is a binary variable. This dicretization leads to problem \(\{BPd\}_{s,n}\), being an approximation of problem \(\{BP\}_{s,n}\):

with \(\Omega ^\textrm{BPd}_{s,n} = \) \(\big \{ b_{n,s},\) \(g_{i,s,z},\) \(\gamma _{k,n,s},\) \(\Gamma _{k,n,s,z},\) \(\lambda _{s,z},\) \({\overline{\mu }}_{l,s,z},\) \({\underline{\mu }}_{l,s,z},\) \(\nu _{i,s,z},\) \(\omega _{i,s,z} \; | \; i\in I {\setminus }\{ n \}, \ l\in L, \ z \in Z_n \big \}\). Constraint group (12m)–(12n) and (12o)–(12p) model \(\Gamma _{k,n,s,z} = \gamma _{k,n,s} g_{n,s,z}\) and \(\Gamma _{k,i,s,z} = {\hat{\gamma }}_{k,i,s,z} g_{n,s,z}\) for \(i\in I{\setminus }\{n\}\), respectively. \({\hat{b}}_{i,s,z}\) are treated as parameters in (12). This assumption however will change as we move to incorporating strategic rival price offers discussed ahead in (15), and necessitated the dicretization via \({\hat{\gamma }}_{k,i,s,z}\) and \(\Gamma _{k,i,s,z}\).

We assume that the rival PO is fixed for (12). At this stage we recast the problem (12) in the form of an exhaustive search using set of inequalities. We enumerate the set of strategies defined by \(\gamma _{k,n,s}\) in a set \({\mathcal {S}}_n\). The profit calculation for the strategic generator n is now a function of \(\gamma _{k,n,s}\) and \({\hat{\gamma }}_{k,i,s,z}\). Assuming rival POs are fixed (i.e. \({\hat{\gamma }}_{k,i,s,z}\) are known), for any strategy \(\gamma _{k,n,s}\) the profit made by the strategic generator n is described by:

Let the different strategies in \({\mathcal {S}}_n\) be represented by \(\{\gamma _{{\mathcal {s}}_n,n,s} \}\). The profit maximization for strategic generator n can be written as:

We note that in (14), we identify particular strategy \(\gamma _{k,n,s}\) from the set \({\mathcal {S}}\), that maximizes the strategic generator’s profits through a set of constraints

5.2 Strategic price offer of a rival

The rival tries to optimize its PO strategies by \({\hat{\gamma }}_{k,i,s,z}\) with an objective of profit maximization \(\forall i\in I\setminus \{n\},~z\in Z_n\). The profit calculation for a rival generator at node e in the informed scenario z when the PO of strategic generator n and the other rival generators \(e|-i\) (including the main strategic generator n discussed in last Section) are known can be represented as:

We enumerate the set of strategies defined by \( {{\hat{\gamma }}_{k,e,s,z}}\) in a set \({\mathcal {S}}_{e,z}\). Let the different elements of the strategy set \({\mathcal {S}}_{e,z}\) be represented by \({\hat{\gamma }}_{{\mathcal {s}}_{e,z},e,s}\). The profit maximization for rival generator e in the informed scenario z can be written as:

5.3 The Bayesian Nash Equilibrium (BNE) of competing generators (proposed MILP model)

The Bayesian Nash Equilibrium from the perspective of strategic generator n in stochastic scenario s can be calculated by solving the strategic PO problem (14) and (16) together. The model allows us to consider all the POs to be variables.

Before we delve into the formulation for a BNE, one might ask the need for a limited rationality model, i.e. need to model information scenarios rather than going ahead with a deterministic Nash Equilibrium model. The reason is more practical rather than technical. The practical information about true heat rates and variable operation costs of generators are proprietary to the generator and not generally disclosed publicly. The generators can only make a guess as to what the true costs of the rival generators could be based on estimated fuel costs and estimate heat rates—which is obtained by finding the make of the generator boiler and turbines and comparing with information available of a similar generator in public domain. The details of open sources for such information \(<1>\) are given in Verma et al. (2021).

Since the true cost of rival generators is unknown, the strategic generator creates informed scenarios z. The strategic generator n optimizes its PO in such a way that its expected profit, over all information scenarios is maximized. In each of the informed scenarios z, the rivals submit their price offers \(\hat{{b}}_{i,s,z}\) with the objective to maximize their individual profits. This is shown for one strategic generator G1 and one rival generator over two information scenarios (\(G2_{z=1}\) and \(G2_{z=2}\)) in conceptual form in Fig. 5.

The Bayesian Nash equilibrium and virtual rivals

Full size image

As shown in Fig. 5, each rival has different virtual models over different informed scenarios z (\(G2_{z=1}\) and \(G2_{z=2}\)). The full BNE model is condensed as (17)

5.4 Multiple Nash Equilibria

There can exist multiple BNEs. The best strategic PO for generator n is obtained from a MILP problem as follows:

This allows the most rational move to be selected by the strategic generator to increase profits. As a consequence of (18) we obtain the strategic PO of n: \(b_{n,s}=BNE^{(n)}_s(C_{n})\). Similar POs are calculated from the perspectives of all the strategic generators \(b_{i,s}=BNE^{(i)}_s(C_{i}),\forall i\in I\).

5.5 The electricity demand investment (MILP model)

In the previous section the stochastic scenarios s have not been discussed in depth. These stochastic scenarios appear because the demand investor does not have true cost information over all generators and, in addition, the future utility of consumption may be uncertain. From a joint probability distribution of generator cost and future utility estimates, the demand investor samples costs as scenario inputs. It is further assumed that the cost estimates made by the demand side investor on the costs of the generators match the estimates that individual generators make about the costs of their rivals. This is the rational expectations hypothesis assuming that all the generators and demand investors have access to common information and interpret it rationally. There are no heterogeneous beliefs. The Demand Side Investor calculates \(b_{i,s}=BNE^{(i)}_s(C_{i,s}),\forall i\in I\) on all these samples \(s\in S\). We represent the POs calculated on these samples as: \(b_{i,s}=BNE^{(i)}_s(C_{i,s}),\forall i\in I,~s\in S\). The POs at each sample \(s\in S\) leads to an optimal dispatch—spot market problem (SM) and yields a Locational Marginal Price (LMP) at the demand investor’s node represented by \(\mu _{y,s}\). These samples are associated with a probability \(\rho _s^{(2)}\). The demand investor solves an optimization problem to select the most optimal investment level. Let the demand investor invests at a node y, then it solves (Fig. 6):

\(<2>\) Note that the determination of \(D_{y,s}\) is made at DSI stage and shared with OD. This is critically possible due to Proposition 2 as mentioned before in this paper.

Problem formulation

Full size image

5.6 The mixed-integer bilevel program

We presented the full formulation in (19). We note (19c) and (19d) are condensed representation of OD and BNE. Further we see that there are two issues with problem (19)

1. 1.

   Since the load at node y is now a variable, we find that the strong duality equations in (19c) and (19d) have new bilinear terms in the form of \(\lambda _{s,z}D_{y,s}\).

2. 2.

   Constraint (19d) is a set of optimization problems.

The first issue for new bilinear terms is addressed by discretizing the \(D_{y,s}\) variable and creating new disjunctions in (19c) and (19d) similar to (12). Now the whole problem (19) is expressed as two stage MILP problem as described below in Fig. 7.

2-Level MILP formulation for Demand investor

Full size image

The OD optimality conditions and the objective for DSI forms the upper level of a 2-stage MILP problem while the BNE calculation forms the lower level of MILP problem.

In this section we develop a technique to recast the bilevel MILP problem posed into a single-level MILP problem. We start by making the following assumptions regarding the bilevel MILP problem:

1. 1.

   All the upper-level variables are discrete (this is true as we have proposed the discretization of upper-level variables to counter the non-convexity arising due to the bilinear terms)

2. 2.

   The lower-level problem is a general MILP with discrete and continuous variables.

We understand that if the lower-level problem were to be linear, then we could have replaced it with its optimality conditions and recast the problem into single stage. But currently we find integer variables in lower-level problem. If we were able to identify cuts for every possible upper-level realization of x, we could replace the lower level problem with its required LP relaxation and cuts. However, there is one issue: we do not know a priori these cuts. There are two ways we can generate these required cuts:

1. 1.

   Gomory cuts method proposed in Küçükyavuz and Sen (2017)

2. 2.

   Manually look for these cuts at suitable intervals.

The parametric Gomory cuts method proposed in Küçükyavuz and Sen (2017) is lexiographic in nature and not suitable for the algebraic modeling in the problem discussed in this paper. We propose a more practical approximate cut generation to recast the bilevel MILP problem to a single-stage MILP problem.

6.1 From continuous upper level to integer upper level

Lemma 9

There exists a closed ball \(B(x,\epsilon )\) and \(\epsilon \ge 0\) around \(x\in {\mathcal {X}}\) in the upper-level feasible region, for which a unique lower-level cut, \(f(y)\le 0\), leads to an integer lower-level solution \(\forall ~ x\in B\)

Proof

Let us assume that \(f_1(y)\le 0\) is a required cut for \(x_1 \in B\) that leads to a feasible integer solution at lower level. Let \(\{x_2\in B\}\). By making this assumption, we have not ruled out a scenario where B is a singleton set and \(x_1=x_2\). In general let us join these with a line \(\{m\in M: m=\theta x_1 + (1-\theta )x_2,~\forall \theta \in [0,1]\}\). By convexity of B, we identify line segment \(M\in B\). We postulate that at some \(q \in [0,1],~ m(\theta )~\forall \theta \in [0,q]\) which is close enough to \(x_1\), the cut \(f_1(y)\le 0\) will lead to integral solution for lower-level problem. This postulate is not true in the case \(\theta =0\) and B is a singleton set [for counter example see example 1 in page 913 of reference Moore and Bard (1990)]. One cannot prove that an open set around x would ever exist due to this counter example but still a closed set can be proven while including the singleton. \(\square \)

This lemma can be rewritten for integer upper-level problems as:

Lemma 10

There exists a closed set partition B(x) and \(\epsilon \ge 0\) around \(x\in {\mathcal {X}}\) in the upper-level feasible region such that \(||x-x_1||\le \epsilon ~\forall x_1\in B(x)\), for which a unique lower-level cut, \(f(y)\le 0\), leads to integer lower-level solution \(\forall ~ x\in B\).

We can conclude that the nature of cuts required for integer lower-level solutions can change for every \(x\in {\mathcal {X}}\). However, if we assume suitable \(\epsilon \), then we can control the error with non-singleton partitions.

We start by making partitions on the upper-level search space. The partitions are made in such a way, that, for the elements in each partition, the maximum distance between the median element and any other element is less than a tolerance level. This tolerance level determines the accuracy of the technique.

It is still necessary to show the existence of an \(\epsilon \) that will lead to non-singleton partitions and lead to optimal integral lower-level solutions. The existence of such a constant is attributed to the finite number of demand investment levels that will lead to a change in the BNE at the lower level, as in Lemma 11.

Lemma 11

For the demand investment problem, the upper-level feasible region can be broken into open set portions, each of which is associated with a unique lower-level cut for integer optimality.

Proof

There are finite number of transmission lines and generators. It is clear from previous results that the BNE changes whenever the demand level rises and one of the constraints become active. This implies the lower-level integer solution changes whenever there are certain active constraints in the system which are directed by the upper-level variable (demand investment). Therefore a particular lower-level cut is valid for a range of demand investment levels (upper-level partition) for integer optimality in the lower-level up until another constraint binds and the BNE changes. Since the number of generators and lines are finite, therefore the possible constraints that can change the lower level BNE are finite. This will lead to an open set partition in the upper level that associates with a unique lower-level cut for integer optimality, wherein no new constraint becomes active and BNE does not change. \(\square \)

We notice that if the upper-level variable is fixed, the bilevel MILP problem collapses into a single-level MILP problem. In the next step, the median element from each partition is selected. Let us consider the median of any partition. The lower-level MILP and a relaxed lower-level LP problem are solved by keeping the upper-level variable fixed at the median. The relaxed LP solution and MILP solution may not match necessarily. In case they do not match, cuts are added to the relaxed LP lower-level problem in order to force a match. We go through all the partitions in this way, adding cuts to the lower-level problem. In this way we generate a unique LP corresponding to each partition of the upper-level search space.

6.2 Approximate cuts method

In the approximate cuts method, we postulate that the entire search space of the upper-level variable x can be segmented into non-empty partitions, in such a way that a unique cut for each partition leads to an integer solution of the relaxed lower-level problem. However, due to Lemma 10, and non existence of open set partitions, it may require that we partition upper-variable search space into singleton sets. In some cases, it may happen that the singleton partition of the upper-level variable may result in a huge number of cuts and, in turn, a huge number of independent lower-level relaxed problems. The approximate cuts method partitions upper-level space into non-singleton partitions in such a way that \(||x-x_1||\le \epsilon ~\forall x_1\in B(x)\) and \(\cup _{i=1}^{N}B(x_i)={\mathcal {X}}\) and assigns a unique cut to each partition. This leads to the following approximation errors:

1. 1.

   Error due to non-integer solution

2. 2.

   Error due to sub-optimal lower level solution

These errors can be reduced by reducing \(\epsilon \) constant.

We have added cuts to the lower-level problems corresponding to each partition based on the median upper-level variable fixes introduced in the previous step. We assume, that this cut is valid for all members of the partition to enforce the LP solution to match the exact MILP lower-level solution. This is an approximation and may not necessarily hold true at all specific cases (see Moore and Bard 1990 example). However, one can clearly see that with a sufficiently low tolerance level, we may achieve an acceptably low approximation error.

6.2.1 Generating the cuts for each partition

1. 1.

   We choose one member from each partition of upper-level partitions.

2. 2.

   Solve MILP and LP relaxation for this member x as constant.

3. 3.

   Enforce a cut \(f(y)\le 0\) for each member so that LP relaxation matches MILP solution (This is achieved by manually tuning the cuts or enforcing \(y=Y_{MILP}\) in a crude implementation)

6.2.2 After generating cuts

After generating cuts, we make a unique copy of lower-level LP relaxation with appropriate cuts for each copy representing each partition over \({\mathcal {X}}\). Optimality conditions for each copy of LP relaxation are now added to the upper-level problem. The optimality conditions for a particular partition should be activated when the solver searches and fixes the upper-level variable in the corresponding partition. This is achieved in the next step by synthesizing a larger MILP problem that allows a switch between the optimality conditions by using binary switches and disjunction techniques. An illustrative example for recasting a bilevel MILP problem to a single-level MILP problem is shown in “Appendix 17”.

We test the computability of proposed Demand Side Investment problem (19) on the Modified IEEE 14-node benchmark system and use the Approximate Cuts method discussed in Sect. 6.2\(<6>\) The problem was formulated and solved using GAMS with a i7-7700K CPU and 16 GB RAM. Most of the data for the modified IEEE 14-node benchmark system was taken from “https://matpower.org/docs/ref/matpower5.0/case14.html" and the line limits were modified to 50 MW. The system under study has 4 strategic generators at Bus No 1, 2, 3, and 6. The corresponding true costs of these strategic generators are modified to \(2\$/MWh,~2\$/MWh,~,5\$/MWh,~5\$/MWh\). In case of PC, the POs are fixed to the generators’ true cost while in case of IC, we assume that the strategic generators can submit a PO in the interval [0, 100]. It was assumed that the utility of a demand investor varies in different stochastic scenarios s as \(U_s\in [20,30]\). The demand investor has three possible options \(DI=\{0,25,50\}~MW\) to invest at node 3 of the system. We calculated the expected net utility for this demand investor under PC and IC. Table 3 shows the expected LMP values under different Demand Investment strategies in PC and IC. These results endorse the previous theory.

The optimal demand investment levels are shown in Table 4

It can be observed that the Demand Investor invests less under IC as compared to PC. \(<6>\)The social benefit shows that the IC can have harmful impact on consumers as well as discourage the higher demand investment. The run times for IC was 720 min, while the run time for PC case was 15 min.

7.1 Why bother with such a sophisticated model?—illustative example

Consider a two-strategic-generator system with two generators G1 and G2 and no congestion in the network, and no capacity limits. For illustration and understanding the impact of the sophisticated model let us assume that the PC and IC price offers (POs) are given below in Table 5. By doing so we are condensing the bi-level BNE model, to a “look up table" in this illustrative example for clarity. Let us further assume that the utility function of demand investor is deterministic and \(U=6\$/MWh\). The demand investment variable \(5 MW \le D \le 10 MW\). The strategic generators have made a demand forecast of 9 MW based on historical data for the planning horizon. This type of demand forecast is typical treatment of the consumer as a passive entity in standard modeling tools such as PLEXOS and PROMOD used in the industry.

In both the cases IC and PC, we see that the price could be set by any strategic generator and the system price is shown in Table 5. To this system price, the demand investor would react and decide the investment levels. We see in Table 5 that the optimal demand investment levels could be drastically different between PC and IC, and further the forecasted demand could be an over or under estimate of real consumer demand in future. A forecast based on the historical data might hold true if the inflation in demand utility matches the inflation in system equilibrium prices (influenced by true cost inflation). A mismatch in inflation rates in the demand sector and generation sector could lead to a situation where forecast based on historical data does not always hold true for active demand consumers.

Whilst it is quite intuitive that large demand-side investors will consider their effects on the supply and demand balance of the market before investing, research on how this may affect prices through increasing the market power of incumbent generators has not previously been undertaken. Furthermore, how this in turn leads to the optimal investments by large consumers has been a open question. We have provided a theoretical analysis and a computational methodology which has led to support for the following conjectures that we introduced. Thus a substantial demand investor will face higher prices in a market with imperfect competition than would have been apparent before the investment. It is possible that a market without substantial market power will have substantial market power because it has attracted demand investment. A demand investor will find it optimal to invest less than it would have intended if an analysis of its effect on generator \(<1>\) behavior is considered. In particular, we have developed a new methodology to determine this optimal size of investment by a large consumer facing latent market power on the generation side.

\(<7>\) We started this paper in Sect. 1 with the following conjectures which have now been clearly addressed.

1. 1.

   A substantial demand investor will face higher prices in a market with imperfect competition than would have been apparent before the investment. This is clearly due to Lemma 7.

2. 2.

   It is possible that a market without substantial market power will have substantial market power because it has attracted demand investment. This is clear due to Proposition 4 and 6. See “Appendices 8, 12, and 13” to see the analysis regarding rate of change of price offers with respect to system demand.

3. 3.

   A demand investor will find it optimal to invest less than it would have intended if an analysis of its effect on generator behavior is considered. This is clear due to Theorem 8.

It is the short-term exercising of market power by generators, which sets a precedence in system operation and results in a lower demand side investment.

Methodologically, for a general nodal power system, we proposed a quad-level program to capture the market interaction between the different players. This was reformulated as a bilevel model to facilitate an efficient solution algorithm. A realistic application was to a case study based on the IEEE 14-bus system. The computational results confirmed the theoretical result that the optimal demand investment decreases in the presence of latent market power.

Apart from developing a methodology to aid the investment deliberations of a large demand side investor, there are some policy and regulatory observations that follow form this work. Generally, market power and competition remedies are activated by the regulators and the competition authorities in response to manifest abuse. This research however shows the need to be more proactive. If consumer investment is deterred by the potential exercise of market power, there is economic harm through the lack of efficient market entry. Mitigation of potential, or latent, market power should therefore be more proactively considered as part of industrial policy.

\(<9>\) We note that in this paper, as motivated in Sect. 1 we use consumer as a leader in a Stackelberg game. However, it might be true that strategic generators might start anticipating such lower demand side investments in future and resort to a Parrondo’s game equilibrium in shorter term for long term benefits. This will require modeling the strategic generators at the same level as demand side investor and leads to a future extension of the work presented in this paper. To the best of authors’ knowledge this is the first work to study the influence of strategic market operation on long term demand side investments. Traditionally the consumer is always modeled as a passive demand forecast as is the case in industry leading software packages like PLEXOS and PROMOD. Another future extension of this work could be analyzing the same results under the assumption that startegic generators are leaders and demand investor is a follower.