# Subsidies-Free Renewable Energy Trading: A Meta Agent Approach

Distributed energy resources (DER) are becoming ubiquitous in deregulated electricity markets. Renewables sources take a significant share of the total DER installed capacity. The variability and uncertainty characteristic of renewable energy has a tremendous impact on profit making. A work around this problem are government support in the form of subsides and tax incentives. These schemes aim to foster investment in renewables by socializing risk. In the foreseen future the renewable/fossil-fuel ratio is required to expand significantly deeming support schemes no longer bearable. Hence, countries had begun to update their energy legislation and market design. The new laws require renewable energy producers to be liable of balancing their positions; whereby coming closer to market fairness. For example, in Ireland the Renewable Electricity Feed-in Tariff support scheme, a public service obligation levy charged to electricity consumers, will cease to exist. Instead, new energy markets will be introduced to balance close to real-time the traders’ energy commitments.

This work tries to provide a sustainable solution for electricity retailers of hybrid wind power plants (HWPP). The work addresses the central question of the HWPP operation: How to dynamically allocate the HWPP generation and operate the pumped storage in order to reduce the trading costs? In fact, revenues from intermittent power trading depend on two main factors: 1) The reliability of the committed power and 2) Wholesale prices. The renewable energy producer is subject, as a consequence of weather variability, to up and down regulating prices. A better outlook of wholesale price would signify less exposure to the more volatile balancing market; thereby reducing costs and increasing profit.

Two mainstream approaches exist to balance supply and demand. On the one hand, accurate forecasting of high impact variables (e.g. weather and energy price). On the other hand, energy storage. The sought result is the smoothing of the total power output. Recent studies in simulated hybrid power scenarios have proposed agent modeling to minimize trading cost. However, most works have focused on consumption management in office or residential areas and no relevant study has been done on the energy management of the supply side.

In this article we presents the MAL, an holistic agent-based methodology to power supply management. The MAL incorporates a tiered framework to manage a hybrid energy system on behalf of a power producer. The three fundamental instances of the MAL are 1) A clustering algorithm to draw knowledge out of the raw data; 2) A deep sequence-to-sequence recurrent neural network to forecast spot prices; and 3) A multi-policy $Q$-learning algorithm for decision-making.

To train and validate the agent’s performance we used real weather and market data. In particular, the data for the latter was conveyed from NordPool, The NordPool a leading energy market in Europe consisting of 20 countries and expanding. The validation was done for a price-taker HWPP and benchmarked with three traders: 1) Perfect information; 2) Markovian agent; and 3) A vanilla artificial neural network.

The contributions are threefold. First, a robust machine intelligence framework for autonomous energy trading. A wait-and-see strategy and the foundational Q-learning (QL) algorithm performs better than a more sophisticated variation. Most importantly, this translates as a reliable operation of the hydro plant. Second, a sequence-to-sequence model to predict spot energy prices that advances previous ANN implementations. Third, a thorough analysis of energy consumption and prices of real-valued market data. Recent studies have focused on the scheduling and trading challenges while regarding the dynamics of the market price as periodic or simply as scenarios from a normal distribution. In the long-term the periodic assumption is not applicable. Whereas our method considers real-time adaptive decision-making. Unlike existing approaches it does not rely on a probability density of market data.

Figure 1 shows the hourly costs and control of the HWPP, the wholesale price, demand, wind power, and hydro usage. The agent learns a cost-effective charge/discharges of the hydro plant. It discharges the hydro plant when there is lack of wind power. The effect of the wait-and-see strategy can be seen in holding the hydro capacity and discharging when the spot price is disadvantageous. For example, the shaded areas highlight a discharge of the hydro plant to cover the lack of wind power. However, based on the predictions, waits until a peak price. Also, prevents wind spillage and curtailment by pumping water to the reservoir when the spot price is low or with surplus wind power (left most shaded area).

The robustness of MAL is tested with random outages of the hydro plant and/or the wind farm. As a consequence of lack of internal energy capacity, the results show an increase in the overall TPC as expected. However of more significance is the control of the hydro plant. In Fig. 2 is observed that the outages does not impact the management of the power flows by either dry running or overflowing the reservoirs. The shaded areas of the figure $\upsilon$ stands for wind power outage, $\zeta$ for a failure of the hydro plant and in $\lambda$ both plants are out of service. In the first outage event, both plants are out for a period of 5 hrs. Followed by the hydro plant for 24 hrs. Lastly, in the third event the wind plant is out of service for 24 hrs.

The MAL brings together energy and the stat-of-the-art in machine intelligence. The work facilitates the integration of renewables reducing the gap of self-sustainment and support-free markets. A direct impact could be realized in lower energy tariffs to electricity customers and a seamless increase of the renewable quota.

Authors: Genaro Longoria, Alan Davy and Lei Shi.

Journal: IEEE Transactions on Sustainable Energy