Renewable power sources such as wind and solar are inflexible in their energy production, which requires demand to rapidly follow supply to maintain an energy balance. Promising controllable demands are heat buffers that use electricity to maintain a temperature at a setpoint. Such Thermostatically Controlled Loads (TCLs) have been shown to be able to follow a power curve using reactive control. In this paper, we investigate the use of planning under uncertainty to pro-actively control an aggregation of TCLs to overcome temporary imbalances. We present a formal definition of the planning problem under consideration, which we model using the Multi-Agent Markov Decision Process (MMDP) framework. Since we consider hundreds of agents, solving these MMDPs directly is intractable. Instead, we propose decomposing the problem by decoupling the interactions through arbitrage. Decomposition of the problem means relaxing the joint power consumption constraint, which means that joining the plans ...
Frits de Nijs, Matthijs T. J. Spaan, Mathijs de We