An intelligent negotiator agent design for bilateral contracts of electrical energy

In this paper, an intelligent agent (using the Fuzzy SARSA learning approach) is proposed to negotiate for bilateral contracts (BC) of electrical energy in Block Forward Markets (BFM or similar market environments). In the BFM energy markets, the buyers (or loads) and the sellers (or generators) submit their bids and offers on a daily basis. The loads and generators could employ intelligent software agents to trade energy in BC markets on their behalves. Since each agent attempts to choose the best bid/offer in the market, conflict of interests might happen. In this work, the trading of energy in BC markets is modeled and solved using Game Theory and Reinforcement Learning (RL) approaches. The Stackelberg equation concept is used for the match making among load and generator agents. Then to overcome the negotiation limited time problems (it is assumed that a limited time is given to each generator-load pairs to negotiate and make an agreement), a Fuzzy SARSA Learning (FSL) method is used. The fuzzy feature of FSL helps the agent cope with continuous characteristics of the environment and also prevents it from the curse of dimensionality. The performance of the FSL (compared to other well-known traditional negotiation techniques, such as time-dependent and imitative techniques) is illustrated through simulation studies. The case study simulation results show that the FSL based agent could achieve more profits compared to the agents using other reviewed techniques in the BC energy market.

[1]  M. N. Ahmadabadi,et al.  Fuzzy Sarsa Learning and the proof of existence of its stationary points , 2008 .

[2]  Katia P. Sycara,et al.  Automated Bilateral Multiple-issue Negotiation with No Information About Opponent , 2013, 2013 46th Hawaii International Conference on System Sciences.

[3]  Ho-fung Leung,et al.  ABiNeS: An Adaptive Bilateral Negotiating Strategy over Multiple Items , 2012, 2012 IEEE/WIC/ACM International Conferences on Web Intelligence and Intelligent Agent Technology.

[4]  N. R. Jennings,et al.  To appear in: Int Journal of Group Decision and Negotiation GDN2000 Keynote Paper Automated Negotiation: Prospects, Methods and Challenges , 2022 .

[5]  Ariel Rubinstein,et al.  A Course in Game Theory , 1995 .

[6]  Keiki Takadama,et al.  Micro- and Macro-Level Validation in Agent-Based Simulation: Reproduction of Human-Like Behaviors and Thinking in a Sequential Bargaining Game , 2008, J. Artif. Soc. Soc. Simul..

[7]  Katia P. Sycara,et al.  Bayesian learning in negotiation , 1998, Int. J. Hum. Comput. Stud..

[8]  Nicholas R. Jennings,et al.  Using Gaussian Processes to Optimise Concession in Complex Negotiations against Unknown Opponents , 2011, IJCAI.

[9]  Mohammad Shahidehpour,et al.  Market operations in electric power systems , 2002 .

[10]  R. Baldick,et al.  Reanalysis of "Nash equilibrium bidding strategies in a bilateral electricity market" , 2004, IEEE Transactions on Power Systems.

[11]  Enrico H. Gerding,et al.  Scientific approaches and techniques for negotiation. A game theoretic and artificial intelligence perspective , 2000 .

[12]  Hamed Kebriaei,et al.  An agent-based system for bilateral contracts of energy , 2011, Expert Syst. Appl..

[13]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[14]  Guoling Lao,et al.  A Negotiation Model and Its Simulation Based on Q-Learning Algorithm , 2010, 2010 International Conference on Computational Intelligence and Software Engineering.

[15]  Gerhard Weiss,et al.  An Efficient and Adaptive Approach to Negotiation in Complex Environments , 2012, ECAI.

[16]  J. Lawarree,et al.  Nash Equilibrium Bidding Strategies in a Bilateral Electricity Market , 2002, IEEE Power Engineering Review.

[17]  Shaoqing Qu,et al.  The two-stage negotiation mechanism based on multi-agent using Q-learning in electricity bilateral contract , 2010, 2010 Conference Proceedings IPEC.

[18]  Fushuan Wen,et al.  Bilateral transaction bargaining between independent utilities under incomplete information , 2001 .

[19]  Fu-Ren Lin,et al.  Using temporal-difference learning for multi-agent bargaining , 2008, Electron. Commer. Res. Appl..

[20]  Karim Faez,et al.  Applying SAQ-Learning Algorithm for Trading Agents in Bilateral Bargaining , 2012, 2012 UKSim 14th International Conference on Computer Modelling and Simulation.

[21]  Nicholas R. Jennings,et al.  Negotiation decision functions for autonomous agents , 1998, Robotics Auton. Syst..