The article deals with reinforcement learning methods adapted to the gas transport management. Reinforcement learning, a part of machine learning, is nowadays one of the most active research areas in artificial intelligence. The goal of reinforcement learning is to learn good policies for sequential decision problems by optimizing a cumulative future reward signal. Notwithstanding of successes of reinforcement learning in playing games, its main practical role can be seen in exploring optimal decisions in managing real processes such as acting robots in an unknown environment, riding autonomous vehicles, and controlling industrial processes.

The purpose of the paper is to show how the reinforcement learning approaches can solve problems of a gas transport control in transient conditions. We will consider two situations. In first, an agent (controller) is learned to optimally and sequentially operate a real compressor station by adjusting its pressure set point. In second, it is learned to control revolutions of a single compressor (working area constrains are explicitly modeled). In both cases technical, operational and contract restrictions of pressure and flow are fulfilled. The agent must properly react when offtakes from the network change in time.

Learning process is simulation based and it utilizes an appropriate simulator of transient gas flow. Employed reinforcement learning methods must cope with continuity of space and actions. It is shown that, despite the long learning process, once the agent is trained, it is able to make right autonomous decisions, or at least help a dispatcher.


There are three main areas in machine learning: supervised learning, unsupervised learning and reinforcement learning. Supervised learning is based on training set or labeled examples provided by external supervisor. On the other hand, methods in unsupervised learning try to find patterns in collection of unlabeled data.

Reinforcement learning belongs somewhere between supervised and unsupervised learning. It depicts an autonomous agent who has interactions with surrounding environment and tries to learn how to choose optimal actions for achieving a desired goal.

This content is only available via PDF.
You can access this article if you purchase or spend a download.