Undulating fins provide biological swimmers with a remarkable propulsion system. Replication with autonomous underwater robots is not simple, both on the technical side and due to the complexity of the control system. Reinforcement learning is applied to a bio-inspired vessel for maneuver control by manipulating the kinematics of its undulating membrane. The controller is initially trained with a numerical model recreating the distributed forces and moments on the fin, and then applied to an underwater vehicle. It was found that using Reinforcement Learning the vehicle was able to perform key maneuvers including control for speed, heading, and turning. The results demonstrate that reinforcement learning has the potential to overcome high levels of system complexity while delivering an optimal solution.
Biological swimmers have achieved remarkable propulsion capabilities through undulatory fins. Nature has evolved to a point where the use of a single fin in combination with pectoral fins are able to control swimming in multiple directions with high levels of accuracy. The bio-inspired application of these traits to underwater vessel locomotion presents a non-trivial task. The bio-inspiration was taken from the Gymnotiformes, known as South American knifefish. Normally, the fish has an elongated fin along its body, driven by hundreds of bones in an undulatory manner, producing the necessary forces and moments to move freely in all directions. It has been observed that these kinds of fish have developed several fin kinematics for different kinds of motion, ranging from sinusoidal traveling wave-like kinematics for longitudinal motion, to combination of more than one wave for vertical displacement among others, all combined with body bending for lateral-directional movement. The potential artificial mimicking by underwater robotic devices imposes a level of complexity in the design of control systems that quickly rules out traditional linear model-based controllers. The operation of a robotic undulatory fin composed of several rotatory rays in place of the fish fin bones, entails a multivariable process that requires a different approach. Our current research involves an autonomous device equipped with an undulatory fin actuated by 16 rays, with the capability of controlling each ray asynchronously, regulating its position and speed for arbitrary path following. The degree of intricacy of the system, while trying to keep a close resemblance to the natural swimmer, strongly suggests the need for a machine learning approach, especially technique such as reinforcement learning that is capable of learning during operation. We have developed a two-input two-output 3 DOF path-following control system for a bio-inspired robot based on a specific pattern of ray motion. The controller modulates the amplitude of a traveling wave for speed control, and coordinates the offsetting of a subset of rays for the lateral-directional dynamics, while using the closest distance to the trajectory and its current speed as inputs. The reinforcement learning controller was tested in various conditions experimentally, including an indoor flume, and indoor and outdoor open tanks to perform different maneuvers, with different levels of external disturbances.
