Learn to swim: Online motion control of an underactuated robotic eel based on deep reinforcement learning

Animal movements result from a complex balance of many different forces. Muscles produce force to move the body; the body has inertial, elastic, and damping properties that may aid or oppose the muscle force; and the environment produces reaction forces back on the body. The actual motion is an emergent property of these interactions. To examine the roles of body stiffness, muscle activation, and fluid environment for swimming animals, a computational model of a lamprey was developed. The model uses an immersed boundary framework that fully couples the Navier-Stokes equations of fluid dynamics with an actuated, elastic body model. This is the first model at a Reynolds number appropriate for a swimming fish that captures the complete fluid-structure interaction, in which the body deforms according to both internal muscular forces and external fluid forces. Results indicate that identical muscle activation patterns can produce different kinematics depending on body stiffness, and the optimal value of stiffness for maximum acceleration is different from that for maximum steady swimming speed. Additionally, negative muscle work, observed in many fishes, emerges at higher tail beat frequencies without sensory input and may contribute to energy efficiency. Swimming fishes that can tune their body stiffness by appropriately timed muscle contractions may therefore be able to optimize the passive dynamics of their bodies to maximize peak acceleration or swimming speed.

Dielectric elastomer actuators (DEAs) are a promising enabling technology for a wide range of emerging applications, including robotics, artificial muscles, and microfluidics. This is due to their large actuation strains, rapid response rate, low cost and low noise, high energy density, and high efficiency when compared with alternative actuators. These properties make DEAs ideal for the actuation of soft submersible devices, although their use has been limited because of three main challenges: (i) developing suitable, compliant electrode materials; (ii) the need to effectively insulate the actuator electrodes from the surrounding fluid; and (iii) the rigid frames typically required to prestrain the dielectric layers. We explored the use of a frameless, submersible DEA design that uses an internal chamber filled with liquid as one of the electrodes and the surrounding environmental liquid as the second electrode, thus simplifying the implementation of soft, actuated submersible devices. We demonstrated the feasibility of this approach with a prototype swimming robot composed of transparent bimorph actuator segments and inspired by transparent eel larvae, leptocephali. This design achieved undulatory swimming with a maximum forward swimming speed of 1.9 millimeters per second and a Froude efficiency of 52%. We also demonstrated the capability for camouflage and display through the body of the robot, which has an average transmittance of 94% across the visible spectrum, similar to a leptocephalus. These results suggest a potential for DEAs with fluid electrodes to serve as artificial muscles for quiet, translucent, swimming soft robots for applications including surveillance and the unobtrusive study of marine life.