Hydrogels can play Pong by “remembering” previous patterns of electrical simulation

Non-living hydrogels can play the video game Pong and improve their gameplay with more experience, researchers report August 23 in the Cell Press journal Cell Reports Physical Science.  The researchers hooked hydrogels up to a virtual game environment and then applied a feedback loop between the hydrogel’s paddle—encoded by the distribution of charged particles within the hydrogel—and the ball’s position—encoded by electrical stimulation. With practice, the hydrogel’s accuracy improved by up to 10%, resulting in longer rallies. The researchers say that this demonstrates the ability of non-living materials to use “memory” to update their understanding of the environment, though more research is needed before it could be said that hydrogels can “learn.”

“Ionic hydrogels can achieve the same kind of memory mechanics as more complex neural networks,” says first author and robotics engineer Vincent Strong of the University of Reading. “We showed that hydrogels are not only able to play Pong; they can actually get better at it over time.”

The researchers were inspired by a previous study that showed that brain cells in a dish can learn to play Pong if they are electrically stimulated in a way that gives them feedback on their performance.

“Our paper addresses the question of whether simple artificial systems can compute closed loops similar to the feedback loops that allow our brains to control our bodies,” says corresponding author and biomedical engineer Yoshikatsu Hayashi of the University of Reading. “The basic principle in both neurons and hydrogels is that ion migration and distributions can work as a memory function that can correlate with sensory-motor loops in the Pong world. In neurons, ions run within the cells; in the gel, they run outside.”

Hydrogels are complex polymers that become jelly like when hydrated—gelatin and agar are natural examples. In this case, the researchers used an “electro-active polymer,” meaning a hydrogel that can respond to electrical stimulation thanks to the presence of ions (charged particles) in the media surrounding its polymer matrix. When the hydrogel is electrically stimulated, the ions move, dragging water molecules with them, and this movement causes the hydrogel to temporarily change shape.

“The rate at which the hydrogel de-swells takes much longer than the time it takes for it to swell in the first place, meaning that the ions’ next motion is influenced by its previous motion, which is sort of like memory occurring,” says Strong. “The continued rearrangement of ions within the hydrogel is based off of previous rearrangements within the hydrogel, continuing back to when it was first made and had a homogeneous distribution of ions.”

To test whether the hydrogel’s physical “memory” could enable it to play Pong, the researchers used electrodes to connect the hydrogel to a virtual game environment and started up the game by sending the ball in a random direction. They used electrical stimulation to inform the hydrogel of the ball’s position and measured the movement of ions within the hydrogel to determine the position of its paddle.

As the Pong games played out, the researchers measured the gel’s hit rate and examined whether its accuracy improved. They showed that, with more experience, the hydrogel was able to hit the ball more frequently, resulting in longer rallies. Whereas the Pong-playing neurons achieved their optimal ball-skills within around 10 minutes, the hydrogel took closer to 20 minutes to reach its maximum Pong potential.

“Over time, as the ball moves, the gel gathers a memory of all motion. And then the paddle moves to accommodate that ball within the simulated environment,” says Strong. “The ions move in a way that maps a memory of all motion over time, and this “memory” results in improved performance.”

Because most existing AI algorithms are derived from neural networks, the researchers say that hydrogels represent a different kind of “intelligence” that could be used to develop new, simpler algorithms. In the future, the researchers plan to further probe the hydrogel’s “memory” by examining the mechanisms behind its memory and by testing its ability to perform other tasks.

“In our follow-up projects, we are thinking about how to extract the algorithm from the hydrogels that allows memory acquisition,” says co-author William Holderbaum of the University of Reading.

“We’ve shown that memory is emergent within the hydrogels, but the next step is to see whether we can also show specifically that learning is occurring,” says Strong.

###

This research was supported by Process Vision Ltd.

Cell Reports Physical Science, Strong, Holderbaum, and Hayashi, “Electro-Active Polymer Hydrogels Exhibit Emergent Memory When Embodied in a Simulated Game-Environment”

Cell Reports Physical Science (@CellRepPhysSci), published by Cell Press, is a broad-scope, open access journal that publishes cutting-edge research across the spectrum of the physical sciences, including chemistry, physics, materials science, energy science, engineering, and related interdisciplinary work. Visit: https://www.cell.com/cell-reports-physical-science/home. To receive Cell Press media alerts, please contact press@cell.com.