Filling in the blanks

Scientists at the U.S. Army’s corporate research laboratory are developing a new algorithm that could improve image and audio identification for intelligence gathering on the battlefield.

U.S. Army Combat Capabilities Development Command Army Research Laboratory scientist Dr. Michael S. Lee and co-workers are developing a deep-learning algorithm called a shortcut autoencoder that can restore single audio clips and images corrupted by various types of random noise.

What sets their work apart from previous studies is that they have improved applicability to 1-D signals (e.g., human speech), and are testing against stronger noise sources than usually considered, i.e., noise/signal ratios beyond 1.0.

“Deep learning is well known for being able to accurately detect objects in images, but it is also capable of synthesizing realistic-looking data, such as observed in the recently popular FaceApp,” Lee said. “In our work, we use deep learning to reconstruct an image based on limited input information, for example, with only one percent of the pixel channels retained.”

Lee said his team’s model is trained with a lot of data of what other real pictures look like, and a variant of their image model can be used to reconstruct human speech from noisy audio signals even when the noise is much louder than the signal.

According to Lee, target Army applications are numerous, including eavesdropping, demodulating communications in the presence of strong jammers and perception of objects in image/video that are obscured intentionally, by darkness (low-light) or by weather events such as fog and rain.

“In the short run, this technology could provide a ‘Zoom/Enhance’ function for intelligence analysts,” Lee said. “In the long run, this type of technology may be seamlessly integrated into a camera’s hardware for improved image quality under various scenarios such as low-light and fog.”

In addition to Army applications, Lee noted that the commercial sector could benefit from this technology as well.

“In low-bandwidth environments, such as areas far away from cell towers, algorithms like ours could provide clearer phone calls,” Lee said. “Self-driving cars may benefit from this technology in extreme weather scenarios like rain and fog to infer what objects are ahead. Commercial video cameras will be able to operate in lower light conditions with higher frame rates and/or lower exposure times.”

This work addresses challenges within the Network Command, Control, Communication and Intelligence Cross-Functional Team.

“Part of CCDC ARL’s mission is to explore the realm of what is possible,” Lee said. “Here, we show that beyond detection and classification, machine learning can be used for the elucidation of weak and/or noisy signals and images.”

Moving into the future, Lee and his colleagues would like to explore how this method will work on data types beyond human speech and optical images, such as physical environment sensor data and wireless communication.

###