AI helps discover new space anomalies

The SNAD team, an international network of researchers including Matvey Kornilov, Associate Professor of the HSE University Faculty of Physics, has discovered 11 previously undetected space anomalies, seven of which are supernova candidates. The researchers analysed digital images of the Northern sky taken in 2018 using a k-D tree to detect anomalies through the ‘nearest neighbour’ method. Machine learning algorithms helped automate the search. The paper is published in New Astronomy.

Most astronomical discoveries have been based on observations with subsequent computations. While the total number of observations in the 20th century was still relatively small, the volumes of data drastically increased with the arrival of large-scale astronomical surveys. For example, the Zwicky Transient Facility (ZTF), which uses a wide-field view camera to survey the Northern sky, generates ∼1.4 TB of data per night of observation and its catalogue contains billions of objects. Manually processing such enormous amounts of data is both expensive and time consuming, so the SNAD team of researchers from Russia, France and the US came together to develop an automated solution.

When scientists examine astronomical objects, they observe their light curves, which show variations of an object’s brightness as a function of time. The observers first identify a flash of light in the sky and then follow its evolution to see whether the light becomes brighter or weaker over time, or goes out. In this study, the researchers examined a million real light curves from the ZTF’s 2018 catalogue and seven simulated live curve models of the types of objects under study. In total, they followed some 40 parameters, including the amplitude of an object’s brightness and the timeframe. 

‘We described the properties of our simulations using a set of characteristics expected to be observed in real astronomical bodies. In the dataset of approximately a million objects, we were looking for super-powerful supernovae, Type Ia supernovae, Type II supernovae, and tidal disruption events,’ explains Konstantin Malanchev, co-author of the paper and postdoc at the University of Illinois at Urbana-Champaign. ‘We refer to such classes of objects as anomalies. They are either very rare, with little-known properties, or appear interesting enough to merit further study.’

The light curve data from real objects was then compared to that of simulations using the k-D tree algorithm. A k-D tree is a geometric data structure for splitting space into smaller parts by cutting it with hyperplanes, planes, lines or points. In the current research, this algorithm was used to narrow down the search range when looking for real objects with properties similar to those described in the seven simulations.

Subsequently, the team identified 15 nearest neighbours, ie real objects from the ZTF database, for each simulation—105 matches in total, which the researchers then examined visually to check for anomalies. The manual verification confirmed 11 anomalies, of which seven were supernova candidates, and four were active galactic nuclei candidates where tidal disruption events could occur. 

‘This is a very good result,’ comments Maria Pruzhinskaya, a co-author of the paper and research fellow at the Sternberg Astronomical Institute. ‘In addition to the already-discovered rare objects, we were able to detect several new ones previously missed by astronomers. This means that existing search algorithms can be improved to avoid missing such objects.’

This study demonstrates that the method is highly effective, while relatively easy to apply. The proposed algorithm for detecting space phenomena of a certain type is universal and can be used to discover any interesting astronomical objects, not limited to rare types of supernovae. 

‘Astronomical and astrophysical phenomena which have not yet been discovered are in fact anomalies,’ according to Matvey Kornilov, Associate Professor of the HSE University Faculty of Physics. ‘Their observed manifestations are expected to differ from the properties of known objects. In the future, we will try using our method to discover new classes of objects.’