A new artificial intelligence system designed to identify fish species in difficult underwater images is promising to transform the way scientists monitor marine ecosystems. Researchers from the Computer Vision Research Laboratory at Sahand University of Technology in Iran have introduced Fish-GCvT, a hybrid deep-learning architecture that combines convolutional neural networks, Vision Transformers, and graph convolutional networks. According to the study, the system achieved classification accuracies above 99 percent across several benchmark underwater fish datasets, outperforming a range of existing approaches. The work addresses a growing need for rapid, automated biodiversity assessment as scientists, fisheries managers, and conservation groups increasingly rely on image-based monitoring.
Recognizing fish from photographs or video is far more complicated than it may appear. Species that are visually distinct in a field guide can look remarkably similar when captured underwater, especially when images are blurred, poorly illuminated, partially obstructed, or surrounded by coral, vegetation, sediment, or other animals. Water selectively absorbs light, often weakening red and yellow wavelengths and producing blue-green colour casts. Suspended particles scatter light, while changing depth and camera position alter contrast and apparent shape. Fish may also turn away from the camera, hide behind objects, or appear only as small regions within a large frame. These conditions can cause conventional image-classification models to focus on misleading details such as background colour, reflections, or water artefacts rather than on the animal itself.
Fish-GCvT was developed around the idea that no single neural-network design is equally effective at capturing every visual clue. Convolutional neural networks, commonly known as CNNs, are particularly good at detecting local patterns. Their filters can learn edges, scales, fin textures, spots, stripes, body contours, and other fine details that are often essential for distinguishing closely related species. However, CNNs typically build their understanding through local receptive fields. Although deeper layers can eventually incorporate broad context, the relationship between distant regions of an image may not be represented as directly or efficiently as it is in an architecture designed for global modelling.
The second component of the system is a Vision Transformer, or ViT. Rather than processing an image only as a continuous grid of pixels, a Vision Transformer divides it into fixed-size patches. Each patch is converted into a numerical feature representation, often called an embedding, and the collection of embeddings is processed using self-attention. This mechanism allows the model to compare patches with one another and estimate which regions are relevant to the classification decision. A patch containing a fish’s tail can therefore be related to another containing its head, even when background regions lie between them. In underwater recognition, this ability to combine separated visual evidence may help the model interpret body proportions, fin placement, colour patterns, and posture as parts of a single biological structure.
Fish-GCvT adds a third layer of reasoning by treating image patches as nodes in a graph. In this representation, each node contains the learned features of one patch, while edges describe relationships between patches. The researchers construct a dynamic graph using both feature similarity and spatial proximity. Patches that appear visually related or occupy nearby positions can be connected, enabling information to move between them. A graph convolutional network then performs neighbourhood aggregation: each node updates its representation by combining its own features with information received from connected nodes. This process can reinforce meaningful relationships while reducing the influence of isolated or ambiguous regions.
The graph stage is important because an underwater image is not merely a collection of independent squares. The appearance of one region can change the interpretation of another. A small patch containing a fin may be difficult to classify on its own, but its position relative to the body, its texture compared with the surrounding fish, and its similarity to other patches can provide valuable context. Likewise, the graph can help connect visually separated portions of an animal when the body is interrupted by occlusion or clutter. Because the connections are generated from the features and spatial arrangement of each image, the structure is dynamic rather than fixed in advance. The network can therefore adapt its relational reasoning to different fish poses and environments.
The overall architecture is intended to create a multi-level feature representation. CNN layers first extract local spatial and textural information. Vision Transformer blocks refine those representations by modelling long-range dependencies through self-attention. The graph convolutional network subsequently propagates contextual information across related image regions. A global aggregation layer combines the resulting patch-level features, and a fully connected classifier produces the final species prediction. In practical terms, the system attempts to answer three questions simultaneously: what fine details are present, how do distant regions belong together, and which image patches share a meaningful structural relationship?
The reported performance is striking, but the significance of the research extends beyond a single accuracy figure. Automated fish identification could support long-term ecological surveys in locations where manual annotation is slow, expensive, and dependent on a limited number of specialists. Underwater cameras, remotely operated vehicles, autonomous platforms, and diver-mounted systems can collect enormous quantities of imagery. Turning those recordings into usable ecological information requires models capable of processing images at scale. A reliable classifier could help estimate species distributions, detect changes in community composition, identify invasive organisms, and provide earlier warnings of environmental disruption. It could also assist fisheries operations by improving catch documentation and supporting more transparent monitoring.
The researchers’ emphasis on visually challenging underwater conditions reflects one of the central problems in deploying artificial intelligence outside the laboratory. A model may achieve exceptional results when training and test images come from the same dataset, yet perform less reliably when confronted with a new camera, location, water depth, season, or lighting environment. Differences in image resolution, species abundance, habitat structure, and annotation quality can all influence performance. The study states that Fish-GCvT was evaluated on multiple publicly available benchmark datasets, suggesting an effort to examine its behaviour across more than one source of imagery. Even so, real-world validation across geographically separated ecosystems and continuously changing underwater conditions will be important before the system can be considered universally dependable.
The architecture also illustrates a broader direction in computer vision: combining specialised forms of reasoning instead of forcing one model to handle every task. CNNs remain effective at learning local visual features, Transformers excel at modelling global interactions, and graph networks provide a framework for relational structure. Their integration increases computational and architectural complexity, but it may offer a better match for biological images in which identity depends on both small details and the arrangement of parts. The approach could eventually be extended beyond fish to coral species, plankton, seabed organisms, or other wildlife whose appearance is distorted by natural environments. It may also be adapted for video, where information from consecutive frames could help classify animals that are briefly hidden or poorly oriented.
The claim of accuracy above 99 percent should therefore be interpreted in context rather than as a guarantee that every underwater photograph will be classified correctly. The most difficult cases are likely to involve rare species, severe occlusion, multiple animals in one frame, unusual life stages, or species absent from the training data. A system trained to choose among known categories may still assign a confident but incorrect label when it encounters an unfamiliar animal. Future studies will need to examine class imbalance, uncertainty estimation, cross-dataset generalization, energy consumption, and inference speed on field equipment. Explainability will also matter: tools such as activation maps could help researchers determine whether the model is using biologically meaningful regions rather than accidental correlations.
By joining local texture analysis, global attention, and graph-based context, Fish-GCvT offers a technically ambitious response to one of marine science’s most persistent bottlenecks: converting underwater imagery into dependable species-level knowledge. Its reported results suggest that hybrid architectures can capture the complexity of fish appearance more effectively than isolated CNN or Transformer pipelines. If the performance holds under diverse field conditions, systems of this kind could become an important part of automated biodiversity observatories, allowing scientists to monitor aquatic life more frequently and over larger areas. The broader message is clear: the future of marine conservation may depend not only on putting more cameras underwater, but also on building smarter methods to understand what those cameras see.
Subject of Research: Automated fish species classification using hybrid deep learning and underwater images.
Article Title: Fish-GCvT: a hybrid deep learning architecture based on Convolutional Vision Transformers and graph convolutional networks for fish species classification
Article References: Asgari, A., Ebrahimnezhad, H. & Sedaaghi, M.H. “Fish-GCvT: a hybrid deep learning architecture based on Convolutional Vision Transformers and graph convolutional networks for fish species classification.” Knowledge and Information Systems 68, Article 247 (2026).
Image Credits: AI Generated
DOI: https://doi.org/10.1007/s10115-026-02863-6
Keywords: Fish species identification, convolutional patch embedding, Vision Transformer, graph convolutional networks, multi-level feature representation, underwater image classification.
