AI Model Improves Mini-LED Display Inspection Accuracy

Published in the journal Frontiers of Optoelectronics, the study introduces a modified high-resolution network (Hrnet) incorporating a residual global context attention (RGCA) module and a mixed dilated convolution and dense upsampling convolution (MDC-DUC) module. The primary objective of this work is to address the challenges posed by the miniature scale of Mini-LEDs, which constrains the massive detection of dead pixels and the precise placement of individual light-emitting diodes in high-performance liquid crystal displays.

The Mini-LED technology has emerged as a leading solution for automotive flat-panel displays, offering locally dimmable backlighting for high dynamic range (HDR) screens. This technology provides superior brightness, enhanced contrast, and a long operational lifespan compared to conventional lighting solutions. However, the manufacturing and integration of these panels require high-precision inspection algorithms. Conventional inspection systems and manual methods struggle to meet the requirements for the compact Mini-LED layout, where even a small deviation in LED position or brightness can seriously affect the overall display quality, leading to uneven brightness, color distortion, or localized shadowing.

To overcome these limitations, the research team proposed a new network architecture built upon the Hrnet backbone. The Hrnet is chosen for its ability to maintain high-resolution feature maps and integrate information at different scales simultaneously, which is crucial for detecting objects with unclear boundaries or fine details. The proposed model integrates the RGCA module to enhance the network’s ability to focus on the luminous region of each Mini-LED. This attention mechanism allows the network to prioritize specific regions of the input data, improving the detection efficiency of small differences in the densely distributed chips.

In addition to the attention mechanism, the model incorporates the MDC-DUC module to broaden the receptive field of the network. Mixed dilated convolutions with varying dilation rates enable the effective featuring of Mini-LED locations at different scales. The dense upsampling convolution component replaces standard bilinear interpolation, improving the capture of individual details for small Mini-LEDs with distinct boundaries. This combination allows for accurate semantic segmentation of Mini-LEDs by smoothly fusing multi-target feature maps, ensuring that the model can distinguish individual chips that might otherwise be misclassified as background.

The experimental setup involved the use of a CCD camera model MV-CS050-10GM paired with a 12 mm industrial lens to capture high-resolution images of the Mini-LED backlight panels. The camera achieves a resolution of 2448 by 2048 pixels with a pixel size of 3.45 micrometers. The specific Mini-LED backlight panel used in the study measures approximately 76 cm in length and 11.8 cm in width, containing 1733 individual Mini-LED chips. These chips measure about 100 micrometers by 200 micrometers with a spacing of 850 micrometers, arranged in a non-standard configuration with a curvature of 4 to 5 degrees.

To create a robust dataset for training and validation, the researchers simulated various dead pixel scenarios using a customized baffle plate. This method involved randomly obstructing the Mini-LEDs on the backlight panels to generate 2118 distinct images, each showcasing different illuminated areas. This data augmentation strategy addresses the critical issue of limited sample data availability, which often restricts the development of detection algorithms using deep learning. The images were labeled using the Segment Anything model combined with Labelimg, ensuring objective and precise annotation of each Mini-LED light-emitting area.

The performance of the proposed model was evaluated against several baseline semantic segmentation networks, including Unet, Pspnet, Deeplabv3+, and the original Hrnet. The models were trained for over 1000 epochs using a system equipped with three NVIDIA GTX 1080 Ti GPUs. The evaluation metrics included mean intersection over union (Miou), mean pixel accuracy (mPA), and overall Accuracy. The proposed model outperformed all baseline networks, achieving a Miou of 86.91%, an mPA of 92.45%, and an Accuracy of 98.04%.

Visual evaluations further demonstrated the superiority of the proposed model. While baseline models such as Pspnet, Unet, and Deeplabv3+ exhibited varying degrees of contour inaccuracy, with multiple Mini-LED illumination contours often concatenated and indistinguishable, the proposed model produced prediction maps that closely matched the ground truth labels. The red rectangles in the visual comparisons highlighted regions where baseline models failed to segment individual Mini-LEDs, whereas the proposed model maintained clear boundaries. This precision is vital for ensuring that the automated optical inspection system can reliably identify defective components.

Beyond segmentation accuracy, the study assessed the model’s ability to count defective Mini-LEDs and determine their positions using the Canny edge detection algorithm. The root mean square error (RMSE) was calculated to quantify the discrepancies between the predicted and actual values. The proposed model achieved the lowest RMSE for defect count, with a value of 3.94, reflecting higher accuracy in calculating the actual number of Mini-LEDs per panel. For position accuracy, the model exhibited the lowest positional error with an RMSE of 176.85, demonstrating its capability to reproduce the true distribution of Mini-LEDs on the backlight panel.

The operational efficiency of the model was also a key consideration for its potential application in production lines. The latency, defined as the forward propagation time per image, was measured to evaluate the model’s speed. While the original Hrnet achieved a latency of 1.43 seconds per image, the proposed model closely matched this with a latency of 1.47 seconds per image. This similarity in prediction time, despite the increased complexity of the network, highlights that the modifications maintain competitive efficiency while delivering enhanced precision.

The study concludes that the integration of MDC-DUC and RGCA modules into the Hrnet backbone significantly improves the detection of Mini-LED chips. The enhanced model demonstrates superior precision in identifying dead pixels and chip placement, outperforming baseline models across all primary evaluation metrics. The use of simulated dead pixel scenarios effectively augmented the dataset, addressing the data scarcity issue inherent in specialized industrial inspection tasks. The quantitative analysis using Canny edge detection provided a robust framework for evaluating both the count and position of Mini-LEDs, further optimizing the inspection process.

This research provides a practical solution for the quality control of Mini-LED backlight manufacturing, leveraging deep learning to overcome the physical constraints of miniature components. By enabling the automated optical inspection device to detect individual Mini-LED regions with high precision, the work facilitates the efficient acquisition of position and defective count data.

The research team addressed the scarcity of Mini-LED backlight sample data, a factor that restricts the development of detection algorithms using deep learning and pattern recognition, by simulating various dead pixel scenarios. This approach involved using a customized baffle plate to randomly obstruct the Mini-LEDs on the backlight panels, creating a dataset of 2118 distinct images. The labeling process for these images utilized the Segment Anything model combined with Labelimg, ensuring objective and precise annotation of each Mini-LED light-emitting area for subsequent analysis.

The experimental hardware setup was designed to capture high-resolution images of the Mini-LED backlight panels with precision. The system employed a CCD camera model MV-CS050-10GM paired with a 12 mm industrial lens, achieving a resolution of 2448 by 2048 pixels. The sensor format measured 2/3 inches, with a pixel size of 3.45 micrometers by 3.45 micrometers. The captured images were transmitted directly to a PC for further analysis and processing, ensuring that the data acquisition was precise enough to evaluate the performance of the Mini-LEDs accurately.

The specific Mini-LED backlight panel used in the study featured a non-standard configuration with a curvature ranging from 4 to 5 degrees. The panel measured approximately 76 cm in length and 11.8 cm in width, containing a total of 1733 individual Mini-LED chips. These chips were divided into 869 on the left panel and 864 on the right panel. Each Mini-LED chip measured about 100 micrometers by 200 micrometers, with a spacing of 850 micrometers between them. This dense distribution highlights the challenge of accurately measuring and analyzing the positions of LED chips, as even a small deviation in position or brightness can seriously affect display quality.

The main control board of the Mini-LED display included a timing controller (TCON) specialist IC, flash storage integrated circuit (IC), and power IC. The TCON processed embedded display port (EDP) video signals from an external host computer, deciphering, compensating, and merging the backlight serial peripheral interface (SPI) signal into the EDP signal internally. Flexible printed circuit (FPC) 1 controlled power on the left, while FPC3 controlled EDP output on the right. The main control board sent each picture frame to the Mini-LEDs via the FPC2 EDP signal and the SPI signal to the backlight board, performing central control and processing. In the simulation, yellow squares indicated illuminated Mini-LEDs, while black squares represented defective Mini-LEDs simulated by covering them with customized light baffles.

The study’s conclusions emphasize that the integration of MDC-DUC and RGCA modules into the Hrnet backbone significantly improves the detection of Mini-LED chips. The enhanced model demonstrates superior precision in identifying dead pixels and chip placement, outperforming baseline models across all primary evaluation metrics. The use of simulated dead pixel scenarios effectively augmented the dataset, addressing the data scarcity issue inherent in specialized industrial inspection tasks. The quantitative analysis using Canny edge detection provided a robust framework for evaluating both the count and position of Mini-LEDs, further optimizing the inspection process for manufacturing quality control.

Subject of Research: Deep learning-based automated optical inspection for vehicular Mini-LED backlight displays

Article Title: Vehicular Mini-LED backlight display inspection based on residual global context mechanism

Article References: Zhao, G., Zheng, X., Huang, X., Lu, Y., Chen, Z., & Guo, W. (2024). Vehicular Mini-LED backlight display inspection based on residual global context mechanism. Frontiers of Optoelectronics, 17(1), Article 35. https://doi.org/10.1007/s12200-024-00140-4

Image Credits: AI Generated

DOI: 10.1007/s12200-024-00140-4

Keywords: Mini-LED, Automated Optical Inspection, Deep Learning, Semantic Segmentation, Display Technology, Quality Control, Computer Vision, Automotive Displays, Vehicular, backlight, display, inspection