DETR-CLIP study evaluates detection, semantics and creativity scoring in 5,000 children’s drawings

A machine-learning study combining DETR object detection with CLIP semantic representations reported strong performance across separate evaluations of detection, semantic understanding, zero-shot symbolic interpretation and creativity assessment. On a newly assembled dataset of 5,000 children’s drawings, the DETR-CLIP model achieved 78.6% ± 0.4% mean average precision at an intersection-over-union threshold of 0.5, although its results declined on an independent external dataset.

The researchers constructed ChildArt-Open from the Google Open Images dataset. Three researchers in children’s art education independently verified the selected images, achieving a Cohen’s κ of 0.86. From 12,400 images identified through the initial screening, the researchers removed duplicates, low-resolution images, non-drawing photographs, adult artworks, digitally synthesized or AI-generated images, and images containing identifiable personal information. The final set comprised 5,000 drawings divided at the image level into 3,500 training, 750 validation and 750 test images. Its categories included people, animals, family scenes, nature, vehicles, fantasy characters and abstract doodles.

DETR served as the spatial backbone, treating object detection as a set-prediction problem within an encoder-decoder Transformer. CLIP supplied image and text representations from a shared semantic space. The researchers inserted a cross-modal attention module after self-attention in each of DETR’s six decoder layers. Twenty CLIP prompt templates were embedded to update object-query features alongside semantic representations. The CLIP ViT-B/32 encoders were frozen, the DETR branch was fine-tuned from COCO weights, and the new projection and cross-modal layers were trained from random initialization.

Training combined DETR classification, bounding-box regression and generalized intersection-over-union losses with a bidirectional CLIP contrastive loss, a Smooth L1 loss for creativity regression and weight regularization. Three experts in children’s art education and developmental psychology independently annotated semantic labels and rated creativity from 1 to 5 across originality, imagination, expressiveness, composition and emotion. Reported agreement measures included Fleiss’ κ of 0.82, Krippendorff’s α of 0.84 and an intraclass correlation of 0.88. The model was trained for 200 epochs over 72 hours using 32 NVIDIA A100 GPUs, AdamW, an initial learning rate of 10^-4 and FP16 mixed precision.

On the 750-image ChildArt-Open test set, DETR-CLIP reached 78.6% ± 0.4% mAP@0.5, compared with 62.3% ± 0.6% for DETR, 71.5% ± 0.5% for Deformable DETR, 73.8% ± 0.6% for Faster R-CNN with CLIP, 74.2% ± 0.5% for RT-DETR, 75.1% ± 0.4% for DINO-DETR and 72.6% ± 0.6% for YOLOv8. The comparison with DETR had p < 0.01 and a Cohen’s d of 1.32. Semantic-understanding accuracy was 87.3% ± 0.5%, with a macro-F1 of 0.86; Faster R-CNN with CLIP had an accuracy of 76.5% ± 0.7%.

For continuous creativity scores, the model had a Spearman correlation of 0.81 ± 0.02 with expert ratings, an intraclass correlation of 0.79 ± 0.03, a mean absolute error of 0.34 ± 0.02 and an RMSE of 0.46. Thresholds determined on the validation set divided creativity into low, medium and high categories. Test-set classification accuracy was 85.5%, with a macro-F1 of 0.84. Ablation experiments reported that removing the cross-modal interaction module was associated with a 15.1% decrease in creativity-scoring accuracy, while removing CLIP semantic guidance was associated with a 9.1% decrease in semantic-understanding accuracy. These ablation results concern the stated evaluation components and do not by themselves establish improved metaphor or creativity understanding from detection gains alone.

Zero-shot semantic interpretation was tested on five concepts not seen during training: loneliness, dream, fear, family warmth and freedom. Using the prompt “a child’s drawing expressing [concept],” DETR-CLIP achieved 78.4% top-1 accuracy, 89.6% top-3 accuracy, 72.8% mAP and a semantic-consistency score of 0.83. Its concept-specific top-1 results ranged from 74.8% for fear to 82.1% for family warmth. CLIP zero-shot classification reached 61.2% top-1 accuracy, while an ordinary object-recognition model reached 52.1% top-1 accuracy.

Among 95 test images that the authors classified as incorrect semantic interpretations, 40 cases, or 42.1%, were attributed to symbolic ambiguity; 29, or 30.5%, to cultural differences; 17, or 17.9%, to multi-object occlusion; and nine, or 9.5%, to other causes. In an external evaluation on the KidsDraw dataset, with model weights fixed and no fine-tuning, the authors reported decreases of 6.7% in mAP@0.5, 8.1% in semantic-understanding accuracy, 9.4% in metaphor-interpretation macro-F1 and 7.5% in creativity-level consistency relative to ChildArt-Open. The results therefore indicated incomplete generalization across differences in drawing style, theme and object-class distribution.

The complete model contained 196.9 million parameters and required 156.2 G floating-point operations. On one NVIDIA A100 GPU with batch size 1 and FP16 precision, measured inference latency was 22.3 milliseconds, equivalent to 44.8 frames per second. Important limitations included the inability to group samples by individual child because the source data contained no identity information, creating a possible data-leakage concern. The study also reported unverified robustness to long text prompts and no explicit modeling of painting media or paper texture. ChildArt-Open identifiers, filtering and annotation files, split information, annotation guidance and model code were released, while the original images remained subject to source copyright restrictions.

Subject of Research: Machine-learning analysis of object detection, semantic interpretation and creativity scoring in children’s drawings

Article Title: A model for children’s drawing content analysis and creativity assessment Integrating DETR target detection and CLIP semantic understanding

Article References: Yuan, Y. (2026). A model for children’s drawing content analysis and creativity assessment Integrating DETR target detection and CLIP semantic understanding. Discover Artificial Intelligence, 6(1), Article 1498. https://doi.org/10.1007/s44163-026-02466-8

Image Credits: AI Generated

DOI: 10.1007/s44163-026-02466-8

Keywords: children’s drawings, DETR, CLIP, object detection, semantic understanding, creativity assessment, zero-shot interpretation, multimodal learning, computer vision, ChildArt-Open, model, children