Quantization Beats Redesign: 4-Bit Small Language Models Cut Memory 71% and Boost Speed 83%
A controlled benchmark study finds that 4-bit quantization of a compact Transformer model delivers the strongest overall efficiency gains, while ...
Tag: knowledge distillation
Lightweight Knowledge Distillation Speeds Up Vulnerability Detection in Python Code
A new knowledge distillation method compresses large pre-trained models into lightweight detectors that identify vulnerabilities in Python code with Macro-F1 ...
AI transformer detects REM sleep behavior disorder with 28-millisecond speed
A new multimodal transformer framework detects REM sleep behavior disorder with an AUC of 0.95 while running in 28 milliseconds, ...
AI Learns to Spot Clouds in Lidar Data Without Site-Specific Retraining
A self-supervised, distillation-guided AI framework identifies cloud layers in multi-wavelength Raman lidar data and transfers to unseen CARLNET sites without ...
AI Reads Police Logs to Count Vulnerability, But Only With Heavy Human Correction
A study of nearly 3,000 UK police incident logs shows fine-tuned language models can estimate vulnerability prevalence, but only after ...
Survey maps the rise of open-ended visual segmentation
A new open access survey in Artificial Intelligence Review organises open-ended visual segmentation into three research directions and maps the ...
One Teacher, Many Datasets: New AI Framework Boosts Medical Image Analysis Across Tasks
A new teacher-student framework aggregates knowledge from multiple medical imaging datasets and consistently improves segmentation, classification, and detection performance across ...
Knowledge-Distilled Lightweight AI Reads Breast Ultrasounds With Radiologist-Beating Accuracy
A knowledge-distilled convolutional network reached 92.64 percent accuracy on an independent breast ultrasound dataset, outperforming chief physicians in a retrospective ...
Pyramid vision transformer cuts attention cost and boosts image classification
A knowledge-distilled pyramid transformer with reduced attention achieved F1-scores of 90.92%, 97.76% and 94.59% on three 32 × 32-pixel image-classification ...
Adaptive Temperature Distillation Shrinks Multilingual AI Models for Classroom Use
A new distillation framework that adjusts its temperature per language and sample compresses a 560-million-parameter multilingual model into a 66-million-parameter ...
-
Mothers who receive childcare support from maternal grandparents show more parental warmth, finds NTU Singapore study
-
University of Seville Breaks 120-Year-Old Mystery, Revises a Key Einstein Concept
-
Bee body mass, pathogens and local climate influence heat tolerance
-
Researchers record first-ever images and data of a shark experiencing a boat strike
-
Groundbreaking Clinical Trial Reveals Lubiprostone Enhances Kidney Function
Embark on a thrilling journey of discovery with Scienmag.com—your ultimate source for cutting-edge breakthroughs. Immerse yourself in a world where curiosity knows no limits and tomorrow’s possibilities become today’s reality!