Tag: parameter-efficient fine-tuning

A matched-budget study of LoRA fine-tuning finds that genetic algorithms and other search optimizers offer no consistent advantage over well-tuned ...

Researchers have introduced PLS-ZO, a zeroth-order optimization framework that confines gradient-free perturbations to a low-dimensional latent subspace, dramatically reducing the ...

Researchers at Tarbiat Modares University have developed DuoDiT, a dual-stream architecture that adapts large diffusion transformers for image generation by ...

A new systematic study shows that low-rank adaptation can match or approach full fine-tuning of transformer models across four sentiment ...

Penn State researchers have developed a continual learning method that adaptively adjusts LoRA ranks layer by layer, helping large language ...

The advent of large language models, particularly in the field of biology, has transformed our understanding of complex systems at ...