Knowledge Distillation: Compressing Large Models
·1 min read

Knowledge Distillation: Compressing Large Models

Transfer knowledge from large to small models—but distilled models lose capabilities

By August Park, Model Optimization Engineerknowledge distillationmodel compressionteacher-student

Knowledge Distillation: Compressing Large Models

Knowledge distillation trains smaller student models to mimic larger teacher models.

Related Chronicles: The Compression Catastrophe (2035)

Share this article

Related Research