Small Language Models Are Gaining Popularity Among Researchers
While larger models offer versatility, smaller models are becoming attractive due to their reduced footprint and efficiency.
Large language models (LLMs) like those from OpenAI, Meta, and DeepSeek are powerful but come with significant costs. These models, which use hundreds of billions of parameters, require massive computational resources for training and operation. For example, Google reportedly spent $191 million to train its Gemini 1.0 Ultra model. Additionally, a single ChatGPT query consumes about 10 times more energy than a Google search, according to the Electric Power Research Institute.
In response, researchers are turning to small language models (SLMs). Companies like IBM, Google, Microsoft, and OpenAI have released SLMs with just a few billion parameters. While these models are not as versatile as LLMs, they excel in specific tasks such as summarizing conversations, serving as health care chatbots, and collecting data in smart devices. Zico Kolter, a computer scientist at Carnegie Mellon University, noted that "For a lot of tasks, an 8 billion–parameter model is actually pretty good." SLMs can also run on devices like laptops or cell phones, reducing reliance on large data centers.
Training and Optimizing Small Models
To make SLMs effective, researchers use techniques like knowledge distillation, where large models generate high-quality training data for smaller ones. This method ensures that SLMs learn from cleaner, more organized data. Another approach is pruning, which involves removing unnecessary parts of a neural network to improve efficiency. Pruning was inspired by the human brain and was first proposed by Yann LeCun in a 1989 paper. This technique allows researchers to fine-tune SLMs for specific tasks or environments.
Advantages of Small Models
SLMs offer several benefits:
- Cost-Effectiveness: They require fewer computational resources, making them cheaper to train and deploy.
- Transparency: With fewer parameters, their reasoning processes are easier to understand.
- Experimentation: Researchers can test new ideas with lower stakes, as noted by Leshem Choshen of the MIT-IBM Watson AI Lab.
While LLMs will remain useful for tasks like generalized chatbots and drug discovery, SLMs provide a practical alternative for targeted applications. As Choshen put it, "These efficient models can save money, time, and compute."
Original story reprinted with permission from Quanta Magazine.
Related News
Zscaler CAIO on securing AI agents and blending rule-based with generative models
Claudionor Coelho Jr, Chief AI Officer at Zscaler, discusses AI's rapid evolution, cybersecurity challenges, and combining rule-based reasoning with generative models for enterprise transformation.
Human-AI collaboration boosts customer support satisfaction
AI enhances customer support when used as a tool for human agents, acting as a sixth sense or angel on the shoulder, according to Verizon Business study.
About the Author

Dr. Lisa Kim
AI Ethics Researcher
Leading expert in AI ethics and responsible AI development with 13 years of research experience. Former member of Microsoft AI Ethics Committee, now provides consulting for multiple international AI governance organizations. Regularly contributes AI ethics articles to top-tier journals like Nature and Science.