Research Papers Institution
Company behind the open-source machine-learning hub of the same name, known for the Transformers library, the model and dataset Hub, and fully open releases like the SmolLM small-model family.
Small Language Models · Hugging Face
DistilBERT turns knowledge distillation for compact language models into a concrete research object, with evidence anchors, method tradeoffs, and limits for practical use.
Small Language Models · Hugging Face
SmolLM2 is a 1.7B model overtrained on ~11T tokens through four data stages. It scores 68.7 on HellaSwag and 19.4 on MMLU-Pro, beating Llama3.2-1B — and ships every dataset, not just the weights.