Articles filed in category 'Local LLMs'

  • Distilling Models

    By: Sahil Malik

    Last updated: Thursday, October 29, 2026

    Published in: CODE Magazine: 2026 - November/December

    Sahil demonstrates that model distillation is the practical path to making AI useful, affordable, and private by compressing the knowledge of large “teacher” models into small, fast “student” models that run locally. Using a Git error–explainer as an example, he shows how to generate synthetic training data with Ollama, fine-tune a tiny model in Google Colab with Unsloth and LoRA, and export it for local use.

  • Fine Tuning Large Language Models, Part 2

    By: Wei-Meng Lee

    Last updated: Thursday, October 29, 2026

    Published in: CODE Magazine: 2026 - November/December

    Wei-Meng expands the fine-tuning toolkit beyond basic LoRA by explaining how quantization enables QLoRA on modest hardware, how prompt tuning can steer a frozen model with only a few learned virtual tokens, and how to judge whether training truly improved performance through baselines, standard metrics, and LLM-as-a-judge methods. This article also introduces knowledge distillation as a way to compress a strong teacher into a smaller student, then shows practical deployment paths with direct Transformers use or ONNX export.