Détails

Source

TAO Daily

Publication

28 oct. à 13h21

Lien direct

https://taodaily.io/ingot-sn97-is-making-powerful-ai-models-small-enough-to-run-on-laptops/

Contenu source (brut)

<p class="wp-block-paragraph">Every time you type a question into an AI chatbot, it leaves your phone or computer, travels to a giant data centre owned by centralized companies, gets processed on expensive chips, and comes back as an answer.</p> <p class="wp-block-paragraph">That arrangement works, but it has costs most people never think about. Your words now live on someone else’s server, and people are sharing far more than they realise.</p> <p class="wp-block-paragraph">A DuckDuckGo survey of nearly 2,000 US adults found that 32% of chatbot users have told an AI things they kept from friends, relatives, colleagues or even their doctors.</p> <p class="wp-block-paragraph">The machines doing all this thinking are also hungry. The International Energy Agency projects that global data centre electricity demand will more than double by 2030 to around 945 terawatt-hours, slightly more than everything Japan consumes today.</p> <p class="wp-block-paragraph">Ingot, Subnet 97 on Bittensor, is attacking this problem from the other direction. Instead of building bigger data centres, it is trying to shrink powerful AI models until they fit on hardware ordinary people already own, such as a MacBook, a gaming PC or a single consumer graphics card.</p> <p class="wp-block-paragraph">The subnet was previously known as <a href="https://taodaily.io/consts-keynote-at-exploit-summit-harness-the-exploit-build-a-mind-outside-the-state/">Albedo</a>, and before that as Distil, and on 16 October 2026 it was renamed Ingot and narrowed its focus to a technique called <em>quantization</em>.</p> <h2 class="wp-block-heading">Making AI Models Smaller Without Losing Their Intelligence</h2> <figure class="wp-block-image size-large"><img fetchpriority="high" decoding="async" width="1024" height="576" src="https://taodaily.io/wp-content/uploads/2026/10/compress-vs-quantize-1024x576.png" alt="" class="wp-image-25942" srcset="https://taodaily.io/wp-content/uploads/2026/10/compress-vs-quantize-1024x576.png 1024w, https://taodaily.io/wp-content/uploads/2026/10/compress-vs-quantize-300x169.png 300w, https://taodaily.io/wp-content/uploads/2026/10/compress-vs-quantize-678x381.png 678w, https://taodaily.io/wp-content/uploads/2026/10/compress-vs-quantize-768x432.png 768w" sizes="(max-width: 1024px) 100vw, 1024px" /><figcaption class="wp-element-caption">Quantization explained</figcaption></figure> <p class="wp-block-paragraph">Large AI models require substantial computing power and memory because they contain billions of numerical values used to process information.</p> <p class="wp-block-paragraph">A technique called <em>quantization</em> reduces the amount of space needed to store these values, allowing the same model architecture to operate with significantly less memory.</p> <p class="wp-block-paragraph">Think of it like compressing a high-resolution photograph. The original file might occupy 50 MB, but a well-compressed version could take up just 5 MB while looking almost identical to the human eye.</p> <p class="wp-block-paragraph">AI quantization follows a similar principle, although reducing model size too aggressively can affect how accurately the model responds.</p> <p class="wp-block-paragraph">Ingot applies this technique to <strong>Qwen3.8-27B</strong>, an open AI model containing approximately 27 billion parameters.</p> <p class="wp-block-paragraph">Miners compete to produce the most faithful compressed version, while validators compare each submission against the original model. The evaluation examines how closely the smaller model reproduces the original&#8217;s predictions and internal calculations.</p> <p class="wp-block-paragraph">A submission that improves on the current best model becomes the new winner, with its weights published openly for anyone to download.</p> <h2 class="wp-block-heading">Ingot&#8217;s Early Results and the Push Toward Smaller Models</h2> <p class="wp-block-paragraph">During its first six days, Ingot&#8217;s team reported that 30 miners produced 64 quantized versions of Qwen3.8-27B.</p> <p class="wp-block-paragraph">The winning model, Ingot-V-27B-FP8, was reportedly <strong>35.6% closer to the original model than Qwen&#8217;s official FP8 version</strong>, based on the subnet&#8217;s evaluation measurements. This does not mean the model became 35.6% smarter, but that its compressed version more closely reproduced the original&#8217;s behavior.</p> <figure class="wp-block-embed is-type-rich is-provider-x wp-block-embed-x"><div class="wp-block-embed__wrapper"> <blockquote class="twitter-tweet" data-width="500" data-dnt="true"><p lang="en" dir="ltr">35.6% closer to the original than Qwen&#39;s own FP8.<br><br>That&#39;s Ingot-V-27B-FP8, the best of 64 quantizations of Qwen3.8-27B that 30 miners built in 6 days.<br><br>FP8 round: done, and we&#39;re happy with it.<br>Now we go to 2-bit.<a href="https://t.co/yDwqnLx5Ob">https://t.co/yDwqnLx5Ob</a></p>&mdash; Ingot | Weather Boss (@ingot_wthrboss) <a href="https://x.com/ingot_wthrboss/status/2113278045163270506?ref_src=twsrc%5Etfw">October 22, 2026</a></blockquote><script async src="https://platform.x.com/widgets.js" charset="utf-8"></script> </div></figure> <p class="wp-block-paragraph">The subnet subsequently moved to more aggressive compression, producing models that use approximately 1.75 bits per weight. Its <a href="https://www.theingot.ai/">Ingot-IV-27B-Q2 model</a> is available in a compressed file of approximately <strong>5.95 GB</strong>, making a 27-billion-parameter model practical to load on ordinary MacBooks.</p> <figure class="wp-block-image size-large"><img decoding="async" width="1024" height="495" src="https://taodaily.io/wp-content/uploads/2026/10/image-274-1024x495.png" alt="" class="wp-image-25934" srcset="https://taodaily.io/wp-content/uploads/2026/10/image-274-1024x495.png 1024w, https://taodaily.io/wp-content/uploads/2026/10/image-274-300x145.png 300w, https://taodaily.io/wp-content/uploads/2026/10/image-274-767x371.png 767w" sizes="(max-width: 1024px) 100vw, 1024px" /><figcaption class="wp-element-caption">Ingot is already beating Bonsai 2. <a href="https://theingot.ai/">Source</a></figcaption></figure> <p class="wp-block-paragraph">This progress is significant, but smaller files do not automatically guarantee identical performance. A <a href="https://arxiv.org/abs/2605.19645">2026 study by researchers at Lund University</a> found that heavily compressed AI models can retain useful capabilities, although some experience considerable performance losses depending on the tasks involved.</p> <p class="wp-block-paragraph">Ingot&#8217;s competition attempts to reduce these losses by continually rewarding better compression methods.</p> <h2 class="wp-block-heading">Why This Matters Beyond Bittensor</h2> <p class="wp-block-paragraph">Today, many popular AI services rely on remote data centers to process requests. This creates dependencies on internet connectivity, external infrastructure, and service providers.</p> <p class="wp-block-paragraph">Running AI locally offers another option, particularly for users who want greater control over their data or need tools that work without constant connectivity.</p> <p class="wp-block-paragraph">For example, a lawyer could review confidential documents using an AI model running entirely on their computer. A developer could use an offline coding assistant, while a student in an area with unreliable internet could access an AI-powered study tool without sending every question to a cloud service.</p> <p class="wp-block-paragraph">The wider tech industry is already moving in this direction. In a <a href="https://developers.googleblog.com/on-device-genai-in-chrome-chromebook-plus-and-pixel-watch-with-litert-lm/">September 2025 technical update</a>, Google highlighted how running AI directly on devices supports offline availability and eliminates per-request cloud API charg