Edge AI4 min read
A 1-Bit AI Trick Shrinks a 70B Model to Fit on a $200 GPU
Samsung's sub-1-bit quantization paper gets a reality check, plus fresh research on tiny LLMs and ESP32 voice silicon.
78 views
Read Tag
small-language-models
13 posts
e=mc²9 min read
Why Momo Recites: Train Four Sizes and Watch Memorisation Begin
Lesson 03: the same small language model trained at 0.95M, 3.06M, 5.85M and 12.3M parameters on the same stories. Quiz perplexity falls from 16.0 to 7.4, and verbatim recitation of training stories switches on somewhere between the first and second size. Real numbers, and what they do not prove.
41 views
Read 
Edge AI4 min read
Small AI Models Steer Robots While ESP32 Boards Do the Grunt Work
A dual-core Arduino runs robot policies on-device, a paper argues for skill-sized models over one big brain, and two ESP32 boards ship.
80 views
Read 
Edge AI5 min read
Seven $5 Chips Learn to Share an LLM, One Bit at a Time
A BitNet cluster splits an LLM across seven ESP32-S3 boards, a 2-bit tool-calling model lands on GitHub, and a quantization paper gets llama.cpp 15x faster on an M4 Pro.
237 views
Read 
Edge AI4 min read
A Classifier That Only Picks, a Model That Skips Tokens, and an AI Firm Bets 2.6B Parameters Can Do Agent Work
Liquid AI ships a 2.6B agentic model for edge hardware, a 144M classifier skips text generation entirely, and a new paper shows byte-level LLMs can beat tokenized ones with sc
100 views
Read 
Edge AI4 min read
Tiny Audio AI Models Learn to Whisper Without Ever Calling Home
Two new speech stacks and a tool-calling model push voice AI fully on-device, while Nordic and Zephyr quietly harden the hardware underneath.
118 views
Read 
Edge AI5 min read
A 27B AI Model Shrinks to 5.9GB as Ternary Quantization Keeps Pushing the Floor Down
PrismML compresses a 27B model to 5.9GB, Intel's BITCOS beats the 1.585-bit ternary limit, and a 44M-parameter model claims exact arithmetic on a laptop CPU.
137 views
Read 
Edge AI4 min read
Ternary Weights and Tiny Tanks: Small-Model Quantization Gets Concrete for Edge AI
A ternary 8B model, a BitNet toy for TinyStories, a fish tank run by a 14M LLM, and a Rust retrieval encoder push quantization research toward real hardware.
125 views
Read 
Edge AI4 min read
Quantization Papers Pile Up as Researchers Argue Over Where Small AI Should Spend Its Bits
Fresh arXiv work rethinks quantization strategy and sustainability, an Apple-adjacent paper shrinks the dictation encoder, and new silicon and Jetson guidance round out the ed
152 views
Read 
Edge AI4 min read
A Million-Token LLM Tries to Fit On Your Phone as Tiny Voice Models Multiply
An iFLYTEK spin-off open-sources a 1.7B model claiming native million-token context on-device, while a 14MB tool-caller and an open voice-agent LLM push the small-model race f
274 views
Read 
Edge AI4 min read
A 27B LLM Shrinks to Phone Size as Edge Quantization Keeps Rewriting the Rules
A 1-bit Qwen derivative fits an iPhone, a healing trick beats its own teacher at 4-bit, and a dense LLM limps along on an $8 chip.
95 views
Read 
Edge AI4 min read
Edge Dispatch: Hearing Aids Get Their Own AI Chips as On-Device Voice and Vision Push Past the Phone
Hearing aids ship dedicated on-device AI chips, new AI glasses land, and real-phone benchmarks show why raw specs don't tell the whole story.
150 views
Read 
Edge AI4 min read
Edge Dispatch: Liquid AI Trains Its Way Around the Q4_0 Quality Tax, While New Papers Chip Away at What Quantization Quietly Breaks
Liquid AI distills instead of just rounding for 4-bit LFM2.5 checkpoints, as fresh research flags what low-bit quantization costs in memory and multilingual accuracy.
151 views
Read