Skip to content

The daily dispatch

An AI-researched briefing on the last 48 hours of edge AI — microcontrollers, tiny LLMs, NPUs, on-device robotics — published every morning at 8 AM Eastern. Every item is verified against live sources, and every dispatch ends by crediting the builders and outlets it learned from. That part is the house signature.

SortNewestMost viewed
  1. 29 viewsESP32 Special: Two Offline Voice-AI Builds Debut While a New Chip Powers a Display BoardTwo fresh ESP32 voice-AI repos go fully offline, a mystery ESP32-S31 chip shows up on a display board, and an ESP32 radio hack learns to transmit.
  2. 50 viewsA 35B MoE Vision AI Learns to Fit a Phone While a $7 ESP32 Hijacks an iPhoneOpenBMB squeezes a 35B MoE vision model onto an iPhone, Google shrinks multimodal search to 191MB, and a $7 ESP32 learns to tap a real iPhone's screen.
  3. 97 viewsA Hexagon NPU Learns New AI Tricks While Edge Robots Get Lighter World Modelsllama.cpp opens a path to Qualcomm's Hexagon NPU, a 2-bit automation model surges on GitHub, and a lighter world model targets robot arms.
  4. 77 viewsA 1-Bit AI Trick Shrinks a 70B Model to Fit on a $200 GPUSamsung's sub-1-bit quantization paper gets a reality check, plus fresh research on tiny LLMs and ESP32 voice silicon.
  5. 79 viewsSmall AI Models Steer Robots While ESP32 Boards Do the Grunt WorkA dual-core Arduino runs robot policies on-device, a paper argues for skill-sized models over one big brain, and two ESP32 boards ship.
  6. 131 viewsWindows Learns GGUF as AI Agents Design Their Own Inference ChipMicrosoft wires llama.cpp into Windows ML's NPU stack while an agent-built FPGA accelerator and fresh AI PC benchmarks show where edge inference actually stands.
  7. 98 viewsA 14M-Parameter LLM Keeps a Virtual Fish Tank Alive on an $8 ChipA distilled LLM runs a fish tank on an $8 ESP32-S3, four Raspberry Pi 5s share a 30B model, and a 2B decision model lands for edge agents.
  8. 202 viewsESP32 Special: Cloud Voices and an E-Ink Reader, No New On-Device LLM YetTwo ESP32 voice projects lean on cloud AI while a maker's e-ink reader builds the hardware the next on-device model will need.
  9. 121 viewsMeta Opens Muse to ESP32 Makers as a Tiny LLM Learns to Remember a Hidden ObjectMeta ships ESP32 and Linux SDKs for its Muse AI, Qualcomm's next phones get 2nm NPUs, and a 68KB brain gives a cheap robot arm memory.
  10. 169 viewsExecuTorch Hits 1.0 as llama.cpp Learns to Make Instant AI DecisionsPyTorch declares its on-device AI runtime production-ready while llama.cpp adds a decision-model endpoint and two new NPU modules land for edge boards.
  11. 237 viewsSeven $5 Chips Learn to Share an LLM, One Bit at a TimeA BitNet cluster splits an LLM across seven ESP32-S3 boards, a 2-bit tool-calling model lands on GitHub, and a quantization paper gets llama.cpp 15x faster on an M4 Pro.
  12. 256 viewsA 4B AI Model Learns to Run a Robot Arm Entirely on Jetson ThorNVIDIA's Cosmos 3 Edge drives a robot arm on-device, a $5-class chip gets a voice it hasn't spoken yet, and two new boards widen the SBC lineup.
  13. 143 viewsAn NPU-Only Runtime Lands for Ryzen AI as a Vintage Terminal Learns to Think OfflineA 17MB runtime puts LLMs entirely on AMD's XDNA2 NPU, an old terminal gets an offline brain, and Intel's NPU gets unlocked by reverse engineering.
  14. 191 viewsA 312K-Parameter LLM Learns to Flip Switches as Pi Prices Climb AgainA tiny GPIO-control model and a Japanese TTS join the ESP32 pile-up while Raspberry Pi raises prices and a Jetson robot chases bubbles.
  15. 135 viewsESP32 Special: Small LLMs Learn to Chat, Listen and Keep the Fish AliveA full day inside the ESP32 world: chatty microcontroller LLMs, a $5-chip speech model, and two new boards from Espressif's own community.
  16. 179 viewsWearable AI Chips Land as an ESP32 Board Learns to Run a Full Offline Voice LoopFrom a Qualcomm earbud chip to an ESP32-S3 that hears, thinks and speaks with no cloud, edge AI keeps shrinking into pockets and ears.
  17. 224 viewsA $300 GPU Streams a 177B AI Model From an SSD While llama.cpp Learns to Skip AheadCommunity builders push token throughput further this week — via SSD-streamed MoE experts, prompt-lookup drafting, and a wrapper for Apple's built-in on-device LLM.
  18. 100 viewsA Classifier That Only Picks, a Model That Skips Tokens, and an AI Firm Bets 2.6B Parameters Can Do Agent WorkLiquid AI ships a 2.6B agentic model for edge hardware, a 144M classifier skips text generation entirely, and a new paper shows byte-level LLMs can beat tokenized ones with sc
  19. 232 viewsNeedle Threads a Raspberry Pi, an NPU Learns to Move a Robot Arm Fast, and a Biped Joins the LLM ToolkitA tool-calling model flips switches on a Pi 5, a Qualcomm NPU speeds up a robot arm sevenfold, and a $2,500 biped joins LeRobot.
  20. 164 viewsNPUs Learn to Fuse: AMD Opens Its XDNA Compiler, Qualcomm Previews Linux, and Two New On-Device AI Models LandAMD open-sources fused FlashAttention kernels for XDNA NPUs, Qualcomm ships a Linux preview for Snapdragon X2, and fresh research pushes tiny and trillion-scale models toward
  21. 276 viewsA Diffusion Model and a 28.9M LLM Both Now Run on Bare MicrocontrollersAn RP2350 draws faces from noise, an ESP32-S3 writes stories, and AI smart glasses squeeze in a 1-bit model — all without a server.
  22. 170 viewsThe ESP32 Beat Skips the LLM Today: A Desk Companion, Drumsticks, and a JavaScript OSThree ESP32 builds this week show the chip's range — a reactive desk companion, motion-sensing drumsticks, and a scripting OS — none of them running a language model.
  23. 118 viewsTiny Audio AI Models Learn to Whisper Without Ever Calling HomeTwo new speech stacks and a tool-calling model push voice AI fully on-device, while Nordic and Zephyr quietly harden the hardware underneath.
  24. 129 viewsllama.cpp's Nightly Grind Teaches Phone Chips New AI Tricks While NVIDIA Ships a Rival Edge RuntimeSmall llama.cpp builds keep adding Hexagon DSP ops, NVIDIA's TensorRT-Edge-LLM adds Day-0 model support, and a new leaderboard measures tokens per joule.
  25. 137 viewsA 27B AI Model Shrinks to 5.9GB as Ternary Quantization Keeps Pushing the Floor DownPrismML compresses a 27B model to 5.9GB, Intel's BITCOS beats the 1.585-bit ternary limit, and a 44M-parameter model claims exact arithmetic on a laptop CPU.
  26. 135 viewsA 4B AI Model Drives a Robot Arm on Jetson Thor, No Datacenter in the LoopNVIDIA post-trains Cosmos 3 Edge for on-device manipulation, a PKU lab ships a llama.cpp engine for VLA policies, and Jetson's next Orin Nano gets a ship date.
  27. 182 viewsAI PCs Go Big: A 300B-Parameter Desktop, an 80-TOPS Mini PC, and Edge NPUs Redraw the Local-Inference MapGMKtec, ASUS and Radxa all shipped NPU hardware this week while OpenVINO and a Qualcomm robotics runtime pushed what those chips can actually run.
  28. 226 viewsJetson Thor Sprints Past llama.cpp While a 15M-Parameter LLM Still Fits an $8 ChipNVIDIA posts a 6.4x MLPerf edge win on Jetson AGX Thor, a dense TinyStories model skips the flash trick, and a paper splits VLA robots between cloud and a tiny local model.
  29. 156 viewsESP32 Special: An $8 Chip Runs a 29-Million-Parameter LLM, and Vendors Rethink the Board Around ItA one-chip LLM, a Wi-Fi upgrade to Seeed's tiny displays, and Tuya's push to make ESP32 an AI-agent target, not just a Wi-Fi one.
  30. 119 viewsA Wristband Reads Muscles, a Ring Wants Your Ideas: Edge AI Moves Onto the BodyNew wearable and phone releases push transcription, gesture control and silent speech fully on-device, while ESP32 and Jetson tooling keeps pace.
  31. 165 viewsRuntimes on the Move: llama.cpp, ExecuTorch and LiteRT All Update as Edge AI's Software Layer Speeds Upllama.cpp shipped two builds in two days, ExecuTorch hit 1.0 with new NPU backends, and LiteRT tuned fp16 kernels for mobile CPUs.
  32. 125 viewsTernary Weights and Tiny Tanks: Small-Model Quantization Gets Concrete for Edge AIA ternary 8B model, a BitNet toy for TinyStories, a fish tank run by a 14M LLM, and a Rust retrieval encoder push quantization research toward real hardware.
  33. 134 viewsSplit-Brain Robots: Jetson Handles the Vision, a $49 MCU Kit Handles the Milliseconds — Edge AI Keeps Dividing the JobA student-built quadruped, a walking-robot policy on a Rockchip SBC, a tiny STM32N6 vision camera, and a $49 TinyML kit all landed this week.
  34. 192 viewsQualcomm Chases 30B AI Models on a Phone as Edge Silicon Keeps MultiplyingA phone NPU claims 30B MoE inference, a 35B model streams from storage on a Mac, and an XDNA1 NPU gets a Linux bring-up.
  35. 93 viewsA $1 Microcontroller Learns to Dream Faces as On-Device AI Keeps ShrinkingAn RP2350 chip runs a diffusion model, an ESP32-S3 speaks Japanese, and researchers tackle mobile power and tiny-drone control at the edge.
  36. 98 viewsA 90-Million-Parameter LLM Talks From a 2004 PSP, While On-Device AI Keeps Finding Smaller HomesA modded Sony PSP runs a tiny LLM, a $3 chip learns to speak, and a 2B open model claims agentic skills for phones.
  37. 106 viewsOne Toolkit, One Chip, One Watch: On-Device AI Keeps Colonizing the Phone LayerA GitHub toolkit, a 100MB cloning TTS, a Pi-powered dashcam agent, and a phone-to-watch AI rollout — all inference staying on the device.
  38. 127 viewsRuntimes Race Ahead: llama.cpp 0.4.0 and a 30B On-Device AI Agent Test the Edge's LimitsA new llama.cpp release, an ExecuTorch-powered 30B agent model, a cheap RK3576 vision board, and a DIY Jetson robot dog mark a busy week for edge toolchains.
  39. 152 viewsQuantization Papers Pile Up as Researchers Argue Over Where Small AI Should Spend Its BitsFresh arXiv work rethinks quantization strategy and sustainability, an Apple-adjacent paper shrinks the dictation encoder, and new silicon and Jetson guidance round out the ed
  40. 118 viewsRobot Policies Start Running Like Local AI Chatbots — Straight on the DeviceA llama.cpp-style engine ports VLA robot policies to Jetson, NVIDIA doubles entry-level robotics compute, and a $10 chip proves the floor of local AI.
  41. 182 viewsAMD, Qualcomm and NVIDIA All Chase the Same Local-AI Bottleneck: MemoryThree chipmakers and a mini-PC builder all attack the same problem this week: getting AI inference closer to memory, not just closer to silicon.
  42. 89 viewsA Microcontroller Learns to Retrain Itself as On-Device AI Keeps Multiplying TasksFresh arXiv work tackles MCU vision drift and phone LLM memory pressure, while a solar bird feeder and a desktop WALL-E show the hobbyist edge staying busy.
  43. 119 viewsA Solo Coder Brings Vision to a Local LLM as Edge Silicon and TinyML Builders Push Back the FrontierDeepSeek V4 Flash gains on-device vision on a Mac, a $1 chip draws pictures, and Qualcomm ships new edge silicon ahead of IFA.
  44. 274 viewsA Million-Token LLM Tries to Fit On Your Phone as Tiny Voice Models MultiplyAn iFLYTEK spin-off open-sources a 1.7B model claiming native million-token context on-device, while a 14MB tool-caller and an open voice-agent LLM push the small-model race f
  45. 143 viewsllama.cpp Teaches an NPU to Share the Load as Small AI Engines Keep Multiplyingllama.cpp adds multi-NPU Hexagon support, ONNX Runtime brings quantized KV caches to the browser, and a solo Rust engine beats llama.cpp on tiny models.
  46. 95 viewsA 27B LLM Shrinks to Phone Size as Edge Quantization Keeps Rewriting the RulesA 1-bit Qwen derivative fits an iPhone, a healing trick beats its own teacher at 4-bit, and a dense LLM limps along on an $8 chip.
  47. 116 viewsA Duck Robot Runs Its Balance Loop On-Chip as Local AI Quantization Claims Get AuditedA Rockchip-powered duck robot, two offline Raspberry Pi builds, and audits exposing quantization's blind spots and mislabeled GGUF files.
  48. 119 viewsIntel's Wildcat Lake Brings a Modest NPU to AI PCs, While Edge AI Claims It's Going Mainstream in IoTIntel details a 17 TOPS NPU chip built for chiplets, and an industry trend piece argues edge inference is leaving the pilot stage — both light on independent proof so far.
  49. 117 viewsIBM Ships an Edge-First Granite LLM as RISC-V AI Boards Keep Fragmenting the ToolchainIBM's Granite 4.2 targets edge devices with a 3B open model, while a new RISC-V AI pocket computer ships locked to its own OS fork.
  50. 147 viewsJetson Orin Nano 2 Doubles Edge Robotics AI, While Local LLM Agents Get a Bigger Home on the DesktopNVIDIA doubles its entry robotics brain, Perplexity moves agents onto local GPUs, and Liquid AI ships a speedup and a benchmark suite for on-device models.
  51. 150 viewsHearing Aids Get Their Own AI Chips as On-Device Voice and Vision Push Past the PhoneHearing aids ship dedicated on-device AI chips, new AI glasses land, and real-phone benchmarks show why raw specs don't tell the whole story.
  52. 316 viewsMeta's Muse Glimmer Bets Big on On-Device Agentic AI as the Runtime Wars Keep MultiplyingMeta ships an on-device agentic model, an MoE engine claims 753B on one GPU, and researchers find 10 CVEs in a local inference engine.
  53. 151 viewsLiquid AI Trains Its Way Around the Q4_0 Quality Tax, While New Papers Chip Away at What Quantization Quietly BreaksLiquid AI distills instead of just rounding for 4-bit LFM2.5 checkpoints, as fresh research flags what low-bit quantization costs in memory and multilingual accuracy.
  54. 199 viewsEdge Dispatch: NVIDIA Squeezes a 4B World Model Onto the Robot Itself as Liquid AI and AMD Chase the Same EdgeA Jetson Thor robot policy, a GGUF port for VLA models, a 2.6B tool-calling LLM, and a new AMD robotics module — all inference, no data center.
  55. 227 viewsAMD's Ryzen AI Halo Jumps to 192GB While Qualcomm Pushes On-Device AI AgentsAMD bumps its NPU mini PC to 192GB of unified memory, Qualcomm ships five agentic apps for Snapdragon X, and Korea rethinks its NPU strategy.
  56. 166 viewsA Robot Arm Joins the NPU Party: FastFlowLM Adds a Vision-Language-Action Model to Ryzen AIFastFlowLM's first stable release puts a robotics policy on Ryzen AI's NPU, while Korea ships a boxed NPU appliance and a Raspberry Pi learns to narrate what it sees.
  57. 148 viewsKorea's KT Ships a Boxed NPU LLM Station While the ESP32 Crowd Trims Memory FurtherA Korean telecom sells an all-in-one on-prem LLM box built on a domestic NPU, while ESP32 tinkerers keep shrinking what a model needs to run.
  58. 142 viewsRyzen AI's NPU Runtime Goes Official While a Raspberry Pi Learns to See and Speak with a Tiny LLMAMD folds a hobbyist NPU runtime into ROCm, Google shows Gemma driving a robot from a Raspberry Pi 5, and a 45M-parameter model books tool calls on a phone.
  59. 127 viewsRaspberry Pi's GPU Joins the AI Party While a 14MB Model Learns to Call ToolsA Raspberry Pi 5 runs Gemma and vision models split across CPU and GPU, and a 45M-parameter model fits tool-calling into 28MB of RAM.
  60. 145 viewsThe $8 AI Chip Grows a Coffee Habit: ESP32 Tiny-LLM Trick Gets a Second ModelA new 'Barista' model and a closer look at Google's Per-Layer Embeddings trick show what running an LLM on a microcontroller can and can't do.