Jetson Thor Sprints Past llama.cpp While a 15M-Parameter LLM Still Fits an $8 Chip

Jetson AGX Thor Beats the llama.cpp Reference by 6.4x in a New MLPerf Edge Result

NVIDIA published MLPerf Inference v6.1 Edge Agentic results on September 16, 2026 showing its TensorRT Edge-LLM running Qwen3.6-27B on a single Jetson AGX Thor Developer Kit at 52.33 tokens per second, finishing all 1,007 turns of the benchmark in 24 minutes and 36 seconds versus 2 hours 37 minutes for the llama.cpp reference submission. The gain comes from NVFP4 quantization, tree-based multi-token prediction and KV cache reuse, per NVIDIA's developer blog.

This is a vendor benchmark against a stock reference implementation, not a head-to-head against a tuned llama.cpp build, and the Thor kit's 128GB of unified memory puts it at the far end of "edge" from a microcontroller. Still, it shows agentic workloads — multi-turn tool calls, not single-prompt chat — running entirely on a box that sits in a robot or a vehicle rather than a rack, a distinction SMNTCN's summary also flags.

A Dense 15-Million-Parameter Model Skips the Flash Trick

Where recent ESP32 LLM projects lean on Google's Per-Layer Embeddings to keep most weights parked in flash, 6farsi's NanoMind-S3 goes the other way: it puts Andrej Karpathy's real stories15M model — a 15.2 million parameter dense LLaMA-2 transformer trained on TinyStories — entirely resident and multiplies every stored weight on every token. It runs on a $4 ESP32-S3 DevKitC-1 with no WiFi or cloud dependency, generating short children's stories over a serial terminal at about 2.96 tokens per second, per the Hackster write-up.

A second builder, JARACH-209, pushed the same dense approach further with a 30.72-million-parameter stories42M variant on an $8 ESP32-S3, self-reporting 0.95 tokens per second with no SD card, per the project repo. Both are slower than the PLE-based 28.9M model this beat has already covered, but they trade speed for architectural simplicity — no flash lookup table, just weights and math — which matters for anyone trying to port the trick to a chip without much flash to spare.

A Paper Splits VLA Robots Between the Cloud and a Tiny Local Model

A preprint posted to arXiv on September 16, 2026, titled VLA-ULAP, argues that billion-parameter vision-language-action policies are too heavy and too latency-prone to run fully on a robot's own compute. Its proposed fix interleaves occasional calls to a large cloud VLA model with an ultra-lightweight local action predictor that handles the moments in between, aiming to keep a robot responsive when the network round-trip would otherwise stall it.

This is not on-device inference in the strict sense — the heavy model still lives off-board — and the paper is a design proposal, not a shipped robot. The honest reading is that it names the real constraint driving edge robotics right now: onboard power and bandwidth, not raw model accuracy, and it's a preprint without independent hardware benchmarks yet.

An Open Camera Module Puts 1.5 TOPS Behind Night Vision

AIMORELOGY launched a crowdfunding campaign on September 16, 2026 for the Ovis, an open-source AI vision camera module built on the CVITEK CV1842H-P SoC. It offers full-color 1080p night vision, AI-ISP support and a rated 1.5 TOPS of edge AI inference, according to CNX Software's coverage.

Crowdfunded hardware ships late and specs sometimes shift before production, so treat the TOPS figure as a stated target rather than a benchmark result. The interesting part is the pairing: night-capable full-color sensing plus an ISP tuned for on-chip AI, aimed at makers who want a security-camera-class module without sending frames anywhere.

A Color E-Paper Badge Runs Open Firmware on an ESP32-S3

ENILINX launched the TICKEY, a 3.7-inch color e-Paper smart badge and desk display built on an ESP32-S3, on a crowdfunding page published September 17, 2026, according to CNX Software. The company is shipping it with open firmware rather than a closed app ecosystem.

No AI model runs on the badge itself; it belongs here as a hardware pick because it's a fresh, dated ESP32-S3 board with an unusually generous display and an open toolchain — the kind of low-cost, well-documented board that ends up hosting someone's tiny local-inference demo within weeks of shipping.

Two very different edges show up today: a Jetson dev kit with 128GB of memory chasing agentic benchmarks, and $4–$8 microcontrollers arguing over whether dense or flash-offloaded weights make more sense for a 15-to-30-million-parameter story generator. Both are the same beat.