Category
24 posts
Your vector index and your query encoder drifted apart months ago. Retrieval quality is quietly collapsing, and nothing in your observability stack noticed.
Kimi K3 proved open-weight LLMs can reach the frontier — and proved they're far too big to run where users actually are. The next race isn't a bigger model; it's the architecture review that puts frontier AI on an ordinary laptop.
A four-bit model file tells you how many elements it has — and almost nothing else that matters. Not the scale-derivation rule, not the zero-point convention, not the packing order. In 2026 alone, six documented incidents across vLLM and SGLang turned those silent agreements into silently wrong model output. GRIT is my answer: a 64-byte descriptor and an O(1) boundary check for block-scaled reduced-precision arrays, with five zero-dependency implementations that agree bit-for-bit on 96/96 cross-language fingerprints — and a read-only scanner that found real convention ambiguity in checkpoints you can download today.
I measured what MCP tool definitions actually cost a context window: ~6.6× redundancy, ~17% of 200k at 72 tools — and the brand-new spec doesn't touch it. Plus the false-positive hunt that broke my own 'zero FP' claim and the detector that came out of it.
I wanted semantic search with no server — and every Rust HNSW crate refused to compile to WebAssembly. So I wrote ferrovec: a hand-rolled HNSW vector index whose only Rust dependencies are serde and postcard, that denies unsafe code crate-wide, uses no system randomness, and produces a wasm build the project reports at ~33 KB gzipped. This is the launch: the algorithm, the determinism, the compaction, and the leap into the browser — the WASM core, transformers.js auto-embedding on a Web Worker, OPFS persistence, and single-writer leader election across tabs.
crimson-crab is a Rust SDK for Anthropic's Claude API: v0.1.0, 191 passing tests, zero clippy warnings, and a library that denies unwrap, expect and panic at compile time. This is the launch post: why tokio sits in the native dependency tree and is absent from the wasm32 one, why 113 of the 191 tests are the documentation, and what happens when a response arrives from a model the SDK has never heard of.
Meta made open-weight LLMs a movement, then quietly went closed in 2026. But the open frontier didn't collapse — it moved, mostly to Chinese labs, and the gap to the best closed models shrank to roughly four months.
For three years, on-device AI was a demo that almost worked. In June 2026 it stopped being one. Sparse models like Apple's AFM 3 and Google's Gemma 4 made intelligence large in flash, small in motion, free to run, and offline by default.
In 2024, you needed a 70B model to get good answers. In 2026, a 7B model trained on the right data beats it on most real-world tasks. The mechanism isn't a secret — it's distillation done well, and it's reshaping the entire model economy.
In April 2026, Anthropic blocked the year's most viral open-source agent framework from spending Claude subscriptions. The fight wasn't really about one developer in Vienna — it was about an unspoken truth the labs had been dodging for a year: chat is the past, agents are the interface, and subscriptions cannot price what comes next.
Every frontier model now claims a 1M-token context window. In production, almost no one uses more than 64K. Here's the gap between the benchmark and the reality, and what to do about it.
Model Context Protocol started as Anthropic's spec for hooking Claude into tools. A year later, every major AI provider, IDE, and SaaS vendor speaks it. This is what protocol-winning looks like in real time.
Enterprise teams are ditching traditional SaaS dashboards for autonomous AI agents that monitor, decide, and act. Here's what's driving the shift and what it means for software builders.
RAG is the most deployed LLM pattern in production today. After building RAG systems for 18 months, here are the architectural decisions that matter and the mistakes that don't show up until scale.
Token costs are just the tip of the iceberg. After running LLM workloads in production for a year, here's where the money actually goes — and how to cut costs without cutting quality.
AI systems fail differently than traditional software. After investigating 50 production incidents across ML systems, here are the patterns — and the engineering practices that prevent them.
Three approaches to customizing LLM behavior, each with different tradeoffs. A decision framework based on your data, budget, and accuracy requirements.
AI agents don't just write code — they review it. Autonomous code review catches bugs, security flaws, and design issues that human reviewers miss. Here's how it works.
The single most important capability that turned language models into agents wasn't better reasoning — it was tool use. Here's the technical story of how function calling changed everything.
Traditional RAG retrieves documents and stuffs them into context. Agentic RAG plans queries, evaluates results, and iterates until it finds the right answer.
Building a demo agent is easy. Shipping one that handles edge cases, recovers from failures, and earns user trust is hard. Here are the lessons learned.
Not all AI agents are created equal. A practical comparison of Claude, GPT-4, and Gemini on real software engineering tasks — coding, debugging, and system design.
Single agents are powerful. Teams of specialized agents working together are transformative. Here's how multi-agent architectures are reshaping complex problem-solving.
The shift from AI-as-tool to AI-as-agent represents the biggest paradigm change since the internet. Here's how we got here and where it's heading.