Blog

Thoughts, research, and insights on technology.

All AI & Machine Learning Data Engineering Opinion Research

The First SQL Engine for Apple Silicon GPUs Is Now a DuckDB Community Extension

In May 2026 I shipped gpudb v0.1 — the first SQL execution engine targeting Apple Silicon GPUs, built as a DuckDB extension with a CUDA backend on Linux. Three releases later, the project crossed two lines at once. v0.3.0's streaming-aggregate rewrite reached parity with native DuckDB on end-to-end TPC-H queries — the worst cell improved roughly 100×, from 11.05 s to 0.109 s. And gpudb became an official DuckDB Community Extension: INSTALL gpudb FROM community now works in any DuckDB ≥ 1.5.5, signed, no flags. This is the full arc — what v0.1 proved, what v0.2 honestly lost, what v0.3 fixed, and why the next GPU frontier is joins.

Aug 11, 20268 min read

Read

AI & Machine Learning7 min read

The Hidden Cost of Embedding Model Drift in Production RAG

Your vector index and your query encoder drifted apart months ago. Retrieval quality is quietly collapsing, and nothing in your observability stack noticed.

Aug 15, 2026Read

AI & Machine Learning8 min read

The Future of Local AI: Every AI Lab Should Redesign Its LLM Architecture to Run on Your Laptop

Kimi K3 proved open-weight LLMs can reach the frontier — and proved they're far too big to run where users actually are. The next race isn't a bigger model; it's the architecture review that puts frontier AI on an ordinary laptop.

Aug 9, 2026Read

Research7 min read

Speculative Decoding: The Inference Trick Hiding in Plain Sight

Speculative decoding promises faster LLM inference without touching model weights. The math holds up — but the production story is messier than the papers admit.

Aug 8, 2026Read

AI & Machine Learning9 min read

I Gave Quantized LLM Checkpoints a Type, and the Type Immediately Caught Real Bugs

A four-bit model file tells you how many elements it has — and almost nothing else that matters. Not the scale-derivation rule, not the zero-point convention, not the packing order. In 2026 alone, six documented incidents across vLLM and SGLang turned those silent agreements into silently wrong model output. GRIT is my answer: a 64-byte descriptor and an O(1) boundary check for block-scaled reduced-precision arrays, with five zero-dependency implementations that agree bit-for-bit on 96/96 cross-language fingerprints — and a read-only scanner that found real convention ambiguity in checkpoints you can download today.

Aug 6, 2026Read

Research7 min read

Mechanistic Interpretability: What We Actually Know So Far

Mechanistic interpretability has produced real, reproducible findings. Here is an honest accounting of what those findings actually mean for engineers.

Aug 1, 2026Read

AI & Machine Learning8 min read

Your MCP Tools Cost 6.6× More Context Than They Need. I Measured It Against the New Spec.

I measured what MCP tool definitions actually cost a context window: ~6.6× redundancy, ~17% of 200k at 72 tools — and the brand-new spec doesn't touch it. Plus the false-positive hunt that broke my own 'zero FP' claim and the detector that came out of it.

Jul 30, 2026Read

Research13 min read

We Shipped a “Provable” Bound. It Wasn’t Provable.

samkhya's whole reason to exist is a join-cardinality ceiling you can prove — so a model clamped under it can't wreck the query plan. In July I materialized the true output of 926 join trials and checked. The ceiling came in below the true answer in 2,179 of 3,704 evaluations. This is the bug, the repair that took it to zero, and every number I had to retract on the way.

Jul 25, 2026Read

Data Engineering10 min read

The Agent-Written Data Pipeline: The Review Bottleneck Nobody Priced In

AI agents can now write dbt models, SQL transforms, and backfills that pass CI and ship. The catch: a wrong number doesn't crash, it quietly poisons every dashboard downstream. The hard part moved from authoring to verification.

Jul 25, 2026Read

Data Engineering9 min read

We Published Our 110× Loss. One Release Later, It Was Gone.

A reviewer on gpudb's DuckDB community-extensions PR asked the question every GPU project dreads: forget the kernel benchmarks — what does a user actually see end-to-end? We ran it honestly. Native DuckDB won every query shape, by 3× to 109×, against our own extension. We published those numbers in our own release notes — and the act of writing them down produced the structural diagnosis that closed the entire gap in the very next release. The fix was the opposite of what a GPU database is supposed to do: delete the GPU from the hot path. This is the full story, with every number.

Jul 20, 2026Read

AI & Machine Learning10 min read

ferrovec: a Tiny Rust HNSW Vector Index That Runs Semantic Search Inside the Browser Tab

I wanted semantic search with no server — and every Rust HNSW crate refused to compile to WebAssembly. So I wrote ferrovec: a hand-rolled HNSW vector index whose only Rust dependencies are serde and postcard, that denies unsafe code crate-wide, uses no system randomness, and produces a wasm build the project reports at ~33 KB gzipped. This is the launch: the algorithm, the determinism, the compaction, and the leap into the browser — the WASM core, transformers.js auto-embedding on a Web Worker, OPFS persistence, and single-writer leader election across tabs.

Jul 18, 2026Read

Data Engineering10 min read

The Inference Hardware Wars: Why Your Token Bill Is Decided in a Fab, Not a Prompt

The token-price crash everyone cheers isn't software magic. It's a hardware war: Cerebras and Groq attacking on speed, NVIDIA's Rubin counterpunching on cost-per-token. The winner of that fight, not your prompt, sets your inference bill and your latency floor.

Jul 17, 2026Read

AI & Machine Learning8 min read

crimson-crab: a Production-Grade Rust SDK for Claude — and Why tokio Leaves the Dependency Tree on wasm32

crimson-crab is a Rust SDK for Anthropic's Claude API: v0.1.0, 191 passing tests, zero clippy warnings, and a library that denies unwrap, expect and panic at compile time. This is the launch post: why tokio sits in the native dependency tree and is absent from the wasm32 one, why 113 of the 191 tests are the documentation, and what happens when a response arrives from a model the SDK has never heard of.

Jul 16, 2026Read

Data Engineering10 min read

samkhya v1.1: Never Regress — Putting a Model in Your Query Optimizer Without Letting It Wreck the Plan

samkhya is a Rust SDK that lets a model — a gradient-boosted tree, TabPFN-2.5, even an LLM — correct the row-count estimates your query optimizer runs on, under a provable ceiling that a hallucinating model can never breach. This is the deep dive: the never-regress clamp, the portable Iceberg sidecar, the three swappable backends, and the honest benchmark I pre-registered and then failed — reported as such.

Jul 15, 2026Read

AI & Machine Learning8 min read

The Open-Weight Frontier Didn't Die — It Moved

Meta made open-weight LLMs a movement, then quietly went closed in 2026. But the open frontier didn't collapse — it moved, mostly to Chinese labs, and the gap to the best closed models shrank to roughly four months.

Jul 11, 2026Read

Opinion9 min read

Sol, Terra, Luna vs. Fable 5: The Week the AI Frontier Got a Price War

On July 9 OpenAI shipped GPT-5.6 as three tiers — Sol, Terra, Luna — from $1/$6 to $5/$30 per million tokens. Four days later, Anthropic's best model, Fable 5, comes off subscriptions and onto a $10/$50 meter. Same week, opposite directions. Is Fable 5 losing ground — and does Anthropic need to blink on July 12?

Jul 10, 2026Read

Data Engineering5 min read

My DuckDB Extension PR Sat Red for Eight Weeks. The Bug Wasn't What the CI Said.

A DuckDB community-extension PR sat red for eight weeks over a job named linux_arm64. The real bug was three words in the log — and a fleet of AI agents took it green on all four platforms in about seventy minutes.

Jul 5, 2026Read

Opinion7 min read

The 10x-a-Year Price Collapse Is an Architecture Bet, Not a Prompt Trick

Token-shaving is fighting yesterday's war. The price of any fixed capability tier falls roughly 10x a year, so the real edge isn't a leaner prompt — it's an architecture that treats today's frontier model as a swappable, soon-to-be-cheap component.

Jul 4, 2026Read

AI & Machine Learning7 min read

On-Device AI Just Got Real

For three years, on-device AI was a demo that almost worked. In June 2026 it stopped being one. Sparse models like Apple's AFM 3 and Google's Gemma 4 made intelligence large in flash, small in motion, free to run, and offline by default.

Jun 27, 2026Read

Opinion10 min read

The Coding-Agent Arms Race: Who Survives the H1-2026 Shakeout

In six months, AI coding agents went from features to a brutal platform war: $26B startups, a new frontier model every six weeks, pricing whiplash, and a reverse-acquihire that gutted a unicorn. The agent you build on is now a strategic bet.

Jun 20, 2026Read

Data Engineering10 min read

Streaming OLAP: The Post-Kafka Stack for Real-Time Analytics

The Kafka + Flink + ClickHouse/Pinot/Druid stack we built between 2018 and 2024 is fragmenting into three forks: single-engine streaming SQL, table-format-as-stream, and OLAP databases that eat the streaming layer entirely. Kafka isn't dying — it's becoming plumbing.

Jun 13, 2026Read

Data Engineering6 min read

What I Learned Writing GPU Kernels for SQL Aggregates

Three months, two abandoned designs, one breakthrough. The one-paragraph version: Apple Silicon GPUs don't have 64-bit atomic_fetch_add until very recent OS versions, and that single missing instruction shapes every other architectural decision in a Metal SQL aggregate engine.

Jun 6, 2026Read

Data Engineering5 min read

Multi-Aggregate Fusion: One Read, Four Answers

Every analytical engine treats SELECT SUM(x), MIN(x), MAX(x), COUNT(x) FROM t as four passes of the column. Fuse them into a single kernel and the speedup ratio against four-pass code becomes 9x to 25x. Here's why the technique works, and the data shape where it doesn't.

May 30, 2026Read

Data Engineering6 min read

Apple Silicon's Unified Memory Is the Quiet Revolution in Analytical Compute

M3 Ultra ships 512 GB of memory at 819 GB/s, addressable by the GPU with zero PCIe transfer cost. Every GPU database project from the past decade was architected around the assumption that memory bandwidth came at PCIe-tax prices. That assumption is now wrong on a fifth of the developer laptops in the world.

May 23, 2026Read

Data Engineering16 min read

samkhya v1.0: Plug Claude, GPT-4o-mini, or Local Ollama Into Your SQL Query Optimizer

samkhya v1.0 ships an LLM-pluggable corrector backend for embedded analytical engines — DataFusion, DuckDB, Polars, Postgres, Iceberg, gpudb. Plug Claude, GPT-4o-mini, or local Ollama into the cardinality-estimation slot via a simple HTTP wire contract (Python FastAPI and Node TypeScript reference servers ship in the box). Every LLM output is clamped from above by a provable pessimistic ceiling (LpJoinBound — 40.95× tighter than the 2008 AGM bound) so the LLM can never make your plan worse than the engine's native estimate. Transport-floor latency measured at P95 0.07–0.11 ms; live-LLM end-to-end cells honestly marked PROJECTED pending API budget.

May 17, 2026Read

Data Engineering27 min read

Why I built a GPU SQL engine in 2026 — when every other one died

Every standalone GPU database built between 2013 and 2024 was acqui-hired or pivoted. So why ship gpudb in 2026? Because nobody had wired Apple Silicon's unified memory into a SQL engine — and DuckDB hands you a hundred-thousand-user distribution channel without writing a database from scratch.

May 9, 2026Read

Opinion4 min read

The Cost of Being Right: AI-Generated Code at Production Scale

Generating code with AI is cheap. Reviewing, testing, and deploying it isn't. The new bottleneck for engineering teams isn't writing the code — it's trusting it.

May 1, 2026Read

Data Engineering5 min read

Databricks vs Snowflake vs The New Wave: The Data Engineering Paradigm Shift

Snowflake just posted $4.68B in FY26 revenue at 29% growth. Databricks crossed $5.4B ARR in February at 65% growth. And neither chart explains why the most interesting data infrastructure being shipped in 2026 is single-process, embeddable, and runs on a laptop.

Apr 27, 2026Read

AI & Machine Learning4 min read

Why Small Models Are Eating Their Teachers

In 2024, you needed a 70B model to get good answers. In 2026, a 7B model trained on the right data beats it on most real-world tasks. The mechanism isn't a secret — it's distillation done well, and it's reshaping the entire model economy.

Apr 25, 2026Read

Opinion3 min read

The Vibe Coding Economy: When 'Make This' Is the Spec

The phrase 'vibe coding' was a joke in 2024. By 2026, it describes how a non-trivial fraction of new software is actually built. The economics of this shift are stranger and more durable than the meme suggested.

Apr 18, 2026Read

AI & Machine Learning13 min read

OpenClaw vs. Anthropic: The Week the Subscription Era Ended for AI Agents

In April 2026, Anthropic blocked the year's most viral open-source agent framework from spending Claude subscriptions. The fight wasn't really about one developer in Vienna — it was about an unspoken truth the labs had been dodging for a year: chat is the past, agents are the interface, and subscriptions cannot price what comes next.

Apr 15, 2026Read

AI & Machine Learning3 min read

A Million Tokens, A Thousand Disappointments

Every frontier model now claims a 1M-token context window. In production, almost no one uses more than 64K. Here's the gap between the benchmark and the reality, and what to do about it.

Apr 10, 2026Read

AI & Machine Learning3 min read

MCP and the Quiet Standardization of AI Tool Use

Model Context Protocol started as Anthropic's spec for hooking Claude into tools. A year later, every major AI provider, IDE, and SaaS vendor speaks it. This is what protocol-winning looks like in real time.

Apr 1, 2026Read

Opinion3 min read

The Death of Prompt Engineering

For two years, the most-clicked role on LinkedIn was Prompt Engineer. In 2026, that role is quietly disappearing — because the model is now the one writing the prompts.

Mar 22, 2026Read

Data Engineering10 min read

Iceberg's Puffin Sidecars: Portable Stats for the Open Lakehouse

Apache Iceberg's Puffin file format is the most strategically important subsystem nobody is talking about. It is the mechanism by which an open lakehouse can carry warehouse-grade statistics across vendors — write the sketch once in Trino, read it tomorrow in Snowflake, plan a join correctly on the first cold query.

Mar 18, 2026Read

AI & Machine Learning9 min read

Why AI Agents Are Replacing SaaS Dashboards in 2026

Enterprise teams are ditching traditional SaaS dashboards for autonomous AI agents that monitor, decide, and act. Here's what's driving the shift and what it means for software builders.

Mar 14, 2026Read

AI & Machine Learning14 min read

Understanding Retrieval-Augmented Generation: Architecture, Pitfalls, and Production Lessons

RAG is the most deployed LLM pattern in production today. After building RAG systems for 18 months, here are the architectural decisions that matter and the mistakes that don't show up until scale.

Mar 10, 2026Read

AI & Machine Learning10 min read

The Real Cost of Running LLMs in Production: A Breakdown

Token costs are just the tip of the iceberg. After running LLM workloads in production for a year, here's where the money actually goes — and how to cut costs without cutting quality.

Mar 3, 2026Read

AI & Machine Learning13 min read

Building Reliable AI Pipelines: Lessons from 50 Production Failures

AI systems fail differently than traditional software. After investigating 50 production incidents across ML systems, here are the patterns — and the engineering practices that prevent them.

Feb 22, 2026Read

Data Engineering6 min read

DuckDB Ate the Modern Data Stack

An embedded analytical engine with no servers, no cluster, no migration cost just quietly displaced Spark for small data and Snowflake XS for medium data. MotherDuck closed Series B at a $400M post-money. Here's the part everyone undercounts.

Feb 17, 2026Read

AI & Machine Learning11 min read

Fine-Tuning vs. Prompting vs. RAG: Choosing the Right LLM Strategy

Three approaches to customizing LLM behavior, each with different tradeoffs. A decision framework based on your data, budget, and accuracy requirements.

Feb 12, 2026Read

Data Engineering5 min read

Iceberg, Delta, Hudi: Pick One in 2026 and Move On

The table-format wars are functionally over. Iceberg won on interop. Delta won on installed base. Hudi won on streaming upserts. The decision tree for a new project in 2026 is shorter than the comparison-blog industry wants you to believe.

Feb 12, 2026Read

Data Engineering9 min read

Polars vs DuckDB in 2026: When To Pick Which

Polars ate Pandas. DuckDB ate everything below the warehouse. The 2023 expectation was a cage match between two in-process analytical engines — the 2026 reality is they ate different cake, and the decision is mostly about whether your team thinks in DataFrames or SQL.

Jan 8, 2026Read

AI & Machine Learning9 min read

Autonomous Code Review: How AI Agents Are Raising the Bar for Software Quality

AI agents don't just write code — they review it. Autonomous code review catches bugs, security flaws, and design issues that human reviewers miss. Here's how it works.

Nov 18, 2025Read

Data Engineering10 min read

Vector Indexes in OLAP Engines: 2025 Is Where Search Ate Analytics

DuckDB, ClickHouse, Snowflake, BigQuery, Postgres — by late 2025 every serious analytical engine ships a native vector index. That wasn't an AI-hype reflex. It was the realization that embedding search is just a column scan with a different distance function, and the warehouse-plus-vector-DB split was operational waste for the 90% case.

Oct 20, 2025Read

AI & Machine Learning8 min read

The Tool-Use Revolution: How Function Calling Transformed LLMs Into Agents

The single most important capability that turned language models into agents wasn't better reasoning — it was tool use. Here's the technical story of how function calling changed everything.

Oct 5, 2025Read

Data Engineering10 min read

Apache Arrow IPC vs JSON: The Numbers Behind the Switch

Most data-API traffic in 2025 still moves as JSON because humans need to read it. But for any system actually shipping columnar batches between services — analytical pipelines, feature stores, embedding services, MCP-style tool calls — Arrow IPC is 3-30× faster end-to-end. Honest accounting of when the switch pays off and when JSON is still correct.

Aug 15, 2025Read

AI & Machine Learning9 min read

RAG Is Dead, Long Live Agentic RAG: The Evolution of AI Knowledge Systems

Traditional RAG retrieves documents and stuffs them into context. Agentic RAG plans queries, evaluates results, and iterates until it finds the right answer.

Jul 22, 2025Read

AI & Machine Learning9 min read

Building Production AI Agents: Lessons from Shipping Autonomous Systems

Building a demo agent is easy. Shipping one that handles edge cases, recovers from failures, and earns user trust is hard. Here are the lessons learned.

Jun 15, 2025Read

AI & Machine Learning10 min read

Claude, GPT, Gemini: Comparing AI Agent Capabilities in Real-World Tasks

Not all AI agents are created equal. A practical comparison of Claude, GPT-4, and Gemini on real software engineering tasks — coding, debugging, and system design.

May 20, 2025Read

AI & Machine Learning8 min read

Multi-Agent Systems: When AI Agents Learn to Collaborate

Single agents are powerful. Teams of specialized agents working together are transformative. Here's how multi-agent architectures are reshaping complex problem-solving.

Apr 10, 2025Read

AI & Machine Learning9 min read

The Agentic Paradigm Shift: Why 2025 Changed Everything in AI Development

The shift from AI-as-tool to AI-as-agent represents the biggest paradigm change since the internet. Here's how we got here and where it's heading.

Mar 1, 2025Read