Apple Silicon's Unified Memory Is the Quiet Revolution in Analytical Compute

The PCIe assumption

For the last fifteen years, every paper, product, and pitch deck about GPU-accelerated analytics has had the same shape. Step one: load the data into host RAM. Step two: copy the relevant batch over PCIe into the GPU's VRAM. Step three: run the kernel. Step four: copy the result back over PCIe. Steps two and four were the dominant cost on small kernels and a meaningful tax even on large ones, which is why the literature is full of optimizations like "pinned memory," "async streams," "RDMA," and "GPUDirect Storage" — every one of them an attempt to make the PCIe round-trip hurt less.

That whole architectural premise was correct on every NVIDIA discrete-GPU system ever shipped. It is wrong on Apple Silicon, and it has been wrong since the M1 launched in November 2020. Five years later, the SQL world had still not noticed.

What unified memory actually means

Apple's M-series chips share a single physical memory pool between the CPU cores, the GPU cores, the Neural Engine, and the media encoders. There is no host RAM and device VRAM — there is one pool, addressed by all engines, with cache coherency handled in hardware. A GPU kernel reading a buffer that the CPU just wrote does not trigger a copy; it reads the same physical page. The bandwidth available to the GPU is the full memory bandwidth of the chip, not the bandwidth of a PCIe link.

The numbers are striking on the high-end SKUs:

Read this chart carefully. The H100 number is dazzling — 3.35 TB/s — but it applies only to data already in HBM. To get data into HBM, you still pay PCIe. So for any analytical workload that doesn't fit in 80 GB of VRAM, the effective end-to-end bandwidth is gated by PCIe, which is two orders of magnitude slower than HBM. UMA dodges this entirely. M3 Ultra's GPU sees 819 GB/s of bandwidth across the full 512 GB of memory, with no copy step.

That changes which workloads make sense on a GPU. On a discrete-GPU system, you only put a workload on the GPU if it does enough compute per byte to amortize the PCIe round-trip. SQL aggregates almost never clear that bar — they're bandwidth-bound, with maybe four or eight FLOPs per byte read. That's why GPU SQL engines have historically struggled on small queries: the dispatch and copy overhead crushes the kernel speedup. On UMA, that calculus changes. The dispatch is still real, but the copy is gone, so the bar moves down by an order of magnitude.

What the field had built

Look at the public GPU OLAP ecosystem as of late 2025 and a pattern jumps out:

• cuDF / RAPIDS: CUDA-only.
• BlazingSQL: CUDA-only. Project went dormant in 2024.
• HEAVY.AI / HeavyDB: CUDA-only. Acqui-hired by NVIDIA in 2025.
• Sirius (UW-Madison + NVIDIA, CIDR 2026): CUDA-only.
• Voltron Data: CUDA-focused. Shed half its staff.

Nobody had wired Metal or MLX behind a SQL execution engine. The reason is partially historical (the GPU-database research community grew up on CUDA in the 2010s) and partially structural (Apple Silicon was a consumer/creator chip until the Mac Pro M2 Ultra in 2023). But the silicon is now sitting on roughly a fifth of the developer laptops in the world. That's a real wedge.

What changes when you build for it

Two things become possible on UMA that were prohibitively expensive on discrete GPUs:

1. Small-query GPU acceleration. A 100 ms aggregate over 5 GB of column data was uneconomic on a discrete GPU because the PCIe copy alone took 100 ms. On UMA the kernel just runs against the existing memory page. You get GPU acceleration on workloads where it never made sense before.
2. "Hot pool" workloads. Any workflow where the working set is smaller than RAM but larger than VRAM — which is most analytical work — becomes a UMA sweet spot. The GPU sees the entire working set at full bandwidth. On a discrete system, you're constantly streaming over PCIe.

This is the architectural premise behind gpudb. The Metal backend is not a port of the CUDA path — it's a different design that takes UMA seriously, doesn't allocate device buffers, doesn't stage anything over PCIe, and dispatches kernels directly against the same Arrow batches DuckDB already has in memory. That's why a 60M-row TPC-H lineitem aggregate fuses to 1 ms on M4 Max and beats a 16-thread DuckDB CPU run by 22-25× on the multi-aggregate workload. It's not magic. It's just that nobody had bothered to build for the architecture that's been sitting on developers' desks for half a decade.

The disclaimer

Two honest caveats. First, UMA peak bandwidth is shared between the CPU and GPU. If both engines are saturating memory simultaneously, neither gets the headline number — they share. For pure GPU compute it's still 800+ GB/s; for mixed CPU/GPU it depends on the workload. Second, not every Mac is an Ultra. The M4 base chip ships with 120 GB/s of bandwidth — respectable, but a quarter of M4 Max and a sixth of M3 Ultra. The story matters most on the prosumer-and-up SKUs.

But the architectural point holds: for the first time in the GPU-database era, there's a mainstream silicon platform where the host-to-device copy isn't the bottleneck. That changes which problems are worth solving on the GPU, and which engines are worth building. The companies that figure this out first will own a wedge that the CUDA-only field structurally cannot reach.