Polars vs DuckDB in 2026: When To Pick Which

Not the comparison everyone thought it would be

In 2023, every data-engineering newsletter on the planet was running some flavor of "Polars vs DuckDB — who wins?" The implicit assumption was that the two new in-process columnar engines were going to slug it out for the same wedge — analytical workloads on a single machine that Spark and Snowflake couldn't economically serve — and one would emerge dominant.

Three years later, the cage match never happened. Polars went one direction; DuckDB went another. Both ate enormous amounts of real workload. Neither displaced the other. The part everyone undercounted in 2023 is that the choice is almost never about raw speed — it's about whether the human writing the query thinks in DataFrames or in SQL, and what the data on the other end of the pipeline looks like.

The deceptively shared substrate

Read the architecture docs of Polars and DuckDB side by side and the convergence is striking. Both keep data in Apache Arrow columnar buffers. Both push compute down hard — predicate and projection pushdown, late materialization, vectorized execution. Both have a query optimizer that rewrites intent into something cheaper before running. Both read Parquet directly off S3, scan Iceberg tables, talk to Postgres. Both ship as a single dependency that drops into a notebook with no infrastructure.

The differences are real, but they live on the surface — the API the user touches — far more than they live in the engine. Polars exposes a Rust-native DataFrame API with a Python binding that has eaten Pandas. DuckDB exposes ANSI SQL with a thin client in every language. Same Arrow batches underneath, different ergonomic frame on top. The obvious benchmark question — "which one runs SUM(x) GROUP BY y faster" — is the wrong question. They're close enough on most workloads that the answer flips depending on dataset shape, column cardinality, and engine version.

Polars wins when

Polars is the answer for any team whose primary mental model is the DataFrame. That's a much bigger group than the SQL camp wants to admit. Most ML feature pipelines are written in Pandas. Most data-science notebooks are written in Pandas. Most exploratory analysis in a Jupyter cell is written in Pandas. Polars eats that workflow whole.

Specifically, Polars wins when:

• You're migrating off Pandas. The API similarity is close enough that most Pandas codebases port over in days, not months, and the performance jump is typically 5-30× depending on the workload. The Polars migration guide is the reference; for most teams, this is the single highest-leverage refactor of 2025-2026.
• You're chaining lazy transforms. pl.scan_parquet("s3://...").filter(...).group_by(...).agg(...).collect() is the Polars idiom, and the lazy planner aggressively pushes filters and projections into the Parquet reader so you only pay for the columns and row groups you actually use. The lazy API is the single biggest reason Polars feels fast on Parquet — it's not that the kernels are magic, it's that the planner reads less data.
• Your dataset is larger than memory. The Polars streaming engine — officially recommended as of the Polars team's December 2025 update (default-flip on the roadmap; v1.31 itself shipped June 2025) — processes batches rather than loading the full frame, which means a 200 GB Parquet file on a laptop is no longer an out-of-memory error. The streaming engine still has rough edges on certain group-bys (see the open issues in the tracking issue), but for scan-filter-aggregate it's reliable.
• You want a Rust eager fast path. Polars is written in Rust and the Python binding is a thin shim. If you're building a service in Rust that needs a DataFrame, Polars is the only serious choice.
• You're doing ML feature engineering. The Polars expression API — composable, lazily-evaluated, type-checked at plan time — is a much better fit for feature transformations than SQL is. You can build reusable expression libraries, parameterize them, unit-test them in Python.

DuckDB wins when

DuckDB is the answer for any team whose primary mental model is SQL — which is most of the analytics, BI, and warehouse-adjacent world. The thing DuckDB does better than anything else, including Polars, is federate: read from Parquet and Postgres and Iceberg and a CSV in one SQL statement, with the optimizer deciding which pushdowns belong on which source.

Specifically, DuckDB wins when:

• You're querying heterogeneous sources in a single statement. SELECT * FROM postgres_scan(...) JOIN read_parquet('s3://...') JOIN iceberg_scan(...) is a one-liner in DuckDB; in Polars, you'd be wiring three readers together by hand. The pg_duckdb extension and the broader DuckDB federation surface — Parquet, JSON, CSV, Iceberg, Delta, MySQL, SQLite, Postgres — make it the natural choice when the source data is a zoo.
• Your consumers already speak SQL. Analysts, BI tools, dbt models, jupyter SQL-magic users — the whole pre-existing SQL ecosystem speaks to DuckDB without translation. dbt-duckdb in particular has become the default way to run small-to-medium dbt projects without paying for Snowflake or BigQuery.
• You need complex joins. DuckDB's join optimizer is more mature. On TPC-H-style multi-way joins with selective predicates, it tends to find the better plan, especially where join reordering matters.
• You need the browser. DuckDB-WASM runs the full engine inside a browser tab and has unlocked an entire category of "no-backend dashboard" applications. Polars has experimental WASM builds, but nothing close.
• You're embedding analytics into something bigger. TUIs, CLIs, IDE plugins, MotherDuck's cloud — DuckDB's single-file C++ binary slots into any host process.

The DataFusion question

The most interesting development underneath both Polars and DuckDB is Apache DataFusion — a Rust-native query engine built directly on Arrow, promoted to an ASF top-level project in 2024, and now the substrate beneath what InfluxData estimates is 1,000+ production systems in 2025: InfluxDB 3 (a full ground-up rebuild of the time-series database around DataFusion), GlareDB, Comet (Apple's Spark accelerator, donated to ASF), Ballista, and a long tail of analytics startups.

DataFusion's pitch is that the hard parts of building an analytical engine — physical operators, SIMD-optimized kernels, columnar memory, Substrait plan exchange, pluggable optimizer rules — should be a shared library, not something every team rebuilds. You write your engine's surface and inherit the execution layer.

This makes the Polars/DuckDB framing look increasingly transitional. Polars's planner is its own (Rust, not DataFusion-derived). DuckDB's engine is its own (C++, not DataFusion-derived). But both interoperate with DataFusion via Arrow, both ship Substrait support, and the gravitational pull of "use the shared substrate" is real. Apple did not rebuild Spark execution from scratch — they wrote DataFusion-Comet on top of DataFusion and got a ~2× TPC-DS SF1000 speedup for free.

The honest read for 2026: if you're a consumer of analytical compute, pick Polars or DuckDB based on team taste. If you're a builder of analytical compute — a new database, a query engine, an embedded analytics product — start with DataFusion. The "build your own engine in C++ or Rust" path looks increasingly like the last era's choice.

The performance numbers

For the record: on aggregation-heavy workloads, all three engines land within roughly 2× of each other on most queries, and the winner flips depending on dataset shape, hardware, and version. The DuckDB Labs fork of the H2O.ai db-benchmark — the most-cited apples-to-apples comparison and last run in late 2025 — shows DuckDB leading on group-by and join across most data sizes, with Polars close behind and frequently winning on subsets of the workload. Real-world numbers track the same shape: independent Parquet benchmarks typically find them within 20-50% of each other.

What benchmark blogs do badly is communicate this. They show a single workload where engine X beats engine Y by 17% and treat it as a verdict. Run twenty more queries with different cardinalities, group counts, predicate selectivities — the ranking reshuffles every time. If you're picking an engine on a microbenchmark delta, you're optimizing the wrong variable.

The team-shaped decision

Here's the honest framework. Forget the benchmark spreadsheet. Ask three questions.

What do your engineers write right now? If the answer is "a lot of Pandas," the migration path to Polars is shorter than the migration path to SQL, and the productivity win is immediate. If the answer is "a lot of dbt models" or "warehouse SQL," DuckDB drops in without retraining anyone. If the answer is "we're building a new analytical engine in Rust," start with DataFusion and skip the rest of this article.

What do your consumers read? If your output feeds Tableau, Looker, or a BI tool, it wants a SQL endpoint — DuckDB plays that role more cleanly. If your output feeds a Python ML pipeline that consumes a DataFrame, Polars cuts the type-conversion step entirely.

How heterogeneous is your data? If everything you query is Parquet on S3, both engines handle it equivalently. If you need to join Parquet with a live Postgres table and an Iceberg snapshot in one statement, DuckDB's federation surface is the bigger lever.

What you don't need to ask is "which one is faster." On the workloads where the answer matters, they're close enough that it doesn't. On the workloads where the answer doesn't matter, you have bigger problems than your engine choice. The 2023 framing was wrong — there was no cage match. Pick the one your team can write fluently in by Friday and spend the saved cycles on the data model, the pipeline reliability, and the cost story. That's where the real differentiation lives.