Asynchronous I/O in DuckDB: Work, Thread, Work duckdb.org ↗ HN ↗

Pedro Holanda on how DuckDB stopped letting network latency stall computation: a worker pool decodes data while a separate, larger pool waits on blocking network calls, feeding a read-ahead queue. An 80GB CSV read from S3 dropped from 877 seconds to 45, about 19 times faster.