Threads & Processes
A track of P32 · Concurrency & Parallelism.
Real threads and processes under the GIL: why threads never speed up CPU-bound work, how ProcessPoolExecutor and fork side-step it, and how Locks, Queues and Semaphores keep shared state correct under contention.
Your data-prep script runs for an hour, and a colleague says "just thread it."
You add a ThreadPoolExecutor, run it on your 8-core machine — and it is exactly
as slow. Nothing sped up. If you know why, that is a two-second diagnosis; if you
don't, it is an afternoon of confusion. The answer is one lock.
This track is the concurrency toolkit, taught against the thing that governs it — the Global Interpreter Lock. Only one thread runs Python bytecode at a time, so threads never speed up CPU-bound work; but the GIL is released during blocking I/O, so threads do overlap waiting. Real CPU parallelism comes from processes, each with its own interpreter — which you prove for yourself by watching thread tasks all report one process ID while process tasks each report their own.
From there it is machinery: concurrent.futures gives a uniform submit/future
API across thread and process pools, with the pickling rule that trips everyone
up on ProcessPoolExecutor; queue.Queue moves work between threads without a
hand-written lock. Then the sharp edges: a read-modify-write is not atomic, so
concurrent updates lose data — you reproduce that race and fix it with a Lock,
then cap concurrency with a Semaphore. Every practice rung runs on the
kernel, so the threads and processes are real, not simulated.