← all records

Async distributed output

pitch-9c4e12 · in Replace blueline

  • No No-gos section. The pitch template asks for one — it is what keeps the appetite honest.

Problem

Distributed output landed in cycle 37 (pitch-370007, #1403): every rank can now write, either by gathering to rank 0 or by writing its own block into a shared store. Both paths are synchronous — the model stops while the data goes out.

The shaping note for that bet named this as its "next and final step", and the cycle-38 brainstorming note carries it as "Async Distributed Output (Christos)". It did not reach the betting table, but it is what Christos worked on during the cycle, so it is written here as a bet of its own rather than left as a loose PR.

Appetite

Two weeks. This is the tail of a bet that has already delivered, not a new line of work, and the synchronous path it builds on is merged and unchanged.

Solution

Split the store write in two, as #1430 does. Everything that involves communication or store metadata stays on the calling thread, exactly as the synchronous path has it: the distribution's prepare (the gather and the halo stripping), creating and resizing the arrays in the store, and the broadcast that rank-block mode uses to order them. Only the write itself moves off the calling thread, so the model can carry on while the data goes out.

Rabbit holes

Dedicated output ranks, which is how the cycle-37 note phrased the end state, are a larger change than overlapping the write with computation. #1430 does the latter.

No-gos

Progress

  • #1430 opened 13.08.2026, rebased onto main once #1403 merged; reviewed by msimberg
  • Merge