Problem
pitch-370008 was bet at one week and closed on the NVIDIA side with #1446, the
thread block configuration for GH200. The AMD side did not fit in that week and
is still open.
What was measured in cycle 37: running the JW experiment on one node with
R02B07 on MI300 gave results about three times worse than the ones measured a
month earlier for PASC. Nobody has yet established whether that is a problem with
JW and communication, or whether the kernels really are that much slower.
Appetite
Two weeks. Ioannis carries no figure in the cycle-38 roster and already holds
task-802688, so this is deliberately smaller than the work that could be done
here.
Solution
- Settle the 3x regression on MI300: JW and communication, or the kernels.
- Check the optimal grid and thread block configuration for GCDs on AMD GPUs in
DaCe, the
chiplet_old_codegen branch the PASC measurements were taken with.
- Extend the block size heuristic across the whole dycore, dumping the
independent and dependent bytes, and then to the
icon_benchmark_global grid.
Rabbit holes
The heuristic tuned in cycle 37 was fitted to two stencils,
vertically_implicit_solver_at_predictor_step and
compute_rho_theta_pgrad_and_update_vn. It gave about 5% on GH200 across the
whole JW experiment, which is to say it may simply not generalise. Extending it
is worth a bounded attempt, not an open-ended one.
No-gos
The GT4Py-feature refactoring of the programs themselves. That is the Audit bet,
whose own "additional performance optimizations" section overlaps this one on the
A100 sweep and on limiting Map fusion. Coordinate rather than duplicate.
Progress