Automatic tolerances inspection

task-370011 · in quality_of_life

Status: Thinking — at the foot of the hill Shaping — halfway up the hill Ready — at the top of the hill In progress — halfway down the hill Done — at the bottom, over the hill Shelved — off the hill A decision, not a failure: we looked, and we are not doing this.
Priority: Medium
Appetite: — A budget in person-weeks, not an estimate: what this is worth, not how long it takes.
Tags: icon4py, tolerances, testing
Owner: jcanton
Assignees: jcanton
Reviewers: nfarabullini
Review waived: —
Parent: quality_of_life
Blocked by: —
PRs: C2SM/icon4py#1357
Cycle: 37 A bet is for one cycle. Work that overruns is re-bet, not extended by default.
Start date: 01.07.2026
End date: —
Scheduled: —
Blocks: —

Reported at the cycle 37 review; not bet at the betting table, so it has no shaping document and no appetite. It is filed under cycle 37 because that is the review it was presented at, not because the work happened there: #1357 was opened on 01.07.2026 and its last commit is 08.07.2026, so all of it predates the cycle opening on 14.07.

Datatest tolerances are set when a test is written and then forgotten about. A tolerance nobody revisits hides the bugs that appear after it was set: measure the diffs weekly and report when a tolerance could be tightened, so it goes on meaning something.

Since the review

Shelved. The RBF pilot is written and nfarabullini reviewed it, but nothing has moved on #1357 since 08.07.2026 and nobody has been bet on picking it up. The PR stays open as the start whoever returns to this will build from.