HackMD
Problem
ICON4Py now has it's own config language. However, that config language is not yet documented. We expect relatively frequent changes, therefore manually documenting it is not an option.
A new user with no experience with the codebase will require some way of knowing
- what configuration options they are
- what data can go in them
- which part of the model they affect in what way
- what the syntax is to express data types beyond simple Python builtins
Solution
It should be possible to work backwards from the ExperimentConfiguration class to gather and present the above information; to the extent it is available (not every config option comes with a description of what it does yet).
Some experimentation will be required to find a sufficient (but not hard to build) way to present the information to an unexperienced user. It might be worth exploring how well the presentation format lends itself to being used by LLM agents.
The end result could be a file in the repo (kept up to date via pre-commit). Or a CLI command which generates some sort of view to the user. This could be as simple as an annotated reference configuration or as complex as a TUI, which guides the user through creating a configuration. The rule is: as simple as possible.
If we can generate a good-enough-for-now view we can generate a nicer one later.
If there is extra time it could be used to work out the missing config option descriptions (with a focus on making them understandable to someone in the field but not necessarily with ICON background specifically).
Rabbit holes
- Presentation: keep it simple!
- Option descriptions: do not try to go for self-contained full explanations, rather start simple and use feedback from users in the future to improve
- Option descriptions: adding / improving them is strictly optional
No-gos
Although the information we want to make available to the user is similar to a formal schema of the config language, generating such a schema is explicitly a non-goal.
Progress