¶Life of an mdtest
Red Knot uses mdtests extensively to test the type checker.
mdtests README [astral-sh/ruff]
The red_knot_test crate is the harness that parses a Markdown file, extracts all of test cases in the file, and runs them.
The red_knot_python_semantic crate has an mdtest test suite that uses this harness to run all of the tests in every markdown file in the resources/mdtest directory. (It uses the dir-test crate to handle the “run a Rust test for each file in a directory” part.)
¶High-level steps
For each file in the test:
- Parse the file via the ruff_db and ruff_python_parser crates, returning a ruff_python_ast::ModModule, which corresponds to the Module variant of the mod rule in the Python abstract grammar.
- Check the types of everything in the module via the red_knot_python_semantic::check_types function.
- The above two steps produce a list of diagnostics, which are the warning and errors messages that would be printed out by the CLI type checker. If there are any assertion comments in the test file, use the red_knot_test::matcher module to verify that the diagnostics match the assertions.
¶Checking types
To “check the types” of a module:
- Generate a “semantic index” for the file, which is a collection of all of the scopes in the file, along with some useful information about each one. (Primarily a symbol table and a use/def map)
- For each scope in the file, infer the types of everything in the scope. This checks the types along the way, generating diagnostics for any warnings or errors.
Note that we don't pass the parsed representation of the file into check_types. The semantic_index function calls parsed_module again, relying on salsa caching to return the already parsed AST.