Description of the problem
Currently, GatorGrade loads YAML configuration files using yaml.safe_load, yaml.load_all, which returns the data as an Any type. This creates significant issues (in this issue I focus on type checkers related issue):
- Because the data is treated as
Any, type checkers (like Mypy or Pyrefly) cannot verify the logic inside the processing functions.
- Type-related bugs remain hidden during development because the type checker "trusts" the
Any variable, even when the logic is actually unsafe.
- Currently it is not possible to check some of the files to detect type related errors because of the above explained issues.
Reproduced Issue
I have created a yaml_type_safety_test.yml file. It uses a list where the program expects a single string.
- description: ["This", "is", "a", "list", "not", "a", "string"]
check: ConfirmFileExists
options:
path: "README.md"
I have used the following code to test this file: uv run gatorgrade --config yaml_type_safety_test.yml
Result
╭───────────────────── Traceback (most recent call last) ──────────────────────╮
│ /home/benedek-kaibas/Desktop/gatorgrade/gatorgrade/main.py:66 in gatorgrade │
│ │
│ 63 │ # that, by default, gatorgrade should run in checking mode │
│ 64 │ if ctx.invoked_subcommand is None: │
│ 65 │ │ # parse the provided configuration file │
│ ❱ 66 │ │ checks = parse_config(filename) │
│ 67 │ │ # there are valid checks and thus the │
│ 68 │ │ # tool should run them with run_checks │
│ 69 │ │ if len(checks) > 0: │
│ │
│ ╭──────────────────────────── locals ────────────────────────────╮ │
│ │ ctx = <click.core.Context object at 0x7838e620cbc0> │ │
│ │ filename = PosixPath('yaml_type_safety_test.yml') │ │
│ │ no_status_bar = False │ │
│ │ report = (None, None, None) │ │
│ │ run_status_bar = False │ │
│ ╰────────────────────────────────────────────────────────────────╯ │
│ │
│ /home/benedek-kaibas/Desktop/gatorgrade/gatorgrade/input/parse_config.py:27 │
│ in parse_config │
│ │
│ 24 │ │ # use it to generate all of the checks; │
│ 25 │ │ # these will be valid checks that are now │
│ 26 │ │ # ready for execution with this tool │
│ ❱ 27 │ │ parse_con = generate_checks(reformat_yaml_data(parsed_yaml_file │
│ 28 │ │ return parse_con │
│ 29 │ # return an empty list because of the fact that the │
│ 30 │ # parsing process did not return a list with content; │
│ │
│ ╭──────────────────────────── locals ─────────────────────────────╮ │
│ │ file = PosixPath('yaml_type_safety_test.yml') │ │
│ │ parsed_yaml_file = [ │ │
│ │ │ [ │ │
│ │ │ │ { │ │
│ │ │ │ │ 'description': [ │ │
│ │ │ │ │ │ 'This', │ │
│ │ │ │ │ │ 'is', │ │
│ │ │ │ │ │ 'a', │ │
│ │ │ │ │ │ 'list', │ │
│ │ │ │ │ │ 'not', │ │
│ │ │ │ │ │ 'a', │ │
│ │ │ │ │ │ 'string' │ │
│ │ │ │ │ ], │ │
│ │ │ │ │ 'check': 'ConfirmFileExists', │ │
│ │ │ │ │ 'options': { │ │
│ │ │ │ │ │ 'path': 'README.md' │ │
│ │ │ │ │ } │ │
│ │ │ │ } │ │
│ │ │ ] │ │
│ │ ] │ │
│ ╰─────────────────────────────────────────────────────────────────╯ │
│ │
│ /home/benedek-kaibas/Desktop/gatorgrade/gatorgrade/input/in_file_path.py:44 │
│ in reformat_yaml_data │
│ │
│ 41 │ if len(data) == 2: │
│ 42 │ │ setup_commands = data.pop(0) # Removes the setup commands │
│ 43 │ │ run_setup(setup_commands) │
│ ❱ 44 │ add_checks_to_list(None, data[0], reformatted_data) │
│ 45 │ return reformatted_data │
│ 46 │
│ 47 │
│ │
│ ╭──────────────────────────── locals ─────────────────────────────╮ │
│ │ data = [ │ │
│ │ │ [ │ │
│ │ │ │ { │ │
│ │ │ │ │ 'description': [ │ │
│ │ │ │ │ │ 'This', │ │
│ │ │ │ │ │ 'is', │ │
│ │ │ │ │ │ 'a', │ │
│ │ │ │ │ │ 'list', │ │
│ │ │ │ │ │ 'not', │ │
│ │ │ │ │ │ 'a', │ │
│ │ │ │ │ │ 'string' │ │
│ │ │ │ │ ], │ │
│ │ │ │ │ 'check': 'ConfirmFileExists', │ │
│ │ │ │ │ 'options': { │ │
│ │ │ │ │ │ 'path': 'README.md' │ │
│ │ │ │ │ } │ │
│ │ │ │ } │ │
│ │ │ ] │ │
│ │ ] │ │
│ │ reformatted_data = [] │ │
│ ╰─────────────────────────────────────────────────────────────────╯ │
│ │
│ /home/benedek-kaibas/Desktop/gatorgrade/gatorgrade/input/in_file_path.py:62 │
│ in add_checks_to_list │
│ │
│ 59 │ │ │ │ │ path = item │
│ 60 │ │ │ │ else: │
│ 61 │ │ │ │ │ path = f"{path}/{item}" │
│ ❱ 62 │ │ │ │ add_checks_to_list( │
│ 63 │ │ │ │ │ path, ddict[item], reformatted_data │
│ 64 │ │ │ │ ) # Runs this same function on the list inside of a di │
│ 65 │ │ │ │ path = current_path │
│ │
│ ╭────────────────────────── locals ───────────────────────────╮ │
│ │ current_path = None │ │
│ │ data_list = [ │ │
│ │ │ { │ │
│ │ │ │ 'description': [ │ │
│ │ │ │ │ 'This', │ │
│ │ │ │ │ 'is', │ │
│ │ │ │ │ 'a', │ │
│ │ │ │ │ 'list', │ │
│ │ │ │ │ 'not', │ │
│ │ │ │ │ 'a', │ │
│ │ │ │ │ 'string' │ │
│ │ │ │ ], │ │
│ │ │ │ 'check': 'ConfirmFileExists', │ │
│ │ │ │ 'options': { │ │
│ │ │ │ │ 'path': 'README.md' │ │
│ │ │ │ } │ │
│ │ │ } │ │
│ │ ] │ │
│ │ ddict = { │ │
│ │ │ 'description': [ │ │
│ │ │ │ 'This', │ │
│ │ │ │ 'is', │ │
│ │ │ │ 'a', │ │
│ │ │ │ 'list', │ │
│ │ │ │ 'not', │ │
│ │ │ │ 'a', │ │
│ │ │ │ 'string' │ │
│ │ │ ], │ │
│ │ │ 'check': 'ConfirmFileExists', │ │
│ │ │ 'options': {'path': 'README.md'} │ │
│ │ } │ │
│ │ item = 'description' │ │
│ │ path = 'description' │ │
│ │ reformatted_data = [] │ │
│ ╰─────────────────────────────────────────────────────────────╯ │
│ │
│ /home/benedek-kaibas/Desktop/gatorgrade/gatorgrade/input/in_file_path.py:56 │
│ in add_checks_to_list │
│ │
│ 53 │ for ddict in data_list: │
│ 54 │ │ for item in ddict: │
│ 55 │ │ │ if isinstance( │
│ ❱ 56 │ │ │ │ ddict[item], list │
│ 57 │ │ │ ): # Checks if the current dictionary has another list as │
│ 58 │ │ │ │ if not path: │
│ 59 │ │ │ │ │ path = item │
│ │
│ ╭─────────────────────────────── locals ───────────────────────────────╮ │
│ │ current_path = 'description' │ │
│ │ data_list = ['This', 'is', 'a', 'list', 'not', 'a', 'string'] │ │
│ │ ddict = 'This' │ │
│ │ item = 'T' │ │
│ │ path = 'description' │ │
│ │ reformatted_data = [] │ │
│ ╰──────────────────────────────────────────────────────────────────────╯ │
╰──────────────────────────────────────────────────────────────────────────────╯
TypeError: string indices must be integers, not 'str'
When I ran the command, the parse_config.py file will load the YAML configuration. Because the description is a list [...] instead of a string "...", any part of the code that tries to print that description or manipulate it (like calling .strip() or .lower()) will cause a Runtime Crash. The "bad data" passes through main.py and parse_config.py without any warnings. It only crashes when it reaches the recursive logic in in_file_path.py.
This is a problem that would need to be fixed because the code in in_file_path.py had no idea that description was a list instead of a string. It just kept running until it hit a piece of logic it couldn't perform.
The "bad data" went through multiple files before it exploded:
- 1:
main.py
- 2:
parse_config.py
- 3:
in_file_path.py It only exploded here
Proposed Goal
We need to remove the Any loophole. By implementing a validation layer (like Pydantic models), we can:
- Stop the crash at Step 2 (parse_config.py) with a helpful error message.
- Enable Type Checkers to verify the logic in
in_file_path.py by providing explicit type hints instead of Any.
- Modify other files just like the
in_file_path.py file, so it is possible to run type checkers on them during the development process and detect type related errors.
Description of the problem
Currently,
GatorGradeloadsYAMLconfiguration files usingyaml.safe_load,yaml.load_all, which returns the data as an Any type. This creates significant issues (in this issue I focus on type checkers related issue):Any, type checkers (likeMypyorPyrefly) cannot verify the logic inside the processing functions.Anyvariable, even when the logic is actually unsafe.Reproduced Issue
I have created a
yaml_type_safety_test.ymlfile. It uses a list where the program expects a single string.I have used the following code to test this file:
uv run gatorgrade --config yaml_type_safety_test.ymlResult
When I ran the command, the
parse_config.pyfile will load theYAMLconfiguration. Because thedescriptionis a list[...]instead of a string"...", any part of the code that tries to print that description or manipulate it (like calling.strip()or.lower()) will cause a Runtime Crash. The "bad data" passes throughmain.pyandparse_config.pywithout any warnings. It only crashes when it reaches the recursive logic inin_file_path.py.This is a problem that would need to be fixed because the code in
in_file_path.pyhad no idea that description was a list instead of a string. It just kept running until it hit a piece of logic it couldn't perform.The "bad data" went through multiple files before it exploded:
main.pyparse_config.pyin_file_path.pyIt only exploded hereProposed Goal
We need to remove the
Anyloophole. By implementing a validation layer (likePydanticmodels), we can:in_file_path.pyby providing explicit type hints instead ofAny.in_file_path.pyfile, so it is possible to run type checkers on them during the development process and detect type related errors.