Skip to content

Adding a lrpar::codegen module. - #656

Draft
ratmice wants to merge 19 commits into
softdevteam:masterfrom
ratmice:lrpar_codegen2
Draft

Adding a lrpar::codegen module.#656
ratmice wants to merge 19 commits into
softdevteam:masterfrom
ratmice:lrpar_codegen2

Conversation

@ratmice

@ratmice ratmice commented Aug 6, 2026

Copy link
Copy Markdown
Collaborator

This the beginning of attempt No. 2 at pr #655 just trying to split up the giant commit from that patch into smaller more easily reviewable ones. Below is the original pr description:

Here is an attempt at pulling a codegen module out of lrpar, it migrates the CTParserBuilder to use it, and passes
the testsuite. I haven't gone over this with a fine toothed comb, there have been some obscure timing related problems I introduced during development that didn't cause any testsuite failures.

(Like calling check_unused_header_keys() too early before the callback.

I wasn't able to do this patch in a way that was even remotely incremental because of ownership issues. Nor really figure out a way to do it in a way that didn't require making slight changes to things as they moved over (mostly changing references to self, removing calls unwrap()).

There are 3 passes to this and a 4th structure ParserBuildEnvArgs:

  1. ParserSrcEnv
  2. ParserBuildEnv
  3. ParserCodeGen

The general idea is:

ParserSrcEnv: source, source path, diagnostics generator, header collection which contains the default values for the %grmtools section.
ParserBuildEnvArgs, these are essentially the builder arguments including the essential Option<> types. It's just a minimal builder.
ParserBuildEnv: This structure contains fields derived from the BuildEnvArgs and also the inner types from options in BuildEnvArgs. By derived fields I mean things like ASTWithValidationInfo
ParserCodegen: This one contains a YaccGrammar, StateTable, and StateGraph. In order to generate code, though it still needs the SrcEnv and the BuildEnv.

Currently the BuildEnv takes ownership of the BuildEnvArgs, it was easiest this way because the rebuild_cache code expects Options it may be we should drop BuildEnvArgs from BuildEnv.

But generally the idea (in a pseudo functional syntax) is something to the effect of:

(SrcEnv?, BuildEnvArgs) -> BuildEnv? -> CodeGen::generate(&build_env, &src_env)?;

There is a pretty high probability that this contains some code duplicated from CTParserBuilder, which
didn't go fully unused in CTParserBuilder, given that the diff +/- is only a few hundred lines, I don't expect an enormous amount. But perhaps if it is really used in both places like fn indent appears to be we can use ctbuilder::indent instead.
I'll try and read through it in this regard tomorrow.

@ratmice

ratmice commented Aug 6, 2026

Copy link
Copy Markdown
Collaborator Author

A couple of oopsies where I copied some code but didn't delete it, or still had things refer to self when they could/should refer to build_env. But now that all the codegen doesn't have access to self that shouldn't be possible.

At least I think that is all of it in that the use items for quote, syn, proc_macro2, used by codegen.

Edit: Anyhow I'm not sure if that helps making it easier to review or not, it's still a lot.

Comment thread lrpar/src/lib/ctbuilder.rs Outdated
let grm = code_gen.grm();
let mod_name = build_env.mod_name();
let visibility = self.visibility.clone();
let visibility = build_env.visibility();

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably would be better if I squashed all these little changes to output_file into one patch.

Comment thread lrpar/src/lib/codegen.rs
}
}

fn extract_ast_validation(

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think rather than extract_ast_validation this should be something like derive_ast_validation.
For the other extract functions, I wonder if they would be better named resolve_*?

Comment thread lrpar/src/lib/codegen.rs
fn extract_ast_validation(
&mut self,
from_ast: Option<&ASTWithValidityInfo>,
) -> Result<ASTWithValidityInfo, Box<dyn Error>> {

@ratmice ratmice Aug 6, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With my nimbleparse_lsp hat on for that project it wants the raw errors,
so it can send spans directly to the editor for highlighting rather than Box<dyn Error>.

Probably something to think about later

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants