Skip to content

Refactor recipe index caching and improve mixin reflection handling - #6

Open
cryolithic wants to merge 4 commits into
Lumengrid:mainfrom
cryolithic:main
Open

Refactor recipe index caching and improve mixin reflection handling#6
cryolithic wants to merge 4 commits into
Lumengrid:mainfrom
cryolithic:main

Conversation

@cryolithic

Copy link
Copy Markdown

Running this mod in All The Mods 10 with some large recursions was causing me to timeout at login. Made a large fix performance wise, while cleaning up a few other things with claude. Tested on my personal server.

Performance
===========

Measured on an All the Mods 10 server: 44,798 crafting recipes, 11 pattern
providers and assembler matrices, ~23,100 generated dependency patterns per
full round.

The old path traversed the whole recipe list twice over. getAllRecipesFor is
List.copyOf(byType), so every recursion copied all 44,798 entries, and every
distinct input item then drove another full linear scan with a getResultItem
call per entry. A complete round across the network works out to roughly
23,100 full copies plus ~58,000 full scans, on the order of 3.6 billion recipe
visits.

Version 1.0.7

[09Aug2026 11:02:58.739] Generated 61 dependency patterns for Assembler Matrix Pattern
[09Aug2026 11:03:02.071] Generated 364 dependency patterns for Assembler Matrix Pattern
[09Aug2026 11:03:50.396] Generated 2614 dependency patterns for Assembler Matrix Pattern
[09Aug2026 11:03:50.774] Generated 43 dependency patterns for Assembler Matrix Pattern

Four providers spanning ~52 seconds; the 2,614-pattern matrix alone took ~48.
Extrapolated to all 11 providers, a full round is six to seven minutes of
server-thread time, repeated on every pattern insertion or removal.

1.0.8 changes

[10Aug2026 11:36:52.822] Indexed 44798 crafting recipes by output item in 37 ms
[10Aug2026 11:36:52.935] Generated 3231 dependency patterns for Assembler Matrix Pattern
[10Aug2026 11:36:52.987] Generated 61 dependency patterns for Assembler Matrix Pattern
[10Aug2026 11:36:52.998] Generated 364 dependency patterns for Assembler Matrix Pattern
[10Aug2026 11:36:53.077] Generated 2751 dependency patterns for Assembler Matrix Pattern
[10Aug2026 11:36:53.097] Generated 43 dependency patterns for Assembler Matrix Pattern
[10Aug2026 11:36:53.199] Generated 2711 dependency patterns for Assembler Matrix Pattern
[10Aug2026 11:36:53.291] Generated 3083 dependency patterns for Assembler Matrix Pattern
[10Aug2026 11:36:53.376] Generated 2530 dependency patterns for Assembler Matrix Pattern
[10Aug2026 11:36:53.450] Generated 2581 dependency patterns for Assembler Matrix Pattern
[10Aug2026 11:36:53.615] Generated 2649 dependency patterns for Pattern Provider

Ten providers and 20,004 patterns in 793 ms end to end, including the one-time
37 ms index build. Like for like, the 2,614-pattern matrix at ~48 s becomes a
2,751-pattern matrix at 79 ms, about 600x. The index is built exactly once per
server session and shared by every provider, so the margin widens as more
providers are added rather than shrinking.

cryolithic and others added 4 commits August 10, 2026 05:03
Extract the duplicated recursive-pattern generation from the three mixins
into RecursivePatternGenerator, backed by a shared output-item -> recipes
index built once per recipe manager, and fix the defects a code review
found in that new cache.

RecursivePatternGenerator:
- Hold the RecipeManager through a WeakReference inside a single volatile
  CachedIndex record. The manager and index were previously separate static
  fields published in two writes, so a concurrent reader could pass the
  identity check and still be handed the pre-reload index. The strong static
  reference also pinned an unloaded world's whole recipe and registry graph.
- Validate the cache on holder identity rather than emptiness, so a
  legitimately empty index counts as a hit instead of forcing a full rescan
  and an INFO log on every call.
- Fingerprint the cache with RecipeManager.getRecipes().size() so in-place
  recipe replacement (KubeJS, CraftTweaker) invalidates it. getAllRecipesFor
  copies the whole list per call and is unsuitable on the fast path. A swap
  that leaves the total count unchanged is still not detected.
- Key processedItems on AEItemKey instead of Item.toString(). Recipe matching
  uses isSameItemSameComponents, so keying dedup on the Item alone made a
  second component-variant get skipped and never receive a dependency pattern.
- Skip generation on the client, so the client and server levels no longer
  alternate through the single cache slot and force a full re-index.

Mixins:
- Replace the cached Field/Method reflection in both provider mixins with
  @shadow members and a @shadow getPatternInv(). The lazy init guard only
  null-checked the first handle, so a failure resolving a later member left
  it permanently null and every later call reported a misleading NPE in place
  of the real lookup failure.
- Share the pattern collection and append loops through
  collectRecursivePatterns() and appendGenerated().

Performance
===========

Measured on an All the Mods 10 server: 44,798 crafting recipes, 11 pattern
providers and assembler matrices, ~23,100 generated dependency patterns per
full round.

The old path traversed the whole recipe list twice over. getAllRecipesFor is
List.copyOf(byType), so every recursion copied all 44,798 entries, and every
distinct input item then drove another full linear scan with a getResultItem
call per entry. A complete round across the network works out to roughly
23,100 full copies plus ~58,000 full scans, on the order of 3.6 billion recipe
visits.

Version 1.0.7

[09Aug2026 11:02:58.739] Generated 61 dependency patterns for Assembler Matrix Pattern
[09Aug2026 11:03:02.071] Generated 364 dependency patterns for Assembler Matrix Pattern
[09Aug2026 11:03:50.396] Generated 2614 dependency patterns for Assembler Matrix Pattern
[09Aug2026 11:03:50.774] Generated 43 dependency patterns for Assembler Matrix Pattern

Four providers spanning ~52 seconds; the 2,614-pattern matrix alone took ~48.
Extrapolated to all 11 providers, a full round is six to seven minutes of
server-thread time, repeated on every pattern insertion or removal.

Initial 1.0.8 changes

[10Aug2026 10:15:51.621] Generated 61 dependency patterns for Assembler Matrix Pattern
[10Aug2026 10:15:51.638] Generated 364 dependency patterns for Assembler Matrix Pattern
[10Aug2026 10:15:51.769] Generated 2721 dependency patterns for Assembler Matrix Pattern
[10Aug2026 10:15:51.805] Generated 43 dependency patterns for Assembler Matrix Pattern

Second round of 1.0.8 changes

[10Aug2026 11:36:52.822] Indexed 44798 crafting recipes by output item in 37 ms
[10Aug2026 11:36:52.935] Generated 3231 dependency patterns for Assembler Matrix Pattern
[10Aug2026 11:36:52.987] Generated 61 dependency patterns for Assembler Matrix Pattern
[10Aug2026 11:36:52.998] Generated 364 dependency patterns for Assembler Matrix Pattern
[10Aug2026 11:36:53.077] Generated 2751 dependency patterns for Assembler Matrix Pattern
[10Aug2026 11:36:53.097] Generated 43 dependency patterns for Assembler Matrix Pattern
[10Aug2026 11:36:53.199] Generated 2711 dependency patterns for Assembler Matrix Pattern
[10Aug2026 11:36:53.291] Generated 3083 dependency patterns for Assembler Matrix Pattern
[10Aug2026 11:36:53.376] Generated 2530 dependency patterns for Assembler Matrix Pattern
[10Aug2026 11:36:53.450] Generated 2581 dependency patterns for Assembler Matrix Pattern
[10Aug2026 11:36:53.615] Generated 2649 dependency patterns for Pattern Provider

Ten providers and 20,004 patterns in 793 ms end to end, including the one-time
37 ms index build. Like for like, the 2,614-pattern matrix at ~48 s becomes a
2,751-pattern matrix at 79 ms, about 600x. The index is built exactly once per
server session and shared by every provider, so the margin widens as more
providers are added rather than shrinking.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Enhance the build workflow to upload the compiled mod jar as a
downloadable artifact on every push, and add a release workflow that
attaches the jar to a GitHub Release (triggered by a v* tag or manually).
Document both download paths in the README.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JBirm2uhKDDYp2CHukTVrC
…wobi

ci: publish downloadable builds and releases
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants