FEXOfflineCompiler exits silently on Windows: null MetaLayer deref in main()

[tompollok]

FEXOfflineCompiler's Windows startup path faults on the first statement of
main(), exiting without printing anything — not even its own usage text.

Upfront about what I have and have not tested: I observed this under
FEX/ARM64EC on macOS via Wine, running the Windows PE build — not on real
Windows. The faulting code is in the Windows-only branch and the defect is
visible by inspection, which is why the case below is built from source rather
than from my repro. I cannot claim to have watched it fail on Windows itself.

This is a null dereference by inspection, so it can be confirmed from the source
without reproducing anything:

• main() calls FEX::Windows::Logging::Init() as its first statement
  (Main.cpp:850).
• Init() immediately reads SILENTLOG
  (Logging.cpp:36).
• That read reaches FEXCore::Config::GetConv, which dereferences Meta
  unconditionally
  (Config.cpp:433).
• Meta is a null-initialised static
  (:116)
  assigned only in Initialize()
  (:239),
  which has not run yet.
• The fault materialises one frame further in, at MetaLayer::GetConv's first
  statement — OptionMap.find(Option)
  (:140) — a member access
  through a null this.

Nothing loads config before that point. FEX::Config::LoadConfig is first reached
inside GenerateCache
(Main.cpp:615),
well after Logging::Init(), and ProcessAll() never calls it at all. Since the
fault is in main() itself, neither subcommand is reachable — including the
generate child that ProcessAll() would otherwise spawn on Windows
(Main.cpp:834),
which would hit the same fault at startup. Fixing main() covers both.

Both other Windows entry points already order this correctly:

Entry point	Order of calls
ARM64EC/Module.cpp:585-587	LoadConfig → ReloadMetaLayer → Logging::Init
WOW64/Module.cpp:519-521	same
FEXOfflineCompiler/Main.cpp:850	Logging::Init only

Adding the same LoadConfig + ReloadMetaLayer pair before Logging::Init()
fixes it.

Note the call sits under a plain #ifndef _WIN32 / #else, so this affects every
Windows build, not only ARM64EC.

What I actually ran, and what I could not

Being precise about this, because it limits what I can claim.

I could not run the binary under its installed name. On my setup any PE with a
filename of 20 characters or more faults before entry — a separate, macOS-side
problem I have not root-caused and am not reporting here.
FEXOfflineCompiler64.exe is 24 characters, so it never starts for me for a
reason that has nothing to do with this bug. What I ran was the same binary
renamed:

fexofc.exe          # 10 chars, so the name-length problem does not apply

• Expected: the usage block, and exit 1.
• Actual: exits immediately with no output at all.

Any argument behaves the same; the fault is before argument parsing. Debugging
that silent exit is what produced the analysis above — the fault is in
MetaLayer::GetConv<bool>, reached from Logging::Init().

Why that produces no output at all here, rather than an unhandled-exception
message, I have not established; it is not part of the claim above.

The honest limitation: at the stock name I cannot reach this fault at all —
the name-length problem fires first, before entry — so the configuration you
actually ship is untestable on my machine, and the short name is what makes this
one reachable. The argument above therefore rests on source inspection rather than
on my repro: the ordering in main() is wrong regardless of filename, and that is
checkable in your tree without running anything.

If someone with Windows-on-ARM hardware runs FEXOfflineCompiler64.exe with no
arguments, that settles it in ten seconds.

Secondary: size_t underflow in the sibling-binary path

Unrelated to the above, and latent rather than blocking, but it is a one-line
fragility in the same file
(Main.cpp:789):

auto NewExecName = fmt::format("FEXOfflineCompiler{}.exe", ExecutableIt->second.ExecutableBitness.value());
SelfPath.replace(SelfPath.size() - NewExecName.size(), NewExecName.size(), NewExecName);

NewExecName is 24 characters. If SelfPath is shorter, the subtraction
underflows and replace throws std::out_of_range, after the code maps have
already been aggregated. It also assumes the binary is always invoked as
FEXOfflineCompilerNN.exe; under any other name the replacement silently corrupts
the path instead of pointing at a sibling. Computing the last path separator and
replacing only the filename component avoids both.

How I hit it. With the config-ordering fix above applied locally — without it
the tool never starts, so this is unreachable — and the binary renamed to
fexofc.exe for the platform reason above, process-all aggregated the code maps
(Main.cpp:765)
and then threw std::out_of_range on the first executable entry it found, at
Main.cpp:789:
SelfPath was C:\fexofc.exe, shorter than the 24-character name the code assumes
it still has. That is before it prints Checking caches for executable
(:792),
and well before the _spawnv the path is being built for
(:834).

To be clear on the order: the rename works around a quirk of my platform, not the
config-ordering defect — that one needed the source change above. What the rename
does is expose this second defect. Renaming a binary is an ordinary thing to do,
and I would not expect the tool to assume its own installed filename — which is the
actual fix here, independent of my platform.

At the stock name and install path this will not fire, so it is latent upstream
rather than blocking. Raising it as a fragility worth hardening, not as something
that breaks a supported configuration.

Notes

• I used an AI assistant (Claude Code) during this investigation. Per
  CONTRIBUTING.md ("No AI/ML/LLM/etc code contributions") I am therefore not
  opening a PR — I would rather you write anything that lands in the tree.
• What I changed locally is here for checking, not for merging:
  tompollok/FEX@7fb37b1, a
  single commit touching only Main.cpp on top of
  9686454.
• Runs were on 9686454 plus two local commits: macOS build shims for the ARM64EC
  unix library (unrelated to this report) and the ordering change linked above.
  ARM64EC, macOS 26.6 (Apple M3 Max), Wine 11.14, llvm-mingw ARM64EC toolchain.
  The analysis itself is against stock 9686454.

Happy to test a patch and report back.