feat(deps): add opt-in flashinfer-aot group for air-gapped installs - #3251
Open
samsja wants to merge 1 commit into
Open
feat(deps): add opt-in flashinfer-aot group for air-gapped installs#3251samsja wants to merge 1 commit into
samsja wants to merge 1 commit into
Conversation
vLLM 0.26 no longer depends on flashinfer-cubin, so the vLLM bump in 9f0d41e dropped it from the lock file. On air-gapped fleets (no internet on GPU nodes) this broke first runs: vLLM tries to download cubins and JIT-compile kernels at runtime. Add a flashinfer-aot dependency group pinning flashinfer-cubin and flashinfer-jit-cache to the locked flashinfer-python version (0.6.14, cu129 to match the vLLM wheel). A dependency group rather than an extra so the default `uv sync --all-extras` setup doesn't pull ~2GB of kernel artifacts; opt in with `--group flashinfer-aot`. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
samsja
marked this pull request as ready for review
August 12, 2026 18:57
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
The vLLM 0.26 bump (9f0d41e, #3166) dropped
flashinfer-cubinfrom the lock file because vLLM no longer depends on it. For air-gapped deployments (GPU fleet with no internet access), this broke first runs: vLLM now tries to download cubins and JIT-compile kernels at runtime, forcing users to trial-and-error rebuild their venv with the missing packages (Slack thread).Change
Add a
flashinfer-aotdependency group (following the existingmamba-ssmpattern) with the pre-built FlashInfer kernel artifacts:flashinfer-cubin==0.6.14— pre-downloaded cubinsflashinfer-jit-cache==0.6.14+cu129— pre-built JIT cache, so nothing compiles on first runOpt in with:
Notes
uv sync --all-extrassetup doesn't pull ~2GB of kernel wheels by default — the thread explicitly asked for this to stay opt-in.flashinfer-python(0.6.14) — mismatched artifact packages are ignored at runtime and vLLM falls back to downloading. Pins need re-checking on vLLM bumps (comment added inpyproject.toml).flashinfer-jit-cachenever is — it's published per-CUDA-version, cu129 here to match the vLLM wheel). Both x86_64 and aarch64 are covered.uv lock --checkpasses anduv export --group flashinfer-aotresolves the two wheels on both arches.🤖 Generated with Claude Code
Note
Low Risk
Dependency and documentation only; no runtime code paths change, though version pins must stay in sync with vLLM/flashinfer-python on future bumps.
Overview
Adds an opt-in
flashinfer-aotuv dependency group so offline fleets can install pre-built FlashInfer artifacts (flashinfer-cubinandflashinfer-jit-cacheat 0.6.14, aligned with lockedflashinfer-python) without pulling ~2GB into the defaultuv sync --all-extrasflow.Wiring includes GitHub-release wheel URLs for cubin and cu129 jit-cache on x86_64 and aarch64,
tool.uv.sources/ exclude-newer entries for those packages, and lockfile updates. README documents opting in viauv sync --all-extras --group flashinfer-aotand why it matters when vLLM would otherwise download cubins or JIT-compile on first run.Reviewed by Cursor Bugbot for commit ff76ecf. Bugbot is set up for automated code reviews on this repo. Configure here.