Skip to content

Add Transformer Engine compatibility shim, fallbacks, and modernize TE pins - #2

Open
LilBadger wants to merge 1 commit into
codex/add-te-compatibility-layer-and-updatesfrom
codex/add-te-compatibility-for-rtx-6000-pro
Open

Add Transformer Engine compatibility shim, fallbacks, and modernize TE pins#2
LilBadger wants to merge 1 commit into
codex/add-te-compatibility-layer-and-updatesfrom
codex/add-te-compatibility-for-rtx-6000-pro

Conversation

@LilBadger

Copy link
Copy Markdown
Owner

Motivation

  • Ensure the repo runs on Blackwell GPUs (CUDA 12.8 / cuDNN 9.x) without crashing when private Transformer Engine (TE) internals have moved or been removed.
  • Provide stable, public-API-first replacements for TE features (RMSNorm, RoPE, attention helpers) and fall back to pure-PyTorch implementations when TE is absent.
  • Relax strict TE version pins so modern TE releases (e.g. 2.11.x and later minor versions) can be used without API breakage.

Description

  • Added a compatibility layer at cosmos_predict1/utils/te_compat.py that exposes RMSNorm, apply_rotary_pos_emb, check_set_window_size, split_along_dim, DotProductAttention (if available), and InferenceParams fallbacks; it tries public TE APIs first and falls back to pure-PyTorch implementations when needed.
  • Removed reliance on TE private symbol _RMSNorm by switching to the compat RMSNorm in cosmos_predict1/autoregressive/utils/parallel.py and using a safe forward path when TE internals are not present.
  • Updated cosmos_predict1/diffusion/module/attention.py to use the compat RMSNorm and the existing compat apply_rotary_pos_emb so modules no longer import TE internals directly.
  • Relaxed TE pins in requirements.txt to transformer-engine>=2.11,<3 and transformer-engine-torch>=2.11,<3, and updated scripts/install_blackwell_wsl.sh accordingly.
  • Added a smoke-check script scripts/check_te_compat.py that imports main inference entrypoints and prints TE_AVAILABLE, TE_DotProductAttention_available, and whether the TE fast-path is enabled (script returns WARN for environment/system-lib import issues instead of failing).

Testing

  • Ran the smoke script python scripts/check_te_compat.py; the script executed and reported TE_AVAILABLE=False, TE_DotProductAttention_available=False in this environment and returned compat_smoke=WARN (imports warned due to missing system lib libGL.so.1 in the minimal CI container, not TE symbol breakage).
  • Compiled affected modules with python -m compileall cosmos_predict1/utils/te_compat.py cosmos_predict1/autoregressive/utils/parallel.py cosmos_predict1/diffusion/module/attention.py scripts/check_te_compat.py and compilation succeeded.
  • Performed lightweight runtime check from cosmos_predict1.utils import te_compat; te_compat.RMSNorm(16) which returned the fallback RMSNorm type successfully.

Codex Task

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant