Resolve_issues_Task 2: Digit Recognition

[NassiraNguadii]

Guide to Resolve Setup Issues for Task 2: Digit Recognition

To help others avoid common setup issues for Task 2, here’s a streamlined guide to clone repositories, install dependencies, and run the training script successfully.

1. Clone Required Repositories

git clone https://github.com/groundlight/r1_vlm.git
git clone --branch release_2025_03_06 --single-branch https://github.com/groundlight/verifiers.git
git clone --branch release_2025_03_06 --single-branch https://github.com/groundlight/trl.git

2. Set Up Virtual Environment
pip install uv
uv venv
source .venv/bin/activate  # Linux/Mac
# or .venv\Scripts\activate  # Windows

3. Install Dependencies
echo "Installing dependencies..."
uv pip install --system torch==2.5.1 setuptools wheel
uv pip install --system flash-attn==2.7.3 --no-build-isolation || echo "Flash Attention failed, continuing..."

# Adjust Python version requirements
sed -i 's/requires-python = ">=3.11"/requires-python = ">=3.10"/' verifiers/pyproject.toml
sed -i 's/requires-python = ">=3.12"/requires-python = ">=3.10"/' r1_vlm/pyproject.toml

# Install TRL
cd trl
uv pip install --system -e . --no-build-isolation
cd ..

# Install Verifiers
cd verifiers
uv pip install --system -e . --no-build-isolation
cd ..

# Install additional dependencies
uv pip install --system hatchling editables

# Install r1_vlm
cd r1_vlm
uv pip install --system -e . --no-build-isolation
cd ..

echo "Installation complete!"

4. Update train.py Parameters
Optimize GPU memory usage and training settings:
sed -i 's/vllm_device="cuda:3"/vllm_device="cuda:0"/' r1_vlm/src/r1_vlm/environments/digit_recognition_env/train.py
sed -i 's/per_device_train_batch_size=5/per_device_train_batch_size=2/' r1_vlm/src/r1_vlm/environments/digit_recognition_env/train.py
sed -i 's/num_generations=15/num_generations=2/' r1_vlm/src/r1_vlm/environments/digit_recognition_env/train.py
sed -i 's/vllm_gpu_memory_utilization=0.8/vllm_gpu_memory_utilization=0.45/' r1_vlm/src/r1_vlm/environments/digit_recognition_env/train.py
sed -i 's/gradient_checkpointing = False/gradient_checkpointing = True/' r1_vlm/src/r1_vlm/environments/digit_recognition_env/train.py
sed -i 's/max_completion_length=512/max_completion_length=256/' r1_vlm/src/r1_vlm/environments/digit_recognition_env/train.py
sed -i 's/gradient_accumulation_steps=4/gradient_accumulation_steps=8/' r1_vlm/src/r1_vlm/environments/digit_recognition_env/train.py

5. Fix qwen_grpo_trainer.py
Issue: TypeError: unhashable type: 'slice'Diagnosis: The error occurs because simple_vision_env.py returns a dictionary {"ids": [...], "messages": [...], "mask": [...]} for completion_ids, but the trainer expects a list. This causes a failure when slicing is attempted.Solution: Update trl/trl/trainer/qwen_grpo_trainer.py to handle the dictionary and extract the ids list.
# Modify line 624 to store dictionary
sed -i '624s/completion_ids = /completion_ids_dict = /' trl/trl/trainer/qwen_grpo_trainer.py

# Add line after 629 to extract ids
sed -i '629a\                    completion_ids = completion_ids_dict["ids"]' trl/trl/trainer/qwen_grpo_trainer.py

6. Run Training Script
cd r1_vlm
WANDB_MODE=disabled PYTHONPATH=src CUDA_VISIBLE_DEVICES=0 PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True python src/r1_vlm/environments/digit_recognition_env/train.py
cd ..

Notes

Ensure the virtual environment is activated before running commands.
These changes reduce GPU memory usage and fix common errors, including the TypeError in qwen_grpo_trainer.py.
If issues persist, verify Python version (3.10+) and dependency compatibility.

Hope this helps others get Task 2 running smoothly!```