AlphaZero From Scratch: Attaxx & Go

An end-to-end implementation of the AlphaZero algorithm from scratch using PyTorch and Monte Carlo Tree Search (MCTS). This repository provides complete self-play reinforcement learning, neural network heuristic evaluation, and deployment capabilities across two classic adversarial games: Attaxx (4x4, 5x5, 6x6) and Go (7x7, 9x9).

---

Overview

AlphaZero revolutionized zero-knowledge reinforcement learning by replacing handcrafted domain heuristics with Monte Carlo Tree Search (MCTS) guided by a dual-head deep residual neural network.

This project explores:

• Spatial Heuristics vs. Sequential Architectures: Comparative study between 2D Convolutional ResNets and Recurrent LSTMs (AttaxRNN / GoRNN).
• Domain Generalization: Training across games with fundamentally different mechanics—Attaxx (cloning & piece capture dynamics) and Go (territory control & group liberty management).
• Hyperparameter Optimization: Systematic evaluation of L2 weight decay vs. dropout regularization to mitigate early-stopping and policy collapse.
• Client-Server Deployment: Multi-agent TCP socket architecture for real-time competitive evaluation.

---

Features

• Modular Architecture: Separate modules for game rules, MCTS lookahead search, neural network definitions, self-play training, and evaluation for both Attaxx and Go.
• Dual-Head ResNet Heuristics: Shared feature extractor with a Policy Head (probability distribution over actions) and a Value Head (board state evaluation scalar $v \in [-1, 1]$).
• Customizable MCTS Engine: Configurable PUCT exploration constant $c_{puct}$, Dirichlet noise $\text{Dir}(\alpha)$, and search iterations.
• Baseline Models: Comparative baseline models including a Recurrent LSTM network (AttaxRNN, GoRNN) and unguided MCTS.
• Client-Server Environment: Real-time deployment scripts (client.ipynb and server.ipynb) for socket-based AI competition.
• Pre-trained Checkpoints: Model checkpoints (.pt) and optimizer state dictionaries for 4x4, 5x5, and 6x6 Attaxx board sizes.
• Automated Benchmarking: Performance metrics, loss logs, and visualization figures saved in results/ and figures/.

---

Architecture

1. Self-Play & MCTS

The agent selects moves using MCTS guided by prior probabilities $P(s, a)$ and value estimations $v(s)$ from the neural network: $$a_t = \arg\max_a \left[ Q(s, a) + c_{puct} P(s, a) \frac{\sqrt{N(s)}}{1 + N(s, a)} \right]$$

2. Dual-Head ResNet Architecture

The neural network consumes an $N \times N \times C$ board tensor, processes it through $K$ Residual Blocks (Conv-BN-ReLU-Conv-BN with residual skip connection), and outputs both policy logits and scalar state values.

3. MCTS Four-Phase Search

1. Selection: Traverse the tree selecting actions with maximum PUCT value.
2. Expansion: Expand unvisited leaf states.
3. Evaluation: Compute policy $P(s, a)$ and value $v(s)$ using ResNet.
4. Backpropagation: Update node visit counts $N$ and mean action values $Q$.

---

Repository Structure

alphazero-from-scratch/
├── README.md                  # Project documentation & overview
├── LICENSE                    # MIT open-source license
├── CITATION.cff               # Citation reference format
├── TECHNICAL_REPORT.pdf       # Comprehensive PDF technical report
├── requirements.txt           # Python dependencies
├── .gitignore                 # Git ignore rules
│
├── figures/                   # Architecture, pipeline, & gameplay figures
│   ├── pipeline.png
│   ├── alphazero_architecture.png
│   ├── mcts.png
│   ├── training_curve.png
│   ├── go_gameplay.png
│   └── attaxx_gameplay.png
│
├── results/                   # Benchmark metrics, tables, & evaluation plots
│   ├── go/
│   │   ├── benchmark_summary.json
│   │   └── go_board_scalability.png
│   ├── attaxx/
│   │   ├── benchmark_summary.json
│   │   └── regularization_experiments.png
│   └── comparison/
│       ├── architecture_comparison.csv
│       └── cnn_vs_rnn_comparison.png
│
├── attaxx/                    # Attaxx game implementation & models
│   ├── game/
│   │   └── AttaxGame.ipynb
│   ├── mcts/
│   │   └── MCTS_Attax.ipynb
│   ├── neural_network/
│   │   └── NeuralNetwork_Attax.ipynb
│   ├── training/
│   │   └── AlphaZero_Attax.ipynb
│   ├── evaluation/
│   │   └── Attax_Test.ipynb
│   ├── agents/
│   │   └── Agent1.ipynb
│   ├── deployment/
│   │   ├── client.ipynb
│   │   └── server.ipynb
│   ├── models/
│   │   ├── model14Attax5.pt
│   │   ├── model7Attax6.pt
│   │   └── model8Attax4.pt
│   └── optimizers/
│       ├── optimizer14Attax5.pt
│       ├── optimizer7Attax6.pt
│       └── optimizer8Attax4.pt
│
└── go/                        # Go game implementation
    ├── game/
    │   ├── GoGame.ipynb
    │   └── GoPvP.py
    ├── mcts/
    │   └── MCTS_Go.ipynb
    ├── neural_network/
    │   └── NeuralNetwork_Go.ipynb
    ├── training/
    │   └── AlphaZero_Go.ipynb
    └── evaluation/
        └── Go_Test.ipynb

---

Installation & Setup

1. Clone the repository:

   git clone https://github.com/LaraSousa34/LabIACD-Project2.git
   cd LabIACD-Project2

2. Create a virtual environment (recommended):

   python3 -m venv venv
   source venv/bin/activate

3. Install dependencies:

   pip install -r requirements.txt

---

Training & Evaluation

Training Attaxx

Open attaxx/evaluation/Attax_Test.ipynb or attaxx/training/AlphaZero_Attax.ipynb in Jupyter Notebook or VS Code to configure training hyperparameters and execute self-play:

args = {
    'C': 2,                       # Exploration constant
    'num_searches': 100,          # MCTS searches per move
    'num_iterations': 500,        # Training iterations
    'num_selfPlay_iterations': 1000,
    'num_epochs': 50,             # Training epochs per iteration
    'batch_size': 64,
    'epsilon': 0.25,              # Dirichlet noise weight
    'alpha': 0.3                  # Dirichlet alpha parameter
}

Training Go

Run go/evaluation/Go_Test.ipynb or go/training/AlphaZero_Go.ipynb for 7x7 or 9x9 board sizes:

game = GoGame(dimension=7)
model = ResNet(game, num_resBlocks=4, num_hidden=64, device=device)
alphaZero = AlphaZero(model, optimizer, game, args)
alphaZero.learn()

Socket Client-Server Deployment

To test real-time agent competition over local sockets:

1. Start the server notebook: attaxx/deployment/server.ipynb
2. Connect two agents using attaxx/deployment/client.ipynb

---

Key Results

Metric / Architecture	ResNet (CNN)	LSTM (RNN)
Spatial Awareness	High (2D Convolutions)	Low (Flattened 1D)
Policy Convergence	Fast (Iterations 8–14)	Slow / Flat Loss
Win Rate vs MCTS Baseline	88.5%	42.0%
Evaluation Speed (ms/move)	4.2 ms	12.8 ms

Regularization Findings

• L2 Weight Decay: A lower weight decay (0.0001) prevents premature early stopping. Higher weight decay (0.01) induces severe underfitting.
• Dropout: Adding a light dropout layer (15%) yields smoother loss convergence and stabilizes self-play against overconfident value targets.

---

Technical Report

For full theoretical derivations, empirical tables, loss progression analysis, and architectural breakdowns, refer to the included PDF report: 👉 TECHNICAL_REPORT.pdf

---

Future Work

• Action-Space Factorization: Scaling Go to 19x19 dimensions via hierarchical policy heads.
• Vision Transformers (ViT): Evaluating ViT backbones with self-attention for global group liberty and territory assessment.
• Distributed Self-Play: Asynchronous Ray / TorchDistributed workers to accelerate self-play data generation.

---

Citation & License

This project is licensed under the MIT License.

If you use or reference this work in your research or portfolio, please cite:

@software{Sousa_AlphaZero_From_Scratch_2026,
  author = {Sousa, Lara},
  title = {AlphaZero-From-Scratch: Deep Reinforcement Learning for Attaxx and Go},
  url = {https://github.com/LaraSousa34/LabIACD-Project2},
  version = {1.0.0},
  year = {2026}
}