Skip to content

README.md

Latest commit

 

History

History
199 lines (145 loc) · 6.7 KB

README.md

File metadata and controls

199 lines (145 loc) · 6.7 KB
Raw

GPT-5 vs Claude Opus 4.1: Real-World Project Implementation Comparison

🎯 Project Overview

This repository contains the results of a head-to-head comparison between OpenAI's GPT-5 and Anthropic's Claude Opus 4.1, both running in Cursor IDE. The challenge was to have both AI models independently implement the same real estate investment opportunity analyzer using identical requirements and Zillow housing market data.

Video Demo: Watch the full comparison on YouTube

🏆 Competition Setup

The Challenge

Both AI models were tasked with:

  • Creating a complete full-stack application from a Product Requirements Document (PRD)
  • Analyzing Zillow real estate datasets to identify investment opportunities
  • Building a dashboard with authentication, data visualization, and AI-powered insights
  • Implementing the solution with Python backend, React frontend, and SQLite database

Fair Testing Conditions

  • Identical PRD: Both models received the same requirements document
  • Same Data: Identical Zillow CSV files (home values, inventory, rentals)
  • Parallel Execution: Both started at the same time
  • No Human Intervention: Minimal guidance, letting models work autonomously
  • Same Environment: Both running in Cursor IDE with same tools available

📊 Results Summary

Metric GPT-5 Claude Opus 4.1
Time to Complete ~18 minutes (initial) + debugging ~12 minutes (initial)
UI/UX Quality Basic, console-like output Polished, professional interface
Stability Multiple issues, required fixes More stable, fewer errors
LLM Integration Eventually working Working with proper narratives
Overall Score 6/10 8/10

Winner: Claude Opus 4.1 🥇

🗂️ Repository Structure

.
├── PRD.md                    # Product Requirements Document used by both models
├── transcript.md             # Full transcript of the comparison video
├── gpt_5/                    # GPT-5's implementation
│   ├── backend/              # Python FastAPI backend
│   ├── frontend/             # React TypeScript frontend
│   └── data/                 # Zillow datasets
└── opus41/                   # Claude Opus 4.1's implementation
    ├── backend/              # Python backend with agents
    ├── frontend/             # React frontend
    └── data/                 # Zillow datasets

🚀 Running the Applications

Prerequisites

  • Python 3.8+
  • Node.js 16+
  • Google Gemini API key

Opus 4.1 Implementation

cd opus41

# Setup backend
cd backend
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate
pip install -r requirements.txt

# Create .env file with your Gemini API key
echo "GOOGLE_API_KEY=your_key_here" > .env

# Run the application
cd ..
chmod +x start.sh
./start.sh

Access at: http://localhost:5173

  • Username: admin
  • Password: admin

GPT-5 Implementation

cd gpt_5

# Setup backend
cd backend
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate
pip install -r requirements.txt

# Create .env file
echo "GOOGLE_API_KEY=your_key_here" > .env

# Start backend
python main.py

# In a new terminal, start frontend
cd ../frontend
npm install
npm run dev

Access at: http://localhost:5174

🎯 Features Implemented

Both implementations include:

  • ✅ User authentication
  • ✅ Dashboard with investment opportunity scores
  • ✅ Regional data visualization with charts
  • ✅ Z-score based investment analysis
  • ✅ Integration with Google Gemini for AI insights
  • ✅ SQLite database for data persistence

Key Differences

Claude Opus 4.1:

  • Better UI/UX with professional styling
  • More comprehensive error handling
  • Cleaner code structure
  • Functional LLM integration with investment narratives
  • Proper data pagination and filtering

GPT-5:

  • More basic interface design
  • Required more debugging iterations
  • Console-like output for AI narratives
  • Simpler implementation overall

📈 Investment Analysis Methodology

Both systems analyze regions based on:

  • Home Value Trends: Historical price appreciation
  • Rental Market: Rental price growth and yield
  • Inventory Levels: Supply/demand dynamics
  • Risk Assessment: Market volatility and data availability

The systems calculate composite Z-scores to rank investment opportunities.

🔍 Key Observations

  1. Speed: Opus 4.1 was significantly faster, completing initial implementation in ~12 minutes vs ~18 minutes for GPT-5

  2. Quality: Opus 4.1 produced more polished, production-ready code with better UI/UX

  3. Debugging: GPT-5 required more iterations to fix issues, particularly with authentication and frontend-backend integration

  4. Documentation: Opus 4.1 was more verbose but comprehensive, while GPT-5 was more concise

  5. Tool Usage: Opus 4.1 showed better understanding and usage of Cursor's tools and Context7 documentation

💡 Lessons Learned

  • Model Performance: Despite the hype around GPT-5, Claude Opus 4.1 demonstrated superior performance in this real-world coding task
  • Context Management: Opus 4.1 handled the complex full-stack context better
  • Autonomous Coding: Both models can create functional applications with minimal human intervention
  • UI/UX Matters: Default design choices significantly impact the final product quality

🛠️ Technologies Used

  • Backend: Python, FastAPI/Flask, SQLite, SQLAlchemy
  • Frontend: React, TypeScript/JavaScript, Vite
  • AI/ML: Google Gemini API, LangChain
  • Data: Zillow housing market datasets (CSV)
  • Deployment: Docker-ready configurations

📝 License

This project is for educational and demonstration purposes. The Zillow data is used under their terms of service.

🤝 Contributing

Feel free to fork this repository and experiment with improvements to either implementation. Some ideas:

  • Add more sophisticated investment algorithms
  • Implement real-time data updates
  • Enhance the UI/UX
  • Add more regions and data sources
  • Improve the AI agent system

📺 Watch the Full Comparison

For a detailed walkthrough of the entire comparison process, including live coding and debugging sessions, watch the full video: https://youtu.be/XAWonUynpVQ

🙏 Acknowledgments

  • Zillow for providing the housing market datasets
  • Cursor IDE for the development environment
  • Google for the Gemini API
  • The AI models themselves for their impressive capabilities

Note: This comparison was conducted on [Date], and model capabilities may have evolved since then. The results reflect performance at the time of testing.