PROGRESS.md

AI PM Technical Fluency - Progress Tracker

Started: Sessions Completed: 0 / 25

See MY_PRODUCT.md for how each session builds your AI feature. See LESSONS.md for detailed session plans.

---

Phase 1: Foundations

Session 1: LLM Architecture & API Fundamentals

• Temperature (creativity dial: 0 = deterministic, 1 = varied)
• System prompts (shape personality/UX)
• Tokens & costs (output costs 5x input)
• Max tokens (hard cutoff, balance cost vs UX)
• Evaluate tab (QA prompts like features)
• Hallucination vs grounding (why RAG exists)

Session 2: Prompt Engineering Patterns

• Few-shot learning (teaching by example)
• Chain-of-thought (forcing the model to show its work)
• Structured outputs (getting JSON reliably)
• Combined all three patterns together

Session 3: Embeddings & Semantic Similarity

• What embeddings are (text → numbers)
• Semantic similarity (finding related content)
• Chat API vs Embedding API
• Quiz (8 questions)

Session 4: Vector Databases Deep Dive

• Why embeddings need special databases
• Pinecone / Chroma / pgvector comparison
• Similarity search and indexing
• Metadata filtering
• Quiz (8 questions)

Session 5: Document Processing & Chunking

• Splitting documents into pieces
• Chunk size tradeoffs (small vs large)
• Overlap strategies
• Metadata tagging
• Quiz (8 questions)

Session 6: Building Your First RAG Pipeline

• End-to-end RAG flow
• Query → embed → search → inject → generate
• Failure modes and debugging
• Quiz (8 questions)

---

Phase 2: Agentic AI

Session 7: Function Calling & Tool Use

• LLMs calling external functions/APIs
• Define tools and schemas
• Claude deciding when to use tools
• Quiz (8 questions)

Session 8: ReAct Agents & Extended Thinking

• Reason → Act → Observe → Repeat
• Extended Thinking (built-in reasoning)
• Interleaved thinking + tools
• Quiz (8 questions)

Session 9: Memory Systems for Stateful Agents

• Short-term memory (within session)
• Long-term memory (across sessions)
• Memory retrieval strategies
• Quiz (8 questions)

Session 10: Advanced RAG Patterns

• Hybrid search (keywords + embeddings)
• Reranking retrieved results
• Query expansion/transformation
• Quiz (8 questions)

Session 11: Corrective & Autonomous RAG

• Self-RAG pattern
• Corrective RAG (check and retry)
• Confidence scoring
• Quiz (8 questions)

Session 12: When RAG Fails — Agentic Search Alternatives

• RAG's hidden costs (staleness, reliability, security)
• Agentic search pattern (grep/glob + iterative reasoning)
• Case study: Why Anthropic abandoned RAG for Claude Code
• Decision heuristic: When to index vs search dynamically
• Quiz (8 questions)

Session 13: Voice & Multimodal Agents

• Image input to Claude
• Analyzing visual content
• Voice input concepts
• Quiz (8 questions)

---

Phase 3: Multi-Agent Systems

Session 14: Multi-Agent Architecture Patterns

• Single agent vs multi-agent comparison
• Orchestration patterns (sequential, parallel, hierarchical)
• Communication between agents
• Quiz (8 questions)

Session 15: CrewAI for Role-Based Teams

• Define agent roles
• Agent collaboration
• Handoffs between agents
• Quiz (8 questions)

Session 16: LangGraph for Complex Workflows

• Graph-based workflows
• Conditional branching
• Human-in-the-loop patterns
• Quiz (8 questions)

Session 17: OpenAI Agents SDK Deep Dive

• OpenAI's agent patterns
• Compare to Anthropic's approach
• Vendor tradeoffs
• Quiz (8 questions)

Session 18: Google ADK for Enterprise Agents

• Google's approach
• Enterprise requirements
• Multi-cloud considerations
• Quiz (8 questions)

Session 19: MCP & Standardized Tool Calling

• MCP architecture (hosts, clients, servers)
• Claude Desktop + MCP
• Build vs integrate decision
• Quiz (8 questions)

---

Phase 4: Production

Session 20: Evaluation Frameworks That Matter

• Define success metrics
• Human eval vs automated eval
• A/B testing AI features
• Quiz (8 questions)

Session 21: Safety, Guardrails & Responsible AI

• Input/output guardrails
• Content filtering
• Bias detection
• Quiz (8 questions)

Session 22: RLHF & Alignment Deep Dive

• RLHF conceptual walkthrough
• Constitutional AI
• Alignment implications for products
• Quiz (8 questions)

Session 23: System Design for AI Products

• Draw the architecture
• Identify bottlenecks
• Plan for scale
• Quiz (8 questions)

Session 24: Fine-tuning vs RAG vs Agentic Search Decision Framework

• The four options spectrum (Prompt → Agentic → RAG → Fine-tune)
• Decision flowchart with real criteria
• When agentic search beats RAG / when RAG still wins
• Hybrid strategies
• Quiz (8 questions)

Session 25: Capstone Integration

• Design the full feature
• Build vs buy decisions
• Create a roadmap
• Quiz (8 questions)

---

Learning Notes

Key Insights

• (Added as you learn)

Questions to Revisit

• (none yet)

Favorite Experiments

• (Added as you discover them)