Skip to content

test: Add unit tests for AI scanners with mocked AI service #5

Description

@nmogil

Summary

Add Vitest unit tests for the 6 AI-powered scanners by mocking the AI gateway service. Currently only the 4 deterministic scanners have tests.

Motivation

Test coverage is the biggest quality gap in the project. AI scanners contain complex prompt construction, response parsing, and tier assignment logic — all of which can break silently. Tests with mocked AI responses verify the surrounding logic without hitting a real LLM.

Currently tested: contentFlags, optOut, urls, rollup
Currently untested: description, sampleMessages, optIn, shaft, affiliateMarketing, consistency, privacyPolicy, termsOfService

Implementation Steps

1. Add AI service mock helper

File: worker/test/helpers/mockAi.ts (new)

The AI service is called in each scanner via callAi() from worker/src/services/ai.ts. Mock this function using vi.spyOn:

import { vi } from 'vitest';

export interface MockAiResponse {
  tier: 'RED' | 'YELLOW' | 'GREEN';
  rationale: string;
  issues?: { severity: string; message: string; twilioErrorCode?: string }[];
  suggestions?: { issue: string; fix: string; example?: string }[];
}

export function mockAiCall(response: MockAiResponse) {
  return vi.fn().mockResolvedValue(response);
}

export const mockEnv = {
  CF_AIG_TOKEN: 'test-token',
  AI_GATEWAY_URL: 'http://test-gateway.local',
  DB: {} as any,
  RATE_LIMIT: {} as any,
  FIRECRAWL_API_KEY: 'test-firecrawl',
  ALLOWED_ORIGINS: 'http://localhost:3000',
  RULES_VERSION: '2026-test.1',
};

2. Test scanDescription

File: worker/test/scanners/description.test.ts (new)

import { describe, it, expect, vi, beforeEach } from 'vitest';
import { scanDescription } from '../../src/scanners/description';
import { mockEnv } from '../helpers/mockAi';
import * as aiService from '../../src/services/ai';
import { goodCampaign } from '../fixtures/campaigns';

describe('scanDescription', () => {
  beforeEach(() => vi.restoreAllMocks());

  it('returns GREEN for clear, specific description', async () => {
    vi.spyOn(aiService, 'callAi').mockResolvedValue({
      tier: 'GREEN', rationale: 'Description is clear and specific',
      issues: [], suggestions: [],
    });
    const result = await scanDescription(goodCampaign, mockEnv);
    expect(result.tier).toBe('GREEN');
    expect(result.field).toBe('campaignDescription');
    expect(result.evidence.source).toBe('ai');
  });

  it('returns RED for vague generic description', async () => { /* mock RED response */ });
  it('returns YELLOW with timeout fallback when AI fails', async () => { /* mock rejection */ });
  it('handles malformed AI response gracefully', async () => { /* mock invalid shape */ });
});

3. Test remaining AI scanners (same pattern)

Each scanner needs these 4 test cases at minimum:

File Scanner Key test scenarios
worker/test/scanners/sampleMessages.test.ts scanSampleMessages GREEN for realistic msgs, RED for placeholder/lorem ipsum, RED for use case mismatch
worker/test/scanners/optIn.test.ts scanOptIn GREEN for web form opt-in, RED for SMS-based initial opt-in
worker/test/scanners/shaft.test.ts scanShaft GREEN for medical context, RED for promotional alcohol/cannabis, error code 30883
worker/test/scanners/affiliateMarketing.test.ts scanAffiliateMarketing GREEN for single-brand, RED for third-party lead gen
worker/test/scanners/consistency.test.ts scanConsistency GREEN for aligned fields, RED for contradicting use case vs messages

4. Add campaign fixtures

File: worker/test/fixtures/campaigns.ts — extend existing file

Add named fixtures: goodCampaign, vagueCampaign, shaftCampaign, affiliateCampaign, inconsistentCampaign

Testing

cd worker
npm test                    # Run all tests
npm test -- description     # Run only description tests
npm run test:watch          # TDD mode

Acceptance Criteria

  • All 6 AI scanners have unit tests
  • Each tests: happy path, failure path, AI timeout fallback, malformed response handling
  • No tests hit a real AI service
  • npm test passes in worker/

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions