feat(posthog): add per-metric summary reports (#3824)

[sanskar-singh-2403]

Fixes #3824

Describe the changes you have made in this PR -

Adds PostHog analytics per-metric summary reports, mirroring the existing
Sentry morning-digest ("Viktor-style") pattern so the surface, output shape,
and scheduling behaviour stay consistent across integrations.

• posthog-summary skill (integrations/posthog/tools/skills/posthog-summary/SKILL.md):
  pulls metrics via the existing PostHog MCP tools (list_posthog_tools /
  call_posthog_tool) and formats a per-metric report with current vs. previous
  deltas, percent change, and trend direction.
• report_runner (integrations/posthog/report_runner.py): runs one headless
  posthog-summary turn and returns the assistant report text; guards on PostHog
  being configured.
• report_prerequisites (integrations/posthog/report_prerequisites.py):
  PostHog integration + delivery-provider readiness checks.
• CLI (surfaces/cli/commands/posthog_report.py):
  opensre posthog report run (on-demand) and
  opensre posthog report schedule add/list/remove/run (scheduled), registered
  in the CLI command list.
• Scheduler wiring: new POSTHOG_METRIC_REPORT TaskKind, a message builder
  in platform/scheduler/tasks.py (strips credential keys, defaults to a 7d
  window), and a route in integrations/scheduled_agent_bootstrap.py.
• Assistant prompt: a PostHog report output-shape rule so the per-metric
  table format stays deterministic.
• Tests: builder coverage in tests/scheduler/test_tasks.py and runner
  coverage in tests/integrations/posthog/test_report_runner.py.

Acceptance criteria from the issue: connect to PostHog data source ✅,
per-metric summary report ✅, Viktor-style output format ✅, on-demand and
scheduled triggering ✅.

Demo/Screenshot for feature changes and bug fixes -

---

Code Understanding and AI Usage

Did you use AI assistance (ChatGPT, Claude, Copilot, etc.) to write any part of this code?

• No, I wrote all the code myself
• Yes, I used AI assistance (continue below)

If you used AI assistance:

• I have reviewed every single line of the AI-generated code
• I can explain the purpose and logic of each function/component I added
• I have tested edge cases and understand how the code handles them
• I have modified the AI output to follow this project's coding standards and conventions

Explain your implementation approach:

The issue asks for automated per-metric PostHog reports "like Viktor reports."
The codebase already had that exact pattern for Sentry (the morning digest), so
rather than inventing a new mechanism I mirrored it: a bundled skill defines the
report workflow/format, a headless runner dispatches a single agent turn driven
by that skill, and the scheduler multiplexes the task onto the shared
agent-runner seam via a new TaskKind. PostHog connectivity already existed
(integrations/posthog REST config + posthog_mcp tool surface), so no new
transport code was needed — the runner just reuses those tools.

Alternative considered: querying the PostHog REST API directly and templating a
report in Python. I rejected that because it would duplicate query logic the MCP
tool surface already exposes, wouldn't reuse the shared skill/prompt formatting,
and would diverge from the Sentry precedent that reviewers already know.

Key components: run_posthog_report (headless turn + report extraction),
_build_posthog_metric_report (scheduler builder, credential stripping,
default window), the posthog report CLI group (on-demand + schedule
management), and the posthog-summary skill/prompt rule that pin the output
shape to a Metric | Current | Previous | Change | % | Trend table.

---

Checklist before requesting a review

• I have added proper PR title and linked to the issue
• I have performed a self-review of my code
• I can explain the purpose of every function, class, and logic block I added
• I understand why my changes work and have tested them thoroughly
• I have considered potential edge cases and how my code handles them
• If it is a core feature, I have added thorough tests
• My code follows the project's style guidelines and conventions

---

Note: Please check Allow edits from maintainers if you would like us to assist in the PR.

[github-actions]

Greptile code review

This repo uses Greptile for automated review. Before merge, aim for Confidence Score: 5/5 with zero unresolved review threads — see CONTRIBUTING.md.

Run a review — add a PR comment with:

@greptile review

Give it ~5-10 minutes (sometimes longer) for results, then fix feedback and re-trigger until you reach Confidence Score: 5/5.

Optional: automate with the greploop skill.

[greptile-apps]

Greptile Summary

This PR adds per-metric PostHog analytics summary reports, mirroring the existing Sentry morning-digest pattern: a bundled posthog-summary skill drives a headless agent turn, a new POSTHOG_METRIC_REPORT TaskKind plugs into the shared scheduler, and on-demand / scheduled CLI commands (opensre posthog report run|schedule) are registered alongside the existing Sentry commands.

• New runner & prerequisites (integrations/posthog/report_runner.py, report_prerequisites.py): headless AgentHarness turn with fail-fast PostHog config guard; stats_period_to_hours() converts period strings to window hours.
• Scheduler wiring (platform/scheduler/tasks.py, types.py, scheduled_agent_bootstrap.py): _build_posthog_metric_report builds the agent payload with credential stripping and source-string routing; delivery.py is promoted from integrations/sentry/ to platform/scheduler/ so both integrations share it.
• CLI & interactive shell (surfaces/cli/commands/posthog_report.py, cli_parity.py): full run / schedule add|list|run|remove surface with prerequisite gates and Rich table output.

Confidence Score: 4/5

Safe to merge after adding the missing tests/integrations/posthog/init.py; all other changes are clean and follow established patterns.

The implementation faithfully mirrors the Sentry digest pattern and all critical paths work correctly. The only blocking gap is the absent tests/integrations/posthog/init.py: every other integration test subdirectory in the repo has one, and tests/platform/scheduler/init.py was added in this very PR for the same reason — the omission will break test collection for the new test file.

Files Needing Attention: tests/integrations/posthog/ — missing init.py that the project's test package convention requires.

Important Files Changed

Filename	Overview
integrations/posthog/report_runner.py	New headless runner; directly mirrors sentry/morning_digest_runner.py pattern. Result extraction and error handling are sound.
integrations/posthog/report_prerequisites.py	New prerequisites module; stats_period_to_hours correctly handles h/d/w units with graceful fallback for invalid input.
surfaces/cli/commands/posthog_report.py	New CLI command group; prerequisite gates are correctly placed. schedule add omits rocketchat as a provider choice, inconsistent with the Sentry digest command and the underlying Provider enum.
tests/integrations/posthog/test_report_runner.py	Good test coverage for prompt building and prerequisite checks, but the new tests/integrations/posthog/ directory is missing init.py, required by this project's test package convention.
platform/scheduler/tasks.py	New _build_posthog_metric_report builder added; credential stripping, default period, and source/task_id override ordering are all correct and consistent with existing builders.
platform/scheduler/delivery.py	Delivery module moved from integrations/sentry/digest_delivery.py; renamed function correctly generalises it for all report types.
integrations/scheduled_agent_bootstrap.py	PostHog routing via 'posthog' in source correctly intercepts both CLI and scheduled task payloads before the Sentry fallback.
tests/scheduler/test_tasks.py	Thorough new tests for the PostHog builder: agent-runner routing, default period, credential stripping, and failure propagation all covered.

Sequence Diagram

sequenceDiagram
    participant CLI as opensre posthog report run
    participant Runner as scheduled_agent_bootstrap
    participant Report as run_posthog_report
    participant Harness as AgentHarness
    participant Agent as HeadlessAgent
    participant MCP as PostHog MCP Tools

    CLI->>CLI: require_posthog_integration()
    CLI->>Runner: invoke_agent_runner(payload)
    Runner->>Runner: posthog in source
    Runner->>Report: run_posthog_report(payload)
    Report->>Report: _require_posthog_configured()
    Report->>Harness: startup()
    Harness-->>Report: session
    Report->>Agent: dispatch(prompt)
    Agent->>MCP: list_posthog_tools / call_posthog_tool
    MCP-->>Agent: metric data
    Agent-->>Report: ShellTurnResult
    Report-->>CLI: report text

Loading

_{Reviews (3): Last reviewed commit: "feat(posthog): add per-metric summary re..." | Re-trigger Greptile}

[sanskar-singh-2403]

@greptile review

[sanskar-singh-2403]

@muddlebee can you kindly let me know your thoughts on this?

[muddlebee]

There's no demo, need screen recording?

[sanskar-singh-2403]

@muddlebee

Demo -

** Recording 2026-07-20 212111.mp4 - Google Drive
** github.com/user-attachments/files/30195791/posthog_demo_log.txt
Verified end-to-end against a live PostHog project via the hosted PostHog MCP server, with the headless agent turn driven by the posthog-summary skill:

integrations verify posthog_mcp — connection confirmed, event schema fetched
posthog report run --period 7d — runs schema discovery, then one bounded HogQL query per metric, and renders the per-metric table (Metric | Current | Previous | Change | % | Trend)
The project I connected is a fresh PostHog project with no ingested events, so the counts are 0 — this exercises the skill's "genuine no-data vs. query-error" handling (it reports a real zero-ingestion result, not a failure). Happy to seed sample events and re-record with non-zero movement if you'd prefer a data-rich table.

[muddlebee]

thank you will take a look shortly

[Davidson3556]

Nice work . One correctness issue I'd want fixed, left inline, plus two process gaps that have no line to anchor to:

Docs — this adds a user-facing opensre posthog report group with no docs change. docs/posthog.mdx and docs/posthog-mcp.mdx already exist and docs/sentry.mdx documents the Sentry digest, so there's a clear home for it. Needs the docs/docs.json entry too or Mintlify won't surface the page.

[sanskar-singh-2403]

@Davidson3556 I have made all the changes suggested by you, kindly review this and let me know your thoughts.

[Devesh36]

@Davidson3556 can you please check this again

[Devesh36]

lot of ci failurs too

[sanskar-singh-2403]

lot of ci failurs too

hey @Devesh36 can we re-run the pipeline i have fixed the test

[sanskar-singh-2403]

Hey @muddlebee @Davidson3556, just a friendly reminder whenever you get a chance to review this PR. Thanks!

[Davidson3556]

ci is failing.

[Davidson3556]

1. report_prerequisites.py pulls delivery helpers from integrations.sentry.digest_delivery, but that module is provider-generic. A second consumer is the cue to move it somewhere neutral (platform/scheduler/) instead of coupling posthog to sentry.

2. _install_scheduler_runners() is copy-pasted from sentry_digest.py, and the "7d" default is repeated in a few spots.

[sanskar-singh-2403]

@Davidson3556 i added this for python.exe path to be taken as well in case of windows OS

[sanskar-singh-2403]

1. report_prerequisites.py pulls delivery helpers from integrations.sentry.digest_delivery, but that module is provider-generic. A second consumer is the cue to move it somewhere neutral (platform/scheduler/) instead of coupling posthog to sentry.
2. _install_scheduler_runners() is copy-pasted from sentry_digest.py, and the "7d" default is repeated in a few spots.

Done both addressed in the latest push.

1. Moved the provider-generic delivery helpers out of sentry/ into platform/scheduler/delivery.py; both the sentry and posthog prerequisites now import downward from there instead of posthog→sentry. Old integrations/sentry/digest_delivery.py deleted, test moved to tests/platform/scheduler/.

2. Extracted the duplicated _install_scheduler_runners() into one shared install_scheduler_runners(), and replaced the repeated "7d" with a DEFAULT_POSTHOG_PERIOD constant.

Tested locally lint-imports --strict passes (1 kept, 0 broken) and the affected suites are green. The only local failure is tests/config/test_runtime_metadata_perf.py::test_baseline_stability, which is unrelated to this change: it's a wall-clock threshold test (asserts build_runtime_metadata runs in <5ms) and it blows the budget only because I'm running under pytest -n auto on a WSL2 box where CPU contention across workers pushes it to ~240ms. It should pass on CI's dedicated runner 🤞 .

[sanskar-singh-2403]

Hey @Davidson3556 can we re-run the CI? if you are ok with the changes? Thanks!

[muddlebee]

@sanskar-singh-2403 thank you for the efforts, will review in next 1-2 days

[sanskar-singh-2403]

@sanskar-singh-2403 thank you for the efforts, will review in next 1-2 days

hey @muddlebee @Davidson3556 bumping this up for a review, Thanks!

[muddlebee]

First step ideally would be to discuss in the github issue about your approach and thoughts rather than sending PRs directly.

[muddlebee]

I think I missed to mention that earlier.

[sanskar-singh-2403]

First step ideally would be to discuss in the github issue about your approach and thoughts rather than sending PRs directly.

Got it, thanks for flagging @muddlebee, that makes sense. I'll write up my approach and reasoning on the linked issue so we can align there first. Happy to adjust the implementation based on that discussion, and I'll keep this PR as a draft in the meantime.

[sanskar-singh-2403]

@larsspinetta12 this is ready for review, Thanks!