Skip to content

Commit a2392b2

Browse files
karngyankarngyan
committed
initial scaffold: rate-limiter starter (python)
Five-stage byox starter for the karnstack Rate Limiter primitive. Failing skeletons for token bucket, sliding-window counter, pluggable counter backend, burst + Retry-After math, and client jitter strategies. Includes the canonical .karnstack/ helper scripts and verify-stages workflow that posts results back to karnstack.com/api/v1/byox/verify.
0 parents  commit a2392b2

17 files changed

Lines changed: 1113 additions & 0 deletions

‎.github/workflows/verify-stages.yml‎

Lines changed: 119 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,119 @@
1+
name: verify-stages
2+
3+
# Runs the BYOX test suite on every push to main, hashes the canonical
4+
# files, mints a GitHub OIDC token bound to audience=karnstack, and
5+
# reports the result to karnstack.com so verified stages light up in
6+
# your karnstack dashboard.
7+
#
8+
# You should not edit this workflow or anything in .karnstack/ - those
9+
# files are part of the karnstack canonical template and their hashes
10+
# are checked server-side. Modifying them causes verification to be
11+
# rejected with `files_modified`. Re-fetch them from the template if
12+
# you do edit by accident:
13+
#
14+
# git checkout origin/template -- .github/workflows/verify-stages.yml .karnstack/
15+
16+
on:
17+
push:
18+
branches: [main]
19+
workflow_dispatch:
20+
21+
permissions:
22+
contents: read
23+
id-token: write # required to mint the OIDC token
24+
25+
env:
26+
PROJECT_SLUG: rate-limiter
27+
LANGUAGE: python
28+
KARNSTACK_URL: https://karnstack.com
29+
30+
jobs:
31+
verify:
32+
# Skip the workflow on the karnstack canonical template itself.
33+
# A template-marked repo cannot produce a valid `template_repository`
34+
# claim, so karnstack's verify endpoint would reject it anyway with
35+
# `not_from_template`. Skipping here saves the action minutes and
36+
# keeps the template's CI history clean.
37+
if: ${{ !github.event.repository.is_template }}
38+
runs-on: ubuntu-latest
39+
steps:
40+
- name: Checkout
41+
uses: actions/checkout@v6
42+
43+
- name: Install mise toolchain
44+
uses: jdx/mise-action@v4
45+
46+
- name: Run all stages
47+
id: tests
48+
continue-on-error: true
49+
run: mise run all 2>&1 | tee test-output.txt
50+
51+
- name: Compute canonical file hashes
52+
run: bash .karnstack/compute-hashes.sh > hashes.json
53+
54+
- name: Parse passing stages from test output
55+
run: bash .karnstack/parse-stages.sh test-output.txt > stages.json
56+
57+
- name: Capture template metadata
58+
id: tmpl
59+
# Workflow-attested template lineage. karnstack's verify endpoint
60+
# prefers this over hitting the unauthenticated GitHub API, which
61+
# is required for private forks (the API call 404s on those).
62+
#
63+
# The push event payload does NOT include
64+
# repository.template_repository (it's a stripped-down version of
65+
# the repo object), so we ask the REST API directly with the
66+
# auto-issued GITHUB_TOKEN. This call is authenticated and works
67+
# on private forks.
68+
#
69+
# Trust is gated server-side by the hash check above: editing this
70+
# workflow to lie about template_repo busts the canonical hash and
71+
# the verification is rejected with `files_modified`.
72+
env:
73+
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
74+
run: |
75+
set -euo pipefail
76+
TMPL=$(gh api "repos/${GITHUB_REPOSITORY}" --jq '.template_repository.full_name // ""')
77+
echo "full_name=${TMPL}" >> "$GITHUB_OUTPUT"
78+
79+
- name: Mint OIDC token and post to karnstack
80+
env:
81+
TEMPLATE_REPO: ${{ steps.tmpl.outputs.full_name }}
82+
run: |
83+
set -euo pipefail
84+
85+
OIDC_TOKEN=$(curl -sLS \
86+
-H "Authorization: Bearer ${ACTIONS_ID_TOKEN_REQUEST_TOKEN}" \
87+
"${ACTIONS_ID_TOKEN_REQUEST_URL}&audience=karnstack" \
88+
| jq -r '.value')
89+
90+
PAYLOAD=$(jq -n \
91+
--arg t "$OIDC_TOKEN" \
92+
--arg p "$PROJECT_SLUG" \
93+
--arg l "$LANGUAGE" \
94+
--arg tr "$TEMPLATE_REPO" \
95+
--slurpfile s stages.json \
96+
--slurpfile h hashes.json \
97+
'{ oidc_token: $t, project: $p, language: $l, template_repo: $tr, stages: $s[0], hashes: $h[0] }')
98+
99+
RESPONSE=$(curl -sS -X POST "${KARNSTACK_URL}/api/v1/byox/verify" \
100+
-H "Content-Type: application/json" \
101+
-d "$PAYLOAD")
102+
103+
echo "karnstack response:"
104+
echo "$RESPONSE" | jq .
105+
106+
OK=$(echo "$RESPONSE" | jq -r '.ok // false')
107+
if [ "$OK" != "true" ]; then
108+
REASON=$(echo "$RESPONSE" | jq -r '.reason // "unknown"')
109+
if [ "$REASON" = "files_modified" ]; then
110+
echo "::error title=karnstack verify::your starter is out of sync with karnstack. run 'mise run sync' (or bash .karnstack/sync.sh) to pull the latest canonical files, then commit and push."
111+
echo "::error::if .karnstack/sync.sh does not yet exist in your fork, bootstrap it first:"
112+
echo "::error:: curl -fsSL https://raw.githubusercontent.com/karnstack/byox-rate-limiter-python/main/.karnstack/sync.sh -o .karnstack/sync.sh && chmod +x .karnstack/sync.sh && bash .karnstack/sync.sh"
113+
else
114+
echo "::warning title=karnstack verify::verification not accepted: ${REASON}"
115+
fi
116+
else
117+
STAGES=$(echo "$RESPONSE" | jq -r '.verified_stages | join(", ")')
118+
echo "::notice title=karnstack verify::verified stages: ${STAGES}"
119+
fi

‎.gitignore‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
__pycache__/
2+
*.py[cod]
3+
.pytest_cache/
4+
.venv/
5+
*.egg-info/
6+
dist/
7+
build/
8+
9+
.vscode/
10+
.idea/
11+
*.swp
12+
.DS_Store

‎.karnstack/compute-hashes.sh‎

Lines changed: 61 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,61 @@
1+
#!/usr/bin/env bash
2+
# Emit SHA-256 hashes for every karnstack-canonical file as JSON:
3+
#
4+
# {
5+
# ".github/workflows/verify-stages.yml": "<sha256hex>",
6+
# ".karnstack/compute-hashes.sh": "<sha256hex>",
7+
# ".karnstack/parse-stages.sh": "<sha256hex>",
8+
# ".karnstack/sync.sh": "<sha256hex>",
9+
# ".mise.toml": "<sha256hex>",
10+
# "tests/test_stage01_token_bucket.py": "<sha256hex>",
11+
# ...
12+
# }
13+
#
14+
# karnstack's /api/v1/byox/verify endpoint compares this map against the
15+
# canonical set recorded for this template version. Any mismatch rejects
16+
# the verification with reason `files_modified`.
17+
#
18+
# Files included:
19+
# - the workflow file itself + the two helper scripts
20+
# - .mise.toml
21+
# - every tests/test_stage*.py
22+
#
23+
# Files NOT included: ratelimit/__init__.py (your implementation),
24+
# README.md, LICENSE, pyproject.toml.
25+
26+
set -euo pipefail
27+
28+
if command -v sha256sum >/dev/null 2>&1; then
29+
hash_file() { sha256sum "$1" | awk '{print $1}'; }
30+
elif command -v shasum >/dev/null 2>&1; then
31+
hash_file() { shasum -a 256 "$1" | awk '{print $1}'; }
32+
else
33+
echo "neither sha256sum nor shasum found" >&2
34+
exit 1
35+
fi
36+
37+
declare -a FILES=(
38+
".github/workflows/verify-stages.yml"
39+
".karnstack/compute-hashes.sh"
40+
".karnstack/parse-stages.sh"
41+
".karnstack/sync.sh"
42+
".mise.toml"
43+
)
44+
45+
shopt -s nullglob
46+
for f in tests/test_stage*.py; do
47+
FILES+=("$f")
48+
done
49+
shopt -u nullglob
50+
51+
JSON="{}"
52+
for f in "${FILES[@]}"; do
53+
if [ ! -f "$f" ]; then
54+
echo "missing canonical file: $f" >&2
55+
exit 1
56+
fi
57+
sha=$(hash_file "$f")
58+
JSON=$(echo "$JSON" | jq --arg k "$f" --arg v "$sha" '. + {($k): $v}')
59+
done
60+
61+
echo "$JSON" | jq -S .

‎.karnstack/parse-stages.sh‎

Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
#!/usr/bin/env bash
2+
# Parse `pytest -v` output and emit JSON:
3+
#
4+
# {"passing_stages": [1, 2, 3]}
5+
#
6+
# A stage passes only when every test_stageNN_* function for that stage
7+
# reports PASSED. Any FAILED / ERROR on a test_stageNN_* fails that
8+
# stage.
9+
#
10+
# Portable across bash 3.2 (macOS) and bash 4+ (Linux).
11+
12+
set -euo pipefail
13+
14+
LOG="${1:-/dev/stdin}"
15+
TMP=$(mktemp -t parse-stages.XXXXXX)
16+
trap 'rm -f "$TMP"' EXIT
17+
18+
# pytest -v lines look like:
19+
# tests/test_stage07_foo.py::test_stage07_added_key_is_present PASSED [ 20%]
20+
# tests/test_stage07_foo.py::test_stage07_bit_array_boundary FAILED [ 40%]
21+
#
22+
# Extract one line per (RESULT, NN). Map ERROR -> FAIL (a crashed test is a fail).
23+
grep -E '::test_stage[0-9]{2}_[A-Za-z0-9_]+ +(PASSED|FAILED|ERROR)' "$LOG" \
24+
| sed -E 's/.*::test_stage([0-9]{2})_[A-Za-z0-9_]+ +(PASSED|FAILED|ERROR).*/\2 \1/' \
25+
| sed -E 's/^ERROR/FAIL/; s/^FAILED/FAIL/; s/^PASSED/PASS/' \
26+
> "$TMP" || true
27+
28+
FAILED=$(awk '$1=="FAIL" {print $2}' "$TMP" | sort -u)
29+
SEEN=$(awk '{print $2}' "$TMP" | sort -u)
30+
31+
if [ -z "$SEEN" ]; then
32+
echo '{"passing_stages": []}'
33+
exit 0
34+
fi
35+
36+
PASSING=$(comm -23 <(printf '%s\n' "$SEEN") <(printf '%s\n' "$FAILED"))
37+
38+
if [ -z "$PASSING" ]; then
39+
echo '{"passing_stages": []}'
40+
exit 0
41+
fi
42+
43+
printf '%s\n' "$PASSING" \
44+
| sed -E 's/^0+//' \
45+
| sort -n \
46+
| jq -R 'tonumber' \
47+
| jq -sc '{passing_stages: .}'

‎.karnstack/sync.sh‎

Lines changed: 86 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,86 @@
1+
#!/usr/bin/env bash
2+
# .karnstack/sync.sh - pull canonical files from the karnstack template.
3+
#
4+
# Run this whenever karnstack ships an update to the workflow, helper
5+
# scripts, mise config, or test files and your push starts failing with
6+
# `files_modified` from /api/v1/byox/verify.
7+
#
8+
# Safe by design: only canonical files are touched. Your implementation
9+
# (ratelimit/__init__.py) and any non-canonical edits are left alone.
10+
# The script aborts if you have local uncommitted changes to any canonical
11+
# file, so nothing of yours gets clobbered without you noticing.
12+
#
13+
# Usage:
14+
# bash .karnstack/sync.sh
15+
# mise run sync # same, via mise
16+
#
17+
# Override the upstream for testing:
18+
# TEMPLATE_OVERRIDE=acme/byox-rate-limiter-python bash .karnstack/sync.sh
19+
20+
set -euo pipefail
21+
22+
TEMPLATE="${TEMPLATE_OVERRIDE:-karnstack/byox-rate-limiter-python}"
23+
REMOTE="karnstack-template"
24+
BRANCH="main"
25+
26+
echo "syncing canonical files from ${TEMPLATE}@${BRANCH}"
27+
28+
if ! git remote get-url "$REMOTE" >/dev/null 2>&1; then
29+
git remote add "$REMOTE" "https://github.com/${TEMPLATE}.git"
30+
fi
31+
git fetch --quiet "$REMOTE" "$BRANCH"
32+
33+
declare -a FILES=(
34+
".github/workflows/verify-stages.yml"
35+
".karnstack/compute-hashes.sh"
36+
".karnstack/parse-stages.sh"
37+
".karnstack/sync.sh"
38+
".mise.toml"
39+
)
40+
41+
REMOTE_TESTS=()
42+
while IFS= read -r line; do
43+
[ -n "$line" ] && REMOTE_TESTS+=("$line")
44+
done < <(
45+
git ls-tree -r --name-only "${REMOTE}/${BRANCH}" \
46+
| grep -E '^tests/test_stage[0-9]+_[a-z0-9_]+\.py$' \
47+
|| true
48+
)
49+
if [ "${#REMOTE_TESTS[@]}" -gt 0 ]; then
50+
FILES+=("${REMOTE_TESTS[@]}")
51+
fi
52+
53+
DIRTY=$(git status --porcelain -- "${FILES[@]}" 2>/dev/null | awk '{print $2}')
54+
if [ -n "$DIRTY" ]; then
55+
echo
56+
echo "ERROR: you have uncommitted changes to canonical files:"
57+
echo "$DIRTY" | sed 's/^/ /'
58+
echo
59+
echo "stash or commit them first, then re-run sync."
60+
exit 1
61+
fi
62+
63+
for f in "${FILES[@]}"; do
64+
git checkout "${REMOTE}/${BRANCH}" -- "$f"
65+
echo " synced $f"
66+
done
67+
68+
shopt -s nullglob
69+
LOCAL_TESTS=( tests/test_stage*.py )
70+
shopt -u nullglob
71+
for f in "${LOCAL_TESTS[@]}"; do
72+
found=0
73+
for r in "${REMOTE_TESTS[@]}"; do
74+
if [ "$r" = "$f" ]; then found=1; break; fi
75+
done
76+
if [ "$found" = "0" ]; then
77+
echo " note: $f exists locally but not upstream; review whether to remove"
78+
fi
79+
done
80+
81+
echo
82+
echo "done. review the diff:"
83+
echo " git diff"
84+
echo "then commit and push:"
85+
echo " git commit -am 'sync canonical files from karnstack'"
86+
echo " git push"

‎.mise.toml‎

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
# mise pins the toolchain and exposes karnstack BYOX tasks.
2+
# Install mise first: https://mise.jdx.dev/getting-started.html
3+
#
4+
# Then from this directory:
5+
# mise trust # one-time, allow this config to run
6+
# mise install # installs Python 3.14 + creates .venv
7+
# mise run setup # installs the ratelimit package + pytest into .venv
8+
# mise run stage 1 # runs the tests for stage 1
9+
# mise run all # runs every stage's tests
10+
#
11+
# Stage prefix:
12+
# stage 1 -> test_stage01_*
13+
# stage 2 -> test_stage02_*
14+
# ...
15+
16+
[tools]
17+
python = "3.14"
18+
19+
[env]
20+
_.python.venv = { path = ".venv", create = true }
21+
22+
[tasks.setup]
23+
description = "Install the ratelimit package in editable mode plus test deps."
24+
run = "pip install -e '.[test]'"
25+
26+
[tasks.stage]
27+
description = "Run tests for one stage. Usage: mise run stage <n>"
28+
usage = '''
29+
arg "n" help="Stage number, 1 through 5."
30+
'''
31+
run = '''
32+
stage=$(printf 'stage%02d' "$usage_n")
33+
pytest -v -k "$stage" tests/
34+
'''
35+
36+
[tasks.all]
37+
description = "Run tests for every stage."
38+
run = "pytest -v tests/"
39+
40+
[tasks.sync]
41+
description = "Pull canonical files from the karnstack template."
42+
run = "bash .karnstack/sync.sh"

‎LICENSE‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
MIT License
2+
3+
Copyright (c) 2026 Karnstack
4+
5+
Permission is hereby granted, free of charge, to any person obtaining a copy
6+
of this software and associated documentation files (the "Software"), to deal
7+
in the Software without restriction, including without limitation the rights
8+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9+
copies of the Software, and to permit persons to whom the Software is
10+
furnished to do so, subject to the following conditions:
11+
12+
The above copyright notice and this permission notice shall be included in all
13+
copies or substantial portions of the Software.
14+
15+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21+
SOFTWARE.

0 commit comments

Comments
 (0)