Goals

Use FlexLib's API source code to create a program that can reliably generate API documentation in markdown format.
Categorize the APIs, creating a separate markdown page for each category. Categories should be defined by a JSON file in the models folder.
Describe the APIs at the TCP command level, including commands and options, responses and know status messages that are sent as a result of the command.
Support hand-written supplementary markdown files that are merged into the final generated document for a given API category, allowing additional context, usage notes, or corrections to be authored manually and preserved across regeneration runs.

Models

Generate the markdown documents using a standard format defined in a file in the models folder. The file example-api-format.md describes this format.
The format model must cover three message types:
- Commands — sent by the client: C[D]<seq>|<command> [parameters]
- Responses — sent by the radio in reply: R<seq>|<result_code>|<data>
- Status messages — unsolicited broadcasts sent by the radio: S<client_handle>|<object_type> [id] <key>=<value> ...
(Future — may be discarded) The example-api-format.md file could be updated to include a "Where:" field breakdown of the status message wire format. However, the radio emits hundreds of distinct status messages with widely varying formats, and it is often impossible to determine whether a given status message is a direct response to a command or a firmware-triggered event. This documentation may instead be deferred to hand-written supplementary text on a per-command basis.

Implementation Language

The program will be implemented in Python.

Reference Documents

The example_docs/ folder contains existing hand-written markdown files covering approximately 30 command categories.
These files serve as an initial reference for the structure and content that the program should produce.
The program now produces output for all 34 categories defined in models/categories.json; the generated pages live in docs/. The example_docs/ files are retained for comparison only and are not regenerated.

Output Directory

The program writes all final generated markdown files directly to docs/.
docs/ is the authoritative output location for the build pipeline; the pipeline does not use a staging folder.
A separate, optional step (src/publish_wiki.py) transforms docs/ into a GitHub-wiki-ready page set under wiki/ for publishing. wiki/ is a transient build artifact and is gitignored. See Wiki Publishing.

Supplementary Content

Supplementary content allows hand-authored notes to be preserved across regeneration runs and merged into the generated pages.

File location and naming

Supplement files live in supplementary/ at the repo root.
Each file is named to match the category output file it augments (e.g., supplementary/Slice-Commands.md for docs/Slice-Commands.md).
Both the supplementary/ folder and the generated docs/ output folder are committed to git. The flexlib/ source tree and the transient wiki/ build folder are not (see .gitignore).

File format

A supplement file is a plain markdown file.
Each ### heading must exactly match a generated command heading (the text after the ## on the generated page), with no backtick wrapper.
Generated headings are derived from the command template with each <placeholder> rendered as ..., and any trailing ... dropped. For example:
- slice tune <index> <freq> → heading ### slice tune ... ...
- slice set <index> active=<active> → grouped heading ### slice set (trailing ... dropped)
When in doubt, run build_docs.py and copy the exact ## ... heading from the generated docs/ page.
Everything under that heading (until the next ### or end of file) is the supplement body for that command.
The supplement body may contain any markdown: prose, code blocks, tables, sub-headings (#### and below).
Within the ### section of a grouped key=value command (e.g. slice set), a #### <key> sub-heading supplies a per-parameter description override for that key, keyed by the parameter name. The override replaces the auto-generated description for that row in the parameter table.

Injection point

Supplement content is injected after the auto-generated command description and before the Syntax block.
If a supplement exists for a command, its full body is inserted at that point verbatim.
If no supplement file exists for a category, or no heading matches a given command, the page renders as if no supplement were present.
A supplement heading that matches no generated command is not silently dropped: the renderer prints a [WARN] supplement heading '<heading>' in <output_file> matched no command; its content was not included. so stale or mistyped headings are caught (e.g. after a FlexLib command is renamed).

Example supplement file (`supplementary/Slice-Commands.md`)

### slice create

Creates a new receiver slice. The radio returns the index of the created slice
in the response data field.

### slice set

Sets one or more operational parameters on the specified slice receiver.

#### active

Only one slice can be active at a time. Setting this to `1` on one slice
automatically deactivates the previously active slice.

Program Design

Entry Point and CLI

The program lives in a src/ folder and is invoked as python src/build_docs.py.
It accepts a --version argument (e.g., --version 4.2.18) that selects the FlexLib source tree under flexlib/v<version>/FlexLib/. Defaults to the only version present if omitted.
It runs the full pipeline on every invocation: extract → intermediate → generate → index.

Pipeline Stages

Stage 1 — Extract

For each source class listed in models/categories.json, open the corresponding .cs file.
Scan for calls to SendCommand(), SendReplyCommand(), and SendCommandAsync() using regex.
Extract the string argument passed to each call. The argument may be:
- A string literal: "slice lock " + _index
- A C# interpolated string: $"display pan set 0x{_streamID:X} pan_position={value}"
- A local variable built in a prior conditional block (harder — best-effort only)
Normalize the extracted string into a command template by replacing C# variable expressions with <placeholder> tokens derived from the variable name where possible (e.g., {_streamID:X} → <streamID>, + _index + → <index>).
The first word(s) of the string form the TCP verb used for category routing.
Scan ParseStatus() and StatusUpdate() methods to extract key names from switch cases, if conditions, and string comparisons (e.g., case "freq":, if (kv == "mode")).
Scan public enum definitions and record member names for use as parameter valid-value lists.

Stage 2 — Categorize and serialize intermediates

Route each extracted command to a category by matching its TCP verb against each category's tcp_verbs list in categories.json.
Write one JSON file per source class to models/intermediate/<ClassName>.json.
Write unmatched commands to models/intermediate/unmatched.json and print a warning per unmatched command.

Stage 3 — Generate markdown

For each category in categories.json, merge the intermediate JSON files listed in source_classes.
Render one markdown page per category to docs/<output_file> using the format defined in models/example-api-format.md.
Each command becomes a ## section. Commands that share a prefix and end in key=<value> form (e.g. all slice set <index> <key>=<value> variants) are consolidated into a single ## section with one parameter table. Parameters and a response example are rendered beneath each section.
Merge any matching per-category supplement from supplementary/<output_file> (see Supplementary Content).
Include a version tag in the page header (e.g., > Generated from FlexLib v4.2.18).

Stage 4 — Generate index

Regenerate docs/Command-Index.md from models/categories.json, grouped by group, with relative links to each output_file.

Parsing Approach

Use regex, not a full C# AST parser. This is intentional: the source patterns are consistent enough for regex, and a full parser would add significant complexity.
String concatenation and interpolation mean many extracted command strings will be partial templates. This is acceptable — the goal is to capture the verb, sub-command, and parameter key names, not to produce executable strings.
Commands built dynamically across multiple lines (e.g., cmd = "slice lock " + _index; if (!_lock) cmd = "slice unlock ...") may only capture one branch. Flag these for manual review in the intermediate JSON using an "extraction_note" field.

Description Generation

The renderer populates the {Description of command} field automatically — no  stubs.
Primary source: scan backwards from each SendCommand call for the nearest /// <summary> XML doc comment block; clean up the text (strip /// markers, collapse whitespace).
Fallback (when no doc comment is found): derive a description from the enclosing property or method name by splitting on camelCase and underscores and capitalising the result (e.g. level_32Hz → "Level 32Hz", EQ_enabled → "EQ enabled").
Store the derived description in the intermediate JSON under "description". The renderer writes it verbatim.

Source Location

All program source lives under src/.
Module breakdown:
- build_docs.py — CLI entry point and pipeline orchestration
- extractor.py — C# source parsing (Send* calls, ParseStatus, enums)
- categorizer.py — verb-to-category routing and intermediate JSON output
- renderer.py — intermediate JSON → markdown page rendering
- supplements.py — per-category supplement file loader (parses ###/#### sections)
- indexer.py — Command-Index.md generation from categories.json
- publish_wiki.py — transforms docs/ into a GitHub-wiki-ready page set (see Wiki Publishing); not part of the build pipeline

Wiki Publishing

The generated docs/ pages are published to this repository's own GitHub wiki (the companion <repo>.wiki.git repository, surfaced under the repo's Wiki tab). Publishing is a separate concern from the build pipeline.

Transform (`src/publish_wiki.py`)

Reads docs/*.md plus models/categories.json and writes a staging directory (wiki/ by default, gitignored).
GitHub wikis are flat and use extensionless links, so the transform:
- rewrites inter-page links from ](Name.md) to ](Name) (preserving any #anchor);
- renames Command-Index.md to Home.md (the wiki landing page) and retargets links to it;
- generates _Sidebar.md from categories.json, grouped like the index, so every wiki page shows category navigation.
Stale .md pages from a previous run are cleared so renames and removals propagate. The transform is idempotent.
Runnable locally for preview: python3 src/publish_wiki.py (writes ./wiki/).

Automation (`.github/workflows/publish-wiki.yml`)

Runs on push to main touching src/, models/, supplementary/, or docs/, and on manual workflow_dispatch.
Rebuilds docs (when the FlexLib source tree is present), runs the transform, then clones the *.wiki.git repo, replaces its pages, and pushes. Commits only when there is a change. A concurrency group serialises wiki pushes.

Required one-time setup

Initialize the wiki: create a first page via the repo's Wiki tab. The *.wiki.git repository does not exist (and cannot be cloned/pushed) until at least one page exists.
WIKI_TOKEN secret: the default GITHUB_TOKEN cannot push to wiki repositories. Add a Personal Access Token (classic repo scope, or fine-grained Contents read/write) as the WIKI_TOKEN Actions secret.
For a public repo, enable Settings → Features → Wikis → "Restrict editing to collaborators only" so the public can read but not edit the wiki.

1. Create a JSON file that represents the command categories and store it in the models folder. (Complete)

The file is models/categories.json. It has been created and contains all current categories.
Each entry in the categories array has the following fields:
- name — display name used in the command index (e.g., "Slice Commands")
- description — one-line description used in the command index link list
- group — section heading in Command-Index.md (e.g., "Core Radio Control")
- output_file — filename written to docs/ (e.g., "Slice-Commands.md")
- source_classes — list of .cs filenames whose commands contribute to this category
- tcp_verbs — list of TCP command verb prefixes (e.g., ["slice"]) used to assign extracted commands to this category

2. Create a program to build documents.

Analyze the FlexLib C# API methods in the following classes:

Core Radio Control

Radio.cs
Slice.cs
Panadapter.cs
Waterfall.cs
Tuner.cs (ATU)
Amplifier.cs
Equalizer.cs
Filter.cs
TNF.cs
Spot.cs
Memory.cs
Meter.cs
Waveform.cs
Xvtr.cs
CWX.cs
DVK.cs
GUIClient.cs
HAAPI.cs

ALE (Automatic Link Establishment)

ALE2G.cs
ALE3G.cs
ALE4G.cs
ALEComposite.cs

USB Cables

UsbCable.cs
UsbBitCable.cs
UsbBcdCable.cs
UsbCatCable.cs
UsbPassthroughCable.cs
UsbLdpaCable.cs
UsbOtherCable.cs

Streams

DAXIQStream.cs
DAXRXAudioStream.cs
DAXTXAudioStream.cs
DAXMICAudioStream.cs
NetCWStream.cs
RXAudioStream.cs
RXRemoteAudioStream.cs
TXRemoteAudioStream.cs

Methods that contain command handling:

SendCommand() — immediate commands (fire and forget)
SendReplyCommand() — commands requiring a response callback
SendCommandAsync() — asynchronous variant used in some classes (e.g., DVK.cs)

Methods that contain status message parsing:

ParseStatus() — parses unsolicited key=value status messages broadcast by the radio
StatusUpdate() — variant of the same pattern used in some classes

These methods reveal the full set of key=value pairs the radio broadcasts for each object type and must be extracted alongside the command strings.

Unmatched commands:

If a SendCommand, SendReplyCommand, or SendCommandAsync call contains a TCP verb that does not match any category's tcp_verbs list, the command must not be silently dropped.
Write all unmatched commands to models/intermediate/unmatched.json using the same schema as other intermediate files, with an additional tcp_verb field showing the unrecognised verb.
The program should also print a warning to stdout for each unmatched command found.

Enum extraction:

Enum definitions in each source file (e.g., AGCMode, InterlockState, PTTSource) represent the valid discrete values for command parameters.
The program should extract enum names and their members and associate them with the parameters that reference them.

3. Create intermediate documents that contain the information found in the APIs.

The program must create the models/intermediate/ folder if it does not exist.
Store intermediate data in models/intermediate/ as JSON files, one per source class.
Each per-class file is an object with these top-level keys:
- source_class — originating class name (e.g. "Slice")
- commands — list of command objects (see below)
- status_keys — key names parsed in ParseStatus/StatusUpdate
- enums — map of enum name → list of member names, used as parameter valid-value lists
Each entry in commands includes:
- template — the TCP command string template, with <placeholder> tokens for variable parts (e.g. "slice set <index> active=<active>")
- verb — the first word of the template, used for category routing
- description — auto-generated description (from the nearest /// <summary> or derived from the member name)
- parameters — list of parameter names extracted from the template's placeholders
- category — the category the command was routed to
- extraction_note — (optional) present only when extraction captured a partial/ambiguous template that warrants manual review
Unmatched commands are written to models/intermediate/unmatched.json using the same command schema plus a tcp_verb field showing the unrecognised verb.

4. Version-tag the generated output.

The FlexLib source at flexlib/v4_2_18/ corresponds to version 4.2.18.
Every generated markdown file should include a version tag in its header indicating the FlexLib version it was produced from.
The version should also be stored in the intermediate JSON files so the full pipeline is traceable.
When new FlexLib versions are added under flexlib/, the program should accept a version argument to target the correct source tree.

5. Add a USB Cable Commands category. (Complete)

USB-Cable-Commands has been added to models/categories.json covering the seven Usb*Cable.cs classes and the usb_cable verb (usb_cable set, usb_cable setbit, usb_cable write).
The corresponding markdown page will be generated by the program on first run.

6. Regenerate the command index.

docs/ is initially empty. docs/Command-Index.md does not pre-exist and will be created by this task on first run.
The program must regenerate docs/Command-Index.md from the category JSON (Task 1) as part of every build run.
Links must use relative paths to local markdown files (e.g., Slice-Commands.md), not absolute or external URLs.

FilesExpand file tree

system_requirements.md

Latest commit

History