Evaluations cannot ask a learner to explain: add a short-answer question type

[blueandyellow44]

Beads: not yet created — bd is not installed on the filing machine, see "Three-way circle" below
Plane: not yet created — see below

The gap

NOW-LMS evaluations support exactly two question types. now_lms/forms/__init__.py:129 is the complete set:

return [("multiple", _l("Opción múltiple")), ("boolean", _l("Verdadero/Falso"))]

There is no storage for a written response and no rubric field. Answer (now_lms/db/__init__.py:1067) persists only selected_option_ids, a JSON array of option UUIDs. Question carries text and explanation; both are instructor-facing and neither is a grading criterion.

So the platform can assess recognition and cannot assess explanation. A question can ask a learner to pick the correct answer from a list. It cannot ask a learner to predict what a command will do before running it, justify an architectural choice, or say what they expect to break — and then hold that written answer against stated criteria.

That ceiling binds any practice-based curriculum on this platform. It is also a general limitation rather than an Intent Solutions one: every comparable LMS ships a short-answer or essay question type.

Why it is smaller than it looks

• The whole grading path is four functions in one file (now_lms/vistas/evaluations.py:134, 150, 161, 180).
• Question.type is String(20) with no enumeration constraint (now_lms/db/__init__.py:1022), so a new type value needs no schema change.
• EvaluationAttempt.passed is already nullable=True (now_lms/db/__init__.py:1056), so a pending-grade state is representable in the schema as it stands. It is simply never produced, because take_evaluation scores every attempt synchronously on submit.

Estimated: one additive migration (all columns nullable, no backfill), roughly 150 lines of application code, and three templates — question editor, learner view, instructor grading queue.

This estimate comes from reading the code, not from building it. Treat it as an estimate until the migration exists.

Proposed decision

ADR-6, filed as 000-docs/012-AT-ADEC-short-answer-evaluations.md in the linked PR, status Proposed. It applies ADR-1 rather than excepting it: the capability is generic, so it is authored upstream to bmosoluciones/now-lms rather than carried as a fork patch.

Automated grading of written answers is explicitly out of scope of that ADR. It carries an external model provider, a cost model, and a revision loop the current one-shot attempt model does not support, and it should not ride along on a question-type decision. The proposed interface admits a grader of any kind later without a second migration.

Open questions for the reviewer

1. Upstream first, or fork first? ADR-1 says upstream. Upstream PR fix(db): bootstrap a fresh PostgreSQL database correctly on first boot bmosoluciones/now-lms#179 has been open and waiting on the maintainer for a while, so upstream-first carries real schedule risk. The alternative is carrying the change on deploy/now-lms-fixed while the upstream PR is open and dropping it on acceptance — a temporary divergence rather than a permanent one, which ADR-1 does not currently describe.
2. Multiple short answers per evaluation, or capped at one to bound grading load?
3. Partial credit as a free score, or fixed bands?

Three-way circle — incomplete, needs closing

Per CLAUDE.md every tracked unit exists as bead ↔ GitHub issue ↔ Plane issue, each carrying the other two IDs. This issue was filed from a machine with no bd binary and no .beads/ database (it is local-only and gitignored), and no Plane access. The bead and the Plane issue still need to be created and linked back here. Flagging it in the open rather than leaving the circle silently broken.

[jeremylongshore]

Linked bead: now-lms-dkq (source of truth). Use bd-sync to mirror updates.

[jeremylongshore]

Owner decisions on the three open questions, 2026-08-02. These are the calls; fold them into the ADR and move it Proposed → Accepted.

Q1 — Upstream first, or fork first?

Both, in the pattern this repo just proved: author upstream immediately, carry the same commits on deploy/now-lms-fixed while the upstream PR is open, drop the carry on acceptance.

The schedule-risk argument (PR bmosoluciones#179 languishing) is stale evidence — the maintainer merged ten of our PRs (bmosoluciones#227–bmosoluciones#236) within days this past week. And the fork side is already governed by the closing convention from the merge-down epic: nothing closes as "shipped" until it is on the branch members run. This is exactly how PR #65 landed the enrollment fixes: upstream-first authorship, vendor-branch carry, provenance preserved for the drop. ADR-1 stays intact rather than gaining an exception clause.

Q2 — Multiple short answers per evaluation, or capped at one?

No schema-level cap. Authoring guidance instead: our own courses start with at most one short answer per evaluation until grading throughput is known.

The cap is an operational policy about our three-person grading bandwidth; encoding it into an upstream schema imposes it on every other operator, and no comparable LMS (Moodle, Canvas) caps constructed-response questions structurally. Policies change per cohort; migrations shouldn't.

Q3 — Partial credit: free score or fixed bands?

Free score — a 0–100 float on the graded answer. Bands, if we want them, are grading-UI presets layered on top, never a storage constraint.

This follows the ADR's own design principle: the interface must admit a grader of any kind later without a second migration. Arbitrary-points storage with rubric conventions above it is the established pattern in every major LMS; fixed bands in the column would be a second migration the day a course needs different bands.

---

One addition to carry into the ADR's design: the instructor grading queue produces the pending state through EvaluationAttempt.passed = NULL, which the schema already represents — the ADR should state explicitly that no attempt is auto-failed while awaiting grading, and the flat pass mark (72.0) applies only once every answer is scored.

This issue stays open as the build item (bead now-lms-dkq / LEARN-26). @blueandyellow44 — with these three answered, ADR-6 in PR #37 can go to Accepted and the migration can start on the decided shape.

• Jeremy Longshore
  intentsolutions.io

[jeremylongshore]

bd-sync from now-lms-dkq: ADR-6's three open questions decided by owner 2026-08-02 (full rationale on GH #36): (1) upstream-first WITH a deploy-line carry while the upstream PR is open, PR #65 pattern, carry dropped on acceptance; (2) no schema-level cap on short answers per evaluation - authoring guidance starts our courses at one; (3) partial credit stored as a free 0-100 float, bands are UI presets not storage. Plus: pending attempts (passed=NULL) are never auto-failed. Max asked to fold into PR #37 and flip Proposed -> Accepted; the build starts on the decided shape after that merges.