Translating Marine Mission Reports into PDDL with Large Language Models
- Research Questions
- System Architecture
- Methodology
- Evaluation Framework
- Repository Structure
- Datasets
- Installation
- Usage
- Results
- Acknowledgements
Autonomous Underwater Vehicles (AUVs) and marine research vessels accumulate knowledge through missions documented in Incident Response Plans (IRPs) or cruise reports. These documents describe, in natural language, what equipment was deployed, what tasks were executed, how long each task took, and whether the task succeeded or failed. This accumulated operational knowledge is a rich but largely untapped resource for generating realistic planning scenarios.
Planning Domain Definition Language (PDDL) provides a formal, constraint solver-ready representation of such scenarios. A PDDL domain encodes what actions are possible; a PDDL problem encodes the concrete objects, initial state, and goal. Automated planners (FastForward, LPG-td, Optic-clp, etc.) then find executable action sequences — plans — that transition the system from the initial to the goal state.
MarineLLM-PDDL treats PDDL generation as a code-translation problem: given the free text of a cruise report, an LLM must produce a domain.pddl and problem.pddl pair that correctly captures the mission's types, predicates, actions, preconditions, and effects.
IRP / cruise report text ──► LLM translator ──► domain.pddl + problem.pddl
│
▼
classical planner
│
▼
timed plan
This is a non-trivial translation: PDDL is a formal language with well-defined semantics, while mission reports are narrative prose written for human readers. Success requires the LLM to both understand the mission and express that understanding in formal syntax.
v1.1 is the text-only, LLM-translation slice of the MarineLLM-PDDL pipeline:
- Input modality: free text only (tables, figures, and PDF metadata are deferred to future work).
- Translator: an Interactive Single Prompt LLM stages.
- Evaluation: static validation using VAL tool, Solvability using classical-planners, Correctness Analysis, and Diversity measurement.
v1.1 is organized around two questions about the LLM's role as a translator:
| # | Question | Evaluation |
|---|---|---|
| RQ1 | Does an LLM truly understand code semantics? That is, can it recover the types, predicates, preconditions, and effects of a mission from narrative prose — not just lexical tokens? | Understanding correctness and semantic completeness |
| RQ2 | What is the relationship between code understanding and code generation abilities in LLMs? When the model comprehends a mission, does it reliably produce valid PDDL — and vice versa? | Generation score (parsability, solvability, structural validity) |
┌──────────────────────────────────────────────────────────────────────┐
│ IRP / Cruise Reports │
│ (text content only) │
└──────────────────┬───────────────────────────────────────────────────┘
│
┌────────▼────────┐
│ Text Extractor │ RAG chunking over free text
│ (ε^txt) │
└────────┬────────┘
│ Structured intermediate representation r_i
┌────────▼────────┐
│ RAG Index │ Text chunks embedded into a vector store
│ (Chroma / FAISS)│
└────────┬────────┘
│
┌────────▼────────────────────────────────────────┐
│ LM Translator — Interactive Chain (f) │
│ │
│ [ScenarioQA] ──► 13 generic answers │ understanding
│ ↓ │
│ [BuildGenericScenario] ──► scenario template │ abstraction
│ ↓ │
│ [ExtractSpecificDetails] ──► types/predicates │ formalization
│ ↓ │
│ [GeneratePDDL] ──► domain + problem │ generation
└────────┬────────────────────────────────────────┘
│
┌────────▼────────┐
│ Validator │ VAL (syntax + typing)
└────────┬────────┘
│
┌────────▼────────┐
│ PDDL Planner │ FastForward / LPG-td / Optic-clp
│ (static check) │ (checks solvability only — no execution)
└────────┬────────┘
│
┌────────▼────────┐
│ Evaluation │ Understanding U · Generation G · Q
│ Framework │
└─────────────────┘
Stages (1)–(2) probe Collect LLM responses to 13 Questions; stages (3)–(4) probe code generation.
Chunking. Free-text content of each PDF is split into semantic chunks and embedded into a RAG vector store.
ScenarioQA. For each document, 13 generic questions (vessel, equipment, tasks, durations, outcomes, location, etc.) are posed against the retrieval-augmented context. The answers form the structured intermediate representation
BuildGenericScenario. The QA answers are synthesized into a scenario template; a natural-language summary of the mission's objects, actions, and timeline.
ExtractSpecificDetails. Types, predicates, and action schemata are extracted from the scenario template.
GeneratePDDL. The final stage emits domain.pddl and problem.pddl.
For the curated Q&A datasets, src/template_pddl_generation.py builds a PDDL domain directly from the structured answers of Q1 (vessel name → domain name) and Q5 (task list + durations + outcomes → actions). It supports two input formats:
- Canonical inline (
- **task**: **duration** - **outcome**) — default parser. - Multi-section (separate "Tasks", "Duration", "Success/Failure" sections) — used for NOAA-style answers.
Evaluation metrics is as follows:
| Metric | Definition |
|---|---|
| Parsability | VAL accepts the PDDL ({0, 1}) |
| Solvability | A classical planner returns a plan ({0, 1}) |
| Structural Validity | Fraction of declared types, predicates, and action schemata that are well-formed |
marineLLM-PDDL/
│
├── src/
│ ├── scenario_generation.py RAG Q&A with LangChain + GPT-4o
│ └── template_pddl_generation.py CSV → PDDL template builder
│ (canonical + NOAA multi-section parsers)
│
├── datasets/
│ ├── CuratedQAs/
│ │ ├── Geomar-Kiel/ Q1–Q13 CSV files
│ │ └── NOAA/ Q1–Q13 CSV files
│ ├── FileNames/
│ └── readme.md
│
├── domains/ Generated PDDL domain files
├── results/
│ ├── plans/ Domain + problem + solution files
│ └── diversity/ Similarity metrics
│
|
└── README.md
| Dataset | Source | Documents | Domain |
|---|---|---|---|
| GEOMAR-Kiel | geomar.de research vessels | 51 | AUV, ROV, research vessel |
| NOAA | repository.library.noaa.gov | 30 | Ocean survey, CTD, ADCP |
Each document has curated Q&A answers for 13 generic questions (see paper Table 1).
pip install -r requirements.txtDownload and build at least one of:
export OPENAI_API_KEY="sk-..."python src/scenario_generation.pypython src/template_pddl_generation.pyThis writes one .pddl per parseable row in domains/:
- Geomar-Kiel rows → canonical parser
- NOAA rows → multi-section parser
./lpg-td -o domains/rv_poseidon_domain.pddl \
-f domains/rv_poseidon_problem.pddl \
-n 1 -out results/plans/rv_poseidon| Dataset | Parsable | Solvable | Correct |
|---|---|---|---|
| GEOMAR-Kiel (51 docs) | 96.1% | 80.0% | 60.0% |
| NOAA (30 docs) | 96.6% | 76.6% | 57.0% |
Pairwise distances across 7 sample scenarios show Wasserstein distance best separates semantically distinct plans from near-duplicates (see results/diversity/).
MarineLLM-PDDL is developed at the IT University of Copenhagen as part of the REMARO project.
Mahya Mohammadi Kashani, Stefan Heinrich, Andrzej Wąsowski. MarineLLM-PDDL: Generation of Planning Domains for Marine Vessels Using Past Incident Response Plans. European Robotics Forum (ERF) 2025. SPAR 36, pp. 307–313. DOI: 10.1007/978-3-031-89471-8_47
This project received funding from the European Union's Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No. 956200 REMARO.
