OpenLiTGen

An open source implementation of LiTGen (Lightweight Traffic Generator), based on the model described in:

Rolland, C., Ridoux, J., Baynat, B. — "LiTGen, a lightweight traffic generator: application to P2P and mail wireless traffic", 2007. (Full paper available at Docs/LiTGen_a_Lightweight_Traffic_Generator_Application.pdf. It is the ultimate source of truth for the statistical model implemented here — when in doubt about why the code does something, check the paper first.)

LiTGen models network traffic as a hierarchical, per-user statistical process with three levels — session, object, and packet — instead of relying on a full TCP/network-stack emulator. That makes it possible to reproduce realistic traffic burstiness and scaling behavior with a very lightweight generator that anyone can run on a single machine.

OpenLiTGen implements that model as two independent command-line tools:

• litgen-model — learns a traffic model from a real capture (a .pcap file or a .csv export) and produces a .lit model file.
• litgen-tg — generates synthetic traffic from a .lit model file, and ships it to a destination of your choice: the console, a .csv file, a .pcap file, or a real network interface.

flowchart LR
    A[Real traffic capture<br/>.pcap / .csv] --> B(("litgen-model"))
    B --> C[.lit model file]
    C --> D(("litgen-tg"))
    D --> E1[console]
    D --> E2[.csv file]
    D --> E3[.pcap file]
    D --> E4[real network interface]

Loading

---

Table of contents

1. Quickstart tutorial
2. Building the project
3. Usage tutorial
4. Project directory layout
5. Regression tests
6. Architecture overview
7. Standard application workflow
8. Further reading

---

Quickstart tutorial

This is the fastest path from a clean checkout to synthetic traffic on your screen.

# 1. Install system dependencies (once per machine)
cd Sources
./build.sh --install-deps

# 2. Build both binaries
./build.sh --build

# 3. Model traffic from a sample capture shipped with the repo
./bin/litgen-model -p ../Pcap/http_PPI.cap -m "HTTP sample traffic"
# -> produces ./http_PPI.lit in the current directory

# 4. Generate synthetic traffic from that model, straight to your terminal
./bin/litgen-tg -m http_PPI.lit -i console --max-time 10

You should see synthetic packets (arrival time, source/destination IP:port, size) being printed to the console, statistically resembling the traffic captured in http_PPI.cap.

---

Building the project

OpenLiTGen uses CMake under the hood, orchestrated by a single script: Sources/build.sh. This is the only supported way to build the project.

cd Sources
./build.sh --install-deps   # install apt dependencies (build-essential, cmake,
                             # pkg-config, libpcap-dev, libtins-dev)
./build.sh --build           # configure + compile both binaries
./build.sh --clean           # remove all build artifacts (not source files)
./build.sh --help            # full manual: usage + how it works internally

After --build, both executables are available at Sources/bin/:

Sources/bin/litgen-model
Sources/bin/litgen-tg

Run ./build.sh --help for a detailed, didactic explanation of what each option does and how the build works internally (CMake configuration steps, directory layout, dependency installation strategy, etc). That manual is kept in the script itself so it can never go out of sync with the actual build logic.

Dependencies are declared in Sources/libs/apt/deps.txt (one apt package per line) and installed one at a time, fail-fast, by Sources/libs/apt/install.sh — if a dependency fails to install, you'll know exactly which one and why, instead of parsing a big batched apt install log.

---

Usage tutorial

OpenLiTGen supports two independent axes of usage:

• Input to litgen-model: a real .pcap capture, or a .csv export with the same fields (useful when you don't have raw captures handy, or want to hand-craft synthetic traffic to model).
• Output of litgen-tg: console, .csv, .pcap, or a real network interface — chosen automatically from the -i/--interface value.

A) Mocked pipeline (CSV in, console/CSV out)

This is the easiest way to experiment without touching real network interfaces at all.

cd Sources

# Model traffic from a CSV capture (see Sources/fakeTraffic2.lit for the
# resulting model format, and Sources/litgen-tg/testFakeTrafficCsv.csv
# for an example input CSV schema)
./bin/litgen-model -p fakeTraffic2.csv -m "Mocked CSV traffic"

# Generate synthetic traffic from the resulting model, printed to console
./bin/litgen-tg -m fakeTraffic2.lit -i console --max-time 30

# ...or dump it to a CSV file instead
./bin/litgen-tg -m fakeTraffic2.lit -i output.csv --max-time 30

The CSV output/input schema is:

arrivalTime,pktSize,ipSrc,ipDst,transportProtocol,portSrc,portDst,syn,ack,fin,rst

B) Real captures and real interfaces (.pcap in, .pcap/interface out)

cd Sources

# Model traffic straight from a real capture
./bin/litgen-model -p ../Pcap/SkypeIRC.cap -m "Skype P2P traffic sample" -t 300

# Generate synthetic traffic and dump it to a new .pcap file
# (this uses litgen-tg's pcap driver -- packets are crafted with libtins
#  and written with libpcap directly, see Docs/Architecture.md for why)
./bin/litgen-tg -m SkypeIRC.lit -i synthetic_traffic.pcap --max-time 60

# ...or replay it live on a real network interface
sudo ./bin/litgen-tg -m SkypeIRC.lit -i eth0 --max-time 60

Sending on a real interface (-i eth0, -i wlan0, ...) typically requires raw-socket privileges (root, or CAP_NET_RAW). litgen-tg validates the interface name before doing any traffic modelling work (fail-fast): if the interface doesn't exist, you'll get an immediate, clear error instead of a silent no-op.

litgen-model reference

Options:
  -h, --help              Show this help message and exit.
  -v, --version           Show version information and exit.
  -p, --pcap <path>       Path to the input pcap or csv file (mandatory).
  -m, --comment <text>    Human-readable description of the traffic (mandatory).
  -t, --session-threshold <seconds>
                          Threshold to separate sessions (default: 300.00 seconds).

litgen-tg reference

Options:
  --max-time <seconds>     (optional) Maximum time in seconds for the traffic
                            to be generated. Default is 3600 seconds.
  -m, --model <lit-model>  LitGen model to be loaded and used.
  -i, --interface <target> Destination for the generated traffic:
                              - 'console' or 'stdout'  -> print to terminal
                              - a path ending in .csv   -> write a CSV file
                              - a path ending in .pcap  -> write a pcap file
                              - anything else           -> treated as a real
                                                            network interface name
  -v, --version             Show version information and exit.
  -h, --help                 Show this help message and exit.

---

Project directory layout

OpenLiTGen/
├── README.md                 <- you are here
├── LICENSE
├── Docs/                     <- documentation & design references
│   ├── LiTGen_..._Application.pdf   <- the original paper (source of truth for the model)
│   ├── Architecture.md       <- deep dive: components, class hierarchy, extension guide
│   ├── Workflow.md           <- deep dive: end-to-end sequence diagrams, worked examples
│   ├── notes.md              <- personal notes on the LiTGen model levels
│   └── Todo.md / ReleaseNotes.md
├── Pcap/                     <- sample .pcap/.cap captures used for testing
└── Sources/
    ├── build.sh              <- single entry point: install deps / build / clean / help
    ├── litgen-model/         <- traffic MODELLING binary (capture -> .lit model)
    │   ├── CMakeLists.txt
    │   ├── include/          <- BaseSniffer, TinsSniffer, CsvSniffer headers
    │   └── src/               <- their implementations + main.cpp
    ├── litgen-tg/             <- traffic GENERATION binary (.lit model -> traffic)
    │   ├── CMakeLists.txt
    │   ├── include/           <- TgEngine, TgConsole, TgCsv, TgPcap, TgTins, PDU headers
    │   └── src/                <- their implementations + main.cpp
    ├── commons/                <- code shared by BOTH binaries (compiled into each)
    │   ├── include/             <- LitModel, NetworkTraffic, Session, Object, Packet, ...
    │   └── src/
    ├── libs/
    │   ├── cpptools_0.1.0.0/    <- vendored in-house C++ helper library (logging, utils, ...)
    │   └── apt/                  <- system dependency manifest + fail-fast installer
    │       ├── deps.txt
    │       └── install.sh
    └── bin/                       <- final copies of both compiled binaries land here

See Architecture overview below (and Docs/Architecture.md for the full deep dive) for why the code is split this way.

---

Regression tests

Tests/ holds a small, self-contained regression suite that formalizes every behavior validated (and every bug found and fixed) during this project's stabilization work — see each test's header comment for the specific regression it guards against.

./Tests/run_all.sh

Layout:

Tests/
├── run_all.sh        <- runs every test below, in order, prints a summary
├── lib/common.sh      <- shared assertion helpers (sourced by every test)
├── fixtures/           <- small, static, version-controlled input files
│                          (e.g. a hand-crafted .lit model with healthy,
│                          non-zero parameters for every random variable)
├── sandbox/             <- scratch space: compiled binaries + everything
│   └── .gitignore          each test generates. Fully .gitignore'd except
│                            for the .gitignore file itself -- never touch
│                            this directory's contents directly.
├── 00-setup/run.sh        <- builds the project and copies binaries into
│                             sandbox/; every other test depends on this
│                             running first (see Tests/run_all.sh)
├── 01-litgen-model-input-drivers/run.sh
├── 02-litgen-tg-console-output/run.sh
├── 03-litgen-tg-csv-output/run.sh
├── 04-litgen-tg-interface-failfast/run.sh
├── 05-litgen-tg-pcap-output/run.sh
└── 06-createsamples-single-session-no-hang/run.sh

Each Tests/<name>/run.sh is independent and self-contained (it sources lib/common.sh itself), and its header comment explains, in detail, what it checks and — where applicable — exactly which past bug it's a regression guard for.

---

Architecture overview

Both binaries follow the same design pattern: a small abstract base class defines what needs to happen, and one concrete subclass per "driver" defines how it happens for a specific data source/destination. A simple factory (a chain of if/else in each main.cpp, dispatching on a CLI argument) decides, at runtime, which concrete class to instantiate.

classDiagram
    class BaseSniffer {
        <<abstract>>
        +analyze(path, pkts) bool
        +free(pkts) bool
        +echo(pkts) bool
    }
    BaseSniffer <|-- TinsSniffer : reads .pcap (libtins)
    BaseSniffer <|-- CsvSniffer  : reads .csv

    class TgEngine {
        <<abstract>>
        +createSamples(model, timeout)
        +generate(target)* void
    }
    TgEngine <|-- TgConsole : prints to stdout
    TgEngine <|-- TgCsv     : writes .csv
    TgEngine <|-- TgPcap    : writes .pcap (libtins + libpcap)
    TgEngine <|-- TgTins    : sends on a real NIC (libtins)

Loading

litgen-model's side (input drivers, BaseSniffer hierarchy) turns raw packets into a normalized std::vector<PACKET_INFO*>, regardless of where they came from. litgen-tg's side (output drivers, TgEngine hierarchy) turns a generated std::vector<PDU*> into whatever the destination needs, regardless of where they're going. Neither hierarchy needs to know about the other — the .lit model file is the sole contract between them.

Adding a new input driver to litgen-model

1. Create include/MySniffer.h / src/MySniffer.cpp, subclassing BaseSniffer and implementing analyze(path, pkts) (fill pkts with PACKET_INFO*, then call BaseSniffer::userAndFlowIdAssignment(pkts) before returning).
2. Register it in litgen-model/src/main.cpp's dispatch (currently: .csv → CsvSniffer, anything else → TinsSniffer), e.g. add a new .dump/protocol check before the TinsSniffer fallback.
3. That's it — NetworkTraffic, LitModel, and the rest of the pipeline are format-agnostic; they only see PACKET_INFO*.

Adding a new output driver to litgen-tg

1. Create include/MyEngine.h / src/MyEngine.cpp, subclassing TgEngine and implementing generate(target) (iterate this->packetVector, a std::vector<PDU*>, and ship each PDU to your destination).
2. Register it in litgen-tg/src/main.cpp's dispatch (currently: .csv → TgCsv, .pcap → TgPcap, console/stdout → TgConsole, anything else → TgTins).
3. If your driver needs upfront validation (like TgTins validating the network interface exists), do it in the constructor and let it throw — main.cpp wraps engine construction in a try/catch so invalid targets fail fast, before any traffic sample generation happens.

See Docs/Architecture.md for the full breakdown of every class involved (LitModel, NetworkTraffic, Session/Object/Packet, PDU, ExponentialDistribution, ...) and their responsibilities.

---

Standard application workflow

sequenceDiagram
    actor User
    participant Model as litgen-model
    participant Sniffer as BaseSniffer subclass
    participant NT as NetworkTraffic
    participant Lit as LitModel (.lit file)
    participant TG as litgen-tg
    participant Engine as TgEngine subclass

    User->>Model: litgen-model -p capture.pcap -m "comment" -t 300
    Model->>Sniffer: analyze(capture.pcap)
    Sniffer-->>Model: vector<PACKET_INFO*>
    Model->>NT: populateTraffic(pkts, threshold)
    NT-->>Model: users -> sessions -> objects -> packets
    Model->>Lit: calc(netTraffic) + save()
    Lit-->>User: capture.lit written to disk

    User->>TG: litgen-tg -m capture.lit -i target --max-time N
    TG->>Lit: load(capture.lit)
    TG->>Engine: createSamples(model, N)
    Engine-->>TG: vector<PDU*> (synthetic packets)
    TG->>Engine: generate(target)
    Engine-->>User: console / .csv / .pcap / real NIC

Loading

Worked example, end to end:

cd Sources
./build.sh --install-deps
./build.sh --build

# Learn a model from a real capture
./bin/litgen-model -p ../Pcap/http_PPI.cap -m "HTTP traffic" -t 300
#  -> writes ./http_PPI.lit

# Generate synthetic traffic for 60 seconds, as a pcap file
./bin/litgen-tg -m http_PPI.lit -i http_PPI_synthetic.pcap --max-time 60
#  -> writes ./http_PPI_synthetic.pcap, ready to open in Wireshark

---

Further reading

• Docs/LiTGen_a_Lightweight_Traffic_Generator_Application.pdf — the original paper. Source of truth for the statistical model (session/object/packet levels, exponential/heavy-tailed distributions, validation methodology).
• Docs/Architecture.md — deep dive into every class, its responsibility, and how the two binaries relate to each other.
• Docs/Workflow.md — deep dive into the runtime data flow, with more worked examples and edge cases (CSV mocked mode vs. real pcap/NIC mode).
• Docs/notes.md — quick personal notes on the LiTGen levels (in Portuguese).
• Docs/Todo.md — known pending work.