How Many Bits per Character? Triangulating the Information Density of the 24 Official EU Languages with Corpora, Neural Translators, and Large Language Models — Manuel Dellabona & Alberto Roggero.
We estimate the information density (bits per character/byte) of all 24 official EU languages three independent ways on identical, professionally translated content (FLORES-200, NTREX-128): (A) corpus statistics and compression, (B) the surprisal of six open LLMs (~577k scored sentences), and (C) the behaviour of one NMT system across all 24×23 = 552 translation directions, scored with chrF++, COMET-22, and reference NLL.
📄 Papers: paper/main.pdf (ACL-style preprint) ·
paper/article.pdf (companion essay for linguists) ·
📋 research plan · 📚 annotated literature
- Density is instrument-independent only under a fixed-meaning design — LLM- and MT-based estimates converge (Spearman ρ = 0.85); corpus statistics on comparable (non-parallel) text correlate with neither (ρ ≤ 0.16).
- The unit decides whether there is a measurement at all — per-token rankings barely replicate across the six models (mean Kendall τ = 0.34); per-byte rankings do (τ = 0.81, Kendall's W = 0.93).
- Written language nearly conserves the information cost of a message — total bits for fixed meaning vary by only 8% across languages under the EU-balanced control model, and density trades off against length with log–log slope −1.11 (the written analogue of the ~39 bits/s speech result).
- Two classic “density” signals are largely instrument artifacts — the “easier out of a dense language than into it” MT asymmetry is strong under chrF++ (β = 2.89, permutation p < 10⁻⁴) but absent under COMET on the same translations; and every LLM prices data scarcity as density (−1.1 to −1.5 z per tenfold fewer resources). After exposure correction, Maltese and Irish drop from the top of the ranking and morphologically rich Estonian, Czech, and Croatian remain — where typology expects them.
configs/ languages.yaml (the 24 languages), models.yaml (instruments,
runnable checkpoints + quantisation actually used)
src/eurodensity/ pipeline package: data/ metrics/ llm/ mt/
scripts/ one CLI per phase (see below)
data/results/ ✅ tracked measurements — every number in the papers
pillar_a.csv corpus/compression metrics (24 langs)
pillar_b*.csv LLM surprisal aggregates + reliability
pillar_b/sentences_*.csv.gz per-sentence surprisal, all 6 models
pillar_b/meta_*.json exact checkpoint / revision / quantisation
pillar_c*.csv 552-direction chrF++/BLEU/COMET/forced-NLL
analysis/ RQ1–RQ4 outputs + resource proxy
data/{raw,processed}/ gitignored (rebuilt by scripts/p2_build_data.py)
paper/ LaTeX sources, vendored ACL style, figures, compiled PDFs
docs/ research plan, literature review, data cards
Python 3.12; versions pinned in requirements.txt.
Reproduction is tiered — each tier needs strictly more hardware.
The tracked measurement records in data/results/ are the primary artifact.
All aggregates, statistics, and figures re-derive from them exactly:
pip install -r requirements.txt
python scripts/p4_pillar_b.py --aggregate-only # pillar_b*.csv from per-sentence records
python scripts/p5_pillar_c.py --aggregate-only # pillar_c*.csv from per-direction records
python scripts/p6_analysis.py # RQ1–RQ4 CSVs + paper figuresWe verify this end-to-end: after running all three commands on a clean
checkout, git status reports no modified CSVs (bit-identical
re-derivation). Figure PDFs embed a creation date; set SOURCE_DATE_EPOCH=0
to make them bit-identical too (PNGs are deterministic regardless). All
stochastic steps (bootstrap, permutation tests, subsampling) use fixed seeds.
python scripts/p2_build_data.py --steps flores,ntrex # downloads + normalises + integrity-checksThe Wikipedia sampling step (--steps wiki, needs datasets) downloads
current dumps: pillar-A raw inputs are date-dependent and not
redistributable, so the committed pillar_a.csv is the canonical record of
the original samples (statistics in docs/DATA_CARDS.md).
| Phase | Command | Wall time (RTX 5070) | Peak disk |
|---|---|---|---|
| P4 pillar B (6 LLMs × 96k sentences) | python scripts/p4_pillar_b.py |
≈ 3.5 h | 15 GB transient |
| P5 matrix (552 directions, beam 4) | python scripts/p5_pillar_c.py --batch-size 64 |
≈ 10 h | 6 GB |
| P5 COMET-22 scoring | .venv_comet/Scripts/python scripts/p5_comet.py |
≈ 1.2 h | 3 GB |
Notes for exact replication:
- Checkpoints. Official Llama/Gemma weights are gated and EuroLLM-9B ships
fp32 (37 GB), so the runs use ungated mirrors / community 4-bit
quantisations, all recorded in
configs/models.yaml(llm_checkpoints:) and, with exact git revisions, indata/results/pillar_b/meta_*.json. Quantisation per model: bf16 (mGPT), bnb-NF4 (Llama/Qwen/Gemma/XGLM), AutoRound-GPTQ-4bit (EuroLLM, loaded via theauto-roundpackage). - COMET runs in a separate venv (
requirements-comet.txt) becauseunbabel-cometpins transformers 4.x / numpy 1.x; theUnbabel/wmt22-comet-dacheckpoint requires a (free) Hugging Face licence acceptance + token. COMET does not cover Maltese (excluded where relevant). - Determinism. GPU float arithmetic is batch-order dependent, so re-measured per-sentence bit values can differ in low decimals; all paper-level statistics are robust to this (cross-domain replication ρ ≥ 0.96, cross-model W = 0.93). Both runs are resumable per model / per direction; per-sentence records are written before aggregation.
tectonic paper/main.tex # 13-page preprint
tectonic paper/article.tex # 8-page companion essayNo system LaTeX needed (tectonic
fetches packages on demand); ACL style files are vendored. The bibliography is
fully source-verified (see docs/LITERATURE.md). To rebuild
the arXiv submission tarball (source + .bbl + figures):
bash paper/make_arxiv.sh.
Code (src/, scripts/, configs/, LaTeX sources) is MIT-licensed; the data
and results (data/results/), figures, and paper text are CC-BY-4.0. See
LICENSE. Upstream corpora, models, and the vendored ACL style
files keep their own licenses.
See CITATION.cff, or:
@misc{dellabona2026eurodensity,
author = {Dellabona, Manuel and Roggero, Alberto},
title = {How Many Bits per Character? Triangulating the Information Density
of the 24 Official EU Languages with Corpora, Neural Translators,
and Large Language Models},
year = {2026},
url = {https://github.com/manudella/EuroDensity}
}This repository previously hosted EuroTranslation (2022–23), a course project by the same authors comparing NMT systems across four European languages (see git history). The present research is independent of it — no data, models, or results are shared; its observation of directional translation asymmetries inspired RQ3.