Entity Identifier Resolver

Entity Identifier Resolver (EIR) is a local, strongly typed entity database and search engine written in Rust.

It is designed for applications that need to find an entity from names, metadata, or relationships rather than from a single known identifier.

Status

EIR is under active development. The CLI, storage, and search systems are functional and tested. The server is currently a CLI placeholder.

⚠️ Warning: EIR is not encrypted. Do not use it to store sensitive data.

Entity Model

An EIR database contains entities identified by an EntityID.

An entity can have several aliases, tags, sources, attributes, and relationships:

{
  "id": 1001,
  "aliases": [
    "FizzBerry Spark",
    "FizzBerry",
    "Berry Spark"
  ],
  "tags": [
    "drink",
    "berry"
  ],
  "sources": [
    {
      "provider": "Open Food Facts",
      "verified": true
    }
  ],
  "attributes": [],
  "relationships": []
}

Aliases

Aliases are names that can be used to find an entity.

For example, the same entity might be known as:

FizzBerry Spark
FizzBerry
Berry Spark

EIR can search aliases using exact, prefix, and fuzzy matching.

Sources

Sources record where information about an entity came from.

An entity can have information from several providers, and source information can be used during search and inspection.

Attributes

Attributes describe properties of an entity.

For example:

brand    = FizzBerry
category = Soft Drink
country  = Denmark

Unlike an alias, an attribute describes the entity rather than providing another name for it.

Relationships

Relationships connect one entity to another.

For example:

FizzBerry Spark
      │
      └── manufactured_by ──> FizzBerry Foods

This allows EIR to use information about related entities when resolving a query.

Search

EIR does not use one search algorithm for every query.

A query can be handled by several search operations:

flowchart LR
    QUERY["Query"] --> PLANNER["Planner"]

    PLANNER --> EXACT["Exact Alias"]
    PLANNER --> PREFIX["Prefix Alias"]
    PLANNER --> FUZZY["Fuzzy Alias"]
    PLANNER --> TOKEN["Token"]
    PLANNER --> TAG["Tag"]
    PLANNER --> PROPERTY["Property"]
    PLANNER --> RELATIONSHIP["Relationship"]

    EXACT --> CANDIDATES["Candidates"]
    PREFIX --> CANDIDATES
    FUZZY --> CANDIDATES
    TOKEN --> CANDIDATES
    TAG --> CANDIDATES
    PROPERTY --> CANDIDATES
    RELATIONSHIP --> CANDIDATES

    CANDIDATES --> RANKER["Ranker"]
    RANKER --> RESULTS["Results"]

The search system records which operations produced each candidate. This makes it possible to see why an entity matched a query.

For example, a result might have matched through:

ExactAlias
Token
Tag

rather than simply returning a score with no explanation.

Database

The database contains the entities and the structures needed to search them.

Database
├── Entities
├── Tag Registry
├── Source Registry
├── Attribute-Key Registry
├── Relationship-Type Registry
└── Indexes

Registries assign internal identifiers to repeated values such as tags, source names, attribute keys, and relationship types.

Indexes provide the structures used to find entities efficiently.

The indexes are built from the current database contents and include structures for aliases, prefixes, fuzzy matching, tokens, tags, sources, attributes, and relationships.

Storage

The logical database is stored using persistent storage managed by the EIR engine.

A database has a layout similar to:

database/
├── database.eir
├── eir.toml
├── segments/
└── wal/

The storage backend uses DEIR segments and a write-ahead log.

The WAL records database mutations so they can be recovered if necessary.

Compaction can later rewrite the stored data to remove obsolete storage.

Architecture

The main runtime object is Engine.

flowchart TB
    CLI["eir-cli"]

    ENGINE["Engine"]

    DATABASE["Database"]
    RESOLVER["Resolver"]
    BACKEND["Backend"]

    QUERY["Query"]
    SEARCH["Search"]
    INDEXES["Indexes"]

    WAL["WAL"]
    SEGMENTS["DEIR Segments"]

    CLI --> ENGINE

    ENGINE --> DATABASE
    ENGINE --> RESOLVER
    ENGINE --> BACKEND

    RESOLVER --> QUERY
    QUERY --> SEARCH
    SEARCH --> INDEXES

    BACKEND --> WAL
    BACKEND --> SEGMENTS

    DATABASE --> INDEXES

Engine coordinates the database, resolver, and storage backend.

The database contains the entity data and indexes, while the resolver uses those indexes to perform searches.

Whole System

flowchart TB
    USER["Application / CLI"]

    ENGINE["Engine"]

    DATABASE["Database"]
    ENTITIES["Entity Documents"]
    REGISTRIES["Registries"]
    INDEXES["Search Indexes"]

    QUERY["Query"]
    PLANNER["Planner"]
    EXECUTOR["Executor"]
    RANKER["Ranker"]
    RESULTS["Search Results"]

    STORAGE["Storage Backend"]
    WAL["Write-Ahead Log"]
    SEGMENTS["DEIR Segments"]

    USER --> ENGINE

    ENGINE --> DATABASE
    ENGINE --> STORAGE

    DATABASE --> ENTITIES
    DATABASE --> REGISTRIES
    DATABASE --> INDEXES

    ENGINE --> QUERY
    QUERY --> PLANNER
    PLANNER --> EXECUTOR
    EXECUTOR --> INDEXES
    EXECUTOR --> RANKER
    RANKER --> RESULTS

    STORAGE --> WAL
    STORAGE --> SEGMENTS

This is the overall flow:

Entity data
    │
    ▼
Database
    │
    ├── Registries
    └── Search indexes
             │
             ▼
          Resolver
             │
             ▼
           Query
             │
             ▼
          Results

CLI

The eir CLI provides tools for working with databases.

Current commands include:

init
build
stats
inspect
search
insert
remove
update
compact
merge
server
completions

For example:

cargo eir init data nutrition

cargo eir build \
  --input entities.json \
  --database data/nutrition

cargo eir search \
  data/nutrition/nutrition.eir \
  "FizzBerry"

See docs/cli.md for the complete command reference.

Project Structure

EntityIdentifierResolver/
├── crates/
│   ├── eir-core/
│   ├── eir-cli/
│   └── eir-version/
├── apps/
├── docs/
└── fixtures/

`eir-core`

The core database and search engine.

It contains the entity model, engine, storage, indexing, query, and search systems.

`eir-cli`

The command-line interface built on top of eir-core.

`eir-version`

Version-related functionality used by the database and merge system.

Development

Clone the repository:

git clone https://github.com/HandsOnDigits/EntityIdentifierResolver.git
cd EntityIdentifierResolver

Check the workspace:

cargo check --workspace

Run the tests:

cargo test --workspace

Format the code:

cargo fmt --all

Documentation

docs/cli.md — CLI commands and database operations
docs/model.md — entity model reference

License

See LICENSE for the license.

Design Goals

EIR is designed around a few simple principles:

Local — no remote service required
Structured — entities contain more than just names
Fast — specialized indexes for different search operations
Explainable — search results expose matching signals
Embeddable — the core engine is independent of the CLI
Recoverable — persistent storage uses snapshots and WAL

Entity Identifier Resolver — TODO

🏗️ Core Architecture

Completed

Planned

Stabilize public core API
Document core architecture
Define database compatibility/versioning policy
Improve error model

🔎 Search & Entity Resolution

Completed

Planned

Improve ranking quality
Tune fuzzy matching
Add more query operators
Improve relationship queries
Improve search explanations
Benchmark search performance
Test against larger real-world datasets
Add configurable ranking strategies

💾 Database & Storage

Completed

✏️ Entity Mutations

Completed

🧹 Compaction

Completed

Planned

Add automatic compaction policy
Add configurable compaction thresholds

🔀 Database Merge

Completed

Planned

Support merging more than two databases
Improve merge performance
Add merge conflict strategies
Document registry remapping
Add large-database merge benchmarks

🖥️ CLI

Completed

Planned

🌐 Server / API

Completed

Server command scaffold
Server lifecycle command structure

Planned

📦 Data & Fixtures

Completed

Planned

CSV entity fixtures

🧪 Testing

Completed

Planned

📚 Documentation

Completed

Planned

🚀 Performance

Planned

🔐 Security & Privacy

Current

Local-first architecture
No search history by default
No external service required for core search
No telemetry in the core engine

Planned

🎯 Project Milestones

Phase 1 — Core Engine

Phase 2 — Database Lifecycle

Phase 3 — CLI

Phase 4 — Production Readiness

Phase 5 — API & Applications

HTTP server
API
TypeScript client

License

See the repository for the current license.

The documentation was created with the help of AI, but otherwise, all the code was written by humans.

Name		Name	Last commit message	Last commit date
Latest commit History 284 Commits
.cargo		.cargo
apps		apps
crates		crates
docs		docs
fixtures		fixtures
.gitignore		.gitignore
Cargo.lock		Cargo.lock
Cargo.toml		Cargo.toml
LICENSE		LICENSE
README.md		README.md

Uh oh!

Folders and files

Latest commit

History

Repository files navigation

Entity Identifier Resolver

Status

Entity Model

Aliases

Tags

Sources

Attributes

Relationships

Search

Database

Storage

Architecture

Whole System

CLI

Project Structure

eir-core

eir-cli

eir-version

Development

Documentation

License

Design Goals

Entity Identifier Resolver — TODO

🏗️ Core Architecture

Completed

Planned

🔎 Search & Entity Resolution

Completed

Planned

💾 Database & Storage

Completed

✏️ Entity Mutations

Completed

🧹 Compaction

Completed

Planned

🔀 Database Merge

Completed

Planned

🖥️ CLI

Completed

Planned

🌐 Server / API

Completed

Planned

📦 Data & Fixtures

Completed

Planned

🧪 Testing

Completed

Planned

📚 Documentation

Completed

Planned

🚀 Performance

Planned

🔐 Security & Privacy

Current

Planned

🎯 Project Milestones

Phase 1 — Core Engine

Phase 2 — Database Lifecycle

Phase 3 — CLI

Phase 4 — Production Readiness

Phase 5 — API & Applications

License

About

Topics

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

`eir-core`

`eir-cli`

`eir-version`