fedserv/echo
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Rewrite the remaining wiki pages

Jean Chevronnet 2026-07-21 14:49:20 +00:00
parent e2df67f24d
commit e9432f4747
No known key found for this signature in database
5 changed files with 78 additions and 101 deletions
Download patch file Download diff file Expand all files Collapse all files

34
Architecture.md

@ -1,26 +1,34 @@
# Architecture # Architecture
Echo is a small core plus a set of module crates. Three layers, each a clean boundary. echo is a small core with a set of module crates around it. Three layers, each a clean boundary: the protocol that talks to the ircd, the engine that holds state and routes messages, and the services that do the work.
## Protocol ## The protocol layer
An ircd link implements the `Protocol` trait. It turns raw server-to-server lines into a normalized `NetEvent` vocabulary the engine understands, and turns the engine's `NetAction`s back into raw lines. The engine never sees a raw protocol line, so supporting another ircd is one new crate. InspIRCd is the first implementation, at `modules/protocol/inspircd`. An ircd link is one crate implementing the `Protocol` trait. It parses raw server-to-server lines into a normalized `NetEvent` vocabulary, and renders the engine's `NetAction`s back into raw lines. The engine never sees a raw line, which is what makes another ircd a single new crate. InspIRCd is the reference, in `modules/protocol/inspircd`; it learns the server's channel modes, extbans, and capabilities from `CAPAB` at link time instead of hardcoding them.
## Engine ## The engine
The engine (`src/engine`) holds the live network view and the account and channel store, and routes a message addressed to a service into that service. It owns the parts a module must not touch: the append-only log, the gossip layer, and all credential material. The engine (`src/engine`) holds the live network view and the account and channel store, and routes a message addressed to a service into that service. It owns the things a module must never touch: the append-only log, the replication layer, and every credential.
A service reaches the engine through exactly two traits: It runs on one thread. An event is handled to completion before the next begins, so there are no locks on the shared state and no data races to reason about. The I/O around it (the uplink, the gRPC API, email, key derivation) runs on Tokio, and the one expensive thing a command can trigger, deriving a SCRAM verifier at registration, is handed off so it never stalls the event loop.
- `NetView`, a read-only view of the live network: who is online, their nick and host, channel membership, who holds ops, last-seen times. ## State as an event log
- `Store`, the account and channel store. Reads return credential-free views; writes are ordinary methods. A write committed here replicates like any other event, with no replication code in the service.
## Store There is no database. Every change is one `Event` appended to a per-node JSON-lines log, and the running state is a fold over that log. A single fold serves three paths: replay on restart, ingest from a peer, and compaction. The code that applies an event on replay is the code that applies it live, so what is on disk and what is in memory cannot drift.
State is event-sourced. Every change is an `Event` appended to a per-node JSONL log, and the in-memory state is a fold over that log. One fold serves three paths: replay on restart, ingest from a peer, and compaction. Passwords are hashed with argon2id and SASL uses stored SCRAM verifiers, none of which is reachable through the `Store` trait. Every committed change is `fsync`'d before its command reports success, so a crash loses nothing. Compaction rewrites the log to a temp file and renames it atomically, so a crash mid-compaction leaves the old log intact.
## Modules Passwords are never stored in the clear. SASL uses stored SCRAM verifiers (PBKDF2), and none of that material is reachable through the traits a service sees.
The pluggable parts are separate crates in a Cargo workspace, each depending only on the `echo-api` SDK and nothing else: not the engine, not the storage, not the network code. If a module compiles against `echo-api`, the core can run it. Services live under `modules/`, one crate per service; ircd links live under `modules/protocol/`. ## Services and the two traits
Each service crate follows one convention: `lib.rs` holds the struct and the command dispatcher, and every command lives in its own file. See [Writing a Module](Writing-a-Module). A service is a pseudo-client in its own crate. It reaches the engine through exactly two traits and nothing else:
- **`NetView`** is a read-only view of the live network: who is online, their nick and host, channel membership, who holds ops, last-seen times.
- **`Store`** is the account and channel store. Reads return credential-free views; writes are ordinary methods. A write committed here replicates like any other event, with no replication code in the service.
That boundary is the point. A service can read and change accounts and channels, but it cannot reach the log, the replication layer, or a password hash. A module can be wrong without being dangerous.
## The workspace
The pluggable parts are separate crates in a Cargo workspace, each depending only on the `echo-api` SDK. Services live under `modules/`, one crate per service; ircd links live under `modules/protocol/`. If a crate compiles against `echo-api`, the core can run it. A service keeps `lib.rs` for the struct and command dispatch, with each command in its own file. See [Writing a Module](Writing-a-Module).

53
Deployment.md

@ -1,8 +1,6 @@
# Deployment # Deployment
Running Echo in production. Echo is the account and channel **authority** for a Running echo in production. echo is the account and channel authority for a network, so the two things that matter most are durability and backups. Both are covered below.
network, so the two things that matter most are durability and backups; both are
covered below.
## Build ## Build
@ -16,42 +14,31 @@ The binary is `target/release/echo`.
## Install ## Install
Run Echo as a dedicated unprivileged user, with `config.toml` and the event log Run echo as a dedicated unprivileged user, with `config.toml` and the event log in one data directory.
in one data directory.
```sh ```sh
useradd --system --home-dir /opt/echo --shell /usr/sbin/nologin echo useradd --system --home-dir /opt/echo --shell /usr/sbin/nologin echo
install -d -o echo -g echo /opt/echo install -d -o echo -g echo /opt/echo
install -o echo -g echo target/release/echo /opt/echo/echo install -o echo -g echo target/release/echo /opt/echo/echo
install -o echo -g echo config.example.toml /opt/echo/config.toml # then edit it install -o echo -g echo config.example.toml /opt/echo/config.toml # then edit it
install -m0644 scripts/echo.service /etc/systemd/system/echo.service # edit paths/User install -m0644 scripts/echo.service /etc/systemd/system/echo.service # edit the paths and User
systemctl daemon-reload systemctl daemon-reload
systemctl enable --now echo systemctl enable --now echo
``` ```
`scripts/echo.service` is a hardened unit (no new privileges, read-only system, `scripts/echo.service` is a hardened unit (no new privileges, read-only system, private tmp, restart on failure). Relax `ProtectSystem=strict` if you use a network mail relay rather than a local sendmail.
private tmp, restart on failure). Adjust `ProtectSystem=strict` for a read-only
root if you use a network mail relay rather than a local sendmail.
## Configure the ircd link ## The ircd link
On the InspIRCd uplink, add a `<link>` block matching `[uplink]` / `[server]` in On the InspIRCd uplink, add a `<link>` block matching `[uplink]` and `[server]` in `config.toml`: the same name, SID, password, and port. echo links as its `[server] name`. The full reference is on the [Configuration](Configuration) page. For a TLS link, set `[uplink] tls = true` and pin the server's SPKI fingerprint.
`config.toml` (name, SID, password, port). Echo links as its `[server] name`.
The full `config.toml` reference is on the [Configuration](Configuration) page.
## Durability ## Durability
Every committed change (registration, password, access, ban, …) is `fsync`'d to Every committed change (registration, password, access, ban) is `fsync`'d to the event log before its command reports success, so a crash or power loss cannot lose it. Compaction rewrites the log to a temp file, `fsync`s, and renames atomically, so a crash during compaction leaves the previous log intact. The log tolerates a truncated final line on load (it is skipped), which is what makes a plain file copy a safe backup.
the event log before the command reports success, so a crash or power loss can
not lose it. Compaction rewrites the log to a temp file, `fsync`s, and renames
atomically, so a crash during compaction leaves the previous log intact. The log
tolerates a truncated final line on load (it is skipped), which is what makes a
plain file copy a safe backup.
## Back up ## Back up
The event log is the whole database. Back it up regularly with The event log is the whole database. Back it up on a schedule with `scripts/backup.sh`, which gzips a timestamped snapshot and prunes old ones:
`scripts/backup.sh` (gzips a timestamped snapshot and prunes old ones):
```ini ```ini
# /etc/systemd/system/echo-backup.service # /etc/systemd/system/echo-backup.service
@ -71,7 +58,7 @@ Persistent=true
WantedBy=timers.target WantedBy=timers.target
``` ```
`systemctl enable --now echo-backup.timer`. Copy backups off-box as well. `systemctl enable --now echo-backup.timer`, and copy backups off-box as well.
## Restore ## Restore
@ -90,26 +77,18 @@ install -o echo -g echo target/release/echo /opt/echo/echo
systemctl restart echo systemctl restart echo
``` ```
The on-disk format is an append-only event log; new event kinds are additive and The on-disk format is an append-only event log: new event kinds are additive, and old logs replay unchanged. Take a backup before upgrading regardless.
old logs replay unchanged. Take a backup before upgrading regardless.
## Monitor ## Monitor
- `systemctl status echo` and `journalctl -u echo -f` (the log records write - `systemctl status echo` and `journalctl -u echo -f`. The log records write failures, compaction, and the account-store load at startup.
failures, compaction, and account-store load at startup). - The optional [`[health]`](Configuration#health) endpoint serves `/health` for a liveness probe and `/metrics` for a Prometheus scrape.
- Optional `[jsonrpc]` HTTP endpoint exposes stats for a status page.
- Systemd `Restart=on-failure` brings it back after a crash. - Systemd `Restart=on-failure` brings it back after a crash.
## Federation (multiple nodes) ## Running more than one node
Give each node a `[gossip]` bind + shared secret and a `[[peer]]` for the others Optional; start with a single node. If you scale out, give each node a `[gossip]` bind and shared secret and a `[[peer]]` for the others, over mutually-authenticated TLS. Account identity replicates to every node; channel state stays local to the node that owns it. See [Federation](Federation).
(mutually-authenticated TLS via `[gossip.tls]`). Account identity replicates to
every node; channel state stays local to the node that owns it. See
[Federation](Federation). Start with a single node until you are comfortable.
## Honest caveats ## Caveats
Echo is pre-1.0. It is memory-safe, panic-free on the wire, and durable, but it echo is pre-1.0. It is memory-safe, does not panic on malformed protocol input, and is durable, but it has not accumulated a long production track record yet. Run it on your own network first, keep backups off-box, and watch the logs.
has not yet accumulated a long production track record. Run it on your own
network first, keep backups, and watch the logs. Declare a license before anyone
else runs it.

27
Directory-API.md

@ -1,20 +1,35 @@
# Directory API (gRPC) # Directory API (gRPC)
`src/grpc.rs` exposes the account and channel directory over gRPC (see `proto/echo.proto`) so a website can mirror it without touching IRC. Omit `[grpc]` in the config to run without it. `src/grpc.rs` exposes echo over gRPC so a website or backend can read the directory and manage accounts without touching IRC. The schema is `proto/echo.proto`. It is off unless you set [`[grpc]`](Configuration#grpc); every call carries a bearer token, and the transport can be TLS.
Four services share the one port.
## Directory (read) ## Directory (read)
Mirror the account and channel directory into your own store.
- `Snapshot` returns the current accounts and channels for the initial load. - `Snapshot` returns the current accounts and channels for the initial load.
- `Subscribe` streams every change from that point on. - `Subscribe` streams every change from that point on.
It subscribes to the same committed-entry channel gossip does, so a change reaches a subscriber in the same push, with no polling. It deliberately excludes anything credential-shaped (password hashes, SCRAM verifiers, cert fingerprints) and the finer channel-ops events. Identity and metadata only. It rides the same committed-entry channel gossip uses, so a change reaches a subscriber in the same push, with no polling. It carries identity and metadata only: anything credential-shaped (password hashes, SCRAM verifiers, cert fingerprints) and the finer channel-ops events are left out.
The usual pattern is `Snapshot` once, then hold `Subscribe` open and apply each event as it arrives.
## Accounts (write) ## Accounts (write)
The same port serves the write side, for a trusted caller such as a website's own backend that has already done its own login or session check: `Register`, `Authenticate`, `SetPassword`, `SetEmail`, `Confirm`, `Drop`, `ForceLogout`, `GroupNick`, `UngroupNick`. This manages accounts the way an IRC user does through NickServ, minus the command syntax. Manage accounts the way a NickServ user does, minus the command syntax, for a trusted backend that has already done its own login or session check.
The bearer token is the authorization. Unlike the NickServ commands these mirror, most calls do not re-check the account's own password (an admin-level override). `Register` and `Authenticate` are the exceptions, since the password is the input there. A write committed this way replicates to every other node exactly like an IRC-originated one. `Register`, `Provision`, `Authenticate`, `SetPassword`, `SetEmail`, `Confirm`, `Drop`, `ForceLogout`, `GroupNick`, `UngroupNick`.
## Auth and transport The bearer token is the authorization. Unlike the NickServ commands these mirror, most calls do not re-check the account's own password (an admin-level override); `Register` and `Authenticate` are the exceptions, since the password is the input there. `Provision` creates an already-confirmed account in one step, for a site that ran its own signup. A write committed this way replicates to every other node exactly like an IRC-originated one.
Bearer-token authenticated, optional server TLS. Pair it with `[auth] external = true` (see [Configuration](Configuration)) to let the website own account identity while Echo owns channels and metadata. Pair this with [`[auth] external = true`](Configuration#auth) to let the website own account identity while echo owns channels, vhosts, and bans, keyed by the account name.
## Stats and Admin
- **Stats** (`Get`) returns the flat, namespaced counter map every service contributes to (the same numbers `/metrics` exposes), for a status page.
- **Admin** (`Version`, `Rehash`, and other operational calls) drives the daemon from a trusted control plane.
## Transport
Every call is bearer-token authenticated. TLS is optional and server-side only (a subscriber is a website backend, not a federation peer, so no client certificate is required). Keep the port on a private or loopback hop.

12
Federation.md

@ -1,24 +1,28 @@
# Federation # Federation
Several Echo nodes on different servers share one account and channel database with no central store. They do it by replicating the event log. Optional. A single node is the usual setup, and most networks never need this. If you run several echo nodes on different servers, they share one account and channel database with no central store, by replicating the event log. Turn it on with [`[gossip]` and `[[peer]]`](Configuration#running-more-than-one-node).
## Gossip ## Gossip
Each log entry carries an origin, a per-origin sequence number, and a Lamport clock. Peers connect, exchange version vectors, and send each other the entries the other is missing. A freshly committed entry is also pushed immediately, so a registration propagates in milliseconds. Ingest is idempotent, so re-delivery and reconnection after a network split both converge. The peer link can run over mutually-authenticated TLS. Each log entry carries an origin, a per-origin sequence number, and a Lamport clock. Peers connect, exchange version vectors, and send each other the entries the other is missing. A freshly committed entry is also pushed straight away, so a registration propagates in milliseconds. Ingest is idempotent, so re-delivery and reconnection after a split both converge on the same state. The peer link can run over mutually-authenticated TLS.
## Global and local scope ## Global and local scope
Not everything replicates. Every event has a scope, and the match on it is exhaustive, so a new event has to pick a side. Not everything replicates. Every event has a scope, and the match on it is exhaustive, so a new event has to pick a side.
- **Global**: account identity. Registration, certificates, password, grouped nicks, and the network-wide operator lists. Gossiped to every node. - **Global**: account identity. Registration, certificates, passwords, grouped nicks, and the network-wide operator lists. Gossiped to every node.
- **Local**: channel state. It stays on the node that owns the channel and never leaves. - **Local**: channel state. It stays on the node that owns the channel and never leaves.
The reason is authority. Account identity is meant to be one thing across the whole network. Channel authority is live and local: a node must not be handed ownership of a channel registered on a network it was never part of. A node refuses to ingest any non-global entry a peer sends it, which is the guarantee behind that rule. The reason is authority. Account identity is meant to be one thing across the whole network. Channel authority is live and local: a node must not be handed ownership of a channel registered on a network it was never part of. A node refuses to ingest any non-global entry a peer sends it, which is the guarantee behind that rule.
## Conflict resolution ## Conflict resolution
If two nodes register the same account name before gossip converges, the log still folds to the same owner on every node. Each account records the node that first registered it; the earliest registration wins, with ties broken deterministically. The merge is commutative and idempotent, so delivery order does not matter. When a peer's registration wins a name a local session was using, that node logs the session out and tells the user, and releases any channels the losing account founded, since they cannot transfer to the name's new owner. If two nodes register the same account name before gossip converges, the log still folds to the same owner on every node. Each account records the node that first registered it; the earliest registration wins, with ties broken deterministically. The merge is commutative and idempotent, so delivery order does not matter. When a peer's registration wins a name a local session was using, that node logs the session out, tells the user, and releases any channels the losing account founded, since they cannot transfer to the name's new owner.
## Compaction ## Compaction
The log is compacted to one entry per account as it grows, so it does not grow without bound. Local channel state is kept for the owning node's own replay and excluded from what peers pull. The log is compacted to one entry per account as it grows, so it does not grow without bound. Local channel state is kept for the owning node's own replay and excluded from what peers pull.
## Signing
By default trust is flat: any peer that holds the shared secret can assert any origin. For a network where nodes do not fully trust each other, per-origin Ed25519 signing (`[gossip.signing]`) lets a node verify that an entry claiming a given origin was really signed by that origin's key. Generate a node's key with `echo --gen-gossip-key` and list each origin's public key under `trust`.

53
Writing-a-Module.md

@ -1,18 +1,8 @@
# Writing a module # Writing a module
Echo is a small core plus a set of module crates. A module depends on one echo is a small core with a set of module crates around it. A module depends on one crate, `echo-api`, and nothing else: not the engine, not the storage, not the network code. That crate carries the traits a module implements and the normalized vocabulary the engine speaks. If your module compiles against `echo-api`, the core can run it.
crate — `echo-api` — and nothing else: not the engine, not the storage, not
the network code. That crate carries the traits a module implements and the
normalized vocabulary the engine speaks. If your module compiles against
`echo-api`, the core can run it.
There are two kinds of module. There are two kinds of module. A **service** (a pseudo-client like NickServ) implements `Service`. A **protocol** (an ircd link like InspIRCd) implements `Protocol`. Both live in their own crate under `modules/`. Copy `modules/example/` for a minimal service, or `modules/protocol/inspircd/` for the reference protocol.
- A **service** (a pseudo-client like NickServ) implements `Service`.
- A **protocol** (an ircd link like InspIRCd) implements `Protocol`.
Both live in their own crate under `modules/`. `modules/example/` is a complete,
minimal service to copy from; `modules/protocol/inspircd/` is the reference protocol.
## A service, end to end ## A service, end to end
@ -52,24 +42,13 @@ impl Service for MyServ {
`on_command` is the whole surface. What you are handed: `on_command` is the whole surface. What you are handed:
- **`from: &Sender`** — who sent it: `uid`, `nick`, and `account` (set when the - **`from: &Sender`** is who sent it: `uid`, `nick`, and `account` (set once the user is logged in).
user is logged in). - **`args: &[&str]`** is the message split on spaces; `args[0]` is the command.
- **`args: &[&str]`** — the message split on spaces; `args[0]` is the command. - **`ctx: &mut ServiceCtx`** is the only way to affect anything. You push intents and the engine performs them: `notice`, `login` and `logout`, `channel_mode`, `kick`, `topic`, `invite`, `force_nick`, `send_email`, and the deferred `defer_register` and `defer_password`, whose expensive key derivation the engine runs off-thread.
- **`ctx: &mut ServiceCtx`** — the only way to affect anything. You push - **`net: &dyn NetView`** is a read-only view of the live network: `uid_by_nick`, `nick_of`, `host_of`, `account_of`, `is_op`, `channel_members`, `last_seen`.
*intents* and the engine performs them: `notice`, `login` / `logout`, - **`store: &mut dyn Store`** is the account and channel store. Reads hand back plain views (`AccountView`, `ChannelView`, ...) carrying only non-secret fields, never a password hash or SCRAM verifier. Writes are ordinary methods (`set_email`, `register_channel`, `access_add`, ...). The log, gossip, and credential material are not reachable from here, by design.
`channel_mode`, `kick`, `topic`, `invite`, `force_nick`, `send_email`, and the
deferred `defer_register` / `defer_password` (whose expensive key derivation
the engine runs off-thread).
- **`net: &dyn NetView`** — a read-only view of the live network: `uid_by_nick`,
`nick_of`, `host_of`, `account_of`, `is_op`, `channel_members`, `last_seen`.
- **`store: &mut dyn Store`** — the account and channel store. Reads hand back
plain views (`AccountView`, `ChannelView`, ...) that carry only non-secret
fields — never a password hash or SCRAM verifier. Writes are ordinary methods
(`set_email`, `register_channel`, `access_add`, ...). The event log, gossip and
credential material are not reachable from here, by design.
A change you commit through `store` replicates to every other node the same way A change you commit through `store` replicates to every other node the same way an IRC-originated one does. You write no replication code.
an IRC-originated one does — you do not write any replication code.
**3. Register it.** Add the crate to the workspace and to the daemon: **3. Register it.** Add the crate to the workspace and to the daemon:
@ -99,20 +78,12 @@ if enabled("mymod") {
services = ["nickserv", "chanserv", "mymod"] services = ["nickserv", "chanserv", "mymod"]
``` ```
Omitting `[modules]` starts the full standard suite (every pseudo-client). A Omitting `[modules]` starts the full standard suite. A name that isn't built in is ignored.
name that isn't built in is ignored.
## A protocol module ## A protocol module
A protocol crate implements `Protocol`: it turns raw server-to-server lines into A protocol crate implements `Protocol`: it turns raw server-to-server lines into the normalized `NetEvent`s the engine understands, and turns the engine's `NetAction`s back into raw lines. The engine never sees a raw line, so supporting another ircd is one new crate under `modules/protocol/`. Start from `modules/protocol/inspircd/` and wire it in `src/main.rs` where `InspIrcd` is constructed.
the normalized `NetEvent`s the engine understands, and turns the engine's
`NetAction`s back into raw lines. The engine never sees a raw line, so supporting
another ircd is one new crate under `modules/protocol/` — see
`modules/protocol/inspircd/`. Wire it in `src/main.rs` where `InspIrcd` is
constructed.
## What the SDK deliberately does not give you ## What the SDK deliberately withholds
A module cannot reach the append-only log, the gossip layer, the storage engine, A module cannot reach the append-only log, the gossip layer, the storage engine, or any credential material. It reads through views and writes through curated methods. That is the boundary: a module can be wrong without being dangerous.
or any credential material. It reads through views and writes through curated
methods. This is the boundary: a module can be wrong without being dangerous.