Rework the README and move the guides to the wiki
Turn the README into a lean landing page: what Echo is, highlights, a quick start, the service suite, a three-layer architecture summary, and links into the wiki for the detail. The extended docs (architecture, federation, services, configuration, the directory API, and the module-writing guide) now live as wiki pages, so MODULES.md is retired.
This commit is contained in:
parent
993f5b2eea
commit
973d2826de
2 changed files with 61 additions and 229 deletions
118
MODULES.md
118
MODULES.md
|
|
@ -1,118 +0,0 @@
|
||||||
# Writing a module
|
|
||||||
|
|
||||||
Echo is a small core plus a set of module crates. A module depends on one
|
|
||||||
crate — `echo-api` — and nothing else: not the engine, not the storage, not
|
|
||||||
the network code. That crate carries the traits a module implements and the
|
|
||||||
normalized vocabulary the engine speaks. If your module compiles against
|
|
||||||
`echo-api`, the core can run it.
|
|
||||||
|
|
||||||
There are two kinds of module.
|
|
||||||
|
|
||||||
- A **service** (a pseudo-client like NickServ) implements `Service`.
|
|
||||||
- A **protocol** (an ircd link like InspIRCd) implements `Protocol`.
|
|
||||||
|
|
||||||
Both live in their own crate under `modules/`. `modules/example/` is a complete,
|
|
||||||
minimal service to copy from; `modules/protocol/inspircd/` is the reference protocol.
|
|
||||||
|
|
||||||
## A service, end to end
|
|
||||||
|
|
||||||
**1. The crate.** One dependency:
|
|
||||||
|
|
||||||
```toml
|
|
||||||
# modules/mymod/Cargo.toml
|
|
||||||
[package]
|
|
||||||
name = "echo-mymod"
|
|
||||||
version = "0.0.1"
|
|
||||||
edition = "2021"
|
|
||||||
|
|
||||||
[dependencies]
|
|
||||||
echo-api = { path = "../../api" }
|
|
||||||
```
|
|
||||||
|
|
||||||
**2. The service.** A plain struct implementing `Service`:
|
|
||||||
|
|
||||||
```rust
|
|
||||||
use echo_api::{NetView, Sender, Service, ServiceCtx, Store};
|
|
||||||
|
|
||||||
pub struct MyServ {
|
|
||||||
pub uid: String, // assigned by the daemon
|
|
||||||
}
|
|
||||||
|
|
||||||
impl Service for MyServ {
|
|
||||||
fn nick(&self) -> &str { "MyServ" }
|
|
||||||
fn uid(&self) -> &str { &self.uid }
|
|
||||||
fn gecos(&self) -> &str { "My Service" }
|
|
||||||
|
|
||||||
fn on_command(&mut self, from: &Sender, args: &[&str], ctx: &mut ServiceCtx, net: &dyn NetView, store: &mut dyn Store) {
|
|
||||||
// args[0] is the command; reply by pushing onto ctx.
|
|
||||||
ctx.notice(&self.uid, from.uid, "hello");
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
`on_command` is the whole surface. What you are handed:
|
|
||||||
|
|
||||||
- **`from: &Sender`** — who sent it: `uid`, `nick`, and `account` (set when the
|
|
||||||
user is logged in).
|
|
||||||
- **`args: &[&str]`** — the message split on spaces; `args[0]` is the command.
|
|
||||||
- **`ctx: &mut ServiceCtx`** — the only way to affect anything. You push
|
|
||||||
*intents* and the engine performs them: `notice`, `login` / `logout`,
|
|
||||||
`channel_mode`, `kick`, `topic`, `invite`, `force_nick`, `send_email`, and the
|
|
||||||
deferred `defer_register` / `defer_password` (whose expensive key derivation
|
|
||||||
the engine runs off-thread).
|
|
||||||
- **`net: &dyn NetView`** — a read-only view of the live network: `uid_by_nick`,
|
|
||||||
`nick_of`, `host_of`, `account_of`, `is_op`, `channel_members`, `last_seen`.
|
|
||||||
- **`store: &mut dyn Store`** — the account and channel store. Reads hand back
|
|
||||||
plain views (`AccountView`, `ChannelView`, ...) that carry only non-secret
|
|
||||||
fields — never a password hash or SCRAM verifier. Writes are ordinary methods
|
|
||||||
(`set_email`, `register_channel`, `access_add`, ...). The event log, gossip and
|
|
||||||
credential material are not reachable from here, by design.
|
|
||||||
|
|
||||||
A change you commit through `store` replicates to every other node the same way
|
|
||||||
an IRC-originated one does — you do not write any replication code.
|
|
||||||
|
|
||||||
**3. Register it.** Add the crate to the workspace and to the daemon:
|
|
||||||
|
|
||||||
```toml
|
|
||||||
# Cargo.toml
|
|
||||||
[workspace]
|
|
||||||
members = [..., "modules/mymod"]
|
|
||||||
|
|
||||||
[dependencies]
|
|
||||||
echo-mymod = { path = "modules/mymod" }
|
|
||||||
```
|
|
||||||
|
|
||||||
Construct it in `src/main.rs` alongside the others, behind its config name:
|
|
||||||
|
|
||||||
```rust
|
|
||||||
if enabled("mymod") {
|
|
||||||
services.push(Box::new(echo_mymod::MyServ {
|
|
||||||
uid: format!("{}AAAAAD", cfg.server.sid), // a stable, unique suffix
|
|
||||||
}));
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
**4. Enable it.** In the config:
|
|
||||||
|
|
||||||
```toml
|
|
||||||
[modules]
|
|
||||||
services = ["nickserv", "chanserv", "mymod"]
|
|
||||||
```
|
|
||||||
|
|
||||||
Omitting `[modules]` starts the full standard suite (every pseudo-client). A
|
|
||||||
name that isn't built in is ignored.
|
|
||||||
|
|
||||||
## A protocol module
|
|
||||||
|
|
||||||
A protocol crate implements `Protocol`: it turns raw server-to-server lines into
|
|
||||||
the normalized `NetEvent`s the engine understands, and turns the engine's
|
|
||||||
`NetAction`s back into raw lines. The engine never sees a raw line, so supporting
|
|
||||||
another ircd is one new crate under `modules/protocol/` — see
|
|
||||||
`modules/protocol/inspircd/`. Wire it in `src/main.rs` where `InspIrcd` is
|
|
||||||
constructed.
|
|
||||||
|
|
||||||
## What the SDK deliberately does not give you
|
|
||||||
|
|
||||||
A module cannot reach the append-only log, the gossip layer, the storage engine,
|
|
||||||
or any credential material. It reads through views and writes through curated
|
|
||||||
methods. This is the boundary: a module can be wrong without being dangerous.
|
|
||||||
172
README.md
172
README.md
|
|
@ -1,124 +1,74 @@
|
||||||
# Echo
|
# Echo
|
||||||
|
|
||||||
A federated IRC services daemon in Rust. Protocol-agnostic core, InspIRCd link first.
|
A federated IRC services daemon, written in Rust.
|
||||||
|
|
||||||
Clean-room, inspired by the ideas behind modern event-log/gossip services, not
|
Echo runs the services side of an IRC network, the accounts, channels, and operator tooling, as a set of pseudo-clients (NickServ, ChanServ, and the rest) linked to an ircd over its server-to-server protocol. State is an append-only event log folded into memory, and that log gossips between nodes, so several Echo instances on different servers share one account and channel database with no central store.
|
||||||
derived from any project's source.
|
|
||||||
|
|
||||||
## Design
|
Clean-room work, informed by the ideas behind modern event-log and gossip services, not derived from any project's source.
|
||||||
|
|
||||||
The core lives in `src/`. The pluggable parts are separate crates in a Cargo
|
## Highlights
|
||||||
workspace, each depending only on the `echo-api` SDK crate.
|
|
||||||
|
|
||||||
- **`api/`** (`echo-api`) the module SDK: the `Service`, `Protocol`, `Store`
|
- **Federated.** Every change is an event; nodes exchange logs and converge. Account identity is global across the network, while channel state stays with the node that owns it.
|
||||||
and `NetView` traits a module implements or is handed, plus the normalized
|
- **Event-sourced.** State is a fold over an append-only log. Restart replays it, compaction folds it, replication ships it, all through one code path.
|
||||||
`NetEvent`/`NetAction` vocabulary and the read views. It has no storage or
|
- **Modular.** The core is one crate. Each service and each ircd protocol is a separate crate depending only on the `echo-api` SDK, so a new service or a new ircd is one new crate.
|
||||||
runtime dependencies, so a third-party module builds against it alone.
|
- **Memory-safe.** No manual memory management and no data races, in a domain that has historically shipped both.
|
||||||
- **`src/engine/`** the services engine: live network `state`, the account store,
|
|
||||||
and the log. The store is event-sourced: every change is an `Event` appended to
|
|
||||||
a per-node log, and state is a fold over that log. A service touches it only
|
|
||||||
through the `Store`/`NetView` traits, so the log, gossip and credential material
|
|
||||||
stay out of a module's reach.
|
|
||||||
- **`src/gossip.rs`** node-to-node replication over the logs.
|
|
||||||
- **`modules/`** the loadable modules, each its own crate depending only on the
|
|
||||||
`echo-api` SDK:
|
|
||||||
- **`modules/protocol/inspircd/`** (`echo-inspircd`) the ircd link layer. A
|
|
||||||
`Protocol` impl maps raw server-to-server lines to and from the normalized
|
|
||||||
model, so the engine never touches a raw line and a new ircd is one new crate
|
|
||||||
under `modules/protocol/`.
|
|
||||||
- the pseudo-clients, each a crate implementing `Service` and reaching the
|
|
||||||
network only through the `Store`/`NetView` traits:
|
|
||||||
- **`modules/nickserv/`** — account registration, identification, grouped nicks.
|
|
||||||
- **`modules/chanserv/`** — channel registration, access lists, mode-lock.
|
|
||||||
- **`modules/botserv/`** — per-channel service bots, fantasy commands, kickers.
|
|
||||||
- **`modules/hostserv/`** — virtual hosts, self-serve and operator-approved.
|
|
||||||
- **`modules/memoserv/`** — messages to accounts, delivered whether online or not.
|
|
||||||
- **`modules/operserv/`** — network-operator tools: akills, DEFCON, jupes, sessions, log search.
|
|
||||||
- **`modules/statserv/`** — network and channel statistics.
|
|
||||||
- **`modules/groupserv/`** — user groups (`!group`) whose members inherit channel access.
|
|
||||||
- **`modules/chanfix/`** — reops an opless channel from accumulated op-time.
|
|
||||||
- **`modules/helpserv/`** — a help-desk ticket queue worked by operators.
|
|
||||||
- **`modules/infoserv/`** — network bulletins shown on connect and on login.
|
|
||||||
- **`modules/reportserv/`** — user abuse reports feeding the operator audit trail.
|
|
||||||
- **`modules/diceserv/`** — dice rolls and math for tabletop games.
|
|
||||||
|
|
||||||
Writing your own module — a service or a new ircd protocol — is documented in
|
## Quick start
|
||||||
[MODULES.md](MODULES.md); `modules/example/` is a minimal service to copy from.
|
|
||||||
|
|
||||||
## Replication
|
|
||||||
|
|
||||||
Each log entry carries an `origin`, a per-origin `seq`, and a Lamport clock. Peers
|
|
||||||
connect, exchange version vectors, and send each other the entries the other
|
|
||||||
lacks; a freshly committed entry is also pushed immediately, so a registration
|
|
||||||
propagates in milliseconds. Ingest is idempotent, so re-delivery and reconnect
|
|
||||||
after a split both converge. The log is compacted to one entry per account as it
|
|
||||||
grows. An account registered on any node works on all of them.
|
|
||||||
|
|
||||||
## Directory API (gRPC)
|
|
||||||
|
|
||||||
**`src/grpc.rs`** exposes the account/channel directory over gRPC (see
|
|
||||||
`proto/echo.proto`) so a website can mirror it without touching IRC: a
|
|
||||||
one-shot `Snapshot` for the initial load, then `Subscribe` for a live stream of
|
|
||||||
every change from that point on. It subscribes to the same committed-entry
|
|
||||||
channel gossip does, so a change reaches a subscriber in the same push, no
|
|
||||||
polling. Deliberately excludes anything credential-shaped (password hashes,
|
|
||||||
SCRAM verifiers, cert fingerprints) and the finer channel-ops-list events
|
|
||||||
(access/akick/mlock/entrymsg) — identity and metadata only. Bearer-token
|
|
||||||
authenticated, optional server TLS. Omit `[grpc]` in config.toml to run
|
|
||||||
without it.
|
|
||||||
|
|
||||||
The same port also serves **`Accounts`**, the write side: `Register`,
|
|
||||||
`Authenticate`, `SetPassword`, `SetEmail`, `Confirm`, `Drop`, `ForceLogout`,
|
|
||||||
`GroupNick`, `UngroupNick` — a trusted caller (e.g. a website's own backend,
|
|
||||||
already having done its own login/session check) managing accounts the same
|
|
||||||
way an IRC user does through NickServ, minus the command syntax. The bearer
|
|
||||||
token *is* the authorization: unlike the NickServ commands these mirror, most
|
|
||||||
calls do not re-check the account's own password (an admin-level override, the
|
|
||||||
same trust model a JSON-RPC integration to another services package would
|
|
||||||
use) — `Register` and `Authenticate` are the two exceptions, since the
|
|
||||||
password is the actual input there. A write committed this way replicates to
|
|
||||||
every other Echo node exactly like an IRC-originated one does — gossip
|
|
||||||
doesn't know or care where it came from.
|
|
||||||
|
|
||||||
## Config
|
|
||||||
|
|
||||||
```toml
|
|
||||||
[uplink]
|
|
||||||
host = "127.0.0.1"
|
|
||||||
port = 7000
|
|
||||||
password = "linkpw"
|
|
||||||
|
|
||||||
[server]
|
|
||||||
name = "services.example"
|
|
||||||
sid = "42S"
|
|
||||||
|
|
||||||
[gossip] # omit to run a single, non-replicating node
|
|
||||||
bind = "0.0.0.0:16700"
|
|
||||||
secret = "shared-secret"
|
|
||||||
|
|
||||||
[gossip.tls] # omit for a plaintext link
|
|
||||||
cert = "certs/nodeA.crt"
|
|
||||||
key = "certs/nodeA.key"
|
|
||||||
ca = "certs/ca.crt" # a peer must present a certificate signed by this CA
|
|
||||||
|
|
||||||
[[peer]]
|
|
||||||
addr = "nodeB.example:16700"
|
|
||||||
name = "nodeB" # TLS name to expect; must match the peer certificate
|
|
||||||
```
|
|
||||||
|
|
||||||
Generate certificates with `scripts/gen-certs.sh`.
|
|
||||||
|
|
||||||
## Run
|
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
cp config.example.toml config.toml # edit uplink + link password
|
git clone https://git.devtronic.pro/fedserv/echo.git
|
||||||
cargo run -- config.toml
|
cd echo
|
||||||
|
cp config.example.toml config.toml # set the uplink host and link password
|
||||||
|
cargo run --release -- config.toml
|
||||||
```
|
```
|
||||||
|
|
||||||
|
Echo links to an InspIRCd uplink out of the box. Point `config.toml` at your ircd's server port and add a matching `<link>` block on the ircd side. The full reference is on the [Configuration](https://git.devtronic.pro/fedserv/echo/wiki/Configuration) page.
|
||||||
|
|
||||||
|
## Services
|
||||||
|
|
||||||
|
Every service is a first-class pseudo-client, all started by default.
|
||||||
|
|
||||||
|
| Service | Role |
|
||||||
|
| --- | --- |
|
||||||
|
| NickServ | account registration, identification, grouped nicks, SASL |
|
||||||
|
| ChanServ | channel registration, access lists, mode-lock, auto-op |
|
||||||
|
| BotServ | per-channel bots, fantasy commands, kickers |
|
||||||
|
| HostServ | virtual hosts, self-serve and operator-approved |
|
||||||
|
| MemoServ | messages to accounts, delivered online or not |
|
||||||
|
| OperServ | akills, DEFCON, jupes, session limits, log search |
|
||||||
|
| StatServ | network and channel statistics |
|
||||||
|
| GroupServ | account groups whose members inherit channel access |
|
||||||
|
| ChanFix | reop an opless channel from accumulated op-time |
|
||||||
|
| HelpServ | a help-desk ticket queue for operators |
|
||||||
|
| InfoServ | network bulletins shown on connect and login |
|
||||||
|
| ReportServ | user abuse reports feeding the operator audit trail |
|
||||||
|
| DiceServ | dice and math for tabletop games |
|
||||||
|
|
||||||
|
## Architecture
|
||||||
|
|
||||||
|
Three layers, each a clean boundary:
|
||||||
|
|
||||||
|
- **Protocol** turns raw server-to-server lines into a normalized event and action vocabulary. The engine never sees a raw line, so supporting another ircd is one new crate.
|
||||||
|
- **Engine** holds the live network view and the event-sourced store, and routes messages to services. Services reach it only through the `Store` and `NetView` traits, keeping the log, gossip, and credential material out of a module's reach.
|
||||||
|
- **Services** are pseudo-clients, one crate each, implementing a single `Service` trait.
|
||||||
|
|
||||||
|
The full write-up is in the wiki: [Architecture](https://git.devtronic.pro/fedserv/echo/wiki/Architecture) and [Federation](https://git.devtronic.pro/fedserv/echo/wiki/Federation).
|
||||||
|
|
||||||
|
## Documentation
|
||||||
|
|
||||||
|
The [wiki](https://git.devtronic.pro/fedserv/echo/wiki) has the detail:
|
||||||
|
|
||||||
|
- [Architecture](https://git.devtronic.pro/fedserv/echo/wiki/Architecture) — the engine, the store, the module boundary
|
||||||
|
- [Federation](https://git.devtronic.pro/fedserv/echo/wiki/Federation) — gossip, the global/local scope model, conflict resolution
|
||||||
|
- [Services](https://git.devtronic.pro/fedserv/echo/wiki/Services) — every service and its commands
|
||||||
|
- [Configuration](https://git.devtronic.pro/fedserv/echo/wiki/Configuration) — the `config.toml` reference
|
||||||
|
- [Directory API](https://git.devtronic.pro/fedserv/echo/wiki/Directory-API) — the gRPC account and channel directory for websites
|
||||||
|
- [Writing a Module](https://git.devtronic.pro/fedserv/echo/wiki/Writing-a-Module) — build your own service or ircd link
|
||||||
|
|
||||||
## Status
|
## Status
|
||||||
|
|
||||||
Links to an InspIRCd uplink and runs NickServ (REGISTER, IDENTIFY, LOGOUT, CERT,
|
Echo links to a live InspIRCd uplink and runs the full service suite over an event-sourced, gossip-replicated store. It supports SASL (PLAIN, SCRAM-SHA-256/512, EXTERNAL), an optional mutually-authenticated TLS peer link, an email subsystem for registration confirmation and password recovery, and a gRPC directory API for websites to mirror accounts and channels. It is verified against a real ircd with the irctest compliance suite.
|
||||||
and SASL PLAIN / SCRAM-SHA-256/512 / EXTERNAL) and ChanServ over an event-sourced
|
|
||||||
account store replicated between nodes by gossip, with an optional mutually
|
## License
|
||||||
authenticated TLS peer link, plus a gRPC directory API for websites to mirror
|
|
||||||
the account/channel directory (bearer-token authenticated, optional TLS).
|
Not yet declared. This is original, clean-room work; a license will be set before any public release.
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue