# Complete Knowledge Graph Corpus

---
id: "auth/abac-conditions"
title: "ABAC Conditions"
---

# ABAC Conditions

> **Design RFC**
>
> ABAC conditions (time-of-day, IP, fresh-MFA gates) are part of the planned authorisation stack. Today's Omnitron does not evaluate request-attribute predicates; the policy shapes below are the target design.

When a role + permission check isn't enough — when access also
depends on **when**, **where**, or **how recently authenticated**
the caller is — write an ABAC policy.

A `PolicyDefinition` is a named, async, side-effect-free
function that takes the request's `ExecutionContext` and returns
a `PolicyDecision`.

```ts
interface PolicyDefinition {
  name: string;
  description?: string;
  tags?: string[];
  evaluate: (ctx: ExecutionContext) => Promise<PolicyDecision> | PolicyDecision;
}

interface PolicyDecision {
  allowed: boolean;
  reason?: string;
  metadata?: Record<string, unknown>;
}
```

## Built-in policies

`BuiltInPolicies` ships the common shapes:

| Factory                                        | Purpose                                                                                            |
| ---------------------------------------------- | -------------------------------------------------------------------------------------------------- |
| `requireAuth()`                                | Reject anonymous callers                                                                           |
| `requireRole(role)`                            | Single role required                                                                               |
| `requireAnyRole([…])` / `requireAllRoles([…])` | Combine multiple roles                                                                             |
| `requirePermission(perm)`                      | Single permission required (wildcard-aware)                                                        |
| `requireAnyPermission([…])`                    | Any-of                                                                                             |
| `requireResourceOwner()`                       | `auth.userId === resource.owner`                                                                   |
| `requireTimeWindow(start, end, tz?)`           | Time-of-day window                                                                                 |
| `requireIP([…])` / `blockIP([…])`              | IP allow / block lists                                                                             |
| `requireAttribute(path, value)`                | Arbitrary dotted-path match against the `ExecutionContext` (e.g. `auth.metadata.tier`) `=== value` |
| `requireScope(scope)` / `requireAnyScope([…])` | OAuth2/OIDC scope                                                                                  |
| `rateLimit(maxRequests, windowMs)`             | Per-caller rate limit (policy returns deny when over budget)                                       |
| `requireTenantIsolation()`                     | Multi-tenant invariant: actor's tenant === resource's tenant                                       |
| `requireEnvironment(env)`                      | Only allow in a specific deploy environment                                                        |
| `requireFeatureFlag(flag, enabled?)`           | Gate behind a feature flag                                                                         |

## Custom policies

A custom policy is a plain object — register it on the
PolicyEngine and reference it by name from `@Auth`:

```ts
// my-policies.ts
import type { PolicyDefinition } from '@omnitron-dev/titan/netron/auth';

export const requireRecentMfa = (maxAgeSeconds: number): PolicyDefinition => ({
  name: `recent-mfa:${maxAgeSeconds}s`,
  description: `Caller must have stepped up MFA within the last ${maxAgeSeconds} seconds`,
  tags: ['mfa', 'step-up'],
  evaluate: (ctx) => {
    const stepUpAt = (ctx.auth?.metadata as { stepUpAt?: number } | undefined)?.stepUpAt;
    if (!stepUpAt) return { allowed: false, reason: 'No step-up MFA in session' };
    const ageMs = Date.now() - stepUpAt;
    if (ageMs > maxAgeSeconds * 1000) {
      return { allowed: false, reason: `Step-up MFA expired (${Math.floor(ageMs / 1000)}s ago)` };
    }
    return { allowed: true };
  },
});

// app bootstrap
policyEngine.registerPolicy(requireRecentMfa(300));

// service
@Auth({ policies: ['recent-mfa:300s'] })
async transferLargeAmount(...) { … }
```

## Composition

```ts
// AND (default)
@Auth({ policies: ['role:admin', 'recent-mfa:300s'] })

// OR
@Auth({ policies: { any: ['role:admin', 'role:security'] } })

// NOT
@Auth({ policies: { not: 'block-ip:tor-exits' } })

// Nested
@Auth({
  policies: {
    and: [
      'role:admin',
      { or: ['recent-mfa:300s', 'feature-flag:mfa-bypass'] },
    ],
  },
})
```

The engine short-circuits on the first decisive branch — AND
fails as soon as one leaf denies, OR succeeds as soon as one
allows.

## Failure semantics

`PolicyDecision.reason` is captured into the audit-log alongside
the `false` decision so the operator sees which gate denied and
why. Surface the reason to admins via the same audit-log UI
(see [Audit trail](/docs/auth/audit-trail)); never echo it back to
the caller — it's an information leak.

## Per-call configuration

A few of the built-ins take parameters specific to the gate.
`@Auth({ policies })` resolves **policy name strings** against the
`PolicyEngine` registry — it does not accept inline policy
objects — so register the parametrised factory once at bootstrap,
then reference it by the name the factory generates:

```ts
// bootstrap — register the parametrised instances
policyEngine.registerPolicies([
  BuiltInPolicies.rateLimit(100, 60_000),                                   // name: ratelimit:100/60000
  BuiltInPolicies.requireTimeWindow('09:00', '18:00', 'Europe/Berlin'),     // name: time:09:00-18:00:Europe/Berlin
  BuiltInPolicies.requireFeatureFlag('beta-checkout-v2', true),             // name: feature:beta-checkout-v2:true
]);

// service — reference by generated name
@Auth({
  policies: [
    'ratelimit:100/60000',
    'time:09:00-18:00:Europe/Berlin',
    'feature:beta-checkout-v2:true',
  ],
})
```

Each factory derives its `name` deterministically from its
arguments (see the `BuiltInPolicies` source), so the registered
instance and the `@Auth` reference stay in sync.

## See also

- [Permissions](/docs/auth/permissions) — the simpler permission-only
  gate that covers \~80% of cases.
- [Per-user overrides](/docs/auth/overrides) — additive grants on
  top of role-derived permissions.
- [Audit trail](/docs/auth/audit-trail) — every policy decision is
  audit-logged with the deny reason.

---

---
id: "auth/access-policy"
title: "Shared access policy"
description: "Small, portable roles and permissions shared by issuers, servers and interfaces."
---

# Shared access policy

`@omnitron-dev/access` is the implemented, dependency-free policy core. It runs
in browsers, servers and workers. Use your own role vocabulary and permission
catalog, or import the optional platform preset. Authentication, persistence,
resource ownership and infrastructure remain separate choices.

This guide follows the current source workspace. For a registry release that
includes the package, install it normally:

```bash
pnpm add @omnitron-dev/access
```

Before that release, build and pack it from the framework workspace, then add
the resulting archive to your application and commit its lockfile:

```bash
pnpm --filter @omnitron-dev/access build
pnpm --filter @omnitron-dev/access pack --pack-destination /tmp/access-package
# In the consuming application:
mkdir -p vendor
cp /tmp/access-package/omnitron-dev-access-0.3.1.tgz vendor/
pnpm add ./vendor/omnitron-dev-access-0.3.1.tgz
```

For a reproducible checkout, copy the archive into the application's `vendor/`
directory and use a relative `file:vendor/...tgz` dependency. The framework
workspace and installed registry version can differ; verify the installed
package's `exports` before adopting a new entry point.

## Choose exactly what you need

| Entry point                        | Includes                                                 |
| ---------------------------------- | -------------------------------------------------------- |
| `@omnitron-dev/access`             | Role factory and permission primitives                   |
| `@omnitron-dev/access/roles`       | `createRoleHierarchy`, `RoleDefinition`, `RoleHierarchy` |
| `@omnitron-dev/access/permissions` | Matching, compiled checks, catalog helpers and types     |
| `@omnitron-dev/access/platform`    | Optional `user` → `superadmin` preset                    |
| `@omnitron-dev/titan-auth/access`  | Compatibility facade for the core and platform preset    |

The package ships ESM and declarations with `sideEffects: false`. It imports no
Node built-ins, JWT library, Titan container, React or network client. Narrow
imports and a named import from the root both support tree shaking. Importing
permission helpers does not construct the platform hierarchy.

## Define a role policy

```ts
import { createRoleHierarchy } from '@omnitron-dev/access/roles';

export const rolePolicy = createRoleHierarchy({
  reader: { level: 0 },
  author: { level: 10, inherits: ['reader'] },
  reviewer: { level: 20, inherits: ['reader'] },
  editor: { level: 30, inherits: ['author', 'reviewer'] },
  owner: { level: 40, inherits: ['editor'], assignable: false },
});

rolePolicy.expandRoles('reviewer'); // ['reader', 'reviewer']
rolePolicy.canSeeRole(['editor'], 'editor'); // true: visible peers
rolePolicy.canModerateRole(['editor'], 'editor'); // false: strictly higher rank required
rolePolicy.assignableRolesFor(['owner'], ['reader', 'editor', 'owner']);
// ['reader', 'editor']; nonassignable roles stay out of selection lists
```

Rank and inheritance answer different questions. `level` orders visibility,
moderation and assignment. `inherits` explicitly adds capabilities. A reviewer
has a higher rank than an author but does not inherit authorship. Equal-rank
research and customer roles can stay independent. Inheritance must be acyclic;
unknown parents, invalid ranks and empty definitions fail during construction.

Definitions are copied and expansions are precomputed. Returned arrays cannot
mutate the policy. Independent applications can reuse role names without global
configuration or shared state.

| Operation                                                  | Contract                                                                     |
| ---------------------------------------------------------- | ---------------------------------------------------------------------------- |
| `rolePriority(role)`                                       | Known rank; `-1` for unknown, null or undefined                              |
| `effectiveLevel(roles)`                                    | Highest recognised rank; `-1` when none exists                               |
| `canSeeRole(actorRoles, targetRole)`                       | Known target at or below the actor's rank                                    |
| `canModerateRole(actorRoles, targetRole)`                  | Known target strictly below the actor's rank                                 |
| `visibleRolesFor(actorRoles, candidates)`                  | Filters candidates using visibility                                          |
| `assignableRolesFor(actorRoles, candidates, currentRole?)` | Known, assignable candidates within rank; excludes the supplied current role |
| `expandRoles(role)` / `expandRoleSet(roles)`               | Stable, deduplicated inheritance expansion                                   |
| `priorities`                                               | Frozen, prototype-free lookup for adapters such as SQL generation            |

Unknown roles never become visibility, moderation or assignment targets.
Expansion preserves unknown claims for applications with independent roles;
preserving a string does not establish its authority. Authenticate the principal
before invoking policy and validate persisted grants against your catalog.

## Share a permission catalog

```ts
import {
  definePermission, isValidPermissionKey, createPermissionChecker, resolvePermissions,
} from '@omnitron-dev/access/permissions';

export const catalog = [
  definePermission('workspace.documents.read', 'workspace', 'documents', 'content',
    'Read documents', 'Read documents in the current workspace'),
  definePermission('workspace.documents.edit', 'workspace', 'documents', 'content',
    'Edit documents', 'Change documents in the current workspace', 'high'),
];

isValidPermissionKey('workspace.documents.*', catalog); // true
const allows = createPermissionChecker(['workspace.documents.*'], 'explicit');
allows('workspace.documents.edit'); // true
resolvePermissions(['workspace.documents.*'], catalog, 'explicit');
// ['workspace.documents.read', 'workspace.documents.edit']
```

`PermissionDefinition<Scope>` preserves an application's scope vocabulary.
`domain`, `category`, `label`, `description` and `risk` support administrative
interfaces; they do not change matching. Restrict the catalog to the relevant
scope before validating a new grant. `*` is accepted by the validator, so enforce
who may delegate it separately.

## Select matching semantics explicitly

| Grant → required                    | `explicit`                    | `hierarchical` (default) |
| ----------------------------------- | ----------------------------- | ------------------------ |
| `documents.read` → `documents.read` | true                          | true                     |
| `documents` → `documents.read`      | false                         | true                     |
| `documents.*` → `documents.read`    | true                          | true                     |
| `documents.*` → `documents`         | false                         | false                    |
| `documents.*` → `documents.`        | true, transport compatibility | false                    |
| `document` → `documents.read`       | false                         | false                    |
| `research` → `research:read`        | false                         | false                    |
| `*` → any key                       | true                          | true                     |

Dots define hierarchy; colons are literal characters. There is no general glob
or regex language. Only a full `*` and suffix `.*` have wildcard meaning.
Prefer validated, nonempty concrete keys in an application's catalog.

`permissionGrants` compares one grant. `hasPermission` scans a list for one
required key. `createPermissionChecker` snapshots grants once, uses exact Set
lookup and probes the required key's dot boundaries. Repeated checks therefore
do not scan every grant. Rebuild the checker after changing principal or grants;
do not keep a global checker across requests.

`hasAllPermissions` and `hasAnyPermission` compile once per call. Empty requirements
satisfy all-of and fail any-of. `resolvePermissions` returns recognised effective
keys in catalog order. `expandWildcards` is a compatibility helper: it expands
wildcards but retains concrete grants verbatim, including unregistered ones.

## Compose authentication and enforcement

- With opaque sessions, load the account and grants from your session adapter,
  then evaluate the pure policy. JWT and Titan Auth are unnecessary.
- With the [shared-session preset](/docs/titan/modules/shared-session), pass
  `roleExpander: rolePolicy.expandRoleSet`. Use the same definition during token
  issuance and verification. Keep issuer, session namespace and revocation
  policy in the authentication adapter.
- With Netron's `@Auth` gate, permission matching stays **explicit**. Its existing
  helpers now delegate to this core without changing that transport contract.
- In a UI, use the same catalog and matcher to decide which actions to display.
  Every mutation still requires a server decision against current authority.

Before assigning a role, check that the actor may moderate the target's current
role, validate the requested role against the allowed choices, and enforce
self-edit and tenant rules. A dropdown's choices alone are not an access gate.
For SQL filtering, reject unknown target ranks before comparing levels: `-1 <= 3`
alone would incorrectly expose unknown roles. On policy lookup failure, refuse
access; an empty exclusion list is an unrestricted grant.

Service identities are separate from human role rank. A claim named
`service_role` does not itself authenticate a machine or bypass resource policy.
Only a trusted authentication boundary can establish that identity.

## Optional platform preset and migration

The preset exports `PLATFORM_ROLES`, `PlatformRole`, `ROLE_PRIORITY`,
`rolePriority`, `effectiveLevel`, `canSeeUserWithRole`,
`canModerateUserWithRole`, `visibleRolesFor`, `assignableRolesFor`,
`expandRoles` and `expandRoleSet`.

Its ranks are `user: 0`, `moderator: 1`, `security: 2`, `admin: 3`,
`superadmin: 4`. Security inherits user only; admin combines moderation and
security. The retired `boss` claim expands to `user` for compatibility but has
no independent rank or assignability. Unknown claims pass through expansion
without becoming assignable platform roles.

Existing `titan-auth/access` imports remain supported. New shared UI/server
policy should import `access/permissions` and either `access/roles` or
`access/platform` directly. The meaningful tightening is that unknown target
roles are refused. Existing valid platform roles and permission patterns retain
their behavior. Keep Netron's explicit mode and application hierarchical mode
intentional when migrating.

## Verification and build budget

Run `pnpm --filter @omnitron-dev/access test` and
`pnpm --filter @omnitron-dev/access check:bundle` in the source workspace.
The bundle check compiles browser ESM, rejects external runtime modules and
unwanted role presets, and enforces an 8 KiB minified budget per tested entry.
With esbuild 0.28.2, the permission-checker fixture is 328 bytes minified / 242
bytes gzip; the role-factory fixture is 1,651 / 797 bytes. These are isolated
fixtures, not application bundle sizes or latency measurements.

The regression suite covers independent hierarchies, cycles, prototype-shaped
names, unknown roles, immutable snapshots and exhaustive agreement between
compiled and single-grant matching in both modes. Consumer tests must also
cover tenant ownership, token/session revocation and database enforcement.

---

---
id: "auth/audit-trail"
title: "Audit Trail"
---

# Audit Trail

> **Design RFC**
>
> The structured `audit_logs` + actor-from-JWT discipline described here is part of the planned auth stack. Today's Omnitron emits operator activity into the daemon log only; the typed audit table and the helpers below are reference designs.

Every authorisation-relevant mutation lands in the platform's
`audit_logs` table. The trail is the platform's accountability
substrate — without it, post-incident reconstruction is
impossible on a closed, anonymous platform.

## Shape

```ts
interface NewAuditLog {
  actorId?: string;          // user.id (always from JWT — never from request body)
  actorType: 'user' | 'organization' | 'system' | 'admin';
  action: string;            // dot-separated event name
  targetType?: string;       // 'user' | 'employee' | 'organization' | 'shop' | …
  targetId?: string;
  details?: Record<string, unknown>;
  userAgent?: string;
}
```

Two invariants the platform protects mechanically:

1. **`actorId` is always taken from the verified JWT**
   (`actorId()` helper in `@yourorg/auth-utils`). It is NEVER
   read from a request DTO. This closes the actor-spoofing
   class of bugs (forge the DTO, get someone else's name on
   your action).
2. **The `audit_logs` table is in the STRICT RLS tier**
   (`requireContext: true, allowUnfilteredQueries: false`).
   A raw `sql\`SELECT …\``without an authenticated context
   refuses to return rows — the only legitimate writers are
   service-layer code paths running under a real JWT or the`'system'\` escape hatch (migrations, workers).

## What gets logged

| Surface                                          | Action prefix                                                                                          |
| ------------------------------------------------ | ------------------------------------------------------------------------------------------------------ |
| User suspend / ban / role-change                 | `admin.user.statusChanged`, `admin.user.roleChanged`                                                   |
| Role CRUD (platform)                             | `admin.role.created`, `admin.role.updated`, `admin.role.deleted`                                       |
| Per-user platform-permission overrides           | `admin.user.permissionGranted`, `admin.user.permissionRevoked`, `admin.user.permissionsReplaced`       |
| Per-employee org-permission overrides            | `org.employee.permissionGranted`, `org.employee.permissionRevoked`, `org.employee.permissionsReplaced` |
| Force-transfer org ownership (admin recovery)    | `admin.org.ownership.forceTransferred`                                                                 |
| Org lifecycle (approve / suspend / ban / verify) | `org.approved`, `org.suspended`, `org.banned`, `org.verificationUpdated`                               |
| Shop moderation                                  | `shop.approved`, `shop.suspended`, `shop.banned`, `commerce.shop.*`                                    |
| Disputes                                         | `dispute.moderatorAssigned`, `dispute.resolved`                                                        |
| Auth (privileged actions on a user)              | `auth.password.reset`, `auth.totp.disabled`, `auth.pgp.disabled`, `auth.sessions.revoked`              |

Every admin-side RPC method that mutates a privileged resource
either writes through `AuditLogRepository.log()` directly or
delegates to a service that does (the per-permission services
in [Per-user overrides](/docs/auth/overrides) are the canonical
template).

## Recovery + actor capture invariant

The admin recovery surface
(`OrganizationsService.forceTransferOwnership`) requires the
operator to supply a `reason` (minimum 8 characters) and
captures it into the audit-log alongside the
recovering-admin's id:

```ts
{
  actorId,                  // recovering admin (from JWT)
  actorType: 'admin',
  action: 'admin.org.ownership.forceTransferred',
  targetType: 'organization',
  targetId: orgId,
  details: { fromUserId, toUserId, reason },
}
```

Same pattern applies to other recovery-style actions; the
reason is enforced at the RPC-zod schema level so it never
becomes optional by convention.

## Read surface

The admin UI reads via the `Audit` RPC service:

- `/admin/audit` — global feed, filterable by `action`,
  `targetType`, `targetId`, date range.
- Per-user/per-org audit tabs — same query, scoped to the
  entity.

The read path itself is gated on `admin.audit.view` and
double-protected by the STRICT RLS policy on `audit_logs`:

```ts
audit_logs: {
  policies: [
    allow('read', (ctx) => hasPermission(ctx.auth.permissions ?? [], 'admin.audit.view'), { name: 'adminAuditRead' }),
    deny(['create', 'update', 'delete'], undefined, { name: 'systemOnlyWrite' }),
  ],
  skipFor: ['system'],
  defaultDeny: true,
}
```

`@kysera/rls` `deny(op, condition)` blocks when the condition is
true; called with no condition it **always** denies. Combined
with `skipFor: ['system']`, write operations are refused for
every role except `'system'`, so audit entries can only be
written through the authorised service-layer path — never via a
raw client-issued INSERT.

## Redaction

Free-form `details` blobs are written as-is. Two rules:

1. **Never include passwords, secrets, or tokens.** The
   conventional shape is `{ field: 'password', changed: true }`
   — record that the field changed, not its value.
2. **For free-form text fields** (e.g. `reason`), the value
   stays as the operator typed it. The audit log is internal
   to the platform's administrators.

A redaction sweep over historical entries is a separate
batch job — see the security checklist for the procedure.

## See also

- [Permissions](/docs/auth/permissions) — `admin.audit.view` and the
  registry it sits in.
- [RLS bridge](/docs/auth/rls-bridge) — STRICT tier guarantees on
  `audit_logs`.
- [Mental model](/docs/auth/mental-model) — how audit ties the four
  layers together.

---

---
id: "auth/index"
title: "Authentication & Authorisation"
---

# Authentication & Authorisation

> **Implementation status**
>
> The [shared access policy](/docs/auth/access-policy), Titan JWT/session adapters and Netron authorization gates are implemented. Pages explicitly marked Design RFC describe additional application architecture. The current Omnitron daemon ships JWT issuance, session lifecycle, and the `admin`, `operator` and `viewer` roles (see `OmnitronAuth` in [services-reference](/docs/omnitron/services-reference#omnitronauth--authrpc-servicets)). These pages describe the broader application authorisation design. See [operator authentication](/docs/omnitron/auth-rbac) for the implemented daemon role checks.

The Titan/Omnitron auth surface is split into four layers. Each
layer is independently composable; the page you're reading next
should be picked by what you're trying to do.

| If you want to…                                             | Start here                                                            |
| ----------------------------------------------------------- | --------------------------------------------------------------------- |
| Share roles and permissions between server and UI           | [Shared access policy](/docs/auth/access-policy)                      |
| Verify a JWT and resolve a user                             | [`@titan-auth` JWT primitives](/docs/titan/modules/auth)              |
| Gate an RPC method on a role / permission / policy          | [`@Auth` decorator & PolicyEngine](/docs/titan/netron/authentication) |
| Compose a permission string (grammar, wildcards, hierarchy) | [Permissions](/docs/auth/permissions)                                 |
| Grant a permission to a single user on top of their role    | [Per-user overrides](/docs/auth/overrides)                            |
| Apply attribute-based conditions (time window, MFA, IP)     | [ABAC conditions](/docs/auth/abac-conditions)                         |
| Project the auth context onto a `@kysera/rls` policy        | [RLS bridge](/docs/auth/rls-bridge)                                   |
| Capture every authorisation decision for audit              | [Audit trail](/docs/auth/audit-trail)                                 |
| Get tokens to the client — cookies or bearer, and CSRF      | [Token issuance & transports](/docs/auth/token-issuance)              |
| Understand how the four layers fit together                 | [Mental model](/docs/auth/mental-model)                               |
| Migrate from `@RequireRole(...)` to permission strings      | [Migration: auth v1 → v2](/docs/titan/migrations/auth-1-to-2)         |

## Reading order for first-time integrators

1. **[Mental model](/docs/auth/mental-model)** — one page, one diagram,
   how roles / permissions / conditions / RLS combine.
2. **[Permissions](/docs/auth/permissions)** — grammar, wildcards,
   hierarchical prefix grant. Read once, reference forever.
3. **[`@titan-auth`](/docs/titan/modules/auth)** — set up the
   JWT verifier and the `@Auth` decorator.
4. **[ABAC conditions](/docs/auth/abac-conditions)** when you need
   gates beyond role/permission (time of day, MFA-stepped-up,
   request IP, feature flag).
5. **[RLS bridge](/docs/auth/rls-bridge)** when your gates also need
   to apply at the database layer.
6. **[Audit trail](/docs/auth/audit-trail)** when you need to prove,
   after the fact, that a decision was made the way it was.

Every page links back to the runnable examples in
[`recipes/api-service`](/docs/titan/recipes/api-service) and
[`recipes/multi-tenant-saas`](/docs/titan/recipes/multi-tenant-saas).

---

---
id: "auth/mental-model"
title: "Mental Model"
---

# Mental Model

> **Design RFC**
>
> This page describes the target authorisation model. Omnitron today ships only the base auth surface; the role/permission/condition/RLS layering shown below is a design reference.

```mermaid
flowchart TB
  Token["Verified JWT: sub, roles, permissions, pv, sid, cnf.fp"]
  Counter[("Permission version in Redis")]
  Invocation["Netron invocation wrapper"]
  Context["Admitted AuthContext"]
  Token --> Invocation
  Counter -->|Check permission version| Invocation
  Invocation -->|Admit request| Context
  Context --> Gate["Authorization gate"]
  Context --> RLS["Kysera row-level policies"]
  Context --> Policy["Organization policy"]
  Context --> Service["Service permission checks"]
  Gate --> Audit["Privileged-action audit"]
  RLS --> Audit
  Policy --> Audit
  Service --> Audit
```

Four layers. They share **one** identity (the JWT-resolved
`AuthContext`), **one** permission matcher (`permissionMatches` /
`hasPermission` in `netron/auth/utils.ts`), **one** counter
(`perm-v:{userId}`), and **one** audit sink (`audit_logs`).

## Layer 1 — Roles

A user has one or more roles. Each role names a default
permission set. Roles are coarse — fewer than ten platform-
wide and a handful per organisation. Tier predicates
(`canModerateUserWithRole`) live in `@yourorg/auth-utils/role-hierarchy`
and are the only consumer of role-priority numbers; gates that
care about specific capabilities check **permissions**, not
roles.

## Layer 2 — Permissions

Strings in the canonical registry, matched by `permissionMatches`,
baked into the JWT at mint time. The `.*` wildcard expands to the
set of keys under its prefix (a bare prefix without the star is
*not* a hierarchical grant — it matches only the exact key). The
matcher is the **single source of truth** for "does this granted
set authorise this required permission?" — used unchanged on both
the platform and org scopes, on the server `@Auth` decorator,
on the @kysera/rls policy ctx, and on the frontend
`<PermissionGuard>` / `usePermission()`.

## Layer 3 — ABAC conditions

When role + permission isn't enough, a `PolicyDefinition`
attaches an attribute test: time window, recent step-up MFA,
request IP, feature flag, rate-limit. Built-in factories cover
the common shapes; custom policies are plain objects registered
on the `PolicyEngine`. Composition via `{and|or|not}`.

## Layer 4 — RLS

@kysera/rls policies fire on every DB query and consume the
SAME `AuthContext` that the application-side gates see. The
STRICT tier (`audit_logs`, `platform_roles`, `users`,
`user_sessions`, `organization_roles`) fails closed on a
missing auth context — raw SQL without context cannot leak
through. Other tables run permissive for anonymous reads.

## Live propagation

Every mutation to a user's effective permission set bumps
`perm-v:{userId}`. The invocation wrapper compares the
JWT's `pv` snapshot against the live counter on every call;
mismatch ⇒ refresh-token flow ⇒ new JWT with the updated set.
The propagation window is bounded by one RPC round-trip —
no re-sign-in.

## Why one mental model?

Before the unification:

- The platform-scope matcher (`shared/permissions.ts`) accepted
  only `.*` wildcards.
- The org-scope matcher (`dynamic-policy.service.ts`) walked
  segment wildcards AND honoured hierarchical prefix grant.
- Two registries (`shared/permissions.ts` + org-roles JSON)
  could disagree.
- Three frontend copies of `ADMIN_ROLES = ['admin','superadmin']`
  drifted.
- `audit_logs` had a pass-through RLS policy.
- The `/admin` route gated on a hand-coded role-set that
  silently excluded moderator and security tiers.

Each of those was a real bug. The architecture above closes
them by making the layers share substrate — there is exactly
one place to look when something authorises (or doesn't)
unexpectedly.

## See also

- [Permissions](/docs/auth/permissions) — grammar, matcher, registry.
- [Per-user overrides](/docs/auth/overrides) — additive grants and
  live propagation.
- [ABAC conditions](/docs/auth/abac-conditions) — non-permission gates.
- [RLS bridge](/docs/auth/rls-bridge) — DB-layer enforcement +
  permVer wiring.
- [Audit trail](/docs/auth/audit-trail) — accountability layer.

---

---
id: "auth/overrides"
title: "Per-User Overrides"
---

# Per-User Overrides

> **Design RFC**
>
> Per-user permission overrides on top of role defaults are part of the planned authorisation stack. Today's Omnitron stores `(user, role)` only; the override table and merge rules below are the target design.

A user's **effective permission set** is the union of:

1. The default permissions of every role they hold, AND
2. Any **custom permissions** granted directly to them.

The override layer is strictly additive — overrides can only
ADD; to remove a role-derived permission, change the user's
role rather than negating an individual entry. This keeps the
"resolve" path a single set-union with no surprise subtraction
semantics.

## Why overrides exist

Without overrides, the only way to grant one moderator a
single extra capability — say `admin.security.alerts` — would
be to promote them to the `security` tier, which carries the
full `admin.security.*` bag. Overrides express the narrower
intent without the side effects.

## Two scopes, identical shape

| Scope    | Storage column                          | Service                     |
| -------- | --------------------------------------- | --------------------------- |
| Platform | `users.customPermissions: string[]`     | `UserPermissionService`     |
| Org      | `employees.customPermissions: string[]` | `EmployeePermissionService` |

Each service exposes the same three operations:

```ts
service.grant(targetId, permission, actorId)
service.revoke(targetId, permission, actorId)
service.replaceAll(targetId, permissions, actorId)
```

Every mutation does three things atomically:

1. **Validate** the permission key against the canonical
   registry (`isValidPlatformPermission` /
   `isValidOrgPermission`). Typos refused.
2. **Write an audit-log row** (`actorType: 'admin'`,
   `targetType: 'user'|'employee'`, `details: { permission }`).
3. **Bump the target's permission version** so every live
   access token they hold becomes stale on the next request
   and the refresh flow re-mints with the updated set.

## Resolution

`auth.service.signin` / `refresh` reads both the role-derived
set and the overrides and bakes the union into the JWT:

```ts
const rolePerms = await rbacService.resolvePermissions(user.platformRole);
const overrides = parseUserOverrides(user.customPermissions);
const effective = rbacService.resolveEffectivePermissions(
  user.platformRole,
  overrides,
  rolePerms,
);
const accessToken = await createAccessToken(user.id, user.platformRole, effective, …);
```

The same shape applies in the org scope via
`DynamicPolicyService.loadPermissions`.

## Live propagation

A grant or revoke fires `AuthService.bumpPermissionVersion(userId)`
which increments the `omni:perm-v:{userId}` counter in Redis.
On every authenticated call the invocation wrapper
([RLS bridge → permVer enforcement](/docs/auth/rls-bridge)) compares
the JWT's `pv` claim against the live counter:

- Equal → request proceeds.
- Token snapshot older → `PERMISSION_VERSION_STALE` (401).
  The client refresh-token flow mints a new JWT with the
  up-to-date set; the original request retries automatically.
  No re-sign-in.

The propagation window is bounded by one RPC round-trip.

## Admin RPC surface

The platform-side endpoints live on `UsersService` and are
gated on `admin.users.permissions` (declared `critical` in the
registry — a generic `admin` role does NOT pick it up by
default, the operator grants it explicitly):

```
adminListUserPermissions({ userId }): { permissions: string[] }
adminGrantUserPermission({ userId, permission })
adminRevokeUserPermission({ userId, permission })
adminSetUserPermissions({ userId, permissions: string[] })
```

The org-side endpoints live on `Organizations` and use the
same `org.roles.assign` permission that role assignment uses
(per-employee overrides are the same authority surface as
role assignment within an org):

```
grantEmployeePermission({ employeeId, permission })
revokeEmployeePermission({ employeeId, permission })
setEmployeePermissions({ employeeId, permissions: string[] })
```

## See also

- [Permissions](/docs/auth/permissions) — registry, grammar, matcher.
- [RLS bridge](/docs/auth/rls-bridge) — permVer counter + invocation
  wrapper.
- [Audit trail](/docs/auth/audit-trail) — what each mutation writes.

---

---
id: "auth/permissions"
title: "Permissions"
---

# Permissions

Permission matching is implemented in [the shared access core](/docs/auth/access-policy).
Netron's `@Auth({ permissions })` uses **explicit** matching. Application policies
can choose **hierarchical** matching, the default of `access/permissions`.
The examples on this page describe transport gates and application catalogs;
the supervisor's operator roles remain a separate policy.

## Grammar

```
<scope>.<resource>[.<action>][.<sub>]
```

| Token      | Examples                                   | Notes                                       |
| ---------- | ------------------------------------------ | ------------------------------------------- |
| `scope`    | `admin`, `site`, `commerce`, `org`         | Top-level domain                            |
| `resource` | `users`, `shops`, `employees`              | Entity inside the scope                     |
| `action`   | `list`, `view`, `create`, `edit`, `delete` | Verb on the resource                        |
| `sub`      | `own`, `assigned`                          | Optional refinement (e.g. `posts.edit.own`) |

Example keys for an application-owned registry:

```
admin.users.list
admin.users.ban
admin.users.permissions
admin.orgs.recovery
site.posts.edit.own
org.shops.create
org.employees.invite
```

## Wildcards

Granting a wildcard authorises every key matching the prefix.

| Granted         | Authorises                                                                  |
| --------------- | --------------------------------------------------------------------------- |
| `*`             | Every permission in every scope (reserved for the owner role)               |
| `admin.*`       | Every key starting with `admin.` (`admin.users.list`, `admin.users.ban`, …) |
| `admin.users.*` | `admin.users.list`, `admin.users.ban`, …                                    |

The trailing `.*` is the explicit (and only) wildcard form: the
granted entry must end in `.*` for prefix expansion to apply. A
bare prefix without the star (`admin.users`) in **explicit mode** authorises **only**
the exact key `admin.users` — it does not implicitly grant the
keys below it. Spell out the `.*` when you mean "everything under
here".

## Composition

A method that lists two permissions in `@Auth({permissions})`
requires *both* by default:

```ts
@Auth({ permissions: ['admin.orders.view', 'admin.users.view'] })
```

For *either-of* semantics, register `BuiltInPolicies.requireAnyPermission`
on the `PolicyEngine` and reference it by its generated name. The
`policies` field of `@Auth` takes policy **name strings** (or an
`{ all } / { any }` / `and|or|not` expression of them), never
inline policy objects:

```ts
// bootstrap
policyEngine.registerPolicy(
  BuiltInPolicies.requireAnyPermission(['admin.orders.view', 'admin.users.view']),
);

// service — the factory names itself `permission:any:admin.orders.view,admin.users.view`
@Auth({ policies: ['permission:any:admin.orders.view,admin.users.view'] })
```

See [ABAC conditions](/docs/auth/abac-conditions) for richer
composition (and / or / not, time-of-day, MFA gates).

## Matcher

`permissionMatches(granted, required)` returns true when:

1. `granted === '*'` (full wildcard)
2. `granted === required` (exact match)
3. `granted` ends in `.*` and `required` starts with the prefix
   (e.g. `admin.users.*` covers `admin.users.ban`)

```ts
permissionMatches('admin.*', 'admin.users.ban')          // true
permissionMatches('admin.users.*', 'admin.users.ban')    // true
permissionMatches('admin.users', 'admin.users.ban')      // false (no trailing .*)
permissionMatches('admin.users.list', 'admin.users.ban') // false
permissionMatches('admin.*', 'site.posts.create')        // false
```

The matcher lives in
[`@omnitron-dev/titan/netron/auth`](/docs/titan/netron/authentication)
(`netron/auth/utils.ts`). `hasPermission(grantedPermissions, required)`
applies it across a granted set (true when any granted entry
covers `required`). `@Auth({ permissions })` requires **all** of
the listed permissions, each resolved through this matcher.

## Scopes

Two scopes ship out of the box; both share the matcher, the
registry shape, and the override mechanism — they only differ in
where the grant is stored.

| Scope      | Storage                                               | Decorator                       |
| ---------- | ----------------------------------------------------- | ------------------------------- |
| `platform` | `users.platformRole` + `users.customPermissions`      | `@RequirePlatformPermission(p)` |
| `org`      | `employees.roleIds[]` + `employees.customPermissions` | `@RequireOrgPermission(p)`      |

The org-scope decorator accepts an `OrgIdResolver` so a single
permission applies to a specific organisation extracted from the
call's args:

```ts
@RequireOrgPermission('org.shops.create')                          // default: args[0].organizationId
@RequireOrgPermission('org.shops.update', { argIndex: 0, field: 'orgId' })
@RequireOrgPermission('org.disputes.respond', (args) => args[0].dispute.organizationId)
```

## Validation

Every permission key referenced in a role definition is
validated against the registry at write time:

```ts
isValidPermissionKey('admin.users.ban', PLATFORM_PERMISSIONS) // true
isValidPermissionKey('admin.users.*',   PLATFORM_PERMISSIONS) // true (wildcard, at least one key under it)
isValidPermissionKey('admin.users.lban', PLATFORM_PERMISSIONS) // false — typo refused
```

`createRole` / `updateRole` reject the whole mutation on any
unknown key — the registry is the source of truth and typos
never make it to the persistence layer.

## See also

- [Per-user overrides](/docs/auth/overrides) — additive grants on a
  single user without bumping their role.
- [ABAC conditions](/docs/auth/abac-conditions) — non-permission gates
  (time-window, MFA, IP).
- [Mental model](/docs/auth/mental-model) — how the layers compose.

---

---
id: "auth/rls-bridge"
title: "RLS Bridge"
---

# RLS Bridge

> **Design RFC**
>
> The RLS-bridge stack documented here (wrapper, `permVer` enforcement, generic policy projection) is a planned architecture, not what Omnitron ships today. Treat the code excerpts as reference shapes you'd implement in your own auth-utils package.

The auth layer above gates **calls**. `@kysera/rls` gates
**rows**. They use the same identity — the user's
`AuthContext` — bridged from the Netron invocation wrapper
into the @kysera RLS AsyncLocalStorage scope.

This page covers the wrapper, the policy authoring shape, and
the per-request permission-version (`permVer`) enforcement that
shares the wrapper's hot path.

## The shared wrapper

Every backend in the fleet (e.g. `main`, `accounting`, `storage`,
`messaging`, `analytics`) wires the SAME
`createRlsInvocationWrapper` from `@yourorg/auth-utils`. The
wrapper does **four** things on every authenticated RPC:

```ts
// packages/auth-utils/src/rls-invocation-wrapper.ts
export function createRlsInvocationWrapper(options) {
  return async (metadata, fn) => {
    const authCtx = metadata.get('authContext');
    if (!authCtx) return fn();                               // 1. anon → no RLS scope

    // 2. permVer check (stale-permissions defence)
    if (options.permVerRedis) {
      const tokenPv = authCtx.metadata?.pv ?? 0;
      const currentPv = parseInt(await options.permVerRedis.get(`perm-v:${authCtx.userId}`) ?? '0', 10);
      if (currentPv > tokenPv) {
        throw Object.assign(new Error('Permission version stale'), {
          code: 'PERMISSION_VERSION_STALE',
          statusCode: 401,
        });
      }
    }

    // 3. User-activity touch — debounced Redis marker that
    //    the flusher worker in `main` batches into
    //    `users.last_active_at`. Fire-and-forget; a Redis
    //    hiccup must never delay the request.
    if (options.activityRedis) {
      void touchUserActivity(options.activityRedis, authCtx.userId).catch(() => undefined);
    }

    // 4. RLS context propagation
    const rlsCtx = mapAuthToRLSContext(authCtx, { defaultTenantId: authCtx.metadata?.tenantId ?? 'default' });
    return rlsContext.runAsync(rlsCtx, fn);
  };
}
```

In `bootstrap.ts`:

```ts
auth: {
  jwt: { enabled: true, tokenCacheTtl: 60_000 },
  invocationWrapper: createRlsInvocationWrapper({
    permVerRedis: deferredPermVerRedis.ref,
    activityRedis: deferredPermVerRedis.activityRef,
  }),
}
```

The `deferredPermVerRedis` ref is populated in `afterCreate`
once Redis is resolvable from the DI container; the same
underlying client doubles as both the permVer lookup and the
activity-touch sink.

### User activity

The `touchUserActivity` helper performs an atomic
`SET omni:activity:dirty:{userId} <iso-now> NX EX 60` — at most
one Redis write per user per 60 seconds, regardless of request
volume across all five backends. A dedicated
`UserActivityFlusherService` in `main` runs on a 30-second
interval to SCAN the dirty namespace, batch-update
`users.last_active_at` with a `GREATEST()`-clamped UPDATE FROM
VALUES (monotonic and idempotent), then UNLINK the keys.

Worst-case lag from "user did something" to "`users.last_active_at`
advances" is `windowSeconds + flushIntervalMs` ≈ 90 seconds.
No new inter-backend RPC traffic — each backend writes to the
shared Redis namespace; only `main` reads.

## RLS policies

A policy is a per-table allow/deny/filter that runs inside the
plugin BEFORE every SELECT / INSERT / UPDATE / DELETE. The
`defineRLSSchema`, `allow`, `deny`, `filter` builders are the
real `@kysera/rls` API, re-exported from
`@omnitron-dev/titan-database/rls`. The condition function
receives a `PolicyEvaluationContext`: `ctx.auth` for identity and
`ctx.row` (read/update/delete) or `ctx.data` (create) for the
record under evaluation:

```ts
// rls-schema.ts
import { defineRLSSchema, allow } from '@omnitron-dev/titan-database/rls';

export const platformRLSSchema = defineRLSSchema<Database>({
  users: {
    policies: [
      allow('read', () => true, { name: 'publicProfiles' }),
      allow('update', (ctx) => ctx.auth.userId === ctx.row?.id, { name: 'ownUpdate' }),
      allow('update', (ctx) => {
        const targetRole = (ctx.row?.platformRole as string) ?? 'user';
        return canModerateUserWithRole(ctx.auth.roles, targetRole);
      }, { name: 'tieredAdminUpdate' }),
    ],
    skipFor: ['system'],
    defaultDeny: false,
  },
  // …
});
```

`ctx.auth` (an `RLSAuthContext`) carries `userId`, `roles`,
`permissions`, `tenantId`, `attributes`, and `isSystem` — the
bridge populates it from the application's `AuthContext` so the
DB layer sees the same identity the call-side gates check.

## STRICT vs PERMISSIVE plugin tiers

A single plugin instance with `allowUnfilteredQueries: true`
historically reduces RLS to an advisory hint. The
schema is split into two tiers via two plugin instances:

```ts
// modules/rbac/rls-plugins.ts
const strict = rlsPlugin({
  schema: platformRLSSchema,
  bypassRoles: ['superadmin'],
  requireContext: true,         // fail closed on missing context
  allowUnfilteredQueries: false, // fail closed on missing filters
});

const permissive = rlsPlugin({
  schema: platformRLSSchema,
  bypassRoles: ['superadmin'],
  requireContext: false,         // anonymous reads OK
  allowUnfilteredQueries: true,
});

for (const t of STRICT_RLS_TABLES) registerTablePlugins(t, […existing, strict]);
for (const t of PERMISSIVE_RLS_TABLES) registerTablePlugins(t, […existing, permissive]);
```

| Tier       | Tables                                                                                                        | Behaviour                                                                 |
| ---------- | ------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------- |
| STRICT     | `audit_logs`, `platform_roles`, `users`, `user_sessions`, `organization_roles`                                | Queries without auth context → 0 rows / refused. `'system'` escape hatch. |
| PERMISSIVE | `posts`, `comments`, `organizations`, `employees`, `shops`, `products`, `orders`, `notifications`, `disputes` | Anonymous public reads work. Policies apply to authenticated callers.     |

A `'system'` skipFor is built into every policy as the legitimate
escape for migrations, background workers, and explicit
service-layer transactions.

## permVer counter

The `perm-v:{userId}` Redis key is bumped by any code path
that mutates a user's effective permission set:

```ts
authService.bumpPermissionVersion(userId)     // single bump (INCR + EXPIRE)
authService.bumpPermissionVersionMany(userIds) // batched pipeline
```

Bump sites (production):

| Mutation                                  | Caller                                                |
| ----------------------------------------- | ----------------------------------------------------- |
| Platform role change (`adminUpdateRole`)  | `UsersService.adminUpdateRole`                        |
| Per-user platform override grant / revoke | `UserPermissionService.{grant,revoke,replaceAll}`     |
| Per-employee org override grant / revoke  | `EmployeePermissionService.{grant,revoke,replaceAll}` |
| Force-transfer org ownership              | `OrganizationsService.forceTransferOwnership`         |

The JWT carries the snapshot in the `pv` claim. Every backend's
invocation wrapper consults the live counter on every call —
mismatch ⇒ `PERMISSION_VERSION_STALE` (401) ⇒ client refresh
mints a fresh JWT with the up-to-date set.

## Redis fail-open

The permVer check is best-effort: a Redis outage on GET falls
open with an `auth.permver.fail_open` log tag, mirroring
`auth.session.fail_open` (MED-2). The DB-side session-revoke
path remains the authoritative kill switch.

## See also

- [Per-user overrides](/docs/auth/overrides) — what triggers a bump.
- [Audit trail](/docs/auth/audit-trail) — every bump is audit-logged
  alongside the mutation that triggered it.
- `@kysera/rls` upstream docs — schema authoring details
  beyond the bridge above.

---

---
id: "auth/token-issuance"
title: "Token issuance & transports"
---

# Token issuance & transports

Where the tokens go after a successful sign-in is a **transport** decision, not
a service one. The same `signin` handler works whether tokens travel in the
response body for the client to store, or in `HttpOnly` cookies the client
never sees.

Service code says *what* to issue; the configured `ITokenTransport` decides
*how* it reaches the client.

```typescript
import { issueTokens, clearTokens, readRequestCookie } from '@omnitron-dev/titan/netron/auth';
```

## Issuing

```typescript
@Public()
async signin(dto: SigninDto) {
  const { user, accessToken, refreshToken } = await this.svc.signin(dto);
  issueTokens({ access: accessToken, refresh: refreshToken });
  return { user, accessToken, refreshToken };
}
```

Return the tokens in the body as written above, even in cookie mode. **Cookie
mode strips them from the body before serialisation**; bearer mode leaves the
body untouched. Writing the handler once, for both, is the point — a handler
that omits them cannot be switched to bearer, and one that omits `issueTokens`
cannot be switched to cookies.

`issueTokens` is idempotent: the last call in a request wins.

## Clearing

```typescript
@Public()
async signout() {
  await this.svc.revokeSession(...);
  clearTokens();
}
```

Cookie mode emits `Max-Age=0` for every registered cookie. Bearer mode is a
no-op — the client clears its own storage.

## Reading a cookie back

`readRequestCookie(name)` reads from the **current request's `Cookie` header**
and returns `null` when the header, or that cookie, is absent. It is the only
supported way for service code to see a value the client never put in the body:

```typescript
const refresh = request.refreshToken ?? readRequestCookie('omni_refresh');
```

That fallback order is deliberate. An explicit body value wins so the same
handler serves a bearer client; the cookie answers for a cookie-mode client
whose body carries nothing.

## How the context is found

Both helpers use `AsyncLocalStorage` to reach the current request's metadata,
so a `@Public` method does **not** have to accept a framework context argument.
The HTTP transport opens that frame around every handler invocation.

Outside such a frame — a unit test calling the helper directly, or middleware
holding its own context — pass the context explicitly:

```typescript
issueTokens(ctx, { access, refresh });   // ctx: Pick<NetronMiddlewareContext, 'metadata'>
clearTokens(ctx);
```

Calling either with no ambient frame and no explicit context **throws**, naming
the two ways out. It does not silently do nothing, which for a sign-out would
mean a session the user believes is closed.

## The transports

`Netron.tokenTransport` defaults to `BearerTokenTransport`.

| Transport   | `usesCookies`  | `extract` reads        | `issue` writes                        |
| ----------- | -------------- | ---------------------- | ------------------------------------- |
| `bearer`    | `false`        | `Authorization` header | nothing — the body carries the tokens |
| `cookie`    | `true`         | the `Cookie` header    | `Set-Cookie`, and strips the body     |
| `composite` | if either does | both, in order         | delegates                             |

`usesCookies` is not decoration: **two middlewares gate on it** — the CSRF
double-submit check and the origin check. Both skip when it is false, on the
reasoning that a bearer client attaches nothing automatically and so cannot be
made to act by a hostile page. A transport that carries cookies without
declaring it would therefore turn off both protections in silence.

`composite` exists for a migration: accept both while clients move, then drop
one. It reports `usesCookies` true if any member does, so the CSRF check stays
on for the whole window rather than for the half that needs it.

## CSRF is two pieces, and both are optional

Cookie mode does not turn CSRF protection on by itself. Two things have to be
configured, and each defaults to absent:

1. a `csrf` manager on `CookieTokenTransport`, which makes `issue()` emit a
   CSRF cookie (readable by client JS, unlike the auth cookies) on every
   sign-in and refresh, and clear it on sign-out;
2. `createCsrfMiddleware(...)` installed on the server, which enforces
   double-submit on protected RPCs.

| transport `csrf` | middleware | result                                                                                                             |
| ---------------- | ---------- | ------------------------------------------------------------------------------------------------------------------ |
| set              | installed  | protected                                                                                                          |
| set              | absent     | a CSRF cookie nobody checks                                                                                        |
| absent           | installed  | **every cookie-authenticated request rejected** — no cookie is ever issued to submit, so verification always fails |
| absent           | absent     | **no CSRF protection**, and nothing says so                                                                        |

The third row fails loudly and immediately, which is the good case. The fourth
is the one to watch: a cookie-mode deployment with neither piece behaves
perfectly until somebody points a hostile page at it.

The middleware's own decisions are all fail-closed. A request whose `Request`
object is missing gets an empty header set rather than a bypass; verification
failure throws `FORBIDDEN` and logs `auth.csrf_violation` with the origin but
no token bytes. It skips only where CSRF cannot apply: bearer transports, the
exempt list (sign-in, refresh, sign-out — the calls that have no cookie yet),
and requests arriving with no access cookie at all, which are bearer or
anonymous.

## What each mode assumes of the client

Cookie mode means the browser holds credentials the JavaScript cannot read — no
token in `localStorage` to steal, and no token for the app to attach by hand.
In exchange the request needs CSRF protection, because the browser now attaches
credentials on its own.

Bearer mode is the reverse trade: the client is responsible for storage and for
sending the header, and no CSRF check is required because nothing is attached
automatically.

---

---
id: "community/changelog"
title: "Changelog"
description: "Versioning policy + how to find release notes per package."
---

# Changelog

## Where to find release notes

Each package versions independently and publishes its own
changelog. Release notes live in three places:

1. **GitHub Releases** —
   [github.com/omnitron-dev/omni/releases](https://github.com/omnitron-dev/omni/releases).
   One entry per release across the monorepo; lists every
   package version bumped and what changed.

2. **`CHANGELOG.md` per package** — committed to the repo, e.g.,
   `packages/titan-cache/CHANGELOG.md`. Maintained by
   [Changesets](https://github.com/changesets/changesets).

3. **npm versions** — `pnpm view @omnitron-dev/titan versions`
   for the version history of any package.

## Versioning policy

We follow **strict semver**:

| Bump                        | When                                                                                                                                                         |
| --------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| **MAJOR** (`1.0.0 → 2.0.0`) | Any breaking change to the public API. Includes: removed exports, renamed methods, changed method signatures, behaviour changes that break valid prior code. |
| **MINOR** (`1.0.0 → 1.1.0`) | New feature or non-breaking improvement. New module, new option with a safe default, performance improvement.                                                |
| **PATCH** (`1.0.0 → 1.0.1`) | Bug fix. No API change.                                                                                                                                      |

### Pre-1.0 exception

Packages with version `0.x.y` may have breaks in minor versions
(`0.1.0 → 0.2.0` may break). Once a package hits `1.0.0`, full
semver applies.

This matches semver's [definition](https://semver.org/#spec-item-4)
for pre-1.0 packages.

### Per-package versioning

Each `@omnitron-dev/*` package versions independently. This means:

- `titan` can be at v1.2.0 while `titan-cache` is at v0.8.4.
- Upgrading one package doesn't force upgrading others — as
  long as peer-dependency ranges are satisfied.
- Breaking change in `titan-cache` doesn't bump `titan`'s
  version.

Peer-dependency relationships make this manageable; each module
declares which `titan` versions it supports.

## Support windows

We aim for:

- **Current major**: full support — bug fixes, security
  patches, new features.
- **Previous major**: security fixes only for **12 months**
  after the next major's release.
- **Older majors**: unsupported; upgrade.

Pre-1.0 packages are best-effort.

## Breaking change communication

Breaking changes get:

1. **A deprecation warning** in the previous minor release (at
   least one).
2. **Documentation** of the migration path in the release
   notes.
3. **A migration guide** for substantial changes.

We aim to avoid surprise breaks. If you depend on undocumented
internals, expect them to break without notice.

## Release cadence

Roughly:

- **Patch releases**: as needed (often weekly across the
  monorepo).
- **Minor releases**: 2–4 per quarter per actively-developed
  package.
- **Major releases**: 1–2 per year per package, more for early
  packages.

There is no fixed schedule — releases ship when they're ready.

## Upgrade guides

When a major version requires non-trivial migration, an upgrade
guide is added to the docs:

| From → To                                               | Guide |
| ------------------------------------------------------- | ----- |
| *(none yet — published once first major upgrades land)* |       |

For specific packages, the package's `CHANGELOG.md` has the
detail.

## Subscribe to releases

| Channel             | URL                                                      |
| ------------------- | -------------------------------------------------------- |
| GitHub Releases RSS | <https://github.com/omnitron-dev/omni/releases.atom>     |
| Watch the repo      | github.com/omnitron-dev/omni → Watch → Custom → Releases |
| npm package         | `npm info @omnitron-dev/<pkg>`                           |
| Blog                | [/blog](/blog) (significant releases get a post)         |

## See also

- [Contributing / Release process](/docs/community/contributing#release-process)
- [Security policy](/docs/community/security) — security-fix SLA
- [Packages](/docs/reference/packages) — per-package index

---

---
id: "community/contributing"
title: "Contributing"
description: "Set up the source or documentation repository and verify a focused change."
---

# Contributing

Omnitron maintains two public repositories:

| Repository                                                      | Contents                                           |
| --------------------------------------------------------------- | -------------------------------------------------- |
| [omnitron-dev/omni](https://github.com/omnitron-dev/omni)       | Runtime, packages, supervisor, CLI and web console |
| [omnitron-dev/website](https://github.com/omnitron-dev/website) | This documentation site                            |

Each repository has its own package manifest and lockfile. Clone them as
sibling directories when working on documentation that needs source checks.

## Work on the runtime

Install Node.js 22+ and the pnpm version declared in `package.json`.
Docker is needed for tests that use external services.

```bash
git clone https://github.com/omnitron-dev/omni.git
cd omni
pnpm install --frozen-lockfile
pnpm build
```

Run the relevant package checks before broad workspace checks:

```bash
pnpm -F @omnitron-dev/titan-cache test
pnpm -F @omnitron-dev/titan-cache typecheck
pnpm lint
pnpm fm:check
```

Omnitron's `pnpm -F @omnitron-dev/omnitron typecheck` checks the Node.js
implementation and every server test. `typecheck:tests` runs the latter check
on its own. Server fixtures retain the backend's strict settings, with the
module resolution used by Vitest. `pnpm -F @omnitron/console typecheck` checks
the browser implementation and all Console tests against the same Prism and
browser contracts. These commands run in the workspace CI typecheck job.
A test that executes successfully must also satisfy its current public types;
do not hide obsolete fixture fields or missing evidence behind casts.

Pass test paths directly to pnpm scripts, for example:

```bash
pnpm -F @omnitron-dev/omnitron test test/unit/release-owner-admission.test.ts
```

The root `pnpm test` runs the workspace suite. `pnpm test:ci` starts a dedicated
Compose project, runs that suite and removes its test resources on success or
failure. CI runs the complete suite in addition to focused release and security
gates; a passing focused gate does not substitute for the full run. For repeated
local tests, use `pnpm test:up` and `pnpm test:down` instead. Redis-backed suites
use real test infrastructure in CI too. `USE_MOCK_REDIS=true` is an explicit
unit-only mode; unavailable Redis does not silently select it. Read
[Testing](/docs/testing) for the distinctions between unit,
integration and cross-runtime tests.

## Update dependencies together

Update manifests and the lockfile in an isolated checkout, then build and test
before changing a running deployment. Use the package manager version pinned in
`package.json`; an older pnpm cannot read every lockfile produced by pnpm 12.
The workspace keeps resolution rules, patches and explicit lifecycle-script
permissions in `pnpm-workspace.yaml`.

```bash
pnpm -r update --latest --lockfile-only
pnpm install --frozen-lockfile
pnpm peers check
pnpm build
pnpm check:packages
```

Review major-version migrations and peer ranges. A transitive security override
across a major version needs compatibility checks for the actual consuming API;
changing its resolution alone does not establish compatibility. Keep package-manager
supply-chain checks and review version-specific exceptions. Run checks for the
affected runtime, browser and CLI paths.

The workspace uses the hoisted pnpm layout declared in `pnpm-workspace.yaml`.
It enables standard `bundledDependencies` archives for the KB extraction toolchain:
`ts-morph` and its dependency tree are packed from the tested lockfile. This keeps
the corrected glob implementation in the published package. Consumers choose their
own installation layout and do not need Omnitron's workspace overrides. Reinstall
with `pnpm install --frozen-lockfile` when changing the workspace layout; a local
package-manager configuration must not override the tracked layout.

The workspace installs the native TypeScript 7 compiler as `@typescript/native`
and the current compatible Compiler API as `typescript` through the
`@typescript/typescript6` npm alias. `tsc` uses the native compiler; ESLint and
API-based tooling use the compatibility package. Both are development tools,
not runtime requirements for applications.

`pnpm check:packages` packs every public workspace and verifies that all declared
exports, types and command entry points exist in the archive. CI runs it after
building. Its `--install-consumer /absolute/new/directory` option also installs the
real archives with npm, replays the consumer's generated lockfile with `npm ci`
and audits the resulting production dependencies. Both minimal and all-package
installations must pass.
Release preparation requires that audit and the public runtime checks, including
TypeScript extraction and a SQLite journal write/read. An unavailable audit or a
reported vulnerability fails preparation; a clean workspace audit cannot substitute
for this consumer check.

When a library embeds a development dependency, list its exact name in
`package.json#tsup.noExternal` and use that list in the tsup build. The source
check accepts that build input; the archive check rejects a JavaScript import or
type declaration that would still require the package on a consumer's machine.

Netron React supplies both ESM and CommonJS declarations. The testing
package uses Node.js 22.12+ native `require(esm)` support for its CommonJS entry
conditions. Validate
consumer applications against the same package artifacts and ensure Titan
resolves to one instance per application. Preserve the previous lockfile,
artifacts and state snapshot for rollback. An immutable experiment that pins
an older runtime needs a separate registration for an upgraded environment;
replacing its recorded hashes changes the experiment rather than reproducing it.

Omnitron 0.4.9 uses published Xec 0.12.0 packages. Package manifests and the
committed lockfile select their versions and integrity; workspace overrides and
patches are declared in `pnpm-workspace.yaml`. No third-party package source is
currently stored under `vendor/`. Run the dependency and package-consumer checks
when updating these inputs; do not add a second SSH transport to Omnitron.

A consumer that develops against Omnitron source can use relative `link:`
dependencies into one checkout. Keep its manifests and workspace overrides in
agreement. The release builder reproduces that layout with clean Git clones and
checks that every framework link resolves into the selected clone. pnpm 12
lockfiles contain separate YAML documents for tooling and application packages;
release tooling reads the application importer graph across those documents.

## Work on the documentation

From the parent directory of your source checkout:

```bash
git clone https://github.com/omnitron-dev/website.git
cd website
pnpm install --frozen-lockfile
pnpm start
```

The development server prints its local URL. Before submitting a change:

```bash
pnpm check:content
pnpm typecheck
pnpm build
```

The production build rejects broken page links and anchors. Test search
against a production build with `pnpm serve`; the local search index is
generated during the build.

For checks against a sibling source checkout:

```bash
pnpm check:source ../omni
```

These checks compare inventories, CLI commands, package subpaths, defaults,
service names and catalog coverage with source. They do not establish that
every documentation example runs; execute the examples you change.

## Write a useful page

State the outcome and prerequisites first. Use complete file names for
runnable examples and label excerpts as excerpts. Explain configuration
defaults, errors, lifecycle behaviour and the next operational step where
they affect the reader.

- Use sentence case, explicit code languages and relative links.
- Keep third-party references relevant to actual dependencies or integrations.
- Use neutral example names and reserved example domains.
- Link to public source paths that a fresh checkout can resolve.
- Mark proposals **Design RFC** and distinguish them from shipped APIs.
- Add a page to `src/data/sidebars.json` so readers can discover it.

## Change the code

Follow the package's existing conventions. Preserve public API compatibility
unless the change deliberately introduces a documented migration. For a bug
fix, add a regression test that exercises the failed behaviour.

Keep type-only imports explicit and use `.js` extensions for relative ESM
imports. Use the repository's lint and formatting configuration.

## Submit a change

Open a focused pull request in the repository you changed. Describe the
problem, the resulting behaviour and the checks you actually ran. Link a
related documentation PR when the code and manual need to change together.

Use a descriptive conventional commit title, such as
`fix(titan-cache): invalidate tagged entries` or
`docs(netron): explain HTTP startup`.

## Release process

The repository uses Xec as its scripting engine. Run the checked-in recipes through
the pinned CLI installed by `pnpm install --frozen-lockfile`; `pnpm release` is the
same entry point as `pnpm exec xec release`. No temporary publisher or personal
shell setup is required.

From a clean `main`, inspect the inventory and plan:

```sh
pnpm exec xec release --list
pnpm exec xec release <version> --dry-run
```

Prepare and qualify the release with the full commit of the backend being replaced:

```sh
pnpm exec xec release <version> --prepare --previous <full-previous-commit> --push-source
```

This command commits synchronized public versions, builds and checks the package
archives in an isolated source export, creates a local annotated tag, and pushes
only the exact source branch for CI. Omit `--push-source` if your Git publication
is handled separately. The same Xec workflow runs the independent build comparison,
CI checks and native transition/rollback checks concurrently after preparation.
The policy in `.xec/release-policy.mjs` selects the CI executor and exact Node
versions. This repository uses GitHub Actions; the Omnitron platform and
application deployment do not require GitHub.

Local preparation and reproducibility use two independent clean builds in total.
Preparation retains its completed build after every package-consumer gate passes.
Qualification verifies that retained build and performs one further build from a
fresh checkout and frozen installation. Both use the same commit, Node version,
platform, architecture, pinned pnpm and build policy, with build-cache reads
disabled. Both producers run pending dependency builds under the checked-in pnpm
policy without restoring native side-effect caches. The sealed runtime, Console
and normalized public npm archives must match. Missing, changed or incompatible
preparation evidence stops qualification.
CI and native jobs still perform their own target-specific builds. The standalone
`pnpm check:reproducibility` gate performs two fresh builds when no preparation
is supplied.

Within each workspace build, `pnpm build` requests `build` and `build:titan` in
one Turbo graph, so shared prerequisites run once while their declared ordering
remains enforced. The full workspace test job completes that build before running
`pnpm test:ci --only` with `TURBO_FORCE=true`: every normal package test executes
without restoring cached results, and its build prerequisites are already present.
The ordinary developer command `pnpm test:ci` still builds prerequisites. These
changes remove duplicate work; elapsed release time depends on the completed
local and remote runs.

The CI reproducibility job installs this same frozen workspace before running
`pnpm check:reproducibility`, which invokes the pinned Xec engine. A fresh runner
does not depend on a globally installed Xec or another job’s build output.

The qualification binds every result to the candidate commit, previous backend,
policy and build Node version. It requires all configured CI jobs and native transition
and rollback on every configured Node. A lost workflow dispatch response is
reconciled by its saved intent; repeating the command does not dispatch a duplicate.
A failed native run remains failed. Inspect it before explicitly retrying the
GitHub run; Xec observes its new attempt when qualification is resumed.

```sh
pnpm exec xec release <version> --qualify --previous <full-previous-commit>
pnpm exec xec release <version> --status
```

Preparation evidence is retained under
`.release-artifacts/qualification/<commit>/preparation/`: `PREPARATION.json`,
`runtime/`, `sdk/` and `sdk-inventory.json`. The qualification directory retains
`qualification.json` with each stage's evidence, plus the accepted independent
build's `QUALIFICATION.json`, runtime and SDK archives. On resume, completed
stages are reverified and reused. A partial build directory is not a completed
receipt. `--status` checks the selected evidence without builds, workflow
dispatches or receipt changes.

Native CI retains the actual tested Linux runtimes. Xec downloads them, verifies
their source, run, attempt, archive checksum and runtime digest, and extracts them
with Omnitron's existing archive validator. Evidence and artifacts live under
`.release-artifacts/`. Use an external development volume for large builds and
set `TMPDIR` and package manager cache directories before the command if required
by your machine.

Package preparation also verifies exports and two ordinary npm consumers without
workspace overrides: a minimal CLI/Netron runtime and every public package,
including optional dependencies. Inspect `release.json`, SHA-512 digests and
`NOTES.md` under `.release-artifacts/<version>/<commit>/`.

Publish the sealed batch:

```sh
pnpm exec xec release <version> --finish
```

Publication rechecks the saved qualification and archive bytes; it does not rerun
builds or tests. Xec checks npm authorization before pushing the selected commit
and its single tag, submits the archives and completes the GitHub release only
after npm confirms the batch. The separate `--push`, `--publish`, `--resume`
and `--github` operations use the same qualification gate.

If npm reports an expired session, run
`npm login --auth-type=web --registry=https://registry.npmjs.org/` and check
`npm whoami --registry=https://registry.npmjs.org/`. Credentials remain in
npm's normal configuration; qualification receipts contain no tokens.

### Browser approval and recovery

Complete the npm challenge in the browser. If npm offers temporary approval for
the current IP and token, it can cover subsequent uploads during that period.
Xec submits the archives in dependency order, then verifies version metadata,
downloaded SHA-512 and the release channel. Registry scanning can continue after
an upload is accepted. [npm browser authorization](https://github.blog/changelog/2022-07-26-general-availability-of-improved-2fa-experience-in-npm/).

Keep the exact clean checkout and its original artifacts after an interruption.
Repeat `--finish` for that version; Xec reconciles saved uploads and waits for
registry confirmation. Never rebuild a reserved version to resolve a conflict.
Use the same `--registry <url>` when resuming a private registry release.

### Correct an unsubmitted draft

If qualification finds a defect after local preparation, preserve the prepared
commit in a clean checkout of the same repository with its original archives.
Commit the fix on `main`, then run
`pnpm exec xec release <version> --supersede <old-checkout>` from `main`.
Xec verifies the old sealed artifacts, refuses any publication journal or remote
tag, and preserves the annotated object under `refs/release-attempts/` before
removing the old local tag with compare-and-swap. It never changes an npm version
or remote reference. Then repeat preparation on the new commit. A release that
was already submitted must resume its exact bytes or use a new version.

### Managed controller updates

Xec coordinates the update through Omnitron's existing lifecycle API. The daemon
owns role checks, immutable runtime pins, operation locks, progress and rollback.
Xec does not stop services over SSH or implement another process owner.

From the qualified checkout, prepare selected registered remote nodes:

```sh
pnpm exec xec update plan <candidate-commit> --node test-controller --node production-controller
```

The recipe imports the retained native runtimes, stages matching variants and
returns the durable Omnitron plan. It does not activate them. Review that plan and
its impact before applying it:

```sh
pnpm exec xec update apply <plan-id>
pnpm exec xec update status <plan-id>
pnpm exec xec update reconcile <plan-id>
```

A blocked or uncertain operation retains its receipt and returns a nonzero exit
status with the controller’s reason. Migration incompatibility requires a separate
database transition; the recipe does not bypass that refusal.

Use `--request-id <uuid>` to reuse a known planning request. After a lost
activation response, inspect or reconcile that ID; never replay activation
blindly. An already qualified runtime from another builder can be supplied with
`--runtime <directory>`. `--cli <executable>` selects the fleet controller CLI.

The local controller uses the existing supervised backend transition. Xec runs
the sealed candidate’s own Node and CLI so admission checks understand its
requirements; `--cli` cannot substitute an older CLI for these local operations:

```sh
pnpm exec xec update local-check <qualified-runtime>/packages/0/dist
pnpm exec xec update local-apply <qualified-runtime>/packages/0/dist
```

These commands retain the execution owner. A required owner, guardian or schema
transition is reported by Omnitron and must follow its own reviewed plan; it is
never silently folded into a backend update. Updating the Console container is
also a separate service operation when required by its deployment configuration.

The CI rehearsal executes the release through the actual Xec CLI against a
loopback registry. It covers partial publication, delayed visibility, conflicting
digests and exact Git refs. Qualification tests cover failed and skipped jobs,
lost dispatch responses, resumed builds and incomplete evidence. Update tests
verify that planning cannot activate a backend and that uncertain activation is
not repeated.

## Report a problem

Include package versions, Node.js version, operating system, expected
behaviour, observed behaviour and a minimal reproduction in a
[source issue](https://github.com/omnitron-dev/omni/issues/new).
Report documentation problems in the
[website repository](https://github.com/omnitron-dev/website/issues/new).
For security reports, follow the [security policy](/docs/community/security).

Contributions use the project's MIT license.

---

---
id: "community/security"
title: "Security policy"
description: "How to report vulnerabilities + supported versions + disclosure SLA."
---

# Security policy

## Supported versions

| Version                          | Supported |
| -------------------------------- | :-------: |
| `0.x.x` (current)                |     ✓     |
| Pre-`0.1.0` development versions |     ✗     |

Once stable releases begin, support windows will be:

- **Current major** — full security + bug fixes.
- **Previous major** — security fixes only for 12 months after
  the next major's release.

## Reporting a vulnerability

**Do not open public GitHub issues** for security reports.

Email reports to: **<security@omnitron.dev>**

Include:

1. Affected package(s) and version(s).
2. A clear description of the vulnerability.
3. Steps to reproduce or proof-of-concept code.
4. Impact assessment (data exposure, RCE, DoS, etc.).
5. Suggested mitigation, if known.
6. Your name + affiliation (optional — for credit).

Encrypt with our PGP key if the report contains exploit
details:

```
[PGP key fingerprint published at omnitron.dev/.well-known/security.txt]
```

## What happens next

| Timeline     | What we do                                                                 |
| ------------ | -------------------------------------------------------------------------- |
| **24 hours** | Acknowledge receipt of your report.                                        |
| **72 hours** | Initial triage; severity classification.                                   |
| **7 days**   | Confirmed vulnerability → fix in development; assigned CVE if appropriate. |
| **30 days**  | Patch released (sooner for critical / actively exploited).                 |
| **+30 days** | Public disclosure with your credit (if you wish).                          |

For exceptionally severe issues (active exploitation, mass
impact), the timeline compresses — we coordinate with you on
a faster track.

## Severity classification

We use CVSS v3.1 base scores:

| Score      | Severity     | Response                                    |
| ---------- | ------------ | ------------------------------------------- |
| 9.0 – 10.0 | **Critical** | Patch within 7 days; coordinated disclosure |
| 7.0 – 8.9  | **High**     | Patch within 14 days                        |
| 4.0 – 6.9  | **Medium**   | Patch in next regular release (≤ 30 days)   |
| 0.1 – 3.9  | **Low**      | Patch in next regular release (≤ 60 days)   |

## Disclosure

When a fix is released:

1. We publish a **security advisory** on GitHub.
2. The release notes call out the fix with a CVE reference (if
   assigned) and credit the reporter.
3. Users are notified through:
   - The release feed on the docs site.
   - GitHub Security Advisories.
   - The `@omnitron-dev/titan` npm package security advisory.

## Scope

In-scope: every npm package under `@omnitron-dev/*` shipped from
the [omni](https://github.com/omnitron-dev/omni) repository.

Out-of-scope:

- The docs site at omnitron.dev (report to GitHub issues; not
  security-sensitive).
- Third-party plugins / community modules — report to their
  maintainers.
- Issues in upstream dependencies — report to those projects
  first; we'll coordinate downstream patching.

## What we look at first

Highest-risk areas:

1. **Auth** — JWT verification, session handling, token
   binding, refresh rotation.
2. **RPC** — wire-format parsing (MessagePack), middleware,
   route auth.
3. **Database** — RLS, query construction, parameter binding.
4. **Process supervision** — Omnitron daemon RBAC, secret
   storage, child process isolation.
5. **Infrastructure provisioning** — Docker container config,
   bare-metal SSH execution.

## Hall of fame

Researchers who responsibly disclosed vulnerabilities (with
permission to be named):

*None yet — the platform is too young for a list. Be the first.*

## Hardening guidance

Production deployment checklists per module:
[Security checklist](/docs/titan/modules/security-checklist).

Operator-side hardening:
[Omnitron Auth & RBAC](/docs/omnitron/auth-rbac) and
[Best practices](/docs/omnitron/best-practices).

## Past advisories

None published. When a security advisory is published, it
will be linked here with the CVE, severity, affected versions,
and fix version.

## Contact

- **Vulnerability reports**: <security@omnitron.dev> (PGP optional)
- **General security questions**: discuss in GitHub Discussions
- **PGP key**: published at `omnitron.dev/.well-known/security.txt`

Thank you for helping keep the platform secure.

---

---
id: "comparison"
title: "vs alternatives"
description: "How this stack differs from NestJS, tRPC, Fastify, Hono, and the typical TS app stack."
---

# vs alternatives

Comparing this stack to the most common alternatives. Strict —
each row is verifiable. Pick the row that matters most to your
team.

If you're migrating from one of these, the
[migration guides](/docs/titan/migrations) cover the mechanical
parts. This page covers **fit**.

## The big picture

```mermaid
flowchart TB
  subgraph TS["The TypeScript backend space"]
    Nest["NestJS<br/>(framework + DI + RPC)"]
    Trpc["tRPC<br/>(RPC layer only)"]
    Fast["Fastify<br/>(HTTP only)"]
    Hono["Hono<br/>(HTTP only, edge-first)"]
    Express["Express<br/>(HTTP only)"]
    Omni["Omnitron stack"]
  end

  Nest -- "decorator DI" --> Omni
  Trpc -- "end-to-end types" --> Omni
  Fast -- "HTTP perf" --> Omni
  Hono -- "transport flexibility" --> Omni

  classDef framework fill:#e3f2fd,stroke:#1976d2,color:#0d47a1
  classDef rpc fill:#f3e5f5,stroke:#8e24aa,color:#4a148c
  classDef http fill:#fff8e1,stroke:#fb8c00,color:#e65100
  classDef omni fill:#e8f5e9,stroke:#43a047,color:#1b5e20
  class Nest framework
  class Trpc rpc
  class Fast,Hono,Express http
  class Omni omni
```

Omnitron borrows the decorator-DI from NestJS, the end-to-end
type ergonomics from tRPC, transport flexibility from Hono,
the supervised-process model from PM2/systemd — and unifies
them under one TypeScript codebase.

## vs NestJS

[NestJS](https://nestjs.com/) is the closest neighbour.

| Axis                            | NestJS                                                                                           | Omnitron stack                                                                    |
| ------------------------------- | ------------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------- |
| **DI container**                | InversifyJS-style                                                                                | Nexus (custom; class-token-based with contextual injection)                       |
| **Decorators**                  | `@Module`, `@Injectable`, `@Controller`, `@Get`, `@Post`, …                                      | `@Module`, `@Injectable`, `@Service`, `@Public`, `@Validate`, `@Auth`             |
| **Transport**                   | Express / Fastify under the hood (HTTP only by default; microservices add-on for TCP/Redis/NATS) | Native: HTTP / WS / TCP / Unix — same service surface                             |
| **Type safety client ↔ server** | OpenAPI codegen or separate `.d.ts` sync                                                         | Service interface IS the type; no codegen                                         |
| **Validation**                  | class-validator + class-transformer (decorator-heavy DTOs)                                       | Zod schemas inline via `@Validate(schema)`                                        |
| **Module ecosystem**            | Huge — most things have a `@nestjs/*` package                                                    | 14 focused official modules + community                                           |
| **Testing**                     | `Test.createTestingModule()`                                                                     | `createTestApp()` (similar shape)                                                 |
| **Lifecycle**                   | `OnModuleInit`, `OnApplicationBootstrap`, `OnApplicationShutdown`                                | `OnInit`, `OnStart`, `OnStop`, `OnDestroy` (semantically similar)                 |
| **Supervisor / fleet**          | Not included; you run it under PM2 / systemd / k8s                                               | Omnitron daemon — supervised processes, 19+ RPC services, web console, MCP server |
| **Bundle size**                 | Larger; pulls in metadata-reflect + RxJS for parts                                               | Modular; pay-for-what-you-use                                                     |

### Pick NestJS when…

- Your team already knows it.
- You need the broad ecosystem (huge selection of `@nestjs/*`
  packages — auth strategies, GraphQL, gRPC, etc.).
- HTTP-only is fine; transport flexibility doesn't matter.

### Pick Omnitron when…

- End-to-end TypeScript (no OpenAPI / protobuf step) is a hard
  requirement.
- You'll use ≥2 transports (HTTP + WS, or HTTP + TCP, or all
  four).
- You want operator surface (supervisor, fleet, web console, MCP)
  in the same stack — not a separate ops pipeline.
- You want smaller per-app footprint.

→ Migration: [from NestJS](/docs/titan/migrations/from-nestjs)

## vs tRPC

[tRPC](https://trpc.io/) is the gold standard for type-safe RPC.

| Axis                    | tRPC                                                     | Omnitron stack                               |
| ----------------------- | -------------------------------------------------------- | -------------------------------------------- |
| **Type sharing**        | Inferred from router → typed on client; same monorepo    | Service interface shared; same monorepo      |
| **Transport**           | HTTP + WebSocket (built-in)                              | HTTP + WS + TCP + Unix                       |
| **DI**                  | None — plain functions in routers                        | Container DI with lifecycle                  |
| **Validation**          | Zod (or Yup, Superstruct, …) — schema-first              | Zod via `@Validate` decorator                |
| **Server framework**    | Adapter pattern — runs on Express/Fastify/Next/Hono/etc. | Standalone Titan framework                   |
| **Middleware**          | Per-procedure context middleware                         | DI middleware + Netron middleware (per-call) |
| **Subscriptions**       | WebSocket-based via SubscriptionResolver                 | AsyncIterable methods over WS                |
| **Auth**                | Context-based — you wire it                              | `titan-auth` module + decorators             |
| **Caching client-side** | React Query (separate package)                           | `netron-react` cache (built-in)              |
| **Cross-language**      | TS-only by design                                        | TS-only; MessagePack wire format extensible  |
| **Operator surface**    | None — you bring it                                      | Full Omnitron daemon                         |

### Pick tRPC when…

- You want a minimal RPC layer on top of an existing HTTP server
  (Next.js API routes, Hono, Express).
- The backend is small and doesn't need a full framework's DI /
  lifecycle / modules.
- You're already invested in React Query for client cache.

### Pick Omnitron when…

- You want the full stack (DI + lifecycle + 14 official modules), not
  just the RPC layer.
- You need transports beyond HTTP/WS (TCP / Unix for
  service-to-service).
- You want auth / cache / retry / circuit breaker as
  framework-level concerns, not per-procedure code.

## vs Fastify / Express / Hono

These are HTTP servers — different category, but often the
starting comparison.

| Axis            | Fastify / Express / Hono                                 | Omnitron stack                                                |
| --------------- | -------------------------------------------------------- | ------------------------------------------------------------- |
| **Scope**       | HTTP only                                                | Full stack (HTTP + DI + RPC + UI + ops)                       |
| **Routing**     | Path-based (`app.get('/users/:id', ...)`)                | Service-based (`UsersService.findById`) — RPC abstracts paths |
| **DI**          | None — manual wiring                                     | Container DI                                                  |
| **Validation**  | Per-route via plugins (Fastify schema, Zod with adapter) | Decorator on method (`@Validate`)                             |
| **Auth**        | Per-route middleware                                     | App-level + per-method decorators                             |
| **WebSocket**   | Add-on plugin                                            | Native — same service over both transports                    |
| **Bundle**      | Tiny (Hono <15 kB on edge)                               | Larger; framework-level                                       |
| **Performance** | Fastify \~70K req/s, Hono \~150K req/s on edge           | Comparable HTTP perf; the cost is framework features          |

### Pick the HTTP server when…

- You're building a small, focused REST/HTTP service.
- You want full control over routing, middleware order, request
  lifecycle.
- You're targeting edge runtimes (Hono) or single-binary deploys.

### Pick Omnitron when…

- You need RPC semantics (typed methods, not HTTP endpoints).
- You need DI / modules / lifecycle / multi-transport — building
  these on top of Express ends up reinventing 60% of Titan.

## vs PM2 / systemd / Docker Compose

Omnitron is a **supervisor + control plane**, similar in scope
to PM2 (for Node) or systemd (for any process) — but typed,
opinionated for TypeScript services.

| Axis                      | PM2                      | systemd    | Docker Compose        | Omnitron                                      |
| ------------------------- | ------------------------ | ---------- | --------------------- | --------------------------------------------- |
| **Process supervision**   | Yes (Node-only)          | Yes (any)  | Container-level       | Yes (Node-first)                              |
| **Health checks**         | Basic                    | Manual     | `healthcheck:` clause | Per-app, per-process, typed                   |
| **Cluster mode**          | Round-robin Node cluster | n/a        | Swarm (separate)      | Leader election + state sync                  |
| **Logs**                  | File rotation            | journald   | Per-container         | Per-app, structured (pino)                    |
| **RPC into running apps** | None                     | None       | None                  | Yes — same Netron protocol as clients         |
| **Configuration**         | `ecosystem.config.js`    | Unit files | `docker-compose.yml`  | `omnitron.config.ts` + per-app `defineSystem` |
| **TypeScript-native**     | No (any Node app)        | No         | No                    | Yes                                           |
| **Web console**           | PM2+ paid add-on         | No         | Portainer (separate)  | Built-in                                      |
| **MCP for agents**        | No                       | No         | No                    | Built-in (`omnitron kb mcp`)                  |

### Pick PM2 when…

- Single-app Node deployment on one box.
- You want lightweight; no opinion on app code.

### Pick systemd when…

- Mixed-language fleet.
- You manage the OS layer too.

### Pick Docker Compose / k8s when…

- Container-first deployment.
- Multi-language; Omnitron just becomes one service among many.

### Pick Omnitron when…

- All apps are Titan apps (or any decorated services).
- You want typed control of the running fleet, not just process
  babysitting.

## vs the typical "modern TypeScript stack"

A typical Next.js + Prisma + tRPC + Tailwind setup:

| Concern           | Typical                                 | Omnitron stack                              |
| ----------------- | --------------------------------------- | ------------------------------------------- |
| Backend framework | Next.js API routes / Hono / tRPC server | Titan                                       |
| RPC               | tRPC                                    | Netron                                      |
| Frontend          | Next.js / Vite + React                  | Vite + React (Prism + netron-react)         |
| Design system     | shadcn/ui or Material-UI                | Prism (built on MUI v9)                     |
| DB layer          | Prisma / Drizzle                        | titan-database (Kysely + Kysera plugins)    |
| Auth              | NextAuth / Clerk                        | titan-auth                                  |
| Form lib          | react-hook-form + zod                   | react-hook-form + zod (via Prism `<Field>`) |
| Cache             | React Query                             | netron-react QueryCache                     |
| State             | Zustand / Jotai                         | Zustand (via Prism `createPrismStore`)      |
| Deployment        | Vercel / Cloudflare / Render            | Docker + supervised by Omnitron             |

If you're already deep in the typical stack, the migration cost
is real. The tradeoff: Omnitron unifies it under one decorator
grammar, end-to-end types, and one operator surface.

## Honest tradeoffs

What this stack **gives up**:

- **HTTP routing flexibility.** RPC services don't map cleanly
  to `/api/v2/users/:id/profile.png`. Custom routes are
  supported but feel bolted on.
- **Massive ecosystem.** NestJS has more 3rd-party packages.
  npm has more shadcn-style components than Prism.
- **Edge runtime support.** Titan + Omnitron daemon assume
  Node. You can't ship Titan to Cloudflare Workers.
- **One-runtime restriction for Omnitron.** Apps can be Bun/Deno;
  the daemon needs Node.

What you **get back**:

- One language end-to-end with no codegen.
- One decorator grammar across DI / RPC / auth / validation.
- One operator surface (CLI + web console + MCP) for any app.
- Per-module independence (14 modules, opt-in each).
- Built-in fleet primitives (cluster, fleet, replication).

## See also

- [Migration / from NestJS](/docs/titan/migrations/from-nestjs)
- [Migration / from Express](/docs/titan/migrations/from-express)
- [Migration / from prom-client](/docs/titan/migrations/from-prom-client)
- [Principles](/docs/foundations/principles) — why this stack looks like this
- [Architecture](/docs/foundations/architecture) — the layer composition

---

---
id: "deployment/bare-metal"
title: "Bare-metal / systemd"
description: "Run Omnitron as a systemd service on your own server."
---

# Bare-metal / systemd

For when you own the server — bare-metal hardware, VPS, or a
VM you SSH into. systemd as the supervisor of the Omnitron
daemon; Omnitron supervises the Titan apps.

## Topology

```mermaid
flowchart TB
  System[systemd]
  System -- spawns + monitors --> Daemon[omnitron daemon]
  Daemon -- forks + supervises --> Apps[Titan apps]
  Apps --> PG[(Postgres)]
  Apps --> Redis[(Redis)]
```

systemd = OS-level supervisor. Omnitron = app-level
supervisor. Each does its job.

## Prerequisites

```bash
# Node.js 22+ (via nvm or apt):
curl -fsSL https://deb.nodesource.com/setup_22.x | sudo -E bash -
sudo apt install -y nodejs

# pnpm:
sudo npm install -g pnpm

# Postgres + Redis (or managed elsewhere):
sudo apt install -y postgresql redis

# Optional but nice — a dedicated user:
sudo useradd -r -m -s /bin/bash omnitron
sudo -u omnitron mkdir -p /home/omnitron/.omnitron
```

## Ship the build

```bash
# On the build host:
pnpm install --frozen-lockfile
pnpm build
tar -czf platform.tar.gz dist apps packages omnitron.config.ts package.json pnpm-lock.yaml

# To the server:
scp platform.tar.gz omnitron@server__LUMEN_COLON__/home/omnitron/
ssh omnitron@server
tar -xzf platform.tar.gz -C /home/omnitron/platform/
cd /home/omnitron/platform
pnpm install --frozen-lockfile --prod
```

For mature shops, replace with a real deploy pipeline
(`omnitron deploy build <app>` — which writes a versioned tarball
to `<projectRoot>/.omnitron/artifacts/<app>-<version>.tar.gz` —
plus scp + atomic symlink swap).

## systemd unit

Create `/etc/systemd/system/omnitron.service`:

```ini
[Unit]
Description=Omnitron supervisor
Documentation=https://omnitron.dev/docs
After=network-online.target postgresql.service redis.service
Wants=network-online.target

[Service]
Type=notify
User=omnitron
Group=omnitron
WorkingDirectory=/home/omnitron/platform

# Environment from a file (do NOT commit this file):
EnvironmentFile=/etc/omnitron/env

# Boot the daemon in foreground; systemd handles restart:
ExecStart=/usr/bin/pnpm omnitron up --foreground

# Graceful shutdown — give time for in-flight requests to drain:
KillSignal=SIGTERM
TimeoutStopSec=60
KillMode=mixed

Restart=on-failure
RestartSec=5s
StartLimitInterval=60s
StartLimitBurst=5

# Hardening:
NoNewPrivileges=yes
ProtectSystem=strict
ProtectHome=yes
ReadWritePaths=/home/omnitron/.omnitron /var/log/omnitron
PrivateTmp=yes
PrivateDevices=yes
ProtectKernelTunables=yes
ProtectKernelModules=yes
ProtectControlGroups=yes
RestrictSUIDSGID=yes

# Resource limits:
LimitNOFILE=65536
MemoryMax=4G
CPUQuota=200%       # 2 cores

[Install]
WantedBy=multi-user.target
```

Environment file `/etc/omnitron/env`:

```bash
NODE_ENV=production
DATABASE_URL=postgres://platform:strongpass@localhost__LUMEN_COLON__5432/platform
REDIS_URL=redis://localhost:6379
JWT_SECRET=<generated>
# Daemon state lives at $HOME/.omnitron — with User=omnitron that resolves to
# /home/omnitron/.omnitron. There is no OMNITRON_HOME override; set HOME to relocate.
HOME=/home/omnitron
```

Set restrictive perms:

```bash
sudo chmod 0600 /etc/omnitron/env
sudo chown omnitron:omnitron /etc/omnitron/env
```

## Enable + start

```bash
sudo systemctl daemon-reload
sudo systemctl enable omnitron
sudo systemctl start omnitron

# Watch logs:
sudo journalctl -u omnitron -f
```

The Omnitron daemon writes its own log to
`~/.omnitron/logs/omnitron.log` (errors also to
`~/.omnitron/logs/omnitron.error.log`); journald captures
stdout/stderr as a backup.

## Database setup

```bash
sudo -u postgres psql <<'SQL'
CREATE USER platform WITH PASSWORD 'strongpass';
CREATE DATABASE platform OWNER platform;
GRANT ALL PRIVILEGES ON DATABASE platform TO platform;
SQL

# Migrate:
sudo -u omnitron bash -c 'cd /home/omnitron/platform && pnpm omnitron infra migrate'
```

For better security, use Postgres' `pg_hba.conf` to restrict
auth method per source.

## Reverse proxy (nginx)

The `webapp_backend` below assumes the Omnitron Console is
running on `:9800`. That port is served by the `omnitron-nginx`
container that `omnitron webapp start` provisions (it proxies to
the daemon's Netron HTTP on `:9801` / WS on `:9802`), so the
Console still needs Docker on the host. If you don't want Docker
on a bare-metal box, drop the `webapp_backend`/`app.example.com`
server block and operate via the CLI + the daemon RPC on `:9801`.

```nginx
# /etc/nginx/sites-available/platform
upstream api_backend {
    server 127.0.0.1:3001;
    keepalive 32;
}

upstream webapp_backend {
    server 127.0.0.1:9800;   # omnitron-nginx (Console) — needs `omnitron webapp start`
    keepalive 16;
}

server {
    listen 443 ssl http2;
    server_name api.example.com;

    ssl_certificate     /etc/letsencrypt/live/api.example.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/api.example.com/privkey.pem;

    location / {
        proxy_pass http://api_backend;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_read_timeout 7d;
    }
}

server {
    listen 443 ssl http2;
    server_name app.example.com;

    ssl_certificate     /etc/letsencrypt/live/app.example.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/app.example.com/privkey.pem;

    location / {
        proxy_pass http://webapp_backend;
        proxy_http_version 1.1;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }
}

server {
    listen 80;
    server_name api.example.com app.example.com;
    return 301 https://$server_name$request_uri;
}
```

```bash
sudo ln -sf /etc/nginx/sites-available/platform /etc/nginx/sites-enabled/
sudo nginx -t && sudo systemctl reload nginx
```

## TLS via Let's Encrypt

```bash
sudo apt install -y certbot python3-certbot-nginx
sudo certbot --nginx -d api.example.com -d app.example.com
```

Cert auto-renews via certbot's timer.

## Updating

```bash
# Atomic deploy via symlink swap:
ssh omnitron@server <<'EOF'
cd /home/omnitron
NEW=platform-$(date +%s)
mkdir -p $NEW
tar -xzf /home/omnitron/platform.tar.gz -C $NEW
cd $NEW && pnpm install --frozen-lockfile --prod && pnpm omnitron infra migrate
cd /home/omnitron
ln -sfn $NEW platform-next
mv -T platform-next platform
sudo systemctl reload omnitron      # or restart if needed
# Keep previous 5 for rollback:
ls -t platform-* | tail -n +6 | xargs rm -rf
EOF
```

For zero-downtime reload (workers cycle, daemon stays up):

```bash
sudo systemctl reload omnitron      # if unit declares ExecReload
# or directly:
sudo -u omnitron pnpm omnitron reload
```

## Backups

Daily Postgres backup via cron:

```bash
# /etc/cron.d/omnitron-backups
30 2 * * *  postgres  pg_dump platform | gzip > /var/backups/platform-$(date +\%Y\%m\%d).sql.gz
```

Or let the daemon schedule it. `omnitron backup schedule` takes a
target and a specification, persists it, and re-arms it on every
daemon start:

```bash
omnitron backup schedule main "0 3 * * *"   # cron, or hourly|daily|weekly, or a number of ms
omnitron backup schedule all daily          # 'all' = every database in a running stack
omnitron backup schedules                   # list what is configured
omnitron backup unschedule main
```

An unparseable specification is rejected when you set it, rather
than defaulting to some time nobody chose.

Cron on the host works too, and is the right choice when you want
the backup to run whether or not the daemon is up:

```bash
# /etc/cron.d/omnitron-app-backups — 'main' is the app database name
0 3 * * *  omnitron  cd /home/omnitron/platform && /usr/bin/pnpm omnitron backup create main
```

The daemon's state under `~/.omnitron/` (including
the SQLite store at `~/.omnitron/data/daemon-state.db`) is
regenerable — back it up if you want history (uptime bars,
secrets store).

## Multi-server fleet

Omnitron supports a fleet of daemons addressable by alias:

```bash
# On the master:
omnitron remote add web-1 192.168.1.10 --port 9700
omnitron remote add web-2 192.168.1.11 --port 9700
omnitron remote status web-1
omnitron fleet status
```

Each `omnitron up` daemon listens on TCP 9700 (configurable) for
fleet RPC; the master coordinates.

For cluster mode (leader election), configure the daemon's
`cluster` block — not env vars. `omnitron cluster status`
itself prompts with the config shape:

```typescript
// daemon cluster config
cluster: {
  enabled: true,
  discovery: 'redis',   // 'redis' | 'static'
  peers: ['192.168.1.10:9700', '192.168.1.11:9700', '192.168.1.12:9700'],
}
```

Inspect/manage it with `omnitron cluster status` and
`omnitron cluster step-down`. (There are no
`OMNITRON_CLUSTER_*` environment variables.)

See [Cluster + Fleet](/docs/omnitron/cluster).

## Monitoring

- **Application metrics** — Prometheus scrapes the daemon's
  dedicated metrics port `:9803/metrics` (loopback by default;
  bind `0.0.0.0` + bearer token to scrape off-host). `:9803`
  also serves `/healthz`.
- **Host metrics** — node_exporter on each host.
- **Logs** — Promtail / Filebeat tails journald + Omnitron's
  log directory.
- **Alerts** — Prometheus AlertManager + Omnitron's own
  `OmnitronAlerts`.

## Firewall

```bash
# UFW recipe:
sudo ufw default deny incoming
sudo ufw default allow outgoing
sudo ufw allow 22/tcp                     # SSH
sudo ufw allow 80/tcp                     # HTTP (redirects to HTTPS)
sudo ufw allow 443/tcp                    # HTTPS
sudo ufw allow from 192.168.1.0/24 to any port 9700  # Omnitron TCP — internal only
sudo ufw enable
```

Public exposure of 9700 = full operator surface on the
internet. Never.

## Hardening checklist

- [ ] Run as a non-root user (`omnitron` in examples above).
- [ ] systemd hardening directives (above).
- [ ] `chmod 0600 /etc/omnitron/env`.
- [ ] Firewall closed to 9700 from outside.
- [ ] TLS on nginx (Let's Encrypt or your CA).
- [ ] Postgres `pg_hba.conf` restricts auth.
- [ ] SSH key auth only; password auth disabled.
- [ ] fail2ban on SSH + nginx.
- [ ] Auto-security-updates for the OS (`unattended-upgrades`).
- [ ] Periodic vulnerability scan (`pnpm audit`).

## When bare-metal beats Docker/k8s

- **Predictable latency** — no container abstraction overhead.
- **Direct hardware access** — GPUs, custom NICs, NVMe.
- **Cost** — at scale, raw VPS beats managed.
- **Simplicity** — one server, one systemd unit, one nginx.

When it doesn't:

- **Horizontal scaling** — pets vs cattle; bare-metal is pets.
- **Crash recovery** — no automatic node replacement.
- **Multi-region** — need to roll your own DNS / failover.

## See also

- [Deployment overview](/docs/deployment)
- [Docker Compose](/docs/deployment/docker) — containerised alternative
- [Cluster + Fleet](/docs/omnitron/cluster) — multi-server patterns
- [Best practices](/docs/omnitron/best-practices)

---

---
id: "deployment/docker"
title: "Docker Compose"
description: "Deploy an Omnitron-supervised stack as a single docker-compose file."
---

# Docker Compose

The fastest path from `pnpm dev` to "running on a server".
Suitable for staging, single-host production, demo
environments, and dev parity.

## The minimal compose file

```yaml
# docker-compose.yml
version: '3.9'

services:
  app:
    image: my-platform:latest
    restart: unless-stopped
    environment:
      NODE_ENV: production
      DATABASE_URL: postgres://platform:${DB_PASS}@postgres__LUMEN_COLON__5432/platform
      REDIS_URL:    redis://redis:6379
      JWT_SECRET:   ${JWT_SECRET}
      HOME:         /var/lib/omnitron   # daemon state → $HOME/.omnitron
    volumes:
      - omnitron-data:/var/lib/omnitron
    ports:
      - '3001:3001'      # api HTTP (per-app Netron transport)
      - '9801:9801'      # daemon Netron HTTP RPC
    depends_on:
      postgres: { condition: service_healthy }
      redis:    { condition: service_healthy }
    command: ['omnitron', 'up', '--foreground']

  postgres:
    image: postgres:16-alpine
    restart: unless-stopped
    environment:
      POSTGRES_USER:     platform
      POSTGRES_PASSWORD: ${DB_PASS}
      POSTGRES_DB:       platform
    volumes:
      - pg-data:/var/lib/postgresql/data
    healthcheck:
      test:     ['CMD', 'pg_isready', '-U', 'platform']
      interval: 10s
      timeout:  5s
      retries:  5

  redis:
    image: redis:7-alpine
    restart: unless-stopped
    command: redis-server --maxmemory 256mb --maxmemory-policy allkeys-lru
    volumes:
      - redis-data:/data
    healthcheck:
      test:     ['CMD', 'redis-cli', 'ping']
      interval: 10s
      timeout:  5s
      retries:  5

  nginx:
    image: nginx:alpine
    restart: unless-stopped
    ports:
      - '80:80'
      - '443:443'
    volumes:
      - ./infra/nginx/nginx.conf:/etc/nginx/nginx.conf:ro
      - ./infra/nginx/certs:/etc/nginx/certs:ro
    depends_on:
      - app

volumes:
  pg-data:
  redis-data:
  omnitron-data:
```

`omnitron up --foreground` keeps the daemon attached to the
container's foreground so Docker's restart policy works.

> **The Console UI (port 9800) is served by a separate
> `omnitron-nginx` container that the daemon provisions over the
> Docker API**, not by the daemon process itself. The daemon's own
> ports are: `9700` (TCP fleet RPC), `9801` (Netron HTTP RPC),
> `9802` (Netron WebSocket), `9803` (`/metrics` + `/healthz`).
> Nginx proxies `9800 → 9801/9802` via `host.docker.internal`. If
> you run the daemon inside this container and want it to manage
> sibling infra/console containers, mount the Docker socket
> (`/var/run/docker.sock`) and reach the daemon on the host. Many
> single-container deployments instead run the daemon with
> `--no-infra` and front it with their own nginx (the `nginx`
> service below).

## The Dockerfile

```dockerfile
# Dockerfile
FROM node:22-alpine AS builder
WORKDIR /app

# Install pnpm
RUN corepack enable && corepack prepare pnpm@latest --activate

# Copy manifests for cache efficiency
COPY pnpm-lock.yaml pnpm-workspace.yaml package.json ./
COPY packages packages
COPY apps     apps
COPY omnitron.config.ts ./

# Install + build
RUN pnpm install --frozen-lockfile
RUN pnpm build

# ---- Runtime stage ----
FROM node:22-alpine
WORKDIR /app

RUN apk add --no-cache tini && corepack enable && corepack prepare pnpm@latest --activate

# Copy built artefacts only
COPY --from=builder /app/node_modules ./node_modules
COPY --from=builder /app/packages      ./packages
COPY --from=builder /app/apps          ./apps
COPY --from=builder /app/omnitron.config.ts ./

# Non-root user
RUN addgroup -S omni && adduser -S omni -G omni && \
    mkdir -p /var/lib/omnitron && chown -R omni:omni /var/lib/omnitron
USER omni

# Daemon state lives under $HOME/.omnitron — point HOME at the volume.
ENV NODE_ENV=production HOME=/var/lib/omnitron
EXPOSE 3001 9801 9802 9803

ENTRYPOINT ['/sbin/tini', '--']
CMD ['pnpm', 'omnitron', 'up', '--foreground']
```

Key choices:

- **Multi-stage**: build artefacts left behind in the builder
  stage; runtime image stays small (\~150 MB).
- **`tini` PID 1**: reaps zombie processes from forked workers;
  forwards signals correctly to the daemon.
- **Non-root user**: standard hardening.
- **No `npm install` at runtime**: predictable image.

## `.env` file

```bash
# .env (gitignored)
DB_PASS=$(openssl rand -hex 32)
JWT_SECRET=$(openssl rand -hex 48)
```

`docker compose` auto-loads `.env` from the working directory.
For real deploys, source from a secret manager and inject as
env at container start.

## Volumes

Three persistent volumes:

| Volume          | Mount                      | Purpose                                                                                 |
| --------------- | -------------------------- | --------------------------------------------------------------------------------------- |
| `pg-data`       | `/var/lib/postgresql/data` | Postgres data                                                                           |
| `redis-data`    | `/data`                    | Redis snapshots                                                                         |
| `omnitron-data` | `/var/lib/omnitron`        | Daemon state (`$HOME/.omnitron`): SQLite state at `data/daemon-state.db`, secrets, logs |

**Back up `pg-data` regularly** — that's the only volume whose
loss is catastrophic. The rest can be reconstructed.

```bash
# Backup
docker compose exec postgres pg_dumpall -U platform > backup.sql

# Or use the built-in:
docker compose exec app omnitron backup create
```

## Build + ship

```bash
# Build local:
docker compose build

# Tag + push to a registry:
docker build -t registry.example.com/platform:$(git rev-parse --short HEAD) .
docker push registry.example.com/platform:$(git rev-parse --short HEAD)

# Deploy on the target host:
ssh server "cd /srv/platform && docker compose pull && docker compose up -d"
```

## Health checks

The compose `healthcheck` for the app container:

```yaml
services:
  app:
    healthcheck:
      test:         ['CMD', 'pnpm', 'omnitron', 'ping']
      interval:     30s
      timeout:      10s
      retries:      3
      start_period: 30s     # grace for cold start
```

`omnitron ping` returns 0 when the daemon is reachable.

## Logging

App logs go to stdout (pino JSON); compose forwards to its
logging driver:

```yaml
services:
  app:
    logging:
      driver: 'json-file'
      options:
        max-size: '50m'
        max-file: '10'
```

For production, ship to Loki / Datadog / Cloudwatch via the
docker logging driver:

```yaml
services:
  app:
    logging:
      driver: 'loki'
      options:
        loki-url: 'https://loki.example.com/loki/api/v1/push'
```

## Resource limits

```yaml
services:
  app:
    deploy:
      resources:
        limits:
          cpus:   '2.0'
          memory: 2G
        reservations:
          cpus:   '0.5'
          memory: 512M
```

`docker compose up` honours these as soft limits; `docker swarm
deploy` enforces them.

## Updating the stack

Zero-downtime rolling update on a single host:

```bash
# Pull the new image:
docker compose pull app

# Recreate just the app container (DB + Redis stay up):
docker compose up -d --no-deps app
```

For true zero-downtime, run the daemon in the container with
`omnitron reload` instead of restart:

```bash
docker compose exec app omnitron reload
```

This works because reload cycles workers within the running
daemon — no container restart.

## Compose for dev

A separate `docker-compose.dev.yml`:

```yaml
version: '3.9'

services:
  postgres:
    extends:
      file: docker-compose.yml
      service: postgres
    ports:
      - '5432:5432'      # expose to host for psql

  redis:
    extends:
      file: docker-compose.yml
      service: redis
    ports:
      - '6379:6379'

# No 'app' service — run the app locally via `pnpm dev`
```

Boot:

```bash
docker compose -f docker-compose.dev.yml up -d
pnpm dev                                # app runs on host
```

## What this doesn't give you

- **Multi-host** — for that, `docker swarm deploy` or move to
  Kubernetes.
- **Per-app autoscaling** based on CPU/memory metrics —
  Omnitron's autoscaler works inside the daemon, not at the
  container level.
- **Managed TLS / DNS** — bring nginx-proxy / Caddy / Traefik
  in front, or a managed LB.
- **Multi-region** — deploy per region; coordinate via DNS or
  Omnitron cluster mode.

For those, see [Kubernetes](/docs/deployment/kubernetes) or
[Cluster + Fleet](/docs/omnitron/cluster).

## See also

- [Deployment overview](/docs/deployment)
- [Best practices](/docs/omnitron/best-practices)
- [Configuration](/docs/omnitron/configuration) — `omnitron.config.ts` reference

---

---
id: "deployment/index"
title: "Overview"
description: "How to deploy Omnitron-supervised apps — Docker, k8s, PaaS, bare-metal."
---

# Deployment

How to take a Titan app from `pnpm dev` to production. Four
paths covered in detail, plus the cross-cutting concerns.

## Choose your path

| Where                      | Use                                                                                       |
| -------------------------- | ----------------------------------------------------------------------------------------- |
| **One box, you own it**    | [Docker Compose](/docs/deployment/docker) — daemon + apps + infra in one stack            |
| **Container orchestrator** | [Kubernetes](/docs/deployment/kubernetes) — daemon as a stateful pod, apps as deployments |
| **Platform-as-a-Service**  | [Fly.io / Railway / Render](/docs/deployment/paas) — opinionated, fast to ship            |
| **Bare-metal / VM**        | [systemd](/docs/deployment/bare-metal) — daemon as a system service                       |

For dev / staging / prod parity, run the same `omnitron.config.ts`
across environments. Only secrets + scale parameters change per
deploy target.

## Cross-cutting concerns

### What you ship

A production deployment carries:

1. **Compiled app code** — `dist/` for each app.
2. **`omnitron.config.ts`** — the ecosystem config.
3. **Per-app `config/default.json`** — declarative infra.
4. **Static webapp bundle** (optional) — `apps/omnitron/webapp/dist/`.
5. **Migrations** — `migrations/*.sql`.

That's it. No generated client code, no schema sync artefacts,
no shipped node_modules.

### What you provide at the target

- **Node.js 22+**.
- **PostgreSQL** (managed or self-hosted) — connection string in
  env.
- **Redis** (managed or self-hosted) — connection string in env.
- **Object storage** (S3 / MinIO) — credentials in env.
- **Secrets manager** (optional) — Omnitron's encrypted store
  works standalone; for prod prefer a real secret manager.

### Environment variables — the canonical set

| Variable            | Purpose                                                  |
| ------------------- | -------------------------------------------------------- |
| `NODE_ENV`          | `'production'`                                           |
| `HOME`              | Daemon state lives under `$HOME/.omnitron/`              |
| `DATABASE_URL`      | Postgres connection string                               |
| `REDIS_URL`         | Redis connection string                                  |
| `JWT_SECRET`        | Signing key — long random; from secret manager           |
| `OMNITRON_SOCKET`   | Override the daemon Unix socket path (optional)          |
| `OMNITRON_NO_INFRA` | `1` to skip Docker infra provisioning on boot            |
| `<APP>_*`           | Per-app overrides (matches the app's `config.envPrefix`) |

The daemon resolves its home directory from `$HOME`
(`$HOME/.omnitron/`) — there is no `OMNITRON_HOME` override. Set
`HOME` for the daemon's user to relocate state.

Per-environment values flow in through each app's
`config.sources` / `config.envPrefix` (Titan's config module) —
favour env over config files for per-environment values.

### Logs and metrics

- **Logs** stream to stdout (pino JSON) — ingest with your log
  aggregator's stdin-tail or sidecar.
- **Metrics** scrape the daemon's Prometheus endpoint at
  `/metrics` on its dedicated metrics port (`httpPort + 3`,
  default `9803`, bound to loopback unless you opt in via host +
  bearer token), or push to a backend via `titan-telemetry-relay`.
- **Health probes** — the daemon's metrics port also serves
  `/healthz` (always public, returns `200 ok`). Per-app health is
  exposed over Netron RPC by `titan-health` (the `live()` /
  `ready()` methods) — `omnitron health` surfaces it. There are no
  `/healthz` / `/readyz` HTTP routes on the app transport ports.

### Secrets

Three options, in order of preference:

1. **Cloud secret manager** (AWS Secrets Manager, GCP Secret
   Manager, Vault) — inject as env at pod / container boot.
2. **Encrypted file** (`~/.omnitron/secrets.enc`) — works
   without external dependencies; rotate the passphrase via
   env.
3. **Plain env vars** — fine for non-sensitive config, not for
   credentials.

Never commit secrets to `omnitron.config.ts` or
`config/default.json`.

### Migrations

```bash
# In the deploy script, before starting the daemon:
omnitron infra migrate
# or for one app:
omnitron infra migrate api
```

Migrations run inside a Postgres advisory lock — concurrent
deploys are safe; only one will run, others wait.

### Backups

```bash
# Manual (defaults to the 'omnitron' database; pass a name for others):
omnitron backup create
omnitron backup create main

# List / restore:
omnitron backup list
omnitron backup restore <id>
```

For scheduled backups use `omnitron backup schedule <target> <cron|hourly|daily|weekly|ms>`; schedules persist across daemon
restarts, and `omnitron backup schedules` lists them. Host cron
driving `omnitron backup create` is the alternative when the
backup must run whether or not the daemon is up — see the
[bare-metal guide](/docs/deployment/bare-metal#backups).

### Zero-downtime deploys

The `reload` command cycles workers one-by-one:

```bash
omnitron reload api
```

In **module-worker mode**, the worker pool maintains capacity
throughout the reload. In **classic mode**, a new bootstrap is
forked side-by-side; the old one drains.

For blue/green or canary, [`omnitron deploy app <app>`](/docs/omnitron/cli#deployment)
takes `--strategy <rolling|all-at-once|blue-green|canary>`
(default `all-at-once`).

### Multi-region

| Strategy                                 | Pattern                                                           |
| ---------------------------------------- | ----------------------------------------------------------------- |
| **One cluster per region** (recommended) | Each region elects its own leader; cross-region calls via ingress |
| **Single global cluster**                | All daemons peer; sensitive to partition                          |
| **Fleet without cluster**                | Each daemon independent; addressed by alias                       |

See [Cluster + Fleet / Multi-region patterns](/docs/omnitron/cluster#multi-region-patterns).

### Observability checklist

- [ ] Logs → SIEM / log aggregator.
- [ ] Metrics → Prometheus / managed metrics backend.
- [ ] Traces → OTel collector (if `titan-tracing` configured).
- [ ] Health probes → load balancer.
- [ ] Uptime monitoring → external pinger of `/healthz`.
- [ ] Error reporting → Sentry via netron-browser middleware.
- [ ] Alerts → `OmnitronAlerts` with delivery webhooks.

### Security checklist (deploy time)

- [ ] TLS terminated at the gateway / load balancer (not the app).
- [ ] Strong `JWT_SECRET`, rotated quarterly.
- [ ] Omnitron TCP port (9700) firewalled to internal only.
- [ ] Database password from secret manager, never env literal.
- [ ] CORS allowlist limited to known origins.
- [ ] CSP header on webapp.
- [ ] Rate limits configured per public endpoint.
- [ ] Audit logs going to append-only sink.

See [Auth & RBAC](/docs/omnitron/auth-rbac) and the per-module
[Security checklist](/docs/titan/modules/security-checklist).

### Rollback

```bash
# Roll back an app on a target server (alias or tag):
omnitron rollback api --target web-1
```

`omnitron rollback <app> --target <server>` requires a target —
it operates against a registered remote daemon. Build artefacts
are versioned tarballs under `<projectRoot>/.omnitron/artifacts/`
(`<app>-<version>.tar.gz`); prune old ones with the
`ArtifactBuilder` retention helper (keeps the latest N per app).

> ⚠️ **NEEDS REWRITE** — there is no `deployment: { retain,
> versionTag }` block in `omnitron.config.ts`, and `rollback`
> has no `--version` flag. Artefact retention is handled by the
> builder, not declarative config.

## Read the per-target guides

- [Docker Compose](/docs/deployment/docker) — single-host development + staging
- [Kubernetes](/docs/deployment/kubernetes) — production at scale
- [PaaS](/docs/deployment/paas) — Fly.io / Railway / Render quickstart
- [Bare-metal / systemd](/docs/deployment/bare-metal) — own-server deploys

## See also

- [Best practices](/docs/omnitron/best-practices) — production patterns
- [Recipes](/docs/omnitron/recipes) — concrete topologies
- [Architecture](/docs/omnitron/architecture) — what runs where
- [Cluster + Fleet](/docs/omnitron/cluster) — multi-node

---

---
id: "deployment/kubernetes"
title: "Kubernetes"
description: "Production-grade Kubernetes deployment patterns."
---

# Kubernetes

For production at scale. The pattern: one Deployment per
Titan app, one Statefulset for the daemon (with its persistent
state), managed Postgres + Redis as separate clusters.

## Topology

```mermaid
flowchart TB
  Ingress[Ingress / Gateway]
  Ingress --> APIsvc[Service: api]
  Ingress --> WebappSvc[Service: webapp]

  subgraph apps["Apps (one Deployment each)"]
    APIsvc --> ApiDeploy[Deployment: api<br/>replicas: 3]
    WebappSvc --> WebappDeploy[Deployment: webapp]
  end

  subgraph daemon["Omnitron daemon (StatefulSet)"]
    DaemonSts[Daemon Pod<br/>PVC: ~/.omnitron]
  end

  subgraph infra["Managed infrastructure"]
    PG[(Postgres<br/>managed)]
    Redis[(Redis<br/>managed)]
    S3[(S3<br/>managed)]
  end

  ApiDeploy --> PG
  ApiDeploy --> Redis
  ApiDeploy --> S3
  ApiDeploy -. report status .-> DaemonSts
  WebappDeploy --> DaemonSts
```

In Kubernetes you have two valid patterns:

| Pattern                                                      | When                                                                                                           |
| ------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------- |
| **Daemon-per-pod** (Omnitron supervises the apps in the pod) | Small clusters; one or two services per pod                                                                    |
| **K8s-supervises-apps** (Omnitron is the control plane only) | Large clusters; one Deployment per Titan app; Omnitron daemon as a separate StatefulSet for control/inspection |

This page covers pattern 2 — the production default.

## Per-app Deployment

```yaml
# k8s/api/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: api
  namespace: platform
spec:
  replicas: 3
  selector:
    matchLabels: { app: api }
  template:
    metadata:
      labels: { app: api }
    spec:
      containers:
        - name: api
          image: registry.example.com/platform-api:v1.4.2
          ports:
            - { name: http, containerPort: 3001 }
          env:
            - { name: NODE_ENV, value: production }
            - name: DATABASE_URL
              valueFrom: { secretKeyRef: { name: api-secrets, key: database-url } }
            - name: REDIS_URL
              valueFrom: { secretKeyRef: { name: api-secrets, key: redis-url } }
            - name: JWT_SECRET
              valueFrom: { secretKeyRef: { name: api-secrets, key: jwt-secret } }
          resources:
            requests: { cpu: '500m', memory: '512Mi' }
            limits:   { cpu: '2000m', memory: '2Gi' }
          livenessProbe:
            httpGet: { path: /health, port: http }
            initialDelaySeconds: 30
            periodSeconds:       10
            timeoutSeconds:      5
            failureThreshold:    3
          readinessProbe:
            httpGet: { path: /health, port: http }
            initialDelaySeconds: 5
            periodSeconds:       3
            timeoutSeconds:      3
            failureThreshold:    2
          startupProbe:
            httpGet: { path: /health, port: http }
            initialDelaySeconds: 0
            periodSeconds:       2
            timeoutSeconds:      2
            failureThreshold:    60      # 2 min grace for cold start
          lifecycle:
            preStop:
              exec:
                command: ['sh', '-c', 'sleep 10']    # let LB drain
      terminationGracePeriodSeconds: 45
---
apiVersion: v1
kind: Service
metadata:
  name: api
  namespace: platform
spec:
  selector: { app: api }
  ports:
    - { name: http, port: 80, targetPort: http }
```

Key points:

- **Three probes**: liveness, readiness, startup. The Titan
  Netron HTTP transport serves **`/health`** on each process's
  HTTP port — it returns `200` with `{"status":"online"}` once the
  transport is up, `503` otherwise. That single endpoint backs all
  three probes here.

  - **Liveness** → `/health`. Kill if it fails persistently.
  - **Readiness** → `/health`. Stop sending traffic when
    failing; resume when passing.
  - **Startup** → `/health` with longer `failureThreshold` —
    masks liveness during cold-start (especially when
    eager-loading heavy services).

  > ⚠️ **NEEDS REWRITE** — there are **no `/healthz` or `/readyz`
  > HTTP routes**. Richer health (the `live()` / `ready()` probes
  > from `titan-health`, dependency checks) is exposed over Netron
  > RPC, not HTTP, so it isn't directly reachable by a k8s
  > `httpGet` probe. If you need a true readiness signal that
  > reflects DB/Redis health, add a `customRoutes` HTTP handler in
  > the process's `transports.http` config that calls the health
  > service, and point `readinessProbe` at it.
- **`preStop` sleep 10 s** — lets the load balancer's endpoint
  list update before SIGTERM. Avoids 502s during pod rotation.
- **`terminationGracePeriodSeconds: 45`** — > app's
  `shutdown.timeout`. Otherwise k8s SIGKILLs mid-drain.

## Omnitron daemon as StatefulSet

The daemon needs persistent state (`~/.omnitron/`) and a stable
identity:

```yaml
# k8s/daemon/statefulset.yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: omnitron
  namespace: platform
spec:
  serviceName: omnitron
  replicas: 1
  selector:
    matchLabels: { app: omnitron }
  template:
    metadata:
      labels: { app: omnitron }
    spec:
      containers:
        - name: omnitron
          image: registry.example.com/platform-omnitron:v1.4.2
          ports:
            - { name: tcp,     containerPort: 9700 }   # fleet RPC
            - { name: http,    containerPort: 9801 }   # Netron HTTP RPC
            - { name: ws,      containerPort: 9802 }   # Netron WebSocket
            - { name: metrics, containerPort: 9803 }   # /metrics + /healthz
          env:
            # Daemon state lives under $HOME/.omnitron — point HOME at the PVC.
            - { name: HOME,     value: /var/lib/omnitron }
            - { name: NODE_ENV, value: production }
          volumeMounts:
            - { name: state, mountPath: /var/lib/omnitron }
          resources:
            requests: { cpu: '200m', memory: '256Mi' }
            limits:   { cpu: '1000m', memory: '1Gi' }
  volumeClaimTemplates:
    - metadata: { name: state }
      spec:
        accessModes: ['ReadWriteOnce']
        resources: { requests: { storage: 5Gi } }
        storageClassName: 'fast-ssd'
---
apiVersion: v1
kind: Service
metadata:
  name: omnitron
  namespace: platform
spec:
  selector: { app: omnitron }
  ports:
    - { name: tcp,     port: 9700, targetPort: tcp }
    - { name: http,    port: 9801, targetPort: http }
    - { name: ws,      port: 9802, targetPort: ws }
    - { name: metrics, port: 9803, targetPort: metrics }
```

The daemon does **not** serve the Console UI itself — that's the
separate [`webapp` Deployment](#webapp-deployment) (an nginx
container that proxies to the daemon's `9801`/`9802`). To reach
the daemon's RPC directly, operators can
`kubectl port-forward svc/omnitron 9801`; to open the Console,
port-forward the `webapp` service instead.

## Ingress

```yaml
# k8s/ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: platform
  namespace: platform
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt
    nginx.ingress.kubernetes.io/proxy-body-size: '50m'
spec:
  ingressClassName: nginx
  tls:
    - hosts: [api.example.com, app.example.com]
      secretName: platform-tls
  rules:
    - host: api.example.com
      http:
        paths:
          - { path: /, pathType: Prefix, backend: { service: { name: api, port: { number: 80 } } } }
    - host: app.example.com
      http:
        paths:
          - { path: /, pathType: Prefix, backend: { service: { name: webapp, port: { number: 80 } } } }
```

Two hosts — one for the API (Titan apps), one for the Console
webapp (the nginx-served [`webapp` Deployment](#webapp-deployment),
**not** the daemon).

## Secrets

```yaml
# k8s/secrets.yaml — do NOT commit. Use sealed-secrets or external-secrets in real life.
apiVersion: v1
kind: Secret
metadata:
  name: api-secrets
  namespace: platform
type: Opaque
stringData:
  database-url: postgres://platform@platform-pg.svc.cluster.local:5432/platform
  redis-url:    redis://platform-redis.svc.cluster.local:6379
  jwt-secret:   '<set via secret manager>'
```

Production options:

- **Sealed Secrets** (Bitnami): commit encrypted secrets to git.
- **External Secrets Operator**: sync from AWS Secrets Manager /
  GCP Secret Manager / Vault.
- **Pod-level secret manager mount** (e.g., AWS Secrets Store CSI
  Driver).

## HorizontalPodAutoscaler

```yaml
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: api
  namespace: platform
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind:       Deployment
    name:       api
  minReplicas: 2
  maxReplicas: 10
  metrics:
    - type: Resource
      resource: { name: cpu,    target: { type: Utilization, averageUtilization: 70 } }
    - type: Resource
      resource: { name: memory, target: { type: Utilization, averageUtilization: 80 } }
```

`titan-pm`'s own autoscaler operates at the worker-pool level
within a process; HPA operates at the pod level. Use both —
HPA for cross-pod scale, `titan-pm` for cross-worker.

## PodDisruptionBudget

```yaml
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: api
  namespace: platform
spec:
  minAvailable: 2
  selector:
    matchLabels: { app: api }
```

Keeps at least 2 pods up during voluntary disruptions (node
drains, cluster upgrades).

## NetworkPolicy

```yaml
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: api-ingress
  namespace: platform
spec:
  podSelector:
    matchLabels: { app: api }
  policyTypes: [Ingress, Egress]
  ingress:
    - from:
        - namespaceSelector: { matchLabels: { name: ingress-nginx } }
      ports:
        - { protocol: TCP, port: 3001 }
  egress:
    - to:
        - namespaceSelector: { matchLabels: { kubernetes.io/metadata.name: platform } }
    - to:
        - namespaceSelector: { matchLabels: { kubernetes.io/metadata.name: kube-system } }
      ports:
        - { protocol: UDP, port: 53 }
```

Allow ingress only from the nginx ingress controller; restrict
egress to in-namespace + DNS.

## Rolling updates

```yaml
spec:
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge:       1
      maxUnavailable: 0
```

`maxUnavailable: 0` keeps capacity at 100% throughout the
rollout. `maxSurge: 1` adds one new pod, waits for it to become
ready, then terminates an old one.

For high-throughput apps, set `maxSurge: 25%` for faster
rollouts.

## Database migrations

Run as an initContainer or pre-deploy Job:

```yaml
# Pre-deploy Job, runs to completion before pods rotate
apiVersion: batch/v1
kind: Job
metadata:
  name: migrate-{{ .Values.deployVersion }}
  namespace: platform
spec:
  backoffLimit: 1
  template:
    spec:
      restartPolicy: Never
      containers:
        - name: migrate
          image: registry.example.com/platform-api:v1.4.2
          command: ['pnpm', 'omnitron', 'infra', 'migrate']
          env:
            - { name: DATABASE_URL, valueFrom: { secretKeyRef: { name: api-secrets, key: database-url } } }
```

Wire as a Helm pre-install / pre-upgrade hook.

## Webapp deployment

Static assets behind an nginx container:

```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: webapp
spec:
  replicas: 2
  template:
    spec:
      containers:
        - name: webapp
          image: registry.example.com/platform-webapp:v1.4.2
          ports: [{ name: http, containerPort: 80 }]
          # Built static; nginx serves it
```

This mirrors what `omnitron webapp` does on a single host: an
`nginx:alpine` container serving the built Console SPA from
`/usr/share/nginx/html` and proxying `/netron/`, `/ws`,
`/api/health`, `/api/metrics` to the daemon. In k8s, point its
upstream at the `omnitron` Service (`9801` HTTP, `9802` WS)
instead of `host.docker.internal`.

## Observability

- **Logs** → stdout → cluster log aggregator (Loki / Datadog /
  ELK).
- **Metrics** → scrape the **daemon's** Prometheus endpoint on
  port `9803` (`/metrics`), which aggregates per-app gauges:
  ```yaml
  # On the omnitron StatefulSet pod template:
  annotations:
    prometheus.io/scrape: 'true'
    prometheus.io/port:   '9803'
    prometheus.io/path:   '/metrics'
  ```
  The per-app HTTP transport port (e.g. `3001`) also has a
  `/metrics` route, but it returns **JSON transport stats and is
  bearer-token gated** — it is not Prometheus exposition text, so
  don't point a scrape config at it. Bind the daemon metrics port
  to `0.0.0.0` and set a bearer token if Prometheus runs off-host.
- **Traces** → OTel collector as DaemonSet; apps export to it (via
  `titan-telemetry-relay`).
- **Alerts** → `OmnitronAlerts` + PrometheusRules.

## Disaster recovery

Backup priorities:

1. **Postgres** — managed snapshots + WAL archiving.
2. **Omnitron daemon state** (`omnitron-data` PVC) — snapshot
   regularly; survival of the daemon is important for the
   uptime-bar history.
3. **Secrets** — backed up via the secret manager itself.

The Postgres data is the only **catastrophic** loss.
Everything else can be reconstructed.

## Cost-savings tips

- **Spot / preemptible** for worker pools; on-demand for the
  daemon.
- **Memory requests > limits, CPU limits > requests**: gives
  headroom under burst while bin-packing efficiently.
- **One daemon per cluster**, not per app.
- **`titan-cache` L1 only** for read-heavy apps to avoid Redis
  hops.

## See also

- [Deployment overview](/docs/deployment)
- [PaaS](/docs/deployment/paas) — for smaller setups
- [Cluster + Fleet](/docs/omnitron/cluster) — multi-cluster patterns
- [Observability](/docs/omnitron/observability)

---

---
id: "deployment/paas"
title: "PaaS (Fly.io / Railway / Render)"
description: "Opinionated PaaS deploys for fast time-to-prod."
---

# PaaS

For when you want shipping speed over operational control.
Three popular options covered below. All share the same shape:

1. Push your Dockerfile.
2. Configure env vars / secrets.
3. Provision managed Postgres + Redis from the same provider.
4. Done.

## Fly.io

`fly.toml`:

```toml
app = "platform"
primary_region = "fra"

[build]
  dockerfile = "Dockerfile"

[env]
  NODE_ENV = "production"
  HOME     = "/var/lib/omnitron"   # daemon state → $HOME/.omnitron (no OMNITRON_HOME var)

[mounts]
  source      = "omnitron_data"
  destination = "/var/lib/omnitron"

[http_service]
  internal_port = 3001                 # public app (Titan HTTP transport)
  force_https   = true
  auto_stop_machines  = false
  auto_start_machines = true
  min_machines_running = 1
  # The app transport serves /health (JSON {status:"online"}); use that for the
  # HTTP check below. There is no /healthz on this port — the daemon's /healthz
  # lives on port 9803, bound to loopback by default.
  [[http_service.checks]]
    method   = "GET"
    path     = "/health"
    interval = "15s"
    timeout  = "5s"

[[vm]]
  cpu_kind = "shared"
  cpus     = 2
  memory_mb = 2048
```

```bash
# Provision DB:
fly postgres create --name platform-pg --region fra
fly postgres attach platform-pg

# Provision Redis:
fly redis create --name platform-redis --region fra
fly redis attach platform-redis

# Secrets:
fly secrets set JWT_SECRET=$(openssl rand -hex 48)

# Deploy:
fly deploy
```

Fly auto-injects `DATABASE_URL` and `REDIS_URL` env vars from
the attached resources. Persistent volume preserves
`~/.omnitron/` across deploys.

For multi-region:

```bash
fly scale count 3 --region fra,ams,lhr     # 3 machines, one per region
```

Each region's machine sees the local Postgres/Redis (Fly
replicates them). For Omnitron cluster mode, set `cluster.peers`
to the Fly internal DNS names.

## Railway

`railway.json`:

```json
{
  "$schema": "https://railway.app/railway.schema.json",
  "build":   { "builder": "DOCKERFILE" },
  "deploy": {
    "startCommand":         "pnpm omnitron up --foreground",
    "healthcheckPath":      "/health",
    "healthcheckTimeout":   30,
    "restartPolicyType":    "ON_FAILURE",
    "restartPolicyMaxRetries": 10
  }
}
```

Railway probes the published port, so `healthcheckPath` must hit
a route on the app's HTTP transport — that's **`/health`** (JSON
`{"status":"online"}`), not `/healthz`. The `/healthz` route only
exists on the daemon's separate metrics port (`9803`).

Workflow:

1. Push the repo to GitHub.
2. `railway up` or connect via dashboard.
3. Add Postgres + Redis from Railway templates (one click each).
4. Set `JWT_SECRET` via the dashboard.
5. Railway auto-detects the Dockerfile and deploys.

Env vars (auto-injected):

- `DATABASE_URL` — from the linked Postgres
- `REDIS_URL` — from the linked Redis
- `PORT` — Railway-assigned. Omnitron doesn't read `PORT`; each
  process declares an explicit port in its `transports.http.port`
  (e.g. `3001`). Pin the published port to that, or template the
  port into the process config from an env var you set yourself.

For persistent volumes (Omnitron state), add a Volume in the
dashboard, mount it at `/var/lib/omnitron`, and set `HOME` to the
same path so the daemon writes its `~/.omnitron` state there.

## Render

`render.yaml`:

```yaml
services:
  - type: web
    name: platform
    runtime: docker
    dockerfilePath: ./Dockerfile
    plan: standard
    region: oregon
    autoDeploy: true
    healthCheckPath: /health          # app transport route; /healthz is daemon-only (port 9803)
    envVars:
      - { key: NODE_ENV, value: production }
      - { key: HOME, value: /var/lib/omnitron }   # daemon state → $HOME/.omnitron
      - { key: JWT_SECRET, generateValue: true }
      - { fromDatabase: { name: platform-pg, property: connectionString }, key: DATABASE_URL }
      - { fromService:  { name: platform-redis, type: redis, property: connectionString }, key: REDIS_URL }
    disk:
      name: omnitron-state
      mountPath: /var/lib/omnitron
      sizeGB: 5

databases:
  - name: platform-pg
    plan: standard
    region: oregon

  - type: redis
    name: platform-redis
    plan: standard
    region: oregon
```

```bash
# Or just connect the repo via the dashboard — render.yaml is auto-detected.
```

Render handles the rest. `generateValue: true` mints a JWT
secret on first deploy.

## Common patterns across PaaS

### Boot command

Always:

```bash
pnpm omnitron up --foreground
```

`--foreground` keeps Node attached so the PaaS supervisor sees a
running process; the daemon takes over signal handling.

### Health check path

For a check against the **published app port**, use `/health` —
the Titan HTTP transport returns `200` `{"status":"online"}`
there. The Prometheus-style `/healthz` (plain `200 ok`) lives
only on the **daemon's metrics port** (`9803`); target it with a
port-aware check (Fly `[checks]`) if your PaaS allows one.
Neither `/readyz` nor an HTTP readiness route exists out of the
box — see the [k8s note](/docs/deployment/kubernetes#per-app-deployment) for
adding one via `customRoutes`.

### Persistent volume

Mount a volume at, e.g., `/var/lib/omnitron` and set `HOME` to
the same path so the daemon's `~/.omnitron` (PID, SQLite state,
secrets store) persists. Without it, every deploy is a fresh
daemon — fine for stateless workloads, loses uptime history +
secrets on restart.

### Secrets

Always via the PaaS secret manager. Never in `omnitron.config.ts`
or `config/default.json`.

### Database migrations

Most PaaS support a "pre-deploy" or "release" command:

```toml
# Fly.io fly.toml
[deploy]
  release_command = "pnpm omnitron infra migrate"

# Railway — separate one-off command
# Render — pre-deploy script in render.yaml
```

Runs once per deploy, before the new image takes traffic.

### Scaling

| PaaS    | How                                    |
| ------- | -------------------------------------- |
| Fly     | `fly scale count N --region X`         |
| Railway | Dashboard slider (replicas)            |
| Render  | Plan-based; auto-scaling on paid plans |

PaaS scaling is **horizontal** (more pods). Omnitron-level
scaling (worker pools within a pod) is configured **per process**
in `defineSystem` — set `instances` on a process and give it a
`scaling` block (`{ strategy, maxInstances, targetCPU, ... }`).
There is no app-level `scaling` block.

### Logs

Each PaaS aggregates container stdout. For production:

- **Fly** — `fly logs` for live tail; ship to Datadog/Logtail.
- **Railway** — Dashboard live tail; webhook to external log
  aggregator.
- **Render** — Dashboard live tail; log streams to S3/SIEM.

Set `logging.level` to `'info'` in production; `'debug'` only
for time-bounded investigations.

## Multi-app on one PaaS service

A single Omnitron daemon can supervise multiple Titan apps in
one container. Configure in `omnitron.config.ts`:

```typescript
export default defineEcosystem({
  apps: [
    { name: 'api',    bootstrap: './apps/api/dist/bootstrap.js' },
    { name: 'worker', bootstrap: './apps/worker/dist/bootstrap.js' },
    { name: 'ws',     bootstrap: './apps/ws/dist/bootstrap.js' },
  ],
  // ...
});
```

Then expose ports for each via PaaS multi-port config (Fly) or
multiple services (Railway).

Single-container deployment is cheaper but losses isolation —
all apps share the container's resources.

## PaaS limitations

- **Filesystem ephemerality** — without explicit volumes,
  `~/.omnitron/` is reset on every deploy. Volumes are mandatory
  for production.
- **PostgreSQL hosting** — managed Postgres on each PaaS is
  fine for hundreds of req/s. For TB-scale data, move to AWS RDS
  / GCP Cloud SQL.
- **Compute limits** — PaaS billing tiers cap CPU / RAM. Heavy
  workloads outgrow them.
- **Cluster mode** — Omnitron's cluster + fleet patterns assume
  long-lived daemons with known addresses. PaaS often
  reschedules containers on different hosts. Use single-node
  daemon per region with managed Postgres / Redis instead.

For more control, see [Kubernetes](/docs/deployment/kubernetes) or
[Docker Compose on a VM](/docs/deployment/docker).

## Cost rough-order

Roughly per month for a small production setup:

| PaaS    | Compute + DB + Redis                      | Total    |
| ------- | ----------------------------------------- | -------- |
| Fly     | shared CPU, 2 GB + small DB + small Redis | \$30–60  |
| Railway | starter tier + DB + Redis                 | \$20–50  |
| Render  | standard plan + DB + Redis                | \$50–100 |

Numbers vary widely by traffic. PaaS shines for $0–$300/mo
range; beyond that, Kubernetes catches up on cost.

## See also

- [Docker Compose](/docs/deployment/docker) — single-host alternative
- [Kubernetes](/docs/deployment/kubernetes) — when you outgrow PaaS
- [Bare-metal](/docs/deployment/bare-metal) — own-server option
- [Configuration](/docs/omnitron/configuration)

---

---
id: "deployment/production-readiness"
title: "Production readiness"
description: "A practical acceptance checklist for deployment, observability, access control, and recovery."
---

# Production readiness

A process that starts is only the first check. Before admitting production
traffic, verify the application contract, its dependencies, the deployment
artifact and the recovery path.

## Define what is ready

| Gate          | Evidence to collect                                            | Reference                                     |
| ------------- | -------------------------------------------------------------- | --------------------------------------------- |
| Build         | A reproducible artifact from a known commit and lockfile       | [Deployment](/docs/deployment)                |
| Configuration | Required values validated; secrets supplied at runtime         | [Configuration](/docs/omnitron/configuration) |
| Data          | Migrations completed and application credentials tested        | [Database](/docs/titan/modules/database)      |
| Application   | A real RPC call succeeds with the intended identity            | [Netron](/docs/titan/netron)                  |
| Readiness     | Required dependencies are reachable; failures remove readiness | [Health](/docs/titan/modules/health)          |
| Observability | Errors, saturation and dependency failures are visible         | [Observability](/docs/omnitron/observability) |
| Recovery      | A previous artifact and a tested data restore procedure exist  | [CLI reference](/docs/omnitron/cli)           |

Treat these as acceptance criteria for your own workload. Module presence
alone does not prove any gate.

## Set the access boundary

Bind local control endpoints to loopback or a restricted network. Terminate
TLS at a trusted proxy where appropriate. Configure authentication and
method-level authorisation before publishing RPC endpoints.

Test allowed and denied operations with separate identities. Check that
logs and client-visible errors do not expose credentials or infrastructure
details. Keep browser origin policy separate from authentication: CORS
controls browser behaviour, not server-to-server access.

See [module security](/docs/titan/modules/security-checklist) and
[operator authentication](/docs/omnitron/auth-rbac).

## Observe the right signals

Track request errors, latency, process restarts, resource saturation and the
dependencies required by each application. Give each alert an owner and a
concrete action.

The daemon metrics listener serves `/metrics` and `/healthz`. Titan's health
module exposes application health over RPC; importing it does not add HTTP
health routes to your application. Configure your ingress or platform probe
against an endpoint you have actually implemented.

A liveness check answers whether the process should be restarted. A
readiness check answers whether it should receive traffic. A dependency
outage should not automatically create an endless restart cycle.

## Rehearse a release

1. Build from a clean checkout and retain the source revision and artifact.
2. Run database migrations with the intended credentials and locking policy.
3. Deploy to a staging environment with representative configuration.
4. Execute an authenticated RPC call and the important end-user journeys.
5. Interrupt a dependency and confirm timeouts, alerts and recovery.
6. Send SIGTERM under load and verify bounded shutdown.
7. Restore the previous artifact and verify its compatibility with the schema.

When using the supervisor, follow its
[orchestrator](/docs/omnitron/orchestrator) and
[operational practices](/docs/omnitron/best-practices). When running Titan
directly, your process manager owns restart and rollout policy.

## Coordinate independently supervised processes

Record the backend, guardian and execution-owner artifacts separately, together
with the application inventory, enabled stacks, watch state and native service
definitions. A controller handoff should preserve the owner identity and
application processes; verify RPC reconnection and durable log/outbox delivery
afterward. A changed controller PID is expected, while an application restart
requires an explanation.

If the owner needs replacement, prepare a bounded maintenance window and the
previous artifacts before stopping it. Require a clean owner shutdown receipt
and confirm that its children exited before starting a replacement. Do not erase
an ownership checkpoint to bypass an uncertain shutdown. Restore enabled stacks
and watching from declared intent, and check domain readiness as well as process
presence before moving to the next host.

Keep private backup and configuration contents on their owning host where
possible; a review plan can contain paths, checksums and verification results.
Validate online SQLite backups through SQLite's backup API and integrity checks.
Reading a database dump successfully proves archive readability, not a successful
database restore; record which verification actually ran.

## Keep rollback and restore separate

Rolling back code does not undo a schema or data change. Prefer additive
migrations when old and new versions must coexist. Take backups appropriate
to your data store and verify a restore in an isolated environment.

Record the recovery point you can tolerate and the time a restore actually
takes. Store backup credentials separately from application credentials.
Avoid discovering during an incident that the backup depends on the same
failed host.

## Diagnose a failure

| Symptom                                | First checks                                                                                                  |
| -------------------------------------- | ------------------------------------------------------------------------------------------------------------- |
| Listener refuses connections           | Process state, bound address, port and proxy target                                                           |
| RPC service is missing                 | Exact service id, provider registration and exposure logs                                                     |
| Authentication succeeds but calls fail | Method role list, token audience and caller identity                                                          |
| A stopped container reappears          | [Infrastructure reconciliation](/docs/omnitron/infrastructure); change declared intent through the supervisor |
| A release starts but traffic fails     | Migrations, runtime configuration, dependency connectivity and readiness                                      |
| Shutdown exceeds its budget            | Outstanding requests, open resources and lifecycle hook deadlines                                             |

Keep incident notes with the application revision, configuration changes,
timeline and observed evidence. Continue with
[Troubleshooting](/docs/troubleshooting) for package-specific symptoms.

---

---
id: "foundations/architecture"
title: "Architecture"
---

# Architecture

How an Omnitron-based system composes at runtime.

## The layers

```mermaid
flowchart TB
    subgraph Browser["Browser / Mobile"]
      UI[Prism + React UI]
      Hooks[netron-react hooks]
      NetClient[netron-browser client]
      UI --> Hooks --> NetClient
    end

    subgraph Service["Titan Service Process"]
      Netron[Netron RPC plane]
      App[Application + DI Container]
      Modules[Titan modules: auth, cache, db, …]
      Services[User-defined @Service classes]
      Netron --> App
      App --> Modules
      App --> Services
    end

    subgraph Operate["Omnitron Supervisor"]
      CLI[omnitron CLI]
      Console[Web console]
      Orch[Orchestrator]
      CLI --> Orch
      Console --> Orch
    end

    NetClient -- "HTTP / WS" --> Netron
    Orch -- "TCP / Unix (Netron)" --> Netron
```

Three observations:

1. The browser and the supervisor talk to the service over the **same
   protocol**, just different transports. The service does not know
   which one is calling.
2. The DI container is the only thing that owns service instances. The
   transport plane (Netron) is a thin adapter from wire format to
   container-resolved methods.
3. The supervisor is itself a Titan app. It exposes a Netron service
   that the CLI and the web console call. The "operator API" is a
   first-class part of the runtime.

## A request, end to end

1. **Browser** — a React component renders, gets the typed
   service with `const users = useService<UsersService>('users')`,
   and calls `users.findById.useQuery([id])`.
2. **Hook** — `netron-react` looks up the cached result; if missing,
   asks the `netron-browser` client for the call.
3. **Client** — serialises the call as a Netron packet, sends it over
   the configured transport (HTTP for queries by default, WS for
   subscriptions).
4. **Transport** — the Titan service's `HttpTransport` decodes the
   packet and routes it to the Netron dispatcher.
5. **Dispatcher** — looks up the registered `users@1.0.0` service in
   the container, validates arguments against the method's schema,
   invokes the method.
6. **Service method** — runs in the container's scope. Dependencies
   (`Database`, `Logger`, etc.) are constructor-injected; nothing is
   pulled from globals.
7. **Response** — the return value is serialised back through Netron;
   typed errors are mapped to client-side `NetronError` instances.
8. **Hook** — caches the result, surfaces it to the React render.

End to end, the type of `data` in the React component is **the return
type of the service method**. No translation step.

## Where each package lives in this picture

| Package                                                              | Responsibility                                                              |
| -------------------------------------------------------------------- | --------------------------------------------------------------------------- |
| `@omnitron-dev/titan`                                                | DI container, module system, lifecycle, validation, Netron core, transports |
| `@omnitron-dev/netron-browser`                                       | Browser-side Netron client (HTTP + WS), middleware, error mapping           |
| `@omnitron-dev/netron-react`                                         | React hooks built on netron-browser; cache and devtools                     |
| `@omnitron-dev/prism`                                                | Design system: theme, layouts, blocks, forms, accessibility                 |
| `@omnitron-dev/omnitron`                                             | Application supervisor, CLI, web console, observability                     |
| `titan-*` modules                                                    | Optional Titan modules: auth, cache, db, discovery, events, etc.            |
| `@omnitron-dev/common`, `cuid`, `eventemitter`, `msgpack`, `testing` | Shared primitives                                                           |

## Cross-cutting concerns

- **Configuration** — `ConfigModule` validates against a Zod schema at
  boot. Misconfiguration is a startup error, not a 3 AM runtime crash.
- **Logging** — `LoggerModule` produces structured JSON with trace IDs.
  The Omnitron CLI tails them in real time across multiple services.
- **Tracing** — Netron propagates a trace context through every call.
  The supervisor's `telemetry-relay` ships traces to your collector.
- **Errors** — Typed `NetronError` subclasses carry status, code, and
  cause across the wire. Client receives the original error type, not
  a generic `Error`.

The [monorepo map](/docs/foundations/monorepo) locates each layer in the source tree.

---

---
id: "foundations/composition"
title: "Composition and build boundaries"
description: "Select small primitives first, then add transports, user interfaces and infrastructure deliberately."
---

# Composition and build boundaries

Choose packages by responsibility. A shared domain policy should run without a
server container; a browser component should not import database or JWT code.
Keep application catalogs, business invariants and data ownership in the
application. Add framework adapters at the boundary that needs them.

| Need                           | Smallest relevant surface             | Add when required                                          |
| ------------------------------ | ------------------------------------- | ---------------------------------------------------------- |
| Roles and permission decisions | `@omnitron-dev/access`                | Platform preset, session verifier, resource adapters       |
| Browser RPC                    | `@omnitron-dev/netron-browser/client` | React bindings and devtools                                |
| React interface primitives     | `@omnitron-dev/prism/ui`              | Maps, editors, knowledge graphs and their CSS entry points |
| Service process and DI         | `@omnitron-dev/titan/application`     | Authentication, database, cache and other modules          |
| Object bytes                   | `@omnitron-dev/titan-storage`         | Application metadata, quotas, ownership and sharing        |
| Process supervision            | `@omnitron-dev/omnitron`              | Deployment, telemetry and operational workflows            |

## Keep policy portable

Place a role definition and permission catalog in a shared application package.
The server uses it for enforcement; the UI uses it for affordances. A token
issuer and verifier use the same expansion. Separate rank from inheritance so
independent capabilities do not become a single ladder. The
[access policy guide](/docs/auth/access-policy) contains the runnable primitives.

Adapters translate authenticated context and domain state into that policy.
They own tenant isolation, transaction boundaries, cache invalidation and
failure behavior. A failed lookup must not accidentally produce an unrestricted
permission set. Do not make application service names or schemas framework defaults.

## Keep expensive features behind an import boundary

Import Prism UI primitives from `/ui`. Load a map, research editor or knowledge
graph from its own public entry point where that feature is used. Import its
stylesheet explicitly. The [Prism guide](/docs/frontend/prism) and
[API index](/docs/reference/api-index) list supported paths.

Use `import type` for shared contracts. Avoid a shared barrel that re-exports
both browser policy and Node adapters: evaluating that barrel can pull server
code into a browser build. A package manifest's `exports` is the supported
boundary; a source directory is not automatically a public import path.

## Verify the artifact

A typecheck proves type compatibility. Also build the actual browser and server
entry points, inspect their contributing modules and test the packed package in
consumers. Measure isolated fixtures and whole application chunks separately.
Keep a regression budget for small primitives and confirm tree shaking rather
than assuming that `sideEffects: false` guarantees every consumer's bundle.

Use workspace dependencies during framework development. For a consumer before
registry publication, a checked-in package archive with a content digest and
lockfile is portable across machines. A local absolute symlink is a development
convenience, not a reproducible release dependency.

---

---
id: "foundations/monorepo"
title: "Monorepo Layout"
---

# Monorepo Layout

The Omnitron source tree at <https://github.com/omnitron-dev/omni>.

**omni/**

- `apps/`
  - `omnitron/` — Supervisor, CLI and control plane
    - `webapp/` — Console, registered as its own workspace
- `packages/`
  - `titan/` — Application lifecycle, dependency injection and Netron core
  - `titan-auth/` — Authentication
  - `titan-cache/` — Caching
  - `titan-database/` — Kysely integration and migrations
  - `titan-discovery/` — Service discovery
  - `titan-events/` — Event bus
  - `titan-health/` — Health indicators
  - `titan-lock/` — Distributed locks
  - `titan-lumen/` — Authorized document services and durable authoring
  - `titan-mailer/` — Email delivery
  - `titan-metrics/` — Metric collection and persistence
  - `titan-notifications/` — Notification channels
  - `titan-pm/` — Process management
  - `titan-ratelimit/` — Rate limiting
  - `titan-redis/` — Redis integration
  - `titan-scheduler/` — Scheduled tasks
  - `titan-storage/` — Storage contracts and adapters
  - `titan-telemetry-relay/` — Durable telemetry delivery
  - `prism/` — Shared React components and design system
  - `lumen/` — Documentation builds, CLI and static site generation
  - `lumen-ast/` — Document compiler and HTML, Markdown and Typst output
  - `lumen-ui/` — Document navigation and interactive components
  - `lumen-migrate/` — Documentation migration tools
  - `netron-browser/` — Browser RPC client
  - `netron-http-core/` — Shared HTTP transport contracts
  - `netron-protocol/` — Shared wire protocol
  - `netron-react/` — React RPC integration
  - `access/` — Shared access-control primitives
  - `common/` — Shared utilities
  - `cuid/` — Unique identifiers
  - `eventemitter/` — Synchronous and asynchronous events
  - `msgpack/` — MessagePack serialization
  - `testing/` — Test helpers and runtime adapters
  - `kb/` — Knowledge base tools
- `.xec/` — Repository and release commands
- `pnpm-workspace.yaml` — Workspace and dependency policies
- `turbo.json` — Task graph and cache configuration
- `package.json` — Repository entry points

## Workspace boundaries

`pnpm-workspace.yaml` declares three workspace globs:

```yaml
packages:
  - apps/*
  - apps/omnitron/webapp
  - packages/*
```

The web console is registered explicitly because it lives inside the
`omnitron` app's tree but is its own workspace.

## Build orchestration

Xec orchestrates repository qualification and releases. Turbo supplies the
package task graph and build cache declared in `turbo.json`. Package scripts
remain available for focused development:

```bash
pnpm build       # turbo build + turbo build:titan
pnpm test        # turbo test
pnpm lint        # eslint across apps/ + packages/
```

## Package independence

Every package in `packages/` ships independently. A consumer pulls in
only what they need; nothing in `@omnitron-dev/titan` imports
`@omnitron-dev/prism` or vice versa.

Titan modules declare their own dependencies and peer requirements. Select
the integrations an application needs; for example, `titan-cache` does not
require `titan-database`. See each package manifest for its exact contract.

---

---
id: "foundations/principles"
title: "Principles"
description: "Shared contracts, explicit composition, visible lifecycle, and operational accountability."
---

# Principles

Omnitron brings application code and operations into one TypeScript
toolchain. Its value depends on boundaries that a developer can inspect,
test and change.

## Share contracts

Netron clients can compile against service interfaces without generating a
client SDK. Keep those types in a shared package or use type-only imports.
Version the contract when independently deployed callers need different
behaviour.

A TypeScript signature is a compile-time constraint. At runtime, validate
inputs, authenticate callers and authorise each protected operation.
Serialization does not replace those checks.

## Compose explicitly

Declare providers, imports and exports in modules. Use injection tokens
where implementations vary. Configuration and logging are registered as
core modules by default; additional capabilities are opt-in.

Module dependencies still matter. A distributed lock needs Redis; a
database module needs a driver and a live connection. Read the
[module map](/docs/titan/modules/module-map) before composing a system.

Titan supports optional filesystem discovery, but explicit module
registration makes startup dependencies easier to review.

## Make lifecycle observable

Services acquire resources during initialization and startup, then release
them during shutdown. Set timeouts and make shutdown safe for in-flight
work. Test failed startup and interrupted shutdown as well as the happy path.

The [lifecycle reference](/docs/titan/application/lifecycle) describes
ordering. Avoid relying on the exact interleaving of unrelated providers.

## Separate runtime from supervision

A Titan application is a normal process. Use a process supervisor appropriate
to your deployment. Add Omnitron when you need its project model,
infrastructure management, release workflow or remote operations.

The CLI and web console use the daemon's RPC services. Operational access
has its own identity and permissions; sharing a transport does not grant
the same authority to every caller.

## Report what happened

Return typed errors, log enough context to diagnose failures, and keep
sensitive details out of public responses. Readiness, successful process
startup and a successful deployment are different claims. Verify each at
the layer that owns it.

For deployment practice, continue with
[Production readiness](/docs/deployment/production-readiness).

---

---
id: "foundations/titan-vs-omnitron"
title: "Titan vs Omnitron"
description: "Titan runs standalone. Omnitron is an opt-in higher-level platform built on top."
---

# Titan vs Omnitron

These are **two distinct layers**. Most projects use **Titan
alone** — it's a complete backend framework. **Omnitron is
opt-in** — a higher-level platform that supervises, deploys,
and operates many Titan apps together.

This page draws the line precisely so you know what to install
and what to skip.

## At a glance

```mermaid
flowchart TB
  subgraph Titan["Titan — backend framework (always)"]
    direction LR
    App["Application.create(module)"]
    DI[Nexus DI container]
    Net[Netron RPC<br/>HTTP / WS / TCP / Unix]
    Lif[Lifecycle hooks]
    Mods[14 modules: auth, cache, db, ...]
  end

  subgraph Omnitron["Omnitron — supervisor + control plane (opt-in)"]
    direction LR
    Daemon[Long-running daemon]
    CLI[75+ CLI commands]
    Console[Web console]
    Infra[Declarative infra provisioning]
    Cluster[Cluster + fleet]
    MCP[MCP server for AI agents]
  end

  Titan -. you can use this alone .-> Run1[node dist/index.js]
  Titan --> Omnitron
  Omnitron --> Run2[omnitron up — supervises N Titan apps]

  classDef titan fill:#e8f5e9,stroke:#43a047,color:#1b5e20
  classDef omni fill:#f3e5f5,stroke:#8e24aa,color:#4a148c
  class Titan titan
  class Omnitron omni
```

**Titan = the runtime.** Build a backend, expose RPC, run it.

**Omnitron = the operator.** Supervise many backends, provide
the operator UI / CLI / agent surface.

## Titan alone (no Omnitron)

`@omnitron-dev/titan` is **a self-contained backend framework**.
You can build a complete production service with just Titan +
the modules you need. No daemon, no extra runtime, no operator
overhead — just a Node process.

```typescript
// my-app/src/main.ts — that's the whole entrypoint
import { Application, Module, Service } from '@omnitron-dev/titan';
import { Public } from '@omnitron-dev/titan/decorators';
import { TitanDatabaseModule } from '@omnitron-dev/titan-database';
import { TitanAuthModule }     from '@omnitron-dev/titan-auth';

@Service('users@1.0.0')
class UsersService {
  @Public()
  async findById(id: string) { /* ... */ }
}

@Module({
  imports: [
    TitanDatabaseModule.forRoot({
      connection: { dialect: 'postgres', connection: env.DATABASE_URL },
    }),
    TitanAuthModule.forRoot({ jwtSecret: env.JWT_SECRET }),
  ],
  providers: [UsersService],
})
class AppModule {}

const app = await Application.create(AppModule, {
  netron: { http: { port: 3000 } },
});

await app.start();
process.on('SIGTERM', () => app.stop());
```

Run with:

```bash
node dist/main.js
```

You get **everything**:

| Feature                                                             | Where it comes from                                                                              |
| ------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------ |
| Decorator DI                                                        | core Titan                                                                                       |
| **Server-side Netron RPC** with all 4 transports (HTTP/WS/TCP/Unix) | core Titan (subpath exports: `titan/netron`, `titan/netron/transport/{http,websocket,tcp,unix}`) |
| Server-side auth middleware (JWT, RLS context, rate-limit)          | core Titan (`titan/netron/auth`, `titan/netron/transport/http/middleware`)                       |
| Multi-backend server-side routing                                   | core Titan (`titan/netron/multi-backend`)                                                        |
| Module system                                                       | core Titan                                                                                       |
| Lifecycle hooks (`onInit` / `onStart` / `onStop` / `onDestroy`)     | core Titan                                                                                       |
| Graceful shutdown                                                   | core Titan                                                                                       |
| Validation                                                          | core Titan + zod                                                                                 |
| Typed errors over the wire                                          | core Titan                                                                                       |
| Structured logging                                                  | `LoggerModule` (built-in)                                                                        |
| Schema-validated config                                             | `ConfigModule` (built-in)                                                                        |
| JWT auth + JWKS                                                     | `titan-auth`                                                                                     |
| Database + RLS + migrations                                         | `titan-database`                                                                                 |
| Multi-tier cache                                                    | `titan-cache`                                                                                    |
| Cron / interval / timeouts                                          | `titan-scheduler`                                                                                |
| Health probes for k8s                                               | `titan-health`                                                                                   |
| Distributed locks                                                   | `titan-lock`                                                                                     |
| Rate limiting                                                       | `titan-ratelimit`                                                                                |
| Worker pools / process management                                   | `titan-pm`                                                                                       |
| Prometheus metrics                                                  | `titan-metrics`                                                                                  |
| Notifications (email / push / SMS / webhooks)                       | `titan-notifications`                                                                            |
| Service discovery                                                   | `titan-discovery`                                                                                |
| In-process event bus                                                | `titan-events`                                                                                   |
| Cross-tab browser auth                                              | `netron-browser` + `AuthenticationClient`                                                        |
| React hooks for RPC                                                 | `netron-react`                                                                                   |
| UI design system                                                    | `prism`                                                                                          |

**This is a complete stack.** Run it under any process
supervisor: PM2, systemd, Docker, Kubernetes, Fly, Railway,
Render — Titan is just a Node process.

## What Omnitron adds

`@omnitron-dev/omnitron` is **an optional higher-level
platform** that **uses Titan internally** (it's itself a Titan
app) and **supervises other Titan apps**.

Adds, **on top of** what Titan already gives you:

| Capability                          | What it does                                                                                                                                                                                                                                     |
| ----------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| **Long-running daemon**             | One supervisor process owns N apps, survives crashes, persists state                                                                                                                                                                             |
| **Multi-app lifecycle**             | `omnitron up` starts a fleet of Titan apps with declarative ordering (`dependsOn`)                                                                                                                                                               |
| **75+ CLI subcommands**             | `omnitron start/stop/restart/reload/logs/metrics/inspect/exec/scale/...` for any app                                                                                                                                                             |
| **Web console (Omnitron Console)**  | React + Vite + Prism UI: dashboards, logs, metrics, traces, fleet view, MCP — at `http://localhost:9800`                                                                                                                                         |
| **20+ management RPC services**     | `OmnitronDaemon`, `OmnitronAuth`, `OmnitronDeploy`, `OmnitronFleet`, `OmnitronSecrets`, `OmnitronBackups`, `OmnitronPipelines`, `OmnitronKubernetes`, `OmnitronLogs`, `OmnitronTraces`, `OmnitronHealth`, `OmnitronAlerts`, `OmnitronNodes`, ... |
| **Declarative infrastructure**      | `omnitronConfig` per app declares Postgres/Redis/S3/custom-daemons; reconciler provisions via Docker (dev) or bare-metal hooks (prod)                                                                                                            |
| **Project + Stack model**           | Many projects × many environments (dev/staging/prod) supervised side-by-side                                                                                                                                                                     |
| **Cluster mode**                    | Multi-node with simplified Raft leader election + state replication                                                                                                                                                                              |
| **Fleet**                           | Inventory of remote daemons + cross-node operations                                                                                                                                                                                              |
| **Hot reload in dev**               | File watcher rebuilds + restarts apps on save                                                                                                                                                                                                    |
| **Per-app log aggregation**         | Rotation, retention, structured query — `omnitron logs api -f -g error -l warn`                                                                                                                                                                  |
| **Persistent state**                | `~/.omnitron/state.json` survives daemon restart; rehydrates running apps                                                                                                                                                                        |
| **Encrypted secret store**          | `omnitron secret set/get/list/delete` with at-rest encryption                                                                                                                                                                                    |
| **Backup service**                  | `omnitron backup create/restore` for managed databases                                                                                                                                                                                           |
| **Deployment workflows**            | \`omnitron deploy app --strategy rolling                                                                                                                                                                                                         | canary | blue-green\` |
| **CI/CD pipelines**                 | First-class pipeline definition + execution                                                                                                                                                                                                      |
| **Kubernetes integration**          | `omnitron k8s pods/deploy scale` from the same CLI                                                                                                                                                                                               |
| **Node inventory + health monitor** | SSH-based health checks, 90-day uptime bars                                                                                                                                                                                                      |
| **MCP server for AI agents**        | `omnitron kb mcp` exposes \~40 typed tools to agents                                                                                                                                                                                             |
| **Knowledge base**                  | Hybrid full-text + semantic search of the codebase                                                                                                                                                                                               |
| **API gateway**                     | Optional OpenResty gateway with Lua maintenance mode                                                                                                                                                                                             |
| **Tor hidden services**             | Optional `.onion` exposure for operator surfaces                                                                                                                                                                                                 |

It's not "Titan with extras" — it's a separate platform that
happens to *use* Titan as its runtime substrate.

## When you need Omnitron

Reach for Omnitron when **any** of these apply:

- **≥ 2 Titan apps** to start, stop, and inspect together.
- **Multi-environment** — dev / staging / prod stacks on the
  same host without container orchestration.
- **Multi-node fleet** — coordinate apps across machines from
  one operator surface.
- **Web console** — you want a UI for ops, not just a CLI.
- **Declarative infra** — `omnitronConfig: { database: true,
  redis: true, s3: true }` and forget about provisioning.
- **AI agent integration** — `omnitron kb mcp` for agents to
  drive the platform.
- **Operator-level RBAC** — three roles (viewer / operator /
  admin) on top of your app's user auth.

## When you don't

Omnitron is **overkill** for:

- **Single Titan app** running on one box — `node dist/main.js`
  is fine.
- **Already containerised** — if Docker Compose / Kubernetes /
  Fly is your supervisor, Omnitron's supervision overlaps. Run
  the Titan apps directly in containers.
- **Edge / serverless** — Cloudflare Workers, Deno Deploy,
  AWS Lambda. Omnitron daemon assumes Node + Unix sockets;
  doesn't fit these runtimes.
- **Single-purpose microservice** — one team, one service, one
  deploy. The framework overhead doesn't pay back.
- **Smaller projects (≤ 2 apps, single team)** — Titan + your
  preferred process manager is simpler.

## Migration paths

### Start with Titan alone

```bash
pnpm add @omnitron-dev/titan
# ... add modules as needed
node dist/main.js
```

### Add Omnitron later

The same Titan app **runs unchanged** under Omnitron. Just
add the ecosystem config:

```typescript
// omnitron.config.ts (new file at repo root)
import { defineEcosystem } from '@omnitron-dev/omnitron';

export default defineEcosystem({
  apps: [
    { name: 'api', bootstrap: './apps/api/dist/bootstrap.js' },
  ],
});
```

Convert `main.ts` to `bootstrap.ts`:

```typescript
// apps/api/src/bootstrap.ts — was main.ts
import { defineSystem } from '@omnitron-dev/omnitron';

export default defineSystem({
  name:    'api',
  version: '1.0.0',
  processes: [
    {
      name:   'http',
      module: './app.module.js',
      transports: { http: { port: 3000 } },
    },
  ],
});
```

Run:

```bash
omnitron up                      # starts daemon + api
```

The Titan app's source — `@Service`, `@Module`, `@Public`,
business logic, modules — **does not change**. The same Netron
endpoints work; clients keep working without modification.

### Drop Omnitron later

Equally easy in reverse: replace `omnitron up` with `node
dist/main.js` (using the original `main.ts`-style boot) or with
your container orchestrator. Titan keeps running.

## Cost comparison

| Concern                  | Titan alone                        | Titan + Omnitron                            |
| ------------------------ | ---------------------------------- | ------------------------------------------- |
| Bundle size at runtime   | \~5–15 MB resident per app         | + daemon \~50 MB                            |
| Boot time                | Seconds (depends on modules)       | + daemon \~1–2 s                            |
| Operator learning curve  | Read Titan docs                    | Read Titan + Omnitron CLI                   |
| Operations surface       | Process manager + DB/Redis hosting | Daemon + ops UI                             |
| Dev hot-reload           | `tsx watch` / `nodemon`            | Built into `omnitron up`                    |
| Log aggregation          | stdout → log shipper               | Built-in ring buffer + rotation             |
| Multi-app supervision    | Your container orchestrator        | Built-in                                    |
| Configuration management | `omnitron.config.ts` not needed    | `omnitron.config.ts` declares the ecosystem |

## What the docs cover for each

| Section                                      | Layer                                                                         |
| -------------------------------------------- | ----------------------------------------------------------------------------- |
| [Titan overview](/docs/titan/overview)       | Titan core                                                                    |
| [Titan modules](/docs/titan/modules)         | The 14 official modules — all usable without Omnitron                         |
| [Frontend / Netron](/docs/frontend/netron)   | Browser-side; works whether the backend is Titan-alone or Omnitron-supervised |
| [Omnitron overview](/docs/omnitron/overview) | Omnitron-specific                                                             |
| [Omnitron CLI](/docs/omnitron/cli)           | The 75+ commands — only relevant if you use Omnitron                          |
| [Deployment guides](/docs/deployment)        | Both layers — install paths for Titan-alone and Omnitron-supervised           |
| [Tutorial](/docs/tutorial)                   | Builds with Titan; the last step adds Docker (no Omnitron)                    |

## Real numbers

A small project (one Titan app, single Postgres, two
HTTP routes):

| Setup                            | RAM                       | Boot time | Lines of config                                    |
| -------------------------------- | ------------------------- | --------- | -------------------------------------------------- |
| Titan alone, `node dist/main.js` | \~80 MB                   | 1.5 s     | \~15 lines (`main.ts`)                             |
| Titan + Omnitron daemon          | \~130 MB (80 + 50 daemon) | 2.5 s     | + 20 lines (`omnitron.config.ts` + `bootstrap.ts`) |

For 1 app: the ratio is bad. For 5 apps: same daemon, \~150 MB
total vs 5 × 80 = 400 MB if each apps' supervision was
duplicated.

## TL;DR

**Titan is the framework. You probably need it.**

**Omnitron is the platform. You probably don't — until you do.**

| You have…                          | Use                                               |
| ---------------------------------- | ------------------------------------------------- |
| One Titan app                      | Titan alone + your process manager                |
| 2–5 Titan apps in one repo         | Titan + (PM2 or Docker Compose or Omnitron)       |
| Multi-environment / multi-node     | Omnitron                                          |
| Need ops UI + CLI + MCP for agents | Omnitron                                          |
| Edge / serverless                  | Titan alone (Omnitron doesn't fit)                |
| Largest possible scale             | Titan + Kubernetes (Omnitron optional inside k8s) |

## See also

- [Architecture](/docs/foundations/architecture) — how Titan composes
- [Principles](/docs/foundations/principles) — the rules behind both layers
- [Comparison](/docs/comparison) — how the whole stack compares to alternatives
- [Omnitron overview](/docs/omnitron/overview) — if you've decided you want Omnitron
- [Deployment](/docs/deployment) — how to ship either way

---

---
id: "frontend/netron/auth"
title: "Auth client"
description: "AuthenticationClient — token storage, auto-refresh, cross-tab sync, inactivity timeout."
---

# Auth client

> **Info**
>
> For the framework-wide authorisation model (permission strings,
> ABAC, RLS bridge) start at [Authentication & Authorisation](/docs/auth).
> This page is the browser-side **token-lifecycle** reference.

`AuthenticationClient` (from `@omnitron-dev/netron-browser`)
owns browser-side authentication state: where the token lives,
when to refresh, how to propagate sign-in/out across tabs, and
when to time out an idle session.

You attach it to a transport client (HTTP or WebSocket) so every
RPC call carries the token, and a refresh fires automatically on
401\. In React apps it's usually wrapped by netron-react's
[`AuthProvider`](/docs/frontend/netron/react#authentication) — but the client
works standalone.

Verified against `packages/netron-browser/src/auth/`.

## Wiring

Pass an `AuthenticationClient` to the transport client. The
client attaches the token to every request and the built-in
`auth-error-handler` middleware refreshes + retries on 401:

```typescript
import { HttpClient } from '@omnitron-dev/netron-browser';
import { AuthenticationClient, LocalTokenStorage } from '@omnitron-dev/netron-browser';
import { createAuthErrorMiddleware } from '@omnitron-dev/netron-browser/middleware';

const auth = new AuthenticationClient({
  storage:          new LocalTokenStorage('platform:token'),
  autoRefresh:      true,
  refreshThreshold: 5 * 60_000,                 // refresh 5 min before expiry
  refreshConfig:    { endpoint: '/auth/refresh' },
  inactivityConfig: { timeout: 30 * 60_000 },   // 30 min
  crossTabSync:     { enabled: true },
});

const client = new HttpClient({ url: 'https://api.example.com', auth });

// Refresh-then-retry on 401 (and surface 403 / 429):
client.use(createAuthErrorMiddleware({
  authClient:        auth,
  onSessionExpired:  () => location.assign('/sign-in?reason=expired'),
}));

// On sign-in success (an AuthResult from your authenticate call):
auth.setAuth(result);

// On sign-out:
await auth.logout();   // POSTs logoutConfig.endpoint if set, then clearAuth()
```

> `AuthenticationClient` is the real class — there is no
> `AuthManager`, no `setTokens`, no `clear()`. Use `setAuth()` /
> `clearAuth()` / `logout()` and `getToken()`.

## Constructor options

`new AuthenticationClient(options: AuthOptions)` — all fields optional:

| Option             | Default                                                            | Notes                                                                                                      |
| ------------------ | ------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------- |
| `storage`          | `new MemoryTokenStorage()`                                         | A `TokenStorage` instance (see below). **Secure-by-default** — memory, not localStorage.                   |
| `storageKey`       | `'netron_auth_token'`                                              | Convenience: if `storage` is omitted but `storageKey` is set, a `LocalTokenStorage(storageKey)` is created |
| `autoRefresh`      | `true`                                                             | Schedule a refresh before expiry                                                                           |
| `refreshThreshold` | `5 * 60_000`                                                       | Refresh this many ms before `expiresAt`                                                                    |
| `autoAttach`       | `true`                                                             | Attach the token to outgoing requests                                                                      |
| `refreshConfig`    | —                                                                  | `{ endpoint, method?, headers?, buildBody? }`                                                              |
| `logoutConfig`     | —                                                                  | `{ endpoint, method?, headers?, includeToken? }`                                                           |
| `inactivityConfig` | `{ timeout: 30*60_000, events: ['click','keypress','mousemove'] }` | Idle auto-sign-out                                                                                         |
| `crossTabSync`     | `{ enabled: true, syncKey: 'netron_auth_sync' }`                   | Sync sign-in/out across tabs                                                                               |
| `transport`        | —                                                                  | A token transport strategy (Bearer / Cookie / Hybrid — see below)                                          |

## Storage backends

`storage` takes a `TokenStorage` **instance** (not a string).
Four implementations ship from `@omnitron-dev/netron-browser`:

| Class                       | Survives                | Use case                                                                          |
| --------------------------- | ----------------------- | --------------------------------------------------------------------------------- |
| `MemoryTokenStorage`        | nothing (reload clears) | **Default.** Highest security; user re-auths on reload                            |
| `LocalTokenStorage(key?)`   | tab close + reload      | Long-lived sessions; most apps                                                    |
| `SessionTokenStorage(key?)` | tab close (per-tab)     | "Remember me off"                                                                 |
| `NoopTokenStorage`          | —                       | Cookie-mode — the browser holds an HttpOnly cookie, nothing is stored client-side |

```typescript
import { LocalTokenStorage, MemoryTokenStorage, NoopTokenStorage }
  from '@omnitron-dev/netron-browser';

new AuthenticationClient({ storage: new LocalTokenStorage('myapp:token') });
```

The `TokenStorage` interface is `getToken() / setToken() /
removeToken() / hasToken()` plus generic `getValue() / setValue()
/ removeValue()` (the client persists a serialized context
alongside the token).

### Token transports (Bearer / Cookie / Hybrid)

How the token reaches the server is a pluggable
`IClientTokenTransport` (T#176). Pass one as `transport`:

| Transport                    | Sends                                                                            | Use                                             |
| ---------------------------- | -------------------------------------------------------------------------------- | ----------------------------------------------- |
| `BearerClientTokenTransport` | `Authorization: Bearer <token>` header (+ `?token=` on WS)                       | Default bearer-token model                      |
| `CookieClientTokenTransport` | nothing — sets `credentials: 'include'` so the browser sends the HttpOnly cookie | Cookie-mode auth (pair with `NoopTokenStorage`) |
| `HybridClientTokenTransport` | both — cookie credentials + bearer header                                        | Migration / dual-mode                           |

```typescript
import { AuthenticationClient, NoopTokenStorage } from '@omnitron-dev/netron-browser';
import { CookieClientTokenTransport } from '@omnitron-dev/netron-browser';

// HttpOnly-cookie auth: no client-side token, browser sends the cookie.
const auth = new AuthenticationClient({
  storage:   new NoopTokenStorage(),
  transport: new CookieClientTokenTransport(),
});
```

## Token + result shapes

After authenticating, you hand the client an `AuthResult` (the
shape your server's authenticate task returns):

```typescript
interface AuthResult {
  success:   boolean;
  context?:  AuthContext;            // user identity + roles/permissions
  error?:    string;
  metadata?: Record<string, any>;    // tokens live here: { accessToken, refreshToken, refreshTokenExpiresAt }
}

interface AuthContext {
  userId:       string;
  roles:        string[];
  permissions:  string[];
  scopes?:      string[];
  token?:       { type: 'bearer' | 'mac' | 'custom'; expiresAt?: Date; issuer?: string; audience?: string[] };
  metadata?:    Record<string, any>;
}
```

`setAuth(result)` reads the access token from
`metadata.accessToken` and the refresh token from
`metadata.refreshToken`, and uses `context.token.expiresAt` for
proactive refresh. For a bare token (no full result) use
`setToken(token, context?)`.

## Auto-refresh flow

```mermaid
sequenceDiagram
  participant App
  participant MW as auth-error-handler
  participant Auth as AuthenticationClient
  participant Server

  App->>Auth: getAuthHeaders()  (attached to every call)
  Note over Auth: a timer fires refreshToken() ~refreshThreshold before expiry
  Auth->>Server: POST refreshConfig.endpoint(refreshToken)
  Server-->>Auth: AuthResult { metadata.accessToken' }
  Auth->>Auth: setAuth(...)  → emits 'token-refreshed'
  App->>Server: invoke with Authorization
  alt 401 from server
    Server--xMW: 401 Unauthorized
    MW->>Auth: refreshToken()
    Auth->>Server: POST refreshConfig.endpoint
    Server-->>Auth: AuthResult { accessToken' }
    MW->>Server: retry invoke
    Server-->>App: 200
  else 200
    Server-->>App: result
  end
```

Proactive (`autoRefresh` + `refreshThreshold`) covers clock skew
and slow networks; the reactive `auth-error-handler` middleware
covers a refresh that fired mid-request.

### Concurrent-request deduplication

`refreshToken()` coalesces concurrent calls via a single shared
`refreshPromise` — multiple requests that 401 simultaneously
share one refresh call rather than all hitting the endpoint.

## Cross-tab sync

With `crossTabSync: { enabled: true }`, sign-in / sign-out in one
tab propagates to others via the storage `storage` event (the
client writes a record to `crossTabSync.syncKey`):

```mermaid
sequenceDiagram
  participant Tab1
  participant LS as localStorage (storage event)
  participant Tab2

  Tab1->>Tab1: setAuth(...)
  Tab1->>LS: write syncKey record
  LS-->>Tab2: 'storage' event
  Tab2->>Tab2: re-read state → emit 'cross-tab-sync'

  Note over Tab1: user signs out
  Tab1->>Tab1: clearAuth()
  Tab1->>LS: write syncKey record
  LS-->>Tab2: 'storage' event
  Tab2->>Tab2: clear local state
```

(`enableCrossTabSync()` / `disableCrossTabSync()` toggle it at
runtime.) Cross-tab auth sync uses `storage` events; the separate
multi-tab WebSocket leader-election feature is unrelated.

## Inactivity timeout

```typescript
new AuthenticationClient({
  inactivityConfig: {
    timeout: 30 * 60_000,
    events:  ['click', 'keypress', 'mousemove'],   // activity resets the timer
    onInactivity: () => { /* optional callback */ },
  },
});
```

When the timeout expires the client emits `'inactivity'`, calls
`onInactivity` (if given), then `clearAuth()`. Subscribe to react
in your app:

```typescript
auth.on('inactivity', () => navigate('/sign-in?reason=timeout'));
```

## Event subscriptions

Subscribe with `on(event, handler)` / unsubscribe with
`off(event, handler)`. The six event types:

```typescript
auth.on('authenticated',   ({ context })      => { /* signed in */ });
auth.on('unauthenticated', ()                 => { /* signed out / cleared */ });
auth.on('token-refreshed', ({ context })      => { /* token rotated */ });
auth.on('error',           ({ error, context })=> { /* refresh/logout failed */ });
auth.on('inactivity',      ({ lastActivity }) => { /* idle timeout */ });
auth.on('cross-tab-sync',  ({ type })         => { /* another tab changed auth */ });
```

## React integration

netron-react's `AuthProvider` wraps the auth lifecycle and
exposes `useAuth()`. Sign-in runs through the provider's
`onLogin` handler; components call `login` / `logout` (see the
[netron-react auth section](/docs/frontend/netron/react#authentication) for the
full provider API):

```tsx
import { AuthProvider, useAuth } from '@omnitron-dev/netron-react/auth';

<AuthProvider
  config={{ refreshEndpoint: '/auth/refresh', storage: 'local', autoRefresh: true }}
  onLogin={(credentials) => client.invoke('auth', 'signIn', [credentials])}
>
  <Outlet />
</AuthProvider>

function UserMenu() {
  const { user, isAuthenticated, login, logout } = useAuth();

  if (!isAuthenticated) {
    return <Button onClick={() => login(credentials)}>Sign in</Button>;
  }
  return (
    <Menu>
      <MenuItem disabled>{user?.userId}</MenuItem>
      <MenuDivider />
      <MenuItem onClick={() => logout()}>Sign out</MenuItem>
    </Menu>
  );
}
```

`useAuth()` returns `{ isAuthenticated, user, login, logout,
refresh, getAuthHeaders, hasRole, hasPermission, hasAnyRole,
hasAllRoles }`.

### Route guards

```tsx
import { AuthGuard, GuestGuard } from '@omnitron-dev/netron-react/auth';

<Routes>
  <Route element={<GuestGuard redirectTo="/"><AuthLayout /></GuestGuard>}>
    <Route path="/sign-in" element={<SignInPage />} />
  </Route>
  <Route element={<AuthGuard redirectTo="/sign-in"><DashboardLayout /></AuthGuard>}>
    <Route path="/" element={<Dashboard />} />
  </Route>
</Routes>
```

`<AuthGuard>` renders children when authenticated, else its
`fallback` (and can `redirectTo`); `<GuestGuard>` is the inverse.
`<RoleGuard role="…">` / `<PermissionGuard permission="…">` gate
on RBAC.

### Role-gated content

```tsx
import { useAuth } from '@omnitron-dev/netron-react/auth';

function AdminPanel() {
  const { hasRole } = useAuth();
  if (!hasRole('admin')) return null;
  return <DestructiveOperations />;
}
```

`hasRole(role)` checks the context's `roles`; `hasAnyRole(roles)`
/ `hasAllRoles(roles)` cover the array cases.

## Sign-in flow with 2FA

```tsx
async function handleSignIn(values: { email: string; password: string; totpCode?: string }) {
  try {
    const result = await client.invoke('auth', 'signIn', [values]);   // returns AuthResult

    if (result.metadata?.requires2fa) {
      setPendingMfa(true);                  // show 2FA input, call signIn again with totpCode
      return;
    }

    auth.setAuth(result);                   // stores token + context, emits 'authenticated'
    navigate('/');
  } catch (e) {
    form.setError('root', { message: (e as Error).message });
  }
}
```

The two-step flow keeps the 2FA input out of the password form
until needed. (Through React, prefer `useAuth().login(values)` —
it runs the provider's `onLogin` and calls `setAuth` for you.)

## Sign-in flow with WebAuthn / passkey

```typescript
const challenge  = await client.invoke('auth', 'getWebAuthnChallenge', [{ email }]);
const credential = await navigator.credentials.get({ publicKey: challenge });
const result     = await client.invoke('auth', 'verifyWebAuthn', [{ credential }]);
auth.setAuth(result);
```

The client doesn't care about the source — `setAuth` stores the
result the same way regardless of method.

## Programmatic token access (advanced)

```typescript
const token   = auth.getToken();              // string | undefined (synchronous)
const isAuth  = auth.isAuthenticated();
const context = auth.getContext();            // AuthContext | undefined (userId, roles, …)
const session = auth.getSessionMetadata();    // { sessionId, loginTime, … } | undefined
const headers = auth.getAuthHeaders();        // e.g. { Authorization: 'Bearer …' }
```

Useful for direct `fetch` calls outside the RPC client (file
uploads, third-party SDKs) — merge `getAuthHeaders()` into your
request.

## Custom refresh

`refreshConfig` shapes the refresh request — use `buildBody` /
`headers` / `method` when refresh isn't a plain
`POST { refreshToken }`:

```typescript
new AuthenticationClient({
  refreshConfig: {
    endpoint:  '/auth/refresh',
    method:    'POST',
    headers:   { 'X-CSRF-Token': readCsrfCookie() },
    buildBody: (refreshToken) => JSON.stringify({ refreshToken }),
  },
});
```

## Security considerations

- **Default storage is memory** — secure-by-default. Opt into
  `LocalTokenStorage` only when you need persistence across
  reloads, and understand that localStorage tokens are reachable
  by any script on the origin (XSS = token compromise).
- **HttpOnly cookies** are immune to XSS but need CSRF
  protection — use `CookieClientTokenTransport` + `NoopTokenStorage`
  and the `csrf` middleware (`createCsrfMiddleware`). Pick one
  model and stick to it.
- **Don't log tokens.** Even at `debug` level.
- **Inactivity timeout** matters for shared / public computers —
  default 30 min; keep it short for admin surfaces.
- **Token rotation hooks** (`auth.on('token-refreshed', …)`) can
  surface session rotation in a security dashboard.

## Best practices

- **One `AuthenticationClient` per app**, wired before any RPC
  calls fire.
- **Attach it to the transport** (`new HttpClient({ url, auth })`)
  so tokens flow automatically; add `createAuthErrorMiddleware`
  for refresh-on-401.
- **Use a real `expiresAt`** (via the result's
  `context.token.expiresAt`) so `autoRefresh` fires proactively
  instead of one-failed-request-per-cycle.
- **`crossTabSync: { enabled: true }`** unless you have a
  specific reason not to.

## Anti-patterns

- **Storing tokens in both cookies and localStorage.** Pick one
  transport model; mixed approaches cause refresh / clear bugs.
- **Reaching for `AuthManager` / `setTokens`.** Those don't
  exist — it's `AuthenticationClient` + `setAuth` / `clearAuth`.

## See also

- [netron-react auth](/docs/frontend/netron/react#authentication) — the React provider + guards
- [Middleware](/docs/frontend/netron/middleware) — `createAuthMiddleware`, `createAuthErrorMiddleware`, `createCsrfMiddleware`
- [Cookie-mode auth](/docs/auth) — the closed-platform cookie model

---

---
id: "frontend/netron/browser"
title: "netron-browser"
description: "Browser-optimized Netron RPC client — HTTP / WebSocket / multi-backend."
---

# netron-browser

`@omnitron-dev/netron-browser` is the **framework-agnostic
browser RPC client** for Titan services. Dual transport
(HTTP + WebSocket), type-safe service proxies, a middleware
pipeline (auth / logging / timing / error-transform / CSRF), an
optional fluent HTTP interface for caching / retry / circuit
breaking, an `AuthenticationClient` with cross-tab sync, and a
multi-backend pool when you talk to more than one Netron server.

> **Works with any frontend.** Vanilla JS, Vue, Svelte, Solid,
> Angular, Lit, React, Web Workers, Electron renderers —
> anywhere you can `import` an ES module. Not tied to React.
> For React-specific hooks / providers / cache integration, see
> [netron-react](/docs/frontend/netron/react) — an **optional** layer on top of
> this package.
>
> The **server side** lives inside Titan at
> `@omnitron-dev/titan/netron` (with all four transports). Pair
> this client with that server.

Verified against `packages/netron-browser/src/`.

```bash
pnpm add @omnitron-dev/netron-browser
```

## Architecture

```mermaid
flowchart TB
  subgraph App["Your app"]
    Code["Code calls client.service(...)"]
  end

  subgraph Client["NetronClient"]
    Proxy[Service proxy]
    MW[Middleware pipeline<br/>auth / logging / timing / CSRF]
    Auth[AuthenticationClient]
    Transport{Transport selector}
  end

  subgraph Transports
    HTTP[HttpClient<br/>fetch + retry option]
    WS[WebSocketClient<br/>persistent + reconnect]
  end

  Code --> Proxy
  Proxy --> MW
  MW --> Auth
  Auth --> Transport
  Transport -- 'http' --> HTTP
  Transport -- 'websocket' --> WS

  HTTP --> Server[(Netron server<br/>HTTP transport)]
  WS --> Server2[(Netron server<br/>WS transport)]
```

> Caching, richer retry, and circuit breaking are **not** in the
> invocation pipeline — they live in the optional [fluent HTTP
> interface](#fluent-http-interface--caching-retry-circuit-breaking)
> reached via `peer.queryFluentInterface(...)`.

## Quick start

```typescript
import { createClient, WebSocketClient } from '@omnitron-dev/netron-browser';

const client = createClient({
  url:       'http://localhost:3000',
  transport: 'http',
});

await client.connect();

// 1. Direct invocation
const result = await client.invoke('calculator', 'add', [2, 3]);

// 2. Type-safe proxy
interface Calculator {
  add(a: number, b: number): Promise<number>;
}

const calc = client.service<Calculator>('calculator');
const sum  = await calc.add(2, 3);

// 3. Same call over WebSocket
const ws = new WebSocketClient({ url: 'wss://api.example.com/rpc', enableServiceDiscovery: true });
await ws.connect();
try {
  const liveSum = await ws.invoke('calculator@1.0.0', 'add', [2, 3]);
  console.log(liveSum);
} finally {
  await ws.disconnect();
}
```

## Client options

```typescript
interface NetronClientOptions {
  url:        string;
  transport?: 'http' | 'websocket';      // default 'http'
  timeout?:   number;                     // ms
  headers?:   Record<string, string>;
  http?: {
    redirect?:   RequestRedirect;
    retry?:      boolean;
    maxRetries?: number;
  };
  websocket?: {
    protocols?:            string | string[];
    reconnect?:            boolean;
    reconnectInterval?:    number;
    maxReconnectAttempts?: number;
  };
}
```

## Transports

`createClient` selects HTTP by default. Choose `'websocket'` for a binary
Netron listener. Supply an absolute base URL in either case.

```typescript
const client = createClient({
  url: new URL('/api', window.location.origin).href,
  transport: 'http',
  http: { retry: true, maxRetries: 3 },
});

await client.connect();
const items = await client.invoke('Catalog', 'list', [], { idempotent: true });
```

The direct HTTP client retries a network failure only when the client enables
retries and that call declares `idempotent: true`. Cancellation and deadline
expiry end the invocation. A WebSocket reconnect opens a new connection;
calls still pending on the closed socket reject.

```typescript
const state = client.getState();
const { requestsSent, responsesReceived, errors, avgLatency } = client.getMetrics();
```

See [Transports](/docs/frontend/netron/transports) for the wire format, handshake, reconnect
events and retry policies. For per-read caching and deduplication see the
[fluent HTTP interface](#fluent-http-interface--caching-retry-circuit-breaking).

## Service proxies — typed RPC

```typescript
interface UserService {
  findById(id: string):                    Promise<User>;
  list(filter: UserFilter):                Promise<User[]>;
  create(input: CreateUser):               Promise<User>;
  subscribe(filter: UserFilter):           AsyncIterable<UserEvent>;
}

const users = client.service<UserService>('users');

await users.findById('u_42');           // Promise<User>
await users.create({ email: '...' });   // Promise<User>

for await (const evt of users.subscribe({ tier: 'pro' })) {
  console.log(evt);                       // streamed over WS
}
```

The proxy is a `Proxy` that resolves any property access into an
RPC call against the named service. No code generation required —
your shared `.d.ts` is enough.

### Service descriptor

```typescript
const desc = await client.getServiceDescriptor('users');
// desc.methods, desc.subscriptions, desc.types, ...
```

Useful for building generic UIs that inspect a service at
runtime.

## Middleware pipeline

Middleware runs around every invocation. Three stages:

```mermaid
flowchart LR
  In[Invoke] --> Pre[pre-request stage]
  Pre --> Send[send to transport]
  Send --> Post[post-response stage]
  Post --> Out[Return value]
  Send -. throws .-> Err[error stage]
  Err --> Out
```

Each middleware has a `priority` — lower runs first within its
stage.

### Built-in middleware

All built-in middleware are **factory functions** (`create*`):

| Middleware                       | Stage              | Purpose                                                  |
| -------------------------------- | ------------------ | -------------------------------------------------------- |
| `createAuthMiddleware`           | pre                | Attach `Authorization: Bearer ...` from a token provider |
| `createLoggingMiddleware`        | pre + post + error | Structured request/response logging                      |
| `createTimingMiddleware`         | pre + post         | Per-call duration metrics                                |
| `createErrorTransformMiddleware` | error              | Normalise transport errors into a consistent shape       |
| `createCsrfMiddleware`           | pre                | Attach the CSRF double-submit header (cookie mode)       |

> There is **no** retry, cache, or circuit-breaker middleware.
> Retry is an `HttpClient` option; caching and circuit breaking
> live in the [fluent HTTP
> interface](#fluent-http-interface--caching-retry-circuit-breaking).
> See [Middleware → Not middleware](/docs/frontend/netron/middleware#not-middleware).

```typescript
import {
  createAuthMiddleware,
  createLoggingMiddleware,
} from '@omnitron-dev/netron-browser';

client.use(createAuthMiddleware({ tokenProvider: { getToken: () => localStorage.getItem('token') } }));
client.use(createLoggingMiddleware({ logRequestPayload: false }));
```

### Custom middleware

A middleware is a plain `(ctx, next) => Promise<void>` function;
register it with `client.use(fn, config?, stage?)`:

```typescript
import {
  type MiddlewareFunction,
  MiddlewareStage,
} from '@omnitron-dev/netron-browser';

const timingMiddleware: MiddlewareFunction = async (ctx, next) => {
  const start = performance.now();
  try {
    await next();
  } finally {
    console.debug(`${ctx.service}.${ctx.method}`, performance.now() - start, 'ms');
  }
};

client.use(timingMiddleware, { name: 'timing', priority: 100 }, MiddlewareStage.POST_RESPONSE);
```

See [Middleware](/docs/frontend/netron/middleware) for the full context shape,
stages, and registration rules.

## Fluent HTTP interface — caching, retry, circuit breaking

For HTTP-transport calls you can opt into a **fluent interface**
that adds per-call caching, retry, deduplication, timeout, and
more — without touching the middleware pipeline. It is reached
through the HTTP peer's `queryFluentInterface<T>(...)` (HTTP
transport only; not available over WebSocket):

```typescript
import { HttpRemotePeer } from '@omnitron-dev/netron-browser';

// `peer` is the HttpRemotePeer for an HTTP connection.
const users = await peer.queryFluentInterface<UserService>('users@1.0.0');

// Chainable per-call configuration, then call the method:
const user = await users
  .cache(60_000)              // cache for 60s (number → { maxAge })
  .retry({ attempts: 5 })     // fluent RetryOptions
  .timeout(3_000)
  .api.findById('u_42');
```

Each chainable method (`.cache()`, `.retry()`, `.timeout()`,
`.priority()`, `.dedupe()`, `.transform()`, `.fallback()`, …)
returns a configurable proxy; access the method through `.api`
(or call it directly on the proxy) to execute. Configuration
applies to that one call.

> The fluent primitives (`FluentInterface`, `HttpCacheManager`,
> `RetryManager`, `QueryBuilder`, `CircuitBreakerOptions`,
> `CacheOptions`, `RetryOptions`) are re-exported from the package
> root and shared with `@omnitron-dev/netron-http-core`.

### Caching

Caching is provided by the fluent interface's `HttpCacheManager`,
not by a middleware. Configure it per call with `.cache(...)`:

```typescript
// CacheOptions:
//   { maxAge, staleWhileRevalidate?, tags?, cacheOnError?, key? }
const user = await users
  .cache({
    maxAge:               60_000,
    staleWhileRevalidate: 10_000,        // serve stale up to 10s while refreshing
    tags:                 ['user:u_42', 'tier:pro'],
  })
  .api.findById('u_42');

// Invalidate by cache-key pattern (string = exact/prefix, or a RegExp):
users.invalidate('user:u_42*');
users.clearCache();
```

A shared `HttpCacheManager` can be attached to the peer so every
fluent interface it creates uses the same cache + stats — and so
you can invalidate by **tag** (the array form of `invalidate()`
matches tags, not keys):

```typescript
import { HttpCacheManager } from '@omnitron-dev/netron-browser';

const cache = new HttpCacheManager({ maxEntries: 1_000 });
peer.setCacheManager(cache);

// …after a tagged .cache({ tags: ['user:u_42'] }) call:
cache.invalidate(['user:u_42']);     // invalidate everything tagged user:u_42
```

> **Note:** `createClient`'s `http.caching` / `http.cacheTTL`
> options are currently inert — the basic `HttpClient` does not
> cache. Use the fluent interface (or netron-react's query cache —
> see [Caching](/docs/frontend/netron/caching)) for response caching.

`LRUCache` (from `@omnitron-dev/netron-browser/utils`) is a
general-purpose bounded cache utility — not a request cache
middleware.

### Retry & circuit breaking

The fluent `.retry(...)` takes a `RetryOptions`:

```typescript
// RetryOptions:
//   { attempts, backoff?, initialDelay?, maxDelay?, jitter?,
//     shouldRetry?, onRetry?, attemptTimeout?, factor?, idempotent? }
const user = await users
  .retry({
    attempts:     3,
    backoff:      'exponential',         // 'exponential' | 'linear' | 'constant'
    initialDelay: 500,
    maxDelay:     8_000,
    jitter:       0.1,
    idempotent:   true,                  // allow retrying ambiguous failures
  })
  .api.findById('u_42');
```

Circuit breaking is configured on a `RetryManager` via
`CircuitBreakerOptions` and attached to the peer:

```typescript
import { RetryManager } from '@omnitron-dev/netron-browser';

// CircuitBreakerOptions:
//   { threshold, windowTime, cooldownTime, successThreshold? }
const retryManager = new RetryManager({
  circuitBreaker: {
    threshold:    5,           // open after 5 failures…
    windowTime:   10_000,      // …within a 10s window
    cooldownTime: 30_000,      // try half-open after 30s
  },
});

peer.setRetryManager(retryManager);
```

When the breaker is open, calls fail fast with a `TitanError`
(`code: ErrorCode.SERVICE_UNAVAILABLE`, message
`"Circuit breaker is open"`) until the cooldown elapses; after
cooldown one probe runs and success closes the breaker. There is
no `CircuitOpenError` class — check `e.code ===
ErrorCode.SERVICE_UNAVAILABLE`.

## Authentication

The real class is **`AuthenticationClient`** (from
`@omnitron-dev/netron-browser`) — there is no `AuthManager`.
Construct one, attach it to the transport client, then drive it
with `setAuth()` / `clearAuth()` / `logout()`:

```typescript
import { HttpClient } from '@omnitron-dev/netron-browser';
import { AuthenticationClient, LocalTokenStorage } from '@omnitron-dev/netron-browser';

const auth = new AuthenticationClient({
  storage:           new LocalTokenStorage('platform:token'),  // or Session/Memory/Noop
  refreshConfig:     { endpoint: '/auth/refresh' },
  inactivityConfig:  { timeout: 30 * 60_000 },                 // 30 min
  crossTabSync:      { enabled: true },                        // storage-event sync
});

// Attach to the transport (HttpClient / WebSocketClient accept `auth`):
const client = new HttpClient({ url: '/api', auth });

// On sign-in — pass the server's AuthResult:
auth.setAuth(result);

// On sign-out:
await auth.logout();   // POSTs logoutConfig.endpoint if set, then clears state
auth.clearAuth();      // local-only clear (no server call)
```

`AuthenticationClient` features:

- **Token storage** — `LocalTokenStorage` / `SessionTokenStorage`
  / `MemoryTokenStorage` (default) / `NoopTokenStorage` (cookie
  mode).
- **Auto-refresh** — schedules a refresh before expiry via
  `refreshConfig`; `refreshToken()` coalesces concurrent calls.
- **Cross-tab sync** — sign-in / sign-out in one tab propagates
  to all open tabs (via the `storage` event).
- **Inactivity timeout** — auto-`clearAuth()` after N ms of no
  activity (`inactivityConfig`).
- **Events** — `auth.on('authenticated' | 'token-refreshed' |
  'unauthenticated' | 'error' | 'inactivity', cb)`.

Read the current token with `auth.getToken()` (synchronous,
`string | undefined`) and check status with
`auth.isAuthenticated()`. For the full reference — storage
backends, token transports, cookie mode — see
[Auth](/docs/frontend/netron/auth).

## Multi-backend client

When the app talks to multiple Netron servers — e.g., one for
identity, one for media, one for analytics — wrap each in a
`BackendClient` and pool them:

```typescript
import { createMultiBackendClient } from '@omnitron-dev/netron-browser';

// Backends sit behind one origin, separated by path — a gateway with
// /auth, /media, /analytics under it. One auth client and one middleware
// chain then cover all of them.
const client = createMultiBackendClient({
  baseUrl: 'https://api.example.com',
  backends: {
    auth:      { path: '/auth' },
    media:     { path: '/media' },
    analytics: { path: '/analytics' },
  },
  defaultBackend: 'auth',
  routing: {
    // String patterns match by prefix, not as globs.
    patterns: [
      { pattern: 'objects', backend: 'media' },
      { pattern: 'reports', backend: 'analytics' },
    ],
  },
});

const users = client.service<UserService>('users');
await users.findById('u_42');        // unrouted → defaultBackend, 'auth'
```

A string pattern matches by prefix, not as a glob; a `RegExp`
covers anything more. A call matching no rule goes to
`defaultBackend` rather than throwing.

### Health checks

`enableHealthChecks` (off by default) and `healthCheckInterval`
(30s) make the pool poll its backends. Nothing re-routes away
from an unhealthy one — there is no failover map and no
`onUnhealthy` policy; a call to a backend that is down fails like
any other call.

→ [Multi-backend](/docs/frontend/netron/multi-backend) for routing, per-backend
auth, and the React provider.

## Errors

Server-side `TitanError` subclasses arrive as the same class on
the client — wire format preserves the constructor name and
`code`:

```typescript
import { TitanError, ErrorCode } from '@omnitron-dev/netron-browser/errors';

try {
  await users.findById('missing');
} catch (e) {
  if (!(e instanceof TitanError)) throw e;

  switch (e.code) {
    case ErrorCode.NOT_FOUND:        return null;
    case ErrorCode.UNAUTHORIZED:     return triggerReauth();
    case ErrorCode.TOO_MANY_REQUESTS: return scheduleRetry(e);
    default:                          throw e;
  }
}
```

→ See [Titan / Errors catalog](/docs/titan/modules/errors-catalog)
for the full code reference.

## Subpaths

| Subpath                                         | Contents                                                                               |
| ----------------------------------------------- | -------------------------------------------------------------------------------------- |
| `@omnitron-dev/netron-browser`                  | Everything, including `AuthenticationClient` and the token storage + transport helpers |
| `@omnitron-dev/netron-browser/client`           | `NetronClient`, `HttpClient`, `WebSocketClient`, `BackendPool`                         |
| `@omnitron-dev/netron-browser/client/http`      | `HttpClient` alone                                                                     |
| `@omnitron-dev/netron-browser/client/websocket` | `WebSocketClient` alone                                                                |
| `@omnitron-dev/netron-browser/middleware`       | All built-in middleware                                                                |
| `@omnitron-dev/netron-browser/packet`           | Wire-format helpers (rarely used directly)                                             |
| `@omnitron-dev/netron-browser/errors`           | `TitanError` hierarchy mirror                                                          |
| `@omnitron-dev/netron-browser/utils`            | URL parsing, header normalisation, `LRUCache`                                          |
| `@omnitron-dev/netron-browser/types`            | All public types                                                                       |

There is no `/core`, `/core-tasks`, `/transport` or `/routing` subpath — those
rows described a layout the package does not have, and an import following any
of them fails to resolve. What they named is in the root or under `/client`.
`AuthenticationClient` is likewise at the root, not under an `/auth` subpath.

## Performance characteristics

| Operation                    | Cost                                |
| ---------------------------- | ----------------------------------- |
| HTTP RPC over LAN            | 0.5–2 ms                            |
| WebSocket RPC after connect  | 0.1–0.5 ms                          |
| WS reconnect from cold       | 100 ms – 1 s (depends on backoff)   |
| Cached call (LRU hit)        | \~0.01 ms (no network)              |
| Middleware overhead per call | \~0.05 ms per registered middleware |
| Proxy access                 | \~0.01 ms per property read         |

Bundle: \~20–30 kB gzipped for the root import. Subpath imports
can ship just the transport you use (HTTP-only: \~12 kB).

## Examples

The package ships runnable examples at
`packages/netron-browser/examples/` covering:

- Basic HTTP usage
- WebSocket subscriptions
- Authentication flow
- Multi-backend routing
- Custom middleware

## Best practices

- **Use service proxies, not raw `invoke`.** Types flow through;
  refactors stay safe.
- **One client per backend.** Don't recreate on every render —
  treat the client as a long-lived singleton.
- **Wire `AuthenticationClient` once.** Cross-tab sync requires a
  single instance to share state via the `storage` event.
- **Cache idempotent reads, not writes.** Mutations should
  invalidate by tag, not cache.
- **Circuit breaker before retry.** A `RetryManager` configured
  with `circuitBreaker` checks the breaker before each attempt,
  so a tripped breaker prevents the retry loop from hammering a
  dead backend.
- **Set `timeout`** explicitly. Browser fetch defaults are
  effectively no-timeout; surface as `TimeoutError` for clean
  client UX.
- **Reconnect bounded.** `maxReconnectAttempts` prevents tabs
  from connecting forever after a permanent outage.

## Anti-patterns

- **Storing tokens in cookies + localStorage.** Pick one. Cross-
  site contexts work better with HttpOnly cookies; same-site
  apps work better with localStorage + `AuthenticationClient`.
- **Per-call new client.** Defeats caching, breaks WS reconnect,
  wastes connections.
- **Catching `Error` generically.** Lose the typed `code`;
  always check `instanceof TitanError`.
- **Stacking the `HttpClient` `retry` option with fluent
  `.retry(...)`.** Double retries amplify failure load; pick one
  layer.
- **`maxReconnectAttempts: Infinity`.** A genuinely-dead server
  has open connections from every tab forever.

## See also

- [netron-react](/docs/frontend/netron/react) — React hooks built on this client
- [Titan / Netron](/docs/titan/netron) — server side
- [Titan / Errors catalog](/docs/titan/modules/errors-catalog) — typed errors
- [Prism](/docs/frontend/prism) — UI components that pair with this client

---

---
id: "frontend/netron/caching"
title: "Caching"
description: "LRU cache, stale-while-revalidate, tag invalidation."
---

# Caching

netron-browser caches HTTP responses through its **fluent HTTP
interface** (`HttpCacheManager`); netron-react layers a query
cache on top with React-aware subscriptions.

## Two cache layers

| Layer                                                | When to use                                       |
| ---------------------------------------------------- | ------------------------------------------------- |
| **netron-browser fluent cache** (`HttpCacheManager`) | Vanilla JS / web workers / non-React HTTP clients |
| **netron-react QueryCache**                          | React apps — used by `useQuery`                   |

Both share the same TTL / stale-while-revalidate / tag
invalidation semantics. Pick one — don't double-cache.

> The fluent cache is opt-in per call (`.cache(...)`) on a
> service obtained via `peer.queryFluentInterface(...)`. The
> basic `createClient()` / `HttpClient` path does **not** cache —
> its `http.caching` / `http.cacheTTL` options are currently
> inert.

## netron-react QueryCache

Used automatically by every `useQuery` / `useService(...).x.useQuery`:

```tsx
const { data, isLoading, isStale } = users.getUser.useQuery([userId], {
  staleTime:    30_000,
  cacheTime:    5 * 60_000,
  refetchOnWindowFocus:  true,
  refetchOnReconnect:    true,
  refetchInterval:       60_000,
});
```

| Option                 | Default      | Meaning                                                      |
| ---------------------- | ------------ | ------------------------------------------------------------ |
| `staleTime`            | `0`          | Data is fresh for N ms; while fresh, no refetch on subscribe |
| `cacheTime`            | `5 * 60_000` | Drop from cache N ms after last subscriber unmounts          |
| `refetchOnMount`       | `true`       | Refetch when a component mounts subscribing to this key      |
| `refetchOnWindowFocus` | `false`      | Refetch when tab regains focus                               |
| `refetchOnReconnect`   | `true`       | Refetch when WS / network reconnects                         |
| `refetchInterval`      | `false`      | Periodic refetch in ms                                       |
| `enabled`              | `true`       | Skip the query when `false`                                  |
| `placeholderData`      | —            | Initial value before first fetch                             |
| `select`               | —            | Transform `data` per subscription                            |

### Cache key

Generated from `[service, method, args]`:

```typescript
// All these are the same cache key:
users.getUser.useQuery(['u_42']);
users.getUser.useQuery(['u_42']);

// These are different keys:
users.getUser.useQuery(['u_42']);
users.getUser.useQuery(['u_43']);
```

Args are deep-equal-compared — `{a:1, b:2}` and `{b:2, a:1}`
match.

### Programmatic cache access

```tsx
import { useNetronClient } from '@omnitron-dev/netron-react';

function CacheManager() {
  const client = useNetronClient();
  const cache  = client.getQueryCache();

  return (
    <>
      <Button onClick={() => client.invalidateQueries({ queryKey: ['users'] })}>
        Refresh all users queries
      </Button>
      <Button onClick={() => client.removeQueries({ queryKey: ['users', 'getUser', 'u_42'] })}>
        Drop one specific
      </Button>
      <Button onClick={() => cache.set(['users', 'getUser', 'u_42'], updatedUser)}>
        Patch cached value
      </Button>
      <Button onClick={() => client.clear()}>
        Nuke everything
      </Button>
    </>
  );
}
```

The TanStack-style filter methods live on the **client**; the
low-level read/write methods live on the **QueryCache**
(`client.getQueryCache()`):

| Method                           | Lives on | Effect                                  |
| -------------------------------- | -------- | --------------------------------------- |
| `getQueryCache().get(key)`       | cache    | Read cached data without subscribing    |
| `getQueryCache().set(key, data)` | cache    | Write to cache directly                 |
| `invalidateQueries(filters)`     | client   | Mark stale → subscribers refetch        |
| `removeQueries(filters)`         | client   | Drop from cache                         |
| `cancelQueries(filters)`         | client   | Abort in-flight queries matching filter |
| `prefetchQuery(key, fetcher)`    | client   | Warm cache without rendering            |
| `clear()`                        | client   | Drop everything                         |

To patch a cached value functionally, read-then-set:

```tsx
const prev = cache.get(['users', 'getUser', 'u_42']);
cache.set(['users', 'getUser', 'u_42'], { ...prev, ...patch });
```

### Filter patterns

The client's filter methods take a `QueryFilters` object —
`{ queryKey?, exact?, status?, stale?, fetching?, predicate? }`.
`queryKey` is a prefix match by default; add `exact: true` for
an exact match.

```tsx
client.invalidateQueries({ queryKey: ['users'] });                  // every users.* query
client.invalidateQueries({ queryKey: ['users', 'getUser'] });       // every getUser
client.invalidateQueries({ queryKey: ['users', 'getUser', 'u_42'], exact: true }); // one specific
client.invalidateQueries({ predicate: (q) => q.queryKey[0] === 'users' && q.state.dataUpdatedAt < Date.now() - 60_000 });
```

The functional `predicate` form lets you invalidate by age,
state, or any custom condition.

## Stale-while-revalidate

```tsx
users.getUser.useQuery([userId], {
  staleTime: 30_000,
  cacheTime: 5 * 60_000,
});
```

Flow when component mounts:

```mermaid
flowchart LR
  Mount[Component mounts]
  Mount --> Check{In cache?}
  Check -- no --> Fetch[fetch + return]
  Check -- fresh --> Return[return cached]
  Check -- stale --> Revalidate[return stale + fetch in background]
  Revalidate --> UpdateCache[on success: cache update + re-render]
```

`isStale` is exposed in the result — components can show a
subtle "refreshing" indicator while stale-while-revalidate
runs.

## Optimistic updates

```tsx
const updateProfile = users.updateProfile.useMutation({
  onMutate: async (newProfile) => {
    // 1. Cancel any in-flight queries that would clobber the optimistic update
    client.cancelQueries({ queryKey: ['users', 'getUser', userId] });

    // 2. Snapshot the previous value
    const previous = cache.get(['users', 'getUser', userId]);

    // 3. Optimistically update
    cache.set(['users', 'getUser', userId], {
      ...previous,
      ...newProfile,
    });

    // 4. Return context for rollback
    return { previous };
  },
  onError: (err, _newProfile, context) => {
    // Rollback to snapshot
    if (context?.previous) {
      cache.set(['users', 'getUser', userId], context.previous);
    }
  },
  onSettled: () => {
    // Always re-fetch authoritative state
    client.invalidateQueries({ queryKey: ['users', 'getUser', userId] });
  },
});
```

Pattern:

1. **Cancel** to avoid race with in-flight fetches.
2. **Snapshot** for rollback.
3. **Optimistic write** for instant UI feedback.
4. **Rollback on error** — restore snapshot.
5. **Invalidate on settled** — re-sync with server truth.

## Tag-based invalidation

For non-React HTTP clients, attach an `HttpCacheManager` to the
peer and tag calls through the fluent `.cache({ tags })`:

```typescript
import { HttpCacheManager } from '@omnitron-dev/netron-browser';

const cache = new HttpCacheManager({ maxEntries: 1_000 });
peer.setCacheManager(cache);

const users = await peer.queryFluentInterface<UserService>('users@1.0.0');

// Tag a cached call with arbitrary labels:
await users
  .cache({
    maxAge:               60_000,
    staleWhileRevalidate: 10_000,
    tags:                 ['user:u_42', 'tier:pro'],
  })
  .api.getUser('u_42');

// Later, when user u_42 changes — invalidate by tag.
// (The array form of invalidate() matches tags; a bare string
// matches cache keys, so pass tags as an array.)
cache.invalidate(['user:u_42']);

// Or all pro-tier cached data:
cache.invalidate(['tier:pro']);
```

Tags are arbitrary strings — typical patterns:

- `user:<id>` — invalidate one user's queries
- `tenant:<id>` — invalidate one tenant's queries
- `entity:<type>` — invalidate one collection
- `feature:<flag>` — invalidate when a flag flips

A single cache entry can carry multiple tags; invalidating any
one drops the entry. `cache.invalidate()` also accepts a key
`RegExp` or a `'prefix*'` string for pattern-based invalidation.

## Prefetching

```tsx
import { useNetronClient, useService } from '@omnitron-dev/netron-react';

function ProjectLink({ id }: { id: string }) {
  const client   = useNetronClient();
  const projects = useService<ProjectService>('projects');
  return (
    <Link
      to={`/projects/${id}`}
      onMouseEnter={() => {
        client.prefetchQuery(
          ['projects', 'getProject', id],
          () => projects.getProject.call(id),
        );
      }}
    >
      {name}
    </Link>
  );
}
```

When the user hovers a link, prefetch the target's data so the
destination renders instantly.

## Cache stats

The fluent `HttpCacheManager` exposes hit/miss statistics:

```typescript
const stats = cache.getStats();
// CacheStats:
//   { entries, hits, misses, hitRate, sizeBytes, activeRevalidations }
```

Surface in devtools or a metrics dashboard.

## React Query parity

netron-react's QueryCache API is intentionally TanStack
Query–compatible at the high level — if you've used React
Query, the mental model carries over. The differences:

- Cache key is `[service, method, args]` not free-form.
- Backed by netron-browser, not raw fetch.
- Subscribes via `useService` / `useQuery` hooks; same shape.

## Best practices

- **`staleTime` ≥ 30 s** for most read queries — saves needless
  refetches.
- **`refetchOnWindowFocus: true`** for dashboards and admin
  surfaces — keeps data fresh after returning to a tab.
- **Optimistic updates** for fast-feedback mutations — but
  always invalidate on settled to converge with server truth.
- **Invalidate by widest sensible key** after mutations —
  invalidating `['users']` after creating a user catches `list`
  - `count` + `getUser` callers.
- **Use tags for non-React clients** — easier to reason about
  than key patterns.
- **Don't cache mutating calls.** `delete` / `update` should
  never read from cache.

## Anti-patterns

- **`staleTime: 0` everywhere.** Defeats caching; every mount
  refetches.
- **`cacheTime: Infinity`.** Cache grows unboundedly; first call
  after page reload is cold anyway.
- **Manual cache writes from multiple components.** Use
  invalidation; manual writes drift.
- **Caching paginated lists without page-aware keys.**
  `[users, list, {page: 1}]` and `[users, list, {page: 2}]`
  must be separate keys.
- **Invalidating after every keystroke.** Debounce or move to
  `onBlur`.

## See also

- [Browser client](/docs/frontend/netron/browser#fluent-http-interface--caching-retry-circuit-breaking) — fluent `HttpCacheManager` / `.cache()`
- [netron-react](/docs/frontend/netron/react) — `useQuery` / `useMutation`
- [Multi-backend](/docs/frontend/netron/multi-backend) — shared cache across backends

---

---
id: "frontend/netron/errors"
title: "Error handling"
description: "Typed errors across the wire, retry classification, recovery patterns."
---

# Error handling

Remote failures use `TitanError` and a stable numeric `code`. Binary RPC can
restore a registered constructor; HTTP clients do not promise the original
server-side subclass. Branch on `code`, not a constructor name or message.

## TitanError

`TitanError` and `ErrorCode` come from the `/errors` subpath, not
the package root. The root exports the transport-level errors
(`NetronError`, `ConnectionError`, `TimeoutError`, …) — the ones
raised by the client itself — while `/errors` carries what the
server sent, including `ErrorCode`, the `Errors` factory and
`toTitanError`.

```typescript
import { TitanError, ErrorCode } from '@omnitron-dev/netron-browser/errors';

try {
  await users.findById('missing');
} catch (e) {
  if (!(e instanceof TitanError)) throw e;

  switch (e.code) {
    case ErrorCode.NOT_FOUND:         return null;
    case ErrorCode.UNAUTHORIZED:      return reauth();
    case ErrorCode.FORBIDDEN:         return showAccessDenied();
    case ErrorCode.TOO_MANY_REQUESTS: return scheduleRetry(e);
    case ErrorCode.VALIDATION_ERROR:  return showFieldErrors(e.details?.errors);
    case ErrorCode.SERVICE_UNAVAILABLE: return showOutage();
    case ErrorCode.REQUEST_TIMEOUT:   return showTimeout();
    default:                          throw e;
  }
}
```

The full `ErrorCode` enum mirrors HTTP status codes — see
[Titan / Errors catalog](/docs/titan/modules/errors-catalog)
for the complete reference.

## HTTP error contract

Node and browser HTTP clients share the decoder in `netron-protocol`. Peer,
connection and per-request-hint paths preserve numeric codes, decimal code strings
and shared symbolic codes such as `FORBIDDEN`. Application-specific symbols stay
in `details.errorCode`; their numeric category comes from the HTTP status.
Malformed codes fall back to a failure category, never `NaN` or a success status.
`X-Request-Id`, `X-Correlation-Id`, `X-Trace-Id` and `X-Span-Id` remain available on
the error for diagnostics, including authentication and rate-limit failures.

A permission refusal remains 403; an authority outage remains 503. This distinction
controls both the message shown to an operator and the decision an agent can make.
Do not repeat a denied mutation or interpret a timeout as proof of cancellation.
The request deadline includes reading the body, not only receiving its headers;
one expired request does not cancel another request on the same connection.

## Transport-level errors

Errors that don't come from the server's `TitanError` system:

| Class             | When                                             |  Recoverable? |
| ----------------- | ------------------------------------------------ | :-----------: |
| `NetworkError`    | DNS failure, connection refused, browser offline |   ✓ (often)   |
| `TimeoutError`    | Request exceeded `timeout`                       | ⚠ (sometimes) |
| `ConnectionError` | WS upgrade failed; HTTP cert invalid             |       ✗       |
| `TransportError`  | Generic transport-layer failure                  |       ⚠       |

These are exported from the package root alongside the other
Netron error classes (`NetronError`, `ProtocolError`,
`ServiceError`, `MethodNotFoundError`, `InvalidArgumentsError`,
`SerializationError`).

> There is **no** `CircuitOpenError` class. When the fluent HTTP
> interface's circuit breaker is open it throws a `TitanError`
> with `code: ErrorCode.SERVICE_UNAVAILABLE` (message
> `"Circuit breaker is open"`) — check the code, not a class.

```typescript
import { NetworkError, TimeoutError, ConnectionError }
  from '@omnitron-dev/netron-browser';
import { TitanError, ErrorCode } from '@omnitron-dev/netron-browser/errors';

try {
  await users.findById(id);
} catch (e) {
  if (e instanceof NetworkError) {
    return showOfflineBanner();
  }
  if (e instanceof TimeoutError) {
    return showSlowNetworkWarning();
  }
  // Circuit breaker open (fluent HTTP interface):
  if (e instanceof TitanError && e.code === ErrorCode.SERVICE_UNAVAILABLE) {
    return showServiceUnavailable();
  }
  throw e;
}
```

## Errors in `useQuery` / `useMutation`

Errors land on `error`:

```tsx
const { data, error, isError } = users.getUser.useQuery([id]);

if (isError) {
  if (error instanceof TitanError && error.code === ErrorCode.NOT_FOUND) {
    return <NotFoundCard />;
  }
  return <ErrorCard error={error} onRetry={() => refetch()} />;
}
```

For mutations:

```tsx
const invite = users.invite.useMutation({
  onError: (error) => {
    if (error instanceof TitanError && error.code === ErrorCode.VALIDATION_ERROR) {
      for (const fieldErr of error.details?.errors ?? []) {
        form.setError(fieldErr.path, { type: 'server', message: fieldErr.message });
      }
    } else {
      toast.error('Could not send invite');
    }
  },
});
```

## Server-side validation errors

When the server's `@Validate(schema)` rejects, the error
carries field-level details:

```typescript
{
  code: ErrorCode.VALIDATION_ERROR,
  message: 'Validation failed',
  details: {
    errors: [
      { path: 'email',    message: 'Invalid email format',  expected: 'email', received: 'foo' },
      { path: 'password', message: 'Must be ≥ 8 characters' },
    ],
  },
}
```

Map these onto form fields:

```tsx
catch (e) {
  if (e instanceof TitanError && e.code === ErrorCode.VALIDATION_ERROR) {
    for (const err of e.details?.errors ?? []) {
      form.setError(err.path as any, { type: 'server', message: err.message });
    }
    return;
  }
  form.setError('root', { message: 'Something went wrong' });
}
```

The error appears under the matching `<Field>` exactly like
client-side errors — no special UI path.

## Retry classification

The fluent HTTP interface's retry (`RetryOptions`, via
`.retry({ ... })`) ships a default `shouldRetry` that classifies
failures by whether the request provably never reached the
server vs. an *ambiguous* failure where it may have been
processed. Ambiguous failures are retried **only when you mark
the call `idempotent: true`**, so a mutation is never silently
re-executed:

| Failure                                                      |  Retried by default? | Why                                                                                    |
| ------------------------------------------------------------ | :------------------: | -------------------------------------------------------------------------------------- |
| Connection refused / DNS / unreachable                       |           ✓          | Request provably never reached the server                                              |
| `429 TOO_MANY_REQUESTS`                                      |           ✓          | Server rejected without processing; honours `Retry-After`                              |
| Connection reset / timeout                                   | only if `idempotent` | Ambiguous — may have been processed                                                    |
| `5xx` (server) / `408`                                       | only if `idempotent` | Ambiguous                                                                              |
| `SERVICE_UNAVAILABLE` / `REQUEST_TIMEOUT` / `INTERNAL_ERROR` | only if `idempotent` | Ambiguous                                                                              |
| Other `4xx` (`400`/`403`/`404`/`409`/`422`)                  |           ✗          | Deterministic client error — won't change                                              |
| `TypeError` / `ReferenceError`                               |           ✗          | Programming bug                                                                        |
| `401 UNAUTHORIZED`                                           |        special       | An auth-error middleware refreshes the token and re-invokes once (separate from retry) |

A custom `shouldRetry` overrides the gate entirely — you own the
decision (signature `(error, attempt) => boolean | Promise<boolean>`):

```typescript
const users = await peer.queryFluentInterface<UserService>('users@1.0.0');

await users
  .retry({
    attempts: 3,
    shouldRetry: (error, attempt) => {
      // Cap retries on timeout (it may have succeeded server-side):
      if (error instanceof TimeoutError && attempt >= 2) return false;
      // Retry network failures and 5xx:
      return error instanceof NetworkError ||
             (error instanceof TitanError && error.code >= 500);
    },
  })
  .api.findById(id);
```

## Circuit breaker integration

The circuit breaker is **not** a middleware — it lives on a
`RetryManager` (the fluent HTTP interface) configured with
`CircuitBreakerOptions`:

```typescript
import { RetryManager } from '@omnitron-dev/netron-browser';

const retryManager = new RetryManager({
  circuitBreaker: {
    threshold:    5,         // open after 5 failures…
    windowTime:   10_000,    // …within a 10s window
    cooldownTime: 30_000,    // try half-open after 30s
  },
});

peer.setRetryManager(retryManager);

const users = await peer.queryFluentInterface<UserService>('users@1.0.0');
await users.retry({ attempts: 3 }).api.findById(id);
```

The breaker is checked **before each attempt**: while it is open,
calls fail fast — the `RetryManager` throws a `TitanError` with
`code: ErrorCode.SERVICE_UNAVAILABLE` (message `"Circuit breaker
is open"`) without attempting the request. After `cooldownTime`
one probe runs; success closes the breaker. There is no
`CircuitOpenError` class — match on the code.

## Error UI patterns

### Inline form errors (mutations)

`FormAlert` takes the message as **children** — there is no
`error` prop. The distinction matters more than it looks: the
component renders with empty children rather than unmounting (so
a conditional slot causes no layout shift), and it announces
itself with `role="alert"`, `aria-live="assertive"` and
`aria-atomic="true"`. Passing the error object to a prop that
does not exist therefore produces an alert that a screen reader
announces as empty — silence where the failure should be.

```tsx
<form>
  {form.formState.errors.root && (
    <FormAlert title="Couldn't sign in">
      {form.formState.errors.root.message}
    </FormAlert>
  )}
  <Field name="email" />
  <Field name="password" />
</form>
```

### Page-level error (query failure)

```tsx
if (error instanceof TitanError && error.code === ErrorCode.NOT_FOUND) {
  return <EmptyContent
    illustration="error-404"
    title="Not found"
    description="The thing you're looking for doesn't exist."
    action={<Button onClick={() => navigate('/')}>Home</Button>}
  />;
}

if (error) {
  return <EmptyContent
    illustration="error-500"
    title="Something broke"
    description={error.message}
    action={<Button onClick={() => refetch()}>Try again</Button>}
  />;
}
```

### Toast (background failure)

```tsx
const save = useMutation({
  onError: (error) => {
    toast.error(`Save failed: ${error instanceof TitanError ? error.message : 'unknown'}`);
  },
});
```

Toasts for **background events** (autosave failed, webhook
errored); page-level cards for primary content; inline alerts
for form submissions.

## Error boundaries

For synchronous render errors (rare with proper data fetching):

```tsx
import { ErrorBoundary } from '@omnitron-dev/prism';

<ErrorBoundary
  fallback={(error, reset) => (
    <EmptyContent
      illustration="error-500"
      title="Something broke"
      description={error.message}
      action={<Button onClick={reset}>Reload section</Button>}
    />
  )}
  onError={(error, info) => reportToSentry(error, info)}
>
  <SuspectComponent />
</ErrorBoundary>
```

Boundary catches **render-time** errors only — async failures
go through the standard `useQuery` `error` flow.

## Global error handling

```typescript
client.on('error', (error, ctx) => {
  if (error instanceof TitanError && error.code === ErrorCode.UNAUTHORIZED) {
    auth.clear();
    navigate('/sign-in');
    return;
  }
  reportToSentry(error, { service: ctx.service, method: ctx.method });
});
```

Use sparingly — most errors should be handled at the call site
where the context is richer.

## Reporting to Sentry

```typescript
import {
  type MiddlewareFunction,
  MiddlewareStage,
} from '@omnitron-dev/netron-browser';

const sentryMiddleware: MiddlewareFunction = async (ctx, next) => {
  Sentry.withScope((scope) => {
    scope.setTag('rpc.service', ctx.service);
    scope.setTag('rpc.method',  ctx.method);
    scope.setContext('rpc',     { args: ctx.args });
    Sentry.captureException(ctx.error);   // populated in the error stage
  });
  await next();        // re-throw
};

// Register on the error stage:
client.use(sentryMiddleware, { name: 'sentry', priority: 200 }, MiddlewareStage.ERROR);
```

Filter noise — don't report `NOT_FOUND` or `UNAUTHORIZED`,
they're not bugs.

## Anti-patterns

- **Catching `Error` generically.** Loses the typed identity;
  always check `instanceof TitanError`.
- **Mapping every error to "Something went wrong".** Users get
  no actionable info; check `code` and route accordingly.
- **Retry on `4xx`.** Won't help; logs the user out of all
  patience.
- **Logging full `TitanError` details client-side.** Server
  details may be sensitive; log `code` + `message`.
- **Throwing custom error classes without registering them.**
  Wire-format only preserves classes the receiver knows about;
  for app-specific errors, extend `TitanError` and ensure both
  sides import the same definition.
- **No error UI for `useQuery`.** Users see infinite spinner
  on backend failure; always handle `isError`.

## Best practices

- **Switch on `code`, not `message`.** Messages are
  human-readable; codes are stable.
- **Show field-level errors next to fields**; form-level
  errors above the form; transient/background errors in toasts.
- **Wire `SentryMiddleware` once** for global reporting; let
  call sites handle UI.
- **Pair retry with circuit breaker.** Configure the
  `RetryManager` with `circuitBreaker` so retries don't amplify
  failure load on a sick backend.
- **Mark idempotent calls `idempotent: true`.** The default
  fluent retry only retries ambiguous failures (5xx / reset /
  timeout) for idempotent calls — mutations stay safe.
- **Honour `Retry-After`** on `TOO_MANY_REQUESTS` — the fluent
  `RetryManager` reads it automatically; do it manually if you
  write your own retry loop.

## See also

- [Titan / Errors catalog](/docs/titan/modules/errors-catalog) — full server-side error reference
- [Browser client / Fluent HTTP interface](/docs/frontend/netron/browser#fluent-http-interface--caching-retry-circuit-breaking) — fluent retry & circuit breaking
- [Middleware / Not middleware](/docs/frontend/netron/middleware#not-middleware) — why retry/cache/circuit-breaking aren't middleware
- [Auth / 401 handling](/docs/frontend/netron/auth#auto-refresh-flow)

---

---
id: "frontend/netron/index"
title: "Overview"
description: "Browser-side Netron RPC — netron-browser + netron-react."
---

# Netron in the browser

> **Note about server-side Netron:** the server lives inside
> Titan itself at `@omnitron-dev/titan/netron` — including all
> four transports (HTTP / WebSocket / TCP / Unix). You don't
> install a separate package for the server. This section
> covers the **browser-side** clients only.

Two packages cover the browser side of Netron RPC:

| Package                                                         | Role                                                               | Framework                                                                                           |
| --------------------------------------------------------------- | ------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------- |
| [`@omnitron-dev/netron-browser`](/docs/frontend/netron/browser) | Transport layer — HTTP, WebSocket, middleware, auth, multi-backend | **Framework-agnostic** — vanilla JS, Vue, Svelte, Solid, Angular, Lit, React, Web Workers, Electron |
| [`@omnitron-dev/netron-react`](/docs/frontend/netron/react)     | React-specific hooks, providers, cache, devtools                   | **React-only**                                                                                      |

For React apps you install both. For non-React apps install
just `netron-browser`:

```bash
# React apps:
pnpm add @omnitron-dev/netron-browser @omnitron-dev/netron-react

# Vue / Svelte / Solid / Angular / Lit / vanilla JS:
pnpm add @omnitron-dev/netron-browser
```

## How they relate

```mermaid
flowchart LR
  App[Your component]
  Hook[netron-react hook<br/>useQuery / useService]
  Cache[QueryCache]
  Client[netron-browser<br/>NetronClient]
  MW[Middleware pipeline]
  Wire[(HTTP / WS)]
  Server[Titan @Service / @Public]

  App --> Hook --> Cache --> Client --> MW --> Wire --> Server
```

**`netron-browser` is the foundation** and works in any
JavaScript environment. **`netron-react` is an optional layer**
that adds React glue: cache, suspense, providers, devtools,
multi-backend hooks. For Vue / Svelte / Solid / Angular / Lit
apps, wrap `netron-browser` calls in your framework's
reactivity primitives directly.

## Deep dives

Topic-focused references:

- [Transports](/docs/frontend/netron/transports) — choosing an HTTP or WebSocket endpoint, React selection and reconnect behaviour
- [Middleware](/docs/frontend/netron/middleware) — three-stage pipeline, built-in middleware, custom middleware
- [Caching](/docs/frontend/netron/caching) — LRU cache, stale-while-revalidate, tag invalidation
- [Auth manager](/docs/frontend/netron/auth) — token rotation, cross-tab sync, inactivity timeout
- [Multi-backend](/docs/frontend/netron/multi-backend) — `MultiBackendProvider`, routing, per-backend hooks
- [Error handling](/docs/frontend/netron/errors) — typed errors, retry classification, circuit breaker
- [Testing](/docs/frontend/netron/testing) — TestNetronProvider, integration patterns
- [SSR](/docs/frontend/netron/ssr) — dehydration / hydration

## Quick start — single backend

```tsx
import { NetronReactClient, NetronProvider, useService }
  from '@omnitron-dev/netron-react';

interface UserService {
  getUser(id: string): Promise<User>;
}

const client = new NetronReactClient({
  url:       'https://api.example.com',
  transport: 'http',
});

function App() {
  return (
    <NetronProvider client={client}>
      <UserCard userId="u_42" />
    </NetronProvider>
  );
}

function UserCard({ userId }: { userId: string }) {
  const users = useService<UserService>('users');
  const { data, isLoading } = users.getUser.useQuery([userId]);
  return isLoading ? <Skeleton /> : <div>{data.email}</div>;
}
```

## Quick start — multi-backend

```tsx
import { createMultiBackendClient }
  from '@omnitron-dev/netron-browser';
import { MultiBackendProvider, useBackendService }
  from '@omnitron-dev/netron-react';

const client = createMultiBackendClient({
  baseUrl: 'https://api.example.com',
  backends: {
    auth:    { path: '/auth',    transport: 'http' },
    media:   { path: '/media',   transport: 'http' },
    streams: { path: '/streams', transport: 'websocket' },
  },
  routing: {
    patterns: [
      { pattern: 'users.',   backend: 'auth' },
      { pattern: 'objects.', backend: 'media' },
      { pattern: 'events.',  backend: 'streams' },
    ],
  },
});

<MultiBackendProvider client={client}>
  <Outlet />
</MultiBackendProvider>

function Component() {
  const users = useBackendService<UserService>('auth', 'users');
  const { data } = users.getUser.useQuery([id]);
}
```

## See also

- [Prism / Netron integration](/docs/frontend/prism#netron-integration--omnitron-devprismnetron) —
  re-export helpers via Prism
- [Titan / Netron](/docs/titan/netron) — server side
- [Architecture](/docs/titan/concepts/architecture) — full
  cross-stack picture

---

---
id: "frontend/netron/middleware"
title: "Middleware"
description: "Three-stage middleware pipeline, built-ins, custom middleware."
---

# Middleware

The middleware pipeline runs around every RPC invocation —
auth, logging, timing, error transformation, CSRF, custom hooks.
Three stages, prioritised within each.

Middleware in `@omnitron-dev/netron-browser` are **factory
functions that return a `MiddlewareFunction`** — there are no
middleware classes. You register them on an `HttpClient` or
`WebSocketClient` (not on `NetronClient` — see
[Registering middleware](#registering-middleware)).

## Three-stage pipeline

```mermaid
flowchart LR
  Inv["client.invoke(...)"] --> Pre[pre-request stage]
  Pre --> Send[transport.send]
  Send -- ok --> Post[post-response stage]
  Send -- throws --> Err[error stage]
  Post --> Out[return value]
  Err --> Out
```

| Stage           | Constant                        | Runs                            | Typical uses                                        |
| --------------- | ------------------------------- | ------------------------------- | --------------------------------------------------- |
| `pre-request`   | `MiddlewareStage.PRE_REQUEST`   | Before transport call           | Attach auth header, attach CSRF token, start timing |
| `post-response` | `MiddlewareStage.POST_RESPONSE` | After successful transport call | Log/measure the response, read response headers     |
| `error`         | `MiddlewareStage.ERROR`         | After transport throws          | Refresh-token-and-retry (auth), transform errors    |

Within a stage, lower `priority` runs first (default `100`). The
error stage runs your error middleware; on the HTTP transport, a
middleware that signals an auth retry triggers exactly **one**
re-send of the request (see
[`createAuthErrorMiddleware`](#createautherrormiddleware)).

> Retry, caching, and circuit-breaking are **not** middleware in
> this package — see [Not middleware](#not-middleware).

## Middleware shape

A middleware is a plain function: `(ctx, next) => …`. Call
`await next()` to run the rest of the chain.

```typescript
import type {
  MiddlewareFunction,
  ClientMiddlewareContext,
} from '@omnitron-dev/netron-browser';

// type MiddlewareFunction =
//   (ctx: ClientMiddlewareContext, next: () => Promise<void>)
//     => Promise<void> | void;

const myMiddleware: MiddlewareFunction = async (ctx, next) => {
  // ...before
  await next();
  // ...after
};
```

The context (`ClientMiddlewareContext`):

```typescript
interface ClientMiddlewareContext {
  service: string;
  method: string;
  args: any[];

  request?: {
    headers?: Record<string, string>;
    timeout?: number;
    metadata?: Record<string, any>;
    credentials?: RequestCredentials; // cookie/hybrid mode → 'include'
  };

  response?: {
    data?: any;
    headers?: Record<string, string>;
    metadata?: Record<string, any>;
  };

  error?: Error;                       // populated in the error stage

  timing: {
    start: number;
    end?: number;
    middlewareTimes: Map<string, number>;
  };

  metadata: Map<string, any>;          // shared state across middleware
  skipRemaining?: boolean;             // set to short-circuit the chain
  transport: 'http' | 'websocket';
}
```

The per-middleware config (`MiddlewareConfig`, all optional except
`name`):

```typescript
interface MiddlewareConfig {
  name: string;                                       // for metrics/debug
  priority?: number;                                  // default 100; lower runs first
  condition?: (ctx: ClientMiddlewareContext) => boolean;
  onError?: (error: Error, ctx: ClientMiddlewareContext) => void;
  services?: string[] | RegExp;                       // restrict to services
  methods?: string[] | RegExp;                        // restrict to methods
}
```

## Registering middleware

`client.use(middleware, config?, stage?)` registers a middleware
and returns `this` (chainable). Available on `HttpClient` and
`WebSocketClient`. `stage` defaults to
`MiddlewareStage.PRE_REQUEST`.

```typescript
import { HttpClient } from '@omnitron-dev/netron-browser/client';
import {
  createAuthMiddleware,
  createLoggingMiddleware,
  StorageTokenProvider,
  MiddlewareStage,
} from '@omnitron-dev/netron-browser';

const client = new HttpClient({ url: 'https://api.example.com' });

client
  .use(createAuthMiddleware({
    tokenProvider: new StorageTokenProvider(localStorage, 'token'),
  }))
  .use(createLoggingMiddleware({ logRequestPayload: true }));
```

Or build a pipeline up front and pass it in. The `middleware`
option is a **single** `IMiddlewareManager` (a `MiddlewarePipeline`
instance) — **not an array**:

```typescript
import {
  MiddlewarePipeline,
  createLoggingMiddleware,
  createTimingMiddleware,
} from '@omnitron-dev/netron-browser';

const pipeline = new MiddlewarePipeline();
pipeline.use(createLoggingMiddleware());
pipeline.use(createTimingMiddleware());

const client = new HttpClient({
  url: 'https://api.example.com',
  middleware: pipeline,
});
```

**`createClient()` / `NetronClient` has no middleware option.** The
high-level `createClient()` / `NetronClient` API exposes **no**
middleware option. To register middleware, construct an
`HttpClient` or `WebSocketClient` directly and use `.use(...)` (or
the `middleware` pipeline option above).

## Built-in middleware

Most factories are importable from the package root
(`@omnitron-dev/netron-browser`). The two auth **error** helpers
live only on the `/middleware` subpath — they are noted inline.

### `createAuthMiddleware`

Attaches the auth token to outgoing requests in the **pre-request**
stage. `tokenProvider` is required; everything else is optional.
A `transport` strategy (bearer / cookie / hybrid) can be supplied
for T#176 cookie-mode; if omitted, a bearer transport is
synthesised from the legacy `headerName` / `tokenPrefix` options.

```typescript
import {
  createAuthMiddleware,
  StorageTokenProvider,
  SimpleTokenProvider,
} from '@omnitron-dev/netron-browser';

client.use(createAuthMiddleware({
  tokenProvider: new StorageTokenProvider(localStorage, 'access_token'),
  // headerName:  'Authorization',  // default
  // tokenPrefix: 'Bearer ',        // default
  skipServices: ['public'],
  skipMethods:  ['auth@1.0.0.signin'],
}));
```

| Option          | Default              | Notes                                                                                              |
| --------------- | -------------------- | -------------------------------------------------------------------------------------------------- |
| `tokenProvider` | —                    | **Required.** `{ getToken(): string \| null \| Promise<string \| null>; getTokenType?(): string }` |
| `transport`     | bearer (synthesised) | `IClientTokenTransport` strategy; cookie/hybrid set `credentials: 'include'`                       |
| `headerName`    | `'Authorization'`    | Legacy bearer-only; ignored when `transport` is given                                              |
| `tokenPrefix`   | `'Bearer '`          | Legacy bearer-only; ignored when `transport` is given                                              |
| `skipServices`  | `[]`                 | Service names to skip                                                                              |
| `skipMethods`   | `[]`                 | `service.method` pairs to skip                                                                     |

Token-provider helpers:

- `SimpleTokenProvider(token | () => string | null)` — wrap a
  static string or getter.
- `StorageTokenProvider(storage, key)` — read from
  `localStorage` / `sessionStorage`.

### `createAuthErrorMiddleware`

> Import from the **`/middleware` subpath** —
> `@omnitron-dev/netron-browser/middleware`. These two helpers are
> not re-exported from the package root.

An **error**-stage middleware that handles auth failures: on 401
it refreshes the token via the `AuthenticationClient` and signals
a retry (the HTTP transport re-sends once); 403 and 429 surface
through callbacks. Register it on the `ERROR` stage.

```typescript
import { AuthenticationClient } from '@omnitron-dev/netron-browser';
import {
  createAuthErrorMiddleware,
  MiddlewareStage,
} from '@omnitron-dev/netron-browser/middleware';

const authClient = new AuthenticationClient({ autoRefresh: true });

client.use(
  createAuthErrorMiddleware({
    authClient,
    onSessionExpired: () => { window.location.href = '/login'; },
    onAccessDenied:   (details) => showNotification(details),
    onRateLimited:    (retryAfter) => warn(`Retry after ${retryAfter}s`),
    maxRetries: 1,
  }),
  { name: 'auth-error-handler', priority: 10 },
  MiddlewareStage.ERROR,
);
```

| Option             | Default | Notes                                                                             |
| ------------------ | ------- | --------------------------------------------------------------------------------- |
| `authClient`       | —       | **Required.** `AuthenticationClient` used for refresh                             |
| `onSessionExpired` | —       | Called on 401 when refresh fails                                                  |
| `onAccessDenied`   | —       | Called on 403 with `{ service, method, userId?, requiredPermissions?, details? }` |
| `onRateLimited`    | —       | Called on 429 with `retryAfter` seconds                                           |
| `maxRetries`       | `1`     | Max refresh-and-retry attempts                                                    |
| `retryStatusCodes` | `[401]` | Codes that trigger refresh+retry                                                  |
| `emitEvents`       | `true`  | Emit events on the auth client                                                    |

There is also a shorthand:

```typescript
import { createSimpleAuthErrorMiddleware } from '@omnitron-dev/netron-browser/middleware';

client.use(
  createSimpleAuthErrorMiddleware(authClient, {
    onSessionExpired: () => { window.location.href = '/login'; },
  }),
  undefined,
  MiddlewareStage.ERROR,
);
```

### `createLoggingMiddleware`

Logs requests, responses, and errors. Wraps all three stages
around `next()`.

```typescript
import {
  createLoggingMiddleware,
  ConsoleLogger,
} from '@omnitron-dev/netron-browser';

client.use(createLoggingMiddleware({
  logger: new ConsoleLogger('[netron]'),
  logRequestPayload:  true,
  logResponsePayload: false,
}));
```

| Option                         | Default               | Notes                                        |
| ------------------------------ | --------------------- | -------------------------------------------- |
| `logger`                       | `new ConsoleLogger()` | Any `Logger` (`debug`/`info`/`warn`/`error`) |
| `requestLogLevel`              | `'info'`              | `LogLevel`                                   |
| `responseLogLevel`             | `'info'`              | `LogLevel`                                   |
| `errorLogLevel`                | `'error'`             | `LogLevel`                                   |
| `logRequestPayload`            | `false`               | Include `ctx.args` — avoid in prod           |
| `logResponsePayload`           | `false`               | Include `ctx.response.data`                  |
| `skipServices` / `skipMethods` | `[]`                  | Skip lists                                   |

Also exported: the `Logger` interface, `ConsoleLogger` class, and
`LogLevel` type.

### `createTimingMiddleware`

Measures call duration via the `performance` API, records to a
collector, and can flag slow calls.

```typescript
import {
  createTimingMiddleware,
  InMemoryMetricsCollector,
} from '@omnitron-dev/netron-browser';

const collector = new InMemoryMetricsCollector(1000);

client.use(createTimingMiddleware({
  collector,
  slowThreshold: 1000,
  onSlowRequest: (m) => console.warn('slow', m.service, m.method, m.duration),
  onMeasure:     (m) => analytics.record(m),
}));

collector.getAverageDuration('users', 'findById');
collector.getSlowestCalls(10);
```

| Option                         | Default      | Notes                                                   |
| ------------------------------ | ------------ | ------------------------------------------------------- |
| `collector`                    | —            | `MetricsCollector`; `InMemoryMetricsCollector` provided |
| `onMeasure`                    | —            | `(metrics: PerformanceMetrics) => void` per call        |
| `slowThreshold`                | —            | ms threshold for the slow-call callback                 |
| `onSlowRequest`                | console.warn | Called when `duration > slowThreshold`                  |
| `skipServices` / `skipMethods` | `[]`         | Skip lists                                              |

Also exported: `MetricsCollector` / `PerformanceMetrics` types and
`InMemoryMetricsCollector` (package root). For observing the emitted
`performance.measure` entries there is also `createPerformanceObserver(cb)`,
which is exported only from the `/middleware` subpath
(`@omnitron-dev/netron-browser/middleware`), not the package root.

### `createErrorTransformMiddleware`

An **error**-stage middleware that normalises thrown errors into a
consistent `NormalizedError` shape and (optionally) maps codes to
user-facing messages.

```typescript
import {
  createErrorTransformMiddleware,
  CommonErrorMessages,
} from '@omnitron-dev/netron-browser';

client.use(createErrorTransformMiddleware({
  errorMessages: CommonErrorMessages,
  includeStack:  false,
  onError: (normalized) => Sentry.captureException(normalized),
}));
```

| Option                         | Default                   | Notes                                                                 |
| ------------------------------ | ------------------------- | --------------------------------------------------------------------- |
| `transformer`                  | `defaultErrorTransformer` | `(error, { service, method, transport }) => NormalizedError \| Error` |
| `onError`                      | —                         | Called with the normalised error                                      |
| `includeStack`                 | `true`                    | Include `error.stack`                                                 |
| `includeContext`               | `true`                    | Include `service` / `method`                                          |
| `errorMessages`                | `{}`                      | Map of error code → message                                           |
| `skipServices` / `skipMethods` | `[]`                      | Skip lists                                                            |

Also exported: `defaultErrorTransformer`, `CommonErrorMessages`,
and the guards `isRetryableError`, `isClientError`,
`isServerError`.

### `createCsrfMiddleware`

Pre-request middleware for cookie-mode auth (T#176). Reads the
CSRF cookie and echoes it in the `X-CSRF-Token` header
(double-submit). No-ops outside the browser or before the cookie
is set.

```typescript
import { createCsrfMiddleware } from '@omnitron-dev/netron-browser';

client.use(createCsrfMiddleware({
  // cookieName: 'omni_csrf',   // default
  // headerName: 'X-CSRF-Token', // default
  skipMethods: ['auth@1.0.0.signin', 'auth@1.0.0.refresh'],
}));
```

| Option        | Default          | Notes                                          |
| ------------- | ---------------- | ---------------------------------------------- |
| `cookieName`  | `'omni_csrf'`    | CSRF cookie to read                            |
| `headerName`  | `'X-CSRF-Token'` | Header to echo it in                           |
| `skipMethods` | `[]`             | `service.method` pairs the server has exempted |

Register this alongside `createAuthMiddleware` when using the
cookie token transport; bearer mode does not need it.

## Custom middleware

Any `(ctx, next)` function works. Read/write `ctx.request.headers`
before `next()`; inspect `ctx.response` / `ctx.error` after.

```typescript
import type { MiddlewareFunction } from '@omnitron-dev/netron-browser';

const tenantMiddleware: MiddlewareFunction = async (ctx, next) => {
  if (!ctx.request) ctx.request = {};
  if (!ctx.request.headers) ctx.request.headers = {};
  ctx.request.headers['X-Tenant-ID'] = getCurrentTenant();
  await next();
};

client.use(tenantMiddleware, { name: 'tenant', priority: 50 });
```

### Useful patterns

**Skip transport on a cache hit** (set `skipRemaining` and
populate `ctx.response`):

```typescript
const cacheReadMiddleware: MiddlewareFunction = async (ctx, next) => {
  const key = `${ctx.service}.${ctx.method}:${JSON.stringify(ctx.args)}`;
  const hit = memoryCache.get(key);
  if (hit !== undefined) {
    ctx.response = { data: hit };
    ctx.skipRemaining = true;     // short-circuits the chain
    return;                        // do not call next()
  }
  await next();
};
```

**Error tagging for Sentry** (register on the `ERROR` stage):

```typescript
import { MiddlewareStage } from '@omnitron-dev/netron-browser';

const sentryMiddleware: MiddlewareFunction = async (ctx, next) => {
  try {
    await next();
  } catch (error) {
    Sentry.withScope((scope) => {
      scope.setTag('rpc.service', ctx.service);
      scope.setTag('rpc.method',  ctx.method);
      scope.setContext('rpc', { args: ctx.args });
      Sentry.captureException(error);
    });
    throw error;                   // re-throw
  }
};

client.use(sentryMiddleware, { name: 'sentry', priority: 200 }, MiddlewareStage.ERROR);
```

**Restrict to specific services/methods** via config:

```typescript
client.use(myMiddleware, {
  name: 'mutations-only',
  methods: /\.(create|update|delete)$/,
});
```

## Ordering rules

Within a stage, lower `priority` runs first (default `100`).
Service- and method-scoped middleware are merged with global
middleware and the whole set is re-sorted by priority. A typical
ordering:

| Stage         | Priority | Middleware         | Why                              |
| ------------- | -------- | ------------------ | -------------------------------- |
| pre-request   | 10       | Auth               | Attach token before transport    |
| pre-request   | 20       | CSRF               | Attach CSRF header (cookie mode) |
| pre-request   | 30       | Timing (start)     | Bracket the call                 |
| pre-request   | 50       | Tenant / context   | Other request attributes         |
| post-response | 30       | Logging            | Final log                        |
| post-response | 50       | Timing (record)    | Record latency                   |
| error         | 10       | Auth error handler | Refresh + retry on 401           |
| error         | 100      | Error transform    | Normalise the thrown error       |
| error         | 200      | Sentry             | Report, then re-throw            |

## Not middleware

Some capabilities that *look* like middleware are configured
elsewhere in this package — do not look for middleware factories
for them.

### Retry — an `HttpClient` option

Request retry is **not** middleware. It is a built-in
`HttpClient` option: set `retry: true` and `maxRetries`. Backoff
is exponential (`2^n × 1000ms`) and it applies to the **HTTP
transport only**.

```typescript
const client = new HttpClient({
  url: 'https://api.example.com',
  retry: true,
  maxRetries: 3,   // default 3 when retry is enabled
});
```

See [Browser client](/docs/frontend/netron/browser) for the full client options.

### Circuit-breaking & caching — the fluent HTTP interface

Circuit-breaking and response caching live in the **advanced
fluent HTTP interface** (`HttpRemotePeer` / `FluentInterface`,
the same surface as `@omnitron-dev/netron-http-core`), not in the
middleware pipeline:

- **Caching** — `HttpCacheManager`, `CacheOptions` (and the
  per-call `.cache({ ... })` fluent builder).
- **Circuit breaking** — `CircuitBreakerOptions`.
- **Fluent retry** — `RetryOptions` with an `attempts` field
  (e.g. `.retry({ attempts: 3 })`). Note this `attempts` field is
  the *fluent* retry option — it is unrelated to the
  `HttpClient` `retry`/`maxRetries` options above, and there is
  no retry middleware.

These are exported from the package root for advanced use
(`HttpCacheManager`, `RetryManager`, `FluentInterface`,
`CircuitBreakerOptions`, `CacheOptions`, `RetryOptions`). They are
documented with the fluent interface, not here.

## Best practices

- **Always `await next()`.** Fire-and-forget breaks ordering and
  loses the response/error.
- **Initialise `ctx.request.headers` before writing.** Both may be
  `undefined` (`ctx.request ??= {}; ctx.request.headers ??= {}`).
- **Re-throw in error middleware** unless you genuinely recover —
  swallowing the error hides failures.
- **Name your middleware.** `config.name` shows up in the
  pipeline's per-middleware metrics (`getMetrics()`).
- **Don't log payloads in production.** Keep `logRequestPayload`
  off, or use `skipServices` / `skipMethods` for sensitive calls.

## See also

- [Browser client](/docs/frontend/netron/browser) — `HttpClient` options including `retry` / `maxRetries`
- [Auth](/docs/frontend/netron/auth) — `AuthenticationClient`, token transports, cookie mode
- [Error handling](/docs/frontend/netron/errors) — error types and codes
- [Transports](/docs/frontend/netron/transports) — what middleware wraps

---

---
id: "frontend/netron/multi-backend"
title: "Multi-backend"
description: "One frontend, many Netron servers — routing, hooks, components."
---

# Multi-backend

When the app talks to several Netron backends — typical in a
fan-out architecture — build a client for all of them and hand
it to `MultiBackendProvider`.

Two things follow from the shape of that client and are worth
knowing before the examples. Backends live behind **one
`baseUrl`, separated by path** — a gateway with `/auth`,
`/media`, `/streams` under it, not a client per hostname. And the
provider does not build the client: it takes one you built, which
is why the same client can be used outside React (a worker, an
SSR pass, a test) without a second configuration.

## Setup

```tsx
import { createMultiBackendClient } from '@omnitron-dev/netron-browser';
import { MultiBackendProvider, useBackendService }
  from '@omnitron-dev/netron-react';

const client = createMultiBackendClient({
  baseUrl: 'https://api.example.com',
  backends: {
    auth:    { path: '/auth' },
    media:   { path: '/media' },
    streams: { path: '/streams', transport: 'websocket' },
    reports: { path: '/reports', transport: 'http' },
  },
  defaultBackend: 'auth',
  routing: {
    services: { objects: 'media', transforms: 'media' },
    patterns: [
      { pattern: 'users',    backend: 'auth' },
      { pattern: 'sessions', backend: 'auth' },
      { pattern: /^events\./, backend: 'streams' },
    ],
  },
});

function App() {
  return (
    <MultiBackendProvider client={client} autoConnect>
      <Outlet />
    </MultiBackendProvider>
  );
}
```

A string pattern matches by **prefix**, not as a glob — see
[Routing patterns](#routing-patterns). A call that matches
nothing goes to `defaultBackend`; it does not throw.

## Default backend

`defaultBackend` is where anything unrouted lands, and it is
required in practice — without it the first backend in the
config is used, which makes the fallback depend on key order.

```tsx
const client = createMultiBackendClient({
  baseUrl: '/api',
  backends: {
    main:  { path: '/main' },
    media: { path: '/media' },
  },
  defaultBackend: 'main',
  routing: { services: { objects: 'media' } },
});
```

Anything not matched routes to `defaultBackend`. Useful when
most services live on one backend and a few are extracted.

## Per-backend service hooks

```tsx
import { useBackendService } from '@omnitron-dev/netron-react';

function UserCard({ userId }: { userId: string }) {
  const users = useBackendService<UserService>('auth', 'users');
  const { data } = users.getUser.useQuery([userId]);
  return data ? <div>{data.email}</div> : null;
}

function MediaPreview({ id }: { id: string }) {
  const media = useBackendService<MediaService>('media', 'objects');
  const { data } = media.getMetadata.useQuery([id]);
  return data ? <img src={data.thumbnailUrl} /> : null;
}
```

`useBackendService(backend, service)` explicitly addresses a
backend; the router's pattern matching is bypassed.

## Routed `useService`

When you use the routed form, the router resolves the backend
from the service name:

```tsx
import { useService } from '@omnitron-dev/netron-react';

function UserCard({ userId }: { userId: string }) {
  // Router sees 'users' → matches the 'users' prefix → 'auth' backend
  const users = useService<UserService>('users');
  const { data } = users.getUser.useQuery([userId]);
}
```

The router-based form is recommended — components don't need to
know which backend a service lives on; the routing config is
the contract.

## Per-backend hooks

| Hook                                    | Purpose                               |
| --------------------------------------- | ------------------------------------- |
| `useBackend(name)`                      | Get the typed backend client by name  |
| `useBackendConnectionState(name)`       | Per-backend connection state          |
| `useBackendService(backend, service)`   | Typed service from a specific backend |
| `useBackendQuery(backend, query)`       | Query against a specific backend      |
| `useBackendMutation(backend, mutation)` | Mutation against a specific backend   |
| `useAllBackendsConnected()`             | True iff all backends connected       |
| `useAnyBackendConnected()`              | True iff at least one connected       |
| `useMultiBackendContext()`              | Full context value (advanced)         |
| `useMultiBackendConnectionState()`      | All backends' connection states       |

## Connection-aware rendering

```tsx
import {
  BackendConnectionAware,
  RequireBackendConnection,
  RequireAllBackends,
  RequireAnyBackend,
  BackendStatus,
  MultiBackendConnectionAware,
} from '@omnitron-dev/netron-react';

// Render only when this backend is connected:
<RequireBackendConnection backend="media" fallback={<MediaOffline />}>
  <UploadForm />
</RequireBackendConnection>

// Branch based on connection state:
<BackendConnectionAware backend="reports">
  {({ isConnected, isConnecting, error }) =>
    isConnected   ? <Reports /> :
    isConnecting  ? <Spinner /> :
                    <ErrorCard error={error} />
  }
</BackendConnectionAware>

// Require ALL listed backends:
<RequireAllBackends backends={['auth', 'media']}>
  <Dashboard />
</RequireAllBackends>

// Or ANY (failover scenarios):
<RequireAnyBackend backends={['auth-primary', 'auth-backup']}>
  <SignInButton />
</RequireAnyBackend>

// Status indicator:
<BackendStatus backend="streams" showLabel />

// All backends at once:
<MultiBackendConnectionAware>
  {(states) => (
    <Stack direction="row">
      {Object.entries(states).map(([name, s]) => (
        <Chip key={name} label={name} color={s.isConnected ? 'success' : 'error'} />
      ))}
    </Stack>
  )}
</MultiBackendConnectionAware>
```

## Per-backend auth

A backend's `auth` takes an `AuthenticationClient` you already
built, or `AuthOptions` for the client to build one:

```tsx
import { AuthenticationClient } from '@omnitron-dev/netron-browser';

const jwtAuth = new AuthenticationClient({ storage: 'local' });

const client = createMultiBackendClient({
  baseUrl: 'https://api.example.com',
  backends: {
    auth:   { path: '/auth',   auth: jwtAuth },
    public: { path: '/public' },          // omit `auth` — no credentials sent
  },
  defaultBackend: 'auth',
});
```

Omitting `auth` is how a backend goes unauthenticated; there is
no `auth: false`.

For most setups **one shared `AuthenticationClient`** across all
backends is right — the same JWT verifies everywhere in a
fan-out architecture, and passing the same instance is what makes
one refresh serve every backend.

## Health checks

`BackendPool` — the layer under the client — can poll its
backends: `enableHealthChecks` (off by default) and
`healthCheckInterval` (30s). What that buys is **observability,
not switching**.

Nothing inside the library reads the resulting flag:
`invoke(backend, service, method)` takes the backend name from
the caller, so there is no routing decision for a health result
to change. The only consumers are the public
`getHealthyBackends()` / `getUnhealthyBackends()` — useful for a
status indicator, and the place to build failover yourself.

So there is no `failover` map, no `onUnhealthy` policy and no
request queue. A call to a backend that is down fails like any
other call.

> **The same option means something else on the server**
>
> `@omnitron-dev/titan`'s multi-backend does route on health — it
> skips backends marked unhealthy and has a circuit breaker. One
> option name, two packages in the same monorepo, and the browser
> one is where it does not decide anything.

## Shared cache

The provider's query engine owns a single QueryCache shared
across every backend. Cache keys distinguish entries — encode
the backend into the key (e.g. `['auth', 'users', 'getUser', 'u_42']`)
when the same service name lives on more than one backend.

Cross-backend invalidation — the bridged client invalidates over
the shared cache:

```tsx
import { useNetronClient } from '@omnitron-dev/netron-react';

function CacheManager() {
  const client = useNetronClient();
  return (
    <Button onClick={() => client.invalidateQueries({ queryKey: ['users'] })}>
      Refresh users everywhere
    </Button>
  );
}
```

## Backend pool (no React)

For vanilla JS / web workers / SSR:

```typescript
import { createMultiBackendClient } from '@omnitron-dev/netron-browser';

// One gateway, several backends behind it — the model is a shared origin
// with a path per backend, not a client per host. That is what makes one
// auth client, one middleware chain and one set of shared options apply
// across all of them.
const client = createMultiBackendClient({
  baseUrl: 'https://api.example.com',
  backends: {
    auth:    { path: '/auth' },
    media:   { path: '/media' },
    streams: { path: '/streams', transport: 'websocket' },
  },
  defaultBackend: 'auth',
  routing: {
    // Explicit mappings win over patterns.
    services: { objects: 'media' },
    // A string pattern is a PREFIX, not a glob — 'users' matches
    // `users`, `users.admin`, `usersLegacy`. Use a RegExp when you need
    // more than that.
    patterns: [
      { pattern: 'users',        backend: 'auth' },
      { pattern: /^events\./,    backend: 'streams' },
    ],
  },
});

const users = client.service<UserService>('users');
const user = await users.getUser('u_42');   // routed to 'auth'
```

To reach a backend by name rather than by routing, `client.backend('media')`
returns its `BackendClient`. `BackendPool` — the layer underneath — is the
registry alone: `get(name)`, `connect(name)`, `connectAll()`, with no routing
of its own.

`MultiBackendProvider` takes this same client — the routing
lives in the client, the provider adds the React subscription
glue. That separation is why the client above works unchanged in
a worker or an SSR pass.

## Routing patterns

Not glob. A string pattern matches by **prefix** (or exactly);
anything more expressive is a `RegExp`. Patterns are tried in
array order and the first match wins — length plays no part.

| Pattern            | Matches                                          | Does not match |
| ------------------ | ------------------------------------------------ | -------------- |
| `'users'`          | `users`, `users.admin`, `usersLegacy`            | `adminUsers`   |
| `/^admin\./`       | `admin.settings`, `admin.audit`                  | `superadmin.x` |
| `'OmnitronDaemon'` | that name exactly, and anything starting with it | —              |

`'users.*'`, `'*.public'` and `'**'` are ordinary strings here:
`*` has no special meaning, so `'users.*'` matches only a service
literally named `users.*…`. For "everything else", set
`defaultBackend` — that is the fallback, and it applies whenever
no explicit mapping and no pattern matched.

Resolution order: `routing.services` (exact, wins outright) →
`routing.patterns` (in order) → `defaultBackend`.

There is no function form: `routing` takes `services` and
`patterns`, and a `RegExp` pattern is the escape hatch for
anything a prefix cannot express.

```typescript
routing: {
  patterns: [
    { pattern: /^Auth/,  backend: 'auth' },
    { pattern: /^Media/, backend: 'media' },
  ],
},
defaultBackend: 'default',
```

## Prism integration

Prism re-exports `MultiBackendProvider` with extra defaults
(theming, settings store hooks, snackbar host):

```tsx
import { createMultiBackendClient, MultiBackendProvider }
  from '@omnitron-dev/prism/netron';

const client = createMultiBackendClient({
  baseUrl:        '',
  backends: {
    main:    { path: '/api/main' },
    storage: { path: '/api/storage' },
  },
  defaultBackend: 'main',
});

<MultiBackendProvider client={client} autoConnect>
  <Outlet />
</MultiBackendProvider>
```

Prefer the Prism version for apps that already use Prism — it
hooks into the Prism context.

## Best practices

- **One `MultiBackendProvider` per app**, mounted at the root.
- **Routing config is the contract.** Components shouldn't
  know which backend they hit.
- **Match backend boundaries to logical concerns.** Don't split
  arbitrarily; split when scaling / ownership / lifecycle
  genuinely differs.
- **Shared `AuthenticationClient`** across backends in fan-out
  architectures.
- **`RequireBackendConnection`** for routes that need a specific
  backend up — fail-fast UX beats mystery loading state.
- **Handle a backend being down where you can act on it.** The
  client does not re-route away from an unhealthy backend, so a
  component that must survive one going away needs its own
  fallback.

## Anti-patterns

- **Per-route `MultiBackendProvider`.** Loses connection
  sharing and cache; re-connects on every route change.
- **Per-backend `AuthenticationClient`** in a fan-out architecture.
  Multiple instances, mismatched tokens, sign-out doesn't
  propagate.
- **A catch-all pattern.** There is no glob to write one with,
  and `defaultBackend` already is the fallback.
- **Hard-coding backend names in components.** Use `useService`
  (routed) so components stay transport-agnostic.

## See also

- [netron-react](/docs/frontend/netron/react) — single-backend equivalents
- [Auth manager](/docs/frontend/netron/auth) — shared auth across backends
- [Caching](/docs/frontend/netron/caching) — shared cache across backends
- [Transports](/docs/frontend/netron/transports) — transport per backend

---

---
id: "frontend/netron/react"
title: "netron-react"
description: "Production-grade React bindings for Netron RPC — type-safe hooks, multi-backend, auth."
---

# netron-react

`@omnitron-dev/netron-react` is the **optional React layer** on
top of [netron-browser](/docs/frontend/netron/browser). Provides React 18+ / 19
bindings: type-safe hooks for queries, mutations, subscriptions,
infinite queries; a query + mutation cache; multi-backend
support; an authentication system with route guards; and
devtools scaffolding.

> **Install only if your frontend uses React.** For Vue / Svelte
> / Solid / Angular / Lit / vanilla JS — use
> [`netron-browser`](/docs/frontend/netron/browser) directly and wrap calls in
> your framework's reactivity primitives.
>
> **Don't confuse with server-side Netron** at
> `@omnitron-dev/titan/netron` — that's part of the Titan
> framework. This package consumes the server through
> `netron-browser`.

In a React Server Component application, create the Netron client inside the
client component that renders the provider. The root, hooks, authentication and
React test entries preserve the client directive in the published package.
Server-side code can import `@omnitron-dev/netron-react/cache` directly.

Verified against `packages/netron-react/src/` (data layer at
parity across all three query hooks; no external data-fetching
dependency).

```bash
pnpm add @omnitron-dev/netron-react @omnitron-dev/netron-browser
```

`netron-browser` is a peer dependency — install both.

## Provider

```tsx
import { NetronReactClient, NetronProvider } from '@omnitron-dev/netron-react';

const client = new NetronReactClient({
  url:       'https://api.example.com',
  transport: 'http',                 // matches this HTTP RPC endpoint
  timeout:   30_000,
  cache: {
    maxEntries: 1000,
  },
  defaults: {                        // applied to every query unless overridden
    staleTime:            30_000,
    cacheTime:            5 * 60_000,
    retry:                3,         // number | RetryConfig
    refetchOnWindowFocus: true,
    refetchOnReconnect:   true,
  },
  devTools: process.env.NODE_ENV === 'development',
});

function App() {
  return (
    <NetronProvider client={client}>
      <Outlet />
    </NetronProvider>
  );
}
```

`new NetronReactClient(config)` and the `createNetronClient(config)`
factory return the same React client. Use `'websocket'` for an endpoint with
a Netron WebSocket listener. The React-only `'auto'` default initializes HTTP
on the server. In the browser it must complete the WebSocket handshake before
initializing HTTP. See [transport selection](/docs/frontend/netron/transports#react-transport-selection).

> Config field names are exact: it's `cacheTime` (not `gcTime`),
> and `retry` accepts a plain number **or** a `RetryConfig`
> object (`{ attempts, initialDelay?, maxDelay?, backoff?,
> retryCondition? }`) — there is no `maxAttempts`. `defaults` is a
> flat object, not `{ queryOptions: … }`.

## Hooks at a glance

| Hook                | Form                | Purpose                                                                    |
| ------------------- | ------------------- | -------------------------------------------------------------------------- |
| `useQuery`          | standalone or proxy | Cached data fetching from a service method                                 |
| `useMutation`       | standalone or proxy | Mutating call with invalidation + optimistic updates                       |
| `useInfiniteQuery`  | standalone          | Paginated / cursor-based queries                                           |
| `useQueries`        | standalone          | Parallel queries — many at once, optional `combine`                        |
| `useSubscription`   | standalone          | Live event subscription synced to React state                              |
| `useService`        | —                   | Typed service proxy with per-method `.call` / `.useQuery` / `.useMutation` |
| `createServiceHook` | —                   | `useService` curried with a service name                                   |

Plus context hooks:

| Hook                                      | Purpose                            |
| ----------------------------------------- | ---------------------------------- |
| `useNetronClient` / `useNetronClientSafe` | Access the raw client from context |
| `useNetronConnection`                     | Reactive connection state          |
| `useDefaults`                             | Read provider-level defaults       |
| `useHydration`                            | SSR hydration state                |

> **Proxy vs. standalone.** The `useService` proxy exposes
> `.call`, `.useQuery` and `.useMutation` per method.
> `useInfiniteQuery`, `useQueries` and `useSubscription` are
> **standalone hooks** — they are not methods on the proxy. Call
> them directly and build the `queryKey` / `queryFn` (or `event`)
> yourself.

## `useQuery` — the workhorse

Through the typed proxy (recommended — pass a positional args
array, options omit `queryKey`/`queryFn` because the proxy
supplies them):

```tsx
import { useService } from '@omnitron-dev/netron-react';

interface UserService {
  getUser(id: string): Promise<User>;
  list(filter: UserFilter): Promise<User[]>;
}

function UserCard({ userId }: { userId: string }) {
  const users = useService<UserService>('users');
  const { data, isLoading, error, refetch } = users.getUser.useQuery([userId]);

  if (isLoading) return <Skeleton />;
  if (error)     return <ErrorCard error={error} onRetry={refetch} />;
  return <div>{data!.email}</div>;
}
```

Standalone form — supply `queryKey` + `queryFn` yourself:

```tsx
import { useQuery, useNetronClient } from '@omnitron-dev/netron-react';

const client = useNetronClient();
const { data } = useQuery<User>({
  queryKey: ['users', 'getUser', userId],
  queryFn: ({ signal }) => client.invoke('users', 'getUser', [userId], { signal }),
});
```

What you get back (`QueryResult<T>`):

```typescript
interface QueryResult<TData, TError = NetronError> {
  data:           TData | undefined;
  error:          TError | null;
  status:         'idle' | 'loading' | 'success' | 'error';
  isLoading:      boolean;
  isError:        boolean;
  isSuccess:      boolean;
  isIdle:         boolean;
  isFetching:     boolean;   // true also when refetching cached data
  isRefetching:   boolean;   // fetching while not in the initial load
  isStale:        boolean;
  isPreviousData: boolean;   // true while keepPreviousData carries a prior key over
  dataUpdatedAt:  number;
  errorUpdatedAt: number;
  refetch:        () => Promise<QueryResult<TData, TError>>;
  remove:         () => void; // drop this query from the cache
}
```

### Query options

```tsx
users.getUser.useQuery([userId], {
  enabled:              userId != null,
  staleTime:            30_000,
  cacheTime:            5 * 60_000,
  refetchOnWindowFocus: true,
  refetchOnReconnect:   true,
  refetchInterval:      60_000,          // number | false
  retry:                3,               // number | boolean | RetryConfig
  retryDelay:           (attempt) => 2 ** attempt * 1000,
  select:               (user) => user.email,   // project/transform
  placeholderData:      previousData,
  initialData:          () => seedUser,
  keepPreviousData:     true,            // see below
  suspense:             false,           // see "Suspense + error boundaries"
  useErrorBoundary:     false,           // see "Suspense + error boundaries"
  onSuccess:            (user) => track('user.loaded'),
  onError:              (err)  => report(err),
  onSettled:            (user, err) => {},
});
```

`RetryConfig` (when `retry` is an object):

```typescript
interface RetryConfig {
  attempts:        number;
  initialDelay?:   number;
  maxDelay?:       number;
  backoff?:        'exponential' | 'linear' | 'constant';
  retryCondition?: (error: Error, attempt: number) => boolean;
}
```

### Cache key

In the proxy form the key is `[service, method, ...args]`. Two
components calling the same query share data; one invalidation
refreshes both. In the standalone form you provide the `queryKey`
directly.

### `keepPreviousData`

When the `queryKey` changes (paging, filtering, switching
selection), keep showing the previous key's data while the new
one loads instead of blanking to the loading state. The
carried-over render is flagged with `isPreviousData: true`:

```tsx
function UserPager({ id }: { id: string }) {
  const users = useService<UserService>('users');
  const { data, isPreviousData, isFetching } =
    users.getUser.useQuery([id], { keepPreviousData: true });

  return (
    <div style={{ opacity: isPreviousData ? 0.5 : 1 }}>
      {data?.email}
      {isFetching && <Spinner />}
    </div>
  );
}
```

`isPreviousData` returns to `false` once the new key resolves.
Supported on `useQuery`, `useInfiniteQuery` and per-query in
`useQueries`.

## `useMutation` — for writes

Through the proxy (the `mutationFn` is supplied automatically;
`mutate`'s argument is the method's argument):

```tsx
function InviteForm() {
  const users  = useService<UserService>('users');
  const invite = users.invite.useMutation({
    onSuccess:         (newUser) => toast.success(`Invited ${newUser.email}`),
    invalidateQueries: [['users', 'list']],   // exact option name
  });

  return (
    <form onSubmit={(e) => {
      e.preventDefault();
      invite.mutate(new FormData(e.currentTarget).get('email') as string);
    }}>
      <input name="email" />
      <Button disabled={invite.isLoading}>Invite</Button>
    </form>
  );
}
```

`MutationResult` fields:

| Field                              | Purpose                                       |
| ---------------------------------- | --------------------------------------------- |
| `mutate(variables)`                | Fire-and-forget mutation                      |
| `mutateAsync(variables)`           | Returns a `Promise` of the result             |
| `isLoading`                        | Currently running (**not** `isPending`)       |
| `isIdle` / `isSuccess` / `isError` | Status flags                                  |
| `status`                           | `'idle' \| 'loading' \| 'success' \| 'error'` |
| `data` / `error` / `variables`     | Result, error, last variables                 |
| `context`                          | Value returned from `onMutate`                |
| `reset()`                          | Clear status back to idle                     |

Options: `mutationFn` (standalone only), `mutationKey`, `onMutate`,
`onSuccess(data, variables, context)`, `onError(error, variables,
context)`, `onSettled(data, error, variables, context)`, `retry`,
`retryDelay`, and `invalidateQueries: QueryKey[]` (invalidated on
success).

### Optimistic updates

The cache is reached through `client.getQueryCache()` for direct
reads/writes (`get` / `set`), and the client itself exposes the
`*Queries` helpers (`cancelQueries`, `invalidateQueries`):

```tsx
const client = useNetronClient();
const cache  = client.getQueryCache();
const key    = ['users', 'getUser', userId];

const updateProfile = users.updateProfile.useMutation({
  onMutate: async (newProfile) => {
    client.cancelQueries({ queryKey: key });   // QueryFilters, not a bare key
    const previous = cache.get<User>(key);
    if (previous) cache.set(key, { ...previous, ...newProfile });
    return { previous };
  },
  onError: (_err, _newProfile, context) => {
    if (context?.previous) cache.set(key, context.previous); // rollback
  },
  onSettled: () => {
    client.invalidateQueries({ queryKey: key });
  },
});
```

## `useSubscription` — live data

`useSubscription` is a **standalone, event-based** hook (it is not
a method on the service proxy). Subscribe to a named server event
delivered over WebSocket:

```tsx
import { useSubscription } from '@omnitron-dev/netron-react';

function LiveOrders() {
  const { data, history, isConnected, isSubscribed, error } =
    useSubscription<Order>({
      event:  'orders.updated',
      filter: (o) => o.tier === 'pro',
      onData: (o) => console.log('new order', o.id),
    });

  return (
    <div>
      <Status connected={isConnected} />
      <OrderStream items={history} latest={data} />
    </div>
  );
}
```

`SubscriptionResult`: `data` (latest item), `history` (received
items, capped at 100), `isConnected`, `isSubscribed`, `error`,
`unsubscribe()`, `resubscribe()`, `clearHistory()`. Options:
`event` (required), `filter`, `transform`, `buffer` (`{ size,
timeout, strategy }`), `enabled`, `onData`, `onError`, `onConnect`,
`onDisconnect`. The React subscription manager re-registers events after the WebSocket reconnects.
Recover missed events from the service's cursor or a fresh snapshot.

## `useInfiniteQuery` — pagination

Standalone hook. Supply `queryKey`, a `queryFn` that receives the
typed `pageParam`, `getNextPageParam`, and `initialPageParam`:

```tsx
import { useInfiniteQuery, useNetronClient } from '@omnitron-dev/netron-react';

const client = useNetronClient();
const {
  data,
  fetchNextPage,
  hasNextPage,
  isFetchingNextPage,
  isPreviousData,
} = useInfiniteQuery<Page, NetronError, string | null>({
  queryKey:         ['feed', category],
  queryFn:          ({ pageParam, signal }) =>
                      client.invoke('feed', 'page', [category, pageParam], { signal }),
  initialPageParam: null,
  getNextPageParam: (lastPage) => lastPage.nextCursor ?? undefined,
  maxPages:         10,            // optional cap; oldest pages drop out
  keepPreviousData: true,          // keep prior category's pages while switching
});

return (
  <>
    {data?.pages.flatMap(p => p.items).map(item => <Row key={item.id} {...item} />)}
    {hasNextPage && (
      <Button onClick={() => fetchNextPage()} disabled={isFetchingNextPage}>
        Load more
      </Button>
    )}
  </>
);
```

`InfiniteQueryResult`: `data` (`{ pages, pageParams }`),
`fetchNextPage()` / `fetchPreviousPage()`, `hasNextPage` /
`hasPreviousPage`, `isFetching` / `isFetchingNextPage` /
`isFetchingPreviousPage`, `status`, `isLoading` / `isError` /
`isSuccess`, `isPreviousData`, `refetch()`. Options also accept
`getPreviousPageParam`, `staleTime`, `cacheTime`, `enabled`,
`retry`, `onSuccess(data)` / `onError`, plus `suspense`,
`useErrorBoundary` and `keepPreviousData` (same semantics as
`useQuery`).

> The infinite-query types (`InfiniteQueryOptions`,
> `InfiniteQueryResult`, `InfiniteData`,
> `InfiniteQueryFunctionContext`) are exported from the package
> root.

## `useQueries` — parallel reads

Standalone hook. Each entry is a **full query config** (`queryKey`

- `queryFn`, plus any per-query options) — not a
  `{ service, method, args }` shape. An optional `combine` reduces
  the results into a custom value:

```tsx
import { useQueries, useNetronClient } from '@omnitron-dev/netron-react';

const client  = useNetronClient();
const results = useQueries({
  queries: userIds.map(id => ({
    queryKey:         ['users', 'getUser', id],
    queryFn:          ({ signal }) => client.invoke('users', 'getUser', [id], { signal }),
    keepPreviousData: true,
  })),
});

const users = results.map(r => r.data).filter(Boolean);
```

With `combine`:

```tsx
const { users, posts } = useQueries({
  queries: [
    { queryKey: ['users', 'list'], queryFn: () => client.invoke('users', 'list', []) },
    { queryKey: ['posts', 'list'], queryFn: () => client.invoke('posts', 'list', []) },
  ],
  combine: ([u, p]) => ({ users: u.data ?? [], posts: p.data ?? [] }),
});
```

Each entry is a `QueryObserverResult<T>` (the same shape as
`QueryResult`, including `isPreviousData`); all queries fire in
parallel. `suspense`, `useErrorBoundary` and `keepPreviousData`
are honoured **per query**.

## Suspense + error boundaries

There is **no** `useSuspenseQuery` hook. Suspense is an **option**
on the existing hooks — `suspense: true` makes the hook throw the
in-flight fetch so the nearest `<Suspense>` boundary shows its
fallback until data resolves:

```tsx
function UserName({ id }: { id: string }) {
  const users = useService<UserService>('users');
  // With suspense, data is guaranteed defined past this line.
  const { data } = users.getUser.useQuery([id], { suspense: true });
  return <span>{data!.name}</span>;
}

<Suspense fallback={<Skeleton />}>
  <UserName id={id} />
</Suspense>
```

`useErrorBoundary` re-throws a query error to the nearest React
error boundary instead of returning it in `error`. It takes a
boolean or a predicate `(error) => boolean` for selective
escalation:

```tsx
useQuery({
  queryKey: ['report', id],
  queryFn:  () => client.invoke('reports', 'get', [id]),
  useErrorBoundary: (err) => err.status >= 500,   // escalate only server errors
});
```

Both `suspense` and `useErrorBoundary` are supported on `useQuery`,
`useInfiniteQuery` and per-query in `useQueries`. (When `suspense`
is on, query errors escalate to the error boundary automatically.)

## `useService` — typed proxy

The recommended entry point. Returns a proxy where every method
exposes `.call`, `.useQuery` and `.useMutation`:

```tsx
const users = useService<UserService>('users');

await users.getUser.call(id);                 // direct promise
users.getUser.useQuery([id]);                 // cached query hook
users.invite.useMutation({ onSuccess: ... }); // mutation hook
```

Types flow from the imported interface — no schema sync,
no codegen, no shape drift. (`useInfiniteQuery`, `useQueries` and
`useSubscription` are standalone — see their sections.)

### Custom service hook

```tsx
import { createServiceHook } from '@omnitron-dev/netron-react';

export const useUsers = createServiceHook<UserService>('users');

function Component() {
  const users = useUsers();
  const { data } = users.getUser.useQuery([id]);
}
```

`createServiceHook(name, defaultOptions?)` is `useService` curried
with a service name — useful when many components hit the same
service.

## Authentication

`@omnitron-dev/netron-react/auth` provides an auth context + route
guards. Configure the provider with a `config` block and an
`onLogin` handler that performs the actual sign-in (e.g. by running
your Netron auth core-task) and returns an `AuthResult`:

```tsx
import { AuthProvider, AuthGuard, GuestGuard, useAuth }
  from '@omnitron-dev/netron-react/auth';

function App() {
  return (
    <NetronProvider client={client}>
      <AuthProvider
        config={{
          refreshEndpoint: 'OmnitronAuth.refreshSession',
          logoutEndpoint:  'OmnitronAuth.signOut',
          storage:         'local',           // 'local' | 'session' | 'memory'
          autoRefresh:     true,
          refreshThreshold: 5 * 60_000,
        }}
        onLogin={async (credentials) =>
          client.invoke('OmnitronAuth', 'signIn', [credentials])
        }
        onAuthenticated={(user) => track('auth.login', { id: user.id })}
      >
        <Routes />
      </AuthProvider>
    </NetronProvider>
  );
}

// In components — note login/logout (not signIn/signOut):
function UserMenu() {
  const { user, isAuthenticated, login, logout } = useAuth();
  if (!isAuthenticated) return <Button onClick={() => login({ /* creds */ })}>Sign in</Button>;
  return (
    <Menu>
      <Avatar src={user!.avatarUrl} />
      <MenuItem onClick={() => logout()}>Sign out</MenuItem>
    </Menu>
  );
}
```

`useAuth()` returns `{ isAuthenticated, user, login, logout,
refresh, getAuthHeaders, hasRole, hasPermission, hasAnyRole,
hasAllRoles }`. (`useUser`, `useIsAuthenticated` and
`useAuthRequired` are convenience selectors.)

### Guards

```tsx
// Render children only when authenticated; otherwise show `fallback`
// (and optionally redirect via `redirectTo`):
<AuthGuard fallback={<SignInPrompt />} redirectTo="/sign-in">
  <DashboardLayout />
</AuthGuard>

// Inverse — only for unauthenticated visitors:
<GuestGuard redirectTo="/">
  <SignInPage />
</GuestGuard>

// Role / permission gates:
<RoleGuard role="admin" fallback={<NotAllowed />}>
  <AdminPanel />
</RoleGuard>
<PermissionGuard permission="billing.write">
  <BillingControls />
</PermissionGuard>

// Lightweight conditional rendering:
<Show when={isAuthenticated}><AccountMenu /></Show>
<Hide when={isAuthenticated}><SignInButton /></Hide>
```

Exported guards: `AuthGuard`, `GuestGuard`, `RoleGuard`,
`PermissionGuard`, `Show`, `Hide`.

## Multi-backend support

When the app talks to several Netron servers — e.g., one for
identity, one for media, one for analytics — use
`createMultiBackendClient` (from **netron-browser**) +
`MultiBackendProvider`:

```tsx
import { createMultiBackendClient }
  from '@omnitron-dev/netron-browser';
import { MultiBackendProvider, useBackendService }
  from '@omnitron-dev/netron-react';

// `backends` keys → path prefixes under `baseUrl`; `routing`
// maps service-name patterns to a specific backend.
const client = createMultiBackendClient({
  baseUrl: 'https://api.example.com',
  backends: {
    auth:      { path: '/auth',      transport: 'http' },
    media:     { path: '/media',     transport: 'http' },
    analytics: { path: '/analytics', transport: 'http' },
  },
  routing: {
    patterns: [
      { pattern: 'users.',   backend: 'auth' },
      { pattern: 'objects.', backend: 'media' },
      { pattern: 'reports.', backend: 'analytics' },
    ],
  },
});

function App() {
  return (
    <MultiBackendProvider client={client}>
      <Routes />
    </MultiBackendProvider>
  );
}

// Use a specific backend:
function Component() {
  const users = useBackendService<UserService>('auth', 'users');
  const { data } = users.getUser.useQuery([id]);
}
```

`MultiBackendClientOptions` also accepts `defaultBackend` and a
`shared` block (timeout / headers / auth applied to all backends);
`routing` supports `patterns` and explicit `services` maps.

### Backend-aware hooks

| Hook                                    | Purpose                               |
| --------------------------------------- | ------------------------------------- |
| `useBackend(name)`                      | Get the typed backend client by name  |
| `useBackendConnectionState(name)`       | Per-backend connection state          |
| `useBackendService(backend, service)`   | Typed service from a specific backend |
| `useBackendQuery(backend, query)`       | Query against a specific backend      |
| `useBackendMutation(backend, mutation)` | Mutation against a specific backend   |
| `useAllBackendsConnected()`             | True iff all backends connected       |
| `useAnyBackendConnected()`              | True iff at least one connected       |

### Backend-aware components

```tsx
// Only render children when this backend is connected:
<RequireBackendConnection backend="media">
  <UploadForm />
</RequireBackendConnection>

// Branch based on connection:
<BackendConnectionAware backend="analytics">
  {({ isConnected, isConnecting, error }) =>
    isConnected ? <Reports /> :
    isConnecting ? <Spinner /> :
    <ErrorCard error={error} />
  }
</BackendConnectionAware>

// Render only when ALL listed backends are connected:
<RequireAllBackends backends={['auth', 'media']}>
  <Dashboard />
</RequireAllBackends>

// Or ANY:
<RequireAnyBackend backends={['auth-primary', 'auth-backup']}>
  <SignInButton />
</RequireAnyBackend>

// Status indicator:
<BackendStatus backend="analytics" />
```

## Connection-aware rendering (single backend)

```tsx
import { ConnectionAware, RequireConnection } from '@omnitron-dev/netron-react';

<ConnectionAware>
  {({ isConnected, isConnecting, error, reconnect }) =>
    isConnected ? <App /> :
    isConnecting ? <Spinner /> :
    <Disconnected error={error} onReconnect={reconnect} />
  }
</ConnectionAware>

<RequireConnection fallback={<OfflineBanner />}>
  <SensitivePage />
</RequireConnection>
```

## Cache management

The **client** exposes the broad-stroke cache operations; the
`QueryCache` (via `client.getQueryCache()`) exposes per-key
reads/writes.

```tsx
import { useNetronClient } from '@omnitron-dev/netron-react';

function AdminControls() {
  const client = useNetronClient();

  return (
    <>
      <Button onClick={() => client.invalidateQueries({ queryKey: ['users'] })}>
        Refresh users
      </Button>
      <Button onClick={() => client.removeQueries({ queryKey: ['users'] })}>
        Clear users cache
      </Button>
      <Button onClick={() => client.clear()}>
        Clear ALL cache
      </Button>
    </>
  );
}
```

Client methods:

| Method                                             | Effect                                                      |
| -------------------------------------------------- | ----------------------------------------------------------- |
| `invalidateQueries(filters?)`                      | Mark matching queries stale; subscribed components refetch  |
| `removeQueries(filters?)`                          | Drop matching queries from cache                            |
| `cancelQueries(filters?)`                          | Abort in-flight matching queries                            |
| `prefetchQuery(queryKey, fetcher, { staleTime? })` | Warm the cache without rendering (dedups against in-flight) |
| `clear()`                                          | Drop everything                                             |
| `getQueryCache()` / `getMutationCache()`           | The underlying caches                                       |
| `getQueryState(queryKey)`                          | Read a query's full state                                   |
| `dehydrate()` / `hydrate(state)`                   | SSR (see below)                                             |

`QueryCache` methods (for direct reads/writes in `onMutate`):

| Method                                      | Effect                               |
| ------------------------------------------- | ------------------------------------ |
| `get(queryKey)`                             | Read cached data without subscribing |
| `set(queryKey, data, staleTime?)`           | Write to cache                       |
| `invalidate(queryKey)`                      | Mark one key stale                   |
| `remove(queryKey)`                          | Drop one key                         |
| `findAll(filters?)`                         | List matching queries                |
| `cancelAll(filters?)`                       | Abort matching in-flight queries     |
| `subscribe(queryKey, observer, cacheTime?)` | Low-level change subscription        |

Filters are a `QueryFilters` object — `{ queryKey?, exact?,
status?, stale?, fetching?, predicate? }` — **not** a bare key
array. Partial key matching is the default:

```tsx
client.invalidateQueries({ queryKey: ['users'] });                       // every users query
client.invalidateQueries({ queryKey: ['users', 'getUser'] });            // every getUser
client.invalidateQueries({ queryKey: ['users', 'getUser', id], exact: true }); // one
client.invalidateQueries({ stale: true });                               // every stale query
```

## SSR support

```tsx
// On the server, after running your queries:
const dehydratedState = client.dehydrate();   // synchronous; returns DehydratedState

// Serialize `dehydratedState` into the HTML, then on the client:
const client = createNetronClient({ url });
client.hydrate(dehydratedState);               // a method call, not a config field
```

Set `ssr: { enabled: true, dehydrateTimeout? }` on the client
config for SSR mode. `useHydration()` exposes the hydration flag
for components that must avoid a hydration mismatch on first
render.

## State management

`netron-react` does **not** ship a general-purpose state library.
(An earlier unexported `state/` atom module was removed — there is
no `@omnitron-dev/netron-react/state` subpath.) For local UI state
use React's own primitives; for app-level stores prefer Prism's
`createPrismStore` / `createPersistedStore` from
`@omnitron-dev/prism/state` (version-aware persistence). Server state
belongs in the query cache, not a
separate store.

## Devtools

```tsx
import { NetronDevtools } from '@omnitron-dev/netron-react/devtools';
```

> **Scaffolded, not yet rendering.** `NetronDevtools` is currently
> a placeholder that renders `null` (full panel implementation is
> pending). The `/devtools` subpath also exports the logging
> helpers (`setLogger`, `getLogger`, `setLogLevel`, `log`) and the
> `DevToolsConfig` type. Don't rely on a visual devtools panel
> yet.

## Testing

`@omnitron-dev/netron-react/test` ships an in-memory test
provider + mock helpers (no real WebSocket / fetch):

```tsx
import { TestNetronProvider } from '@omnitron-dev/netron-react/test';

render(
  <TestNetronProvider
    testConfig={{
      mocks: [
        { service: 'users', method: 'getUser', response: { id: '1', email: 'a@b.c' } },
        { service: 'users', method: 'deleteUser', error: new Error('forbidden') },
      ],
      defaultDelay: 0,
    }}
  >
    <UserCard userId="1" />
  </TestNetronProvider>
);

await screen.findByText('a@b.c');
```

`TestNetronProvider` takes `testConfig` (from which it builds a
client) or a `client` you built yourself with
`createTestClient` — there is no `services` prop. Mocks are
matched on the `service.method` pair and answer with a **static**
`response` or a thrown `error`; `delay` is per-mock,
`defaultDelay` the fallback. There is no per-call function form,
so a mock whose answer must depend on the arguments belongs in a
hand-built client rather than here.

`createMockService<T>(implementations)` is a separate tool and
takes **one** argument — the object of method implementations,
which may be dynamic. It returns a plain stub object to inject
where a service instance is expected; it does not register
anything with the provider.

For hooks, `createWrapper(client?, testConfig?)` returns a
`renderHook` wrapper around the same provider.

Also exported: `createTestClient`, the query helpers
`waitForQuery` / `createQueryKey`, the assertion helpers
`expectLoading` / `expectSuccess` / `expectError`, and timer
utilities `advanceTimersAndFlush` / `nextTick`.

## Subpaths

The package exposes exactly these entry points:

| Subpath                               | Contents                                                                                                      |
| ------------------------------------- | ------------------------------------------------------------------------------------------------------------- |
| `@omnitron-dev/netron-react`          | Root — client, provider, all hooks, cache classes, multi-backend, context hooks                               |
| `@omnitron-dev/netron-react/hooks`    | The hooks in isolation (`useQuery`, `useMutation`, …)                                                         |
| `@omnitron-dev/netron-react/cache`    | `QueryCache`, `MutationCache`, `SubscriptionManager`, key utilities                                           |
| `@omnitron-dev/netron-react/auth`     | `AuthProvider`, `useAuth`, guards (`AuthGuard`, `GuestGuard`, `RoleGuard`, `PermissionGuard`, `Show`, `Hide`) |
| `@omnitron-dev/netron-react/devtools` | `NetronDevtools` (placeholder) + logging helpers                                                              |
| `@omnitron-dev/netron-react/test`     | `TestNetronProvider`, `createMockService`, test utilities                                                     |

`NetronReactClient`, `MultiBackendProvider`, the multi-backend
hooks and `useService` are all on the **root** entry — there is no
`/core`, `/service`, `/multi-backend` or `/state` subpath.

## Best practices

- **Prefer `useService` over raw `useQuery`.** Types flow from
  the interface; refactors are safe.
- **One client per backend.** Treat as long-lived singletons —
  recreating on render destroys cache and connection.
- **Invalidate with the broadest sensible filter.** Mutations that
  affect multiple views should `invalidateQueries({ queryKey })`
  at the broadest shared prefix.
- **Use `select` to extract slices.** Reduces re-renders when
  only part of `data` changes.
- **`keepPreviousData`** for paging/filtering so the UI doesn't
  flash empty between keys.
- **Optimistic updates** for fast feedback; roll back in
  `onError`; invalidate in `onSettled`.
- **`enabled: false`** for conditional queries — better than an
  early return from the component (preserves cache + hook order).

## Anti-patterns

- **Calling `useQuery` inside loops/conditionals.** React hooks
  rules — use `useQueries` for a dynamic set.
- **Storing query results in component state.** Defeats cache
  sharing; read from the cache instead.
- **Reaching for a `useSuspenseQuery` hook.** It doesn't exist —
  pass `suspense: true` to `useQuery`.
- **Passing a bare key array to `invalidateQueries`.** It takes a
  `QueryFilters` object: `{ queryKey: [...] }`.
- **Custom retry on top of provider retry.** Doubled retries
  amplify failure load.
- **Mounting `<MultiBackendProvider>` per route.** It's app-level
  state — mount once at the root.
- **Polling instead of `useSubscription`** when the server emits
  the event. Subscriptions are cheaper and more responsive.

## See also

- [netron-browser](/docs/frontend/netron/browser) — the underlying transport
- [Caching](/docs/frontend/netron/caching) — cache internals + invalidation
- [SSR](/docs/frontend/netron/ssr) — server rendering + hydration
- [Testing](/docs/frontend/netron/testing) — the test provider + mocks
- [Multi-backend](/docs/frontend/netron/multi-backend) — multi-server routing
- [Prism](/docs/frontend/prism) — UI components that pair with these hooks
- [Frontend overview](/docs/frontend/overview) — the three-package picture

---

---
id: "frontend/netron/ssr"
title: "SSR"
description: "Server-side rendering — dehydrate / hydrate, hydration safety."
---

# SSR

netron-react supports server-side rendering by serialising the
QueryCache on the server and rehydrating on the client.

## Pattern — Next.js / Remix style

```tsx
// server-side route loader:
export async function loader(req) {
  const client = new NetronReactClient({
    url:       process.env.API_URL,
    transport: 'http',
    ssr:       { enabled: true },
  });
  await client.connect();

  // Prefetch everything the page needs:
  const users    = client.service<UserService>('users');
  const projects = client.service<ProjectService>('projects');
  await client.prefetchQuery(['users', 'list', { filter: 'active' }], () => users.list.call({ filter: 'active' }));
  await client.prefetchQuery(['projects', 'getMine'], () => projects.getMine.call());

  const dehydratedState = client.dehydrate();

  return { dehydratedState };
}

// client-side root:
function App({ dehydratedState }: { dehydratedState: DehydratedState }) {
  const [client] = useState(() => {
    const c = new NetronReactClient({
      url: new URL('/api', window.location.origin).href,
      transport: 'http',
    });
    c.hydrate(dehydratedState);    // populate cache from server
    return c;
  });

  return (
    <NetronProvider client={client}>
      <Outlet />
    </NetronProvider>
  );
}
```

The server's `dehydrate()` returns a plain-object snapshot of
the cache (synchronously — no `await`). The client's `hydrate()`
method populates the fresh client's cache from that snapshot.

## Hydration semantics

After hydration:

- Cached queries from the server **don't** refetch on mount
  (until they go stale per `staleTime`).
- Subscriptions don't carry over — they re-subscribe on the
  client side.
- Mutation state isn't dehydrated (it's request-local).

```tsx
useQuery(['users','list', { filter: 'active' }], {
  staleTime: 30_000,
  // Server-fetched + dehydrated; client mount won't refetch unless > 30s stale.
});
```

## `useHydration`

When client-side rendering needs to **branch** on whether
hydration has completed:

```tsx
import { useHydration } from '@omnitron-dev/netron-react';

function ColorMode() {
  const isHydrated = useHydration();

  // Avoid a hydration mismatch by deferring browser-only state entirely.
  // Reading it above this line would run it on the server too, where
  // `window` does not exist.
  if (!isHydrated) return null;

  const dark = window.matchMedia('(prefers-color-scheme: dark)').matches;
  return <Icon name={dark ? 'sun' : 'moon'} />;
}
```

Useful for any UI that depends on browser-only state (localStorage,
`window.matchMedia`, etc.) — render `null` until hydrated to
avoid SSR/client divergence.

## Streaming SSR

```tsx
// Next.js 13+ / React 19 server components:
import { renderToReadableStream } from 'react-dom/server';

const stream = await renderToReadableStream(<App />, {
  onError: console.error,
});
```

For streaming, prefer `<Suspense>` boundaries around data-bound
sections; pass `{ suspense: true }` to `useQuery` and it
integrates cleanly.

## Per-user dehydration

Use a request-scoped cache for authenticated data. For an upstream service
that accepts bearer credentials, put the token on the HTTP client that makes
the call. Keep the upstream base URL in trusted server configuration.

```typescript
import { HttpClient } from '@omnitron-dev/netron-browser';
import { NetronReactClient } from '@omnitron-dev/netron-react';

async function prefetchWorkspace(apiUrl: string, accessToken: string) {
  const rpc = new HttpClient({
    url: apiUrl,
    redirect: 'error',
    headers: { Authorization: `Bearer ${accessToken}` },
  });
  const client = new NetronReactClient({ url: apiUrl, transport: 'http' });
  await rpc.connect();
  try {
    await client.prefetchQuery(['workspace', 'current'], () =>
      rpc.invoke('Workspace', 'current', []),
    );
    return client.dehydrate();
  } finally {
    await rpc.disconnect();
  }
}
```

Only include data the requesting user may receive in the page.

## Excluding queries from dehydration

`dehydrate()` takes no arguments — it automatically dehydrates
only **successful** queries. Errored / in-flight queries are
skipped so the client retries fresh, and mutation state is
request-local. To keep sensitive entries out of the snapshot,
remove them from the cache before dehydrating:

```typescript
// Drop sensitive queries before serialising:
client.removeQueries({ queryKey: ['admin'] });
client.removeQueries({ queryKey: ['tokens'] });

const dehydratedState = client.dehydrate();
```

## Bundle size considerations

The browser and SSR use the same package entry point. Selecting
`transport: 'http'` runs the HTTP path and avoids opening a socket on the server.
Bundle size depends on the imports and bundler output.

## Caveats

- **Subscriptions don't persist.** Server-side they aren't
  initiated; client takes over.
- **Optimistic mutations** in flight at dehydration time are
  lost — let them complete first.
- **Time-sensitive data** (clocks, "X minutes ago") needs
  `suppressHydrationWarning` on the wrapper element or
  `useHydration`-gated rendering.

## Best practices

- **Prefetch above-the-fold data.** Don't dehydrate the whole
  app — pick the queries that fill the initial render.
- **Per-user clients.** Don't reuse a server-side client
  across requests.
- **`staleTime > 0` for dehydrated queries.** Otherwise the
  client refetches immediately, defeating SSR.
- **`useHydration` for browser-only UI.** Theme toggle, geolocation,
  feature flags — gate them.

## Anti-patterns

- **Sharing a dehydrated cache across users.** Leaks data.
- **Streaming WebSocket on the server.** WS doesn't work
  pre-hydration; the cache rehydrates, but live subscriptions
  start on the client.
- **`client.hydrate()` without a matching server prefetch.** The
  cache is empty; you've added complexity for nothing.

## See also

- [Caching](/docs/frontend/netron/caching) — `dehydrate` / `prefetchQuery`
- [netron-react](/docs/frontend/netron/react) — `useQuery` SSR semantics
- [Transports](/docs/frontend/netron/transports) — HTTP for edge runtimes

---

---
id: "frontend/netron/testing"
title: "Testing"
description: "TestNetronProvider, integration patterns, deterministic fixtures."
---

# Testing

netron-react ships testing utilities (under
`@omnitron-dev/netron-react/test`) that let you drive components
without a real backend.

## `TestNetronProvider`

```tsx
import { TestNetronProvider } from '@omnitron-dev/netron-react/test';

render(
  <TestNetronProvider
    testConfig={{
      mocks: [
        { service: 'users', method: 'getUser', response: { id: '1', email: 'a@b.c' } },
        { service: 'users', method: 'list',    response: [{ id: '1', email: 'a@b.c' }] },
      ],
    }}
  >
    <UserCard userId="1" />
  </TestNetronProvider>
);

await screen.findByText('a@b.c');
```

No real transport — `TestNetronProvider` builds a test client
(via `createTestClient`) whose `invoke` is intercepted by the
`mocks` array. Hooks behave identically (`useQuery`,
`useMutation`, `useService`).

Each mock entry is a `MockResponse`: `{ service, method, response?,
error?, delay? }`. Provide `error` to make the matching call
reject, and `delay` to simulate latency.

## Custom mock responses

Each entry in `mocks` returns a **static** `response` for one
`service.method`. To branch on arguments, override the client's
`invoke` — it's a plain method. `createMockService` builds an
object of async method implementations you can dispatch to:

```tsx
import { createTestClient, createMockService, TestNetronProvider }
  from '@omnitron-dev/netron-react/test';

const usersMock = createMockService<UserService>({
  getUser: async (id: string) => {
    if (id === 'missing') {
      throw new TitanError({ code: ErrorCode.NOT_FOUND, message: 'not found' });
    }
    return { id, email: `${id}@example.com` };
  },
});

const client = createTestClient();
// Dispatch every call to the mock service object:
client.invoke = (async (_service, method, args) =>
  (usersMock as Record<string, (...a: unknown[]) => Promise<unknown>>)[method](...args)
) as typeof client.invoke;

render(
  <TestNetronProvider client={client}>
    <UserCard userId="missing" />
  </TestNetronProvider>
);
```

`createMockService(implementations)` returns a plain object of
async methods. It is **not** auto-wired into the provider —
dispatch to it via `invoke` as above (or pass `vi.fn()`
implementations so you can assert calls).

## Mock latency

```tsx
render(
  <TestNetronProvider
    testConfig={{
      mocks: [
        { service: 'orders', method: 'getOrder', response: order, delay: 100 },
      ],
    }}
  >
    <OrderCard orderId="1" />
  </TestNetronProvider>
);
```

A per-mock `delay` (or `testConfig.defaultDelay`) lets you assert
intermediate loading states before the response settles.

## Multi-backend tests

For multi-backend apps, build a `MultiBackendProvider` around a
mock multi-backend client in your test setup — the test utils
focus on the single-client path; the multi-backend hooks
(`useBackendService`, …) read from whatever
`<MultiBackendProvider>` you supply.

```tsx
import { MultiBackendProvider } from '@omnitron-dev/netron-react';

render(
  <MultiBackendProvider client={mockMultiBackendClient} autoConnect={false}>
    <Dashboard />
  </MultiBackendProvider>
);
```

See [Multi-backend](/docs/frontend/netron/multi-backend) for the production
`MultiBackendProvider` API the mock client must satisfy.

## Cache control in tests

```tsx
import { createTestClient, TestNetronProvider }
  from '@omnitron-dev/netron-react/test';

const client = createTestClient({
  mocks:  [{ service: 'users', method: 'getUser', response: user }],
  config: { cache: { staleTime: 0, cacheTime: 0 } },   // disable cache for predictability
});

render(<TestNetronProvider client={client}><App /></TestNetronProvider>);
```

`createTestClient(config?)` builds the client; pass cache tuning
through `config`. With `staleTime: 0`, every mount re-fetches —
useful for asserting "fetched N times" without caching surprises.

## Loading + error states

```tsx
it('shows skeleton while loading', async () => {
  render(
    <TestNetronProvider
      testConfig={{
        mocks: [{ service: 'users', method: 'getUser', response: user, delay: 100 }],
      }}
    >
      <UserCard userId="1" />
    </TestNetronProvider>
  );

  expect(screen.getByTestId('skeleton')).toBeInTheDocument();
  await screen.findByText('a@b.c');
  expect(screen.queryByTestId('skeleton')).not.toBeInTheDocument();
});

it('shows error on failure', async () => {
  render(
    <TestNetronProvider
      testConfig={{
        mocks: [{
          service: 'users',
          method:  'getUser',
          error:   new TitanError({ code: ErrorCode.NOT_FOUND }),
        }],
      }}
    >
      <UserCard userId="1" />
    </TestNetronProvider>
  );

  await screen.findByText(/not found/i);
});
```

## Mutation assertions

Spy on the client's `invoke` to assert a mutation fired with the
right arguments — the proxy calls `invoke(service, method, args)`:

```tsx
it('calls invite on submit', async () => {
  const user   = userEvent.setup();
  const client = createTestClient({
    mocks: [{ service: 'users', method: 'invite', response: { id: 'new', email: 'x@y.z' } }],
  });
  const invokeSpy = vi.spyOn(client, 'invoke');

  render(
    <TestNetronProvider client={client}>
      <InviteForm />
    </TestNetronProvider>
  );

  await user.type(screen.getByLabelText('Email'), 'x@y.z');
  await user.click(screen.getByRole('button', { name: 'Invite' }));

  await waitFor(() => {
    expect(invokeSpy).toHaveBeenCalledWith('users', 'invite', ['x@y.z'], expect.anything());
  });
});
```

`useMutation` exposes `isLoading` while the mutation is in flight
(plus `isSuccess` / `isError` / `data`).

## Suspense + error boundary tests

Suspense is the `suspense: true` option on `useQuery` — it makes
the hook throw the in-flight fetch to the nearest `<Suspense>`,
and escalates errors to the nearest error boundary:

```tsx
it('falls back to error boundary on render error', () => {
  render(
    <TestNetronProvider
      testConfig={{
        mocks: [{ service: 'users', method: 'getUser', error: new Error('boom') }],
      }}
    >
      <ErrorBoundary fallback={() => <div>caught</div>}>
        <Suspense fallback={<div>loading</div>}>
          {/* SuspenseUserCard calls useQuery([...], { suspense: true }) */}
          <SuspenseUserCard userId="1" />
        </Suspense>
      </ErrorBoundary>
    </TestNetronProvider>
  );

  return waitFor(() => expect(screen.getByText('caught')).toBeInTheDocument());
});
```

## Integration tests against a real backend

For higher confidence, point tests at a running Titan dev
server:

```tsx
import { NetronReactClient, NetronProvider } from '@omnitron-dev/netron-react';

const realClient = new NetronReactClient({
  url:       process.env.TEST_API_URL ?? 'http://localhost:3001',
  transport: 'http',
});

beforeAll(async () => {
  await realClient.connect();
  // Seed test data via direct RPC
  await realClient.invoke('test-utils', 'reset', []);
});

afterAll(() => realClient.disconnect());

it('full flow', async () => {
  render(<NetronProvider client={realClient}><SignInForm /></NetronProvider>);
  // ...
});
```

Run alongside the omnitron daemon's `test` stack:

```bash
omnitron stack start my-project test
pnpm test:integration
omnitron stack stop my-project test
```

## Deterministic clock

For time-sensitive behaviour (`refetchInterval`, retries,
debounce):

```tsx
import { createTestClient, advanceTimersAndFlush }
  from '@omnitron-dev/netron-react/test';

vi.useFakeTimers();

const client = createTestClient({ /* ... */ });
render(<NetronProvider client={client}><Polling /></NetronProvider>);

await screen.findByText('initial');

await advanceTimersAndFlush(30_000);    // trigger refetch interval + flush microtasks

await screen.findByText('refreshed');

vi.useRealTimers();
```

`advanceTimersAndFlush(ms)` advances vitest fake timers and
flushes pending microtasks; `nextTick()` awaits a single tick.

## Best practices

- **Mock per test file**, not globally. Sharing mocks across
  tests causes order dependencies.
- **Reset mocks** (`vi.clearAllMocks()`) in `beforeEach` if you
  share across `it` blocks.
- **Assert calls + UI**. Calling-the-mock isn't proof; the
  user-visible result is.
- **`staleTime: 0`** in tests for predictability.
- **One test per behaviour**, not per method. "Shows loading,
  then data" + "shows error on failure" are two tests.
- **Integration tests** for the critical happy path; unit
  tests with mocks for edge cases.

## Anti-patterns

- **Mocking `fetch` directly.** Bypasses the client logic;
  use `TestNetronProvider`.
- **Stale mocks shared across tests.** State leaks; mysterious
  failures.
- **Testing implementation details.** "Calls `cache.set`"
  is fragile; "shows the updated value" is robust.
- **Real backend in unit tests.** Slow, flaky, hard to seed —
  reserve for integration suite.

## See also

- [netron-react](/docs/frontend/netron/react) — hooks under test
- [Multi-backend](/docs/frontend/netron/multi-backend) — `MultiBackendProvider`
- [Caching](/docs/frontend/netron/caching) — `staleTime` tuning for tests

---

---
id: "frontend/netron/transports"
title: "Transports"
description: "Choose a Netron transport, bound the lifetime of a call, and observe connection recovery."
---

# Transports

Choose the transport the service exposes at the URL you are calling. A gateway
can serve HTTP RPC at one path and WebSocket upgrades at another.

| Client                          | Use it for                                              | Transport selection                          |
| ------------------------------- | ------------------------------------------------------- | -------------------------------------------- |
| `createClient` / `NetronClient` | Framework-independent RPC to one backend                | `http` by default, or explicit `websocket`   |
| `HttpClient`                    | HTTP RPC with authentication or custom middleware       | HTTP                                         |
| `WebSocketClient`               | Binary RPC, server-initiated tasks and reconnect events | WebSocket                                    |
| `createMultiBackendClient`      | Several backends with separate endpoints and policies   | Configure `http` or `websocket` per backend  |
| `NetronReactClient`             | React queries, mutations and event subscriptions        | `http`, `websocket` or React-specific `auto` |

## HTTP

Supply an absolute base URL. The browser client appends `/netron/invoke` and
sends a JSON POST. HTTP `connect()` initializes the client locally. To check
whether a service is reachable, invoke a permitted read or its readiness probe.

```typescript
import { createClient } from '@omnitron-dev/netron-browser';

const client = createClient({
  url: new URL('/api', window.location.origin).href,
  transport: 'http',
  timeout: 10_000,
  headers: { 'X-Client': 'web/1.0' },
  http: { retry: true, maxRetries: 3, redirect: 'error' },
});
await client.connect();
```

### Invocation lifetime and replay

```typescript
const controller = new AbortController();
const result = await client.invoke('Catalog', 'list', [], {
  signal: controller.signal,
  timeout: 10_000,
  idempotent: true,
});
```

`signal` is local control state, never part of the wire payload. Cancellation
rejects with `signal.reason`. The deadline covers transport attempts, backoff,
response-body decoding and response middleware. A response whose headers arrive
before the deadline can still time out while its body is being read.
Cancellation and deadline expiry do not trigger network retry or auth refresh.
Aborting the caller does not roll back an operation already accepted by the server.

Network retries require **both** client `http.retry: true` and invocation
`idempotent: true`. The second option is the caller's assertion about the operation,
not an idempotency key or a server deduplication guarantee. Do not set it for a
write merely because a failed connection returned no answer. Use a domain
operation ID with server-side replay semantics when a write must be retried.
The direct HTTP client does not infer this property from a method name.

For a fixed RPC endpoint, use `http.redirect` on `createClient` or pass
`redirect: 'error'` directly to `HttpClient`.
Fetch then rejects a redirect before replaying the request at another address.
This protects publication bodies and custom credential headers. With multiple
backends, set `http.redirect` on the backend that needs this policy. Omitting
the option preserves the default Fetch redirect behavior.

Service query hooks forward the query cancellation signal. A component unmount does
not automatically cancel a mutation: its server-side outcome may already be committed.

### HTTP wire format

The direct browser transport posts JSON to `/netron/invoke`:

```json
{"id":"request-id","service":"Catalog","method":"list","input":[]}
```

A successful reply contains the same `id`, `success: true`, and `data`.
A failed invocation contains `success: false` and a structured `error`. Titan may
also reject a request before RPC admission with an HTTP transport error. The
client distinguishes that refusal from a successful RPC envelope. A mismatched
ID, malformed envelope or an HTTP failure carrying an RPC success is rejected.
Application payloads still require their own runtime schemas; a TypeScript
return type cannot validate bytes received over the network.

## WebSocket

Use the URL of a binary Netron WebSocket listener. Await `connect()` before
issuing calls. It waits for the Netron handshake. Use the direct client for
connection events, service discovery or server-initiated task handlers.

```typescript
import { WebSocketClient } from '@omnitron-dev/netron-browser';

const socket = new WebSocketClient({
  url: 'wss://api.example.com/rpc',
  enableServiceDiscovery: true,
  reconnect: true,
  reconnectInterval: 1_000,
  maxReconnectAttempts: 12,
});

socket.on('reconnecting', ({ attempt, delay, announced }) => {
  console.info(announced ? 'Server restarting' : 'Connection lost', { attempt, delay });
});
await socket.connect();
try {
  const items = await socket.invoke('Catalog@1.0.0', 'list', []);
  console.info(items);
} finally {
  await socket.disconnect();
}
```

Service discovery resolves a service's wire definition ID on the connected
server. It is opt-in. Only enable it for a trusted endpoint. Alternatively,
supply a known `serviceDefinitions` map. Register a named server-initiated task
with `socket.onTask(name, handler)` when the service contract requires it.

### Reconnect behaviour

Automatic reconnect is enabled by default. The base retry interval is 1,000 ms
and the default attempt limit is `Infinity`. Ordinary retries use exponential
backoff with jitter, capped at 30 seconds with the default base. A server restart
announced by close code 1012 gets an initial jittered delay of 1–3 seconds.
The `reconnecting` event exposes the actual delay. Exhausting a finite limit
sets the state to `failed` and emits `reconnect-failed`. Explicit `disconnect()`
ends the reconnect loop.

A closed socket rejects its pending invocations. For an interrupted write,
check its recorded outcome before submitting it again. For events, the React
subscription manager re-registers subscribed names after reconnect. Recover
events missed during the outage using the service's cursor or a fresh snapshot.

## React transport selection

Use `transport: 'http'` for an HTTP-only API, including a React application
behind an HTTP gateway. Use `'websocket'` for a WebSocket-only service.

`NetronReactClient` defaults to `'auto'`. Without a browser `window` it selects
HTTP. In the browser it opens WebSocket first, waits for the handshake, then
initializes HTTP with the same configured URL. Calls prefer the connected
WebSocket and can use the initialized HTTP client while the socket is down.
If the initial WebSocket connection fails, `connect()` rejects before HTTP
initialization. For this mode, supply an absolute HTTP(S) base that serves
both the WebSocket upgrade and the HTTP RPC path. For separate endpoints,
configure each backend explicitly.

## Connection state and metrics

The direct browser clients expose `getState()` and `getMetrics()`.
Connection states are `disconnected`, `connecting`, `connected`,
`reconnecting` and `failed`. A React component inside `NetronProvider`
can read `state`, `error` and `reconnect` from `useNetronConnection()`.

```typescript
const { transport, state, requestsSent, responsesReceived, errors, avgLatency } =
  client.getMetrics();
```

These counters belong to this client. `avgLatency` is optional and measured
in milliseconds. Use a query cache or the [fluent HTTP interface](/docs/frontend/netron/browser#fluent-http-interface--caching-retry-circuit-breaking)
when individual reads require caching or deduplication.

## See also

- [HTTP and WebSocket clients](/docs/frontend/netron/browser)
- [Authentication and cookie transport](/docs/frontend/netron/auth)
- [React subscriptions](/docs/frontend/netron/react)
- [Multi-backend routing](/docs/frontend/netron/multi-backend)

---

---
id: "frontend/overview"
title: "Frontend overview"
description: "Three packages that cover everything browser-side in a Titan app."
---

# Frontend overview

> The **server-side** Netron lives inside Titan itself
> (`@omnitron-dev/titan/netron`, including all four
> transports). This section covers the **browser side**.

Browser-side packages:

| Package                                                         | Role                                                                   | Framework                                                                                 | Size                                                                   |
| --------------------------------------------------------------- | ---------------------------------------------------------------------- | ----------------------------------------------------------------------------------------- | ---------------------------------------------------------------------- |
| [`@omnitron-dev/netron-browser`](/docs/frontend/netron/browser) | RPC transport (HTTP + WebSocket), middleware, auth, multi-backend pool | **Framework-agnostic** — vanilla JS, Vue, Svelte, Solid, Angular, Lit, React, Web Workers | \~25 kB gz                                                             |
| [`@omnitron-dev/netron-react`](/docs/frontend/netron/react)     | React hooks, query/mutation cache, auth guards, devtools               | **React-only** — optional layer on top of netron-browser                                  | \~15 kB gz                                                             |
| [`@omnitron-dev/prism`](/docs/frontend/prism)                   | Design system: MUI v9 components, blocks, layouts, forms, hooks        | **React-only** (MUI v9 is React) — optional                                               | Depends on selected components and peers; unused modules are discarded |

`netron-browser` works everywhere. Add `netron-react` + `prism`
**only** for React apps. For Vue / Svelte / Solid / Angular / Lit,
use `netron-browser` directly and wrap calls in your framework's
reactivity primitives.

## How they compose

```mermaid
flowchart LR
  Comp["React component<br/>built with Prism"]
  Hook["netron-react hook<br/>(useQuery / useService)"]
  Client["netron-browser<br/>NetronClient"]
  Wire["HTTP / WebSocket"]
  Server["Titan @Service / @Public"]

  Comp --> Hook --> Client --> Wire --> Server
```

A typical screen:

```tsx
import { DataGridBlock }   from '@omnitron-dev/prism/blocks';
import { useService }      from '@omnitron-dev/netron-react';

interface UserService {
  list(filter: UserFilter): Promise<User[]>;
}

function UsersPage() {
  const users = useService<UserService>('users');
  return (
    <DataGridBlock
      title="Users"
      columns={[
        { field: 'email',  header: 'Email' },
        { field: 'role',   header: 'Role'  },
      ]}
      query={({ page, sort, filter }) =>
        users.list.useQuery([{ page, sort, filter }])
      }
    />
  );
}
```

Three packages, one screen, no codegen.

## End-to-end types

There is no schema file, no codegen step, no manual sync. The
TypeScript compiler is the source of truth:

```typescript
// Shared package — the contract:
export interface UsersService {
  findById(id: string): Promise<User | null>;
}

// Browser — type flows through:
const users = useService<UsersService>('users');
const { data } = users.findById.useQuery([id]);
//      ^? User | null | undefined
```

A method signature change on the server fails the build on
every caller in the same `tsc` pass.

## Wiring an app from scratch

```tsx
import { NetronReactClient, NetronProvider }   from '@omnitron-dev/netron-react';
import { AuthProvider }                        from '@omnitron-dev/netron-react/auth';
import { PrismProvider, ProviderStack }        from '@omnitron-dev/prism/core';
import { RouterProvider }                      from 'react-router-dom';

const client = new NetronReactClient({
  url:       import.meta.env.VITE_API_URL,
  transport: 'http',
});

// `AuthProvider` takes a `config` (refreshEndpoint?, logoutEndpoint?,
// storage?: 'local' | 'session' | 'memory', autoRefresh?, refreshThreshold?)
// and an `onLogin` handler that performs the actual sign-in and returns
// an AuthResult.
function App() {
  return (
    <ProviderStack
      providers={[
        [NetronProvider, { client }],
        [AuthProvider,   {
          config:  { storage: 'local', autoRefresh: true, refreshEndpoint: '/auth/refresh' },
          onLogin: (credentials) => signIn(client, credentials),
        }],
        [PrismProvider,  { defaultSettings: { mode: 'dark', primaryColor: '#7c4dff' } }],
        [RouterProvider, { router }],
      ]}
    >
      <Outlet />
    </ProviderStack>
  );
}
```

Four providers, one app. Anything Titan-side becomes
discoverable through `useService`. The auth API is `login` / `logout`
(from `useAuth()`) with `<AuthGuard>` / `<GuestGuard>` /
`<RoleGuard>` / `<PermissionGuard>` route guards — see
[netron-react / Auth](/docs/frontend/netron/auth) for the full surface.

## Multi-backend out of the box

Talking to several Netron servers? Swap `NetronProvider` for
`MultiBackendProvider`:

```tsx
import { createMultiBackendClient }
  from '@omnitron-dev/netron-browser';
import { MultiBackendProvider, useBackendService }
  from '@omnitron-dev/netron-react';

const client = createMultiBackendClient({
  baseUrl: 'https://api.example.com',
  backends: {
    auth:      { path: '/auth',      transport: 'http' },
    media:     { path: '/media',     transport: 'http' },
    analytics: { path: '/analytics', transport: 'http' },
  },
  routing: {
    patterns: [
      { pattern: 'users.',   backend: 'auth' },
      { pattern: 'objects.', backend: 'media' },
      { pattern: 'reports.', backend: 'analytics' },
    ],
  },
});

<MultiBackendProvider client={client}>
  <Outlet />
</MultiBackendProvider>

// In components:
const users = useBackendService<UserService>('auth', 'users');
```

→ [netron-react / Multi-backend](/docs/frontend/netron/react#multi-backend-support).

## What lives where

| You want…                                         | Reach for                                                  |
| ------------------------------------------------- | ---------------------------------------------------------- |
| Theme, colors, typography                         | `@omnitron-dev/prism/theme`                                |
| Dashboard / auth / data-grid shell                | `@omnitron-dev/prism/blocks`                               |
| Sidebar + topbar layout                           | `@omnitron-dev/prism/layouts`                              |
| Card / Table / Drawer / Form field                | `@omnitron-dev/prism` or `@omnitron-dev/prism/components`  |
| Schema-aware forms                                | `@omnitron-dev/prism/forms`                                |
| Persisted UI state (sidebar open, dark mode)      | `@omnitron-dev/prism/state`                                |
| Plain React hooks (`useArray`, `useFocusTrap`, …) | `@omnitron-dev/prism/hooks`                                |
| Type-safe RPC call inside a component             | `useService` from `@omnitron-dev/netron-react`             |
| Live data over WebSocket                          | `useSubscription` from `@omnitron-dev/netron-react`        |
| Sign-in flow + route guards                       | `@omnitron-dev/netron-react/auth` + `<AuthGuard>`          |
| Multi-backend setup                               | `MultiBackendProvider` from `@omnitron-dev/netron-react`   |
| Raw RPC client (no React)                         | `createClient` from `@omnitron-dev/netron-browser`         |
| Cross-tab auth sync                               | `AuthenticationClient` from `@omnitron-dev/netron-browser` |
| Cache, retry, circuit breaker                     | `@omnitron-dev/netron-browser/middleware`                  |

## Bundle strategy

| Stage                     | Bundle goal                                                         |
| ------------------------- | ------------------------------------------------------------------- |
| First paint (auth screen) | Load the auth screen and its providers                              |
| Authenticated shell       | Add the dashboard layout and shared navigation after authentication |
| Per-route page            | Lazy-load with `React.lazy()` and measure the emitted route chunks  |
| Charts / editor / maps    | Load engines and styles with the routes that use them               |

Named imports from the root and `@omnitron-dev/prism/components` discard unused
modules. Public subpaths such as `components/card`, `components/chart`,
`components/map`, `components/editor` and `components/emoji-picker` are also
available for explicit composition and lazy loading. Only declared exports are
public; there is no wildcard mapping from every source directory.

Import each component's documented CSS entry when required. CSS remains in the
production output. The package archive check verifies that a simple `Card`
consumer does not retain optional engines and that selected calendar, map and
stylesheet dependencies remain. It does not set an application-wide size promise:
measure initial requests, route chunks and shared peer dependencies in the actual
production build.

## Production reference

The Omnitron Console (`apps/omnitron/webapp/`) is built entirely
from these three packages. \~20 pages, multi-backend, full RBAC,
real-time subscriptions, devtools — see
[Console](/docs/omnitron/console) for the route map.

## Read on

- [Prism](/docs/frontend/prism) — design system in depth
- [netron-browser](/docs/frontend/netron/browser) — the transport client
- [netron-react](/docs/frontend/netron/react) — React hooks

---

---
id: "frontend/prism/accessibility"
title: "Accessibility"
description: "Vision simulation, keyboard interaction and accessible composition with Prism."
---

# Accessibility

Prism's accessibility subpath exports **vision simulation** tools:
`VisionModeProvider`, `useVisionMode`, `getVisionModeStyles` and
`VISION_MODE_OPTIONS`. It does not export a general ARIA utility library.
Component accessibility also depends on the labels, semantics and focus
behaviour supplied by the application.

## Simulate colour perception

```tsx
import {
  VisionModeProvider, useVisionMode, VISION_MODE_OPTIONS,
  type VisionMode,
} from '@omnitron-dev/prism/accessibility';

function VisionSelector() {
  const { mode, setMode } = useVisionMode();
  return (
    <label>
      Vision simulation
      <select value={mode} onChange={(event) => setMode(event.target.value as VisionMode)}>
        {VISION_MODE_OPTIONS.map((option) => (
          <option key={option.value} value={option.value}>{option.label}</option>
        ))}
      </select>
    </label>
  );
}

export function Preview() {
  return (
    <VisionModeProvider defaultMode="normal" disablePersistence>
      <VisionSelector />
      <p>Review your application controls in each mode.</p>
    </VisionModeProvider>
  );
}
```

The modes are `normal`, `protanopia`, `deuteranopia`, `tritanopia` and
`achromatopsia`. The hook also exposes `toggle`, `reset` and `isFiltered`.
The provider installs SVG filters and applies the selected mode globally.
Use it in a development or preview surface with a visible reset control.

| Provider option      | Meaning                                                    |
| -------------------- | ---------------------------------------------------------- |
| `defaultMode`        | Initial mode when no persisted preference takes precedence |
| `storageKey`         | Persistence key; defaults to `prism-vision-mode`           |
| `disablePersistence` | Starts without reading or writing the stored preference    |
| `attributeName`      | Attribute used for the filter; defaults to `data-vision`   |

Simulation helps find colour-dependent distinctions. It does not measure
contrast, establish conformance or replace keyboard and assistive-technology
testing.

## Compose accessible screens

| Surface             | Application responsibility                                                |
| ------------------- | ------------------------------------------------------------------------- |
| Icon-only actions   | Supply a clear accessible name such as `aria-label`                       |
| Inputs              | Associate labels and descriptions; connect validation errors to the field |
| Dialogs and drawers | Verify initial focus, Escape behaviour and focus restoration              |
| Tables              | Preserve headings; expose sort state and label row actions                |
| Async results       | Provide loading, empty and failure states; announce meaningful changes    |
| Status colours      | Include text or an icon shape in addition to colour                       |
| Motion              | Respect reduced-motion preferences                                        |

Start with the semantics of the underlying control. Avoid replacing a
button with a clickable `div`. Custom renderers and slots can remove
semantics that a default component supplies, so test the final composition.

## Menus and popovers

Under `createPrismTheme` or `PrismProvider`, menus, Select dropdowns and
popovers release pointer input when `open` becomes `false`. Their exit
animation may finish in the DOM, but the next click on a neighbouring control
works immediately. Open overlays keep MUI's click-away behaviour, keyboard
navigation and focus restoration. Closing a menu inside a dialog leaves
the parent dialog modal.

Keep the trigger mounted and give it an accessible name. Use the native
Escape and focus-restoration behaviour; disabling it to compensate for
an animation can strand keyboard users. Where an interaction needs immediate
visual removal, the MUI `Menu` exported from `@omnitron-dev/prism/ui` accepts
`transitionDuration={{ exit: 0 }}` through its ordinary public API.

Test the first click after selection, immediate reopening, Escape followed
by Tab, and menus inside dialogs. A test that retries a blocked click until
an animation ends can hide a lost first click.

## Verification loop

Navigate a representative screen using only Tab, Shift+Tab, Enter, Space
and Escape. Check visible focus in both themes, then test zoom and a narrow
viewport. Run the repository's `pnpm -F @omnitron-dev/prism test:a11y`
suite for its test application and add checks for your own screens.
Read [React testing](/docs/testing/react) for component-level examples.

---

---
id: "frontend/prism/blocks"
title: "Blocks"
description: "Full-page composites — drop one in, fill the slots, ship a screen."
---

# Blocks

Blocks sit one level above [layouts](/docs/frontend/prism/layouts) and components.
They're higher-level composites — auth forms + guards, a
collapsible dashboard panel, a full data-grid — that you drop
into a route, wire to your data callbacks, and ship.

Import `AuthBlock`, `DashboardBlock` and `DataGridBlock` from `@omnitron-dev/prism/blocks`.

## `AuthBlock` — auth forms + route guards

`AuthBlock` is a **namespace** of ready-made auth forms —
`AuthBlock.Login`, `AuthBlock.Register`, `AuthBlock.ForgotPassword`,
`AuthBlock.ResetPassword`, `AuthBlock.VerifyCode` (the same
components are also exported individually as `LoginForm`,
`RegisterForm`, etc.). There is no single `<AuthBlock mode="…">`
component — render the form you need and wire its `onSubmit`:

```tsx
import { AuthBlock } from '@omnitron-dev/prism/blocks';

function SignInPage() {
  return (
    <AuthBlock.Login
      onSubmit={async ({ email, password, rememberMe }) => {
        await authService.signIn({ email, password });
        navigate('/');
      }}
      showRememberMe
      showForgotPassword
      onForgotPassword={() => navigate('/recover')}
      showSocialLogin
      socialProviders={[
        { id: 'google', name: 'Google', icon: <GoogleIcon />, onClick: oauthGoogle },
        { id: 'github', name: 'GitHub', icon: <GithubIcon />, onClick: oauthGithub },
      ]}
    />
  );
}
```

### The forms

| Member (namespace)         | Standalone export    | Submit payload                     |
| -------------------------- | -------------------- | ---------------------------------- |
| `AuthBlock.Login`          | `LoginForm`          | `{ email, password, rememberMe? }` |
| `AuthBlock.Register`       | `RegisterForm`       | `RegisterFormData`                 |
| `AuthBlock.ForgotPassword` | `ForgotPasswordForm` | `{ email }`                        |
| `AuthBlock.ResetPassword`  | `ResetPasswordForm`  | `ResetPasswordFormData`            |
| `AuthBlock.VerifyCode`     | `VerifyCodeForm`     | `{ code }`                         |

### Common form props

| Prop        | Type                                   | Notes                                           |
| ----------- | -------------------------------------- | ----------------------------------------------- |
| `onSubmit`  | `(data) => Promise<void> \| void`      | Submit handler for the form                     |
| `loading`   | `boolean`                              | Disables + shows progress. Optional — see below |
| `disabled`  | `boolean`                              | Disable the form                                |
| `schema`    | `z.ZodSchema`                          | Override the built-in Zod validation            |
| `labels`    | per-field label overrides              | i18n / copy customisation                       |
| `sx`        | `SxProps<Theme>`                       | Styles for the form wrapper                     |
| `slotProps` | `{ wrapper, submitButton, textField }` | MUI slot overrides                              |

The forms use react-hook-form + Zod internally — you get email
validation and field-level errors for free.

**You do not have to pass `loading` to prevent a double submit.**
The form awaits your `onSubmit` and falls back to react-hook-form's
own `isSubmitting`, so every control is disabled for as long as the
promise is pending. Pass `loading` when the *parent* owns a pending
state the form cannot see — a redirect that runs after sign-in, say,
where the button should stay disabled past the resolution of
`onSubmit`.

**Error messages from `onSubmit` are shown, sanitized.** A rejection
is caught and its message rendered above the form, unless it looks
like something internal — a stack frame, an errno, a SQL fragment, a
file path, or a leaked credential value. It matches the *value*, so
"Password must contain at least 2 digits" and "Reset token expired"
reach the user; `password=hunter2` and a JWT do not.

### Per-form props

| Form                     | Prop                                                               | Notes                                         |
| ------------------------ | ------------------------------------------------------------------ | --------------------------------------------- |
| Login                    | `showRememberMe`, `showForgotPassword`, `onForgotPassword`         |                                               |
| Login                    | `showSocialLogin`, `socialProviders`                               | `{ id, name, icon, onClick }[]`               |
| Register                 | `showNameFields`, `requireTerms`, `onTermsClick`, `onPrivacyClick` |                                               |
| Register / ResetPassword | `minPasswordLength`                                                | Feeds the built-in schema                     |
| ForgotPassword           | `onBackToLogin`                                                    |                                               |
| VerifyCode               | `codeLength`                                                       | Default 6                                     |
| VerifyCode               | `onResendCode`, `resendCooldown`                                   | Seconds before Resend re-enables (default 60) |

Resend is guarded against a second click while the first request is
still open, so a double-click sends one code rather than two — which
matters on a rate-limited endpoint, where the second attempt is what
locks the user out of the flow they are finishing.

### Route guards

The block also exports composable route guards (each takes a
`useAuth` hook so you can plug in your own auth store):

```tsx
import { AuthGuard, GuestGuard, RoleBasedGuard }
  from '@omnitron-dev/prism/blocks';

<AuthGuard useAuth={useAuth} onUnauthenticated={() => navigate('/sign-in')}>
  <Dashboard />
</AuthGuard>

<RoleBasedGuard
  useAuth={useAuth}
  roles={['admin', 'moderator']}
  roleMatchStrategy="any"
  accessDeniedComponent={<Forbidden />}
>
  <AdminPanel />
</RoleBasedGuard>
```

`hasRole` / `hasPermission` / `createConditionalRender` helpers
are exported for inline checks.

## `<DashboardBlock>` — collapsible content panel

`<DashboardBlock>` is a flexible **panel/card** primitive — a
titled, optionally-collapsible container with header, content,
and footer slots, plus built-in loading and error states. It is
the building block you compose dashboards *from* (it does not
itself render stat tiles or charts — drop those in as children):

```tsx
import { DashboardBlock } from '@omnitron-dev/prism/blocks';

function RequestsPanel({ loading, series }) {
  return (
    <DashboardBlock
      title="Requests / min"
      subtitle="last 24h"
      icon={<ActivityIcon />}
      actions={<IconButton><RefreshIcon /></IconButton>}
      loading={loading}
      collapsible
    >
      <Chart series={series} type="area" />
    </DashboardBlock>
  );
}
```

| Prop                             | Type                              | Purpose                                    |
| -------------------------------- | --------------------------------- | ------------------------------------------ |
| `title` / `subtitle`             | `string`                          | Header text                                |
| `icon`                           | `ReactNode`                       | Leading header icon                        |
| `actions`                        | `ReactNode`                       | Top-right header slot                      |
| `children`                       | `ReactNode`                       | Panel body                                 |
| `footer`                         | `ReactNode`                       | Footer slot                                |
| `variant` / `size`               | `DashboardBlockVariant` / `…Size` | Visual style                               |
| `loading`                        | `boolean`                         | Shows the loading skeleton                 |
| `error`                          | `boolean`                         | Shows the error state (`errorConfig`)      |
| `collapsible`                    | `boolean`                         | Enables collapse controls; default `false` |
| `defaultCollapsed`               | `boolean`                         | Initial state for an uncontrolled block    |
| `collapsed` / `onCollapseChange` | `boolean` / callback              | Controlled state and requested changes     |

`DashboardBlockHeader`, `DashboardBlockContent`, and
`DashboardBlockFooter` are exported for fully custom composition,
and `useDashboardBlock` drives collapse/loading state.

Capability and state are separate: without `collapsible`, the block remains
expanded, renders no collapse button and ignores context collapse actions, even
if a collapsed state was supplied. Custom headers read the same `collapsible`
flag from `useDashboardBlockContext`. An enabled block collapses its complete
body, including loading, error and empty states; its header stays available.

Use `defaultCollapsed` for internal state, or pair `collapsed` with
`onCollapseChange` for state owned by the caller. The callback reports requested
changes in both modes. The public hook supplies state props, so enable the
capability explicitly when spreading them:

```tsx
import { DashboardBlock, useDashboardBlock } from '@omnitron-dev/prism/blocks';

function ActivityPanel() {
  const { blockProps } = useDashboardBlock({ defaultCollapsed: false });
  return (
    <DashboardBlock title="Activity" collapsible {...blockProps}>
      <ActivityFeed />
    </DashboardBlock>
  );
}
```

External controls can use the hook's `actions.expand()`, `actions.collapse()` and
`actions.toggleCollapse()` to update that same controlled state.

## `<DataGridBlock>` — filterable / sortable / paginated table

The most-used block. It wraps **MUI X DataGrid** and adds a
toolbar, row actions, selection, and empty/error states behind
one prop API. Columns are MUI `GridColDef`s and rows are passed
directly via the `rows` prop:

```tsx
import { DataGridBlock } from '@omnitron-dev/prism/blocks';

function UsersPage({ users, loading }) {
  return (
    <DataGridBlock
      rows={users}
      getRowId={(row) => row.id}
      loading={loading}
      columns={[
        { field: 'email',  headerName: 'Email', flex: 1, sortable: true },
        { field: 'role',   headerName: 'Role' },
        { field: 'status', headerName: 'Status',
          renderCell: (p) => <StatusChip status={p.value} /> },
        { field: 'createdAt', headerName: 'Created',
          renderCell: (p) => <DateCell value={p.value} /> },
      ]}
      onRowClick={(params) => navigate(`/users/${params.id}`)}
      rowActions={[
        { key: 'edit',   label: 'Edit',   icon: <EditIcon />,  onClick: (row) => navigate(`/users/${row.id}/edit`) },
        { key: 'remove', label: 'Remove', icon: <TrashIcon />, color: 'error', onClick: handleRemove },
      ]}
      selection={{ enabled: true, checkboxSelection: true }}
      onSelectionChange={(model) => setSelected(model)}
      toolbar={{
        quickFilter:      { enabled: true, placeholder: 'Search by email' },
        export:           { csv: true },
        columnVisibility: { enabled: true },
        densitySelector:  true,
      }}
      pagination={{ defaultPageSize: 25, pageSizeOptions: [25, 50, 100] }}
      empty={{
        title:       'No users yet',
        description: 'Invite your first user.',
        action:      { label: 'Invite', onClick: onInvite },
      }}
    />
  );
}
```

### Columns

Columns are standard MUI X `GridColDef`s — `field`, `headerName`,
`flex` / `width`, `sortable`, `renderCell`, `valueGetter`, etc.
(Not a bespoke `ColumnDef`.) Column visibility is controlled via
`columnVisibilityModel` / `onColumnVisibilityChange`, or the
toolbar's `columnVisibility` toggler.

### Row actions

`rowActions` render in a per-row menu — each is keyed and gets
the row + event:

```tsx
rowActions={[
  { key: 'edit',   label: 'Edit',   icon: <EditIcon />,  onClick: (row, e) => { /* … */ } },
  { key: 'view',   label: 'View',   icon: <EyeIcon />,   onClick: (row, e) => { /* … */ } },
  { key: 'remove', label: 'Remove', icon: <TrashIcon />, color: 'error',
    disabled: (row) => row.locked,
    divider: true,
    onClick: (row, e) => { /* … */ },
  },
]}
```

Actions support `disabled` / `hidden` (boolean or `(row) =>
boolean`), `color`, and a `divider`. (There is no built-in
`confirm` field — open your own `<ConfirmDialog>` from `onClick`.)

### Selection & bulk operations

```tsx
<DataGridBlock
  selection={{ enabled: true, mode: 'multiple', checkboxSelection: true }}
  onSelectionChange={(model) => setSelected(model)}
  // …
/>
```

`onSelectionChange` reports the MUI `GridRowSelectionModel`; drive
your own bulk-action bar from that selection.

### Toolbar config

```typescript
interface DataGridToolbarConfig {
  show?:             boolean;
  quickFilter?:      { enabled?, placeholder?, debounceMs? };
  export?:           { csv?, excel?, print?, fileName? };
  columnVisibility?: { enabled?, defaultModel? };
  densitySelector?:  boolean;
  actions?:          ReactNode;          // custom toolbar actions
  startContent?:     ReactNode;          // left-aligned custom content
  endContent?:       ReactNode;          // right-aligned custom content
}
```

### Server-side data — `useDataGridBlock`

For server-driven paging/sorting/filtering, use the
`useDataGridBlock` hook: pass a `fetcher` and spread its return
onto the block (server pagination mode):

```tsx
import { DataGridBlock, useDataGridBlock } from '@omnitron-dev/prism/blocks';

function UsersPage() {
  const grid = useDataGridBlock({
    fetcher: async ({ page, pageSize, sortModel, filterModel, quickFilterValue }) => {
      const res = await usersService.list({ page, pageSize, sortModel, filterModel, q: quickFilterValue });
      return { rows: res.items, total: res.total };   // DataGridFetchResult
    },
    initialPageSize: 25,
  });

  return (
    <DataGridBlock
      columns={columns}
      rows={grid.rows}
      loading={grid.loading}
      pagination={{ mode: 'server', rowCount: grid.total }}
      paginationModel={grid.paginationModel}
      onPaginationChange={grid.onPaginationChange}
      sortModel={grid.sortModel}
      onSortChange={grid.onSortChange}
      filterModel={grid.filterModel}
      onFilterChange={grid.onFilterChange}
    />
  );
}
```

The toolbar applies space-separated search terms to
`filterModel.quickFilterValues`. Client grids filter their rows immediately
after the debounce window. Server grids pass the same terms to the fetcher,
with `quickFilterValue` available as joined text. Forward the request's
`signal` to your HTTP client to cancel superseded work. The grid displays
only the latest response, even when the server completes an older request
later. Escape clears search. Pass `toolbar={{ show: false }}` to omit it.

The hook also returns `refresh()` and `reset()`. For client-side
data, just pass the full `rows` array and use the default
(`'client'`) pagination mode.

## When to use a block vs compose your own

| Reach for a block when…                          | Compose components when…                    |
| ------------------------------------------------ | ------------------------------------------- |
| The screen matches a standard pattern            | Layout differs from the template            |
| You want consistency across many similar screens | The screen is unique                        |
| Speed of iteration matters more than control     | Pixel-precision matters more than speed     |
| You're building admin / operator surfaces        | You're building a marketing / brochure page |

The blocks are compositions of public components + hooks, so
dropping out is cheap. If you outgrow `<DataGridBlock>`, drop to
MUI X `<DataGrid>` directly (or `<FilterToolbar>` + the Prism
`<Table>`) without changing the rest of your page.

## Subpath imports

```tsx
import { AuthBlock, DashboardBlock, DataGridBlock } from '@omnitron-dev/prism/blocks';
```

## See also

- [Components catalog](/docs/frontend/prism/components) — what blocks are
  composed of
- [Layouts](/docs/frontend/prism/layouts) — shells that hold blocks
- [Forms](/docs/frontend/prism/forms) — `<AuthBlock>` uses these primitives

## Guard-only entry point

Use `@omnitron-dev/prism/auth` when you need guards without forms or layout dependencies:

```tsx
import { RoleBasedGuard, hasPermission, type AuthState } from '@omnitron-dev/prism/auth';

function TerminalArea() {
  return (
    <RoleBasedGuard useAuth={useConsoleAuthorization} permissions={['OmnitronTerminal.open']}
      loadingComponent={<p>Checking access…</p>}
      accessDeniedComponent={<p>Access unavailable</p>}>
      <TerminalWorkspace />
    </RoleBasedGuard>
  );
}
```

The entry exports `AuthGuard`, `GuestGuard`, `RoleBasedGuard`, `hasRole`, `hasPermission`, `createConditionalRender` and their types. Guards are router-independent. Supply an authentication hook backed by the application's session and current server policy. `RoleBasedGuard` evaluates grants during rendering, so revoked children cannot appear for a stale render while an effect updates state. UI guards never replace backend authorization.

Omnitron uses this entry with [RPC-derived capabilities](/docs/omnitron/access), preserving the same role policy in Console and CLI.

---

---
id: "frontend/prism/cli"
title: "Component CLI"
description: "Initialise, inspect, copy and maintain Prism components with the shipped CLI."
---

# Component CLI

The `prism` binary ships with `@omnitron-dev/prism`. Use package imports
for components you want to update with the library. Use the CLI when you
want a local copy whose source your application owns.

From a project with Prism installed:

```bash
pnpm exec prism init
pnpm exec prism list --components
pnpm exec prism add card
pnpm exec prism doctor
```

The commands are `init`, `add`, `remove`, `list`, `diff`, `update` and
`doctor`. Component names are arguments to `add`; `component` is not an
intermediate subcommand.

## Command reference

| Command                  | Purpose                               | Useful options                                         |
| ------------------------ | ------------------------------------- | ------------------------------------------------------ |
| `init`                   | Initialise project configuration      | `--defaults`, `--yes`, `--cwd`                         |
| `list`                   | Inspect available items               | `--components`, `--blocks`, `--category`, `--json`     |
| `add [components...]`    | Copy selected items                   | `--all`, `--path`, `--install`, `--overwrite`, `--cwd` |
| `diff [components...]`   | Compare local items with the registry | `--cwd`                                                |
| `update [components...]` | Update copied items                   | `--all`, `--overwrite`, `--yes`, `--cwd`               |
| `remove <components...>` | Remove installed items                | `--force`, `--yes`, `--cwd`                            |
| `doctor`                 | Diagnose the project's setup          | `--cwd`                                                |

Run `pnpm exec prism <command> --help` for the exact options in the version
installed in your lockfile.

## Work with local changes

Commit or otherwise preserve local modifications before updating copied
components. Inspect `diff` first. `--overwrite` allows replacement of
modified files; `remove --force` allows removal even when an installed
item was modified. Those flags are explicit decisions about your source.

`add --install` also installs the selected item's npm dependencies.
Review the package and lockfile diff together with the copied source.

## Programmatic use

`@omnitron-dev/prism/cli` exports `createCli`, `cli`, command factories,
`PRISM_VERSION`, `DEFAULT_CONFIG` and the configuration types. This is a
Node.js entry point. Keep it out of a browser application's import graph.

For the component APIs themselves, use the
[component catalogue](/docs/frontend/prism/components), [blocks](/docs/frontend/prism/blocks) and
[layouts](/docs/frontend/prism/layouts). The [API index](/docs/reference/api-index)
lists declared package entry points.

---

---
id: "frontend/prism/components"
title: "Components catalog"
description: "Per-component reference for the widgets in Prism."
---

# Components catalog

Explore each component, change its state and open **Source code** to see the TypeScript that runs the example. The API and usage notes remain available with JavaScript disabled.

For full-page composites see [Blocks](/docs/frontend/prism/blocks); for layouts
see [Layouts](/docs/frontend/prism/layouts). For scientific documents, block editing and typed knowledge graphs, see [Research workspaces](/docs/frontend/prism/research).

## Status & feedback

### `<Alert>` / `<FormAlert>`

This sample starts in a valid state. Select **Show validation error** to inspect
`FormAlert`; the message belongs to this example.

```tsx
import { Alert, FormAlert } from '@omnitron-dev/prism';

<Alert severity="warning" title="Storage low">
  90% of the bucket quota is in use.
</Alert>

<FormAlert title="Invalid email">
  Please enter a valid email address.
</FormAlert>
```

| Prop       | Type                                          | Default      | Notes                         |
| ---------- | --------------------------------------------- | ------------ | ----------------------------- |
| `severity` | `'info' \| 'success' \| 'warning' \| 'error'` | `'info'`     | Drives icon + colour          |
| `title`    | `ReactNode`                                   | —            | Bold leading line             |
| `onClose`  | `() => void`                                  | —            | Renders dismiss button        |
| `variant`  | `'standard' \| 'filled' \| 'outlined'`        | `'standard'` | Visual weight                 |
| `icon`     | `ReactNode \| false`                          | (derived)    | Override (or `false` to hide) |
| `closable` | `boolean`                                     | `false`      | Show the dismiss button       |

**`<FormAlert>`** is the canonical surface for **inline** form
errors — toast/snackbar is reserved for transient background
events. It defaults to `severity="error"`; pass the message as
**children** (there is no `error` prop). Pair with
`react-hook-form`:

```tsx
const { formState, register } = useForm();

<form>
  {formState.errors.root && (
    <FormAlert>{formState.errors.root.message}</FormAlert>
  )}
  <input {...register('email')} />
</form>
```

### `<Snackbar>`

Transient global notifications. Mounted once by `<PrismProvider>`;
fire from anywhere:

```tsx
import { useSnackbar } from '@omnitron-dev/prism';

function CopyButton({ text }: { text: string }) {
  const { show } = useSnackbar();
  return (
    <Button onClick={() => {
      navigator.clipboard.writeText(text);
      show({ message: 'Copied', severity: 'success', duration: 2_000 });
    }}>
      Copy
    </Button>
  );
}
```

| Prop       | Type                                              | Default  |
| ---------- | ------------------------------------------------- | -------- |
| `message`  | `string`                                          | —        |
| `severity` | `'info' \| 'success' \| 'warning' \| 'error'`     | `'info'` |
| `duration` | `number` (ms)                                     | —        |
| `action`   | `ReactNode`                                       | —        |
| `position` | `{ vertical, horizontal }` (MUI `SnackbarOrigin`) | —        |
| `key`      | `string \| number`                                | —        |

`useSnackbar()` also exposes `success` / `error` / `warning` /
`info` / `close` shortcuts. Max-stack is configured at the
provider level (`maxSnackbars`).

### `<ConfirmDialog>`

`<ConfirmDialog>` is a controlled component — thread `open` /
`onClose` / `onConfirm` (there is no `useConfirmDialog` hook):

```tsx
import { ConfirmDialog } from '@omnitron-dev/prism';

function DeleteButton({ onDelete }: { onDelete: () => void }) {
  const [open, setOpen] = useState(false);

  return (
    <>
      <Button color="error" onClick={() => setOpen(true)}>Delete</Button>
      <ConfirmDialog
        open={open}
        onClose={() => setOpen(false)}
        onConfirm={onDelete}
        title="Delete project?"
        content="This cannot be undone."
        confirmLabel="Delete"
        cancelLabel="Keep"
        confirmColor="error"
      />
    </>
  );
}
```

`onConfirm` may be async — pass `loading` to show a spinner
while it resolves. A `<DeleteDialog>` preset is also exported.

### `<Tooltip>`

```tsx
import { Tooltip } from '@omnitron-dev/prism';

<Tooltip title="Restart this app" placement="top">
  <IconButton onClick={restart}>
    <RefreshIcon />
  </IconButton>
</Tooltip>
```

Standard MUI tooltip with Prism theme integration. Wrap a
single child element (not a fragment).

### `<LoadingScreen>`

Full-viewport loading bar — typical use is the Suspense fallback:

```tsx
import { LoadingScreen } from '@omnitron-dev/prism';

<Suspense fallback={<LoadingScreen />}>
  <Outlet />
</Suspense>
```

Pass `portal` to render through a portal, or override the
progress indicator via `slots.progress` / `slotProps.progress`.
`SplashScreen` (branded full-screen) and `Spinner` (inline) are
also exported.

### `<Skeleton>`

Animated placeholder while data loads.

```tsx
import { Skeleton, CardSkeleton, TableSkeleton }
  from '@omnitron-dev/prism';

{isLoading ? <Skeleton variant="rectangular" height={200} /> : <Chart type="line" series={series} />}
```

`<Skeleton>` wraps MUI's Skeleton (`variant`: `text` /
`circular` / `rectangular`). For composite placeholders use the
dedicated `<CardSkeleton>` and `<TableSkeleton>` exports.

### `<EmptyContent>`

Friendly empty-state card. Use on every list that may be empty.

```tsx
import { EmptyContent } from '@omnitron-dev/prism';

{items.length === 0 ? (
  <EmptyContent
    title="No projects yet"
    description="Create your first project to get started."
    action={<Button onClick={onCreate}>Create project</Button>}
    icon={<FolderIcon />}
  />
) : (
  <ItemList items={items} />
)}
```

`illustration` and `icon` both take a `ReactNode` (there are no
built-in named illustrations). `icon` is rendered inside a
circular coloured wrapper; `illustration` is rendered as-is.
`SearchEmptyContent` and `LoadingEmptyContent` are preset
variants.

### `<ErrorBoundary>`

`fallback` receives a single `FallbackProps` object
(`{ error, errorInfo, parsedStack, resetErrorBoundary }`):

```tsx
import { ErrorBoundary } from '@omnitron-dev/prism';

<ErrorBoundary
  fallback={({ error, resetErrorBoundary }) =>
    <ErrorScreen error={error} onReset={resetErrorBoundary} />}
  onError={(error, errorInfo) => reportToSentry(error, errorInfo)}
  resetKeys={[location.pathname]}
>
  <Outlet />
</ErrorBoundary>
```

Catches synchronous render errors. Doesn't catch async / event-
handler errors — those go through global window error handlers.
Omit `fallback` to use the built-in error screen (with dev-mode
stack details).

### `<LinearProgress>` / `<CircularProgress>` / `<ProgressBar>` / `<CountdownRing>`

```tsx
import { LinearProgress, CircularProgress, ProgressBar, CountdownRing }
  from '@omnitron-dev/prism';

<ProgressBar segments={[{ value: 75, color: 'success' }, { value: 25, color: 'info' }]} />
<LinearProgress />                         {/* indeterminate */}
<CircularProgress variant="determinate" value={50} />
<CountdownRing deadline={expiresAt} totalMs={30_000} />  {/* timed ring */}
```

Linear and circular variants; determinate and indeterminate
modes. (There is no single `<Progress>` export — pick the
specific component.)

`CountdownRing` counts down to a **deadline** (`Date` or ISO
string), not for a duration: it survives a remount and a
backgrounded tab, both of which a duration timer gets wrong.
`totalMs` is only the denominator for the filled fraction — omit
it and the ring starts full. It switches to `warningColor` below
`warningThresholdMs` and to `expiredColor` past the deadline.

## Data display

### `<Card>`

```tsx
import { Card, StatCard } from '@omnitron-dev/prism';

<Card title="Active users" subheader="last 30 days">
  <Typography variant="h3">1,234</Typography>
</Card>

<Card
  variant="outlined"
  title="Project Alpha"
  headerAction={<IconButton><MoreIcon /></IconButton>}
  actions={<Button>Open</Button>}
>
  <Typography variant="body2">Last updated 2 hours ago</Typography>
</Card>

{/* Pre-composed stat tile: */}
<StatCard label="Revenue" value="$48,200" change={12.5} />
```

`StatCard` takes `label` and `value` (both required), plus
`change` (a percentage, rendered with direction), `icon`,
`color`, `subtitle` and `loading`. It extends
`Omit<CardProps, 'children'>`, so `title` is a valid prop —
inherited from `Card` — but it is the *card header*, not the
stat. A tile given `title` and `total` renders a header with an
empty stat below it, which is why the wrong pair does not look
broken at a glance.

| Prop             | Type                                  | Default       | Notes                 |
| ---------------- | ------------------------------------- | ------------- | --------------------- |
| `variant`        | `'elevation' \| 'outlined' \| 'soft'` | `'elevation'` | Visual weight         |
| `title`          | `ReactNode`                           | —             | Card header title     |
| `subheader`      | `ReactNode`                           | —             | Card header subtitle  |
| `headerAction`   | `ReactNode`                           | —             | Top-right header slot |
| `actions`        | `ReactNode`                           | —             | Footer actions row    |
| `disablePadding` | `boolean`                             | `false`       | Drop content padding  |

Card extends MUI's `Card`, so MUI props (`onClick`, `elevation`,
`sx`, …) pass through. `CardSection` and `StatCard` are also
exported. Cards are typically arranged in a `<Grid>` — `<Card>`
itself doesn't handle layout.

### `<Avatar>`

```tsx
import { Avatar, CustomAvatarGroup } from '@omnitron-dev/prism';

<Avatar src={user.avatarUrl} name={user.name} size="md" />
<Avatar name="Alice" online />
<CustomAvatarGroup
  max={3}
  avatars={team.map(u => ({ name: u.name, src: u.avatarUrl }))}
/>
```

| Prop                 | Type                                                                      | Default      | Notes                             |
| -------------------- | ------------------------------------------------------------------------- | ------------ | --------------------------------- |
| `src`                | `string`                                                                  | —            | Image URL; falls back to initials |
| `name`               | `string`                                                                  | —            | Used for initials + alt text      |
| `size`               | `'xs' \| 'sm' \| 'md' \| 'lg' \| 'xl'`                                    | `'md'`       |                                   |
| `online` / `offline` | `boolean`                                                                 | —            | Status indicator dot              |
| `badge`              | `ReactNode`                                                               | —            | Corner badge (e.g. count)         |
| `badgeColor`         | `'primary' \| 'secondary' \| 'success' \| 'warning' \| 'error' \| 'info'` | —            |                                   |
| `shape`              | `'circular' \| 'rounded' \| 'square'`                                     | `'circular'` |                                   |

The fallback is **deterministic** — same name always produces
the same colour + initials, so users are visually identifiable
even without photos.

### `<Badge>`

`<Badge>` wraps MUI Badge (use `badgeContent` / `variant="dot"`).
For a count overlay, the `<CountBadge>` helper (with `count` /
`max`) is more ergonomic; `<StatusDot>` renders a standalone
status dot:

```tsx
import { Badge, CountBadge, StatusDot }
  from '@omnitron-dev/prism';

<Badge badgeContent={5}>
  <NotificationsIcon />
</Badge>

<Badge variant="dot" color="error">
  <Avatar src={src} />
</Badge>

<CountBadge count={150} max={99}>
  <ShoppingCartIcon />
</CountBadge>

<StatusDot status="online" pulse />
```

`max` caps the display value with a `+` suffix (`99+`);
`variant="dot"` shows a colour dot without a number.

### `<Table>`

The low-level table. For most app tables, prefer
[`<DataGridBlock>`](/docs/frontend/prism/blocks#datagridblock--filterable--sortable--paginated-table) or
`<AdminDataTable>` (re-exported from `components/admin-filters`)
which add filtering, sorting, pagination, row actions.

`<Table>` is config-driven — pass `columns` + `data` (it is not
a MUI-style composition of `<TableRow>` / `<TableCell>`):

```tsx
import { Table } from '@omnitron-dev/prism';

<Table
  rowKey="id"
  data={users}
  columns={[
    { id: 'name',  label: 'Name' },
    { id: 'email', label: 'Email' },
    { id: 'karma', label: 'Karma', align: 'right',
      render: (u) => u.karma },
  ]}
/>
```

Columns are `{ id, label, align?, sortable?, render?, format? }`.
The component also supports `selectable`, `sortable`, `paginated`,
and `loading` props (see `TableProps`). Both `Table` and `AdminDataTable` use a
bounded, keyboard-focusable `TableViewport` with a localisable `scrollLabel`.
Pagination remains outside the scroll region and adapts to the container width;
`maxHeight` limits the rows. Sticky header cells use an opaque theme surface.
Rows with `onRowClick` accept Enter and Space when the row itself is focused;
keyboard events from nested controls retain their original behavior.

Use `TableColumn.minWidth` or the table’s `sx` for readable columns.
`AdminDataTable` accepts `minWidth` for the table and `ColumnDef.minWidth` for
individual columns. For a composed MUI table, reuse the same viewport directly:

```tsx
import { TableViewport } from '@omnitron-dev/prism/components/table';
import MuiTable from '@mui/material/Table';

<TableViewport label="Deployment history" maxHeight={480}>
  <MuiTable stickyHeader sx={{ minWidth: 640 }}>{/* table sections */}</MuiTable>
</TableViewport>
```

Keep pagination outside `TableViewport`. It supplies the scroll region, not row
data, sorting or a second pagination implementation.

### `<Chart>`

ApexCharts wrapper with theme-aware defaults.

```tsx
import { Chart, useChart } from '@omnitron-dev/prism/components/chart';

function CpuChart({ series }: { series: TimeSeries[] }) {
  const options = useChart({
    chart:   { type: 'area', stacked: true },
    xaxis:   { type: 'datetime' },
    yaxis:   { labels: { formatter: (v) => `${v}%` } },
    legend:  { position: 'top' },
  });

  return <Chart options={options} series={series} type="area" height={320} />;
}
```

`useChart` merges your overrides into Prism-themed defaults
(colours, gridlines, tooltips, fonts). Always go through it
rather than passing raw ApexCharts options — keeps theming
consistent.

The standalone `components/chart` entry keeps charts out of an application's main
component barrel. The plotting library loads lazily. Numeric heights reserve the
plot's space during loading; the wrapper can shrink inside flex/grid workspaces.
Tooltips and legends inherit the active theme, including custom CSS-variable
prefixes. `prefers-reduced-motion` disables animations even if a caller enables them.

For operational time series, use straight lines, explicit units and local/UTC time
labels. `{ x, y: null }` represents missing samples; do not substitute zero or connect
long gaps. Provide an accessible name (`role="img"`, `aria-label`) and a tabular
alternative when exact values matter. Selection controls should be ordinary
keyboard-operable buttons outside the image. `ChartLegends` wraps long labels.

Supported `type` values: `line`, `area`, `bar`, `pie`, `donut`,
`radialBar`, `scatter`, `bubble`, `heatmap`, `candlestick`,
`boxPlot`, `radar`, `polarArea`, `rangeBar`, `rangeArea`,
`treemap`. (There is no `column` type — use `bar`.)

### `<Carousel>`

Slides are passed as **children** (not a `slides` array prop):

```tsx
import { Carousel } from '@omnitron-dev/prism';

<Carousel autoplay autoplayInterval={5000} dots arrows>
  {images.map((src) => <img key={src} src={src} />)}
</Carousel>
```

| Prop               | Type                   | Default |
| ------------------ | ---------------------- | ------- |
| `children`         | `ReactNode` (slides)   | —       |
| `autoplay`         | `boolean`              | `false` |
| `autoplayInterval` | `number` (ms)          | —       |
| `pauseOnHover`     | `boolean`              | —       |
| `loop`             | `boolean`              | —       |
| `dots`             | `boolean`              | —       |
| `arrows`           | `boolean`              | —       |
| `slidesToShow`     | `number`               | `1`     |
| `spacing`          | `number` (px)          | —       |
| `responsive`       | `CarouselBreakpoint[]` | —       |

### `<Lightbox>`

Drive open/index state with `useLightbox`, then spread
`getLightboxProps()` onto `<Lightbox>` (which takes a `slides`
array). There is no `controller` prop:

```tsx
import { Lightbox, useLightbox } from '@omnitron-dev/prism';

function Gallery({ images }: { images: string[] }) {
  const lightbox = useLightbox({ totalSlides: images.length });
  const slides = images.map((src) => ({ src }));
  return (
    <>
      {images.map((src, i) => (
        <img key={src} src={src} onClick={() => lightbox.onOpen(i)} />
      ))}
      <Lightbox slides={slides} {...lightbox.getLightboxProps()} />
    </>
  );
}
```

Keyboard navigation (arrows, Esc), zoom controls, thumbnails,
counter, optional download/share buttons.

### Map point picker

Choose a point on the local GeoJSON layer, pan the map, or clear the selection. The example runs with the packaged map worker and its own data.

### `<Image>`

```tsx
import { Image } from '@omnitron-dev/prism';

<Image
  src={user.avatarUrl}
  alt={user.name}
  ratio="1/1"
  fallbackSrc="/default-avatar.png"
  placeholder
/>
```

Adds: aspect-ratio container via `ratio` (no layout shift),
fallback on error (`fallbackSrc` URL or a `fallback` ReactNode),
lazy-load by default (disable with `disableLazy`), and an
optional loading `placeholder`. (`ratio` is the prop name — not
`aspectRatio`.)

### `<TagCloud>`

```tsx
import { TagCloud } from '@omnitron-dev/prism';

<TagCloud
  tags={[
    { id: '1', name: 'react',      slug: 'react',      postCount: 142 },
    { id: '2', name: 'typescript', slug: 'typescript', postCount: 98 },
    { id: '3', name: 'rpc',        slug: 'rpc',        postCount: 31 },
  ]}
  variant="cloud"
  onTagClick={(tag) => navigate(`/search?q=${tag.slug}`)}
/>
```

Each tag is `{ id, name, slug, postCount, … }`; the cloud sizes
tags proportional to `postCount`. `variant="list"` renders a
ranked list instead. `onTagClick` receives the full tag object.

## Navigation

### `<Breadcrumbs>`

```tsx
import { Breadcrumbs } from '@omnitron-dev/prism';

<Breadcrumbs
  heading="Alpha"
  links={[
    { name: 'Home',     href: '/' },
    { name: 'Projects', href: '/projects' },
    { name: 'Alpha' },               // current page — no href
  ]}
/>
```

| Prop            | Type                                               | Default | Notes                                 |
| --------------- | -------------------------------------------------- | ------- | ------------------------------------- |
| `links`         | `BreadcrumbLinkProps[]` (`{ name, href?, icon? }`) | —       | The crumb trail                       |
| `heading`       | `string`                                           | —       | Page heading above the crumbs         |
| `activeLast`    | `boolean`                                          | `false` | Keep the last crumb clickable         |
| `backHref`      | `string`                                           | —       | Shows a back arrow before the heading |
| `action`        | `ReactNode`                                        | —       | Right-side action slot                |
| `linkComponent` | `ElementType`                                      | —       | Router `Link` for client-side nav     |

Pass `linkComponent` (e.g. react-router's `Link`) to keep
breadcrumb links client-side.

### `<Menu>`

Dropdown menu. Pairs with `<IconButton>` or any trigger:

`<Menu>` is config-driven — pass `items` (a `MenuItemDef[]`), and
drive open/anchor state with the `useMenu` hook. There are no
`<MenuItem>` / `<MenuDivider>` child components:

```tsx
import { Menu, useMenu } from '@omnitron-dev/prism';

function UserMenu() {
  const menu = useMenu();
  return (
    <>
      <IconButton onClick={menu.handleOpen}><MoreIcon /></IconButton>
      <Menu
        {...menu.menuProps}
        items={[
          { key: 'profile',  label: 'Profile',  icon: <ProfileIcon />,  onClick: () => navigate('/me') },
          { key: 'settings', label: 'Settings', icon: <SettingsIcon />, onClick: () => navigate('/settings') },
          { type: 'divider', key: 'd1' },
          { key: 'signout',  label: 'Sign out', icon: <LogoutIcon />,   danger: true, onClick: signOut },
        ]}
      />
    </>
  );
}
```

Item shape: `{ key, label, icon?, onClick?, danger?, shortcut?, type? }`.
A `<ContextMenu>` variant is also exported for right-click menus.

### `<MegaMenu>`

Multi-column dropdown for sites with deep navigation. It is
config-driven — pass a `data` array (there is no `<MegaMenuColumn>`
child component). Use `MegaMenu` (auto), or the explicit
`MegaMenuHorizontal` / `MegaMenuVertical` / `MegaMenuMobile`
variants:

```tsx
import { MegaMenu } from '@omnitron-dev/prism';

<MegaMenu
  data={[
    {
      title: 'Products',
      path:  '/products',
      children: [
        { subheader: 'Backend',  items: [
          { title: 'Titan',   path: '/titan' },
          { title: 'Modules', path: '/modules' },
        ]},
        { subheader: 'Frontend', items: [
          { title: 'Prism',        path: '/prism' },
          { title: 'netron-react', path: '/netron-react' },
        ]},
      ],
    },
  ]}
/>
```

### `<NavCard>` / `<NavCardGrid>`

Compact cards for navigation hubs. `<NavCard>` navigates via
`to` + `linkComponent` (not `href`); `icon` is a `ReactNode`.
`<NavCardGrid>` auto-wraps based on `minTileWidth` (not a fixed
`columns` count):

```tsx
import { Link as RouterLink } from 'react-router-dom';

<NavCardGrid minTileWidth={260} gap={3}>
  <NavCard title="Apps"     to="/apps"     linkComponent={RouterLink} icon={<BoxIcon />}  description="Manage applications" />
  <NavCard title="Infra"    to="/infra"    linkComponent={RouterLink} icon={<GridIcon />} description="Containers + services" />
  <NavCard title="Settings" to="/settings" linkComponent={RouterLink} icon={<CogIcon />}  description="Per-user preferences" />
</NavCardGrid>
```

### `<NavSection>`

Sidebar navigation builder. The `<DashboardLayout>` uses this
internally; you can use it standalone:

Driven by a `data` prop — an array of sections, each with an
optional `subheader` and an `items` array (active state is
derived from the router automatically):

```tsx
<NavSection
  data={[
    {
      subheader: 'Main',
      items: [
        { title: 'Dashboard', path: '/', icon: 'home' },
        {
          title: 'Settings',
          path:  '/settings',
          icon:  'cog',
          children: [
            { title: 'Profile',  path: '/settings/profile' },
            { title: 'Security', path: '/settings/security' },
          ],
        },
      ],
    },
  ]}
/>
```

`NavSectionVertical` / `NavSectionHorizontal` / `NavSectionMini`
are the explicit layout variants.

### `<NavigationProgress>`

Top-of-page loading bar — shows during route transitions. Feed
it the current `pathname` (and optionally `search`) from the
router; it completes when those change:

```tsx
import { NavigationProgress } from '@omnitron-dev/prism';
import { useLocation } from 'react-router-dom';

const { pathname, search } = useLocation();

<NavigationProgress pathname={pathname} search={search} />
```

`delay` (default 100ms) avoids flashing on fast transitions.

### `<Stepper>`

`<Stepper>` is config-driven — pass a `steps` array (there is no
separate `<Step>` component):

```tsx
import { Stepper } from '@omnitron-dev/prism';

const [activeStep, setActiveStep] = useState(0);

<Stepper
  activeStep={activeStep}
  steps={[
    { label: 'Account', description: 'Email + password' },
    { label: 'Profile', description: 'Name + avatar' },
    { label: 'Plan',    description: 'Choose a tier' },
    { label: 'Confirm', description: 'Review & finish', optional: true },
  ]}
/>
```

Horizontal by default; pass `orientation="vertical"` for a side
stepper. `StepperActions` + `useStepper` cover next/back/reset
wiring.

### `<Tabs>` / `<TabPanel>`

There is no separate `<Tab>` component. Either pass a `tabs`
array of `{ value, label, content }`, or compose `<TabPanel>`
children — `<Tabs>` builds the tab strip from each panel's
`value` / `label` and renders the active one. `onChange`
receives the new value directly.

```tsx
import { Tabs, TabPanel } from '@omnitron-dev/prism';

// Config-driven:
<Tabs
  defaultValue="overview"
  onChange={(value) => console.log(value)}
  tabs={[
    { value: 'overview', label: 'Overview', content: <Overview /> },
    { value: 'logs',     label: 'Logs',     content: <Logs /> },
    { value: 'metrics',  label: 'Metrics',  content: <Metrics /> },
  ]}
/>

// Or with <TabPanel> children:
<Tabs defaultValue="overview">
  <TabPanel value="overview" label="Overview"><Overview /></TabPanel>
  <TabPanel value="logs"     label="Logs"><Logs /></TabPanel>
  <TabPanel value="metrics"  label="Metrics"><Metrics /></TabPanel>
</Tabs>
```

The `useTabs` hook manages active-value state for you when you
need it outside the component.

For individually closable tabs, set `closable: true` on each item and supply
`onClose(value)`. The close icon is part of that tab, and **Delete** requests
closure when the tab has focus. Clicking close does not select an inactive tab.
`closeDisabled` suspends closure while leaving selection available;
`closeLabel` supplies a localized accessible name. The same options work on
`TabPanel` children. Closure is a request: the caller owns confirmation and
removal, plus selection in controlled mode. Focus moves to the next available
tab after removal, or the preceding tab when closing the last one. When all tabs
are removed, the caller should focus its creation control or another logical
next action. Default tabs remain non-closable.

```tsx
import { Tabs } from '@omnitron-dev/prism/components/tabs';

<Tabs
  value={activeId}
  onChange={setActiveId}
  onClose={setPendingCloseId}
  tabs={sessions.map(session => ({
    value: session.id,
    label: session.label,
    closable: true,
    closeDisabled: saving,
    closeLabel: `Close ${session.label}`,
  }))}
  variant="scrollable"
  aria-label="Sessions"
/>
```

### `<ScrollSpy>` / `<ScrollSpyProvider>`

Wrap content in `<ScrollSpyProvider>`, mark each tracked region
with `<ScrollSpySection id="…">`, and read the in-view section
from the `useScrollSpy()` hook (`activeId`) to drive your own
nav. There is no all-in-one `<ScrollSpy items={…}>` nav widget:

```tsx
import { ScrollSpyProvider, ScrollSpySection, useScrollSpy }
  from '@omnitron-dev/prism';

function Toc() {
  const { activeId } = useScrollSpy();
  return (
    <aside>
      {['intro', 'setup', 'api'].map((id) => (
        <a key={id} href={`#${id}`} aria-current={activeId === id}>{id}</a>
      ))}
    </aside>
  );
}

<ScrollSpyProvider>
  <article>
    <ScrollSpySection id="intro">…</ScrollSpySection>
    <ScrollSpySection id="setup">…</ScrollSpySection>
    <ScrollSpySection id="api">…</ScrollSpySection>
  </article>
  <Toc />
</ScrollSpyProvider>
```

Tracks which section is in view via `activeId`. Used by docs
pages.

### `<ScrollToTop>`

Scrolls the window back to the top whenever the route changes —
pass the current `pathname` from the router:

```tsx
import { ScrollToTop } from '@omnitron-dev/prism';
import { useLocation } from 'react-router-dom';

<ScrollToTop pathname={useLocation().pathname} />
```

(This resets scroll on navigation — it is not a floating
"scroll to top" button. For that, see the `useBackToTop` hook.)

### `<Scrollbar>`

```tsx
import { Scrollbar } from '@omnitron-dev/prism';

<Scrollbar sx={{ maxHeight: 400 }}>
  <LongContent />
</Scrollbar>
```

Cross-browser styling of native scrolling. On a fine pointer, the thumb is
transparent while idle and appears on container hover or keyboard focus inside.
Visibility does not change the scrollport's size. Touch input and forced-colors
mode keep native controls discoverable.

`autoHide` defaults to `true`; set `autoHide={false}` for a persistent thumb.
`hideScrollbar` still explicitly hides the bar while preserving scrolling.
The shared CSS baseline gives nested overflow regions the same behaviour.

## Input & form

### `Field` (namespace: `Field.Text`, `Field.Select`, …)

`Field` is a **namespace** of react-hook-form-integrated inputs —
you use `<Field.Text>`, `<Field.Select>`, `<Field.Number>`, etc.,
not a single polymorphic `<Field>` element. Each field reads from
the surrounding react-hook-form `FormProvider` by `name`:

```tsx
import { Field } from '@omnitron-dev/prism';
import { useForm, FormProvider } from 'react-hook-form';

const methods = useForm();

<FormProvider {...methods}>
  <Field.Text   name="email"    label="Email"    type="email" />
  <Field.Text   name="password" label="Password" type="password" />
  <Field.Text   name="bio"      label="Bio"      multiline rows={4} />
  <Field.Select name="role"     label="Role" options={[
    { value: 'viewer', label: 'Viewer' },
    { value: 'admin',  label: 'Admin' },
  ]} />
</FormProvider>
```

Available members include `Field.Text`, `Field.Select`,
`Field.Checkbox`, `Field.Switch`, `Field.Number`, `Field.Radio`,
`Field.Autocomplete`, `Field.MultiSelect`, `Field.Rating`,
`Field.Slider`, `Field.DatePicker`, `Field.TimePicker`,
`Field.DateTimePicker`, `Field.Code`, `Field.Upload`,
`Field.Phone`, `Field.CountrySelect`, `Field.Editor`, and more.

Each text field extends MUI's `TextField` props plus:

| Prop                                                                   | Type              | Notes                                        |
| ---------------------------------------------------------------------- | ----------------- | -------------------------------------------- |
| `name`                                                                 | `string`          | Maps to the react-hook-form field (required) |
| `rules`                                                                | `RegisterOptions` | Per-field react-hook-form validation rules   |
| `label` / `multiline` / `rows` / `type` / `placeholder` / `helperText` | —                 | Inherited from MUI `TextField`               |

When you wrap the form in [`<SchemaProvider>`](/docs/frontend/prism/forms), Zod
schema messages flow into the fields (i18n-aware); `rules`
overrides per field.

### `<Label>`

A small status/category chip (not an HTML form `<label>`). Use
`color` + `variant` for status pills; `StatusLabel` and
`BooleanLabel` are presets:

```tsx
import { Label, StatusLabel, BooleanLabel }
  from '@omnitron-dev/prism';

<Label color="success" variant="soft">Active</Label>
<Label color="warning" startIcon={<ClockIcon />}>Pending</Label>
<BooleanLabel value={isEnabled} />
```

The isolated entry is `@omnitron-dev/prism/components/label`. Variants are
`soft` (default), `filled`, `outlined` and `inverted`. Colored soft labels
measure contrast against their translucent fill composited over the theme's
paper surface, rather than the paper color alone. They select a tone with at
least 4.5 contrast for small text, using Prism's extended tonal scale when
needed. This also handles presets with inverted dark-mode tonal roles. Inverted
labels use their white surface for the comparison.

Colors remain bound to the theme's CSS variables and custom prefix. Plain MUI
palettes without a suitable tone, or brand colors that can only be resolved by
the browser, fall back to semantic text. `sx` can override the presentation;
when changing the surface or text color, check the resulting contrast. Keep a
written state alongside color and use legible text sizes inside scaled
visualizations.

### `<SearchInput>`

Debounced search input with a clear button:

```tsx
import { SearchInput } from '@omnitron-dev/prism/components/search-input';

<SearchInput
  value={query}
  onChange={(v) => setQuery(v)}
  debounce={300}
  placeholder="Search projects…"
  aria-label="Search projects"
/>
```

When `debounce > 0`, `onChange` fires that many ms after the
user stops typing (it receives the raw string) — saves you
wiring `useDebouncedValue` per call site. A clear button is
built in. `aria-label` names the input independently of its placeholder;
`clearTooltip` names the clear button.

### `<DateRangePicker>`

`<DateRangePicker>` is a dialog driven by the
`useDateRangePicker` hook — spread the hook's return onto it
(dates are Day.js values):

```tsx
import { DateRangePicker, useDateRangePicker }
  from '@omnitron-dev/prism';

function Filter() {
  const range = useDateRangePicker();
  return (
    <>
      <Button onClick={range.onOpen}>{range.label || 'Pick dates'}</Button>
      <DateRangePicker {...range} variant="calendar" />
    </>
  );
}
```

`variant` is `'input'` (default) or `'calendar'`; pass
`translations` to localise the dialog. A `<DateRangeInput>` (the
two-field inline form) is also exported.

### `<DurationPicker>`

For "how long" inputs (TTLs, timeouts). The value is a duration
**in seconds** (`null` = permanent/indefinite):

```tsx
<DurationPicker
  value={seconds}
  onChange={(seconds) => setDuration(seconds)}
/>
```

`onChange` receives a `number | null` (seconds). Pass `labels`
to localise the unit strings.

### `<CountrySelect>`

ISO 3166 country dropdown with flags + search. It is
**asset-free** — the consumer supplies the `options` list (and
optionally a `getFlagSrc` resolver); Prism bundles no country
data:

```tsx
<CountrySelect
  value={country}                    // lowercase iso2, or null
  onChange={setCountry}              // receives lowercase iso2 | null
  options={countries}               // ReadonlyArray<CountryOption>
  getFlagSrc={(iso2) => `/flags/${iso2}.svg`}
/>
```

Value / change are the lowercase ISO 3166-1 alpha-2 code.

### `<Editor>` + `<TipTapRenderer>`

TipTap-based rich-text editor and read-only renderer. The
`toolbar` prop takes a **preset** (`'full' | 'standard' |
'compact' | 'minimal' | 'chat' | 'inline'`) or an explicit
`{ items: [...] }` config — not a bare array:

```tsx
import { Editor } from '@omnitron-dev/prism/components/editor';
import { TipTapRenderer } from '@omnitron-dev/prism';

// Edit mode (onChange receives a string — JSON by default, or HTML
// when format="html"):
<Editor
  value={content}
  onChange={setContent}
  format="json"
  toolbar={{ items: ['bold', 'italic', 'link', 'heading', 'bulletList', 'code'] }}
/>

// Read mode (e.g., displaying a saved post):
<TipTapRenderer content={post.content} />
```

Changes are delivered synchronously on the editor transaction; debounce expensive persistence or previews in the host, not the authoritative draft.

Storage format is TipTap's JSON document — portable across
edit/view, indexable for full-text search, safer than raw HTML.
(The renderer export is `TipTapRenderer` — note the capital T.)

### `<ContentRenderer>`

Renders mixed content (TipTap JSON, markdown string, or HTML
string) with consistent typography — it auto-detects the format
from the content shape:

```tsx
<ContentRenderer content={post.body} />
```

Props: `content`, `compact`, `className`, `sx`,
`markdownComponents`, and `routerLinkComponent` (pass a
router-aware `Link` to keep internal links client-side).

### `<CommandPalette>`

`Cmd+K` style command palette. It self-manages open state via
`triggerKey` (default `'k'` + the platform meta key) — just pass
`actions`:

```tsx
import { CommandPalette }
  from '@omnitron-dev/prism';

function App() {
  return (
    <>
      <Outlet />
      <CommandPalette
        actions={[
          { id: 'new-project', title: 'New project', icon: <PlusIcon />,   onSelect: () => navigate('/projects/new') },
          { id: 'sign-out',    title: 'Sign out',    icon: <LogoutIcon />, onSelect: signOut },
        ]}
      />
    </>
  );
}
```

Each action is `{ id, title, subtitle?, group?, keywords?, icon?,
shortcut?, onSelect }`. Fuzzy-matches by title + keywords; arrow
keys + Enter; Esc closes. Pass `open` / `onOpenChange` for
controlled mode.

### `<FilterToolbar>`

Filter toolbar above admin tables — `search`, `select`,
`multi-select`, `date-range`, `boolean`, `number-range` filter
types. State is centralised: declare `filters` (a
`FilterConfig[]`), hold a single `values` object, and update via
one `onChange`:

```tsx
import { FilterToolbar } from '@omnitron-dev/prism';

const [values, setValues] = useState({});

<FilterToolbar
  filters={[
    { key: 'status', type: 'multi-select', label: 'Status',
      options: [{ value: 'active', label: 'Active' }, { value: 'archived', label: 'Archived' }] },
    { key: 'tier', type: 'multi-select', label: 'Tier',
      options: [{ value: 'free', label: 'Free' }, { value: 'pro', label: 'Pro' }] },
    { key: 'q', type: 'search', label: 'Search' },
  ]}
  values={values}
  onChange={setValues}
  onReset={() => setValues({})}
  total={results.length}
/>
```

Each filter is `{ key, type, label, options?, … }`. The module
also exports `AdminDataTable`, `StatusChip`, and `AmountCell` for
admin table surfaces. (There is no `<AdminFilters>` export and no
built-in save-as-view.) Controls wrap within their available width. A `search`
filter grows to use spare space; select and range controls shrink before they
can overflow. Use one debounced search field per toolbar.

Date ranges store `<key>From` and `<key>To`; numeric ranges store `<key>Min` and
`<key>Max`. Updating a date range emits both ends in one `onChange`, preserving
other filter values. A partial range, including numeric zero, counts as active
and can be reset. `onReset` lets the host restore its own defaults.

## Layout & utility

### `<Drawer>`

Side panel — left, right, top, bottom:

```tsx
import { Drawer } from '@omnitron-dev/prism';

const [open, setOpen] = useState(false);

<Drawer
  open={open}
  onClose={() => setOpen(false)}
  anchor="right"
  width={480}           // px or any CSS width (for left/right anchors)
  title="Edit user"
  footer={<><Button onClick={() => setOpen(false)}>Cancel</Button>
            <Button variant="contained" onClick={save}>Save</Button></>}
>
  <UserForm user={user} />
</Drawer>
```

Use `width` for left/right anchors and `height` for top/bottom
(there is no preset `size` prop). Variants via the `variant` prop:

- `temporary` (default) — overlays, closes on backdrop click
- `persistent` — pushes content
- `permanent` — always visible

### `<PageContent>`

A vertical-stack content wrapper (consistent section `gap`,
optional `fill`). It does **not** render a title/breadcrumb
header — compose `<Breadcrumbs>` + a heading above it, or use a
layout. For breadcrumbs use the standalone `<Breadcrumbs>`
component (above):

```tsx
<PageContent gap={3}>
  <Breadcrumbs links={[{ name: 'Home', href: '/' }, { name: 'Projects' }]} />
  <ProjectsList />
</PageContent>
```

### `<DocLayout>`

For documentation pages — collapsible sidebar + main +
on-this-page nav. The right-rail "table of contents" is
`<DocSectionNav>` (passed via the `sectionNav` prop); there is no
`<TableOfContents>` component:

```tsx
import { DocLayout, DocSidebar, DocSectionNav }
  from '@omnitron-dev/prism';

<DocLayout
  sidebar={<DocSidebar items={navItems} />}
  sectionNav={<DocSectionNav headings={headings} />}
>
  <article>{content}</article>
</DocLayout>
```

### `<Accordion>`

`<Accordion>` is config-driven — pass an `items` array (there is
no `<Accordion.Item>` child). Single-open by default; pass
`multiple` to allow several panels open:

```tsx
import { Accordion } from '@omnitron-dev/prism';

<Accordion
  multiple
  items={[
    { id: 'overview',     title: 'Overview',     content: 'What it does.' },
    { id: 'installation', title: 'Installation', content: 'How to install.' },
  ]}
/>
```

Item shape: `{ id, title, subtitle?, content, icon?, disabled? }`.
A `<SimpleAccordion>` convenience wrapper is also exported.

### `<AnimateBorder>` / `<MotionLazy>`

The `animate` module exports `AnimateBorder` — a `Box` with an
animated gradient border — and `MotionLazy`, a provider that
lazy-loads framer-motion features (there is no generic
`<Animate type="fade-up">` entrance wrapper):

```tsx
import { AnimateBorder, MotionLazy }
  from '@omnitron-dev/prism';

// Animated gradient border around any content:
<AnimateBorder duration={8}>
  <Card>…</Card>
</AnimateBorder>

// Lazy-load framer-motion once near the app root:
<MotionLazy>
  <App />
</MotionLazy>
```

### `<SvgColor>`

Inlines an SVG and applies a colour via CSS mask (uses
`currentColor` by default — pass any CSS colour, e.g. a theme
token via `sx`):

```tsx
<SvgColor src="/icons/star.svg" sx={{ color: 'primary.main' }} size={20} />
```

Lets you tint icons without exporting per-colour copies.

### Settings drawer — `<SettingsProvider>` / `<SettingsDrawer>`

In-app settings drawer (theme mode, layout, colour preset).
Wrap the app in `<SettingsProvider>`, mount `<SettingsDrawer>`
once, and open it via the `useSettingsDrawer` hook — there is no
single `<Settings>` component with `open`/`onClose` props:

```tsx
import { SettingsProvider, SettingsDrawer, useSettingsDrawer }
  from '@omnitron-dev/prism';

function Header() {
  const drawer = useSettingsDrawer();
  return <IconButton onClick={drawer.onOpen}><SettingsIcon /></IconButton>;
}

<SettingsProvider>
  <App />
  <SettingsDrawer />
</SettingsProvider>
```

State is backed by the Prism settings store (`useSettingsStore`
from `@omnitron-dev/prism/state`) — persisted to localStorage.

### Changelog — `<ChangelogTimeline>` / `<ChangelogEntry>`

Renders a feed of changes (there is no single `<Changelog>`
component — compose a `<ChangelogTimeline>` of
`<ChangelogEntry>` children):

```tsx
import { ChangelogTimeline, ChangelogEntry }
  from '@omnitron-dev/prism';

<ChangelogTimeline>
  <ChangelogEntry version="1.4.0" date="2026-05-16">
    New dashboard — added project overview.
  </ChangelogEntry>
  <ChangelogEntry version="1.3.1" date="2026-05-10">
    Fixed logs — resolved tail buffering.
  </ChangelogEntry>
</ChangelogTimeline>
```

## Accessibility — across the catalog

Every interactive component:

- **Focus management** — `useFocusTrap` in modals/drawers,
  `useReturnFocus` on close.
- **Keyboard navigation** — full-featured for menus, tabs,
  steppers, command palette, lightbox.
- **ARIA roles** — correct `role` / `aria-*` attributes.
- **Screen-reader text** — `<VisuallyHidden>` for icon-only
  buttons and stateful indicators.
- **Reduced motion** — animations respect `prefers-reduced-motion`.

Buttons / inputs / selects inherit MUI v9's accessibility
foundation, which is WCAG 2.1 AA-compliant out of the box.

## Refs (React 19)

Every prism component exposes its underlying DOM element through
the standard `ref` prop. React 19 routes `ref` through props
directly — there is **no `forwardRef` wrapper** to layer through,
and consumer call sites do not need `React.forwardRef` either.

```tsx
import { useRef, useEffect } from 'react';
import { Card } from '@omnitron-dev/prism';

function ScrollIntoViewExample() {
  const cardRef = useRef<HTMLDivElement>(null);

  useEffect(() => {
    cardRef.current?.scrollIntoView({ behavior: 'smooth' });
  }, []);

  return <Card ref={cardRef}>…</Card>;
}
```

Components that expose an imperative handle attach the handle
type via a `ref?: Ref<HandleType>` field on their props interface
— identical call-site ergonomics, no `useImperativeHandle`
ceremony on the consumer side.

## MUI v9 slot props

For slot-bearing components (`<TextField>`, `<Select>`,
`<Autocomplete>`, …) prism re-exports MUI v9's `slotProps` API.
The legacy `InputProps={…}` / `InputLabelProps={…}` / `MenuProps={…}`
forms still work for backwards compatibility but emit a deprecation
warning in MUI v9 dev mode — migrate to `slotProps={{ input: { … } }}`
etc. as you touch each consumer.

```tsx
// ❌ Legacy (deprecated in MUI v9)
<TextField
  InputProps={{ endAdornment: <PasswordToggle /> }}
  FormHelperTextProps={{ sx: { ml: 0 } }}
/>

// ✅ MUI v9 idiom
<TextField
  slotProps={{
    input: { endAdornment: <PasswordToggle /> },
    formHelperText: { sx: { ml: 0 } },
  }}
/>
```

The same migration applies to Tabs (`TabIndicatorProps` →
`slotProps.indicator`), Modal/Drawer/Popover
(`BackdropProps` → `slotProps.backdrop`), and Autocomplete
(`componentsProps` → `slotProps`).

## Per-component subpaths

Each component is also importable from its own subpath:

```tsx
// Convenience (bigger bundle):
import { Card, Table, Drawer } from '@omnitron-dev/prism';

// Tree-shaken (smaller bundle):
import { Card }   from '@omnitron-dev/prism';
import { Table }  from '@omnitron-dev/prism';
import { Drawer } from '@omnitron-dev/prism';
```

Use subpaths in production for the leanest payload.

## See also

- [Blocks](/docs/frontend/prism/blocks) — full-page composites built from components
- [Layouts](/docs/frontend/prism/layouts) — shell components
- [Forms](/docs/frontend/prism/forms) — `<Field>` + schema-driven forms
- [Hooks catalog](/docs/frontend/prism/hooks-catalog) — 25+ React hooks
- [Theme](/docs/frontend/prism/theme) — colours, typography, dark mode

## Also exported

These are exported from `@omnitron-dev/prism/components` and had no entry
here. Listed with their real signatures rather than invented examples — what
was missing is that they exist.

### Lists and layout

| Component        | Signature                                                           | What it is                                                                                                                                                                                                                                                                                     |
| ---------------- | ------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `VirtualList<T>` | `{ items, getKey, children, estimateSize?, overscan?, maxHeight? }` | Windowed list over `@tanstack/react-virtual`. Measures rows as they mount, so **variable heights work** and a row may expand without breaking the scroll. `getKey` must be stable — it is what survives a reorder. Set `maxHeight` when the parent is not height-bounded, e.g. inside a Dialog |
| `CardGrid`       | `{ children, columns = 2, gap, sx }`                                | Wraps each direct child in a grid cell                                                                                                                                                                                                                                                         |
| `ContentRoot`    | styled `div`                                                        | The typographic container the content renderer paints into                                                                                                                                                                                                                                     |
| `DocSidebar`     | `{ items, activeId, onItemClick, title }`                           | Sidebar for a documentation-style layout                                                                                                                                                                                                                                                       |

### Content

| Component          | Signature                                | What it is                                                 |
| ------------------ | ---------------------------------------- | ---------------------------------------------------------- |
| `Markdown`         | `{ children, className, … }`             | Markdown renderer used by the content pages                |
| `Emoji`            | `{ native, set = 'twitter', size, alt }` | One emoji from a set, without loading the picker's dataset |
| `FlagIcon`         | `{ code, … }`                            | Country flag, as used by `Field.CountrySelect`             |
| `ImageWithDefault` | `(props: ImageProps)`                    | `<Image>` that falls back to an icon when the source fails |

### Feedback and overlays

| Component           | Signature                               | What it is                                                                                                                |
| ------------------- | --------------------------------------- | ------------------------------------------------------------------------------------------------------------------------- |
| `InlineAlert`       | `{ severity = 'info', message, icon }`  | Alert inside the flow rather than at the page edge. For form errors see the policy in [Forms](/docs/frontend/prism/forms) |
| `SimpleSnackbar`    | `{ … }`                                 | Snackbar without the provider — for a surface that has no `<PrismProvider>` above it                                      |
| `CustomPopover`     | `{ … }`                                 | The popover `usePopover()` is built for                                                                                   |
| `CustomAvatarGroup` | `{ avatars, size = 'sm', max = 4, sx }` | Overlapping avatars with an overflow count                                                                                |

### Charts

| Component      | Signature                 | What it is                             |
| -------------- | ------------------------- | -------------------------------------- |
| `ChartLegends` | `{ … }`                   | Legend rows, usable apart from a chart |
| `ChartLoading` | `{ sx, className, type }` | Chart-shaped skeleton                  |

### Navigation internals

`NavSection`, `NavSectionVertical`, `NavSectionHorizontal` and `NavSectionMini`
are the navigation API and are documented above. The parts they are assembled
from — `Nav`, `NavList`, `NavSubList`, `NavItem`, `NavItemBase`, `NavUl`,
`NavLi`, `NavIcon`, `NavTexts`, `NavInfo`, `NavArrow`, `NavSubheader` — are
exported so a project can build a navigation shape the four presets do not
cover. Reach for them only then; the presets carry the accessibility
behaviour.

### Elsewhere

`Map`, `MapMarker`, `MapPointPicker` and `MapCoverageLayer` have their own
page: [Maps](/docs/frontend/prism/maps). The `Field.*` inputs are in [Forms](/docs/frontend/prism/forms).

## Research documents

`@omnitron-dev/prism/components/research` adds a source-preserving Markdown → typed document → React path. Import `@omnitron-dev/prism/components/research/styles.css` once. It supports GFM, stable Unicode heading anchors, admonitions, tabs, KaTeX and strict Mermaid diagrams. `parseResearchMarkdown(source, documentId)` returns the document tree, heading outline and diagnostics; node attributes retain source positions.

```tsx
import { ResearchDocument, defineResearchWidget } from '@omnitron-dev/prism/components/research';
import '@omnitron-dev/prism/components/research/styles.css';
import { z } from 'zod';

const props = z.object({ value: z.number().finite() }).strict();
const widgets = {
  quantity: defineResearchWidget(1, input => props.parse(input),
    ({ value }) => <output aria-label="Quantity">{value}</output>),
};
<ResearchDocument source={markdown} documentId="paper-overview" widgets={widgets} contents />;
```

Use `$...$` / `$$...$$` for math, a `mermaid` fence for diagrams, and a `prism` fence containing `{"type":"quantity","version":1,"props":{"value":3}}` for a registered widget. `:::note`, `:::warning`, and `:::tabs` with nested `:::tab{label="..."}` use directive syntax. Custom widgets are ordinary host-owned React components with explicit versioned prop validation. The source cannot import JavaScript, register components or grant access. Registries belong to each document, not to process-wide state. `TipTapRenderer` also exposes `nodeRenderers` and `inheritRegisteredNodes`; existing global registrations remain supported by default, while `ResearchDocument` disables inheritance.

Raw HTML and unsupported constructs remain inert with source diagnostics. Unsupported diagram engines retain their source; Pintora is not interpreted as Mermaid. TeX runs with trust disabled and expansion/size limits. Mermaid source cannot change renderer configuration; rendered SVG is sanitized and displayed as an isolated SVG image, preserving the layout styles without inserting document-controlled CSS into the page. Active elements and links are removed; the browser image boundary disables scripting and external resources. Theme changes are serialized. Source/AST/block/edge limits bound accepted input, **not execution time**: Mermaid runs on the browser main thread. Host-owned widgets must independently obey the host's accessibility, resource, network and permission policies. This renderer provides presentation, not proof checking or Noesis semantics. The host owns immutable revision identity, digest verification, authorization and publication. Unknown or failing blocks preserve source rather than masquerading as successful verification.

### `<ResearchEditor>`

A block-oriented scientific editor using Tiptap/ProseMirror and the same
Markdown, math, diagram and widget contract as `ResearchDocument`.

```tsx
import { useRef, useState } from 'react';
import {
  ResearchEditor, type ResearchEditorHandle,
} from '@omnitron-dev/prism/components/research-editor';
import '@omnitron-dev/prism/components/research-editor/styles.css';

function PaperEditor({ edition, source, save }) {
  const editor = useRef<ResearchEditorHandle>(null);
  const [dirty, setDirty] = useState(false);
  return <>
    <ResearchEditor key={edition} ref={editor} initialSource={source}
      maxSourceLength={20000}
      onChange={snapshot => setDirty(snapshot.getSource() !== source)} />
    <button disabled={!dirty} onClick={() => save(editor.current!.getSnapshot().getSource())}>
      Save edition
    </button>
  </>;
}
```

`initialSource` is intentionally initial, not a controlled string on every
keystroke. Replace the React `key` when loading another edition or explicitly
reconciling a conflict. `onChange` receives a synchronous immutable
`ResearchSnapshot` (`format`, `blocks`, `getSource()`); a save operation does not
wait for a debounced update. Persist the returned Markdown, not internal
ProseMirror JSON. The host owns revisions, authorization, save requests,
conflicts and collaboration.

| Prop                      | Purpose                                                                                               |
| ------------------------- | ----------------------------------------------------------------------------------------------------- |
| `initialSource`           | Initial Markdown. Original source is retained for unchanged blocks.                                   |
| `onChange`                | Transaction snapshot; retain the latest snapshot in a ref to avoid rerendering the whole application. |
| `ref`                     | `getSnapshot()` and `focus()`.                                                                        |
| `readOnly`                | Locks both controls and the editable surface.                                                         |
| `maxSourceLength`         | Source length limit in UTF-16 code units, bounded by the research parser's global limit.              |
| `labels`, `previewLabels` | Editor controls and shared scientific preview labels.                                                 |
| `widgets`                 | Explicit, scoped `ResearchWidgets` registry.                                                          |
| `insertions`              | Additional `{ id, label, source, keywords? }` block recipes supplied by the host.                     |
| `blockControls`           | Show one contextual block gutter (default `true`). Keyboard commands remain available when hidden.    |
| `className`, `sx`         | Theme-compatible presentation customization.                                                          |

Type `/` in an empty paragraph to open the insertion palette. Headings, lists,
task lists, code, inline equations and plain GFM tables have editable structure.
Block equations, diagrams, registered widgets and unsupported syntax preserve
source as atomic blocks with a scientific preview and a source dialog. The
contextual gutter offers insertion and a block menu at the hovered or selected
block. Drag the grip to reorder; use the menu on touch devices. Ctrl/Command +
Alt + Up/Down moves the current block without a pointer. All three paths share
identity-based ProseMirror transactions, reject stale identities, and preserve
the cursor inside a moved paragraph. Each discrete block operation is an
independent undo step, separated from adjacent typing. `blockMenu` and `dragBlock`
labels are localizable. Only one gutter is mounted for the document; scientific
content is not copied into a second editable DOM tree. Whole-document and
block-source edits are explicit operations. Switching to `readOnly` closes open
editing dialogs and locks transaction commands. Unsupported table spans and
over-budget operations are rejected,
not silently flattened or truncated.

Untouched blocks preserve original line endings and syntax. Edited supported
blocks are serialized with Markdown escaping. This is **not** a claim of exact
byte preservation after every edit: HTML clipboard transfer can canonicalize
ordinary Markdown. Formula atoms, opaque scientific source and supported
formatting are retained. Reference-style links, aligned tables and unknown
constructs remain source blocks rather than being destructively approximated.
Session block IDs are not stable publication or citation IDs; keep immutable
edition identity and source locators in the application's model.

Scientific previews are activated near the viewport. React does not rerender
the parent document for each transaction; unchanged block serialization is
cached by immutable ProseMirror node. Serialization and structural validation
still have costs proportional to the document, and React node views have a
cost. Measure browser input latency separately from parsing or serialization;
no constant-time editing or arbitrary document-size guarantee is made.

### `<KnowledgeGraph>`

A configurable React Flow surface for typed knowledge relations. The data
model is independent of React Flow node coordinates. A relation is a first-class
node with ordered, named input and output occurrences; it is not flattened into
unrelated pairwise edges.

```tsx
import { KnowledgeGraph } from '@omnitron-dev/prism/components/knowledge-graph';
import '@omnitron-dev/prism/components/knowledge-graph/styles.css';

const knowledge = {
  entities: [
    { id: 'premise-a', kind: 'assumption', label: 'A' },
    { id: 'premise-b', kind: 'assumption', label: 'B' },
    { id: 'claim', kind: 'claim', label: 'C' },
  ],
  relations: [{
    id: 'derivation', kind: 'candidate derivation', label: 'Joint premises',
    inputs: [{ entity: 'premise-a', role: 'first' }, { entity: 'premise-b', role: 'second' }],
    outputs: [{ entity: 'claim', role: 'conclusion' }],
  }],
};
<KnowledgeGraph data={knowledge} visibleBudget={300}
  onSelect={selection => inspect(selection)}
  renderDetails={selection => <ObjectDetails selection={selection} />} />;
```

The component does not certify the relation. The host determines whether it
represents citation, hypothesis, derivation or another relation. Repeated
participants in different roles are preserved, and cycles are allowed. Input
validation rejects duplicate identities and dangling references. Entity and
relation IDs have independent namespaces.

| Prop                | Purpose                                                                     |
| ------------------- | --------------------------------------------------------------------------- |
| `data`              | `KnowledgeGraphData` with entities and relations.                           |
| `visibleBudget`     | Maximum visible entity + relation nodes; default 300.                       |
| `labels`            | Localized controls, instructions and diagnostics.                           |
| `onSelect`          | Entity or relation selection, including clearing.                           |
| `layout`            | Host-supplied coordinates for a `KnowledgeProjection`.                      |
| `onLayoutChange`    | Visual coordinates after a drag; not a semantic graph mutation.             |
| `renderDetails`     | Host inspector for the selected object.                                     |
| `height`            | Map height, default 560px.                                                  |
| `edgeLabelGap`      | Minimum canvas-space gap between measured role labels and nodes; default 8. |
| `edgeLabelMaxWidth` | Maximum label width before wrapping, in canvas units; default 240.          |
| `className`, `sx`   | Styling and theme integration.                                              |

The searchable index and map share selection. The expand control opens an accessible full-screen workspace while retaining selection and layout; Escape closes it. Keyboard-accessible index
controls and a list view provide alternatives to spatial navigation. A bounded
neighbourhood admits a relation only together with all its endpoints. The UI
reports omitted relations explicitly. A large graph starts with an index from
which the user chooses a neighbourhood; an absent edge on that view does not
mean an absent relationship. Pass only authorized data: this presentation
budget is not access control, a storage limit or a substitute for server-side
pagination. The default grid is a deterministic starting layout, not a causal
ordering. Use `entityNodeId` and `relationNodeId` when supplying coordinates.

Role labels share one measured placement layer. Every ordered occurrence keeps
its full text and identity, including repeated participants. Prism measures the
rendered label box, wraps long roles, and places boxes clear of other labels and
nodes. Shifted labels have a dotted leader to their own path midpoint. The
placement is deterministic for the measured geometry; it does not assert a
globally minimal layout or encode semantic priority through position.

The component recomputes placement when paths, text, fonts or node dimensions
change. Pan and zoom transform the same world geometry. The fit control includes
labels outside the node bounds; subsequent edits preserve the user's viewport.
All admitted edges remain mounted so viewport culling cannot silently discard
roles or change their placement. `visibleBudget` still limits the admitted
incidence projection. Select a line to inspect its relation and its complete
ordered port list; the searchable index remains a keyboard alternative.

### Engine choices and scope

The scientific editor extends the existing Tiptap/ProseMirror stack, avoiding
a second incompatible document model. BlockNote provides a ready-made block
experience on ProseMirror; Lexical provides another editor state and plugin
system; Slate/Plate, Milkdown, Editor.js and Quill offer different integration
contracts. None is automatically faster for this workload. The selected path
keeps Prism's host-owned MUI design, Markdown preservation and scientific
rendering under one contract. Tiptap's
[performance guidance](https://tiptap.dev/docs/guides/performance) informed
transaction subscriptions and isolated React rendering.

React Flow is used for inspectable interactive relation diagrams. Its
[performance guidance](https://reactflow.dev/learn/advanced-use/performance)
informs stable node components, memoized projections and bounded views.
Canvas or WebGL renderers such as Cytoscape or Sigma remain possible host-level
choices for aggregate overviews after measuring actual scale. The current
component is not an unrestricted million-node canvas or a semantic reasoning
engine. Both components support application-owned labels, data, policies and
extension registries; neither requires a particular domain backend.

The Prism research editor is an independently implemented interface on the open
Tiptap/ProseMirror core. It does not include the paid Tiptap Notion template.
Block interaction style is not a storage format or a proof model. Real-time
collaboration, durable block identities, offline drafts, Noesis semantic commands
and proof checking require explicit host protocols and are not implied by the
block controls. The browser acceptance suite covers actual dragging, keyboard
movement, narrow dark layouts, accessibility and exact serialized source snapshots.
A separate 2000-block typing sample records beforeinput-to-animation-frame timings;
these are local diagnostic measurements, not field INP or a fastest-editor claim.

## Opt-in component imports

Console imports its components through dedicated package entry points. Each entry has both JavaScript and declarations in the published build, so applications can use navigation, feedback and data tables without loading maps or rich-text editors.

```tsx
import { Breadcrumbs } from '@omnitron-dev/prism/components/breadcrumbs';
import { SnackbarProvider, useSnackbar } from '@omnitron-dev/prism/components/snackbar';
import { LoadingScreen } from '@omnitron-dev/prism/components/loading-screen';
import { AdminDataTable } from '@omnitron-dev/prism/components/admin-filters';
```

The following component groups are independently importable:

`@omnitron-dev/prism/components/admin-filters`, `@omnitron-dev/prism/components/alert`, `@omnitron-dev/prism/components/animate`, `@omnitron-dev/prism/components/breadcrumbs`, `@omnitron-dev/prism/components/card`, `@omnitron-dev/prism/components/chart`, `@omnitron-dev/prism/components/command-palette`, `@omnitron-dev/prism/components/confirm-dialog`, `@omnitron-dev/prism/components/empty-content`, `@omnitron-dev/prism/components/error-boundary`, `@omnitron-dev/prism/components/label`, `@omnitron-dev/prism/components/loading-screen`, `@omnitron-dev/prism/components/navigation-progress`, `@omnitron-dev/prism/components/page-content`, `@omnitron-dev/prism/components/popover`, `@omnitron-dev/prism/components/search-input`, `@omnitron-dev/prism/components/scroll-to-top`, `@omnitron-dev/prism/components/skeleton`, `@omnitron-dev/prism/components/snackbar`, `@omnitron-dev/prism/components/stepper`, `@omnitron-dev/prism/components/table`, `@omnitron-dev/prism/components/tabs`.

Use `@omnitron-dev/prism/core` for `PrismProvider` and its core hooks, `@omnitron-dev/prism/layouts` for layout composition, `@omnitron-dev/prism/state` for stores and `@omnitron-dev/prism/auth` for guards. Maps and editors retain their existing explicit subpaths and can be loaded only on the routes that need them. The root barrel remains compatible, but opt-in imports give a consumer tighter control of its dependency graph.

---

---
id: "frontend/prism/forms"
title: "Forms"
description: "Schema-aware forms with react-hook-form + zod + Prism Field."
---

# Forms

Prism's form layer is **react-hook-form** + **zod**, plus a `Field`
namespace of RHF-bound input components and an optional i18n provider
(`SchemaProvider`) for localized validation messages.

There is **no** schema-driven field inference. You write a zod schema,
wire it into `useForm` via `zodResolver`, and place `Field.*` components
addressed by `name`. Validation runs through the resolver; each `Field`
reads its own error from react-hook-form state and renders it inline.

## The pieces

1. **A zod schema** — the source of truth for shape + validation, wired
   in via `zodResolver`.
2. **react-hook-form** — `useForm` + `FormProvider` own the form state.
   `Field.*` components call `useFormContext()` internally, so they must
   be rendered inside a `<FormProvider>`.
3. **`Field.*` components** — RHF-bound inputs from the `Field`
   namespace, addressed by `name`.

`Field` is exported from the package root and from the `components`
subpath. `zodResolver` comes from `@hookform/resolvers/zod` — it is a
dependency of Prism but is **not** re-exported, so import it directly
(this is the same import Prism's own blocks use):

```tsx
import { Field } from '@omnitron-dev/prism/components';
import { FormAlert } from '@omnitron-dev/prism';
import { FormProvider, useForm } from 'react-hook-form';
import { zodResolver } from '@hookform/resolvers/zod';
import { z } from 'zod';

const SignInSchema = z.object({
  email:    z.string().email(),
  password: z.string().min(8).max(128),
  remember: z.boolean().optional(),
});

type SignInInput = z.infer<typeof SignInSchema>;

function SignInForm() {
  const form = useForm<SignInInput>({
    resolver:      zodResolver(SignInSchema),
    defaultValues: { email: '', password: '', remember: false },
  });

  const onSubmit = (data: SignInInput) => authService.signIn(data);

  return (
    <FormProvider {...form}>
      <form onSubmit={form.handleSubmit(onSubmit)}>
        {form.formState.errors.root && (
          <FormAlert>{form.formState.errors.root.message}</FormAlert>
        )}
        <Field.Text     name="email"    label="Email"    type="email" />
        <Field.Text     name="password" label="Password" type="password" />
        <Field.Checkbox name="remember" label="Remember me" />
        <Button type="submit" disabled={form.formState.isSubmitting}>
          Sign in
        </Button>
      </form>
    </FormProvider>
  );
}
```

> **FormAlert takes the message as children**
>
> `<FormAlert>` does not accept an `error` prop. The message is its
> `children`. react-hook-form errors are `FieldError` objects, so pass
> `errors.root.message` (a string), not the error object itself. See
> [Error display](#error-display) below.

## Localized validation messages (optional)

`SchemaProvider` does **not** take a `schema` prop and does **not** drive
field rendering. It supplies an i18n context — a translate function `t`,
a `locale`, and optional `messages` overrides — that produces a set of
pre-translated zod schema builders. Read them with `useSchema()`, which
returns `{ t, locale, schema }`, and compose your form schema from
`schema.*` helpers so validation errors come out localized.

```tsx
import { SchemaProvider, useSchema } from '@omnitron-dev/prism/forms';
import { useTranslation } from 'react-i18next';

function App() {
  const { t, i18n } = useTranslation();
  return (
    <SchemaProvider t={t} locale={i18n.language}>
      <SignInForm />
    </SchemaProvider>
  );
}

// inside a form, build the schema with translated messages:
function useSignInSchema() {
  const { schema } = useSchema();           // throws if no provider
  return z.object({
    email:    schema.email(),               // localized "required"/"invalid"
    password: schema.password({ minLength: 8 }),
  });
}
```

The `schema` helper exposes builders such as `email()`, `phone()`,
`date()` / `dateOptional()`, `boolean()` / `booleanOptional()`,
`editor()`, `file()` / `fileOptional()` / `files()`, `url()` /
`urlOptional()`, `password()`, `required()`, `sliderRange()`,
`nullableInput()`, and `confirm()` — each accepting per-call message
overrides. Without an i18n library you can still get default English
messages via the standalone factory:

```ts
import { createSchemaUtils } from '@omnitron-dev/prism/forms';

const schema = createSchemaUtils();          // English defaults
const userSchema = z.object({ email: schema.email() });
```

`useSchemaOptional()` is the non-throwing variant — it returns `null`
outside a provider, for components that may render either way.

## Field types

Each control is a member of the `Field` namespace. Every field takes a
`name` (the RHF path), reads its error from react-hook-form, and renders
the error message inline below the control. Most pass extra props through
to the underlying MUI component; text-like fields also accept an optional
`rules` prop (react-hook-form `RegisterOptions`) for inline validation.

```tsx
<Field.Text   name="title"    label="Title" />
<Field.Text   name="bio"      label="Bio" multiline rows={4} />
<Field.Text   name="email"    label="Email" type="email" />
<Field.Text   name="password" label="Password" type="password" />
<Field.Number name="age"      label="Age" min={0} max={150} />
<Field.Phone  name="phone"    label="Phone" defaultCountry="US" />
<Field.DatePicker name="dob"  label="Date of birth" />
<Field.Checkbox   name="active"     label="Active" />
<Field.Switch     name="newsletter" label="Subscribe" />
<Field.Select name="role" label="Role" options={[
  { value: 'admin', label: 'Admin' },
  { value: 'user',  label: 'User' },
]} />
<Field.Radio name="plan" label="Plan" options={[
  { value: 'free', label: 'Free' },
  { value: 'pro',  label: 'Pro' },
]} />
<Field.MultiSelect name="tags" label="Tags" options={tagOptions} />
<Field.Code   name="otp" length={6} />
<Field.Upload name="avatar" />
```

The full namespace (`packages/prism/src/components/field/index.ts`):

| Member                 | Component                  | Notes                                                     |
| ---------------------- | -------------------------- | --------------------------------------------------------- |
| `Field.Text`           | text input                 | MUI `TextField`; `type`, `multiline`, `rows` pass through |
| `Field.Number`         | numeric input              | `min`/`max`/`decimals`/`allowNegative`; stores a `number` |
| `Field.Select`         | dropdown                   | `options: { value, label, disabled? }[]` + `placeholder`  |
| `Field.Checkbox`       | checkbox                   | `label`, `helperText`, `indeterminate`                    |
| `Field.Switch`         | toggle                     | `label`, `labelPlacement`                                 |
| `Field.Radio`          | radio group                | `options: { value, label, disabled? }[]`, `row`           |
| `Field.Autocomplete`   | searchable select          | `options`, `loading`, `noOptionsText`                     |
| `Field.MultiSelect`    | multi-select with chips    | `options`, `maxSelections`, `limitTags`                   |
| `Field.Rating`         | star rating                | `showValue`                                               |
| `Field.Slider`         | range slider               | `showValue`, `unit`, `formatValue`                        |
| `Field.DatePicker`     | date picker (MUI X)        | stores ISO string; `minDate`/`maxDate`/`disablePast`      |
| `Field.TimePicker`     | time picker (MUI X)        |                                                           |
| `Field.DateTimePicker` | combined date-time (MUI X) |                                                           |
| `Field.MultiCheckbox`  | checkbox group             |                                                           |
| `Field.MultiSwitch`    | switch group               |                                                           |
| `Field.Code`           | OTP / PIN input            | `length`, `type`                                          |
| `Field.Upload`         | dropzone upload            | `multiple`, `thumbnail`; value is `File`/`File[]`/URL     |
| `Field.UploadBox`      | compact upload box         |                                                           |
| `Field.UploadAvatar`   | circular avatar upload     |                                                           |
| `Field.Phone`          | international phone        | `defaultCountry`, `preferredCountries`                    |
| `Field.CountrySelect`  | country selector           |                                                           |
| `Field.Editor`         | rich text editor           | textarea fallback or custom                               |
| `Field.CustomEditor`   | headless editor            | render-prop                                               |

(Select / Radio / MultiSelect / Autocomplete take an `options` array;
they do **not** take `<MenuItem>` children.)

For inputs without a `Field.*` wrapper, drop to `<Controller>` (or
`useController`) and wire the field manually:

```tsx
import { Controller } from 'react-hook-form';
import { DateRangePicker } from '@omnitron-dev/prism';

<Controller
  name="range"
  control={form.control}
  render={({ field, fieldState }) => (
    <DateRangePicker
      value={field.value}
      onChange={field.onChange}
      error={fieldState.error?.message}
    />
  )}
/>
```

## Validation modes

```typescript
useForm({
  resolver: zodResolver(Schema),
  mode:     'onSubmit',    // default — validate on submit only
  // 'onChange'   — validate every keystroke
  // 'onBlur'     — validate when field loses focus
  // 'onTouched'  — onChange after first blur (recommended for sign-in)
  // 'all'        — onChange + onBlur
});
```

Recommendations:

- **`onSubmit`** — short forms, low-friction UX
- **`onTouched`** — most general-purpose; users see errors only
  after engaging with a field
- **`onChange`** — only for password-strength meter or live
  preview cases

## Error display

### Inline per-field

Every `Field.*` component renders its error message automatically below
the input when `form.formState.errors[name]` is set — it reads
`fieldState.error` from the internal `<Controller>` and feeds the message
into the MUI control's `helperText` (with `error`, `aria-invalid`, and
`aria-describedby` wired up). No extra markup needed.

### Form-level

For root errors (server-side rejections, business-rule failures), use
`<FormAlert>`. It is the platform's canonical inline surface for
form-level errors (per the Error-UX policy: form failures inline via
`FormAlert`, background events via toast). The message is passed as
**children**:

```tsx
const submit = async (data: SignInInput) => {
  try {
    await authService.signIn(data);
  } catch (e) {
    form.setError('root', {
      type:    'server',
      message: e instanceof InvalidTokenError ? 'Invalid credentials' : e.message,
    });
  }
};

return (
  <form onSubmit={form.handleSubmit(submit)}>
    {form.formState.errors.root && (
      <FormAlert onClose={() => form.clearErrors('root')}>
        {form.formState.errors.root.message}
      </FormAlert>
    )}
    {/* fields */}
  </form>
);
```

`<FormAlert>` props (`packages/prism/src/components/alert/alert.tsx`):

| Prop         | Type                                          | Default   | Notes                                              |
| ------------ | --------------------------------------------- | --------- | -------------------------------------------------- |
| `children`   | `ReactNode`                                   | —         | The alert body — the error message goes here       |
| `severity`   | `'error' \| 'warning' \| 'info' \| 'success'` | `'error'` |                                                    |
| `title`      | `ReactNode`                                   | —         | Optional heading above the message                 |
| `onClose`    | `() => void`                                  | —         | Renders a close (×) button; should clear the error |
| `autoScroll` | `boolean`                                     | `true`    | Scrolls the alert into view on mount               |
| `sx`         | `SxProps`                                     | —         | Style overrides                                    |

It sets `role="alert"` + `aria-live="assertive"` so screen readers
announce the message immediately, and auto-scrolls into view so an error
below the fold isn't missed.

**Inline form errors, not toasts** — toasts disappear; the user needs
the error visible while they fix the field.

### Server validation merging

```tsx
catch (e) {
  if (e instanceof ValidationError && e.fieldErrors) {
    for (const [field, message] of Object.entries(e.fieldErrors)) {
      form.setError(field as any, { type: 'server', message });
    }
  } else {
    form.setError('root', { message: 'Something went wrong' });
  }
}
```

Per-field server errors land on the matching `Field.*` exactly like
client-side errors — no special UI path; the field renders them inline.

## Submit state

```tsx
const { formState: { isSubmitting, isValid, isDirty } } = form;

<Button
  type="submit"
  disabled={isSubmitting || !isValid || !isDirty}
  loading={isSubmitting}
>
  Save
</Button>
```

| State                | Meaning                                  |
| -------------------- | ---------------------------------------- |
| `isSubmitting`       | Submission in flight                     |
| `isValid`            | Current values pass schema               |
| `isDirty`            | At least one field changed from defaults |
| `isSubmitted`        | Form has been submitted at least once    |
| `isSubmitSuccessful` | Last submit didn't throw                 |
| `submitCount`        | Total submissions                        |

## Dynamic forms

### Conditional fields

```tsx
const role = form.watch('role');

<>
  <Field.Select name="role" label="Role" options={[
    { value: 'admin', label: 'Admin' },
    { value: 'user',  label: 'User' },
  ]} />

  {role === 'admin' && (
    <Field.Text name="adminScope" label="Admin scope" />
  )}
</>
```

The schema must accept the conditional shape. `z.discriminatedUnion`
is the clean way:

```typescript
const Schema = z.discriminatedUnion('role', [
  z.object({ role: z.literal('admin'), adminScope: z.string() }),
  z.object({ role: z.literal('user') }),
]);
```

### Field arrays

`Field.*` components address array members by index path
(`items.0.label`, `items.1.label`, …), so they compose directly with
react-hook-form's `useFieldArray`:

```tsx
import { useFieldArray } from 'react-hook-form';

const { fields, append, remove } = useFieldArray({
  control: form.control,
  name:    'items',
});

<>
  {fields.map((field, index) => (
    <div key={field.id}>
      <Field.Text name={`items.${index}.label`} label="Label" />
      <Field.Text name={`items.${index}.value`} label="Value" />
      <IconButton onClick={() => remove(index)}>
        <DeleteIcon />
      </IconButton>
    </div>
  ))}
  <Button onClick={() => append({ label: '', value: '' })}>
    Add row
  </Button>
</>
```

The schema for the array:

```typescript
items: z.array(z.object({
  label: z.string().min(1),
  value: z.string().min(1),
})).min(1, 'At least one item required').max(10),
```

## Multi-step wizards

```tsx
import { Stepper } from '@omnitron-dev/prism';

const Schema = z.object({
  account: z.object({ email: z.string().email(), password: z.string().min(8) }),
  profile: z.object({ name: z.string().min(1) }),
  plan:    z.enum(['free', 'pro', 'enterprise']),
});

const steps = ['account', 'profile', 'plan'] as const;

function Wizard() {
  const [step, setStep] = useState(0);
  const form = useForm({ resolver: zodResolver(Schema) });

  const next = async () => {
    // validate just this step's fields before advancing
    const ok = await form.trigger(steps[step] as any);
    if (ok) setStep(s => s + 1);
  };

  return (
    <FormProvider {...form}>
      <Stepper
        activeStep={step}
        steps={[
          { label: 'Account', description: 'Email and password' },
          { label: 'Profile', description: 'Your name' },
          { label: 'Plan' },
        ]}
      />

      {step === 0 && <AccountStep />}
      {step === 1 && <ProfileStep />}
      {step === 2 && <PlanStep />}

      <Stack direction="row" spacing={2}>
        {step > 0 && <Button onClick={() => setStep(s => s - 1)}>Back</Button>}
        {step < 2 && <Button variant="contained" onClick={next}>Next</Button>}
        {step === 2 && <Button type="submit" variant="contained">Finish</Button>}
      </Stack>
    </FormProvider>
  );
}
```

`form.trigger(name)` validates a subset — letting you gate "Next"
without committing to a full submit. (Step components read state via
`useFormContext()`, so they need no `form` prop.)

## Patterns

### Password visibility toggle

`usePasswordVisibility()` returns `{ visible, toggle, show, hide, type }`
— state and handlers only, **no** rendered button. Supply your own
`IconButton` in the field's `endAdornment` and drive the input `type`
from the hook:

```tsx
import { usePasswordVisibility } from '@omnitron-dev/prism/hooks';
import IconButton from '@mui/material/IconButton';
import InputAdornment from '@mui/material/InputAdornment';

function PasswordField() {
  const password = usePasswordVisibility();   // { visible, toggle, type, ... }
  const value    = form.watch('password');
  const strength = getPasswordStrength(value);

  return (
    <Field.Text
      name="password"
      label="Password"
      type={password.type}                      // 'text' | 'password'
      helperText={<PasswordStrengthBar score={strength} />}
      slotProps={{
        input: {
          endAdornment: (
            <InputAdornment position="end">
              <IconButton onClick={password.toggle} aria-label="Toggle password visibility">
                {password.visible ? <VisibilityOff /> : <Visibility />}
              </IconButton>
            </InputAdornment>
          ),
        },
      }}
    />
  );
}
```

### Async field validation

For "is this email taken?" checks:

```typescript
const Schema = z.object({
  email: z.string().email()
    .refine(async (email) => !(await isEmailTaken(email)), {
      message: 'Email already taken',
    }),
});
```

Use `mode: 'onBlur'` — every keystroke would hammer the server.

### Optimistic submit

```tsx
const submit = async (data: ProfileInput) => {
  // Optimistic local update
  setLocalUser({ ...user, ...data });
  try {
    await users.update.mutateAsync(data);
  } catch (e) {
    setLocalUser(user);  // rollback
    form.setError('root', { message: 'Could not save changes' });
  }
};
```

## Accessibility

Every `Field.*`:

- Has a programmatic `<label>` (via MUI's `TextField` / `FormControlLabel`).
- Sets `aria-invalid` when in error state.
- Sets `aria-describedby` linking to the helper/error message.
- Sets `aria-required` when `required` is passed.
- Supports keyboard navigation natively.

For custom controls wired via `<Controller>`, follow the same contract —
pass `error` + `helperText` props that map to ARIA attributes.

## Anti-patterns

- **Mixing controlled and uncontrolled fields.** Pick one (`Field.*` is
  controlled via react-hook-form) and stick to it.
- **`mode: 'onChange'` for everything.** Annoying UX; users see errors
  before they finish typing. Use `onTouched` or `onSubmit`.
- **Toast for form errors.** Use `<FormAlert>` inline (message via
  children).
- **Passing an `error` object to `<FormAlert>`.** It has no `error`
  prop — pass the message string as children.
- **Expecting `SchemaProvider` to validate or render fields.** It only
  supplies localized messages; validation is still `zodResolver`.
- **Custom HTML5 validation on top of zod.** Pick zod; HTML5 attributes
  from the schema are for ergonomics only.
- **Re-validating async on every keystroke.** Debounce or move to
  `onBlur`.
- **Storing form state in a global store.** Use react-hook-form for form
  state; lift only the final submitted values.

## See also

- [Components catalog / Field](/docs/frontend/prism/components#field) — the `Field` API
- [Hooks catalog](/docs/frontend/prism/hooks-catalog) — `usePasswordVisibility`,
  `useFocusTrap`, etc.
- [Blocks / AuthBlock](/docs/frontend/prism/blocks#authblock) — prebuilt sign-in flow
- [zod docs](https://zod.dev/) — schema authoring
- [react-hook-form](https://react-hook-form.com/) — form state management

---

---
id: "frontend/prism/hooks-catalog"
title: "Hooks catalog"
description: "The 48 React hooks exported from @omnitron-dev/prism/hooks."
---

# Hooks catalog

`@omnitron-dev/prism/hooks` ships hooks used internally by the
components and exposed for your code. All SSR-safe via
`useIsomorphicLayoutEffect` where needed.

```tsx
import { useArray, useAsync, useKeyboardShortcut } from '@omnitron-dev/prism/hooks';
```

## State & data

### `useArray<T>`

Reactive array operations without manual setState:

```tsx
const { value, push, removeAt, updateAt, clear } = useArray<Todo>([]);

<Button onClick={() => push({ id: '1', text: 'Buy milk' })}>Add</Button>
<Button onClick={() => removeAt(0)}>Remove first</Button>
<Button onClick={() => updateAt(0, { id: '1', text: 'Buy oat milk' })}>Update</Button>
<Button onClick={clear}>Clear</Button>
```

The current array is `value` (not `items`). Returns:

| Member                            | Effect                                   |
| --------------------------------- | ---------------------------------------- |
| `value`                           | Current array                            |
| `setValue(items)`                 | Replace entire array                     |
| `push(item)`                      | Append                                   |
| `pushMany(items)`                 | Append many                              |
| `remove(itemOrPredicate)`         | Remove first matching value or predicate |
| `removeAt(index)`                 | Remove at index                          |
| `removeWhere(pred)`               | Remove all matching predicate            |
| `updateAt(index, item)`           | Replace at index                         |
| `updateAtPartial(index, partial)` | Merge partial at index                   |
| `updateWhere(pred, update)`       | Update all matching predicate            |
| `insertAt(index, item)`           | Insert at index                          |
| `move(from, to)`                  | Reorder                                  |
| `swap(a, b)`                      | Swap two indices                         |
| `clear()`                         | Reset to `[]`                            |
| `reset()`                         | Reset to initial value                   |
| `reverse()` / `sort(cmp?)`        | Reorder in place                         |

Plus non-mutating helpers (`filter`, `find`, `findIndex`, `includes`) and
computed reads (`length`, `isEmpty`, `first`, `last`).

### `useAsync<T>`

Lifecycle-safe async state — handles unmount-during-fetch.

```tsx
const { data, loading, error, execute, reset } = useAsync(
  async (id: string) => fetchUser(id),
  { immediate: false },
);

useEffect(() => { execute(userId); }, [userId]);

if (loading)        return <Spinner />;
if (error)          return <ErrorCard error={error} />;
return <UserCard user={data!} />;
```

Returns `{ data, loading, error, execute, reset, setData, setError }`.
Options: `{ initialData, immediate, onSuccess, onError }`. Unmount
before resolve → state never updates (no leak warnings). For a
fire-and-forget variant without data tracking, see `useAsyncFn`
(`{ execute, loading, error }`).

### `useUpdateEffect`

`useEffect` that skips the first render — useful for "react to
prop change but not initial mount":

```tsx
useUpdateEffect(() => {
  track('filter.changed', { filter });
}, [filter]);
```

### `useSessionStorage<T>` / `useCookies`

Reactive storage mirrors. Both return an object (not a `[value, setValue]`
tuple):

```tsx
const { state: draft, setState: setDraft, remove } =
  useSessionStorage<Draft | null>('post-draft', null);

// Single key; supports object values with partial/field updates.
const { state, setState, setField } = useCookies('theme', 'dark', {
  maxAge: 365 * 86400,
  path: '/',
});
```

`useSessionStorage` returns `{ state, setState, remove, hasValue }` and
survives reloads within the tab session. `useCookies(key, initialState,
options)` returns `{ state, setState, setField, resetState }` — it manages
a single cookie key, not a list. Pairs with `useLocalStorage` (re-exported
from `@omnitron-dev/prism/hooks`).

### `useConfigFromQuery`

Syncs **theme configuration** (preset / mode / primary / direction /
density / contrast) with URL query params — for shareable demo/preview
links. It is not a generic schema-driven query-state hook.

```tsx
const {
  configFromUrl,   // parsed UrlConfigValues from the current URL
  updateUrl,       // (partial) => void — writes via history.replaceState
  clearUrl,        // () => void — strips all recognised params
  generateUrl,     // (partial, baseUrl?) => string — build a share link
  hasUrlConfig,    // boolean
} = useConfigFromQuery({
  onConfigChange: (cfg) => {
    if (cfg.preset) setPreset(cfg.preset);
    if (cfg.mode)   setMode(cfg.mode);
  },
});

const shareUrl = generateUrl({ preset: 'luxury', mode: 'dark' });
```

Browser Back and Forward update `configFromUrl`. To mirror edits made by a
settings panel, pass `{ syncToUrl: true, values: currentSettings }`. An
incoming shared link takes precedence on mount. Changes preserve other
query parameters, the URL fragment and the router's history state.

Recognised params: `preset`, `mode`, `primary` (hex), `dir`, `density`,
`contrast` (rename via the `paramNames` option). For arbitrary typed
query-state, parse `URLSearchParams` yourself or use a router loader.

## Timers & lifecycle

### `useCountdownDate` / `useCountdownSeconds`

```tsx
// Returns formatted, zero-padded strings (no isExpired flag).
const { days, hours, minutes, seconds } = useCountdownDate(deadline);

// Returns control object; call start() to begin (does not auto-start).
const { value, start, reset, isCounting, setValue } = useCountdownSeconds(60);
```

`useCountdownDate(date, placeholder?)` counts down to a specific
timestamp, ticking every second and returning two-digit strings.
`useCountdownSeconds(n)` is a manually-started countdown (ideal for OTP
resend timers) — it stays at `n` until you call `start()`.

### `useThrottle<T>`

Throttle a fast-changing value to at most one update per `ms`:

```tsx
const throttled = useThrottle(scrollY, 100);
```

Differs from debounce — throttle emits the leading edge then at
most one per window.

### `useOnlineStatus`

Reactive `navigator.onLine`. Returns an object, not a bare boolean:

```tsx
const { isOnline, isOffline, lastChanged } = useOnlineStatus();
if (isOffline) return <OfflineBanner />;
```

Listens to `online` + `offline` events (SSR-safe via
`useSyncExternalStore`).

## Layout & sizing

### `useClientRect`

Bounding rect + scroll dimensions. The hook provides the ref:

```tsx
const { elementRef, width, height, top, left } = useClientRect<HTMLDivElement>();

<div ref={elementRef}>Width: {width}px</div>
```

Values default to `0` (never null) and update on resize / scroll.

### `useWindowSize`

```tsx
const { width, height, isMobile, isTablet, isDesktop } = useWindowSize();
```

Debounced internally (100 ms by default) to avoid render thrash, and
exposes MUI-aligned breakpoint booleans.

### `useScrollPosition` / `useScrollOffsetTop`

```tsx
const { x, y, directionY, isScrolled, isAtBottom } = useScrollPosition();
const { offsetTop } = useScrollOffsetTop(200);   // offsetTop is the boolean

<TopBar elevation={offsetTop ? 4 : 0} />
```

`useScrollPosition` returns scroll `x` / `y` (not `scrollX` / `scrollY`)
plus direction + edge flags. `useScrollOffsetTop(defaultValue)` returns
`{ offsetTop, elementRef }` — attach `elementRef` to track an element's
offset instead of a fixed pixel value.

### `useBackToTop`

Returns visibility state + a scroll-to-top handler. The threshold is a
positional arg — a `'NN%'` string (scroll progress) or a pixel number
(distance from bottom); default `'90%'`:

```tsx
const { isVisible, onBackToTop } = useBackToTop('90%');
{isVisible && <Fab onClick={onBackToTop} />}
```

### `useImageDimensions`

```tsx
const { dimensions, loading, error } = useImageDimensions(src);
// dimensions: { width, height, aspectRatio } | null
```

## Visibility & focus

### `useIntersectionObserver`

The hook returns the `ref` callback to attach (it does not take a ref arg):

```tsx
const { ref, isIntersecting } = useIntersectionObserver({
  threshold: 0.5,
  rootMargin: '100px',
  triggerOnce: true,
});

<div ref={ref}>{isIntersecting && <Image src={src} />}</div>
```

Returns `{ ref, entry, isIntersecting, disconnect }`. Foundation for
lazy-loading patterns.

### `useInfiniteScroll`

Takes a page-fetch function (not a `hasMore`/`onLoadMore` config) and
owns the paging state itself:

```tsx
const { items, isFetchingMore, hasNextPage, fetchNextPage, sentinelRef } =
  useInfiniteScroll<Row>(
    async ({ cursor, pageSize }) => {
      const res = await api.list({ cursor, limit: pageSize });
      return { items: res.rows, pageInfo: { hasNextPage: res.hasMore, endCursor: res.next } };
    },
    { pageSize: 20 },
  );

<>
  {items.map(i => <Row key={i.id} {...i} />)}
  <div ref={sentinelRef}>{isFetchingMore && <Spinner />}</div>
</>
```

The fetch fn returns `{ items, pageInfo: { hasNextPage, endCursor } }`;
the hook flattens pages into `items` and auto-loads when `sentinelRef`
intersects. Use sparingly — pagination is friendlier than infinite
scroll for most apps.

### `useFocusTrap`

The hook returns `{ ref, focusFirst, focusLast }` — destructure `ref`.
The activation option is `enabled` (not `active`):

```tsx
const { ref } = useFocusTrap<HTMLDivElement>({ enabled: open });

<div ref={ref}>
  <TextField autoFocus />
  <Button>Save</Button>
  <Button>Cancel</Button>
</div>
```

Traps Tab navigation inside the container (options: `enabled`,
`autoFocus`, `restoreFocus`, `focusableSelector`). The modal / drawer /
dialog components use this internally.

### `useKeyboardShortcut`

The shortcut is a descriptor object `{ key, ctrl?, shift?, alt?, meta? }`
(one per call), not a `'cmd+k'` string or an array:

```tsx
useKeyboardShortcut({ key: 'k', ctrl: true }, () => commandPalette.open(), {
  preventDefault: true,
  ignoreInputs: true,
});

useKeyboardShortcut({ key: '?', shift: true }, () => helpDrawer.open());
```

`ctrl: true` matches Ctrl or Cmd. Set `ignoreInputs: true` to suppress
the shortcut while focus is in an input / textarea / contenteditable
(off by default); `enabled` toggles the listener entirely.

## Interaction

### `useDoubleClick`

Distinguish single from double click (built-in delay so single
clicks fire only after the double-click window). Options are `click`,
`doubleClick`, `timeout`; the hook returns a single click handler:

```tsx
const handleClick = useDoubleClick({
  click:       () => select(item),
  doubleClick: () => navigate(`/items/${item.id}`),
  timeout:     250,
});

<div onClick={handleClick} />
```

### `useLazyQuery`

Trigger a query on demand rather than on mount. Pass a query function;
call `query(variables)` to run it:

```tsx
const { query, data, isLoading, isSuccess } = useLazyQuery(
  (q: string) => searchApi(q),
  { cacheTime: 60_000 },
);

<>
  <SearchInput value={q} onChange={setQ} debounce={300} />
  <Button onClick={() => query(q)}>Search</Button>
  {isSuccess && <Results items={data!.items} />}
</>
```

Returns `{ data, error, isLoading, isSuccess, isError, isCalled,
isFetching, isRefetching, query, refetch, reset, setData }` with optional
`cacheTime` / `staleTime` / `retry`.

### `useMutation` (Prism's)

```tsx
// useMutation(mutationFn, options) — mutationFn is the first positional arg.
const save = useMutation(
  (data) => api.save(data),
  {
    onMutate:  (data) => { setLocal(data); toast.info('Saving…'); },      // optimistic update
    onSuccess: () => toast.success('Saved'),
    onError:   (e) => { setLocal(previous); toast.error(`Save failed: ${e.message}`); }, // rollback
  },
);

<Button onClick={() => save.mutate(values)} disabled={save.isLoading}>
  Save
</Button>
```

Prism's `useMutation` returns `{ mutate, isLoading, isSuccess, isError,
isIdle, data, error, reset }` — do optimistic updates in `onMutate` and roll
back in `onError` (there are no separate `optimistic`/`rollback` options).

For RPC mutations, prefer `useMutation` from
`@omnitron-dev/netron-react` — it integrates with the cache.
The Prism version is for non-RPC operations.

### `usePopoverHover`

Hover-managed popover open state. Returns the trigger ref + open/close
handlers + anchor element (no `triggerProps` / `popoverProps` bundles):

```tsx
const { open, anchorEl, onOpen, onClose, elementRef } =
  usePopoverHover<HTMLDivElement>();

<Box ref={elementRef} onMouseEnter={onOpen} onMouseLeave={onClose}>
  <Avatar />
</Box>
<Popover open={open} anchorEl={anchorEl} onClose={onClose}>
  <UserCard />
</Popover>
```

### `usePasswordVisibility`

Returns `{ visible, toggle, show, hide, type }` (state + helpers, not a
prebuilt button). Wire `type` and `toggle` into your own adornment:

```tsx
const { type, visible, toggle } = usePasswordVisibility();

<TextField
  type={type}
  slotProps={{
    input: {
      endAdornment: (
        <IconButton onClick={toggle} aria-label="Toggle password visibility">
          {visible ? <VisibilityOff /> : <Visibility />}
        </IconButton>
      ),
    },
  }}
/>
```

## SSR & isomorphism

### `useIsomorphicLayoutEffect`

`useLayoutEffect` on client; `useEffect` on server — avoids
SSR warnings:

```tsx
useIsomorphicLayoutEffect(() => {
  // DOM measurement / sync state
}, [deps]);
```

Use whenever your effect needs synchronous post-render execution
and you SSR.

## Hooks used internally by components

These are exposed but typically consumed via the matching
component:

| Hook               | Component / area                                                |
| ------------------ | --------------------------------------------------------------- |
| `useMenu`          | `<Menu>` (from `@omnitron-dev/prism`)                           |
| `useSnackbar`      | `<Snackbar>`                                                    |
| `useLightbox`      | `<Lightbox>` — open and index only; the component owns its zoom |
| `useChart`         | `<Chart>`                                                       |
| `usePrismContext`  | `<PrismProvider>` (from `/core`)                                |
| `useSettingsStore` | settings / theme mode / density (from `/state`)                 |
| `useLayoutContext` | layouts (from `/layouts`)                                       |

`useCarousel` is deliberately absent from that table. It exports
carousel state — index, bounds, autoplay — for a slider you render
yourself, and it does **not** drive `<Carousel>`: that component keeps
its own copy of the same state and takes no controlled props, so nothing
you do with the hook can move it. Use one or the other.

Dark mode is driven through `useSettingsStore` (`mode`, `setMode`,
`toggleMode`) — there is no `useColorMode` hook. `<ConfirmDialog>` and
`<CommandPalette>` are component-only (no companion `use*` hook).

## Composition patterns

Hooks compose freely:

```tsx
function ProductGrid({ filters }: { filters: ProductFilters }) {
  const { items, isFetchingMore, fetchNextPage, sentinelRef } =
    useInfiniteScroll<Product>(
      async ({ cursor, pageSize }) => {
        const res = await api.list({ ...filters, cursor, limit: pageSize });
        return { items: res.items, pageInfo: { hasNextPage: res.hasMore, endCursor: res.nextCursor } };
      },
      { pageSize: 24 },
    );

  return (
    <Stack>
      <Grid>
        {items.map(p => <ProductCard key={p.id} {...p} />)}
      </Grid>
      <div ref={sentinelRef}>
        {isFetchingMore && <Spinner />}
      </div>
    </Stack>
  );
}
```

## See also

- [Components catalog](/docs/frontend/prism/components) — components paired
  with these hooks
- [Forms](/docs/frontend/prism/forms) — hooks used in form patterns
- [netron-react hooks](/docs/frontend/netron/react) — RPC-specific hooks
  (use those, not these, for data fetching from a Titan backend)

## Also exported

Twenty-one further hooks are exported from `@omnitron-dev/prism/hooks` and had
no entry here. They are listed with their real signatures rather than written
up with invented examples — the point of this section is that they exist and
can be found, which is what was missing.

### State

| Hook                 | Signature                                                      | What it does                                                                                  |
| -------------------- | -------------------------------------------------------------- | --------------------------------------------------------------------------------------------- |
| `useBoolean`         | `(defaultValue = false): UseBooleanReturn`                     | A boolean with `setTrue` / `setFalse` / `toggle`, for the open-close state every dialog needs |
| `useSetState`        | `<T extends Record<string, unknown>>(initial: T \| (() => T))` | `setState` with partial updates, like a class component's                                     |
| `useMultiSelect`     | `<T>(options?: UseMultiSelectOptions<T>)`                      | Multi-selection state — select, deselect, toggle, select-all                                  |
| `usePrevious`        | `<T>(value: T): T \| undefined`                                | The value from the previous render                                                            |
| `useCopyToClipboard` | `(options?: UseCopyToClipboardOptions)`                        | Copy with a `copied` flag that resets itself                                                  |

### Timers and lifecycle

| Hook                    | Signature                                                           | What it does                                                                                                   |
| ----------------------- | ------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------- |
| `useCountdown`          | `(options: UseCountdownOptions)`                                    | Countdown from a duration; see also `useCountdownDate` / `useCountdownSeconds` above                           |
| `useTimeout`            | `(callback: () => void, delay: number)`                             | `setTimeout` that survives a changing callback and clears on unmount                                           |
| `useDebounceCallback`   | `<T extends (...args: never[]) => unknown>(fn: T, delay = 500)`     | Debounces the call, where `useDebounce` debounces the value                                                    |
| `useThrottleCallback`   | `<T extends (...args: unknown[]) => unknown>(fn: T, delay: number)` | The callback counterpart of `useThrottle`                                                                      |
| `useInterval`           | `(callback: () => void, delay: number \| null)`                     | `setInterval` that survives a changing callback; **`null` pauses it** rather than requiring a conditional hook |
| `useEventListener`      | `<K extends keyof WindowEventMap>(…)`                               | Adds a listener and removes it on unmount, typed by event name                                                 |
| `useIsClient`           | `(): UseIsClientReturn`                                             | False during SSR and the first render, true afterwards — the hydration guard                                   |
| `useMounted`            | `(): UseMountedReturn`                                              | Whether the component is still mounted — for guarding async completions                                        |
| `useUpdateLayoutEffect` | `(effect: EffectCallback, deps?: DependencyList)`                   | `useLayoutEffect` that skips the first render                                                                  |

### Layout and sizing

| Hook                         | Signature                                                               | What it does                                                                           |
| ---------------------------- | ----------------------------------------------------------------------- | -------------------------------------------------------------------------------------- |
| `useBreakpoint`              | `(custom?: Partial<Breakpoints>): 'xs' \| 'sm' \| 'md' \| 'lg' \| 'xl'` | The current breakpoint as one value                                                    |
| `useBreakpoints`             | `(): UseBreakpointsReturn`                                              | Every breakpoint's state at once — reach for this when a component branches on several |
| `useBreakpointChecks`        | `(custom?: Partial<Breakpoints>): BreakpointChecks`                     | The same, as a set of booleans to branch on                                            |
| `useMediaQuery`              | `(query: string): boolean`                                              | One media query                                                                        |
| `useResponsiveQuery`         | `(query: string): UseResponsiveQueryReturn`                             | A media query with its match state broken out                                          |
| `useScrollLock`              | `(): { lock, unlock, isLocked }`                                        | Locks body scroll — what a modal needs                                                 |
| `useScrollToTop`             | `(options?: { threshold?: number })`                                    | `scrollToTop` plus whether the button should show                                      |
| `useMultipleImageDimensions` | `(urls: (string \| null \| undefined)[], options?)`                     | Dimensions of several images at once                                                   |

### Pointer and keyboard

| Hook              | Signature                                                                 | What it does                                           |
| ----------------- | ------------------------------------------------------------------------- | ------------------------------------------------------ |
| `useClickOutside` | `<T extends HTMLElement>(handler: (e: MouseEvent \| TouchEvent) => void)` | Fires when a click lands outside the returned ref      |
| `usePopover`      | `<T extends HTMLElement>(): UsePopoverReturn<T>`                          | Anchor + open state for a popover, without a component |
| `useEnterKey`     | `(callback: (e: KeyboardEvent) => void, options?)`                        | Enter, over `useKeyboardShortcut`                      |
| `useEscapeKey`    | `(callback: (e: KeyboardEvent) => void, options?)`                        | Escape, over `useKeyboardShortcut`                     |

Hooks named on other pages — `useSnackbar`, `useLightbox`, `useCarousel`,
`useChart`, `useMenu`, `usePrismContext`, `useLayoutContext`,
`useSettingsStore` — are exported from `/components`, `/core`, `/layouts` and
`/state` rather than from `/hooks`, which is why they are not in this list.

---

---
id: "frontend/prism/http"
title: "HTTP client"
description: "Typed REST requests, cancellation, interceptors and explicit retries."
---

# HTTP client

`@omnitron-dev/prism/http` provides `createHttpClient` and the shared `http`
instance for ordinary HTTP APIs. It is separate from the Netron RPC client.
Use it for REST endpoints and downloads; use
[Prism's Netron bindings](/docs/frontend/prism#netron-integration--omnitron-devprismnetron)
for named RPC services.

## Make a request

```typescript
import { createHttpClient } from '@omnitron-dev/prism/http';

const api = createHttpClient({
  baseUrl: '/api',
  timeout: 10_000,
  retry: { maxRetries: 0 },
});

interface Profile { id: string; displayName: string }
const response = await api.get<Profile>('/profile');
console.log(response.data.displayName, response.status);
```

The result is `HttpResponse<T>` with `data`, `status`, `statusText`,
`headers` and the original `config`. The generic describes the expected
payload; it does not validate JSON at runtime. Validate untrusted payloads
at the application boundary.

## Request surface

| Method                             | Arguments                                        |
| ---------------------------------- | ------------------------------------------------ |
| `get`, `delete`, `head`, `options` | URL and optional request configuration           |
| `post`, `put`, `patch`             | URL, body and optional request configuration     |
| `request`                          | A complete `HttpRequestConfig`                   |
| `extend`                           | Creates another client with merged configuration |
| `getConfig`                        | Reads the current client configuration           |

Use `params` for query parameters, `headers` for per-request headers and
`responseType` for `json`, `text`, `blob` or `arraybuffer`. `withCredentials`
controls cookie credentials; configure the server's CORS and CSRF policy
as well when credentials cross an origin.

## Cancel work on disposal

```typescript
const controller = new AbortController();
const pending = api.get('/reports', { signal: controller.signal });
// On navigation or component cleanup:
controller.abort();
try {
  await pending;
} catch (error) {
  // Distinguish cancellation from an actionable request failure.
}
```

`HttpError` exposes optional `status`, `data`, `headers`, `cancelled` and
`timeout` fields, plus the request `config`. A network failure may have no
HTTP status. Narrow the caught value before reading those properties.

## Interceptors and retries

`addRequestInterceptor`, `addResponseInterceptor` and `addErrorInterceptor`
return removal functions. Install long-lived interceptors once; remove
temporary ones when their owner is disposed.

```typescript
const remove = api.addRequestInterceptor((config) => ({
  ...config,
  headers: { ...config.headers, 'X-Client': 'web' },
}));
// Later, when this integration is disposed:
remove();
```

Retry configuration accepts `maxRetries`, `retryDelay` and
`retryCondition(error, attempt)`. Set retry policy explicitly for the
operation: replaying a payment or creation request may duplicate a side
effect. An idempotency key and server support are needed before such a
request is safe to retry.

The client supplies transport mechanics, not a React cache. Use
[Netron query hooks](/docs/frontend/netron/react) for Netron server state, or let
the consuming application own caching and invalidation for a REST API.

---

---
id: "frontend/prism/index"
title: "Overview"
description: "Design system for production React frontends — MUI v9 + Prism."
---

# Prism

<code>@omnitron-dev/prism</code> is the design system. Pre-composed,
theme-aware components and entire UI blocks for building
production React frontends — built on **MUI v9** + **React 19**,
integrated with **react-hook-form + zod**, ready for **Vite /
Next / Remix** out of the box.

```bash
pnpm add @omnitron-dev/prism
```

For React Server Components, put providers and interactive Prism components at
the client boundary. The published UI entry points declare `"use client"`.
Import `@omnitron-dev/prism/http` directly for server-side HTTP calls.

> **React 19 + MUI v9 idioms**
>
> Prism uses the latest patterns from both upstreams:
>
> - **No `forwardRef`** — React 19 routes `ref` through props
>   directly, so every component takes `ref` as a normal prop.
> - **`slotProps` API** — the v9 replacement for the legacy
>   `InputProps={...}` / `MenuProps={...}` / `TabIndicatorProps={...}` /
>   `BackdropProps={...}` / `componentsProps={...}` families.
> - **Unified `<Grid>` with `size={{ xs, sm, md }}`** — v1's
>   `<Grid item xs={...}>` shorthand is gone.
> - **Position classes for grouped controls** —
>   `ToggleButtonGroup` and `ButtonGroup` mark children with
>   `firstButton` / `middleButton` / `lastButton`; theme overrides
>   target those classes instead of the deprecated
>   `:not(:first-of-type)` sibling selector.
>
> See [Components → Refs (React 19)](/docs/frontend/prism/components#refs-react-19)
> and [Components → MUI v9 slot props](/docs/frontend/prism/components#mui-v9-slot-props)
> for the consumer-side migration cheat sheet.

## Three layers of API

```mermaid
flowchart LR
  Blocks[Blocks<br/>full-page composites]
  Layouts[Layouts<br/>shell + slots]
  Components[Components<br/>composable widgets]
  Core[Core primitives<br/>theme, providers, types]

  Blocks --> Layouts
  Blocks --> Components
  Layouts --> Components
  Components --> Core
  Layouts --> Core
```

Pick the level that matches your need:

- [**Blocks**](/docs/frontend/prism/blocks) — copy a `<DashboardBlock>`, fill the
  slots, ship a screen.
- [**Layouts**](/docs/frontend/prism/layouts) — own the routing; use
  `<DashboardLayout>` for shell.
- [**Components**](/docs/frontend/prism/components) — assemble from `<Card>` /
  `<Table>` / `<Drawer>`.
- [**Theme**](/docs/frontend/prism/theme) — palette, typography, shadows,
  density, dark mode.
- [**Forms**](/docs/frontend/prism/forms) — schema-aware forms with `<Field>` +
  `SchemaProvider`.
- [**Hooks catalog**](/docs/frontend/prism/hooks-catalog) — all 48 React hooks.
- [**Maps**](/docs/frontend/prism/maps) — self-hosted MapLibre map, markers,
  point picker and coverage layer.

## Subpath exports

Public subpaths are explicit package exports with matching JavaScript and
TypeScript declarations. For example, `./components/card`, `./components/table`,
`./components/tabs`, `./components/chart` and `./components/map` are available.
A source directory by itself does not create a public import path; there are no
wildcard component exports.

Named imports from the root or `./components` discard unused component modules.
Importing `Card` does not retain the editor, maps or chart engines. Choose a
public subpath when it makes a route's lazy-loading boundary clearer. Editors,
maps, research documents and emoji pickers should load with the routes that use
them.

Import documented `styles.css` entries explicitly for components that need CSS;
these styles remain side-effectful and survive production optimization. Calendar
locale registration stays with its component. CI builds consumers from the real
package archive and verifies both unused dependency removal and preservation of
calendar, map and stylesheet dependencies. Run `pnpm check:packages` after the
workspace build to repeat this check. Measure final application chunks separately:
the size depends on selected components, peers and route boundaries.

| Subpath                                       | What it exports                                                                                              |
| --------------------------------------------- | ------------------------------------------------------------------------------------------------------------ |
| `@omnitron-dev/prism`                         | Named imports across the public API; unused modules are discarded                                            |
| `@omnitron-dev/prism/theme`                   | `createPrismTheme()`, palette, typography, shadows, presets                                                  |
| `@omnitron-dev/prism/core`                    | `<PrismProvider>`, `<ProviderStack>`, context primitives                                                     |
| `@omnitron-dev/prism/layouts`                 | `<DashboardLayout>`, `<AuthCenteredLayout>` / `<AuthSplitLayout>` / `<AuthSimpleLayout>`, `<LayoutProvider>` |
| `@omnitron-dev/prism/blocks`                  | `<AuthBlock>`, `<DashboardBlock>`, `<DataGridBlock>`                                                         |
| `@omnitron-dev/prism/components`              | Named imports from the component catalog                                                                     |
| `@omnitron-dev/prism/components/card`         | `Card`, `CardSection`, `StatCard`                                                                            |
| `@omnitron-dev/prism/components/table`        | `Table`, `useTable`                                                                                          |
| `@omnitron-dev/prism/components/map`          | MapLibre map, markers, layers and place search                                                               |
| `@omnitron-dev/prism/components/editor`       | `<Editor>` alone, as its own chunk                                                                           |
| `@omnitron-dev/prism/components/emoji-picker` | `<EmojiPicker>` alone — the dataset is \~80 kB gz                                                            |
| `@omnitron-dev/prism/forms`                   | Schema-aware form helpers                                                                                    |
| `@omnitron-dev/prism/hooks`                   | 48 React hooks                                                                                               |
| `@omnitron-dev/prism/state`                   | Zustand-based store factory                                                                                  |
| `@omnitron-dev/prism/accessibility`           | Vision simulation provider and controls                                                                      |
| `@omnitron-dev/prism/netron`                  | Pre-wired Netron auth/UI bindings                                                                            |
| `@omnitron-dev/prism/http`                    | HTTP fetcher helpers                                                                                         |
| `@omnitron-dev/prism/cli`                     | CLI helpers (used by `prism` bin)                                                                            |

Tree-shaking works on every subpath — import the smallest scope
your bundler needs.

## Minimum wiring

```tsx
import { PrismProvider } from '@omnitron-dev/prism/core';

function App() {
  return (
    <PrismProvider
      defaultSettings={{ mode: 'dark', preset: 'midnight', primaryColor: '#7c4dff' }}
    >
      <Outlet />
    </PrismProvider>
  );
}
```

`<PrismProvider>` builds the theme itself — you do not call
`createPrismTheme()` and hand it over, and there is no `theme`
prop to hand it to. It reads the settings store (persisted, so a
user's own choices survive a reload), falls back to
`defaultSettings` for anything unset, resolves `mode: 'system'`
against the OS preference, and calls `createPrismTheme()` with
the result. The Omnitron console supplies `mode`, `preset` and
`direction` through this settings interface.

`defaultSettings` accepts `mode`, `preset`, `direction`,
`primaryColor`, `contrast`, `fontSize`, `fontFamily` and
`navLayout`. For MUI-level escapes use `themeOverrides`, which is
merged into the generated theme; `config` carries non-theme
configuration. `createPrismTheme()` is exported for building a
theme outside the provider (a standalone MUI tree, a snapshot
test), not for feeding one back in.

Beyond the theme, the provider sets up MUI's `ThemeProvider`,
`CssBaseline`, the snackbar host, the icon registry, and the
localization context (`dateAdapterLocale`, `dateLocaleText`).

For more sophisticated apps, use `<ProviderStack>` to layer
multiple providers cleanly:

```tsx
<ProviderStack
  providers={[
    [QueryClientProvider, { client: queryClient }],
    [AuthProvider,         { client: authClient }],
    [PrismProvider,        { defaultSettings: { mode: 'dark', preset: 'midnight' } }],
    [RouterProvider,       { router }],
  ]}
>
  <Outlet />
</ProviderStack>
```

## State management — `createPrismStore` / `createPersistedStore`

Prism ships Zustand-based store factories (Immer + DevTools +
optional localStorage persistence baked in). Use
`createPrismStore` for general state and `createPersistedStore`
for persisted state with version-aware migration:

```tsx
import { createPrismStore } from '@omnitron-dev/prism/state';

interface UIState {
  sidebarOpen: boolean;
  toggleSidebar: () => void;
}

// Immer middleware is enabled — mutate `state` directly in `set`.
export const useUIStore = createPrismStore<UIState>((set) => ({
  sidebarOpen:   true,
  toggleSidebar: () => set((s) => { s.sidebarOpen = !s.sidebarOpen; }),
}), {
  name:    'ui-store',
  persist: { partialize: (s) => ({ sidebarOpen: s.sidebarOpen }) },
});
```

For persisted preferences with migrations, pass a `version`
plus a `migrate` callback through the persist options:

```tsx
import { createPersistedStore } from '@omnitron-dev/prism/state';

export const useSettings = createPersistedStore<SettingsState>(
  (set) => ({ /* … */ }),
  'settings',                       // store name (positional)
  {
    version: 2,
    migrate: (persisted, version) => {
      // version-aware migration
      return persisted as SettingsState;
    },
  },
);
```

When you bump `version`, persisted state from older versions
runs through `migrate` before being adopted. (See also the
versioned-settings helpers in `@omnitron-dev/prism/state`.)

## Accessibility

`@omnitron-dev/prism/accessibility` exports vision simulation through
`VisionModeProvider`, `useVisionMode`, `VISION_MODE_OPTIONS` and
`getVisionModeStyles`. See [Accessibility](/docs/frontend/prism/accessibility) for
the actual surface and an application-level verification workflow.

## Netron integration — `@omnitron-dev/prism/netron`

Pre-wired auth + UI bindings for apps that talk to a Titan
backend via Netron:

```tsx
import { NetronProvider, createNetronClient }
  from '@omnitron-dev/prism/netron';

const client = createNetronClient({ transport: 'http', url: '/api' });

function App() {
  return (
    <NetronProvider client={client}>
      <Outlet />
    </NetronProvider>
  );
}
```

For multi-backend setups:

```tsx
import { createMultiBackendClient, MultiBackendProvider }
  from '@omnitron-dev/prism/netron';

const client = createMultiBackendClient({
  baseUrl: '',
  backends: {
    main:    { path: '/api/main' },
    storage: { path: '/api/storage' },
    realtime:{ path: '/api/realtime' },
  },
  defaultBackend: 'main',
});

<MultiBackendProvider client={client} autoConnect>
  <Outlet />
</MultiBackendProvider>
```

The bindings re-export `@omnitron-dev/netron-react` hooks —
see [netron/react](/docs/frontend/netron/react) for the full hook
reference.

## CLI — `prism` binary

```bash
prism init                 # scaffold Prism config in current project
prism add card   # generate boilerplate using registered template
prism list --components      # show available components
```

The CLI uses templates from `templates/` (shipped with the
package) plus the schema in `registry.json` for available
component metadata.

## Best practices

- **Pick the smallest layer.** A [`<DashboardBlock>`](/docs/frontend/prism/blocks)
  is faster than composing it from 12 components, but constrains
  you to its prop API. Drop to [`<DashboardLayout>`](/docs/frontend/prism/layouts)
  - components when you need flexibility.
- **One `<PrismProvider>` per app.** Multiple providers create
  duplicate snackbar hosts and confused theme contexts.
- **Use declared entry points.** Consult the [API index](/docs/reference/api-index);
  arbitrary per-component paths are not published.
- **Use schema-driven forms.** [`<Field>` +
  `SchemaProvider`](/docs/frontend/prism/forms) produces consistent UX with zero
  per-field boilerplate.
- **Surface form errors with `<FormAlert>` inline**; reserve
  toasts (`<Snackbar>`) for transient background events.
- **`createPrismStore` / `createPersistedStore` over raw
  Zustand** for any state that persists — the versioned-settings
  helpers handle migrations.

## See also

- [Components catalog](/docs/frontend/prism/components) — 50+ widgets with
  props + examples
- [Blocks](/docs/frontend/prism/blocks) — `<AuthBlock>`, `<DashboardBlock>`,
  `<DataGridBlock>`
- [Layouts](/docs/frontend/prism/layouts) — three app shells
- [Theme](/docs/frontend/prism/theme) — palette, typography, dark mode
- [Forms](/docs/frontend/prism/forms) — schema-aware form patterns
- [Hooks catalog](/docs/frontend/prism/hooks-catalog) — React hooks
- [netron-browser](/docs/frontend/netron/browser) — the RPC client
- [netron-react](/docs/frontend/netron/react) — React hooks for RPC
- [Frontend overview](/docs/frontend/overview) — the three-package picture

## Further guides

- [State and persistence](/docs/frontend/prism/state) — stores, hydration and migrations.
- [HTTP client](/docs/frontend/prism/http) — REST requests, cancellation and interceptors.
- [Component CLI](/docs/frontend/prism/cli) — copy and maintain local component source.
- [API index](/docs/reference/api-index) — every declared public export.

---

---
id: "frontend/prism/layouts"
title: "Layouts"
description: "DashboardLayout + the three auth layouts + the core layout primitives."
---

# Layouts

Layouts are page-shell components — they own the chrome (sidebar,
header, footer) and slot in your route content (typically a
react-router `<Outlet />`).

Everything lives under a **single** subpath, `@omnitron-dev/prism/layouts`.
There are **no** `/layouts/dashboard`, `/layouts/auth`, or
`/layouts/core` deep imports — the package `exports` map only declares
`./layouts`, so deep imports will not resolve.

```tsx
import {
  // Application shell
  DashboardLayout,
  // Auth shells
  AuthCenteredLayout,
  AuthSplitLayout,
  AuthSimpleLayout,
  // Context / config
  LayoutProvider,
  useLayoutContext,
  useLayoutConfig,
  useNavActive,
  // Core primitives (build-your-own)
  LayoutSection,
  MainSection,
  HeaderSection,
  UserMenu,
} from '@omnitron-dev/prism/layouts';
```

| Export                            | What it is                                                                  |
| --------------------------------- | --------------------------------------------------------------------------- |
| `DashboardLayout`                 | Full app shell — sidenav / topnav / combo + header + content                |
| `DashboardContent`                | Padded content wrapper for use inside the dashboard `<main>`                |
| `PanelLayout` / `SidebarToggle`   | Nested navigation with independent collapse state and a shared edge control |
| `Sidenav` / `MobileSidenav`       | The desktop / mobile sidebar (rendered by `DashboardLayout`)                |
| `AuthCenteredLayout`              | Card-centered auth shell (sign-in / sign-up / reset)                        |
| `AuthSplitLayout`                 | Split-screen auth shell (illustration + form)                               |
| `AuthSimpleLayout`                | Minimal auth/error/maintenance shell                                        |
| `LayoutProvider`                  | Context provider holding layout config + nav state                          |
| `LayoutSection` / `MainSection`   | Structural primitives for custom shells                                     |
| `HeaderSection` / `HeaderToolbar` | Slot-based sticky header                                                    |
| `UserMenu`                        | Avatar-trigger account dropdown for the header `rightArea`                  |

There is **no** single `<AuthLayout>` component (only an
`AuthLayoutProps` / `AuthLayoutVariant` *type*), and there is **no**
`<CoreLayout>` — the minimal shell is composed from `LayoutSection` +
`HeaderSection`, or you reach for `AuthSimpleLayout`.

## Secondary navigation with `<PanelLayout>`

Use one application `DashboardLayout` and place `PanelLayout` inside it for
corpus contents, document navigation, graph indexes or administration tools.
Each panel can collapse independently. The main editor remains mounted when a
panel collapses or its mobile drawer opens. Render the panel as a function to
provide compact navigation while retaining the same component and its state.
A plain React node remains mounted but hidden when collapsed. Mobile drawers
retain their body between openings.

```tsx
import { PanelLayout } from '@omnitron-dev/prism/layouts';

<PanelLayout
  title="Document contents"
  labels={{ expand: 'Expand contents', collapse: 'Collapse contents', close: 'Close contents' }}
  persistKey="research.document-contents"
  navigationKey={selectedDocumentId}
  panel={({ collapsed }) => <DocumentContents compact={collapsed} />}
>
  <DocumentEditor />
</PanelLayout>
```

`PanelLayout` does not require a router. `navigationKey` closes the mobile
drawer after the host changes selection; route-based applications can pass
`useLocation().key`. It does not reset the desktop preference. Use distinct
`persistKey` values for the application sidebar and each kind of nested panel.
The default desktop breakpoint is `md`, expanded width is 264 px, and sticky
offset is 88 px. The compact rail is 64 px wide. Its height is identical in both
modes: by default `calc(100dvh - (offset + 24px))`. Override `height` for embedded
surfaces such as a graph, and `collapsedWidth`, `layoutQuery`, `width`, `offset`
and `sx` as needed. `DocSidebar` supports `compact`, full tooltip/accessibility
labels, active destination identity and optional item `shortLabel` values.
Hide auxiliary search controls instead of unmounting them when retaining their
input state matters. The dashboard and nested panels share `SidebarToggle` and
the themed `Scrollbar`; scrollbar styling also applies to nested overflow
regions through Prism's CSS baseline. On a fine pointer, thumbs reveal on hover
or keyboard focus inside the region without changing layout. Touch input keeps
the native controls visible; forced-colors mode keeps native colors and dimensions.

It uses native MUI controls and the shared Prism settings store, including its
storage-failure fallback. The panel body may remount when crossing the mobile
breakpoint; lift durable drafts and search state into the owning feature.

`DashboardLayout.labels` localizes opening/closing navigation, collapse/expand,
main landmark and skip link. `slotProps.main.id` and `className` integrate with
the application's focus and fragment navigation. Its mobile drawer closes on
actual router navigation; collapsed destinations retain accessible names and
`aria-current` values. A public website and a workspace should render separate
shells, rather than placing the full application inside a bounded marketing
container.

## `<DashboardLayout>`

Admin / operator console shell. It wraps its children in a
[`LayoutProvider`](#layoutprovider) and renders a header plus a
sidenav, a topnav, or both — driven by `initialConfig.navigationMenuType`
(`'sidenav'` by default).

```tsx
import { DashboardLayout, UserMenu } from '@omnitron-dev/prism/layouts';
import type { LayoutNavData } from '@omnitron-dev/prism/layouts';
import { Outlet } from 'react-router-dom';

const navData: LayoutNavData = [
  {
    id: 'main',
    subheader: 'Overview',
    items: [
      { id: 'dashboard', title: 'Dashboard', path: '/', icon: <DashboardIcon /> },
      { id: 'apps', title: 'Apps', path: '/apps', icon: <AppsIcon /> },
    ],
  },
  {
    id: 'infra',
    subheader: 'Infrastructure',
    items: [
      {
        id: 'infrastructure',
        title: 'Infrastructure',
        icon: <GridIcon />,
        children: [
          { id: 'containers', title: 'Containers', path: '/containers' },
          { id: 'nodes', title: 'Nodes', path: '/nodes' },
        ],
      },
      {
        id: 'settings',
        title: 'Settings',
        path: '/settings',
        icon: <CogIcon />,
        allowedRoles: ['admin'],
      },
    ],
  },
];

function ConsoleShell() {
  return (
    <DashboardLayout
      logo={<Logo size={32} />}
      navData={navData}
      headerSlots={{
        centerArea: <CommandPaletteTrigger />,
        rightArea: (
          <>
            <NotificationsBell />
            <UserMenu user={{ name: 'Ada Lovelace', email: 'ada@example.com' }} />
          </>
        ),
      }}
      initialConfig={{ navigationMenuType: 'sidenav', sidenavVariant: 'default' }}
      persistKey="console"
    >
      <Outlet />
    </DashboardLayout>
  );
}
```

### Bounded workspaces

For terminals, editors and infrastructure consoles, set `contentMode="workspace"`. The shell uses `100dvh`, keeps its header and footer in the available height, and gives the main region `minHeight: 0` and hidden overflow. Children should use flex sizing with `minHeight: 0` and `minWidth: 0`; put scrolling on the panel that owns it. To scroll ordinary pages inside the same shell, use `slotProps={{ main: { sx: { overflowY: 'auto' } } }}`. Default `document` mode retains normal page scrolling for other applications.

### Props

| Prop            | Type                        | Default      | Notes                                                                                                                               |
| --------------- | --------------------------- | ------------ | ----------------------------------------------------------------------------------------------------------------------------------- |
| `children`      | `ReactNode`                 | —            | Page content (your `<Outlet />`)                                                                                                    |
| `contentMode`   | `'document' \| 'workspace'` | `'document'` | Document flow, or a viewport-bounded shell whose panels own scrolling                                                               |
| `navData`       | `LayoutNavData`             | —            | Navigation tree — `LayoutNavSection[]` (see below)                                                                                  |
| `headerSlots`   | `HeaderSlots`               | —            | Header content per slot (`leftArea` / `centerArea` / `rightArea` / `topArea` / `bottomArea`)                                        |
| `logo`          | `ReactNode`                 | —            | Logo shown in the sidenav (and as the mobile header left slot)                                                                      |
| `footer`        | `ReactNode`                 | —            | Rendered at the bottom of the main column                                                                                           |
| `sidenavFooter` | `ReactNode`                 | —            | Footer content pinned inside the sidenav                                                                                            |
| `layoutQuery`   | `Breakpoint`                | `'lg'`       | Breakpoint at/above which the desktop sidenav shows and the mobile drawer/menu-button hides                                         |
| `initialConfig` | `Partial<LayoutConfig>`     | —            | Initial layout config (nav type, sidenav variant, nav color, …)                                                                     |
| `persistKey`    | `string`                    | —            | When set, sidenav collapsed state persists to `localStorage` under this key. Use a unique key per layout (e.g. `'main'`, `'admin'`) |
| `slotProps`     | `{ header?, main? }`        | —            | `header` forwards to `HeaderSection`; `main` accepts `{ sx, disablePadding, maxWidth }`                                             |

There is **no** `brand` / `navItems` / `topRight` / `topLeft` /
`sidebarWidth` / `defaultCollapsed` / `persistCollapse` /
`showBreadcrumbs` / `userRoles` prop. The header is composed via
`headerSlots`; the logo goes through `logo`; nav uses `navData`; and
nav-item visibility is expressed per-item with `allowedRoles` /
`allowedPermissions` (see [`LayoutNavItem`](#layoutnavitem)).

> **Mobile menu button is automatic**
>
> When a sidenav is active, `DashboardLayout` injects a hamburger
> `<IconButton>` into the header `leftArea` below `layoutQuery` (it
> toggles the mobile drawer). You don't need to add one yourself —
> anything you pass in `headerSlots.leftArea` renders **after** it.

### `LayoutNavData`

Navigation is a list of **sections**, each with an optional subheader
and a list of items:

```typescript
type LayoutNavData = LayoutNavSection[];

interface LayoutNavSection {
  id: string;
  subheader?: string;       // group heading
  items: LayoutNavItem[];
}
```

### `LayoutNavItem`

```typescript
interface LayoutNavItem {
  id: string;                       // unique item ID (required)
  title: string;                    // display label
  path?: string;                    // route (leaf items)
  icon?: ReactNode;                 // icon element
  info?: ReactNode;                 // trailing info/label
  caption?: string;                 // secondary description text
  disabled?: boolean;
  external?: boolean;               // external link
  children?: LayoutNavItem[];       // nested menu — expands inline
  allowedRoles?: string[];          // visibility filter (roles)
  allowedPermissions?: string[];    // visibility filter (permissions)
  deepMatch?: boolean;              // active on child routes (default: true)
  badge?: string | number;          // count / "new" indicator
  selectionPrefix?: string | string[]; // keep active across disjoint URL spaces
}
```

Active-state rules (implemented in `isNavItemActive`):

- **Exact match** — `path` equals the current pathname (query/hash and
  trailing slashes are stripped first).
- **`selectionPrefix`** — active when the path starts with any listed
  prefix. Accepts a string or an array (any prefix wins) so one entry
  can span two URL spaces, e.g.
  `selectionPrefix: ['/communities', '/c/']`.
- **`deepMatch`** (default `true`) — active when `path` is a
  segment-aligned prefix of the pathname (`/admin/orgs` stays
  highlighted on `/admin/orgs/list`, but `/product` does **not** match
  `/products`). Set `deepMatch: false` to require an exact match.

Items with `children` render as inline-expandable groups; a parent
stays highlighted when one of its descendants is active.

### Behaviour

- **Navigation type** — `initialConfig.navigationMenuType` selects
  `'sidenav'` (default), `'topnav'`, or `'combo'` (both). Sidenav and
  combo render `Sidenav` + `MobileSidenav`; topnav and combo render the
  `Topnav` bar.
- **Sidenav variants** — `initialConfig.sidenavVariant` is `'default'`,
  `'mini'`, or `'stacked'`; widths come from `DRAWER_WIDTHS`
  (`full: 280`, `mini: 73`, `stackedExpanded: 300`,
  `stackedCollapsed: 72`).
- **Responsive** — at/above `layoutQuery` the desktop `Sidenav` shows
  and the main content is offset by the drawer width; below it, the
  sidebar becomes a `MobileSidenav` drawer toggled by the injected
  header menu button.
- **Collapse persistence** — toggling collapse persists to the settings
  store under `persistKey` (and is restored on mount). Without
  `persistKey`, collapse is in-memory only.
- **Accessibility** — renders a `SkipLink` to `#main-content`, a
  `role="main"` landmark, and ARIA labels for the header / nav.

### `DashboardContent`

Optional padded wrapper for content rendered inside the dashboard
`<main>` (handy when you don't pass `slotProps.main`):

```tsx
import { DashboardContent } from '@omnitron-dev/prism/layouts';

<DashboardContent maxWidth="xl">
  <PageHeader title="Apps" />
  <AppsTable />
</DashboardContent>
```

| Prop             | Type                  | Default |
| ---------------- | --------------------- | ------- |
| `children`       | `ReactNode`           | —       |
| `disablePadding` | `boolean`             | `false` |
| `maxWidth`       | `Breakpoint \| false` | —       |
| `sx`             | `SxProps<Theme>`      | —       |

### Layout density

Density is a setting on Prism's settings store, not a layout prop —
read/write it with `useSettingsStore`:

```tsx
import { useSettingsStore } from '@omnitron-dev/prism';

function DensityToggle() {
  const density = useSettingsStore((s) => s.density);
  const setDensity = useSettingsStore((s) => s.setDensity);
  return (
    <select value={density} onChange={(e) => setDensity(e.target.value as any)}>
      <option value="compact">Compact</option>
      <option value="standard">Standard</option>
      <option value="comfortable">Comfortable</option>
    </select>
  );
}
```

`ComponentDensity` is `'compact' | 'standard' | 'comfortable'`. Tables,
lists, and forms respond to it via theme spacing tokens.

## Auth layouts

There are three auth shells. They share a help-link header pattern but
differ in how the form is framed. None of them require a `LayoutProvider`.

### `<AuthCenteredLayout>`

Card-centered form on a subtle gradient — the default for sign-in /
sign-up / reset / verify.

```tsx
import { AuthCenteredLayout } from '@omnitron-dev/prism/layouts';

function SignInPage() {
  return (
    <AuthCenteredLayout logo={<Logo />}>
      <SignInForm />
    </AuthCenteredLayout>
  );
}
```

| Prop       | Type               | Default |
| ---------- | ------------------ | ------- |
| `children` | `ReactNode`        | —       |
| `logo`     | `ReactNode`        | —       |
| `maxWidth` | `number \| string` | `420`   |
| `sx`       | `SxProps<Theme>`   | —       |

Companion: `AuthCenteredContent` — a standalone card wrapper
(`children`, `maxWidth = 420`, `sx`) for composing your own centered shell.

### `<AuthSplitLayout>`

Split-screen: branded illustration pane on the left (hidden below `md`),
form on the right. Includes a built-in help link in the header.

```tsx
import { AuthSplitLayout } from '@omnitron-dev/prism/layouts';

function SignInPage() {
  return (
    <AuthSplitLayout
      logo={<Logo />}
      title="Welcome to Platform"
      subtitle="Sign in to continue to your dashboard"
      illustration={<img src="/auth-illustration.svg" alt="" />}
    >
      <SignInForm />
    </AuthSplitLayout>
  );
}
```

| Prop             | Type               | Default        |
| ---------------- | ------------------ | -------------- |
| `children`       | `ReactNode`        | —              |
| `logo`           | `ReactNode`        | —              |
| `title`          | `string`           | `'Welcome'`    |
| `subtitle`       | `string`           | —              |
| `illustration`   | `ReactNode`        | —              |
| `showHelp`       | `boolean`          | `true`         |
| `helpUrl`        | `string`           | `'/help'`      |
| `helpText`       | `string`           | `'Need help?'` |
| `settingsButton` | `ReactNode`        | —              |
| `maxWidth`       | `number \| string` | `480`          |
| `illustrationSx` | `SxProps<Theme>`   | —              |
| `sx`             | `SxProps<Theme>`   | —              |

The illustration pane is hidden below the `md` breakpoint; the form
then takes the full width. Companions `AuthSplitIllustration`
(`title` / `subtitle` / `illustration` / `sx`) and `AuthSplitContent`
(`children` / `maxWidth` / `sx`) let you assemble a custom split.

### `<AuthSimpleLayout>`

Minimal shell with just a help-link header — for error pages (404 /
500\), maintenance pages, or compact auth forms.

```tsx
import { AuthSimpleLayout } from '@omnitron-dev/prism/layouts';

function NotFoundPage() {
  return (
    <AuthSimpleLayout logo={<Logo />} compact>
      <NotFoundContent />
    </AuthSimpleLayout>
  );
}
```

| Prop             | Type                  | Default        |
| ---------------- | --------------------- | -------------- |
| `children`       | `ReactNode`           | —              |
| `logo`           | `ReactNode`           | —              |
| `showHelp`       | `boolean`             | `true`         |
| `helpUrl`        | `string`              | `'/help'`      |
| `helpText`       | `string`              | `'Need help?'` |
| `settingsButton` | `ReactNode`           | —              |
| `compact`        | `boolean`             | `false`        |
| `maxWidth`       | `Breakpoint \| false` | `'lg'`         |
| `sx`             | `SxProps<Theme>`      | —              |

With `compact`, content is centered in a `448px` column; otherwise it
fills a `<Container maxWidth={maxWidth}>`. Companion:
`SimpleCompactContent` (`children` / `sx`) for the compact wrapper alone.

## `<LayoutProvider>`

`DashboardLayout` mounts a `LayoutProvider` for you. Use it directly
only when you build a custom dashboard-style shell from the primitives
and need access to layout config + nav state.

```tsx
import { LayoutProvider, useLayoutContext } from '@omnitron-dev/prism/layouts';

<LayoutProvider initialConfig={{ navigationMenuType: 'combo' }} persistKey="admin">
  <CustomShell />
</LayoutProvider>
```

| Prop            | Type                    | Default |
| --------------- | ----------------------- | ------- |
| `children`      | `ReactNode`             | —       |
| `initialConfig` | `Partial<LayoutConfig>` | —       |
| `persistKey`    | `string`                | —       |

`LayoutConfig` holds: `navigationMenuType` (`'sidenav' \| 'topnav' \| 'combo'`),
`sidenavVariant` (`'default' \| 'mini' \| 'stacked'`), `sidenavCollapsed`,
`topnavVariant`, `navColor` (`'default' \| 'vibrant' \| 'integrate'`),
`drawerOpen`, `drawerWidth`, and `compactLayout`.

### Hooks

| Hook                     | Returns                                                                                                                                                                                                                                                                       |
| ------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `useLayoutContext()`     | Full context: `config`, `isMobile`, `setConfig`, `toggleDrawer`, `toggleSidenavCollapse`, `setNavigationMenuType`, `setSidenavVariant`, `setNavColor`, `openItems` / `toggleItem` / `isItemOpen`, `isNavItemActive`, `isNestedItemActive`. Throws outside a `LayoutProvider`. |
| `useLayoutConfig()`      | Just the current `LayoutConfig`.                                                                                                                                                                                                                                              |
| `useSidenavVisible()`    | `true` for `'sidenav'` / `'combo'`.                                                                                                                                                                                                                                           |
| `useTopnavVisible()`     | `true` for `'topnav'` / `'combo'`.                                                                                                                                                                                                                                            |
| `useNavActive(pathname)` | `{ isItemActive, isNestedActive }` for a given pathname (pass `useLocation().pathname` from your router).                                                                                                                                                                     |

```tsx
import { useLayoutContext } from '@omnitron-dev/prism/layouts';

function CollapseToggle() {
  const { config, toggleSidenavCollapse } = useLayoutContext();
  return (
    <IconButton onClick={toggleSidenavCollapse}>
      {config.sidenavCollapsed ? <MenuOpenIcon /> : <MenuIcon />}
    </IconButton>
  );
}
```

## Core primitives (custom shells)

When none of the pre-built shells fit, compose your own from the core
primitives. There is no `<CoreLayout>` — `LayoutSection` is the root
container, `HeaderSection` is the slotted header, and `MainSection` is a
padded `<main>`.

```tsx
import {
  LayoutSection,
  MainSection,
  HeaderSection,
  Sidenav,
} from '@omnitron-dev/prism/layouts';

function CustomShell() {
  return (
    <LayoutSection
      headerSection={
        <HeaderSection
          slots={{ leftArea: <Logo />, rightArea: <UserMenu user={user} /> }}
        />
      }
      sidebarSection={<Sidenav navData={navData} />}
      footerSection={<Footer />}
    >
      <MainSection>
        <Outlet />
      </MainSection>
    </LayoutSection>
  );
}
```

### `LayoutSection`

| Prop             | Type                               | Notes                                        |
| ---------------- | ---------------------------------- | -------------------------------------------- |
| `headerSection`  | `ReactNode`                        | Rendered at the top                          |
| `sidebarSection` | `ReactNode`                        | Renders a flex sidebar+main row when present |
| `footerSection`  | `ReactNode`                        | Rendered at the bottom of the main column    |
| `children`       | `ReactNode`                        | Main content                                 |
| `cssVars`        | `Record<string, string \| number>` | Override layout CSS variables                |
| `sx`             | `SxProps<Theme>`                   | —                                            |

`MainSection` takes just `children` + `sx` (padded `<main>` flex column).

### `HeaderSection`

A sticky, slot-based `AppBar` with a scroll blur/elevation effect. Used
internally by the dashboard and split/simple auth layouts; usable
standalone.

| Prop               | Type                             | Default | Notes                                                              |
| ------------------ | -------------------------------- | ------- | ------------------------------------------------------------------ |
| `slots`            | `HeaderSlots`                    | —       | `topArea` / `leftArea` / `centerArea` / `rightArea` / `bottomArea` |
| `slotProps`        | per-slot `SxProps` + `container` | —       | Style each slot                                                    |
| `disableSticky`    | `boolean`                        | `false` | Use `position="static"`                                            |
| `disableOffset`    | `boolean`                        | `false` | Disable the scroll-blur background layer                           |
| `disableElevation` | `boolean`                        | `false` | Disable the scroll shadow                                          |
| `containerized`    | `boolean`                        | `false` | Wrap the toolbar in a max-width `Container`                        |
| `maxWidth`         | `Breakpoint \| false`            | `'lg'`  | Container width when `containerized`                               |
| `sx`               | `SxProps<Theme>`                 | —       | —                                                                  |

Companion `HeaderToolbar` (`children` / `sx`) is the bare toolbar
without the `AppBar` wrapper.

### `UserMenu`

Avatar-trigger account dropdown, designed for the header `rightArea`.

```tsx
import { UserMenu } from '@omnitron-dev/prism/layouts';

<UserMenu
  user={{ name: 'Ada Lovelace', email: 'ada@example.com', role: 'Admin' }}
  menuItems={[
    { key: 'profile', label: 'Profile', onClick: goToProfile },
    { key: 'logout', label: 'Sign out', destructive: true, onClick: signOut },
  ]}
  showEmail
/>
```

| Prop                 | Type                                                   | Default      |
| -------------------- | ------------------------------------------------------ | ------------ |
| `user`               | `UserMenuUser` (`{ name; email?; avatarUrl?; role? }`) | — (required) |
| `menuItems`          | `UserMenuItem[]`                                       | —            |
| `header` / `footer`  | `ReactNode`                                            | —            |
| `avatarSize`         | `'small' \| 'medium' \| 'large'`                       | `'medium'`   |
| `showEmail`          | `boolean`                                              | —            |
| `showRole`           | `boolean`                                              | —            |
| `sx` / `popoverSx`   | `SxProps<Theme>`                                       | —            |
| `onOpen` / `onClose` | `() => void`                                           | —            |

Each `UserMenuItem` is `{ key, label, icon?, onClick?, destructive?, disabled?, dividerAfter? }`.

## Mixing layouts in one app

Layouts are just route-element wrappers — react-router 7 handles the rest:

```tsx
<Routes>
  {/* Auth flow */}
  <Route element={<GuestGuard><AuthCenteredLayout logo={<Logo />}><Outlet /></AuthCenteredLayout></GuestGuard>}>
    <Route path="/sign-in" element={<SignInForm />} />
    <Route path="/sign-up" element={<SignUpForm />} />
  </Route>

  {/* Authenticated app */}
  <Route element={<AuthGuard><ConsoleShell /></AuthGuard>}>
    <Route path="/dashboard" element={<DashboardPage />} />
    <Route path="/apps" element={<AppsPage />} />
  </Route>

  {/* Error pages */}
  <Route path="*" element={<AuthSimpleLayout logo={<Logo />} compact><NotFound /></AuthSimpleLayout>} />
</Routes>
```

(`ConsoleShell` is the `DashboardLayout` wrapper shown at the top of
this page; it renders `<Outlet />` for the matched child routes.)

## See also

- [Blocks](/docs/frontend/prism/blocks) — composites that often sit inside layouts
- [Components catalog](/docs/frontend/prism/components) — components used in custom layouts
- [Theme](/docs/frontend/prism/theme) — density / spacing tokens

---

---
id: "frontend/prism/maps"
title: "Maps"
description: "Self-hosted MapLibre maps, markers, a point picker and a coverage layer."
---

# Maps

Prism ships reusable map components built on `react-map-gl/maplibre`. They pick up
the Prism theme, and they are written for a platform whose tiles are its own —
Prism makes no automatic requests to public providers. The host is responsible
for every URL referenced by its styles, fonts and tile sources.

```tsx
import { Map, MapMarker, MapPointPicker, MapCoverageLayer, MapPlaceSearch } from '@omnitron-dev/prism/components/map';
import '@omnitron-dev/prism/components/map/styles.css';
```

## The tile source is yours

`<Map>` never falls back to a hosted tile provider. A style must come from one
of three sources, in order of precedence:

1. `styleUrl` accepts a style JSON URL or an inline `StyleSpecification`.
2. `styles={{ light, dark }}` supplies complete styles for both Prism themes.
3. The host can inject `window.__MAP_STYLE_URL__` at boot.

A missing style displays a localizable alert. A progress indicator tracks loading, and a warning reports source errors.

The map entry registers `pmtiles://` before creating a map. Reference your own
HTTP-served PMTiles archive from a vector or raster source in the style.

## `<Map>`

| Prop               | Type                                    | Default                        | Effect                                          |
| ------------------ | --------------------------------------- | ------------------------------ | ----------------------------------------------- |
| `styleUrl`         | `string \| StyleSpecification`          | host configuration, then alert | Vector tile style                               |
| `viewport`         | `Partial<MapViewport>`                  | —                              | Controlled camera; pair with `onViewportChange` |
| `initialViewport`  | `Partial<MapViewport>`                  | world, zoom 1                  | Defaults when uncontrolled                      |
| `onViewportChange` | `(next: MapViewport) => void`           | —                              | Every drag, zoom and rotate                     |
| `onClick`          | `(e: { lngLat: { lng, lat } }) => void` | —                              | Click on the canvas                             |
| `showNavigation`   | `boolean`                               | `true`                         | Zoom + compass control                          |
| `showScale`        | `boolean`                               | `true`                         | Scale control                                   |
| `showAttribution`  | `boolean`                               | `true`                         | Attribution control                             |
| `style`            | `CSSProperties`                         | 400 px tall                    | **Pass `height` here**                          |
| `styles`           | `{ light: MapStyle; dark: MapStyle }`   | —                              | Theme-specific styles                           |
| `messages`         | `Partial<MapMessages>`                  | English                        | Localized label and status text                 |

The container is 400 px tall unless you say otherwise, and height goes in
`style` rather than in a prop of its own.

## `<MapMarker>`

A pin at a coordinate, themed by default and replaceable wholesale.

| Prop                     | Type                                                 | Effect                                                  |
| ------------------------ | ---------------------------------------------------- | ------------------------------------------------------- |
| `longitude` / `latitude` | `number`                                             | Position                                                |
| `color`                  | `string`                                             | Fill; defaults to the theme primary                     |
| `children`               | `ReactNode`                                          | Custom DOM instead of the default pin                   |
| `popup`                  | `ReactNode`                                          | Popup body, opened on click                             |
| `anchor`                 | `'center' \| 'top' \| 'bottom' \| 'left' \| 'right'` | Default `bottom`, so the pin tip sits on the coordinate |
| `onClick`                | `() => void`                                         | Click-through handler                                   |

## `<MapPointPicker>`

Controlled coordinate selection by map click, marker drag or the keyboard-accessible
"Select map centre" action. `labels` localizes every picker label. `selectionZoom`
(default 10) controls focus after selecting a point. `readOnly`, `showDetails`,
`onClear` and typed `mapProps` adapt the picker without an application-specific copy.
Longitude wraps at the antimeridian; latitude is validated in \[-90,90]. The
Mercator camera limit never alters the stored geographical coordinate.

```tsx
const [point, setPoint] = useState<MapPointPickerValue | null>(null);

<MapPointPicker
  value={point}
  onChange={setPoint}
  height={480}
  geocodeResult={address}        // your RPC result, rendered in the side panel
  geocodeLoading={isGeocoding}
  placeholder="Click the map to choose a location"
/>
```

**Reverse geocoding is the host app's job.** The picker takes
`geocodeResult` and `geocodeLoading` and renders them; it does not know which
backend to ask, and deliberately so — the component would otherwise carry a
dependency on one app's RPC client.

## `<MapCoverageLayer>`

`MapCoverageLayer` draws its points in a single MapLibre circle layer. Enable
`showLabels` to add a symbol layer for names. Both layers use the same GeoJSON source.

```tsx
<MapCoverageLayer
  id="drops"
  points={drops.map((d) => ({
    id: d.id,
    longitude: d.lng,
    latitude: d.lat,
    status: d.status,     // bucket → colour
    label: d.title,       // shown at zoom 11+ when showLabels is enabled
  }))}
  colorByStatus={{ active: 'green', expired: 'grey' }}
  showLabels
/>
```

The default palette covers `active`, `pending`, `expired`, `disabled` and `retired`.
Use `colorByStatus` to add or override colors. `radius` defaults to 6 pixels at every zoom.

## `<MapPlaceSearch>`

The host supplies a provider function, typically a Netron call. The component
has no knowledge of authentication, provider credentials or public services.
Default `mode="submit"` performs a request only after explicit submission;
`mode="autocomplete"` opts into debounced search when the provider permits it.
New input aborts the previous request, and stale responses cannot replace the
current results. Provider ranking is preserved; invalid coordinates are rejected.

```tsx
<MapPlaceSearch
  search={(query, { signal, limit }) => placeProvider.search(query, { signal, limit })}
  onSelect={(place) => setPoint(place.point)}
  labels={{ label: 'Место рождения', search: 'Найти' }}
/>
```

Every result has `id`, `label` and `point: { lng, lat }`. Generic result types
preserve provider metadata. `labels`, `minLength`, `limit`, `debounceMs` and
`disabled` configure the control. Failure, empty result and loading are distinct.
The host should propagate aborts to its transport when supported.

## Related

- [Components catalog](/docs/frontend/prism/components) — the rest of the widget set.
- [Theme](/docs/frontend/prism/theme) — the palette these components read.

---

---
id: "frontend/prism/research"
title: "Research workspaces"
description: "Scientific documents, block editing and typed knowledge graphs with a shared MUI theme."
---

# Research workspaces

Prism supplies independently composable components for research applications:

| Component          | Import                                           | Responsibility                                                                          |
| ------------------ | ------------------------------------------------ | --------------------------------------------------------------------------------------- |
| `ResearchDocument` | `@omnitron-dev/prism/components/research`        | Render Markdown, equations, diagrams, source metadata and registered interactive blocks |
| `ResearchEditor`   | `@omnitron-dev/prism/components/research-editor` | Edit scientific blocks and deliver an exact source snapshot                             |
| `MathInput`        | `@omnitron-dev/prism/components/research-editor` | Edit LaTeX visually in place, or through its source field                               |
| `KnowledgeGraph`   | `@omnitron-dev/prism/components/knowledge-graph` | Inspect entities and typed relations through a graph or searchable index                |

Import the corresponding `styles.css` subpath for each component. Render them
under the application's Prism theme. Buttons, search fields, insertion dialogs
and block menus use MUI, so application component overrides and locale settings
apply inside the workspace too.

`parseResearchMarkdown` recognizes inline and block equations without loading an
equation renderer. The reader and editor render through Prism's declared KaTeX
dependency; the Markdown tokenizer ships with Prism. Consumers do not need to
override a second renderer version.

## One visual system

Colours come from `theme.palette`; editor and graph radii come from
`theme.shape.borderRadius`. Override these through the
[Prism theme](/docs/frontend/prism/theme), rather than adding global `button` or `input` CSS.
The scientific components also expose two inherited CSS variables:

```css
.research-workspace {
  --prism-border-width: 1px;
  --prism-focus-width: 2px;
}
```

The first controls structural boundaries. The second controls keyboard focus
and the editor's insertion indicator. Selection changes colour without moving
the layout. React Flow nodes, controls, edges and the searchable index inherit
the same palette. Native canvas controls remain scoped to the graph.

`className` and `sx` allow per-instance composition. Graph and editor labels are
configurable; pass a `labels` object to localize their controls. A graph's
`renderDetails` slot can contain application-specific revision and verification
information without coupling Prism to that application's backend.

## Keep document and knowledge semantics separate

`ResearchEditor` accepts `initialSource`. Its `onChange` callback delivers a
`ResearchSnapshot`; call `getSource()` to obtain the exact current source.
The imperative `ResearchEditorHandle.getSnapshot()` reads the same state,
including the last delivered transaction. Replace the React key when loading
a different edition, or explicitly reconcile an existing draft before replacing
it. `readOnly` disables editing on the contenteditable surface itself.

Registered scientific widgets use explicit type/version entries. A document
cannot authorize new executable widgets. Unsupported source blocks remain
preserved, and changing the host's widget registry does not discard the draft.

`KnowledgeGraph` takes `KnowledgeGraphData` with stable entity and relation
identities. Relations have ordered input/output ports: they are not flattened
into arbitrary pairwise edges. The component validates references before
projection and bounds the displayed neighbourhood with `visibleBudget`.
Omitted relations are counted explicitly. Dragging changes visual coordinates
through `onLayoutChange`; it does not change the knowledge relation.

Edge roles are measured and placed together, with nonoverlapping label boxes
that also avoid node bounds. Dotted leaders retain the association with displaced
labels; repeated roles are never merged. `edgeLabelGap` and `edgeLabelMaxWidth`
configure spacing and wrapping. Fit-to-view includes the label extent. Clicking
a line selects the relation; dragging continues to change only coordinates.

The host owns persistence, permissions, revision selection and proof checking.
A visual selection or rendered equation is not a proof certificate. Supply
verified status and diagnostics from the host through `renderDetails` and
ordinary Prism components.

## Equations in place

Inline and display equations open inside the document. Click an equation, or
enter an adjacent inline equation with an arrow key. `Ctrl/Command + Shift + M`
inserts an inline equation. `Escape` or `Enter` returns to the document;
`Shift + Enter` remains available to MathLive. Boundary arrows navigate out of
the formula. A display equation with no reachable text neighbour creates one
editable paragraph on exit. Formula edits participate in the document's
undo/redo history, including edits from the LaTeX source field.

```tsx
<ResearchEditor
  initialSource={source}
  onChange={(snapshot) => saveDraft(snapshot.getSource())}
  math={{
    locale: 'en',
    maxLength: 16_384,
    labels: { field: 'Equation', done: 'Return to document' },
    options: { mathVirtualKeyboardPolicy: 'manual' },
  }}
/>
```

`math.labels` localizes the host controls; `math.locale` selects MathLive's
locale. MathLive uses a document-wide locale and virtual keyboard. Editors on
one page should share the application's locale. The host may configure macros,
inline shortcuts and keybindings through `math.options`. These are initial,
trusted settings: remount the input to replace them. Imported document text
never supplies executable configuration.

`MathInput` can also be used independently. Supply controlled `value` and
`onChange`; returning `false` rejects the edit synchronously. `onExit` hands
focus back to the host, and `onHistory` delegates undo/redo to its transaction
history. `readOnly`, labels and value remain reactive. The host owns draft
storage and conflict resolution.

The visual engine is [MathLive](https://mathlive.io/mathfield/guides/integration/),
loaded on first edit. Fonts are bundled locally; the component does not load
CDN assets, sound files or a compute engine. MUI controls and keyboard colours
inherit the Prism theme. A sanitizer preserves mathematical layout, the editable
input sink and local SVG icons while removing active markup and external resource
references. If visual loading fails, the LaTeX source field remains usable.

An untouched block retains its exact source, including line endings. An edited
block is serialized as Markdown/LaTeX; neighbouring untouched blocks remain
byte-identical. Unsupported math metadata stays in the source-preserving block
editor. An empty visual slot is stored explicitly as `\placeholder{}`. Syntax
errors remain visible with their source. Input and document limits reject a
change atomically rather than saving a truncated expression.

This is notation editing. It does not establish mathematical equivalence,
validate a proof or grant a certificate; those operations belong to the host's
verification service. Length limits do not establish a time bound for arbitrary
LaTeX or trusted user-defined macros.

### Observation badges in knowledge graphs

`KnowledgeEntity` and `KnowledgeRelation` accept optional `badges: readonly KnowledgeBadge[]`.
Each badge has a `label` and an optional MUI `tone` (`default`, `info`, `success`, `warning`, `error`).
The canvas and index render the same wrapping MUI chips; badge text participates in index search.
Updating observations preserves object identity, selection and the incidence structure.

Badges are host-supplied observations. Prism does not infer proof validity or dependency currency
from a colour, a relation or a successful layout. Keep independent dimensions separate, for example
`Judgment verified` and `Review dependencies`, and retain the exact observation identity in the inspector.

---

---
id: "frontend/prism/state"
title: "State and persistence"
description: "Local UI stores, persisted preferences, migrations and the boundary with server state."
---

# State and persistence

Import store factories from `@omnitron-dev/prism/state`. They wrap Zustand
with Immer updates, optional persistence and DevTools integration. Use
these stores for local interface state. Keep remote query results in
[Netron's query cache](/docs/frontend/netron/caching), which owns their refresh and
invalidation rules.

## Choose a store

| Factory                                                    | Use it for                               | Persistence                        |
| ---------------------------------------------------------- | ---------------------------------------- | ---------------------------------- |
| `createUIStore(initializer, name)`                         | Panels, selection and transient controls | Disabled                           |
| `createPrismStore(initializer, options)`                   | A store with explicit middleware choices | Opt-in through `options.persist`   |
| `createPersistedStore(initializer, name, persistOptions?)` | Preferences that survive reloads         | Enabled; exposes the `persist` API |

```typescript
import { createUIStore } from '@omnitron-dev/prism/state';

interface Selection {
  selectedId: string | null;
  select: (id: string | null) => void;
}

export const useSelection = createUIStore<Selection>((set) => ({
  selectedId: null,
  select: (id) => set((state) => { state.selectedId = id; }),
}), 'selection');
```

Read the smallest slice in a component:

```tsx
function SelectionLabel() {
  const selectedId = useSelection((state) => state.selectedId);
  return <span>{selectedId ?? 'Nothing selected'}</span>;
}
```

Outside React, use the returned store's `getState()` and `subscribe()`.
Call the unsubscribe function when the consumer is disposed.

## Built-in stores

| Hook               | Responsibility                                                                                |
| ------------------ | --------------------------------------------------------------------------------------------- |
| `useSettingsStore` | Theme preferences, density, navigation and application settings                               |
| `useThemeStore`    | The standalone theme store; do not create a second source of truth for a `PrismProvider` tree |
| `useUIStore`       | Shared interface state                                                                        |
| `useAuthStore`     | Local authentication presentation state                                                       |

Each also exports a non-hook alias: `settingsStore`, `themeStore`,
`uiStore` and `authStore`. For a provider-backed application, follow the
[theme guide](/docs/frontend/prism/theme) and update `useSettingsStore`.
Authentication state in a browser store does not enforce server access;
use the [Netron auth provider](/docs/frontend/netron/auth) for RPC sessions.

## Persist preferences deliberately

```typescript
import { createPersistedStore } from '@omnitron-dev/prism/state';

interface Preferences {
  compact: boolean;
  setCompact: (compact: boolean) => void;
}

export const usePreferences = createPersistedStore<Preferences>(
  (set) => ({
    compact: false,
    setCompact: (compact) => set((state) => { state.compact = compact; }),
  }),
  'preferences',
  { version: 1 },
);
```

The persisted factory catches unavailable-storage errors. Its default
migration retains fields whose names and basic shapes still match the
current defaults. A changed domain model needs an explicit migration:
matching primitive types does not establish semantic compatibility.

The `persist` surface includes `hasHydrated`, `rehydrate`, `clearStorage`,
`onHydrate`, `onFinishHydration`, `getOptions` and `setOptions`. Check
hydration before rendering a preference-dependent screen; dispose the
hydration listeners on unmount.

For structured migrations, the same subpath exports `runMigrations`,
`createVersionedStorage`, `createVersionedPersistOptions`,
`createMigration`, `createRenameMigration`, `createAddFieldMigration`,
`createRemoveFieldMigration` and `createTransformMigration`.
`validateMigrations` and `getMigrationInfo` help inspect a migration chain.
`exportSettings`, `importSettings`, `checkSettingsVersion`,
`clearSettings` and `createSettingsValidator` support preference import
and recovery. Consult the [API index](/docs/reference/api-index)
for each current signature.

## Server rendering and tests

Create request-specific stores on the server; a module-level mutable store
can otherwise share user state between requests. Browser persistence is
available only after hydration. Render a stable initial state on both
sides, then apply persisted preferences.

Test initial state, actions, reload, a version upgrade and unavailable
storage. Use isolated storage keys and reset stores between tests.
Never put passwords, private keys or an authoritative permission decision
in a persisted UI store.

---

---
id: "frontend/prism/theme"
title: "Theme"
description: "Palette, typography, shadows, dark mode, presets."
---

# Theme

Prism's theme is MUI v9's theme + extensions (custom shadows,
density tokens, brand presets, runtime primary-color override). The
factory is `createPrismTheme` — it builds a fully-formed MUI theme from
a small set of high-level options.

```tsx
import { createPrismTheme } from '@omnitron-dev/prism/theme';

const theme = createPrismTheme({
  preset:       'luxury',   // brand palette (see Presets below)
  mode:         'dark',     // 'light' | 'dark' | 'system'
  primaryColor: '#7c4dff',  // runtime primary override (hex)
  borderRadius: 8,          // base radius in px (not `shape`)
  density:      'standard', // 'compact' | 'standard' | 'comfortable'
});
```

> Most apps never call `createPrismTheme` directly — `<PrismProvider>`
> builds the theme from the settings store for you. Reach for the factory
> when you need a theme outside React (SSR, Storybook, tests).

## Palette

`createPrismTheme` does not take a raw MUI `palette` object. Instead you
pick a `preset` and (optionally) a `primaryColor`, and Prism builds the
full palette — including light/dark color schemes and channel variants —
for you. The resulting `theme.palette` is a standard MUI palette:

```typescript
theme.palette = {
  mode: 'light' | 'dark',
  primary:   { main, light, dark, contrastText },
  secondary: { main, light, dark, contrastText },
  error / warning / info / success: { main, light, dark, contrastText },
  text:       { primary, secondary, disabled },
  background: { default, paper, neutral },   // 'neutral' for subtle section bg
  action:     { active, hover, selected, disabled, disabledBackground, focus },
  divider,
}
```

To set the brand colour, pass `primaryColor: '#7c4dff'` — Prism derives
the light/dark shades and contrast text. For deeper customisation, pass
through raw MUI options via the `overrides` option
(`createPrismTheme({ overrides: { palette: { … } } })`).

### Semantic colour tokens

Avoid hex codes in app code — use theme tokens:

```tsx
<Box sx={{ color: 'primary.main', bgcolor: 'background.paper' }} />
<Typography color="text.secondary">…</Typography>
```

This keeps dark mode working automatically and lets you re-skin
without touching component code.

### Readable status surfaces

Standard and soft alerts, soft chips and soft labels share one contrast
policy. Prism compares the semantic colour tones against their translucent
fill composited over `background.paper`, selecting a tone with at least
4.5 contrast when available. It preserves the colour token and emits both
scheme rules, so custom CSS variable prefixes and live light/dark switching
continue to work. This also covers MUI `Alert` and `Chip` used directly under
`PrismProvider` or `createPrismTheme`.

Custom palettes can use either ordinary MUI tonal roles or Prism's inverted
dark scale. When none of the supplied tones is readable, or a colour can
only be resolved by the browser, Prism uses `text.primary`. Inverted labels
use the theme's white surface and grey text fallback. Browser-resolved
colours, custom parent backgrounds and application `sx` overrides still
need contrast checks in the final UI; the calculation assumes an opaque
`background.paper` beneath the translucent fill.

An alert's `color` overrides its appearance; `severity` still selects its
meaning and icon. For example, `<Alert severity="warning" color="error">`
uses the error palette with the warning icon. Use text as well as colour
to explain the state.

## Typography

```typescript
typography: {
  fontFamily:     '"Inter", "Roboto", sans-serif',
  fontSize:       14,            // base px
  htmlFontSize:   16,            // for rem conversion
  fontWeightLight:     300,
  fontWeightRegular:   400,
  fontWeightMedium:    500,
  fontWeightBold:      700,

  h1: { fontSize: '2.5rem', fontWeight: 700, lineHeight: 1.2 },
  h2: { fontSize: '2rem',   fontWeight: 700, lineHeight: 1.25 },
  // … h3, h4, h5, h6
  subtitle1: { fontSize: '1rem',     fontWeight: 600, lineHeight: 1.5 },
  subtitle2: { fontSize: '0.875rem', fontWeight: 600, lineHeight: 1.5 },
  body1:     { fontSize: '1rem',     fontWeight: 400, lineHeight: 1.5 },
  body2:     { fontSize: '0.875rem', fontWeight: 400, lineHeight: 1.5 },
  caption:   { fontSize: '0.75rem',  fontWeight: 400, lineHeight: 1.4 },
  overline:  { fontSize: '0.625rem', fontWeight: 700, letterSpacing: '0.1em', textTransform: 'uppercase' },
  button:    { fontWeight: 600, textTransform: 'none' },     // Prism default: no SHOUTY buttons
}
```

Apply via `<Typography>` or `sx`:

```tsx
<Typography variant="h3">Section title</Typography>
<Box sx={{ typography: 'body2', color: 'text.secondary' }}>Small print</Box>
```

## Spacing

`spacing: 8` means `theme.spacing(N) === 8 × N` pixels. The
`sx` shorthand interprets numbers in spacing units:

```tsx
<Box sx={{ p: 2, mt: 4, gap: 1 }} />
// padding: 16px, margin-top: 32px, gap: 8px
```

Convention: use the spacing unit (`p: 2`) not pixel values
(`p: '16px'`) — keeps the rhythm consistent and tunable.

## Shape & shadows

Set the base radius via the `borderRadius` option (a number of px) —
`createPrismTheme({ borderRadius: 8 })`. It surfaces as `theme.shape.borderRadius`
on the resulting MUI theme. Shadows are generated for you:

```typescript
shadows: [
  'none',
  '0 1px 2px rgba(0,0,0,0.05)',
  '0 1px 3px rgba(0,0,0,0.07), 0 1px 2px rgba(0,0,0,0.06)',
  // … 24 steps
]

customShadows: {
  z1:   '...',     // softer cards
  z4:   '...',     // floating panels
  z8:   '...',     // modals
  z16:  '...',     // popovers
  primary:   '...', // primary-tinted shadow
  secondary: '...',
  error: '...',
  warning: '...',
  info: '...',
  success: '...',
}
```

Use `<Paper elevation={N}>` for `shadows`; `boxShadow:
'customShadows.z4'` via `sx` for the custom variants.

## Dark mode

`<PrismProvider>` reads the theme mode from the settings store, so it
takes no `mode` prop — seed the initial mode with `defaultSettings`, then
read and update it via `useSettingsStore`. There is no `useColorMode`
hook.

```tsx
import { PrismProvider } from '@omnitron-dev/prism/core';
import { useSettingsStore } from '@omnitron-dev/prism';

function App() {
  return (
    <PrismProvider defaultSettings={{ mode: 'system' }}>
      <Outlet />
    </PrismProvider>
  );
}

function ModeToggle() {
  const mode       = useSettingsStore((s) => s.mode);
  const toggleMode = useSettingsStore((s) => s.toggleMode);
  return (
    <IconButton onClick={toggleMode}>
      {mode === 'dark' ? <SunIcon /> : <MoonIcon />}
    </IconButton>
  );
}
```

| Mode       | Effect                       |
| ---------- | ---------------------------- |
| `'light'`  | Force light                  |
| `'dark'`   | Force dark                   |
| `'system'` | Track `prefers-color-scheme` |

`mode` is the stored preference; the provider resolves `'system'` against
`prefers-color-scheme` internally before building the theme. State is
persisted via the settings store; survives reload; syncs across tabs.
(`toggleMode` flips light/dark; use `setMode('system')` to re-enable
system tracking.)

### Flash-of-wrong-theme prevention (SSR)

Prism's CSS variables output (`theme/css-variables.ts`) emits
a stylesheet that applies the correct colour scheme **before**
React hydrates. Include in your HTML head:

```html
<style id="prism-css-vars">
  /* Generated by generateCssVariables(theme) at build time */
</style>
```

Or use the Vite plugin (when one is configured) to inject it
automatically.

## Presets

Pre-tuned palettes for common aesthetics. Select one by name via the
`preset` option:

```tsx
const theme = createPrismTheme({ preset: 'luxury' });
```

Available presets (`PRESET_NAMES`): `default-light`, `default-dark`,
`luxury`, `arctic`, `nature`, `ember`, `dracula`, `midnight`, `retro`,
`minimal`. Non-default presets carry an inherent light/dark mode
(e.g. `dracula`/`midnight`/`ember` are dark; `luxury`/`arctic`/`nature`
are light) — selecting them forces that mode, while the two `default-*`
presets respect the `mode` option. Pick a preset, then layer
`primaryColor` / `borderRadius` / `density` on top.

## Density

Three densities adjust spacing + typography + control sizes:

```tsx
useSettingsStore.getState().setDensity('compact');
// 'compact' | 'standard' | 'comfortable'
```

Tables, lists, forms, dialogs respond automatically. Useful for
information-dense admin surfaces.

## Mixins

Reusable style helpers (most are functions returning a `CSSObject` you
spread into `sx`):

```tsx
import { mixins } from '@omnitron-dev/prism/theme';

<Box sx={mixins.textGradient('to right', '#7c4dff', '#00bcd4')}>Gradient text</Box>
<Box sx={mixins.bgBlur({ blur: 6 })}>Backdrop blur card</Box>
<Box sx={(theme) => mixins.scrollbarStyles(theme)}>Custom scrollbar</Box>
```

Built-in mixins (keys on the `mixins` object):

| Mixin                                                  | Effect                                                       |
| ------------------------------------------------------ | ------------------------------------------------------------ |
| `textGradient(direction, ...colors)`                   | Gradient text fill                                           |
| `bgGradient({ direction, colors, ... })`               | Linear gradient background                                   |
| `bgBlur({ blur, color })`                              | Backdrop blur                                                |
| `borderGradient({ color, borderWidth, borderRadius })` | Gradient border                                              |
| `maxLine({ lines, lineHeight })`                       | Multi-line truncation with ellipsis                          |
| `scrollbarStyles(theme, { autoHide: true })`           | Native themed scrollbar; hover/focus reveal on fine pointers |
| `hideScrollX` / `hideScrollY`                          | Hidden-scrollbar containers (objects)                        |

(Standalone scrollbar/ellipsis helpers are also exported by name:
`customScrollbarMixin`, `hideScrollbarMixin`, `textEllipsisMixin`,
`multiLineEllipsisMixin`, `glassMixin`, `focusRingMixin`.)

## Components theme overrides

Pass raw MUI component overrides through the `overrides` option:

```typescript
createPrismTheme({
  overrides: {
    components: {
      MuiButton: {
        defaultProps:  { variant: 'contained', size: 'medium' },
        styleOverrides: {
          root: { borderRadius: 8, textTransform: 'none' },
        },
      },
      MuiTextField: {
        defaultProps: { variant: 'outlined', size: 'small' },
      },
    },
  },
});
```

Useful for enforcing project-wide conventions (e.g., "all
buttons are contained, no uppercase").

## CSS variables export

```tsx
import { generateCssVariables } from '@omnitron-dev/prism/theme';

const css = generateCssVariables(theme);
// → :root { --palette-primary-main: #7c4dff; ... }
```

Useful for:

- Server-side colour-scheme cookie + SSR.
- Sharing theme tokens with non-MUI elements (e.g., plain
  `<svg>` strokes that should track the theme).
- Embedding in `<style>` tag at the top of `<head>` to prevent
  flash.

## Anti-patterns

- **Hard-coded hex colours in components.** Use
  `theme.palette.X` / `sx={{ color: 'primary.main' }}` so dark
  mode + theme switches work.
- **`px` everywhere.** Use spacing units; relative `rem` for
  typography.
- **Multiple `<ThemeProvider>`s in the tree.** Confused
  inheritance, duplicate snackbar hosts.
- **Per-component `styled()` for one-off colours.** Prefer `sx`
  for one-offs; `styled()` for recurring patterns.
- **Bypassing the settings store for mode persistence.**
  Custom localStorage code drifts; use `useSettingsStore`
  (`mode` / `setMode` / `toggleMode`).
- **`:not(:first-of-type)` / `:nth-of-type` sibling selectors
  in `styleOverrides`.** MUI v9 dropped the legacy of-type
  pattern in favour of explicit position classes
  (`firstButton` / `middleButton` / `lastButton` on
  `ToggleButtonGroup`, similar tags on other grouped
  components). The of-type form silently misfires once
  consumers add wrapper elements or fragments; child-position
  (`:first-child` / `:last-child` / `:nth-child`) is the
  general-purpose replacement when no position class exists.
- **`forwardRef` wrappers in component code.** React 19
  threads `ref` through props automatically — declare
  `ref?: Ref<HTMLElement>` on the props interface and use
  it directly. No `forwardRef`, no `displayName` follow-up.

## See also

- [Components catalog](/docs/frontend/prism/components) — components themed via these tokens
- [Layouts](/docs/frontend/prism/layouts) — `<DashboardLayout>` reads density
- [Hooks catalog](/docs/frontend/prism/hooks-catalog) — `useSettingsStore`, theme-aware hooks
- [MUI v9 theming docs](https://mui.com/material-ui/customization/theming/) — underlying foundation

## Application icon families

`PrismIconProvider` from `@omnitron-dev/prism/icons` supplies a semantic icon set
to layouts, editors, graphs and MUI slots, including portals. Its `icons` prop
accepts a partial `PrismIconSet`; missing slots retain the parent or bundled icon.
Use `Record<PrismIconName, PrismIconComponent>` for compile-time coverage of every
slot, and `prismIconNames` to enumerate them. Supplied components accept
`SvgIconProps`, forward the SVG ref and preserve MUI accessibility semantics.

```tsx
import { PrismIconProvider, type PrismIconSet } from '@omnitron-dev/prism/icons';

const icons = { search: ApplicationSearchIcon } satisfies PrismIconSet;
<PrismIconProvider icons={icons}>
  <PrismThemeProvider theme={theme}>{children}</PrismThemeProvider>
</PrismIconProvider>;
```

The application supplies and licenses its artwork. The provider does not fetch
icons or replace rendered DOM. Component-level MUI slots and explicit icon props
take precedence; nested providers inherit the outer set.

---

---
id: "getting-started/developer-map"
title: "Developer map"
description: "Direct routes from a development task to the guide, API reference and verification step."
---

# Developer map

Choose the task you are doing now. Each route connects a working guide to
the reference needed when the application becomes more complex.

## Build a backend

| Task                      | Guide                                                                                           | Verify                                             |
| ------------------------- | ----------------------------------------------------------------------------------------------- | -------------------------------------------------- |
| Start a service           | [Quickstart](/docs/getting-started/quickstart) · [Lifecycle](/docs/titan/application/lifecycle) | Compile, make one RPC call, stop cleanly           |
| Organise dependencies     | [Module system](/docs/titan/modules-system/defining-modules) · [DI](/docs/titan/di/overview)    | Resolve the graph and test overrides               |
| Store data                | [Database](/docs/titan/modules/database)                                                        | Run migrations and test transaction rollback       |
| Authenticate calls        | [Token issuance](/docs/auth/token-issuance) · [RPC auth](/docs/titan/netron/authentication)     | Test missing, expired and insufficient credentials |
| Isolate tenants           | [Context guards](/docs/titan/modules/rls) · [Database](/docs/titan/modules/database)            | Attempt a cross-tenant read and write              |
| Coordinate work           | [Locks](/docs/titan/modules/lock) · [Scheduler](/docs/titan/modules/scheduler)                  | Test lock loss, overlapping jobs and shutdown      |
| Recover from dependencies | [Resilience](/docs/titan/resilience/overview)                                                   | Exercise timeout, exhaustion and recovery          |

## Build an interface

| Task                   | Guide                                                                                         | Verify                                           |
| ---------------------- | --------------------------------------------------------------------------------------------- | ------------------------------------------------ |
| Call RPC from any UI   | [Browser client](/docs/frontend/netron/browser)                                               | Connect, call, handle a typed error, disconnect  |
| Query from React       | [React bindings](/docs/frontend/netron/react) · [Caching](/docs/frontend/netron/caching)      | Loading, empty, error, mutation and invalidation |
| Compose a screen       | [Prism components](/docs/frontend/prism/components) · [Layouts](/docs/frontend/prism/layouts) | Keyboard navigation and narrow viewport          |
| Capture input          | [Forms](/docs/frontend/prism/forms)                                                           | Client validation and server rejection           |
| Persist preferences    | [State](/docs/frontend/prism/state) · [Theme](/docs/frontend/prism/theme)                     | Hydration and a persisted-state migration        |
| Call a REST endpoint   | [HTTP client](/docs/frontend/prism/http)                                                      | Cancellation and explicit retry policy           |
| Adapt component source | [Prism CLI](/docs/frontend/prism/cli)                                                         | Review copied code and registry diff             |
| Validate accessibility | [Accessibility](/docs/frontend/prism/accessibility)                                           | Focus, labels, both themes and zoom              |

## Test and operate

| Task                   | Guide                                                                                 | Verify                                                 |
| ---------------------- | ------------------------------------------------------------------------------------- | ------------------------------------------------------ |
| Test a module          | [Testing](/docs/testing) · [Test helpers](/docs/testing/testing-package)              | Await cleanup; isolate state and storage               |
| Supervise applications | [Operations](/docs/omnitron/overview) · [Configuration](/docs/omnitron/configuration) | Health, restart and shutdown behaviour                 |
| Ship a release         | [Release lifecycle](/docs/omnitron/releases)                                          | Build, gate, deploy, attest and rehearse rollback      |
| Diagnose a deployment  | [CLI](/docs/omnitron/cli) · [Observability](/docs/omnitron/observability)             | Inspect the actual runtime and its diagnostic findings |
| Prepare production     | [Readiness](/docs/deployment/production-readiness)                                    | Exercise failure and recovery procedures               |

## Find the exact API

Use the [package index](/docs/reference/packages) to choose a dependency
and the [API index](/docs/reference/api-index) to find an export or import
path. The [module map](/docs/titan/modules/module-map) explains composition.
The [glossary](/docs/reference/glossary) resolves terminology.

Pages marked **Design RFC** describe application architecture that requires
additional implementation. Read the shipped module reference before
adopting an RFC's code shape.

---

---
id: "getting-started/installation"
title: "Installation"
description: "Install only what you need. Server-side Netron ships inside Titan; the browser client is a separate package."
---

# Installation

The public `@omnitron-dev/` packages share a release version.
Install the packages your application uses.

> **Important to understand upfront:**
>
> - **Server-side Netron** (RPC framework with all 4 transports —
>   HTTP, WebSocket, TCP, Unix) is **part of
>   `@omnitron-dev/titan`** itself. It's exposed via subpath
>   exports like `@omnitron-dev/titan/netron`. **You do not
>   install a separate package** for the server.
> - **`@omnitron-dev/netron-browser`** is the **browser-side
>   Netron client** — usable with **any frontend** (vanilla JS,
>   React, Vue, Svelte, Solid, Angular, Lit, …). It's not tied
>   to React.
> - **`@omnitron-dev/netron-react`** sits on top of
>   `netron-browser` and adds React-specific hooks / providers /
>   cache integration. Install it **only** if you use React.

## Prerequisites

| Tool                     | Version                               | Why                                                                                                                        |
| ------------------------ | ------------------------------------- | -------------------------------------------------------------------------------------------------------------------------- |
| **Node.js for Omnitron** | 22.14+ on 22.x, 23.6+ on 23.x, or 24+ | The daemon and CLI use SQLite through a native Node-API 10 binding.                                                        |
| **pnpm**                 | 12.16.1                               | Use the exact `packageManager` version for source builds; npm/yarn work for published-package consumers                    |
| **TypeScript**           | 7.x native compiler                   | Source builds use native `tsc`. Libraries that consume the Compiler API use the pinned TypeScript 6 compatibility package. |
| **Bun** (optional)       | 1.x                                   | Titan apps may run on Bun; Omnitron daemon needs Node                                                                      |
| **Deno** (optional)      | 2.x                                   | Titan apps may run on Deno; Omnitron daemon needs Node                                                                     |
| **Docker** (optional)    | latest                                | Required for `omnitron infra` provisioning                                                                                 |

## Backend — `@omnitron-dev/titan`

The backend framework. **This is the only package required to
build a Titan service.** Server-side Netron — including every
transport — is already inside.

```bash
pnpm add @omnitron-dev/titan
```

Subpath imports (use any of these — they're all part of the
single `@omnitron-dev/titan` package):

```typescript
// Core framework:
import { Application, Module, Service, Injectable }
  from '@omnitron-dev/titan';

// Lifecycle interfaces. The root exports OnInit and OnDestroy only;
// the other two come from the application subpath:
import { OnInit, OnDestroy }   from '@omnitron-dev/titan';
import { OnStart, OnStop }     from '@omnitron-dev/titan/application';

// Server-side Netron API. The auth managers live under their own
// subpath — `netron` itself does not re-export them:
import { Netron, ServiceDescriptor }        from '@omnitron-dev/titan/netron';
import { AuthenticationManager, AuthorizationManager }
  from '@omnitron-dev/titan/netron/auth';

// Per-transport server (each can be enabled independently):
import { HttpTransport }      from '@omnitron-dev/titan/netron/transport/http';
import { WebSocketTransport } from '@omnitron-dev/titan/netron/transport/websocket';
import { TcpTransport }       from '@omnitron-dev/titan/netron/transport/tcp';
import { UnixTransport }      from '@omnitron-dev/titan/netron/transport/unix';

// Server-side HTTP middleware. Authentication is a factory, and the
// built-ins (rate limiting among them) are static methods on one class:
import { createAuthMiddleware, NetronBuiltinMiddleware }
  from '@omnitron-dev/titan/netron/transport/http/middleware';

// Multi-backend (server-side):
import { MultiBackendClient, createMultiBackendClient }
  from '@omnitron-dev/titan/netron/multi-backend';

// Built-in modules:
import { ConfigModule } from '@omnitron-dev/titan/module/config';
import { LoggerModule } from '@omnitron-dev/titan/module/logger';

// DI container primitives (advanced):
import { Container, createToken } from '@omnitron-dev/titan/nexus';

// Validation (`@Validate` is a decorator — see below):
import { z, Contract } from '@omnitron-dev/titan/validation';

// Typed errors:
import { Errors, ErrorCode, TitanError } from '@omnitron-dev/titan/errors';

// Decorators. The root re-exports only the nine DI ones; every
// other decorator — @Public, @Validate, @Auth, @Cache,
// @RateLimit, @Memoize, @Retry — is available ONLY from here:
import { Public, Validate, Memoize, Retry, Timeout }
  from '@omnitron-dev/titan/decorators';
```

Full subpath list (from `titan/package.json` exports):

| Subpath                                                | Contains                                                                                                                                                                                                                                                                                                                                                                                                                   |
| ------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `@omnitron-dev/titan`                                  | `Application`, `startApp` / `createApp` / `createModule` / `defineModule` / `createToken`, and exactly nine decorators re-exported from `/decorators`: `Service`, `Injectable`, `Inject`, `Optional`, `Singleton`, `Transient`, `Module`, `PostConstruct`, `PreDestroy`. **Not** a root import for everything — `@Public`, `@Validate`, `@Auth`, `@Cache`, `@RateLimit`, `@Memoize` and the rest are only in `/decorators` |
| `@omnitron-dev/titan/application`                      | `Application.create`, and the `OnInit` / `OnStart` / `OnStop` / `OnDestroy` interfaces                                                                                                                                                                                                                                                                                                                                     |
| `@omnitron-dev/titan/lifecycle`                        | The lifecycle *machinery*: `LifecycleController`, `LifecycleTimeoutError`, `bucketOf`, `LifecyclePhase`. The interfaces you implement are in `/application`                                                                                                                                                                                                                                                                |
| `@omnitron-dev/titan/decorators`                       | All core decorators (`@Service`, `@Public`, `@Validate`, `@Memoize`, etc.)                                                                                                                                                                                                                                                                                                                                                 |
| `@omnitron-dev/titan/netron`                           | **Server Netron API** — peers, services, descriptors                                                                                                                                                                                                                                                                                                                                                                       |
| `@omnitron-dev/titan/netron/transport/http`            | HTTP server transport                                                                                                                                                                                                                                                                                                                                                                                                      |
| `@omnitron-dev/titan/netron/transport/websocket`       | WebSocket server transport                                                                                                                                                                                                                                                                                                                                                                                                 |
| `@omnitron-dev/titan/netron/transport/tcp`             | TCP server transport                                                                                                                                                                                                                                                                                                                                                                                                       |
| `@omnitron-dev/titan/netron/transport/unix`            | Unix-socket server transport                                                                                                                                                                                                                                                                                                                                                                                               |
| `@omnitron-dev/titan/netron/transport/http/middleware` | HTTP-transport middleware (auth, rate-limit, …)                                                                                                                                                                                                                                                                                                                                                                            |
| `@omnitron-dev/titan/netron/auth`                      | `AuthenticationManager`, JWT verify, RLS context mapping                                                                                                                                                                                                                                                                                                                                                                   |
| `@omnitron-dev/titan/netron/multi-backend`             | Multi-backend server-side routing                                                                                                                                                                                                                                                                                                                                                                                          |
| `@omnitron-dev/titan/nexus`                            | DI container primitives                                                                                                                                                                                                                                                                                                                                                                                                    |
| `@omnitron-dev/titan/validation`                       | zod re-export (`z`), `Contract` / `contract` / `ContractBuilder`, `ValidationEngine`, `ValidationMiddleware`. `@Validate` is in `/decorators`                                                                                                                                                                                                                                                                              |
| `@omnitron-dev/titan/module/config`                    | `ConfigModule` (built-in)                                                                                                                                                                                                                                                                                                                                                                                                  |
| `@omnitron-dev/titan/module/logger`                    | `LoggerModule` (built-in)                                                                                                                                                                                                                                                                                                                                                                                                  |
| `@omnitron-dev/titan/module/rls`                       | RLS decorators                                                                                                                                                                                                                                                                                                                                                                                                             |
| `@omnitron-dev/titan/errors`                           | `Errors` factories + `TitanError`                                                                                                                                                                                                                                                                                                                                                                                          |
| `@omnitron-dev/titan/types`                            | Public type-only exports                                                                                                                                                                                                                                                                                                                                                                                                   |
| `@omnitron-dev/titan/utils`                            | Internal utilities                                                                                                                                                                                                                                                                                                                                                                                                         |
| `@omnitron-dev/titan/tracing`                          | OpenTelemetry-style tracing primitives                                                                                                                                                                                                                                                                                                                                                                                     |

Tree-shaking works on every subpath. Production bundles import
the smallest scope they need.

## Backend modules (optional, 14 packages)

Independently versioned. Add only the ones your app uses.

```bash
# Pick what you need (any subset):
pnpm add @omnitron-dev/titan-auth          # JWT auth
pnpm add @omnitron-dev/titan-cache         # multi-tier cache (L1 + L2 Redis)
pnpm add @omnitron-dev/titan-database      # Kysely + migrations + RLS
pnpm add @omnitron-dev/titan-discovery     # service discovery
pnpm add @omnitron-dev/titan-events        # event bus
pnpm add @omnitron-dev/titan-health        # health probes
pnpm add @omnitron-dev/titan-lock          # distributed locks
pnpm add @omnitron-dev/titan-metrics       # Prometheus metrics
pnpm add @omnitron-dev/titan-notifications # multi-channel delivery
pnpm add @omnitron-dev/titan-pm            # worker pools, autoscaling
pnpm add @omnitron-dev/titan-ratelimit     # rate limiting
pnpm add @omnitron-dev/titan-redis         # Redis client
pnpm add @omnitron-dev/titan-scheduler     # cron / interval / timeout
pnpm add @omnitron-dev/titan-telemetry-relay  # store-and-forward telemetry
```

→ Full module reference: [Titan modules](/docs/titan/modules).

## Browser client — `@omnitron-dev/netron-browser`

**Framework-agnostic RPC client for the browser.** Works with
any frontend — vanilla JS, React, Vue, Svelte, Solid, Angular,
Lit, plain JS in a Web Worker, Electron renderer, etc.

```bash
pnpm add @omnitron-dev/netron-browser
```

Use directly (no framework):

```typescript
import { createClient } from '@omnitron-dev/netron-browser';

const client = createClient({
  url:       'https://api.example.com',
  transport: 'http',                       // 'http' | 'websocket' | 'auto'
});

await client.connect();

// Type-safe service proxy (type from your shared types package):
import type { UsersService } from '@your-app/contracts';
const users = client.service<UsersService>('users');
const user  = await users.findById('u_42');
```

Carries:

- HTTP + WebSocket transports with auto-reconnect.
- Middleware pipeline (auth, retry, cache, circuit breaker, tracing).
- `AuthenticationClient` with token refresh, cross-tab sync.
- `BackendPool` for multi-backend routing.
- LRU cache with stale-while-revalidate.
- Typed errors that round-trip from the server intact.

Subpaths:

| Subpath                                   | Contains                                                                         |
| ----------------------------------------- | -------------------------------------------------------------------------------- |
| `@omnitron-dev/netron-browser`            | Root — `createClient`, plus `AuthenticationClient`, token storage and transports |
| `@omnitron-dev/netron-browser/client`     | `NetronClient`, `HttpClient`, `WebSocketClient`, `BackendPool`                   |
| `@omnitron-dev/netron-browser/middleware` | All built-in middleware                                                          |
| `@omnitron-dev/netron-browser/errors`     | Typed error hierarchy                                                            |

→ Full reference: [netron-browser](/docs/frontend/netron/browser).

### When you'd skip this

- Your client isn't a browser. Inside another Titan app (Node /
  Bun / Deno), use `@omnitron-dev/titan/netron` directly —
  server and Node-client APIs share the same shape.

## React bindings — `@omnitron-dev/netron-react` (only for React apps)

**React hooks, providers, and cache integration on top of
`netron-browser`.** Install **only** if your frontend uses
React.

```bash
pnpm add @omnitron-dev/netron-browser @omnitron-dev/netron-react
```

The `netron-react` package depends on `netron-browser` as a
peer — install both.

```tsx
import { NetronReactClient, NetronProvider, useService }
  from '@omnitron-dev/netron-react';

const client = new NetronReactClient({ url: 'https://api.example.com' });

function App() {
  return <NetronProvider client={client}>...</NetronProvider>;
}

function UserCard({ id }: { id: string }) {
  const users = useService<UsersService>('users');
  const { data, isLoading } = users.findById.useQuery([id]);
  return isLoading ? <Skeleton /> : <div>{data?.email}</div>;
}
```

Provides:

- `useQuery` / `useMutation` / `useSubscription` /
  `useInfiniteQuery` / `useQueries` / `useService`
- `<NetronProvider>` / `<MultiBackendProvider>`
- `<AuthProvider>` + `<AuthGuard>` + `<GuestGuard>` + `useAuth`
- `<NetronDevtools>` (React DevTools-style panel)
- `TestNetronProvider` for testing

→ Full reference: [netron-react](/docs/frontend/netron/react).

### For Vue / Svelte / Solid / Angular / Lit

Use `@omnitron-dev/netron-browser` directly. The client object
has identical methods regardless of framework — wrap them in
your framework's reactivity primitives (e.g., Vue's
`reactive`, Svelte stores, Solid signals).

Community `netron-vue` / `netron-svelte` packages could mirror
`netron-react`'s API exactly. Until they ship, the direct API
from `netron-browser` works in any framework.

## Design system — `@omnitron-dev/prism` (optional, React)

50+ MUI v9 components, schema-aware forms, theme. **React-only**
because MUI v9 is React.

```bash
pnpm add @omnitron-dev/prism
```

→ Full reference: [Prism](/docs/frontend/prism).

## Utilities (optional)

Five framework-agnostic packages used internally and reusable:

```bash
pnpm add @omnitron-dev/common         # type predicates, promise helpers, data structures
pnpm add @omnitron-dev/cuid           # collision-resistant URL-safe IDs
pnpm add @omnitron-dev/eventemitter   # async event emitter (parallel/serial/reduce)
pnpm add @omnitron-dev/msgpack        # MessagePack + native JS types (Date, Map, Set, …)
pnpm add @omnitron-dev/kb             # knowledge-base framework (server-side)
```

→ Full reference: [Utilities](/docs/utilities).

## Testing utilities (dev-only)

```bash
pnpm add -D @omnitron-dev/testing
```

Cross-runtime testing primitives — Node + Bun + Deno from one
source. → [Testing](/docs/testing).

## Supervisor (opt-in) — `@omnitron-dev/omnitron`

The application supervisor + CLI + web console + MCP server.
**Optional.** Install only when you outgrow a single Titan
app — see
[Titan vs Omnitron](/docs/foundations/titan-vs-omnitron).

Global install (system-wide `omnitron` command):

```bash
pnpm add -g @omnitron-dev/omnitron
```

Or per-project:

```bash
pnpm add -D @omnitron-dev/omnitron
pnpm omnitron up
```

→ Full reference: [Omnitron](/docs/omnitron/overview).

## Install matrices — common setups

### Backend-only service (RPC server, no frontend)

```bash
pnpm add @omnitron-dev/titan @omnitron-dev/titan-database @omnitron-dev/titan-auth
```

Server-side Netron with HTTP + WS + TCP + Unix transports is
already inside `titan` — no extra install.

### Backend + browser client (any frontend framework)

```bash
# Backend:
pnpm add @omnitron-dev/titan @omnitron-dev/titan-database @omnitron-dev/titan-auth

# Browser (works for Vue / Svelte / Solid / Angular / Lit / vanilla):
pnpm add @omnitron-dev/netron-browser
```

### Backend + React frontend

```bash
# Backend:
pnpm add @omnitron-dev/titan @omnitron-dev/titan-database @omnitron-dev/titan-auth

# Frontend (React-specific):
pnpm add @omnitron-dev/netron-browser @omnitron-dev/netron-react @omnitron-dev/prism
```

### Full operator-supervised platform

```bash
# Everything above PLUS:
pnpm add -g @omnitron-dev/omnitron
```

### Service-to-service (one Titan app calling another)

No client package needed. Use `@omnitron-dev/titan/netron`
directly in the caller — same API:

```typescript
import { Netron } from '@omnitron-dev/titan/netron';
import { createNullLogger } from '@omnitron-dev/titan/module/logger';

// A logger is required; `create` also starts it, which `connect` needs.
const netron = await Netron.create(createNullLogger());
const peer   = await netron.connect('tcp://other-service:4001');
const users  = await peer.queryInterface<UsersService>('users@1.0.0');
```

## Verify

```bash
pnpm list @omnitron-dev/titan --depth 0
node --input-type=module -e "import { Application } from '@omnitron-dev/titan'; console.log(Application.name)"
```

For Omnitron:

```bash
omnitron --version
omnitron ping
```

---

---
id: "getting-started/project-structure"
title: "Project Structure"
---

# Project Structure

A reference layout for a Titan-based project. The Omnitron monorepo itself
follows this shape; consumer projects do not need to be a monorepo to use
the same conventions.

## Single-app layout

**my-app/**

- `src/`
  - `modules/`
    - `users/`
      - `users.service.ts`
      - `users.module.ts`
      - `users.types.ts`
    - `orders/`
  - `app.module.ts` — Root module that imports the rest
  - `main.ts` — Application.create + start
- `tsconfig.json`
- `package.json`

## Monorepo layout

For multi-service projects, mirror the Omnitron monorepo:

**my-monorepo/**

- `apps/`
  - `api/` — Titan service exposed over HTTP/WS
  - `worker/` — Titan service exposed over TCP only
  - `web/` — React frontend (Prism + netron-react)
- `packages/`
  - `shared-types/` — Service interfaces shared by all apps
  - `shared-modules/` — Reusable Titan modules
- `pnpm-workspace.yaml`
- `turbo.json`

### `pnpm-workspace.yaml`

```yaml
packages:
  - apps/*
  - packages/*
```

### Sharing service types

Service interfaces should live in a shared package so both the server
implementation and the client `queryInterface<T>()` call see the same type:

**packages/shared-types/**

- `src/`
  - `users.contract.ts` — `interface UsersContract { … }`
  - `orders.contract.ts`
- `package.json`

The server class implements the contract; the client parameterises
`queryInterface<UsersContract>()` against it.

## Configuration

Titan's `ConfigModule` reads from any combination of files, env vars, and
schema-validated overrides. A typical project:

**my-app/**

- `config/`
  - `default.yaml` — Defaults that apply everywhere
  - `development.yaml` — Local-dev overrides
  - `production.yaml` — Prod overrides
- `src/`
  - `app.module.ts` — Root module and configuration registration

See [ConfigModule](/docs/titan/modules/config) for configuration sources, validation and precedence.

## Conventions

- **Module per bounded context.** Group services, repositories, and
  schemas by domain, not by layer.
- **Contracts in a shared package.** Keep `*.contract.ts` files free of
  runtime imports so the browser bundle does not pull in the server.
- **One `main.ts` per app.** The root file boots `Application.create`;
  it should be the only entrypoint that performs side effects at import
  time.

---

---
id: "getting-started/quickstart"
title: "Quickstart"
description: "Compile a Titan service, bind an HTTP transport, and call it from a typed client."
---

# Your first service

Build a small service that reports its status, expose it over HTTP, and call
it from a separate process. You need **Node.js 22+** and a package manager.
No database, Docker or supervisor is required.

## 1. Create the project

```bash
mkdir signal-service
cd signal-service
pnpm init
pnpm add @omnitron-dev/titan @omnitron-dev/netron-browser
pnpm add -D typescript @types/node
mkdir src
```

Set `"type": "module"` in `package.json`. Keep the installed versions in your
lockfile. Create this compiler configuration:

```json title="tsconfig.json"
{
  "compilerOptions": {
    "target": "ES2022",
    "module": "NodeNext",
    "moduleResolution": "NodeNext",
    "rootDir": "src",
    "outDir": "dist",
    "strict": true,
    "skipLibCheck": true,
    "experimentalDecorators": true,
    "emitDecoratorMetadata": true,
    "useDefineForClassFields": false
  },
  "include": ["src/**/*.ts"]
}
```

Compile with `tsc` so legacy decorators and their metadata are emitted.
Node's built-in TypeScript stripping does not compile those decorators.

## 2. Declare the service

```typescript title="src/signals.service.ts"
import { Service, Public } from '@omnitron-dev/titan/decorators';

@Service('signals@1.0.0')
export class SignalsService {
  @Public()
  async read(): Promise<{ status: string }> {
    return { status: 'online' };
  }
}
```

`@Service` gives the service its wire identity. `@Public()` exposes the
method to RPC callers. This example has no authentication and accepts no
arguments; it is intended to run on your local machine.

## 3. Register the module and transport

```typescript title="src/main.ts"
import { Application, Module } from '@omnitron-dev/titan';
import { HttpTransport } from '@omnitron-dev/titan/netron/transport/http';
import { SignalsService } from './signals.service.js';

@Module({ providers: [SignalsService] })
class AppModule {}

const app = await Application.create(AppModule);
const netron = app.netron;
if (!netron) throw new Error('Netron is unavailable');

netron.registerTransport('http', () => new HttpTransport());
netron.registerTransportServer('http', {
  name: 'api',
  options: { host: '127.0.0.1', port: 3000 },
});

await app.start();
console.log('Signals service ready at http://127.0.0.1:3000');
```

Register transports **before** `app.start()`. Registration stores the server
configuration; startup opens the listener and exposes service instances.
The application installs graceful shutdown handlers by default.

Compile and run:

```bash
pnpm exec tsc
node dist/main.js
```

Keep this terminal running. The readiness message appears after startup
completes. Stop the process with Ctrl+C when finished.

## 4. Make a typed call

```typescript title="src/client.ts"
import { createClient } from '@omnitron-dev/netron-browser';
import type { SignalsService } from './signals.service.js';

const client = createClient({ url: 'http://127.0.0.1:3000' });
try {
  await client.connect();
  const signals = client.service<SignalsService>('signals@1.0.0');
  console.log(await signals.read());
} finally {
  await client.disconnect();
}
```

In a second terminal, from the same project directory:

```bash
pnpm exec tsc
node dist/client.js
```

Expected output:

```text
{ status: 'online' }
```

The client imports only the service **type**; the backend implementation is
not included in the client at runtime.

## If it fails

| Symptom                     | Check                                                               |
| --------------------------- | ------------------------------------------------------------------- |
| `EADDRINUSE`                | Another process owns port 3000. Choose a free port in both files.   |
| Connection refused          | Start the server first and confirm its readiness message.           |
| Service not found           | Match `signals@1.0.0` exactly and register the class as a provider. |
| Decorator compilation error | Use the configuration above and compile with `tsc`.                 |
| `Netron is unavailable`     | Keep Titan's core modules enabled for this example.                 |

## Extend the service

Add [validation](/docs/titan/validation/overview) before accepting untrusted
inputs and [authentication](/docs/titan/netron/authentication) before exposing
protected methods. Types alone enforce neither boundary.

Continue with the [application lifecycle](/docs/titan/application),
[transport reference](/docs/titan/netron/transports), or
[React bindings](/docs/frontend/netron/react).

---

---
id: "intro"
title: "Introduction"
description: "Choose the Omnitron layers you need, understand their boundaries, and build your first service."
---

# Build the system you control

Omnitron is a TypeScript stack for building and operating services. **Titan**
provides the application runtime. **Netron** connects services and clients.
**Prism** supplies React interface components. The **Omnitron supervisor**
manages applications and infrastructure.

Start with a Titan application. Add the client, interface and supervision
layers when your system needs them.

## Choose a starting point

| Your goal                     | Start here                                                    | What you will produce                                     |
| ----------------------------- | ------------------------------------------------------------- | --------------------------------------------------------- |
| Run your first service        | [Quickstart](/docs/getting-started/quickstart)                | A compiled application and a typed HTTP client            |
| Understand the boundaries     | [Titan vs Omnitron](/docs/foundations/titan-vs-omnitron)      | A decision about runtime and supervision                  |
| Build a complete application  | [Tutorial](/docs/tutorial)                                    | A backend, authentication, interface and tests            |
| Connect an existing interface | [Browser client](/docs/frontend/netron/browser)               | HTTP calls or WebSocket connections                       |
| Prepare a deployment          | [Production readiness](/docs/deployment/production-readiness) | A release and recovery plan                               |
| Find an API                   | [API index](/docs/reference/api-index)                        | A symbol, supported import path, guide and implementation |

Use the [developer map](/docs/getting-started/developer-map) for task-based
routes through the manual, including a verification step for each route.

## The stack

| Layer          | Package                        | Boundary                                                                             |
| -------------- | ------------------------------ | ------------------------------------------------------------------------------------ |
| Runtime        | `@omnitron-dev/titan`          | Dependency injection, modules, lifecycle, validation, logging and server-side Netron |
| Browser client | `@omnitron-dev/netron-browser` | HTTP and WebSocket RPC; independent of the UI framework                              |
| React bindings | `@omnitron-dev/netron-react`   | Providers and typed query, mutation and subscription hooks                           |
| Interface      | `@omnitron-dev/prism`          | React components, layouts, themes and forms                                          |
| Operations     | `@omnitron-dev/omnitron`       | Daemon, process supervision, infrastructure, CLI, web console and MCP                |

Server-side Netron ships inside Titan. A Titan process can run directly under
Node.js; installing the Omnitron supervisor is optional. Prism and the React
bindings are also optional. See the [architecture](/docs/foundations/architecture)
for the runtime relationships.

## Contracts across the wire

A client can import a service's TypeScript type and call its methods through a
Netron proxy. Sharing that type catches incompatible calls **when both sides
compile against the same contract**. It does not synchronise independently
deployed versions or validate an arbitrary request at runtime.

Use [validation](/docs/titan/validation/overview) for untrusted inputs,
[authentication](/docs/titan/netron/authentication) for access control and
[service versioning](/docs/titan/netron/services) when contracts evolve.
The [quickstart](/docs/getting-started/quickstart) shows the complete registration
and transport setup.

## Add capabilities deliberately

Titan modules are separate packages. Their dependency and lifecycle contracts
are documented alongside their configuration.

| Capability                                    | Reference                                                                                                   |
| --------------------------------------------- | ----------------------------------------------------------------------------------------------------------- |
| Persistence, migrations, transactions and RLS | [Database](/docs/titan/modules/database)                                                                    |
| JWT verification and token transport          | [Authentication](/docs/titan/modules/auth) · [Token issuance](/docs/auth/token-issuance)                    |
| Caching and distributed coordination          | [Cache](/docs/titan/modules/cache) · [Redis](/docs/titan/modules/redis) · [Locks](/docs/titan/modules/lock) |
| Background work                               | [Scheduler](/docs/titan/modules/scheduler) · [Process management](/docs/titan/modules/pm)                   |
| Operational signals                           | [Health](/docs/titan/modules/health) · [Metrics](/docs/titan/modules/metrics)                               |
| A complete module inventory                   | [Module map](/docs/titan/modules/module-map)                                                                |

## Read implementation status before adopting a design

The module and API references describe shipped code. Pages labelled
**Design RFC** describe architectural proposals and application patterns that
may need your own implementation. In particular, the
[authorisation design](/docs/auth) goes beyond the daemon's
[operator roles](/docs/omnitron/auth-rbac).

The project is pre-1.0. Pin package versions, read the
[release notes](/docs/community/changelog), and test server/client combinations
before upgrading. See [compatibility](/docs/reference/compatibility) for runtime
requirements and support boundaries.

## Work from source

The framework, packages and supervisor live in the
[Omnitron repository](https://github.com/omnitron-dev/omni). This manual has its
own [documentation repository](https://github.com/omnitron-dev/website).
Both use the MIT license. See [Contributing](/docs/community/contributing)
for reproducible setup and verification commands.

---

---
id: "lumen/authoring"
title: "Authoring with Lumen"
---

# Authoring with Lumen

Keep the document useful as static HTML. Add an island where interaction helps the reader inspect a result.

## Stable routes and headings

```markdown
---
title: Connection lifecycle
slug: /manual/connections
---
# Connection lifecycle

## Shutdown {#shutdown}

Close the client after the last request.
```

A leading slash makes `slug` an absolute site route. A relative slug is resolved from the document directory. An `index.md` uses its parent route. `id` names the document in the corpus and sidebar; it is separate from the public route.

Use explicit heading IDs when other pages or external readers depend on a link. Repeated automatic headings receive unique numeric suffixes. Duplicate explicit IDs are errors.

## One source, multiple readers

> **Keep the contract visible**
>
> State ownership, prerequisites, failure behavior and cleanup next to the example. Search and agent exports see the same public document tree.

```markdown
:::note{title="Keep the contract visible"}
State ownership, prerequisites, failure behavior and cleanup next to the example. Search and agent exports see the same public document tree.
:::
```

```mermaid
flowchart LR
  Source[Document source] --> AST[Typed AST]
  AST --> HTML[Static HTML]
  AST --> Search[Local search]
  AST --> Markdown[Markdown mirror]
```

Open **Diagram source** below the drawing to see its Mermaid input. It is the same input the browser renders.

Mermaid diagrams use the real browser engine, including its ELK layouts. Static
HTML retains readable diagram source. The client registers `MermaidDiagram` and
loads the engine when the island becomes visible; invalid diagrams show an error
and retain their source. The shared helper is available from
`@omnitron-dev/lumen-ast/diagrams`. It requires a browser DOM for SVG rendering;
PDF and Markdown exports currently retain diagram source. Put labels with punctuation in quotes, for example `A["Infrastructure (Docker / bare-metal)"]`. Mermaid escape codes use `#quot;` or `#40;`, without an HTML `&` prefix. On narrow screens, scroll the diagram horizontally or expand its source.

## File trees

Use the built-in file tree block for directory layouts. Keep filenames in inline
code and descriptions in ordinary Markdown. From Lumen 0.4.8, folders with
children use native keyboard-accessible disclosure controls and start expanded.

**my-service/**

- `src/` — application source
  - `main.ts` — entry point
  - `modules/`
    - `users.module.ts` — feature module
- `package.json` — dependencies and commands

Copy this Markdown to reproduce the tree:

```markdown
:::filetree{title="my-service/"}
- `src/` — application source
  - `main.ts` — entry point
  - `modules/`
    - `users.module.ts` — feature module
- `package.json` — dependencies and commands
:::
```

The block renders as static HTML. Nested lists, filenames and annotations remain
readable in search, Markdown and PDF exports; files are not pretend links.
This is a Lumen build directive. Prism ResearchEditor preserves it as opaque
source; the standalone ResearchDocument reader reports an unknown directive
rather than rendering a file tree.

## Selective React islands

A PDX widget declares literal props. The site's client entry registers its React component with `LumenShell`; the shared portal runtime activates it at the declared trigger.

```jsx
<ApiExplorer hydrate="load" />
```

This site's [API explorer](/docs/reference/api-index) is a working island backed by a generated public export inventory. The rest of the document remains server-rendered and readable without JavaScript.

Supported activation triggers are `load`, `idle`, `visible` and `interaction`.
Props cross a strict JSON boundary shared by SSR and the browser through
`@omnitron-dev/lumen-ast/islands`. Unsupported values, cycles and oversized props
fail explicitly; they are not silently converted to different values. Arbitrary document expressions are not application code. Import and register application components in the trusted client entry.

## Component source

Use `ComponentExample` from `@omnitron-dev/lumen-ui/blocks` to keep a trusted React preview beside the exact module that implements it:

```tsx
import { ComponentExample } from '@omnitron-dev/lumen-ui/blocks';
import Counter from './counter.tsx';
import source from './counter.tsx?raw';

<ComponentExample title="Request counter" code={source} resettable>
  <Counter />
</ComponentExample>
```

> **Version requirements**
>
> `ComponentExample`, its native Source disclosure and Reset control are available from Lumen 0.4.8. Preview error isolation, progressive syntax highlighting and the `@omnitron-dev/lumen-ast/highlight` entry require Lumen 0.4.9 or later.

Source uses a native disclosure and remains readable before hydration. Reset remounts the preview while keeping the source open; a client render error stays in the preview so Source and Reset remain available. The copy control preserves the original text, comments and newlines. The [Prism catalog](/docs/frontend/prism/components) uses this same frame.

`CodeBlock` adds light and dark syntax colors when its source becomes visible. It uses the browser-safe `@omnitron-dev/lumen-ast/highlight` entry, which exports `highlightCode` and literal token types and loads the requested language grammar on demand. The TypeScript compiler and Twoslash stay outside the client bundle. Unknown languages remain plain text; a failed grammar download shows a status beside the complete source.

## Inspect inferred types

A `typescript twoslash` fence requests analysis from the actual TypeScript language service. The caret points to the symbol on the preceding line:

```typescript twoslash
const doubled = [1, 2, 3].map(value => value * 2);
//    ^?
```

The annotation above is generated during this site's build. It is not a handwritten type label. Analysis uses the compiler's packaged standard libraries in a standalone virtual file; it does not execute the code or import this website's workspace dependencies.

## Mathematical documents

Inline mathematics such as $E = mc^2$ and display equations use KaTeX:

$$
\sum_{i=1}^{n} i = \frac{n(n+1)}{2}
$$

This site's client entry imports `@omnitron-dev/prism/components/research/styles.css`, so Vite publishes the shared math stylesheet and its fonts with the other local assets. No font CDN is required.

For a paper, set `pdf.paperSections` to the document IDs or source paths you want, then run:

```bash
lumen pdf --outDir papers
```

The command writes one PDF per selected public document using the offline Typst compiler. Unsupported mathematical commands fail explicitly. HTML `katexMacros` do not define PDF macros; diagram blocks retain their source in PDF output.

## Run an isolated example

This JavaScript snippet runs inside a dedicated browser worker with a capability-free
QuickJS runtime. It has no network, DOM, filesystem or host credentials. Editing the
snippet changes only this local example; it does not call an Omnitron service.

The editor above starts with the literal `code` value from this PDX widget:

```jsx
<Playground hydrate="visible" code="return [1, 2, 3].map(value => value * 2)" />
```

Run the initial expression to produce `[2, 4, 6]`. Change the multiplier and run it again. Reset the expression by reloading the page. A timeout or invalid expression appears in the result area; a subsequent run starts a fresh execution.

## Reading wide content

Code, tables, diagrams and the API symbol list keep their own native scrolling.
Internal scrollbars stay thin and reveal on hover or keyboard focus without
changing the layout. Touch and high-contrast modes retain native visibility.
Use Tab to reach a code block, table or diagram, then arrow keys to scroll; file
trees use their existing disclosure controls. The static HTML remains usable
without client JavaScript.

## Verify the result

Run `lumen check` while authoring, then build and inspect the actual output. A parser accepting a widget is not evidence that its registered component loaded; exercise it in the production preview. Check keyboard focus, narrow viewports and both color themes.

---

---
id: "lumen/overview"
title: "Lumen"
---

# Lumen

Lumen compiles a document corpus into static HTML, local search, Markdown mirrors and an agent index. The same parsed document graph supplies page rendering and link checks. This manual is a reference consumer of that build path.

## Choose the boundary

| Package                       | Responsibility                                                                                       |
| ----------------------------- | ---------------------------------------------------------------------------------------------------- |
| `@omnitron-dev/lumen-ast`     | Markdown/PDX parsing, typed nodes, HTML and Markdown transformations, code highlighting and diagrams |
| `@omnitron-dev/lumen`         | Corpus graph, verification gates, static generation and CLI                                          |
| `@omnitron-dev/lumen-ui`      | Documentation layouts, search UI and selective React island hydration                                |
| `@omnitron-dev/titan-lumen`   | Titan-hosted document services, access policy, live editing and MCP integration                      |
| `@omnitron-dev/lumen-migrate` | Docusaurus content conversion and route/anchor comparison                                            |

A static build needs no Titan server. Add Titan-Lumen only when the application needs authenticated document services. A public static artifact is readable by anyone who receives its files; privileged builds require a corresponding hosting boundary.

## Build a document site

Start with one document and one configuration file. The source directory determines the document routes. With the default route rules, `docs/intro.md` becomes `/docs/intro`.

**Engineering manual**

- `docs/` — public document sources
  - `intro.md` — first page
- `lumen.config.ts` — build configuration

Set the input and output directories explicitly:

```typescript
import { defineLumenConfig } from '@omnitron-dev/lumen';

export default defineLumenConfig({
  title: 'Engineering manual',
  srcDir: 'docs',
  outDir: 'build',
  onBrokenLinks: 'throw',
  onBrokenAnchors: 'throw',
});
```

```bash
lumen check
lumen build
lumen serve --host 127.0.0.1 --port 4173
```

`build` rejects invalid routes and broken internal links before publishing its output. `serve` previews the existing build; it does not start a backend or deploy the site. `dev` rebuilds watched content and reloads the local preview. Restart it after changing the build configuration.

## Read and reuse the corpus

Every public document has a Markdown mirror. [This page as Markdown](/docs/lumen/overview.md), the [agent index](https://omnitron.dev/llms.txt) and [complete public corpus](https://omnitron.dev/llms-full.txt) are produced from the same public projection as the site.

Open the Markdown mirror beside the HTML page and compare the headings. The agent index links to individual documents. The complete corpus combines them for clients that need one download.

The default static audience is anonymous. Restricted documents and sections are projected out before HTML, search or agent exports are produced. Do not put secrets in public content or client props.

## Local MCP access

```bash
lumen mcp --stdio --root .
```

The official MCP SDK serves `search_corpus`, `get_document` and
`lumen_inspect_ast`. Resource paths are corpus-relative, for example
`lumen://corpus/docs/intro.md`. The local endpoint exposes the anonymous
projection only: restricted documents and sections are absent from search,
resources and AST output. It rejects missing or invalid corpora at startup.

The spawning process owns this local trust boundary. Caller-supplied roles do
not grant access. Use Titan-Lumen's adapter for authenticated live calls through
Netron. Both adapters share the isolated Node-only `lumen-ast/mcp` contract;
the AST root and browser entry do not import the server SDK. Diagnostics go to
stderr, and the transport closes an input exceeding its 1 MiB buffer limit even
before a newline arrives.

## Continue

- [Author documents and interactive examples](/docs/lumen/authoring)
- [Inspect this reference site's build](/docs/lumen/this-site)
- [Find exact package exports](/docs/reference/api-index)

---

---
id: "lumen/this-site"
title: "How this manual is built"
---

# How this manual is built

This repository consumes Lumen's ordinary CLI. Its layout uses Lumen navigation components and the shared Prism identity used by Omnitron Console.

## The build path

1. `lumen.config.ts` declares document and page directories, the SSR layout and the client entry.
2. Lumen parses the corpus, resolves public routes and checks internal links and anchors.
3. The public audience projection supplies HTML, search and Markdown exports.
4. Vite bundles the declared client entry. `LumenShell` activates React islands through the shared portal runtime.
5. The completed generation replaces the previous output after its full inventory is written. A failed content check retains the previous generation.

There is no application RPC required to read or search this static manual.

The trusted configuration, layout and renderer are compiled into one Vite SSR graph. Local components and imported React packages share the project's React peers, so state and context hooks also work during static rendering. The compiler preserves source module paths and ESM resolution and releases its temporary files after success or failure.

Browser resources enter through `clientEntry`, `stylesheets` or `publicDir`. Importing a separate CSS, image or font file from the SSR layout fails with a diagnostic instead of leaving a missing resource in the published site. Self-contained inline assets need no separate file.

Publication persists a small recovery journal before moving the previous output.
After interruption, the next build or serve verifies ownership and file hashes
before retaining the complete new generation or restoring the complete previous
one. `lumen serve` holds a verified snapshot until shutdown; restart it to preview
a later build. Do not point another development server at the mutable output
while rebuilding. This is recoverable directory replacement on a local macOS or
Linux filesystem, with a brief rename gap; Windows publication is not supported.

## What is interactive

- The header opens Lumen's local search dialog.
- The API index filters the generated public symbols by package, import path and kind.
- The authoring guide runs editable JavaScript inside a bounded QuickJS worker and renders a Mermaid diagram with the actual browser engine.
- Type annotations and mathematical typesetting are generated during the build and remain readable without JavaScript.
- The theme menu offers Light, Dark and System. It remembers the selected mode;
  System follows changes to the operating-system preference. Shared color tokens
  apply before the client mounts, and PrismProvider owns live theme updates.
- The header mark unfolds on hover or keyboard focus and returns afterward.
  It remains still when reduced motion is requested.
- The API inventory is fetched only when its island is present on the page.
- Navigation and page contents remain ordinary links and static HTML. Theme and
  search controls enable after their client handlers are ready; the document and
  navigation remain usable without JavaScript.

The API inventory is generated from public package manifests and TypeScript re-exports. It checks that each entry point has a guide; it does not prove that every behavior has a complete example.

## A page for people and tools

The interactive reading-format tabs and direct Markdown mirror links in this section require Lumen 0.4.8 or later.

Choose a format for the authoring guide. All three links are part of the static document. Use the arrow keys to change tabs, or Home and End to jump to either end.

### HTML

Use the [authoring guide](/docs/lumen/authoring) to try the diagram and playground.

### Markdown

Read the [authoring guide as Markdown](/docs/lumen/authoring.md) to reuse its text and code.

### Agent index

Find public documents in the [agent index](https://omnitron.dev/llms.txt).

The Markdown below produces this group. Add a shared `syncKey` to several groups on one page when the same choice should follow the reader. Give matching choices the same `value`.

```markdown
:::tabs{label="Public reading formats"}
:::tab{label="HTML" value="html"}
Use the [authoring guide](./authoring.md) to try the diagram and playground.
:::
:::tab{label="Markdown" value="markdown"}
Read the [authoring guide as Markdown](/docs/lumen/authoring.md) to reuse its text and code.
:::
:::tab{label="Agent index" value="agents"}
Find public documents in the [agent index](https://omnitron.dev/llms.txt).
:::
:::
```

## Preserving public links

The migration records the previous built site's route and anchor inventory in `migration/url-contract.json`. Validation compares the new output against that contract and checks internal links in the emitted HTML, including fragment targets. Content changes must retain externally useful anchors or provide an explicit reviewed redirect.

## Source and distribution

The website and framework are separate repositories. Use the normal Omnitron SDK workflow for a development checkout and released package versions for distribution. An unpublished SDK candidate must be qualified explicitly; a local dependency link is not evidence of a reproducible registry installation.

See [contributing](/docs/community/contributing) for the project workflow and [compatibility](/docs/reference/compatibility) for supported versions.

---

---
id: "omnitron/access"
title: "Console access"
description: "Human roles, effective RPC permissions, account administration and revocation shared by Console and CLI."
---

# Console access

The **Access** page (`/access`) shows the permissions of the current account. Administrators can create accounts, change roles and revoke sessions. `omnitron access` calls the same `OmnitronAccess` service. The service runs on a master with PostgreSQL and an audit service.

## One policy across interfaces

The daemon publishes methods with Titan `@Public({ auth: … })` metadata. `OmnitronAccess.describe()` reads that metadata from the services actually registered in Netron and evaluates it with Titan's access validator and authorization manager. The browser receives a catalog of `service.method` capabilities; it does not maintain a second role-to-action matrix.

| Role       | Intended use                 | Examples                                                                                            |
| ---------- | ---------------------------- | --------------------------------------------------------------------------------------------------- |
| `viewer`   | Observe infrastructure       | Status, logs, metrics, health and deployment history                                                |
| `operator` | Operate applications         | Viewer reads plus lifecycle, deployment and permitted node operations                               |
| `admin`    | Administer the control plane | Operator actions plus accounts, configuration, secrets, SSH terminals and arbitrary-code automation |

Trace reads (`getTrace`, `queryTraces`, `getServiceMap`) allow viewers.
Trace ingestion (`ingestSpan`, `ingestBatch`) writes monitoring data and requires
`operator` or `admin`; viewer access does not include submitting telemetry.
These RPC methods do not implicitly grant access to machine identities.

These are the existing daemon roles, defined through `@omnitron-dev/access/roles`. Exact grants come from each RPC method, not this summary. `service_role` is a separate machine identity and cannot be assigned to a human account.

Prism's narrow `@omnitron-dev/prism/auth` entry supplies guards without importing authentication forms or layouts. Console gates navigation, page entry, mutation controls and dialogs using the catalog. It refreshes while visible every 15 seconds; a failed refresh clears effective grants until verification succeeds. A role change is enforced by the server independently of this refresh interval.

A catalog grant describes a method-level rule. Resource ownership, live sessions and other checks still apply during invocation. Dynamic policies that need call arguments are marked **Conditional**, and the UI does not grant them speculatively. A hidden control never replaces server authorization.

## Manage accounts

In **Access → Console accounts**, create a named account with the least powerful suitable role. Creation defaults to viewer. Usernames allow 1–128 letters, digits and `. _ @ + -`, beginning with a letter or digit. The initial password and later password changes use the same validation: at least 15 Unicode characters and at most 256 UTF-8 bytes. Secrets are never included in account listings or audit details.

```bash
omnitron access show
omnitron access users
omnitron access create observer --role viewer --password-stdin
omnitron access create deployer --role operator --display-name 'Deployment operator' --password-stdin
```

On a terminal, `--password-stdin` opens a hidden prompt. A pipe is read until EOF. Do not put a password in shell arguments or checked-in scripts. Account creation in the browser requires HTTPS, with a loopback exception for local development.

Account lists contain at most 100 entries and a continuation cursor. Use `omnitron access users --after <id>` for the next page. Global `--json` is supported.

### Concurrent sign-ins

Failed password checks for the same account accumulate in PostgreSQL even when several requests arrive together. Once locked, the account rejects new sign-ins until the recorded deadline. A successful sign-in rechecks the current password record, lockout and role, then creates the session and resets the counter in a single transaction. A database failure rolls back both writes. The same policy applies when a Console request reaches the service through HTTP or WebSocket.

## Change roles and revoke sessions

Review the current role first, then submit it as the expected previous value:

```bash
omnitron access role <account-id> operator --expect viewer
omnitron access revoke <account-id>
```

Role changes are serialized in PostgreSQL and revalidate the actor inside the transaction. A stale expected role fails rather than overwriting another administrator's change. The last administrator cannot be demoted. A human administrator cannot change their own role; use another administrator. The owner-only local CLI retains its OS-authorized recovery path.

Changing a role deletes that account's sessions and records `account.role.change` in the same transaction. Creating an account records `account.create`; explicit revocation records `account.sessions.revoke`. If the audit insert fails, the transaction rolls back. Read the events in **Audit** or with `omnitron audit --resource account`.

The audit trail distinguishes a human or local CLI caller (`user`), a control-plane peer (`service`) and autonomous daemon work (`system`). A service keeps its authenticated subject ID; having an ID does not turn it into a human. Console and CLI show the service kind beside that ID. Existing audit rows retain their original evidence and are not relabelled retrospectively.

After commit, the daemon invalidates cached tokens and removes authority from admitted persistent peers. A validation already in flight cannot restore a cached grant after revocation. SSH checks the live session on every operation and during its two-second lifecycle sweep. The account signs in again to obtain its new role.

Every authenticated HTTP, WebSocket and TCP invocation also checks its admitted identity. Human JWTs require a live master session (`sid`); a signed role without a session is insufficient. A role change refuses an existing peer rather than changing its permissions after the method gate. An expired JWT must be renewed even if its socket stays connected. A session-store outage returns `503 SESSION_STORE_UNAVAILABLE`, not a false revocation.

Event subscriptions recheck authority before each delivery. Each peer has one in-flight send, at most 256 queued events and 1 MiB of queued payload; an individual event is limited to 256 KiB. An authorization failure, delivery timeout or overflow closes the stream so Console resumes polling. This is a live notification stream, not a replayable event journal; reconnecting refreshes authoritative state.

Revocation ends sessions; it does not disable an account or change its password. The owner can sign in again. Changing one's password preserves the caller's session and revokes the others. Sign-out also invalidates cached authority for the affected session.

## Scope and extension points

Console roles govern infrastructure administration. Application users, organization memberships and tenant-specific policies belong to the application's access layer; see [Authentication and authorization](/docs/auth). Selecting a project in Console filters the workspace and is **not** a tenant security boundary.

This release manages the three fixed daemon roles. It does not add custom role editing, per-project grants, invitations, account suspension, SSO, passkeys or MFA enrollment. The database's second-factor fields alone do not enable those flows. Add such features through the shared access policy and server authorization before exposing controls in Console.

## Activation

Update the master backend and Console as one release. Before switching the
served frontend, verify `omnitron access show` against the restarted master.
An older daemon without `OmnitronAccess` cannot supply the permission catalog;
the new UI then denies access instead of inventing grants. See the
[Console build workflow](/docs/omnitron/console#production-build) for staging validation
assets without replacing a running frontend.

## Implementation and validation

The implementation is in `apps/omnitron/src/services/access.rpc-service.ts`, `shared/roles.ts`, `shared/dto/access.ts` and `webapp/src/auth/access.tsx`. New public methods appear automatically in the catalog; new UI actions must use their typed RPC permission.

Integration tests exercise real PostgreSQL transactions and HTTP authorization: role boundaries, stale edits, last-admin protection, audit failure rollback and cached-token revocation. A separate HTTP regression test verifies that viewers can read traces but cannot
ingest spans or batches, and that rejected calls never reach the collector.
Browser tests cover pending grants, failed refresh, account changes and viewer isolation. See [SSH workspace](/docs/omnitron/terminal) for terminal-specific ownership and limits.

## Automation and future agents

Xec pipeline methods require `admin`, including definition and run-history reads. A run revalidates the human session and requires a durable audit intent and the reviewed definition revision. An operator's application lifecycle rights do not grant arbitrary shell execution. See [Xec automation](/docs/omnitron/automation).

Release builds follow the same boundary: operators build registered project code
using its declared settings. An explicit SDK source or nonempty environment
override requires a revalidated administrator and a durable audit intent before
execution. Both can change which code tools execute. Audit records include only
environment variable names, not their values; revocation during the audit write
prevents the build from starting.

The local MCP server uses the daemon owner's administrative Unix socket. Tool selection in an agent client does not reduce that underlying authority. It is not a delegated identity. Embedded agents require a broker, isolated workers and resource-scoped grants evaluated by this same access layer; the [agent design](/docs/omnitron/agents) describes the boundary and acceptance tests. That delegated path is not enabled yet.

---

---
id: "omnitron/agents"
title: "Agents and delegated operations"
description: "Architecture and acceptance criteria for provider adapters, delegated RBAC, monitoring evidence and controlled automation."
---

# Agents and delegated operations

This document specifies the next agent integration. **Embedded provider sessions, delegated agent identities, custom role editing, isolated agent workers and approval grants are not implemented in this release.** Existing MCP tools and administrator Xec automation are useful building blocks, but do not provide this security boundary.

Start with the [current access model](/docs/omnitron/access), [Xec automation](/docs/omnitron/automation) and [monitoring interfaces](/docs/omnitron/observability). Keep provider integration optional: installing Omnitron must not require an agent SDK, provider account or inference service.

## Boundaries and ownership

| Layer                  | Responsibility                                                                                                       |
| ---------------------- | -------------------------------------------------------------------------------------------------------------------- |
| `@omnitron-dev/access` | Reusable roles, permissions and policy evaluation                                                                    |
| Titan                  | Principal propagation, transport authorization, lifecycle, metrics, diagnostics and notification delivery primitives |
| Omnitron               | Infrastructure resource identity, delegation, approvals, audit, query budgets and operational workflows              |
| Xec                    | Local/remote execution, scripts, graph scheduling, streaming and cancellation                                        |
| Provider adapter       | Provider-specific authentication, sessions, model events, interruptions and usage                                    |
| Application            | Its own domain policy and business operations                                                                        |

Do not add another role hierarchy to an agent plugin. Human, service and agent principals share the policy engine while retaining distinct identity types. A provider login authenticates the model session; it does not grant infrastructure rights.

## Codex provider

OpenAI provides both a [TypeScript Codex SDK](https://learn.chatgpt.com/docs/codex-sdk) and [Codex app-server](https://learn.chatgpt.com/docs/app-server). The SDK fits programmatic thread execution. App-server is the intended adapter for an interactive Console integration requiring streaming sessions, interruptions and approval events. Generate and pin its protocol types with a tested provider version; do not couple core Omnitron DTOs to experimental protocol fields.

Use standard MCP tools through the broker. Treat experimental dynamic tools as an optional adapter capability. Provider approval settings are an additional restriction; they cannot override Omnitron denial.

Prefer an eligible ChatGPT subscription connection when the deployment supports it. The [Sign in with ChatGPT guidance for open-source integrations](https://developers.openai.com/siwc/token-sharing-open-source) and [self-hosted VM guidance](https://developers.openai.com/siwc/self-hosted-vms) define the supported ownership and token-refresh paths. App-server's managed ChatGPT login is not a general authentication solution for a commercial hosted product. Evaluate the applicable partner path before offering subscription login in such a deployment.

Offer API credentials as a distinct, explicit connection. Never silently fall back to billable API usage when subscription access fails or reaches a limit. Show connection identity, provider availability, usage and the applicable billing mode without exposing tokens. Only show quota/usage figures the provider actually returns. Follow the [preview limitations](https://developers.openai.com/siwc/preview-limitations); do not infer unrestricted API compatibility.

The adapter owns token refresh and reconnect/resume behavior. Store credentials in the server vault, scope them to a connection and owner, and keep them out of browser storage, URLs, process arguments, scripts, telemetry and exported support bundles. Disconnect must revoke or delete local credentials as supported by the provider and terminate affected sessions. Provider-side revocation limitations must remain visible.

## Delegation in the existing role system

An agent principal has an immutable agent ID, owner ID, delegation ID and expiry. A run adds its own run ID and grant. It does not inherit the administrator's Unix-socket identity, JWT or `service_role`.

Effective authority is the intersection of:

1. The owner's **current** rights and account/session state.
2. Role bindings evaluated for the agent principal.
3. Explicit delegation scope and expiry.
4. The run's narrower scope and remaining budgets.
5. Organization policy and any required approval.

An explicit deny wins. A missing policy, unavailable authority store, unrecognized resource, expired grant or stale policy version denies a mutation. Approval satisfies an approval requirement; it never creates a permission absent from that intersection.

Resource constraints use server-resolved project, stack, node and application IDs, not a UI filter or a caller-supplied path prefix. A compound request is rejected if any requested resource is outside scope; silently returning a subset can mislead analysis. Recheck authority at admission and each privileged broker call. Revoke active streams and queued operations when the owner, binding or delegation changes. A running external operation needs a documented cancellation or containment policy; revocation does not undo completed effects.

### Permission families

These are proposed semantic permissions, to be mapped centrally to RPC methods rather than implemented as a provider-specific allowlist:

| Family         | Separately grantable operations                                                                       |
| -------------- | ----------------------------------------------------------------------------------------------------- |
| Monitoring     | Read metrics, query logs, read traces, read health, read/subscribe to alerts                          |
| Alerts         | Acknowledge, silence with expiry, resolve where supported, edit rules, edit delivery routes           |
| Automation     | Read definitions, draft scripts, submit review, approve, execute approved revision, cancel owned runs |
| Deployment     | Inspect, prepare, activate a reviewed release, rollback a named release                               |
| Infrastructure | Inspect resources, change a bounded resource, open a terminal                                         |
| Access         | Inspect own grants, administer delegation, administer roles                                           |
| Secrets        | List permitted references, resolve for a specific broker operation, rotate                            |

Reading logs does not permit reading vault values. Drafting a script does not permit executing it. Receiving an alert does not authorize remediation. Do not bundle unrestricted SSH, arbitrary shell or secret resolution into a monitoring role. Treat shell access as the full OS authority of its target identity, not as a narrowly constrained action merely because the launch method has a role check.

## Worker and broker isolation

Run the model/provider adapter in a dedicated worker with a restricted filesystem, bounded CPU/memory/runtime, controlled egress and a short-lived broker credential. Do not mount the daemon's Unix socket, Docker socket, host SSH directory, vault, production database credentials or the user's full home directory. A normal child process is not sufficient isolation.

Only the broker resolves infrastructure credentials and invokes Omnitron operations. It validates typed arguments, resource scope, live authority, quotas and approval, then records a correlated audit intent before mutation. Use an OS/container/VM sandbox appropriate to the deployment and prove containment in integration tests. A container with host sockets or privileged mounts is not isolated.

Provider events and tool calls carry provider-session, agent-run and operation correlation IDs. Enforce one active dispatcher per run, bounded queues and event sizes, and replay-safe sequence handling. A reconnect restores conversation state, not permission to replay a completed mutation. On uncertain delivery, query the operation ID before retrying.

## Review artifacts and approvals

An execution proposal is an immutable artifact containing source revision/hash, target IDs, resolved execution profile, parameter digest, allowed effects, timeout, maximum attempts and rollback instructions where meaningful. Do not expose secret parameter values in the approval view.

Bind a single-use approval to that artifact, requester, approver, expiry and nonce. Consume it atomically with operation admission and retain the decision in audit. An edit to code, parameters, targets or execution profile invalidates it. The actor requesting execution cannot approve their own request when policy requires separation of duties. A denied or unavailable audit write prevents the effect.

Console presents a compact proposal with expandable code, targets, privileges, estimated query/execution cost and the recovery procedure. CLI receives the same artifact and decision endpoints. No browser-only checks or provider-text interpretation may authorize execution.

## Monitoring as evidence

An agent's analysis should cite retrievable evidence: the query, resource scope, time window, source/collector identity, sample or event timestamps, collection time, result completeness and correlation IDs. Distinguish an observed value from an inference. A healthy process does not prove successful email delivery or business correctness.

Current MCP `monitoring.snapshot`, `metrics.series` and `alerts.active` reuse Titan/Omnitron readers and explicitly label unavailable or oversized sources. Collection timestamps are not source freshness. Historical log queries are bounded before SQL construction. These improvements do not yet constitute delegated agent access or complete query cost control.

The next shared query contract must expose:

- **Coverage:** expected/observed nodes and applications, disconnected collectors, last successful ingestion and source watermark.
- **Loss:** dropped events, rejected batches, queue pressure, retention boundaries and gaps after restart or disconnection.
- **Completeness:** limit reached, sampling, aggregation interval, estimated/known population and continuation cursor.
- **Time:** source event time, receipt time and query time separately; clock skew makes cross-node ordering uncertain.
- **Provenance:** stable log/trace/alert references and evidence hashes where retained, with a clear expiry if retention can remove them.

Unknown, stale, sampled, truncated and absent are distinct states. Never convert failed queries to zero errors or no alerts. A sampled error count is not an exact population count; a missing metric is not zero. Alert summaries must reveal evaluation freshness, not only the last persisted firing state.

### Efficiency and scale

Reuse Titan's storage and query abstractions. Do not build a second analytics database in the agent adapter. Apply allowlisted dimensions, metric-series cardinality budgets, bounded batches, backpressure and retention/rollups at the owning layer. Never use raw URLs, user input, tokens, run IDs or source text as unbounded metric labels.

Admit queries with explicit resource and time scope, row/byte/series bounds, deadlines and cancellable database work. Add keyset pagination for deep history, per-principal concurrency and cost budgets, and fair scheduling so agent analysis cannot starve incident response. A response byte cap alone does not cap database work. Prefer ingestion counters and bounded snapshots for polling; full archive counts are a deliberate analytic query.

Use subscriptions with cursor-based resumption and bounded queues for alert/event delivery. Slow consumers receive an explicit gap and recover through a bounded historical read. Do not run a full health probe or archive count on every dashboard refresh. Cache only under the effective scope and policy version; invalidation must not reveal another principal's data.

## Alerts and notifications

Keep evaluation, incident state and delivery separate. An alert's lifecycle includes pending, firing and resolved, with timestamps and rule revision. Acknowledgement records operator ownership; it does not resolve the underlying condition. Silence is scoped, expiring and audited. Dedupe keys and grouping must prevent a flapping service from producing an unbounded notification storm.

Use the existing Titan notification adapter/outbox boundary for future messenger integrations. Persist delivery intent with the state transition, assign an idempotency key, retry transient failures with bounded exponential backoff and jitter, and surface exhausted deliveries for inspection/retry. Track provider acceptance separately from confirmed delivery when a provider supplies receipts. Do not promise exactly-once delivery across external systems.

A delivery adapter receives a redacted event with stable IDs and a link to the authorized Console view. Routing, recipients and secret references remain server policy. Messenger payloads, responses and logs cannot change roles, approve scripts or grant access. No messenger provider is enabled by this design document.

## Untrusted input and remediation

Application logs, issue text, repository content, tool results and notification bodies are untrusted data. Instructions embedded in them never change system policy. Keep them visibly separated from trusted instructions and pass only the evidence needed for the current analysis. Redact before provider egress, and require explicit policy for exporting sensitive operational data.

Begin with observation and proposals. Automatic remediation must select a reviewed, versioned playbook with bounded targets, rate limits, cooldown, idempotency, health criteria and a stop condition. A log message must never become a shell command. Canary changes and rollback require evidence; repeated failed remediation escalates to a human instead of looping.

## Delivery and acceptance

Implement in independently testable stages: scoped read-only broker and principal propagation; provider connection and resumable observation; draft/review artifacts; isolated approved execution; bounded remediation. The current administrator pipeline is not a shortcut around those stages.

Before enabling each stage, verify:

1. Viewer/operator/agent boundaries and cross-project denial through actual HTTP, WebSocket, CLI and broker paths.
2. Mid-run revocation, owner demotion, expiry, stale approvals, replay and simultaneous approval consumption.
3. Audit/store failure before mutation, worker crash, daemon restart, network partition and lost completion messages.
4. Query deadlines, slow consumers, high cardinality, ingestion loss and visible stale/partial results under load.
5. Prompt injection in logs/tool output, secret exfiltration attempts and filesystem/socket/network escape tests.
6. Provider quota/auth failures with no silent switch to paid API usage, plus reconnect without duplicate effects.

Publish measured latency, memory, queue and throughput budgets for the tested deployment profile. Availability or latency targets are acceptance criteria, not guarantees inferred from an architecture diagram.

---

---
id: "omnitron/architecture"
title: "Architecture"
description: "Components, planes, and data flows inside the Omnitron supervisor."
---

# Architecture

Omnitron exposes one control plane per node through Netron RPC. Its backend
provides authentication, project ownership, infrastructure management and
observability. In coupled mode it also owns application processes. In
[resident execution mode](/docs/omnitron/backend-handoff), a separate process owns forks,
workers, pools and local topology while the backend uses a fenced private RPC
contract. The diagram below describes the default coupled deployment.

See the [resilient control-plane design](/docs/omnitron/resilient-control-plane) for the target
contracts for environment promotion, placement, recovery and scaling. Its capability
gates distinguish implemented behavior from planned autonomous operations.

## Component map

```mermaid
flowchart TB
  subgraph DaemonProc["Daemon process (one per node)"]
    direction TB

    subgraph IngressPlane["Ingress planes"]
      US[Unix socket<br/>~/.omnitron/daemon.sock]
      TCP[TCP listener<br/>0.0.0.0:9700]
      HTTP[HTTP/WS listener<br/>0.0.0.0:9800]
    end

    subgraph Core["Core services"]
      Netron[Netron RPC bus]
      Daemon[OmnitronDaemon service]
      Auth[Auth service + JWT]
      State[(SQLite state store<br/>data/daemon-state.db)]
      PIDM[PID manager]
      Sched[Daemon scheduler]
    end

    subgraph BuiltIn["Built-in RPC services (master daemon)"]
      Deploy[deploy]
      Fleet[fleet]
      Secrets[secrets]
      Backup[backup]
      Pipeline[pipeline]
      Project[project]
      Infra[infrastructure]
      K8s[kubernetes]
      Logs[log-collector]
      Metrics[metrics]
      Trace[trace-collector]
      Telemetry[telemetry]
      Health[health-check]
      Sync[sync]
      SysInfo[system-info]
      EventBC[event-broadcaster]
      Discovery[discovery]
      Alert[alert]
      Node[node-manager]
    end

    subgraph Orchestrator["Orchestrator subsystem"]
      AppHandle[App handle]
      Boot[Bootstrap loader]
      TSC[TS compiler]
      Watch[File watcher]
      DepRes[Dependency resolver]
      Router[Service router]
      Janitor[Process janitor]
      Classic[Classic launcher]
      MWP[Module-worker spawner]
    end

    subgraph InfraSub["Infrastructure subsystem"]
      Docker[Docker provider]
      Bare[Bare-metal provider]
      Provisioner[Service provisioner]
    end

    subgraph Cluster["Cluster subsystem"]
      Leader[Leader election]
      Replicate[State replication]
    end

    subgraph Webapp["Webapp host"]
      Static[Vite dev / static]
      Bridge[Netron HTTP bridge]
    end
  end

  US --> Netron
  TCP --> Netron
  HTTP --> Netron
  HTTP --> Static
  Static --> Bridge --> Netron

  Netron --> Auth
  Netron --> Daemon
  Daemon --> State
  Daemon --> PIDM
  Daemon --> Sched
  Daemon --> Orchestrator
  Daemon --> BuiltIn

  Orchestrator --> Classic
  Orchestrator --> MWP
  Orchestrator --> Watch
  Orchestrator --> Janitor

  Classic -- fork --> Children
  MWP -- fork --> Children[Child app processes]

  Daemon --> InfraSub
  InfraSub --> DockerD[Docker daemon]

  Daemon --> Cluster
  Cluster --> RemoteDaemons[Other omnitron daemons]
```

## The three planes

Omnitron exposes three independent transports — same Netron RPC
surface, different network shape.

### 1. Management plane — `unix://~/.omnitron/daemon.sock`

Unix domain socket, file mode `0o600` (owner-only). The **trust
boundary**: if you can open the socket, you've already passed the
OS-level identity check. CLI calls auth-bypass through this
socket for ergonomics.

| Carries                      | Used by                            |
| ---------------------------- | ---------------------------------- |
| All built-in RPC services    | `omnitron` CLI                     |
| The `OmnitronDaemon` service | Webapp dev (when running locally)  |
| MCP server requests          | Agent processes spawned by the CLI |

### 2. Public TCP plane — `tcp://0.0.0.0:9700`

Opt-in. Disabled by default; enable when:

- You run a fleet (other daemons connect for cluster membership).
- You operate `omnitron remote` (alias-addressed daemons).
- You expose a programmatic API to an external CI/CD.

JWT is **required** on this plane. RBAC roles
(`viewer` / `operator` / `admin`) gate every method.

### 3. HTTP / WS plane — `http://0.0.0.0:9800`

Opt-in. The daemon's HTTP plane is the **Netron HTTP + WebSocket
bridge** that browser clients use. With the default `httpPort`
of `9800`, the daemon binds the HTTP bridge on `httpPort + 1`
(`:9801`) and the WebSocket transport on `httpPort + 2`
(`:9802`); the public `:9800` is fronted by the `omnitron-nginx`
container, which serves the static bundle and reverse-proxies
`/netron/*` and `/ws` to those daemon ports.

JWT is required for RPC, except for static asset routes (served
by nginx) and the unauthenticated `signIn` / `validateToken` /
`refreshSession` methods.

## Daemon lifecycle

```mermaid
sequenceDiagram
  participant OS as systemd / launchd
  participant D as Daemon
  participant Orch as Orchestrator
  participant SS as State store
  participant PM as PID manager

  OS->>D: Start configured daemon entry
  D->>PM: acquire lock (~/.omnitron/daemon.pid)
  alt lock taken
    PM-->>D: throw — already running
    D->>OS: exit 1
  end
  D->>SS: Open durable state and project registry
  D->>D: bind Unix socket, [TCP, HTTP]
  D->>D: register all RPC services
  D->>Orch: Read registered projects and enabled stacks
  loop per enabled stack
    Orch->>Orch: Resume selected release or development configuration
  end
  D-->>OS: ready
  Note over D: serve requests indefinitely
  OS->>D: SIGTERM
  D->>Orch: stopAll(graceful)
  D->>SS: persist state
  D->>PM: release lock
  D->>OS: exit 0
```

The state-store is the **persistent intent** — what should be
running. On crash + restart, the daemon reads `state.json` and
relaunches anything that was alive at last write.

## State store — `state.json`

A single JSON file capturing:

```typescript
interface DaemonState {
  apps: Record<string, {
    name:        string;
    status:      'starting' | 'running' | 'stopped' | 'crashed';
    pid?:        number;
    startedAt?:  number;
    lastError?:  string;
    restarts:    number;
    processes:   Record<string, ChildProcessState>;
  }>;
  cluster?: {
    role:    'master' | 'follower';
    leader?: string;
    term:    number;
  };
}
```

Persisted on every status transition, atomic-rename style
(`write tmp → rename → fsync` parent). If the file is corrupt at
boot, the daemon starts fresh — empty apps map — and logs a
warning.

## PID manager

Owns `~/.omnitron/daemon.pid`. Responsibilities:

- **Lock acquisition** at boot: open exclusive, write `process.pid`.
  If lock fails, another daemon is running — abort.
- **Liveness sweep**: periodically check whether the lock-holding
  PID is still alive; if not, reclaim the lock.
- **Atomic ownership transfer** during `cluster.step-down` — the
  outgoing leader explicitly releases.

Child process PIDs are owned by the orchestrator (per `IAppHandle`),
not the PID manager.

## Daemon scheduler

A small in-process scheduler bound to the daemon's lifecycle.
Runs periodic tasks:

| Task                                   | Default interval              |
| -------------------------------------- | ----------------------------- |
| Health probe sweep across all apps     | 15 s                          |
| Metrics aggregation tick               | 5 s                           |
| State persistence flush                | on transition + 30 s baseline |
| Crash-loop backoff timer               | per-app, exponential          |
| Cluster heartbeat (if cluster.enabled) | 2 s                           |
| Health-monitor sweep (cluster nodes)   | 60 s                          |

All scheduler timers are `.unref()`-ed so they don't block
shutdown.

## Orchestrator subsystem

Owns the per-app launch pipeline. One `AppHandle` per running
app:

```mermaid
flowchart LR
  Daemon -->|"launch(name)"| AppH[AppHandle]
  AppH --> Boot[Bootstrap loader]
  Boot --> TSC[TS compiler<br/>if .ts source]
  Boot --> Resolve[Dependency resolver]
  Resolve --> SR[Service router]
  AppH --> Spawn{Spawn mode}
  Spawn -- classic --> CL[ClassicLauncher]
  Spawn -- module --> MWP[ModuleWorkerProcess]
  CL --> Forks
  MWP --> Forks[Child processes]
  AppH --> Watcher[FileWatcher<br/>if dev]
  AppH --> Janitor[ProcessJanitor]
```

Two launch modes:

- **Classic launcher** — `node bootstrap.js` is forked once;
  bootstrap is responsible for creating all `Application`s and
  running its own subprocess management. Used for legacy apps.
- **Module-worker spawner** — one fork per `IProcessEntry`. Each
  child imports a single module file and runs `Application.create`
  directly. Default for new apps; lower memory, faster boot.

→ Full pipeline: [Orchestrator](/docs/omnitron/orchestrator).

## Built-in RPC services

A daemon exposes services according to its configured role. A worker registers a reduced core set —
the PostgreSQL-backed services (`OmnitronAuth`, `OmnitronAlerts`,
`OmnitronTelemetry`, `OmnitronFleet`, `OmnitronDiscovery`,
`OmnitronPipelines`, `OmnitronTraces`, `OmnitronDeploy`,
`OmnitronNodes`) are master-only. All services share
authentication / authorization with the `OmnitronDaemon` service.

The first column is the **Netron service name** (the identifier
clients resolve, e.g. `daemon.OmnitronAuth.signIn(...)`).

| Service              | Purpose                                                   |
| -------------------- | --------------------------------------------------------- |
| `OmnitronDaemon`     | App lifecycle — start / stop / restart / status / inspect |
| `OmnitronAuth`       | JWT issue / verify / RBAC                                 |
| `OmnitronSecrets`    | Encrypted secret CRUD                                     |
| `OmnitronInfra`      | Docker container management (Postgres / Redis / etc.)     |
| `OmnitronDeploy`     | Legacy operation history; per-app mutations refuse        |
| `OmnitronFleet`      | Cross-node fleet operations                               |
| `OmnitronPipelines`  | CI/CD pipeline runs                                       |
| `OmnitronBackups`    | Database backup / restore                                 |
| `OmnitronProject`    | Project + stack registry                                  |
| `OmnitronKubernetes` | k8s integration (apply / scale / observe)                 |
| `OmnitronNodes`      | Infrastructure node inventory (node-manager)              |
| `OmnitronLogs`       | Per-app log streaming + filtering                         |
| `OmnitronMetrics`    | Metrics aggregation (from `titan-metrics`)                |
| `OmnitronTraces`     | Distributed trace ingestion                               |
| `OmnitronTelemetry`  | Telemetry-relay (`titan-telemetry-relay`) aggregator      |
| `OmnitronHealth`     | Active health-check runs                                  |
| `OmnitronDiscovery`  | Service discovery state                                   |
| `OmnitronEvents`     | Cross-process event bus (WebSocket subscriptions)         |
| `OmnitronAlerts`     | Alert rules + delivery                                    |
| `OmnitronSync`       | Cross-daemon state synchronisation                        |
| `OmnitronSystemInfo` | Host CPU / RAM / disk inventory                           |
| `OmnitronCluster`    | Daemon participation and enrolled controller election     |

→ Full reference: [Services reference](/docs/omnitron/services-reference).

## Infrastructure subsystem

When an app's `omnitronConfig.infrastructure` declares a
requirement, the daemon's infrastructure subsystem:

1. Checks if the requirement is already satisfied (running
   container, registered bare-metal service).
2. Resolves connection parameters (host / port / credentials).
3. Provisions if missing — Docker for dev/test, bare-metal hooks
   for prod.
4. Injects resolved env vars (`DATABASE_URL`, `REDIS_URL`, …) into
   the app at startup.

```mermaid
flowchart LR
  App["App declaration<br/>requires: { database, redis, s3 }"]
  App --> Resolver
  Resolver{Provider?}
  Resolver -- dev/test --> Docker[Docker provider]
  Resolver -- prod --> Bare[Bare-metal hook]
  Docker --> Containers[Running containers]
  Bare --> Existing[Existing servers]
  Containers --> Inject[Inject env vars]
  Existing --> Inject
  Inject --> App
```

→ Reference: `omnitron infra status` shows current containers;
the [Infra CLI section](/docs/omnitron/cli#infrastructure) covers commands.

## Cluster subsystem

Optional programmatic `cluster.enabled: true` uses the same writable PostgreSQL
as the fleet registry to arbitrate controller leases. The store commits owner,
incarnation and epoch before acknowledging authority. There is no peer gossip
protocol, anonymous voting or independent consensus log.

```mermaid
flowchart LR
  D1[Controller A] --> Lease[(PostgreSQL lease and durable epoch)]
  D2[Controller B] --> Lease
  Lease --> Status[Authenticated authority observations]
  Status --> Admission[Recipient fencing required for effects]
```

The lease lasts 15 seconds by default and is renewed every 2 seconds. Loss of
confirmation demotes the controller. Database mutations can be fenced in the same
transaction; remote effects need durable recipient-side fencing. Normal master/slave
fleet operation is independent of this optional mode. See [cluster coordination](/docs/omnitron/cluster)
and the [release delivery gates](/docs/omnitron/release-readiness) for implemented boundaries.

## Webapp host

The React console is served by a dedicated `omnitron-nginx`
container — **not** by the daemon process itself. The daemon's
HTTP plane carries only the Netron RPC bridge; nginx serves the
static bundle and proxies RPC to the daemon:

| Mode           | Served by                                                        | Command                                              |
| -------------- | ---------------------------------------------------------------- | ---------------------------------------------------- |
| **Production** | `omnitron-nginx` container fronting `apps/omnitron/webapp/dist/` | `omnitron webapp build` then `omnitron webapp start` |
| **Dev**        | Vite dev server with HMR                                         | `pnpm dev` in `apps/omnitron/webapp/`                |

nginx (`:9800`) proxies `/netron/*` to the daemon's HTTP listener
(`httpPort + 1`, default `:9801`) and `/ws` to the daemon's Netron
WebSocket transport (`httpPort + 2`, default `:9802`). In dev mode
Vite serves the console on `:9810` and proxies the same two paths to
the same daemon ports, so the browser asks for `/netron/*` and `/ws`
on its own origin either way and the client hard-codes neither. See
[Console](/docs/omnitron/console) for the full port map.

## Auth model — three roles

The `OmnitronDaemon` service and others use role-based access:

| Role       | What they can do                                                    |
| ---------- | ------------------------------------------------------------------- |
| `viewer`   | Read-only: list / status / inspect / metrics / health / logs        |
| `operator` | Viewer + lifecycle: start / stop / restart / reload / scale / exec  |
| `admin`    | Operator + destructive: shutdown / reloadConfig / setMetricsEnabled |

The local Unix socket bypasses auth — local CLI calls run as the
implicit `admin` (OS-level trust). TCP/HTTP planes always require
JWT.

## Data flow — log shipping

```mermaid
sequenceDiagram
  participant Child as Child process
  participant Pipe as stdio pipe
  participant LC as log-collector service
  participant File as ~/.omnitron/logs/{app}.log
  participant CLI as omnitron logs

  Child->>Pipe: pino JSON line
  Pipe->>LC: ingest entry
  LC->>File: append + rotate if size > threshold
  LC->>LC: keep ring buffer (last N entries per app)
  CLI->>LC: getLogs({name, lines, level, grep})
  LC-->>CLI: filtered entries (from ring + file tail)
  CLI->>CLI: render to terminal
```

Logs flow through the daemon — no separate log shipper. The
ring buffer means `omnitron logs --follow` returns recent entries
immediately even if the file has been rotated.

## Data flow — metrics aggregation

```mermaid
sequenceDiagram
  participant W as Worker daemon
  participant S as Worker SQLite
  participant C as Controller
  participant DB as PostgreSQL
  W->>S: Commit samples and outbox rows together
  C->>W: Read pending batch over authenticated mesh
  W-->>C: Entries with source and sequence
  C->>DB: Commit accepted entries and persistence evidence
  C->>W: Acknowledge committed delivery
  C->>DB: Read history for CLI and Console
```

The controller stores its own samples directly through Titan's PostgreSQL adapter. Each source retains unacknowledged worker entries. See [replication and receipts](/docs/omnitron/cluster).

## Data flow — leader election (cluster mode)

```mermaid
sequenceDiagram
  participant A as Controller A
  participant B as Controller B
  participant DB as PostgreSQL primary
  participant W as Enrolled worker
  A->>DB: Acquire leased epoch
  A->>DB: Renew while the grant is fresh
  A->>W: Signed operation for this node
  W->>DB: Verify current grant before admitting the effect
  Note over A,DB: A releases its lease or the lease expires
  B->>DB: Acquire a higher epoch
  W->>DB: Reject later effects signed by A
```

An administrator can request a transfer with `omnitron cluster step-down`. The current controller demotes and releases its epoch. A successor reports leadership after PostgreSQL confirms its grant.

## Database and diagnostics boundaries

Kysera implements reusable database mechanisms. Titan supplies module lifecycle,
connection and transaction context. Omnitron exposes application-facing migration
commands and operational diagnostics. Applications provide schema definitions
and domain policies; Console displays the same health data as CLI.

See [migrations](/docs/omnitron/migrations) for adopting existing journals and
[diagnostics](/docs/omnitron/diagnostics) for error capture and notification adapters.

## Putting it together

Read these in order for a complete picture:

1. **[Daemon](/docs/omnitron/daemon)** — the always-on process; its
   internals.
2. **[Orchestrator](/docs/omnitron/orchestrator)** — how apps launch.
3. **[Services reference](/docs/omnitron/services-reference)** — every RPC
   method.
4. **[CLI](/docs/omnitron/cli)** — the operator's daily interface.
5. **[Console](/docs/omnitron/console)** — the web UI.

---

---
id: "omnitron/auth-rbac"
title: "Auth & RBAC"
description: "Authentication, sessions, JWT, RBAC roles, RLS context propagation."
---

# Auth & RBAC

> **Info**
>
> This page covers the **operator-flavoured** auth surface
> (CLI, webapp, MCP, service-to-service). For the framework-
> agnostic authorisation primitives (permission grammar,
> per-user overrides, ABAC conditions, RLS bridge, audit
> trail), start at [Authentication & Authorisation](/docs/auth).

Omnitron authenticates human operators and machine identities on the control plane. [Console access](/docs/omnitron/access) documents the implemented account-management flow shared by webapp and CLI. Examples about end-user identity below are application integration patterns; they are not additional Console login methods.

## Three identity surfaces

| Surface                                   | Trust source                                                            | Auth flow                                                       |
| ----------------------------------------- | ----------------------------------------------------------------------- | --------------------------------------------------------------- |
| Local CLI                                 | OS-level file permission (Unix socket `0o600`)                          | Implicit admin — no JWT needed                                  |
| Webapp / remote CLI                       | JWT issued by `OmnitronAuth.signIn`                                     | Token-backed peer; current authority checked on each invocation |
| Service-to-service (cluster, fleet, sync) | Node-specific signing secret reached over the registered SSH connection | Short-lived `service_role` JWT on an authenticated peer         |

Apps in your ecosystem add a **fourth** surface — end users
authenticating against one of your apps. The recommended
pattern (used by the production reference) is:

| Your surface             | Trust source                      | Auth flow                                     |
| ------------------------ | --------------------------------- | --------------------------------------------- |
| End-user HTTP / WS calls | JWT issued by *your* identity app | Same JWT verified by every app in the fan-out |

## Three RBAC roles

| Role       | Inherits             | What they can do                                                                                       |
| ---------- | -------------------- | ------------------------------------------------------------------------------------------------------ |
| `viewer`   | —                    | Read-only: list / inspect / status / metrics / health / logs                                           |
| `operator` | `viewer`             | Viewer + lifecycle: start / stop / restart / reload / scale                                            |
| `admin`    | `operator`, `viewer` | Operator + arbitrary execution, SSH, automation, shutdown / reloadConfig / secrets / setMetricsEnabled |

Replication ingestion and outbox acknowledgements require administrator or
service authority. Operators can inspect diagnostics but cannot forge persistence
evidence or release queued records.

Methods declare explicit role lists through `VIEWER_ROLES`, `OPERATOR_ROLES` and `ADMIN_ROLES`. The shared hierarchy in `@omnitron-dev/access/roles` also supports live principal checks at account and terminal boundaries. Unknown roles receive no implicit grants.

Human token validation on the master reads the current account role from PostgreSQL. An old JWT role claim cannot preserve a demoted role. `OmnitronAccess.describe()` derives the UI catalog from the running services' decorators and Titan authorization rules. See [revocation behavior](/docs/omnitron/access#change-roles-and-revoke-sessions).

```typescript
// From the daemon RPC service:
@Public({ auth: { roles: VIEWER_ROLES } })   // ['admin','operator','viewer']
async list(): Promise<ProcessInfoDto[]> { /* ... */ }

@Public({ auth: { roles: OPERATOR_ROLES } }) // ['admin','operator']
async startApp(...) { /* ... */ }

@Public({ auth: { roles: ADMIN_ROLES } })    // ['admin']
async shutdown(...) { /* ... */ }

@Public({ auth: { allowAnonymous: true } })  // no token required
async ping() { /* ... */ }
```

→ See [Daemon / RPC surface](/docs/omnitron/daemon#rpc-surface--omnitrondaemon-service)
for the full method-by-method role map.

## Shared JWT across fan-out apps

One identity owner issues user tokens; each backend verifies their signature
and asks the owner for the current session and grants. Signature validity alone
does not establish current authority. Keep human and service credentials under
explicit issuer control, and isolate signing credentials by trust boundary.

```mermaid
sequenceDiagram
  participant U as User
  participant App as Application backend
  participant Auth as Session authority
  participant DB as Canonical database
  U->>App: Request with access token
  App->>App: Verify signature, issuer and expiry
  App->>Auth: Authenticated session and subject lookup
  Auth->>DB: One consistent session, account and grants snapshot
  DB-->>Auth: Current authority
  Auth-->>App: Live session and permissions
  App->>App: Enforce role, permission and resource policy
  App-->>U: Response
```

## Shared auth factory pattern

Use the shipped [shared-session preset](/docs/titan/modules/shared-session)
and provide an application-owned `sessionAuthority(sessionId, userId)` adapter.
The owner reads its database directly; other backends call the protected
`SessionAuthority` service. Share bootstrap wiring in one workspace package,
while keeping policy definitions in the browser-safe Access layer.

Registry compatibility mode supports `sessionRedis` and `sessionLookup`, but
never restores cache entries automatically. Cache restoration and revocation
must share a serialization protocol. A Redis hit does not prove that a newer
database revocation or permission edit was propagated.

## Application authority without duplicate policy engines

Keep three responsibilities separate:

- **Access** evaluates permission keys, system-role hierarchy and session policies.
- **Titan / Titan Auth** validate credentials, resolve current authority, isolate
  invocation context and enforce transport/method admission.
- **The application** owns its account, assignment and resource records. Its
  canonical adapter reads a consistent database snapshot and uses the shared
  predicates; CLI and browser controls consume this authority.

A reusable application permission role should reference a definition, not copy
its grants into every account. Keep system rank separate from such definitions:
renaming a custom role must not create a service identity or administrative tier.
Union applicable grants, but require **all** applicable session policies. Missing
assigned definitions must deny access rather than silently removing restrictions.

Serialize sensitive assignment changes with the current actor, target and role
definitions. Verify the delegation ceiling and commit the audit record in the
same [Titan database transaction](/docs/titan/modules/database). A form should
carry the version or assignments it edited, so concurrent changes return a
conflict rather than overwriting another administrator's work.

Refresh-token rotation, session extension and access-token issuance also belong
in one transaction. A signing or authority-store failure must preserve the
presented refresh credential. Repositories join Titan's transaction context,
which delegates transaction/savepoint behavior to Kysera; they must not start an
independent transaction inside the operation. Update compatibility caches after
commit, and keep them out of authorization decisions.

These are application integration requirements. The shared-session preset does
not create an application's role-assignment schema, refresh-token store or editor.

## Activate an authorization update

Treat the controller, resident execution owner and application runtimes as
separate artifacts. A successful backend handoff updates the controller; it does
not replace code already loaded by the owner or by an application. Inspect each
artifact before declaring a security rollout complete. Restarting the owner
requires an application maintenance window; do not infer a pause-free owner
upgrade from `handoffSupported`.

For a shared session authority, apply compatible additive schema changes first,
then update the authority, its consumers and the account-management UI. Configure
the authority endpoint explicitly. Give simultaneous Netron clients unique
routing identities; a shared diagnostic label is not an identity. Before starting
an artifact, verify that its bootstrap adapter and application resolve the same
Titan runtime.

Compare `omnitron access show --json` with Console **Access**, but account for
their identities: the owner-only local socket grants local administrative
authority, whereas the browser has the signed-in account's role. Verify a viewer
and an operator through the real token transport, including direct calls to
denied methods. Hiding a navigation item alone is not a permission check. Change
the test account's role while connected and verify session revocation; remove
the temporary account and its sessions after the check.

Retain additive columns during code rollback. Once new permission assignments or
session restrictions are used, an older artifact that ignores them is not a safe
rollback target even if it can read the schema. Check both data compatibility and
authorization semantics before restoring it.

## Token cache

The shared-session preset and Omnitron daemon disable authentication-result
caching. Signature verification can be cached independently. Netron revalidates
persistent token-backed peers before each invocation, so a live role change
reaches the method gate without reconnecting. Expiry also bounds cached results
when a different application explicitly enables caching.

Authority-store failures remain 503. They do not permit stale authorization.
In-flight operations retain the context they were admitted with; critical
mutations must additionally use transaction-level ownership and authority checks.

## Token binding via `cnf.fp`

Standard JWT auth has one weakness: a stolen short-lived token
works until it expires. **`cnf.fp` binds the JWT to a refresh
chain** — if the chain rotates or revokes, the JWT becomes
invalid immediately even before expiry.

How it works:

1. Identity app issues a refresh token, records its id `rtid`.
2. When minting the access JWT, it stamps
   `cnf.fp = HMAC(secret, rtid)` into the JWT claims.
3. Every backend, after verifying signature + session, also:
   a. Looks up the session's **currently active** refresh-token id
   from the canonical authority snapshot.
   b. Computes `HMAC(secret, currentRtid)`.
   c. If `cnf.fp` doesn't match → reject as 401.

Result: if the refresh chain rotates (user signed in elsewhere)
or revokes (user changed password), the JWT immediately stops
working without waiting for `exp`.

```typescript
createJwtAuthManager({
  // ...
  cnfHmacSecret: process.env.JWT_SECRET,  // typically the same JWT signing key
});
```

The HMAC secret is conventionally the same as the JWT signing
key — one rotation surface handled by the `kid` mechanism.

## Refresh token rotation

```mermaid
sequenceDiagram
  participant Client
  participant Auth as identity app

  Client->>Auth: POST /sign-in
  Auth-->>Client: { access: JWT(short), refresh: rtid_1 }
  Note over Client: rtid_1 stored in HttpOnly cookie

  loop access expires
    Client->>Auth: POST /refresh (rtid_1)
    Auth->>Auth: mark rtid_1 used<br/>create rtid_2
    Auth-->>Client: { access: JWT(cnf.fp = HMAC(rtid_2)), refresh: rtid_2 }
    Note over Client: rtid_1 now used<br/>only rtid_2 is presentable
  end

  Note over Client: token stolen and used after refresh
  Note over Auth: stolen JWT carries cnf.fp(rtid_1)<br/>session's<br/>currently-active is rtid_2 → mismatch → 401
```

Refresh tokens are **one-use** — using `rtid_N` invalidates it
and produces `rtid_N+1`. Reuse of an old refresh token is a
strong signal of token theft.

## Multi-key JWT verification (rotation without downtime)

```typescript
JwtModule.forRoot({
  algorithm: 'HS256',
  jwtSecret: 'current-key',                  // legacy / fallback
  verificationKeys: {
    'k2025-01': 'older-key-still-valid',
    'k2025-04': 'current-key',
    'k2025-07': 'next-key',
  },
});
```

When issuing, set `kid: 'k2025-04'` in the JWT header. Verifiers
pick the matching key by `kid`. Rotate by:

1. Add new key under a new `kid`.
2. Start signing with new `kid` (everywhere — coordinated deploy).
3. Wait for all old-key tokens to expire (`exp`).
4. Remove old key from `verificationKeys`.

Zero downtime, zero forced sign-outs.

## RLS context propagation

The Netron auth manager populates `authContext` in invocation
metadata; the **invocation wrapper** maps that into a kysera RLS
scope so Postgres policies see the right session variables.

```typescript
// Shared wrapper:
import { mapAuthToRLSContext } from '@omnitron-dev/titan/netron/auth';
import { rlsContext }          from '@omnitron-dev/titan-database/rls';

export function createRlsInvocationWrapper() {
  return async (metadata: Map<string, unknown>, fn: () => Promise<unknown>) => {
    const authCtx = metadata.get('authContext') as AuthContext | undefined;
    if (authCtx) {
      const rlsCtx = mapAuthToRLSContext(authCtx, {
        defaultTenantId: (authCtx.metadata as any)?.tenantId ?? 'default',
      });
      return rlsContext.runAsync(rlsCtx, fn);
    }
    return fn();
  };
}

// In every app's bootstrap:
auth: {
  jwt: { enabled: true, tokenCacheTtl: 60_000 },
  invocationWrapper: createRlsInvocationWrapper(),
},
```

Now every database query — from any `@Service` method, over any
transport — runs inside an AsyncLocalStorage scope that exposes
`user_id`, `is_system`, `tenant_id` to RLS policies. Repositories
pick this up automatically via the kysera RLS plugin.

> **One wrapper, two places that apply it**
>
> `transport/http/server.ts` applies it per request. Every socket
> transport — WebSocket, TCP, Unix — dispatches through
> `remote-peer.ts`, which applies it per invocation on the `CALL`
> branch and builds a metadata map of the same shape, so a wrapper
> written against HTTP works unchanged.
>
> Until recently only the HTTP half existed, and a call arriving
> over WebSocket ran with **no RLS context**. That failed closed
> rather than open — kysera gives `SELECT` an impossible predicate
> and lets `UPDATE` / `DELETE` touch no rows — so it surfaced as a
> realtime page that was simply empty, which is why it lasted: the
> wrong answer looked like no data rather than an error.

## Role hierarchy patterns for end-user RBAC

For your application's own role system (separate from the
operator roles above), declare a hierarchy and let library
functions handle precedence:

```typescript
// packages/auth-utils/src/role-hierarchy.ts
export const ROLE_PRIORITY = {
  guest:     0,
  user:      10,
  contributor: 20,
  moderator: 50,
  admin:     100,
} as const;

export type RoleName = keyof typeof ROLE_PRIORITY;

export function effectiveLevel(roles: readonly RoleName[]): number {
  return Math.max(0, ...roles.map(r => ROLE_PRIORITY[r] ?? 0));
}

export function canModerateUserWithRole(
  actor: readonly RoleName[],
  target: readonly RoleName[],
): boolean {
  return effectiveLevel(actor) > effectiveLevel(target);
}

export function visibleRolesFor(viewerRoles: readonly RoleName[]): RoleName[] {
  const cap = effectiveLevel(viewerRoles);
  return (Object.keys(ROLE_PRIORITY) as RoleName[]).filter(r => ROLE_PRIORITY[r] <= cap);
}
```

Centralised hierarchy = no per-feature ad-hoc checks. Tiered
visibility (search results, profile views) and tiered moderation
(who can ban whom) compose from these primitives.

## Anonymous and service-to-service endpoints

Three auth shapes per method:

```typescript
// 1. Public — anyone with socket access
@Public({ auth: { allowAnonymous: true } })
async ping() { /* ... */ }

// 2. Authenticated user — any logged-in user
@Public({ auth: { roles: ['user', 'admin'] } })
async getProfile() { /* ... */ }

// 3. Recipient protocol — authenticated machine only
@Public({ auth: { roles: ['service_role'] } })
async prepareNodeStackStop(data: NodeStackStopRequest) { /* ... */ }
```

The master-to-node path is implemented in `cluster/mesh-link.ts` and `cluster/slave-connector.ts`. It uses the node's registered SSH connection to obtain that node's signing secret, mints an HS256 JWT with issuer `omnitron` and role `service_role` (five-minute default lifetime), then authenticates the persistent peer. Each RPC must explicitly permit that identity. `CONTROL_PLANE_ROLES` and `CONTROL_PLANE_READ_ROLES` make the distinction reusable.

Machine identity is outside the human role hierarchy. It does not grant account administration or interactive SSH. Do not create a human account with `service_role`, reuse a Console session as a machine credential, or infer machine access from a role rank.

## API key pattern (for headless integrations)

For CI agents, partner integrations, or scripted clients that
shouldn't go through interactive sign-in:

```typescript
// In your identity app:
async createApiKey(userId: string, scopes: string[], expiresAt?: Date) {
  const apiKey = `pk_${crypto.randomBytes(32).toString('base64url')}`;
  const hash   = await argon2.hash(apiKey);
  await db.insertInto('api_keys').values({ userId, hash, scopes, expiresAt }).execute();
  return apiKey;     // returned ONCE; not stored in plaintext anywhere
}

async verifyApiKey(presented: string): Promise<AuthContext | null> {
  // Look up by prefix (first 8 chars) for fast filter
  const candidates = await db.selectFrom('api_keys')
    .where('hashPrefix', '=', presented.slice(3, 11))
    .selectAll().execute();
  for (const row of candidates) {
    if (await argon2.verify(row.hash, presented)) {
      return buildAuthContext(row.userId, row.scopes);
    }
  }
  return null;
}
```

Then in your auth manager, accept either JWT or API key:

```typescript
if (token.startsWith('pk_')) return verifyApiKey(token);
return verifyJwt(token);
```

API keys carry their own scope list — narrower than JWT roles —
so a compromised key only opens the doors you've granted it.

## TOTP / WebAuthn second factor

For human users (especially admins), require a second factor on
sign-in:

```typescript
async signIn({ email, password, totpCode }) {
  const user = await this.users.verifyPassword(email, password);
  if (!user) throw Errors.unauthorized('invalid credentials');

  if (user.totpEnabled) {
    if (!totpCode) return { requires2fa: true };
    const ok = await verifyTotp(user.totpSecret, totpCode, /* window */ 1);
    if (!ok) throw Errors.unauthorized('invalid 2FA code');
  }

  return this.createSession(user);
}
```

WebAuthn / passkeys are stronger; same flow shape.

Console currently has no MFA enrollment or verification flow. The stored `totpEnabled` / `pgpEnabled` fields do not implement one. `requiresMfa` is not an enforced Titan RPC option. A future second-factor gate must verify proof server-side, bind it to the current session and enforce it in the shared authorization path before a UI can advertise it.

## Auth in WebSockets

WS auth is verified on the **upgrade request** only:

```typescript
ws.on('upgrade', async (req) => {
  const token = extractBearer(req) ?? extractCookieToken(req);
  if (!token) return ws.reject(401);
  const authCtx = await authManager.verify(token);
  ws.context = authCtx;
});
```

After upgrade, Netron retains the credential privately and revalidates its
authority before each invocation. Changed roles reach the method gate; revoked
or expired credentials are refused. Authentication credentials are never exposed
in the serialized context.

For token rotation while a socket is open: the client receives
a `token.expiring-soon` push, refreshes the token via HTTP, and
reconnects WS. The application may also use an explicitly enabled authentication task;
HTTP cookie-only applications should retain their established refresh flow.

## Private runtime storage

On POSIX systems, runtime directories are sealed to `0700` and state, backup and
journal files to `0600`. Sealing opens the target without following a leaf symlink,
checks its type and owner, changes permissions on that open file and verifies the
result. Shared hard-linked files and incompatible owners are refused. A failed
permission change is an operation failure, not a successful insecure backup.

The main file must exist. Missing optional SQLite `-wal`/`-shm` companions are
allowed; other access errors propagate. Do not use symlinks as substitutes for
private runtime files. These checks are POSIX-specific; Windows deployments need
an equivalent owner-only ACL policy. They do not turn a writable parent directory
into a sandbox against a privileged local process.

## Audit logging

The daemon stores control-plane events through `AuditService`. The current wire fields are `action`, `resourceType`, `resourceId` and `details`; actor identity comes from the request context. Read events through Console's **Audit** page or `omnitron audit`.

Account creation, role changes and session revocation write their audit rows in the same PostgreSQL transaction as the mutation. An audit failure rolls them back. Interactive SSH requires a successful audit write before host trust changes or session opening; lifecycle completion is best effort. Many older operational paths still use best-effort audit to preserve availability. These guarantees differ and must not be represented as a universal fail-closed audit policy.

Tokens, passwords, SSH keys, commands and terminal transcripts are excluded from the new account/terminal audit details. The database trail is not a tamper-evident external archive; deployments requiring that property need a separately controlled audit sink.

## Anti-patterns

- **Auth check in custom HTTP routes that's weaker than the RPC
  surface.** Always call the same access predicate (`assertAccess`,
  `requireRole`) from both paths.
- **Tokens in URLs.** Show up in proxy logs, browser history,
  referer headers. Always use `Authorization: Bearer`.
- **No `cnf.fp` on high-value operations.** A 15-min JWT is fine
  for read APIs; for destructive actions, prefer token-binding so
  rotation invalidates immediately.
- **Skipping RLS for "internal" services.** Internal callers
  carry their own AuthContext; even the identity app's own
  queries benefit from RLS as defense in depth.
- **API keys without scopes.** A scope-less API key is a
  full-power credential in disguise. Always require a scope list.
- **One key for everything.** Separate keys per integration so
  rotation / revocation is granular.
- **Logging tokens.** Even at debug level, JWTs in logs let
  anyone with log access impersonate users. Log `kid` + `code`
  on failures, never the full token.
- **`allowAnonymous: true` on a mutating endpoint.** Be explicit;
  read-only auto-discovery is fine, but anything that writes
  state should require auth.

## See also

- [Daemon / Auth flow](/docs/omnitron/daemon#auth-flow) — the daemon's own auth
- [Console access](/docs/omnitron/access) — account management and effective permissions
- [SSH workspace](/docs/omnitron/terminal) — pinned terminals and ownership
- [Services reference](/docs/omnitron/services-reference) — `OmnitronAuth`, `OmnitronAccess` and terminal methods
- [Best practices / Shared authentication](/docs/omnitron/best-practices#shared-authentication-across-apps)
- [`titan-auth`](/docs/titan/modules/auth) — JWT module reference
- [`titan-database`](/docs/titan/modules/database) — RLS plugin

---

---
id: "omnitron/automation"
title: "Xec automation"
description: "Author, review, execute and inspect bounded Xec pipelines through Console and CLI."
---

# Xec automation

Use **Pipelines** in Console to create an automation, review its source and targets, run it with parameters, and inspect each step. Definitions belong to the daemon; the section is available in the system workspace without selecting a project. A workspace selection does not restrict execution targets or provide an authorization scope. The CLI calls the same `OmnitronPipelines` service. Omnitron owns authority, persistent run state and presentation; `@xec-sh/ops` owns the execution graph, retries and cancellation; `@xec-sh/core` owns local and SSH process execution.

This executor is for trusted administrators. A separate child process prevents a script from blocking the daemon's JavaScript event loop, but does **not** sandbox its filesystem, network or operating-system privileges. Do not give an untrusted agent the local daemon socket or this executor. See the [agent integration design](/docs/omnitron/agents).

## First automation

Save this as `inspect.json`, replacing the working directory with an existing project directory:

```json
{
  "name": "inspect-runtime",
  "description": "Inspect a runtime before planning a change",
  "steps": [
    {
      "name": "runtime",
      "kind": "xec",
      "cwd": "/srv/projects/example",
      "timeout": 30000,
      "run": "const label: string = String(params.label ?? 'runtime'); console.log(label); await $target`node --version`;"
    }
  ]
}
```

```bash
omnitron pipeline save inspect.json
omnitron pipeline list
omnitron pipeline run <pipeline-id> --params-file parameters.json
omnitron pipeline status <run-id> --output
omnitron pipeline status <run-id> --json
omnitron pipeline cancel <run-id>
```

`parameters.json` contains a JSON object, for example `{"label":"candidate"}`. Keep private parameter files outside version control. Parameter values travel as data and are represented as `[provided]` in run history. This does not make deliberate printing or copying of a secret safe.

Every pipeline command supports the global `--json` flag or `OMNITRON_OUTPUT=json`. Successful requests emit one `{ok: true, data: ...}` object on stdout; request failures emit `{ok: false, error: ...}` on stderr. A status request for a failed, cancelled or interrupted run still returns its record, with exit code 1.

To update an existing automation without replacing its identity or run history, first review its definition and `revision` in `omnitron --json pipeline list`, then run:

```bash
omnitron pipeline save inspect.json --id <pipeline-id> --expected-revision <reviewed-sha256>
```

The update compares that revision inside a database transaction. If another editor changed the definition, the save fails without replacing their work. Console sends the revision loaded when its editor opened and keeps your draft on a conflict. Reopen the current definition and reconcile both edits before saving again. Prefer IDs when names are ambiguous.

## Steps and targets

| Field       | Meaning                                                                                         |
| ----------- | ----------------------------------------------------------------------------------------------- |
| `name`      | Unique graph key, 1–80 letters, digits, `_`, `.` or `-`                                         |
| `kind`      | `shell` by default; `xec` for TypeScript; `release` for a pinned compiled entry                 |
| `run`       | Shell source or TypeScript script; absent for a release step                                    |
| `target`    | Release step only: registered project, stack, app, producer artifact and package-relative entry |
| `args`      | Release step only: literal argument strings                                                     |
| `cwd`       | Existing working directory; module imports resolve relative to it                               |
| `nodeId`    | Registered node ID; omission or `local` selects the local machine                               |
| `dependsOn` | Names of steps that must complete first                                                         |
| `timeout`   | Step deadline in milliseconds; default five minutes, maximum 24 hours                           |
| `retries`   | Additional attempts, 0–5; only use with repeatable operations                                   |
| `env`       | Explicit environment entries                                                                    |
| `condition` | A small parameter expression, such as `params.enabled` or `params.stage == 'test'`              |

An Xec script receives `params`, `$` for local commands and `$target` for the selected target. TypeScript runs in a local child process; selecting a remote node does not move that interpreter to the node. Use `$target` for remote commands. The server resolves SSH credentials from the node registry and vault and requires a trusted host key. The browser never receives credentials.

Shell steps receive parameters as `OMNITRON_PARAMS` JSON. `{{parameter}}` source substitution is rejected. Unsupported conditions and cron/webhook triggers are rejected rather than silently ignored; this release supports manual triggers. Xec validates the complete dependency graph before any step executes.

From Omnitron 0.3.5 with Xec Ops 0.11.4, SSH shell steps receive their declared environment and `OMNITRON_PARAMS`. Local callbacks can still read the controller environment. Configure credentials and paths on the node that owns them.

## Execute a pinned release

Use `kind: 'release'` when an operation must run a compiled member of an installed
application release. The definition pins a producer artifact and a package-relative
entry. Omnitron verifies the selected installation and uses the resident execution
owner's Node through the existing Xec executor. The definition contains no physical
installation path or interpreter path.

A caller running at its registered compiled launch entry can obtain the producer
identity through the public runtime API:

```ts
import { verifyInstalledRuntime } from '@omnitron-dev/omnitron/runtime';
import type { PipelineDef } from '@omnitron-dev/omnitron/dto/services';

const runtime = await verifyInstalledRuntime(new URL(import.meta.url));
if (runtime.state !== 'verified') throw Error(runtime.reason);
const definition: PipelineDef = {
  name: 'publish-results',
  steps: [{
    name: 'publish',
    kind: 'release',
    target: {
      project: 'sample', stack: 'tools', app: 'publisher',
      artifact: runtime.producer,
      entry: 'dist/publish.js',
    },
    cwd: '/srv/sample-output',
    args: ['--checked'],
    timeout: 180000,
    retries: 0,
  }],
};
```

`artifact` records `releaseId`, `manifestSha256`, `artifactSha256`, `producerApp`,
`projectCommit` and `frameworkCommit`. An imported artifact retains its producer;
the run records the selected consumer release separately. The entry must be a
physical compiled JavaScript member of the same verified closure. Arguments are
literal array elements; parameters remain JSON in `OMNITRON_PARAMS`.

Prepare and select the release through the ordinary stack lifecycle before
admitting the pipeline. Admission checks the installation, source identity,
selected release, declaration, environment and native owner binding again before
each attempt. It does not install or prepare a missing artifact, provision
infrastructure, change release selection or start the application. This contract
supports independently owned local stacks; remote release targets are refused.
It keeps the admitted owner during the run. Repeated full verification can be
expensive for large closures.

Use the ordinary pipeline save/run commands or Console's Advanced JSON editor.
Saving retains the target and arguments. Changing the owner can change the
verified physical installation without changing the logical definition, ID or
history; selecting a different producer requires an explicit reviewed update.
Run details show per-attempt artifact, installed-tree, executable, target, native
owner lifetime, process ID and environment fingerprint. Environment values are
not copied into that evidence.

The release step cannot set `PATH`, `NODE_OPTIONS` or `NODE_PATH`. The verified
application supplies its resource-only Node options. Inherited or declared
`LD_*` and `DYLD_*` loader controls are refused; Linux library paths are derived
only from the verified closure. `cwd` must be an absolute physical directory,
and application output belongs outside the immutable installation.

## Review and authority

Every public pipeline method requires `admin`. Invocation revalidates the human session, including its current account role. Sensitive mutations must write an audit intent before taking effect; a failed audit write denies the operation. The owner-only Unix socket remains an OS-authorized administrative interface.

The run review shows source, target name and ID, working directory, dependencies, retries, condition and environment. Expand each step before admitting it.

The definition carries a SHA-256 `revision`. Console submits the revision displayed for review; the CLI fetches the selected definition immediately before admission. Direct RPC and MCP callers must supply `expectedRevision` for both execution and updates. If source, targets, environment or other stored definition fields changed, admission fails before inserting a run. This is optimistic concurrency control, not a separate multi-person approval workflow.

Per-step history records a digest of the admitted definition, target ID, actor, attempts, timing, outcome and bounded stdout/stderr. It does not yet keep a complete immutable source archive. Do not treat the digest alone as recoverable source.

## Upgrading existing integrations

Review saved definitions and their callers when upgrading the pipeline contract:

- Step names are stable identifiers of 1–80 letters, digits, underscores, dots or hyphens. Update dependency references when renaming a step.
- Direct RPC callers fetch the definition and submit its `revision` as `expectedRevision`. Before updating, require `conditionalUpdates: true` in that response; older daemons ignore extra request fields and cannot enforce an update precondition. A recurring operator should pin the reviewed revision and stop dispatching after a conflict until the operator reviews the change; it must not automatically accept a newer definition.
- Replace source interpolation with data parameters and remove unsupported trigger descriptors. Existing external schedulers can continue to dispatch manual runs.
- New release-bound integrations use `kind: 'release'` and a producer artifact identity. The owner resolves the physical executable. Existing shell definitions that compare absolute paths or generated `PATH` values need a separately reviewed caller and an explicit conditional definition update; this framework contract does not rewrite them automatically.

Validate the updated definitions before enabling recurring dispatch. Re-registering a definition is separate from executing its deployment or publication steps.

A pinned recurring application does not automatically pin the programs launched by shell steps. Package one-shot entry points with the application and use a release target for those compiled members. Keep mutable data directories separate. The dispatcher must require the reviewed target, arguments, environment and revision. Moving to another producer requires an explicit conditional definition update; it must not silently accept a changed pipeline or compile a helper from a development checkout during a production run.

## Progress and failure

Console shows individual step states, elapsed time, attempt counts, exit codes and output. Open a run for details; the history table fetches summaries without downloading every run's output. A failed refresh is displayed as stale data. On narrow windows, details use the full viewport and long code/output remains inside its scroll area.

`omnitron pipeline cancel <run-id>` first records `cancelling`, signals active work and awaits executor termination before returning a terminal state. Dependent steps do not start after cancellation. A daemon restart marks unfinished runs `interrupted`; it never replays operations automatically. Inspect external effects before retrying an interrupted run.

Release attempts retain `prepared`, `started`, `completed` or `unconfirmed`
execution evidence. If execution began but its binding or durable completion
cannot be confirmed, the attempt is `unconfirmed` and automatic retries are
refused. Recovery marks unfinished prepared/started attempts unconfirmed while
preserving completed attempts. Missing spawn acknowledgement does not prove that
no effect occurred. Inspect effects before explicitly running again. The owner
proof is an observation, not an operating-system process lease or a promise of
exactly-once execution.

Local process groups and Xec SSH channels participate in cancellation. A command that deliberately detaches a remote process can outlive its SSH channel; channel closure is not proof that every remote side effect stopped. Use managed Omnitron jobs or an explicitly reversible playbook for long-lived services.

## SSH ownership and uncertain effects

Omnitron owns authorization, trusted node identities, durable operation state,
audit and presentation. Xec owns SSH authentication, host verification, connection
reuse, channels, SFTP, forwarding and cancellation. Applications must use the Xec
API and public types rather than importing its transport library or adding a
parallel connection pool.

Xec 0.12 distinguishes `outcome-unknown` from a known command
failure: losing a reply after dispatch does not prove whether an operation ran.
The execution boundary refuses automatic replay and exposes `recoverable: false`. Inspect
the remote effect or reconcile an operation receipt before an explicit retry.
SFTP interruption can leave a partial destination; upload/download helpers are not
atomic deployment transactions. Omnitron's durable operation model must preserve
that distinction through both CLI and Console.

The [release checklist](/docs/omnitron/release-readiness) distinguishes this verified source
candidate from installed fleet behavior. Dependency publication and coordinated
activation remain separate steps. Existing applications do not need a direct SSH
transport dependency.

## Resource bounds

| Resource                 | Bound                               |
| ------------------------ | ----------------------------------- |
| Steps per definition     | 32                                  |
| Active runs per daemon   | 8                                   |
| Concurrent steps per run | 4                                   |
| Source per step          | 128 KiB                             |
| Parameters               | 64 scalar JSON values, 16 KiB total |
| Retained stdout/stderr   | 32 KiB per stream and step          |
| History request          | At most 200 summaries               |

Output persistence is throttled and awaited, applying backpressure instead of starting unbounded database writes. Xec's SSH stream honors writable backpressure and avoids a second unbounded capture. History reports truncation explicitly. Known parameter/credential values and private-key blocks are redacted; redaction is best effort and does not replace secret isolation.

The daemon exposes bounded-label run counters and duration histograms through the existing Titan metrics path. Code, user parameters and run IDs are not metric labels. See [Observability](/docs/omnitron/observability) for querying signals and [Console access](/docs/omnitron/access) for effective grants.

---

---
id: "omnitron/backend-handoff"
title: "Backend replacement"
description: "Replace a control plane while a resident owner retains application processes."
---

# Backend replacement

Resident execution separates the backend from the process that owns applications. `omnitron backend upgrade` replaces the backend while the resident owner keeps application PIDs, pools, local topology routes, watchers and disk log capture. CLI and Console requests pause during replacement and reconnect afterwards; this is not uninterrupted control-plane availability.

Resident execution is opt-in on POSIX. It supports unmanaged operation and separate OS services for the execution owner and backend guardian. A legacy single-service layout is refused: detaching a child does not establish independent service ownership. Replacing the owner or its Node runtime still requires an application maintenance window.

## Choose the update mechanism

| Purpose                                   | Entry point                                                                | Boundary                                                                                                                         |
| ----------------------------------------- | -------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------- |
| Replace the local resident backend        | `backend upgrade <sealed-dist>` through its local supervisor               | Keeps the owner and application processes. The fleet cannot update its initiating controller.                                    |
| Replace remote managed backends           | `fleet rollout` or Console **Nodes → Managed backend updates**             | Uses sealed builds and one shared, durable receipt across CLI and Console.                                                       |
| Bootstrap an older independent guardian   | Native `backend guardian prepare` and reviewed `apply`                     | A separate, qualified window before using `kernelHandoff@1`. Retains the selected backend and owner.                             |
| Replace a coupled host's workspace bundle | `fleet upgrade` or Console **Advanced · workspace source bundle upgrades** | Uses the legacy host/app contract and can restart applications. Independently supervised hosts require the managed backend path. |

The first transition from a coupled daemon to an independent owner still requires
an application maintenance window. Neither a fleet plan nor a guardian update
transfers application ownership to a new owner process.

## Owner compatibility

A compatible RPC envelope does not imply compatible lifecycle behavior. A sealed
backend declares the owner contracts it requires; the owner advertises the
contracts it implements. The current `stackControl@1` contract includes durable
stack intent, admission fences and confirmed process termination.
Stack stop and launch admission cover the same complete stack prefix, including
retained applications with legacy nested names. Watch and recovery cannot restart
such an application while its parent stack is fenced.

The optional `executionScopes@1` owner contract reuses that durable mechanism for
`{ project, stack, application? }`. The private Netron operations
`inspectExecution`, `stopExecution` and `enableExecution` bind an application stop
to its launch generation and parent stack revision. A child fence survives an
owner restart; enabling the parent cannot clear it. Resuming the child requires
its observed intent revision. Installed-stack recovery refuses a separately
fenced child before launching any process. Older owners are refused before RPC
or recovery-state changes.

This contract is an owner primitive. The public named `stop` operation still
controls current execution without committing this durable child fence; it must
not be used to promise exclusion after boot recovery. There is also no Console or
CLI selector for switching one application between its working tree and a release.
Activating it requires an owner transition, independently of a backend update.
Before downgrading the owner, reconcile child fences explicitly: a legacy owner
does not understand them.

The optional `stoppedStackReconciliation@1` contract allows an explicit local
stack start to reconcile a stopped stack's complete application graph, whether
its source is a selected release or the registered development configuration. The
owner validates the current parent revision, every retained generation and the
absence of live process handles. It revokes obsolete source watches and retires
only registrations absent from the selected graph. Logs, release archives,
infrastructure and other stacks remain intact. Stop and reconciliation share
one in-flight execution boundary; a parallel start cannot reopen the stack.

The controller commits its existing SQLite release selection behind the durable
stop fence. A failed write, lost reply or watcher failure leaves the stack fenced
for inspection and explicit retry or rollback. Automatic boot/resume never
retires retained registrations. This contract is optional for backend handoff;
older owners continue supporting unchanged stack membership and refuse a graph
change before the controller changes the selection. Updating the owner itself
still needs an application maintenance window. See
[changing a local stack's applications](/docs/omnitron/releases#changing-a-local-stacks-applications).

Run the **candidate's CLI** with `backend upgrade <sealed-dist> --check` before
planning activation. It validates the selected instance's owner identity and
contracts before delegating to a guardian or draining the backend. The check runs
again before preparing a handoff, and the candidate independently verifies its
owner before acquiring control. An older owner with no capability declaration
cannot satisfy a new requirement. A version mismatch is also refused; a larger
version number is not assumed to be backward compatible.

`backend status --json` exposes `execution.ownerCapabilities`. An absent field
means the backend does not report these contracts; an empty object means its
owner advertised none. `handoffSupported: true` describes the transfer mechanism,
not compatibility with every future backend. If a required contract is absent,
keep the current backend running and prepare a controlled owner transition with
application inventory, readiness budgets and rollback. Never bypass the refusal
by replacing the active files or changing an advertised capability.

Candidate preflight has a 30-second deadline. Failures preserve a structured
`BACKEND_PREFLIGHT_*` code with the runtime hash, child PID, elapsed time, last
reported load phase, exit status and bounded stderr. CLI JSON retains the same
error details carried over Netron. A load-progress message is diagnostic only:
admission still requires a successful protocol response. A timeout with phase
`not-reported` means no valid progress was received; it does not establish a
particular cause or justify activating the candidate.

## First transition

Run `omnitron backend status --json` before planning an update. `mode: coupled` means the backend directly owns its workers. Those live forks and threads cannot be transferred to another parent without changing their lifecycle. **The first transition requires one controlled application restart.** The upgrade command refuses a legacy daemon before draining or stopping anything.

During an approved maintenance window, preserve the current configuration, project registrations, running inventory and previous build. Merge the following into the existing daemon configuration; do not replace its authentication or access settings:

```json
{
  "executionHost": {
    "enabled": true,
    "directory": "/absolute/private/omni-execution"
  }
}
```

The directory must belong to the daemon user, have mode `0700`, and be short enough for Unix socket paths (the longest generated socket must fit within 100 bytes). Checkpoints and sockets use `0600`. Configuration must be serializable; executable hooks belong in application bootstrap modules.

Start from a sealed backend artifact and verify application health, readiness, child PIDs, pool membership and registered runtime integrity. Application bootstrap workers use their application's installed Omnitron adapter and Titan instance, preserving DI identity independently of the backend's dependency tree. An incompatible or missing adapter is refused with an explicit error.

## Build and seal

Build and test in a separate checkout or staging directory. Never rebuild dependencies underneath a live application. The source tree must be clean and the backend must contain a valid `BUILD.json`.

```sh
node apps/omnitron/scripts/freeze-backend.mjs \
  /absolute/staging/apps/omnitron \
  /absolute/releases/omnitron-next
```

The freezer copies the installed runtime dependency closure and the current Node executable into a new tree owned by the instance account, rewrites package links inside the artifact, and records file hashes in `RUNTIME.json`. It performs no install scripts, package upgrades or network operations. Optional `package=compiled-package-directory` arguments select explicitly built workspace packages; their versions must match the installed graph. Keep the current sealed artifact for rollback. If the package includes Console, its complete output must be present at `webapp/dist/index.html`; a partial `webapp` directory is rejected. Headless packages omit the directory. Include the frontend before sealing rather than modifying an installed artifact.

Use the artifact's Node executable and CLI when starting that release:

```sh
/absolute/releases/omnitron-current/node \
  /absolute/releases/omnitron-current/packages/0/dist/cli/omnitron.js up
```

## Permanent installation and cold start

`backend upgrade` installs both the candidate and rollback into the instance's permanent store before draining the backend. Each directory is addressed by its complete SHA-256 under `$OMNITRON_HOME/runtime/releases/`. Copying never moves or rewrites a running release. The copied Node executable, backend and dependency closure are revalidated before publication and execution; an existing directory with different contents is refused.

The startup selector is stored transactionally in `runtime/state.db`, using the existing daemon state store with full SQLite durability. It records the current release, previous release, selector revision and a separately pinned launcher release. An exclusive installation lease prevents concurrent upgraders or selectors; a dead installer's lease can be recovered. Cold startup shares that lease until the daemon has initialized, closing the version-selection/PID-file race.

After the first successful upgrade, use the stable entry point:

```sh
~/.omnitron/runtime/bin/omnitron backend installation --json
~/.omnitron/runtime/bin/omnitron up
```

For a different `OMNITRON_HOME`, the launcher lives in that instance's `runtime/bin` directory and retains the instance path. It follows the selected release and uses that release's own Node. The normal detached and foreground startup paths also honor the selector. A missing, modified or incompatible selected artifact is an error, never an implicit fallback to a source checkout.

Foreground startup releases the installation lease after initialization while keeping the daemon process running. A launcher that forwards to the selected runtime waits for that process and preserves its exit status. The native CLI qualification covers detached startup, installed foreground startup and forwarding from another runtime, including RPC shutdown and a repeated invocation against an already running daemon.

To prepare an installation while the backend and execution owner are stopped:

```sh
omnitron backend install /absolute/sealed-runtime --json
omnitron backend select <complete-sha256> --json
```

`install` only copies and verifies. `select` refuses a running backend or owner, an unfinished activation and an incompatible migration transition. Use `upgrade` for a running resident installation. Selecting a release does not replay an old handoff checkpoint: a normal cold start reads the current project registry and configuration. A living owner with existing applications still requires validated recovery rather than startup reconciliation.

The activation intent is durable before retirement; the selector commits only after public readiness and application identity checks. If the coordinator dies or publication fails, the unfinished activation blocks cold startup. `backend recover` can verify and acknowledge an already-ready successor without launching a second controller; otherwise it restores the verified rollback after proving the backend is absent and the owner inventory is unchanged. Administrative diagnostics and recovery remain accessible through the stable launcher during this state. After a successful activation, the proven management release becomes the recovery bootstrap; a failed candidate cannot replace it. A later legacy rollback retains that capable management release.

`execution/active-backend.json` remains a diagnostic receipt. The installation database is the startup selector. There is no automatic release deletion: retain the selected, rollback and launcher releases, and every artifact still used by an owner or application. In particular, installing a permanent backend does not relocate an already-running owner from its original artifact.

## Replace the backend

```sh
omnitron backend status --json
omnitron backend upgrade /absolute/releases/omnitron-next/packages/0/dist --check
omnitron backend upgrade /absolute/releases/omnitron-next/packages/0/dist
```

`--check` validates both candidate and rollback artifacts without draining requests. Activation repeats validation before retirement. Changed database migration files are refused by default. A compatible master schema expansion requires the separate, exact plan described below. A replacement also inspects the existing migration journal through Omnitron’s shared Kysera runner before opening public ingress. Pending, changed or unapproved unknown migrations refuse readiness; this check never creates history or applies migrations. Legacy records without checksums remain explicitly unverified by `omnitron migrate verify --daemon`. Applying a migration separately does not bypass the artifact compatibility check.

The replacement proceeds through these boundaries:

1. Reject active SSH sessions, file transfers, pipelines and release builds. Drain admitted RPC calls, property operations, Netron tasks and infrastructure reconciliation through their replies.
2. Verify and install candidate/rollback artifacts; validate native container identity and specifications. Capture project ownership, retry backoff, monitoring configuration and application identities in a private checkpoint.
3. Pause metric polling, wait for any child batch already in flight, and flush accepted samples to the configured storage. A failed or timed-out flush refuses the transfer and the incumbent resumes polling. Fence execution mutations with a new controller epoch. Persist the grant before enabling the new controller; stale controllers cannot write or acknowledge observations.
4. Retire only backend services. The owner continues running applications and capturing logs.
5. Start the candidate and adopt existing infrastructure. Open public RPC and resume metric polling once the candidate is ready.
6. Verify public readiness, owner identity, epoch, application PIDs, start times and restart counts. Only then complete retirement.

A pool can replace a worker after the handoff under its normal recycling policy. Compare the inventory captured at transfer time first. For a later PID change, match the worker ID in the owner log to its recycling reason (for example, `max_lifetime`) and its exit code.

A failed candidate is stopped and the previous artifact is started under a fresh epoch. Rollback does not reuse an old grant. A failure to prove identity or inventory preservation stops automatic recovery rather than guessing which processes to kill or relaunch.

A service that failed before creating a container can transfer its failed state when native Docker inspection confirms that the container is absent. The checkpoint binds that absence separately from the desired service and its error; the candidate verifies it again and preserves the failure. An unavailable Docker daemon is not evidence of absence. A missing previously observed container, changed container identity/specification or legacy checkpoint without this evidence still refuses transfer. Handoff does not repair infrastructure or turn a failed service healthy.

## Recovery after a backend crash

A sealed resident backend maintains a private recovery journal beside the owner socket, in `execution/recovery.db`. It reuses the daemon SQLite store with `synchronous=FULL`. Before changing ownership bookkeeping, it durably invalidates the previous certificate; a failed write refuses the mutation. The last snapshot remains available privately for diagnosis but is never accepted as a current recovery certificate.

Every five seconds, a quiet backend reads its owner, project registry and infrastructure bookkeeping without draining ingress, provisioning containers or pausing applications. It publishes only a stable generation with no unfinished execution mutation or lifecycle operation. Active SSH sessions, transfers, pipelines and builds prevent certification. Unchanged snapshots are not rewritten. `execution.recovery` reports `pending`, `ready` or `unavailable` with the reason; a pending operation is not an application health failure.

Run `omnitron backend recover` without a checkpoint to restore a crashed backend from its current certificate. It refuses a live controller, an unfinished activation, an unverified or different selected artifact, an externally changed project registry, a changed owner/controller grant or any difference in the recorded application inventory. Infrastructure adoption checks actual container specifications before public readiness. Recovery obtains a fresh fenced grant and uses the same validated replacement path as a planned update; it never replays application startup or migrations.

A crash during a mutation can leave recovery uncertified. A changed process inventory, including an application restart after the certificate, also requires reconciliation rather than automatic adoption. Use the reported reason and private journal to investigate; do not stop a living owner to make the check pass. If recovery itself is interrupted after preparing its grant, the runtime installation records the explicit handoff checkpoint for `omnitron backend recover /absolute/private/handoff-ID.json`.

The journal does not itself install an OS supervisor or promise recovery from owner/host failure. The unmanaged mode still needs an operator or external controller to invoke recovery after proving the backend is dead.

## Independent OS supervision

With `executionHost.enabled`, `omnitron service install --prepare` writes a reviewable plan under `runtime/supervision-plan` without stopping processes or loading OS services. The plan contains the private runtime configuration and both service definitions. For a living backend, preparation requires its current recovery certificate and takes the environment from that process. It waits up to 15 seconds for a pending certificate, releasing the installation lease between attempts so the publisher can make progress. Every attempt repeats admission checks; the certificate must match both the current backend PID and selected runtime. An expired wait reports the concrete refusal and publishes no partial plan. This wait never stops processes or bypasses recovery validation. Installation retains the prepared environment across the pause instead of replacing it with the installing shell’s values. An active unmanaged owner cannot change OS service ownership in place: complete a controlled initial shutdown before `omnitron service install`. Installation refuses a running backend or owner instead of stopping them implicitly.

The default plan pins both roles to the selected runtime. To prepare a different
installed owner, use `omnitron service install --prepare --owner-runtime <sha256>`.
The ordinary artifact verifier checks its sealed bytes, native compatibility and
execution capabilities against the selected backend. The plan records both hashes
and renders each role from its own executable; installation requires the same owner
and selected backend hashes and service scope. This permits an explicitly qualified
recovery combination without editing generated service files. It does not establish application or data-schema
compatibility: qualify those separately, including any source-watch limitations.

After stopping the instance, repeat `omnitron service install --prepare` to
review a different installed owner or newly selected backend. Preparation validates
the saved plan's structure, instance, execution directory, service scope and native
role identities. It copies the saved private environment exactly, including its
secrets, even when the preparing shell has no corresponding variables. An invalid
saved record is refused; preparation does not replace it with shell defaults.
Review the new plan before installing. Installation still refuses changed owner
or backend selections until an explicit preparation has recorded those bindings.

Preparation and installation share the runtime operation lock. Installation also
requires launchd jobs to be unloaded, or systemd units to be inactive with no queued
job, and rechecks process evidence before writing active files. Unknown OS query
results refuse installation. For a planned owner window, first require a ready
recovery certificate, a ready guardian with no pending operation, and a settled
runtime selection; a certificate can become ready before a handoff finishes.

For an owner maintenance window, review and stop each application stack with
`stack stop <project> <stack> --keep-infrastructure --dry-run`, then execute the
same mode with its review digest. This retains databases, caches, other stack
containers, ownership and selected releases while durably disabling application
resume and fencing watcher/recovery starts. Confirm process exit and every
per-node receipt before shutting down native owner services. Do not substitute a
full stack stop when the plan requires infrastructure to remain available.
Retained containers alone do not prove application or data-schema recovery safety;
qualify the exact releases and domain recovery separately. After the new owner is
verified, explicitly start the reviewed releases and check their health. A backend
or owner restart must not silently undo the maintenance stop.

The owner runs in `dev.omnitron.owner` on macOS or `omnitron-owner.service` on Linux. The existing daemon service runs a guardian, which owns all backend generations. It uses the same replacement and certified recovery functions as the CLI. CLI upgrades and recovery delegate through its private Unix socket, so a shell-launched successor cannot escape OS supervision. The selected backend’s private control identity determines its instance home before supervisor lookup; a custom public socket is never redirected to the invoking shell’s default instance. Recovery obtains the instance from the validated checkpoint or certified journal and checks its execution directory. A guardian refuses another instance’s upgrade or recovery. No public HTTP endpoint exposes this socket.

A periodic guardian health inspection does not represent an operator operation. One command can reserve admission while that read-only inspection finishes, for at most ten seconds. A second command, active recovery or shutdown is refused. Admission repeats isolation checks before starting work. A timed-out command is never executed later, and `operationPending` includes the reserved command. The correction lives in the guardian: replacing only the backend cannot update an older guardian's admission behavior.

Before accepting the managed owner, the backend verifies native launchd/systemd main PIDs, its guardian parent and, on Linux, separate owner/backend cgroups. Configuration or environment flags alone cannot establish isolation. Service entry points verify their pinned artifact before importing the runtime. Backend candidates remain in the guardian's process group; the owner and application processes remain in their separate service.

If a backend dies, the guardian restores it only from a certified checkpoint or a recorded interrupted activation. If the guardian dies, the OS restarts it and the same ownership checks apply. Recovery retries use bounded backoff and preserve the reason when state cannot be proved. `omnitron service status` reads guardian diagnostics even while the backend is unavailable; private `execution/supervisor.log`, `supervisor-state.json` and `backend-startup.log` provide the underlying evidence. A living unverified backend is never duplicated or killed automatically.

The owner service deliberately does not restart after a crash on the same host boot. Some classic applications use separate process groups, so a fresh owner cannot assume all of its predecessor's processes disappeared. A durable receipt distinguishes clean shutdown, a new host boot and uncertain ownership. Reconcile the surviving processes before owner recovery; changing the receipt alone is not reconciliation. Normal `omnitron down` stops both services and applications, and `omnitron up` starts them again. Backend-only OS restarts retain applications when a certified checkpoint is available.

For system scope, prepare as the non-root instance owner with `omnitron service install --scope system --prepare`, then have an administrator install the reviewed unit files at their reported paths. System units name that account explicitly. Do not prepare the application runtime as root or copy secrets into unit environment directives. The private supervision record retains required runtime environment values; unit definitions carry only the instance path.

### Migrating an existing Linux node

A coupled daemon owns its application processes. Its first transition to independent system services requires a controlled application restart. Subsequent **backend** replacements retain the owner, application PIDs and runtime dependency bindings. Replacing the owner itself still requires a maintenance window.

Prepare a sealed copy using the Node executable and installed dependencies of that node. Before stopping anything, validate its application bootstrap readers, operational health, private socket permissions, native supervisor identity and saved environment. Save the old configuration, unit definitions, project registrations and online SQLite backups; prepare the new owner/backend units and a concrete rollback procedure alongside the running installation.

An owner replacement also requires cold application recovery checks in both
directions. Run ordinary release admission against the actual project
registrations, configuration inputs and selected archives for the candidate and
rollback. A running application's retained launch handle does not prove that a
fresh owner can load its release after those inputs change. Successful backend
handoff checks alone do not qualify that recovery. Resolve declaration drift
before the window; do not bypass digest verification during recovery.

Bootstrap children and pools use the application's declared `cwd`, falling back
to its project directory. Persistent files belong there or in an explicitly
configured data directory, never in the sealed owner runtime. Validate this
boundary with real child processes; an unchanged package hash does not excuse
unexpected writes to an installed runtime tree.

Preserve the existing instance account, home and application dependency bindings. Changing a root-owned legacy installation to another account is a separate migration: it can affect application files, credentials and container access. The automatic `service install --prepare` flow continues to require a non-root account for new system installations. An administrator-reviewed migration of an existing root-owned instance must explicitly retain that identity and private control paths; it does not authorize a second daemon or shared sockets.

During the approved window, disable file watching, stop applications gracefully, verify their processes have exited, and then retire the old daemon. Enable resident execution, select the sealed runtime and install the two independently supervised services. Start the owner before the backend. Confirm the original application inventory, operational health, ready recovery certificate, separate native cgroups and durable log delivery before proceeding to another node.

Plan process startup and domain readiness separately. A stream processor can be
online while it reconstructs a complete observation window; a healthy connection
or running PID is not evidence that its outputs are usable. Set the acceptance
budget from the workload's real data requirements, startup latency and sampling
cadence. For example, a five-minute input window can require a further full minute
at the boundary. A generic four-minute health timeout would force a needless
rollback and create a second gap in that input.

Record the latest pending check, its reason and elapsed waiting time. A later
successful connection replaces an earlier socket error in the diagnosis; do not
report that stale error when domain health is the actual blocker. Preserve the
original health result and withhold dependent outputs until it passes. Never
fabricate completeness, replay a missed forecast, or repeatedly restart a healthy
collector to clear a data-window warning. Include the same readiness requirements
in rollback verification. If runtime restoration has succeeded but data readiness
is pending, report those states separately and verify recovery before closing the
maintenance record.

Rollback restores service ownership, configuration and the retained runtime. It must first prove that the new owner and its application processes have stopped. Keep newly written logs, replication acknowledgments and application data: do not restore a pre-transition SQLite snapshot over them. Retain the failed transition's runtime and execution journals for diagnosis.

Remote application deployment recognizes a settled, sealed managed backend through its selected runtime, native guardian parent and separate owner/backend cgroups. A stale selection or mismatched process is refused. The coupled `fleet upgrade` workflow refuses an independently supervised node before stopping or rebinding any application. Use `fleet rollout` for registered remote managed backends and `backend upgrade <sealed-runtime>/packages/0/dist` for the local supervisor. Keep framework bundles still referenced by applications.

## Checksum metadata on existing migrations

Adding a declared checksum to an existing procedural migration does not change
its schema operation. The shared compatibility check used by cold selection and
backend replacement admits this change only when all migration files and
folders are unchanged and the registry's executable syntax is identical after
removing those literal checksum properties. Names, order, imported `up`/`down`
bindings and SQL migration references must remain unchanged. Dynamic entries,
changed bodies, hooks, extra statements and replacement of an already declared
checksum are refused. Inspection parses the registries without importing them or
executing migration code.

An older guardian still compares the complete registry byte for byte. The
candidate CLI explains this limitation before draining anything. Use the
[existing guardian transition](#updating-the-guardian) to install a guardian with
`migrationChecksumMetadata: 1`, retaining the selected backend, execution owner
and application processes. Then repeat `backend upgrade --check` and the
reviewed backend transition. This does not require an owner replacement; other
owner capability changes remain a separate maintenance operation.

No migration-history rows are rewritten or adopted by this compatibility check.
Historical null checksums stay unknown, including their original execution times.
Rollback to an older registry may remove checksum declarations, but never removes
recorded checksums from the database. That older runtime may be unable to verify
those declarations; it must not be reported as fresh integrity proof. Changed
SQL/schema still requires the separate schema plan below.

## Compatible schema expansion

Schema changes and backend replacement are separate operations. `omnitron backend schema prepare` reads the running master’s database and saves a private plan; `omnitron backend schema apply` applies or resumes that plan under the same Kysera advisory lock as normal Omnitron migrations.

If the active launcher predates these commands, invoke the verified candidate directly: `<candidate>/node <candidate>/packages/0/dist/cli/omnitron.js backend ...`. Installing an artifact alone does not select it or update the running launcher. After successful backend activation, the stable launcher uses the selected release.

Before preparing, activate a compatibility bridge that supports schema expansion protocol 1 while retaining the existing migration registry. For independent services, update the guardian to a release that understands schema plans. Both steps preserve the execution owner and application processes. A legacy backend without the compatibility protocol is refused.

```bash
omnitron backend schema prepare /absolute/releases/omnitron-next --json
omnitron backend schema apply /absolute/instance/runtime/schema-transition-ID.json --json
omnitron backend upgrade /absolute/releases/omnitron-next/packages/0/dist \
  --schema-plan /absolute/instance/runtime/schema-transition-ID.json --check
omnitron backend upgrade /absolute/releases/omnitron-next/packages/0/dist \
  --schema-plan /absolute/instance/runtime/schema-transition-ID.json
```

Preparation installs the sealed candidate without selecting or running it as the backend. The plan identifies both complete runtime hashes, the instance, the actual PostgreSQL database and schema identity, unchanged migration bodies, the ordered registry, exact new checksums, current history and retained owner. It contains no database password. Only migrations explicitly declared as backward-compatible expansions in the candidate’s reviewed policy qualify. A new filename alone is insufficient. The current policy permits adding replication receipt outcomes with an `unverified` default, preserving old writers and readers.

Apply verifies the live instance and selection again. It durably publishes the exact compatibility receipt **before** executing DDL, then checks the history again on Kysera’s locked connection. Database lock acquisition is bounded to five seconds per lock and individual statements to sixty seconds; a blocked database produces an error, not an unbounded schema wait. A coordinator interruption can leave an approved plan: rerun the same `apply` command. Committed additions are recognized by their recorded checksums and are not replayed. Unrelated history changes, drift, missing definitions or a different database stop the operation.

The daemon journal now stores new content checksums through Kysera’s nullable checksum column. Existing names, timestamps and unknown legacy checksums remain unchanged; this does not attest old source retroactively. Read-only inspection never creates the column. During migration setup, Kysera adds it while holding its existing lock. The deployed journal and lock identity are retained.

A compatibility runtime accepts future migration history only when a private receipt matches its own immutable artifact, original migration bodies, database identity and every observed future checksum. This also supports a cold restart after a migration commits but the coordinator dies before marking the plan complete. Unknown or checksum-less future rows are refused. A compatibility runtime cannot roll back migrations it does not own.

To return to the retained bridge, use `backend upgrade <bridge-dist> --schema-plan <same-plan>`. The schema stays expanded. Both upgrade and rollback revalidate the applied plan and database; `--schema-plan` is not a general compatibility override. Keep its plan, receipt, receipt index and both sealed releases for as long as rollback remains required.

After an application-level rollback, the additive schema may already be committed.
For a corrected candidate with **identical migration definitions**, prepare a new
binding from the retained, applied plan:

```bash
omnitron backend schema prepare /sealed/corrected-runtime --from-plan /private/applied-plan.json
omnitron backend schema apply /private/new-plan.json
```

Preparation verifies the original sealed runtimes, database identity, registry
order, migration bodies, checksums and complete execution history. It retains the
original baseline; legacy unknown checksums remain unknown. Applying the new
binding takes the same migration lock and performs no DDL when the reviewed
expansion is already complete. An unfinished plan must be resumed first; a new
migration, changed history or foreign rollback runtime is refused. The original
plan and receipt remain available for audit and rollback.

This workflow currently covers master PostgreSQL expansions. Local node SQLite transitions, destructive schema changes, order changes and edits to deployed migration bodies need a separate rollout. No public Console endpoint can execute schema plans or local artifacts.

## Updating the guardian

`omnitron backend guardian` manages the guardian independently of the backend selection and execution owner. For supported user services and qualified root-owned Linux system services, `omnitron backend guardian prepare` saves a plan and `omnitron backend guardian apply` applies or resumes it:

```sh
omnitron --json backend guardian prepare /absolute/releases/omnitron-next
omnitron --json backend guardian apply /absolute/instance/runtime/supervisor-transition-ID.json --check
```

Use a management runtime that provides `--check`. It reads the prepared plan and
rechecks the selected backend, owner, inventory, pending operations, native unit
and Linux stop boundary without starting the transition. Review `phase: prepared`
and `checked: true` before approval. A completed plan returns its earlier receipt
with `checked: false`, which is not a fresh readiness check. Apply performs the
checks again under the installation lease.

After reviewing the current evidence, apply the same plan separately.

```sh
omnitron --json backend guardian apply /absolute/instance/runtime/supervisor-transition-ID.json
```

If the installed launcher predates these commands, invoke the candidate's `node` executable and `packages/0/dist/cli/omnitron.js` directly. Preparation verifies and installs the sealed management runtime, records the loaded unit file and current owner/application identities, and saves a private plan. It does not stop services. The record contains the original supervision configuration and unit for recovery; do not publish its contents.

Apply drains and checkpoints the backend, records an activation intent, stops only the guardian's native OS job, atomically replaces its unit and private configuration, then starts the new guardian. The existing backend release is restored under that guardian with a fresh controller epoch. The owner and application processes remain in their separate job. CLI and Console requests pause while backend control is restored.

A pool can recycle a worker while a plan awaits review. If the read-only check
refuses a changed inventory before retirement, prepare a fresh plan and review its
new PID and inventory evidence. Preserve the old receipt. A change after the
transition started requires inspection of its original phase and checkpoint.

The plan is resumable: run the same `apply` command after a coordinator interruption. It refuses changed units, a different owner, changed application inventory or a conflicting activation. A completed plan returns its original receipt without starting another process; use `omnitron backend status` for current state. To return to a previous guardian, prepare a new plan for its retained sealed runtime. No database migration is performed, and backend candidate compatibility checks remain in effect.

Guardian replacement supports user launchd/systemd services and root-owned Linux
system services with a loaded, independently stoppable unit. For system scope,
preparation checks the installed unit, its executable, stop propagation, hooks
and drop-ins. Apply rechecks these properties and preserves the unit's account,
environment and working directory. Prepare and review a separate guardian plan
for installations that predate the fenced fleet handoff. Updating the execution owner's runtime still requires an application maintenance window.

On Linux, the complete guardian cgroup must contain only the recorded guardian
and backend, with no occupied descendants. A refusal reports the relative group
and expected/observed PIDs separately from frozen or ambiguous group state.
A health inspection can briefly start another process in the root group.
During preparation, Omnitron can wait up to 1.5 seconds between full inspections
when both original processes are present. It verifies the same process lifetimes
and accepts the plan only after the extra process leaves. If the refusal persists,
inspect the reported PIDs. A missing process, an occupied descendant or an
ambiguous group state fails inspection. Review and apply check the current scope
immediately, including the check before stopping the unit.

The kernel defines `cgroup.procs` as an unordered process set and can repeat a PID
during a read. The reader normalizes ordering and repeated entries while retaining
strict rejection of additional or missing processes. Process lifetimes, executable
identities and cgroup membership are independently checked twice; normalization
does not waive those checks. See the [Linux cgroup v2 interface](https://docs.kernel.org/admin-guide/cgroup-v2.html#core-interface-files).

The recovery checkpoint monitor retries a busy snapshot once after 250 ms. This
avoids repeatedly sampling a short metrics drain at the same phase of a shared
five-second period. A busy owner is never certified: controller generation,
in-flight operations and ownership revision must still agree. Persistently busy
state remains `pending`, and stopping the monitor cancels the scheduled retry.

## Owner shutdown boundaries

An owner update is a separate maintenance operation from a backend handoff.
Drain applications through the private owner control service, verify their PIDs
have exited, request graceful owner shutdown, and wait for its durable clean
receipt before unloading or replacing the native service. A timeout is a failed
transition; do not start a second owner or continue to the next host.

New systemd owner units use `SendSIGKILL=no` while retaining their cgroup and
`Restart=no`. A stop timeout leaves unresolved processes for explicit recovery;
it does not authorize a forced kill. Existing installed units retain their old
settings until a reviewed unit update. launchd retains its finite OS timeout:
complete the private graceful shutdown before unloading its job, rather than
setting an infinite system shutdown timeout.

The owner logs shutdown stages (`watchers`, `applications`, `processes`, `state`,
`logs`, `rpc`, `storage`, `complete`). Application-stop errors do not skip the final
process-manager termination attempt. Any failed application stop, manager drain,
termination or persistence step prevents a clean receipt; repeated shutdown calls
retain that failed result. Native PID lifetimes and empty process groups must still
be verified independently. Public application `stopAll` leaves the manager usable;
only the owner/lifecycle performs its terminal shutdown. Preserve an unclean receipt and the native journal. Recovery requires
proof about the recorded owner lifetime, stopped application inventory and empty
OS process groups; changing a receipt alone is never proof. Retain that evidence
and the original receipt when recording an explicit reconciliation.

Explicit crash reconciliation supports launchd on macOS and systemd system
services on Linux. The CLI seals the native observation before a separately
approved drain, then verifies it again before issuing a one-use acknowledgment.
Preparation requires all recorded previous owner identities to be proven gone
and no live owner socket. A surviving backend/guardian and its socket remain in
the sealed inventory; they do not prevent preparing a reviewable plan. They must
be stopped, together with all recorded application processes and sockets, before
acknowledgment. Preparation never authorizes a drain or starts a new owner.
The replacement owner checks the same evidence under an inherited-descriptor
kernel lock (`lockf` on macOS, `flock` on Linux). Helper exit leaves the lock held
by Node; process death releases it without stale lock-file deletion. The original
unclean receipt stays intact, and a durable consumption record prevents replay
across interrupted acknowledgment publication.

macOS evidence binds the launchd domain and boot-session UUID, PID birth identities,
observed descendants, retained process groups and execution-directory Unix sockets.
It includes non-Node workers. The host and process-lifetime fences share the stable
`kern.bootsessionuuid`; changing legacy `kern.boottime` text is never evidence of a
reboot. A dirty legacy receipt is preserved and can be bound to the current UUID
only by explicit sealed reconciliation. Clean legacy shutdown receipts remain
valid for ordinary startup; log assembly retains incomparable live legacy writers.
Process births are collected in UTC through one shared identity contract for log
recovery, native owner evidence and rollout coordination. A legacy local-time
identity remains ambiguous while its PID is alive; changing the observer timezone
cannot prove process death or release an update fence. New macOS recovery plans
use protocol 2, so earlier recovery tools refuse them instead of misinterpreting
the UTC identity. Use the current recovery tool and owner for such a plan; a
legacy owner requires a separately reviewed compatible recovery procedure.
Rollout journals retain a null legacy birth field alongside the canonical lifetime
so older backends conservatively retain a live writer during rollback.
Unlike Linux cgroups, process groups cannot reveal
an unrecorded process that detached and reparented before observation. Supply the
complete recorded cohort and independently audit the drain; incomplete native
observations refuse recovery. Recovery never signals processes or adopts their
handles. See the [CLI recovery procedure](/docs/omnitron/cli#recover-after-an-execution-owner-crash).

## Changing owner log routing

A launchd unit generated by current Omnitron retains structured stdout together
with stderr; systemd uses the native journal. See
[owner and supervisor journals](/docs/omnitron/observability#owner-and-supervisor-journals)
for paths and retention boundaries. Previously generated launchd units may still
contain `StandardOutPath=/dev/null`.

For a logging-only correction, retain the exact owner runtime, Node executable,
arguments, environment and restart policy. Back up the installed unit and change
only its stdout destination. Prepare the rollback before unloading the job:
changing a plist on disk alone does not repair the live owner's stdout.
Stop the backend and owner gracefully, verify the clean owner shutdown receipt,
reload the corrected job, then restore the backend and the exact application
inventory. Confirm new JSON records and stderr reach the destination. If startup
or application health fails, stop the candidate jobs gracefully and reload the
saved unit with the same pinned runtime. Preserve databases and experiment data;
do not restore a stale database snapshot over new writes. An uncertain owner
shutdown requires reconciliation, not an automatic second owner.

## Long-lived RPC clients

The backend also reconnects to the resident execution owner after a broken private socket. Every connection rechecks socket ownership and permissions, then verifies the owner identity before sending retained event cursors, acknowledgments or mutations. A different owner is never adopted automatically. Observation polling uses its own transport with a four-second request deadline; a slow monitoring request cannot accumulate abandoned polls or interrupt a long-running application operation. An acknowledgment failure raises a separate diagnostic from an observation failure. Both resolve after the corresponding operation succeeds.

Concurrent observation requests share the same pending operation. Activating replay waits for that operation, including persistence and acknowledgment, instead of returning early because a periodic poll is already running. A failed write or acknowledgment still leaves records in the owner journal and raises the delivery diagnostic; completion of an observation alone is not proof of durable delivery.

The clients returned by `createDaemonClient()` and the fleet TCP client reconnect on the next call after a backend disconnect. Service handles obtained with `client.service(name)` remain usable: each call resolves the service against the current connection. Netron still owns discovery, serialization, remote properties and stream transport.

Repeated or concurrent `Netron.connect(address)` calls share one persistent peer on that instance. An explicit replacement cancels an obsolete scheduled reconnect. Separate principals or independently owned connections require separate Netron instances with unique IDs.

Binary transport handshakes have a bounded lifetime. Disconnect rejects an unfinished connection immediately; stopping Netron cancels pending handshakes and reconnect timers. A late disconnect from a previous connection cannot remove a replacement peer with the same identity.

An invocation already sent when the connection fails is rejected immediately with `TransportLostError`, including its `pendingPacketId`. This is an unknown outcome, not a request timeout. **It is never replayed automatically:** a missing response does not prove that a mutation was not committed. Inspect the operation ID or current state before deciding whether to dispatch again. Live streams and references owned by the old connection must be reopened by their consumer.

During an orderly handoff, new requests and tasks receive an explicit `SHUTTING_DOWN` refusal while admitted work drains. A forced disconnect can still leave a reply unavailable. Persisted log and metric delivery is a separate protocol: journals/outboxes retain the original IDs until a database commit is confirmed, so lost acknowledgments can be replayed safely. See [capture, persistence and replay](/docs/omnitron/observability#capture-persistence-and-replay). Generic RPC calls, live streams and arbitrary event handlers do not acquire durable delivery from a transport reconnect.

Update long-running consumers to this client before relying on backend replacement. Updating the backend cannot replace client code already loaded in another process; older consumers that retain a raw service proxy need a separate controlled restart with the updated dependency. Pin that dependency independently of the backend, especially for applications with registered runtime integrity checks.

## Short-lived socket probes

Titan PM checks old Unix socket files before considering removal. A live connection remains open until the first server greeting, EOF or a bounded 250 ms deadline; only an explicit `ECONNREFUSED` on an old, unowned path permits removal. This avoids closing immediately before a retained application sends its greeting during backend startup.

Netron also cancels an unsent greeting when the connection closes and observes the write result. An expected pre-handshake close is a debug event; a write failure on an open connection is an error with `connectionId`, transport and `phase: handshake`. Do not suppress all transport write errors: a failed application RPC requires its own delivery assessment.

## Interrupted operation

Check the original command's error and the private execution directory before taking action. Never print or publish checkpoint contents: they include resolved configuration and environment variables.

| State                                                                              | Action                                                                                                         |
| ---------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------- |
| Prepared, original backend has not retired                                         | `omnitron backend abort /absolute/private/handoff-ID.json` resumes it                                          |
| Backend retired or candidate failed; no backend is alive                           | `omnitron backend recover /absolute/private/handoff-ID.json` restores the previous artifact with a fresh epoch |
| Successor is ready, but selection publication was interrupted                      | `backend recover` validates its artifact, owner, epoch and inventory, then acknowledges that existing process  |
| Backend marker belongs to an unrelated live process or the original active backend | Recovery refuses takeover; inspect it or abort a preparation that has not retired                              |
| Owner died, inventory changed or checkpoint is stale                               | Automatic recovery is refused; reconcile actual application state before planning maintenance                  |
| Journal commit is uncertain                                                        | Recovery reads the durable grant before preparing another epoch                                                |

Recovery checkpoints describe a particular replacement, not a continuously updated backup of project state. They must not be replayed after unrelated configuration or application changes.

## Diagnostics and access

`omnitron status --json` includes `execution`; `omnitron backend status --json` reports the owner PID/identity, controller epoch, phase, observation age and blockers. A missing owner connection makes application state unknown. The backend does not infer that applications stopped from a control-channel failure. `execution.connection` reports `connected`, `unavailable` or `owner-changed`, the last successful observation timestamp and a concrete error. An observation older than five seconds is unavailable even before a failed request returns. `omnitron backend status` uses the independent control-status endpoint if application inventory cannot be read. Console Settings → General shows the same connection information and independent-supervision guardian PID; its status bar labels retained application counts as stale and checks backend reachability independently.

The owner's bounded live observation buffer retains 10,000 events. It is not the durable log archive. Updated owner/backend pairs capture full application records in a private SQLite journal, then confirm exact IDs and content only after database commit and file projection. `capture: legacy-owner` identifies the older volatile path, whose live log messages can be truncated to 4,096 characters; a backend-only update cannot repair it. Unacknowledged live observations may replay after replacement, and buffer overflow remains a visible diagnostic. Use [ingestion status](/docs/omnitron/observability#interpreting-diagnostics) and replication receipts to assess durable delivery independently of the live buffer.

Human viewer, operator and administrator roles, plus the explicit mesh observer identity, can read backend status. Artifact activation requires the local administrator Unix socket. HTTP administrators cannot obtain that private control address. Netron enforces the method policy for the caller’s identity. The local CLI uses the OS-authorized administrator identity on the Unix socket. Console uses the signed-in account on HTTP and WebSocket. Executable artifact paths are accepted only through the local administrative interface.

Local application topology stays in the owner. Public backend transports reconnect across an update. Direct external consumers of application services exposed on the daemon's former Netron transport are not automatically forwarded to the owner; validate those integrations before enabling resident execution.

`OMNITRON_HOME=/absolute/instance` selects an isolated daemon state directory for tests or a separately configured instance. Console containers have an instance-specific name and ownership label; the default instance retains `omnitron-nginx`. A legacy container must prove ownership through its configuration mount before it can be replaced or stopped. Shutdown of an instance without Console enabled never removes a Console container. Application paths and stack infrastructure are not remapped; distinct listeners and infrastructure configuration are still required. Do not change the shell's `HOME` to isolate a daemon.

## Upgrade a fleet of managed backends

A standalone controller can distribute and activate an Omnitron backend on up to
128 registered remote Linux servers with independent systemd supervision. All
targets must have a pinned SSH host key, a healthy execution owner and a guardian
advertising `kernelHandoff@1`. The controller updates servers in the requested
order and verifies each handoff before contacting the next server for activation.
Application processes stay with their existing owner.

Fleet planning runs the candidate's ordinary `backend upgrade --check` on each
node. CLI, Console and a local upgrade therefore use the same owner contracts,
attested database scope and migration compatibility rules. A standalone worker
checks its SQLite registry; a controller or enrolled worker also checks the
PostgreSQL registry. Compatible checksum metadata requires a guardian that can
verify it. Actual schema changes still require the separate database transition;
the fleet does not apply migrations or bypass a guardian refusal. The supervisor
revalidates the candidate and live identities when applying the reviewed plan.

For an older guardian, qualify its transition on native Linux before the maintenance
window. Run the repository fixture on an isolated native host or use the
**Qualify a managed backend handoff** workflow with the full serving and candidate
commits and exact target Node version. GitHub and npm publication are not required
for qualification from verified local sources. The result must identify the exact
artifacts and runtime roles being tested; a container-only test does not qualify
native service supervision. The test starts an application, replaces the guardian,
verifies owner identity and exercises a backend update and rollback while checking
application requests and logs.
The full native scenario upgrades the guardian first, then verifies backend replacement and rollback under that guardian, crash recovery, and guardian rollback with an interrupted coordinator. The separate `--backend-only` scenario deliberately retains the original guardian and qualifies only that narrower operation.

The same native fixture also qualifies an execution-owner replacement with
`--owner-roundtrip`. It verifies that the reviewed footprint contains exactly its
private counter application and no owned containers, then uses the ordinary
digest-bound stack stop. The same operation works with the retained old backend
and the candidate; this fixture does not qualify the separate infrastructure
retention extension. It confirms the old owner’s clean shutdown, selects the candidate
through the public CLI, and restarts unique test OS services. The stack must stay
stopped until an explicit start. The fixture repeats the sequence for rollback,
compares the existing migration journal exactly, and checks that every recorded
test process has exited. The producer journal is compared with durable SQLite
log rows across both owner changes: timestamps and multiline text must match,
with no losses or duplicates. Each private Linux owner/guardian service is bounded
to 2 GiB of memory, one CPU and 128 tasks, including its descendants.
This scenario requires an application pause in a real
installation; a successful backend handoff does not qualify owner replacement.

The execution owner, guardian and serving backend can use three different
runtimes. Observe each role independently and qualify the exact combination with
`--initial-guardian` and `--initial-backend`; both default to the owner artifact
when omitted. Rollback restores each original role. For example, from the
repository root:

```sh
node apps/omnitron/test/runtime/managed-backend.mjs \
  /builds/omnitron-handoff-old-owner/packages/0/dist \
  /builds/omnitron-handoff-candidate/packages/0/dist \
  --owner-roundtrip \
  --initial-guardian /builds/omnitron-handoff-serving-guardian/packages/0/dist \
  --initial-backend /builds/omnitron-handoff-serving-backend/packages/0/dist
```

To exercise a development stack and a pinned application release through the
same owner replacement, add `--application-sdk <installed-package>` and
`--application-sdk-commit <full-sha>` to the command. Take the SHA from the
reviewed application release. Before creating services, the fixture verifies the
installed package name, clean build stamp and commit. The result records the
application framework commit alongside the execution owner's commit and artifact
hash. This supports an application SDK released independently of the controller.

Use sealed disposable artifacts built for the host and its exact Node version.
The fixture uses private application data and unique launchd/systemd jobs. Linux
requires permission to create its isolated system services. The native GitHub
qualification includes owner replacement and rollback with the selected previous
and candidate versions; a separately versioned serving owner still needs its
exact role combination checked before activation.

For a project that needs a MASTER controller, use the same fixture with
`--master-scenario /absolute/scenario.mjs --fixture-root /absolute/new/root --master-config /absolute/private-config.json`. This is a separate mode from
`--owner-roundtrip` and `--backend-only`. The JSON file must be owned, regular,
0600 and contain explicit `port` and `httpPort` values. Prepare a separate
loopback PostgreSQL database and pass its URL as `OMNITRON_DATABASE_URL`.
The scenario exports `managedMasterScenario(context)`, uses `context.createClient()`
for public services, registers its child PIDs with `context.track(...)`, and returns
`{ success: true }` only after its assertions pass. The common fixture owns the
native owner/guardian lifecycle and cleanup; the scenario must not launch another
anonymous daemon or duplicate migrations. External database/store fixtures remain
the caller's responsibility. An explicit Unix `DOCKER_HOST` is forwarded to the
native runtime with competing Docker context/TLS selectors cleared.

Fixture OS unit files are created exclusively and tracked by identity, contents
and permissions. A checked guardian plan authorizes only its exact replacement;
cleanup refuses a substituted file. This protects unrelated services even when
setup or the transition is interrupted. These are isolated qualification tools,
not an alternative production activation path.

A failed availability probe records its transition phase, separate HTTP and owner-RPC results and durations, returned process identities, and the observer’s event-loop delay. Inspect `availability-probe-failed` in the run output and `probe-failures.jsonl` in the retained fixture directory. The application also reports slow writes of the fixture’s expected-log evidence. `availability-final` and the retained `availability-summary.json` keep the probe counts even when a later phase fails. The interrupted coordinator keeps its stdout, stderr and terminal error in `coordinator-result.json`; its 180-second checkpoint budget includes full artifact verification. A child failure is reported immediately rather than retried as readiness. HTTP and RPC probe deadlines remain unchanged. An availability failure still rejects qualification even when the lifecycle transitions succeed; complete log retention is a separate check.
Use the [guardian transition procedure](#updating-the-guardian) for the
one-time installation. Fleet handoff starts only after the guardian supports the
protocol. The first management runtime must be installed through a separately
qualified native path, rather than asking an older guardian to perform an
unsupported fleet handoff.

### Publish the batch to the controller

Build and qualify the same clean Git commit for each distinct Linux architecture
and exact Node version in the fleet. Import the sealed directories under the
controller's daemon account. A release pipeline can perform this step once per
variant. Several servers can reuse one build.

```sh
omnitron fleet rollout import /builds/linux-x64-node22/runtime /builds/linux-x64-node24/runtime
omnitron --json fleet rollout builds
```

The private catalog records the source commit, target platform and Node version,
runtime SHA-256, compressed archive SHA-256 and byte count. A repeated import of
the same build is safe. A different runtime for the same commit and target
requires resolving the build discrepancy. The publisher can admit foreign
architectures without executing the contained Node binary. Only the local
publisher can supply filesystem paths.

The receiving node validates archive entries before extraction and restores each validated regular file's permissions before verifying the sealed runtime hash. Executable bits remain identical even when the service uses `umask 077`; the receiver does not change the process-wide mask. Special mode bits, group/other-writable entries, links outside the validated archive structure and ownership changes are refused. Distribution neither changes the selected runtime nor starts a backend.

### Distribute, review and activate

For a canary followed by production, substitute the actual 40-character commit
and registered node names. With no node arguments the controller selects all
registered remote nodes in name order. The local controller has its own handoff
procedure.

```sh
omnitron --json fleet rollout plan <commit> test-node production-node --stage --wait
omnitron --json fleet rollout apply <plan-id> --wait
```

For a qualified batch in a scheduled window, distribution, qualification and
activation can be requested from one command.

```sh
omnitron --json fleet rollout plan <commit> test-node production-node --stage --apply --wait
```

With `--stage`, the controller matches each serving runtime to an exact catalog
variant, transfers it over the enrolled SSH connection and asks the receiving
installation to verify and store it. Staging preserves the active selection.
Omit `--stage` to inspect builds already installed through
`omnitron backend install <runtime>`. A single blocked target prevents activation
of the entire plan. A successful plan records the exact rollback artifact,
candidate, current selection revision, backend PID, owner and application inventory.

In Console, open **Nodes → Managed backend updates → Plan update**. Choose a catalog
commit and select the servers in canary order, or explicitly choose **All
registered remote nodes**. An empty selection does not authorize the whole fleet.
Enable **Stage from controller catalog** if distribution is required. Review the
per-server results and choose **Apply reviewed plan**. The Nodes URL's `?rollout=<plan-id>` and `fleet rollout status <plan-id>` refer to the same persisted operation. CLI and Console show the overall
batch state and each target's outcome. If preparation is blocked, inspect the affected target in Console or its `data.targets[].reason` in `omnitron --json fleet rollout status <plan-id>`. The reason describes the failing check. For example, if the guardian reports an operation in progress, inspect that operation before preparing a fresh plan. The original receipt remains available under its ID.

The advanced workspace source bundle panel
reports a separate legacy operation.
After each successful handoff, the controller checks application readiness and
owner identity and continues to the next server. If your acceptance procedure
also requires a replication watermark to advance, a public transaction to
succeed or an operator to inspect the result, use one plan for the canary.
Complete that check before submitting a second plan for the remaining servers.
Administrative sessions can prepare, start and reconcile a rollout. Read access
lets operators and viewers follow progress. Each server operation rechecks the
initiating administrator's authority.

A healthy pool rotation can change a PID or inventory digest after qualification.
If dispatch has not started, create a new plan and review its fresh identities.
Keep the original receipt for audit. A stale refusal is not evidence that an
activation happened or that rollback ran. After dispatch, inspect the original
ID. Follow an operation that is still running and reconcile an uncertain outcome
before creating another plan.

### Reverse an accepted node

Reverse a successfully accepted, healthy target with a new plan ID. Name exactly
one node and use its previous full source commit. Its retained runtime is already
installed, so omit `--stage`.

```sh
omnitron --json fleet rollout plan <previous-full-commit> production-node --wait
```

Inspect the JSON before applying. Confirm exactly one ready target and unchanged
owner identity and PID. Its `previous` must match the accepted runtime. Its
`candidate` must match the full SHA-256 saved for that node in the forward receipt.
JSON exposes complete hashes.

```sh
omnitron --json fleet rollout apply <new-plan-id> --wait
```

If readiness has degraded, stop further updates and inspect selection, owner and
inventory. Reconcile an uncertain receipt first. Reconciliation observes the
outcome without reversing it. A refused reverse plan requires a separately
reviewed fenced recovery procedure.

### Follow a batch or recover an interrupted handoff

```sh
omnitron --json fleet rollout status <plan-id> --wait
omnitron --json fleet rollout reconcile <plan-id>
```

Planning returns an ID while qualification continues. With `--apply` the CLI
waits for qualification before submitting the reviewed identity. Use `--json`
for automation and `--request-id <uuid>` when a caller needs idempotent planning.
The same ID binds the requested commit, node selection and staging choice.
Closing a client after submission leaves accepted work on the controller.

On activation failure the controller stops dispatching later nodes and records
the observed outcome. Before dispatch, a refusal leaves the incumbent in place.
A `failed` target can describe an unchanged original runtime, PID and selection
revision. A `rolled-back` target requires evidence of activation followed by return
to the pinned previous runtime. Merely seeing the previous artifact does not
prove a rollback.

A confirmed candidate with failed readiness can be handed back when the guardian
still verifies ownership. Previously successful nodes retain their accepted
versions. A batch can therefore be `failed` while an earlier target is
`succeeded` and later targets were never dispatched. CLI and Console retain these
per-target outcomes, including the authoritative reconciliation result while a
follow-up read is unavailable.

A post-dispatch `uncertain` receipt blocks a new rollout until reconciliation
checks live evidence. Reconciliation observes the node without replaying the
handoff or repairing an unhealthy application. If identity cannot be proven, the
operation stays uncertain. After reviewing a terminal result, create a new plan
for any remaining nodes.

Elected multi-controller clusters require recipient-side fencing across leaders;
this workflow refuses dispatch in that configuration. Upgrading the execution
owner, changing Node versions or applying schema changes each requires a
separately qualified transition. The local Console container follows its
controller's publication procedure.

### Service unit file preservation

Systemd recovery preserves the reviewed service file's owner and permission bits,
including under a restrictive process umask. It writes an exclusive temporary
regular file, flushes it, rechecks the original identity and content, and then
atomically replaces the file and flushes its directory. A symlink at either path
or a concurrent change refuses the replacement rather than following the link.

---

---
id: "omnitron/best-practices"
title: "Production best practices"
description: "Patterns proven in real multi-app Omnitron deployments."
---

# Production best practices

A working multi-app Omnitron ecosystem looks remarkably different
from a "hello-world" `defineSystem`. This page collects the
patterns that emerge once you run several apps together — shared
authentication, ordered startup, custom infrastructure, graceful
shutdown ordering, gateway integration, and the half-dozen
quality-of-life decisions that make the difference between "it
boots" and "it stays up".

Every example uses generic placeholder names (`auth`, `ledger`,
`media`, `realtime`, `analytics`). Substitute the names that fit
your domain.

## The five-app fan-out

A common production shape: **one app owns identity**; **N
specialist apps delegate authentication back to it** via a
shared JWT. Each specialist owns its own data plane (database,
queue, blockchain daemon, object store, websockets) but trusts
the central auth.

```mermaid
flowchart LR
  Client[Browser / mobile]
  Gateway[OpenResty<br/>gateway]
  Auth[App: auth<br/>identity, sessions, RBAC]
  Ledger[App: ledger<br/>payments + bookkeeping]
  Media[App: media<br/>object storage + transforms]
  Realtime[App: realtime<br/>websockets + presence]
  Analytics[App: analytics<br/>market data + aggregates]
  Redis[(Redis<br/>session registry)]
  PG[(Postgres<br/>per-app schemas)]

  Client --> Gateway
  Gateway --> Auth
  Gateway --> Ledger
  Gateway --> Media
  Gateway --> Realtime
  Gateway --> Analytics

  Auth -- writes sessions --> Redis
  Ledger -- reads sessions --> Redis
  Media -- reads sessions --> Redis
  Realtime -- reads sessions --> Redis
  Analytics -- reads sessions --> Redis

  Auth --> PG
  Ledger --> PG
  Media --> PG
  Realtime --> PG
  Analytics --> PG
```

Properties of this shape:

- **One signing key**, one JWT format — auth issues; everyone
  else verifies.
- **Session revocation is instant** because every backend checks
  the Redis session registry on each call.
- **Each specialist scales independently** — only `media` needs
  more CPU when traffic spikes uploads.
- **The auth app is `critical: true`**; the others are usually not.

A minimal `omnitron.config.ts` for this shape:

```typescript
export default defineEcosystem({
  project: 'platform',
  infrastructure: {
    services: {
      gateway: { preset: 'openresty', config: { configDir: './infra/nginx' } },
    },
    // postgres / redis / minio are auto-provisioned from app requirements
  },
  apps: [
    { name: 'auth',      bootstrap: './apps/auth/dist/bootstrap.js',      critical: true,
      watch: { directory: './apps/auth' } },
    { name: 'media',     bootstrap: './apps/media/dist/bootstrap.js',     dependsOn: ['auth'],
      startupTimeout: 90_000,  watch: { directory: './apps/media' } },
    { name: 'analytics', bootstrap: './apps/analytics/dist/bootstrap.js',
      watch: { directory: './apps/analytics' } },
    { name: 'ledger',    bootstrap: './apps/ledger/dist/bootstrap.js',    dependsOn: ['auth'],
      startupTimeout: 120_000, watch: { directory: './apps/ledger' } },
    { name: 'realtime',  bootstrap: './apps/realtime/dist/bootstrap.js',  dependsOn: ['auth'],
      watch: { directory: './apps/realtime' } },
  ],
  supervision: {
    strategy: 'one_for_one',
    maxRestarts: 5,
    window: 60_000,
    backoff: { type: 'exponential', initial: 1_000, max: 30_000, factor: 2 },
  },
  monitoring: { healthCheck: { interval: 15_000, timeout: 5_000 },
                metrics: { interval: 5_000, retention: 3_600 } },
  logging:    { level: 'info', maxSize: '50mb', maxFiles: 10, compress: true },
});
```

## Shared authentication across apps

Use the [shared-session preset](/docs/titan/modules/shared-session) rather than
implementing token, session and role validation in each bootstrap. A shared
workspace adapter wires the application's issuer, role hierarchy and canonical
session authority. The authority owns the account schema and reads current grants
in a single consistent database snapshot. Other backends call it through Netron
with a service credential restricted to `sessions.resolve`.

```ts
import { createSharedSessionAuthManager }
  from '@omnitron-dev/titan-auth/preset/shared-session';
import { withAuthRLSContext, type AuthContext }
  from '@omnitron-dev/titan/netron/auth';

app.netron.configureAuth(createSharedSessionAuthManager({
  logger, jwtService,
  expectedIssuer: 'my-platform',
  sessionAuthority: lookupCurrentSessionAndGrants,
  roleExpander: applicationRoles.expandRoleSet,
}));

// Pass to the application's invocationWrapper option.
const invocationWrapper = (metadata: Map<string, unknown>, fn: () => Promise<unknown>) => {
  const auth = metadata.get('authContext') as AuthContext | undefined;
  // Anonymous calls receive an empty frame instead of inheriting outer authority.
  return withAuthRLSContext(auth ?? { userId: '', roles: [], permissions: [] }, fn);
};
```

Configure deadlines and lifecycle cleanup for the authority client. Deploy its
endpoint before consumers and provide redundant owners for high availability.
A failed authority lookup refuses protected traffic with 503; cached grants
must not turn an outage into a permission bypass. Signature caches remain
independent of live session checks.

### Token binding (`cnf.fp`)

With `cnfHmacSecret`, an access token can bind to the active refresh-token ID.
The canonical adapter returns that ID from the same snapshot as the session.
When it disappears or changes, the previous confirmation is refused. Registry
compatibility mode cannot provide this guarantee for legacy entries without a
refresh-token ID, and the preset never rewrites registry entries itself.

## Ordered startup with `dependsOn`

Specialist apps that need the auth app warm before they start:

```typescript
apps: [
  { name: 'auth',     bootstrap: './apps/auth/dist/bootstrap.js',     critical: true },
  { name: 'media',    bootstrap: './apps/media/dist/bootstrap.js',    dependsOn: ['auth'] },
  { name: 'ledger',   bootstrap: './apps/ledger/dist/bootstrap.js',   dependsOn: ['auth'] },
  { name: 'realtime', bootstrap: './apps/realtime/dist/bootstrap.js', dependsOn: ['auth'] },
]
```

The orchestrator waits for an app's `dependsOn[]` to all report
healthy before starting it. Shutdown reverses the order
automatically.

## Tuning `startupTimeout` per app

The default 30 s startup timeout is fine for most apps. Apps
that initialise heavy resources need more:

| App profile                                            | Typical `startupTimeout` |
| ------------------------------------------------------ | ------------------------ |
| Plain HTTP service                                     | 30 000 (default)         |
| Image processor warm-up + S3 connect                   | 90 000                   |
| Blockchain wallet RPC connection check + bootstrapping | 120 000                  |
| Hardware crypto provider warm-up                       | 300 000                  |

Set per app in `IEcosystemAppEntry.startupTimeout`. If an app
hits the timeout without reporting ready, the supervisor counts
it as a failed start and applies the restart policy.

## Worker processes within an app

`defineSystem` lets you fan out workers as separate processes
inside the same app:

```typescript
export default defineSystem({
  name: 'media',
  processes: [
    {
      name: 'http',
      module: './app.module.js',
      critical: true,
      transports: { http: { port: 3002 } },
      topology: { access: ['ImageTransformWorker'] },   // ← consumes sibling
    },
    {
      name: 'image-transform',
      module: './workers/image-transform.module.js',
      critical: false,            // worker pool — don't kill the app if it dies
      startupTimeout: 30_000,
      health: { enabled: true, interval: 30_000 },
      topology: { expose: true }, // ← exposes ImageTransformWorker @Service
    },
    {
      name: 'thumbnail-generator',
      module: './workers/thumbnail.module.js',
      critical: false,
      health: { enabled: false }, // burst worker — no need for health probe
    },
  ],
});
```

The pattern:

- **http** process exposes the public API; never blocks the event
  loop with CPU-bound work.
- **image-transform** process exposes `ImageTransformWorker` via
  `@Service`; http consumes it via `topology.access`.
- **thumbnail-generator** runs as a fire-and-forget worker — low
  criticality, no health probe overhead.

Result: CPU-bound work happens off the request event loop;
restarts of the worker pool don't take down the HTTP surface.

## Shutdown priority

When the daemon shuts down apps, lower `shutdown.priority`
numbers stop first; higher numbers stop later. Use this to drain
in dependency order:

```typescript
// apps/auth/src/bootstrap.ts
shutdown: { priority: 0, timeout: 10_000, drainConnections: true },

// apps/ledger/src/bootstrap.ts — should outlive auth on shutdown so
// in-flight transactions can complete before sessions become invalid
shutdown: { priority: 15, timeout: 10_000, drainConnections: true },

// apps/realtime/src/bootstrap.ts — must shut first to send
// disconnect frames to clients
shutdown: { priority: -10, timeout: 5_000, drainConnections: true },
```

Heuristic: an app that *depends on* another should shut **first**.

## Custom infrastructure with `networkMode` variants

When an app needs a service the presets don't cover (a search
engine, a blockchain daemon, a custom queue broker), declare it
in the app's `config/default.json` under `omnitron.infrastructure`:

```jsonc
{
  "omnitron": {
    "database": true,
    "redis":    true,
    "infrastructure": {
      "search": {
        "description": "Search engine for full-text indexing",
        "type": "daemon",
        "version": ">=8.10.0",
        "networkMode": "single-node",
        "ports": { "http": 9200, "transport": 9300 },
        "env": {
          "SEARCH_URL":      "http://${host}:${port:http}",
          "SEARCH_API_KEY":  "${secret:search_api_key}"
        },
        "secrets": {
          "search_api_key": { "secret": "search_api_key" }
        },
        "healthCheck": {
          "type": "command",
          "target": "curl -sf http://localhost:9200/_cluster/health",
          "interval": "30s", "timeout": "10s", "retries": 5, "startPeriod": "60s"
        },
        "startupTimeout": 120000,
        "docker": {
          "image": "elasticsearch:8.11.0",
          "environment": { "discovery.type": "single-node" },
          "resources": { "memory": "1G" },
          "variants": {
            "cluster": {
              "environment": { "cluster.name": "platform-search" },
              "resources":   { "memory": "4G", "memoryReservation": "2G" }
            }
          }
        },
        "bareMetal": {
          "systemdUnit": "elasticsearch",
          "dataDir": "/var/lib/elasticsearch",
          "user": "elasticsearch",
          "validateCommand": "elasticsearch --version"
        }
      }
    }
  }
}
```

What this delivers:

- Dev / test stacks get the Docker container automatically.
- Production stacks point at the bare-metal service via the
  `bareMetal` block + per-stack override.
- The app receives a literal `SEARCH_URL=http://localhost:9200`
  and `SEARCH_API_KEY=<decrypted>` at startup.
- `networkMode: 'cluster'` swaps the Docker config when a stack
  declares it.

## Redis: one client, many logical databases

Avoid running multiple Redis containers. One instance with
named clients per concern:

```typescript
// In your app:
const redisManager = await app.container.resolveAsync(REDIS_MANAGER);
const sessionRedis = redisManager.getClient('sessions');     // DB 0
const cacheRedis   = redisManager.getClient('cache');        // DB 1
const queueRedis   = redisManager.getClient('queue');        // DB 2
const ratelimitRedis = redisManager.getClient('ratelimit');  // DB 3
const locksRedis   = redisManager.getClient('locks');        // DB 4
```

Configure per-client DBs in app config:

```jsonc
{
  "redis": {
    "clients": {
      "sessions":  { "db": 0 },
      "cache":     { "db": 1 },
      "queue":     { "db": 2 },
      "ratelimit": { "db": 3 },
      "locks":     { "db": 4 }
    }
  }
}
```

`sessions` is intentionally shared across all backends in the
fan-out architecture — every app reads from DB 0 to validate
sessions.

→ Recommended DB split is also documented in
[titan-redis module-map](/docs/titan/modules/module-map#where-redis-sits).

## Per-app HTTP transport tuning

Tuning the HTTP transport per-process by use case:

```typescript
// Auth — short connections, fast turnaround
transports: { http: {
  port: 3001, host: '0.0.0.0', cors: true,
  requestTimeout: 30_000, keepAliveTimeout: 65_000, headersTimeout: 60_000,
}}

// Media — big uploads, long sessions
transports: { http: {
  port: 3002, host: '0.0.0.0', cors: true,
  requestTimeout: 300_000, keepAliveTimeout: 65_000, headersTimeout: 60_000,
  maxRequestSize: '10gb',
}}

// Realtime — HTTP for RPC + WebSocket for live
transports: {
  http:      { port: 3005, host: '0.0.0.0', cors: true,
               requestTimeout: 120_000, keepAliveTimeout: 65_000, headersTimeout: 70_000 },
  websocket: { port: 3006, host: '0.0.0.0', path: '/ws',
               keepAlive: { interval: 30_000, timeout: 10_000 } },
}
```

Headers-timeout > keep-alive-timeout > request-timeout is the
common ordering; setting them all explicitly is more reliable
than relying on Node defaults.

## Custom HTTP routes alongside RPC

Co-locate non-RPC endpoints (object serving, file streaming,
image transforms, webhooks) with the RPC surface:

```typescript
processes: [{
  name: 'http',
  module: './app.module.js',
  transports: { http: { port: 3002, host: '0.0.0.0' } },
  customRoutes: [
    {
      method: 'GET',
      pattern: '/render/image/*',
      handler: async (req: Request): Promise<Response | null> => {
        const url = new URL(req.url);
        if (!url.pathname.startsWith('/render/image/')) return null;
        // Resolve bucket from path; verify auth; transform; serve.
        // Use the SAME services available via Netron (objectService, transformService)
        // so policy is identical across the RPC + REST surfaces.
        // ...
      },
    },
    {
      method: 'POST',
      pattern: '/webhooks/payment-provider',
      handler: async (req) => verifyAndRoute(req),
    },
  ],
}]
```

**Security trap**: if your custom route bypasses a check that
the RPC surface enforces, you have two access tiers. The route
above must call the same `assertAccess('read')` gate the RPC
path uses. Don't skip auth in custom routes just because the
RPC layer "usually handles it".

## Hot-reloadable runtime settings

For settings that should change without restart (commission
rate, feature flags, business rules), store them in the
database and load on boot:

```typescript
hooks: {
  afterCreate: async (app) => {
    // ... auth wiring above ...

    // Apply persisted runtime settings
    const settingsService = await app.container.resolveAsync(RUNTIME_SETTINGS_SERVICE_TOKEN);
    if (settingsService && typeof settingsService.loadAndApply === 'function') {
      await settingsService.loadAndApply();
    }
  },
}
```

A separate admin RPC method calls `setAndReload(key, value)` to
update at runtime; subscribers in-process get notified via
`titan-events`.

This pattern is for **business logic settings**, not connection
strings — those still belong to the static config.

## Eager-loading heavy services

Some services (blockchain RPC clients, encryption providers,
machine-learning model loaders) take seconds to initialise.
Lazy resolution means the first request after startup hangs
waiting for them. Eager-load them in `afterCreate`:

```typescript
hooks: {
  afterCreate: async (app) => {
    // ... auth wiring ...

    // Eagerly resolve to fire @PostConstruct initializers
    try {
      logger.info('eagerly resolving wallet providers');
      await app.container.resolveAsync(WALLET_PROVIDER_A_TOKEN);
      await app.container.resolveAsync(WALLET_PROVIDER_B_TOKEN);
      logger.info('wallet providers warm');
    } catch (err) {
      logger.warn({ err }, 'could not eagerly resolve wallet providers');
    }
  },
}
```

Without this, the supervisor reports the app "ready" but the
first user request still pays the cold-start cost on a
critical path.

## Reverse-proxy with maintenance-mode flag in Redis

A gateway in front of every app that can be flipped to
maintenance mode without restart:

```nginx
# infra/nginx/nginx.conf
worker_processes auto;

env REDIS_HOST;
env REDIS_PORT;
env REDIS_PASSWORD;

http {
  lua_package_path "/etc/nginx/lua/?.lua;;";
  lua_shared_dict maintenance_cache 1m;

  upstream auth_backend     { server ${UPSTREAM_AUTH_HOST}:${UPSTREAM_AUTH_PORT}; }
  upstream media_backend    { server ${UPSTREAM_MEDIA_HOST}:${UPSTREAM_MEDIA_PORT}; }
  upstream ledger_backend   { server ${UPSTREAM_LEDGER_HOST}:${UPSTREAM_LEDGER_PORT}; }
  upstream realtime_backend { server ${UPSTREAM_REALTIME_HOST}:${UPSTREAM_REALTIME_PORT}; }
  upstream realtime_ws      { server ${UPSTREAM_REALTIME_WS_HOST}:${UPSTREAM_REALTIME_WS_PORT}; }

  server {
    listen 8080;
    access_by_lua_file /etc/nginx/lua/maintenance_check.lua;

    location /api/auth/     { proxy_pass http://auth_backend; }
    location /api/media/    { proxy_pass http://media_backend; }
    location /api/ledger/   { proxy_pass http://ledger_backend; }
    location /api/realtime/ { proxy_pass http://realtime_backend; }
    location /ws/           {
      proxy_pass http://realtime_ws;
      proxy_http_version 1.1;
      proxy_set_header Upgrade $http_upgrade;
      proxy_set_header Connection "upgrade";
    }
  }
}
```

`maintenance_check.lua` reads a flag from Redis and serves
`maintenance.html` if set. Operators flip the flag with one
Redis command — no daemon restart, no DNS change, no LB
reconfiguration.

```lua
-- infra/nginx/lua/maintenance_check.lua
local cache = ngx.shared.maintenance_cache
local cached = cache:get("flag")
if cached == nil then
  local redis = require("resty.redis")
  local r = redis:new()
  r:set_timeout(50)
  if r:connect(os.getenv("REDIS_HOST"), os.getenv("REDIS_PORT")) == 1 then
    if os.getenv("REDIS_PASSWORD") then r:auth(os.getenv("REDIS_PASSWORD")) end
    local v = r:get("platform:maintenance")
    cached = (v == "1") and "1" or "0"
    cache:set("flag", cached, 1)   -- 1s TTL; minimal Redis chatter
  else
    cached = "0"   -- fail-open
  end
end
if cached == "1" then
  ngx.status = 503
  ngx.header.content_type = "text/html"
  ngx.say(io.open("/etc/nginx/maintenance.html"):read("*all"))
  ngx.exit(503)
end
```

Configure the OpenResty gateway as a preset under
`infrastructure.services`:

```typescript
infrastructure: {
  services: {
    gateway: {
      preset: 'openresty',
      config: { configDir: './infra/nginx' },
      ports:  { http: 8080 },
    },
  },
}
```

## Tor hidden service for operator console

For sensitive operator surfaces (the webapp, an admin portal),
expose them over a `.onion` address — no DNS, no SNI, no public
TLS termination to manage:

```typescript
infrastructure: {
  services: {
    gateway: { preset: 'openresty', config: { configDir: './infra/nginx' } },
    tor: {
      preset: 'tor',
      config: {
        hiddenServices: [
          { name: 'webapp', virtualPort: 80, target: 'host.docker.internal:9800' },
          { name: 'admin',  virtualPort: 80, target: 'host.docker.internal:8080' },
        ],
      },
    },
  },
}
```

The `.onion` addresses are derived from keys in a persistent
volume. Print them after bootstrap with `omnitron tor`.

This is a control-plane convenience for distributed teams — not a
substitute for proper RBAC. Defense in depth.

## RLS context propagation

Every app in the fan-out has the same auth wrapper:

```typescript
auth: {
  jwt: { enabled: true, tokenCacheTtl: 60_000 },
  invocationWrapper: createRlsInvocationWrapper(),
}
```

The wrapper maps the Netron `authContext` into a kysera RLS
scope before each invocation; row-level policies in the database
see the right `app.user_id` / `app.is_system` / `app.tenant_id`
session variables.

Even the auth app uses this wrapper — its own queries against
`user_sessions` / `refresh_tokens` benefit from RLS just as much.

## Per-app config with env override

A two-source pattern works for almost every app:

```typescript
config: {
  sources: [
    { type: 'file', path: 'apps/media/config/default.json', optional: true },
    { type: 'env',  prefix: 'MEDIA_' },
  ],
  envPrefix: 'MEDIA_',
}
```

In `config/default.json`: defaults, business rules, declarative
infrastructure declarations. Env vars override — used for
secrets, per-environment toggles, and any value that must not
appear in committed files.

Prefix matters: `MEDIA_` for media, `LEDGER_` for ledger, etc.
Avoids cross-app env collisions.

## Restart policies — be sparing with `critical`

`critical: true` means "kill the whole daemon if this crashes
past `maxRestarts`". Reserve it for the **one** app whose
absence makes the rest useless:

```typescript
apps: [
  { name: 'auth',     bootstrap: '...', critical: true  },  // without auth, nothing works
  { name: 'media',    bootstrap: '...', critical: false },  // media down → uploads broken, rest OK
  { name: 'ledger',   bootstrap: '...', critical: false },
  { name: 'realtime', bootstrap: '...', critical: false },
  { name: 'analytics',bootstrap: '...', critical: false },
]
```

If you mark everything critical, any single bug brings the whole
daemon down — including the apps that were working fine.

## Workers: `critical: false` + health-check tuning

Worker subprocesses (`captcha-generator`, `notification-worker`,
`deposit-worker`) have different health-check needs:

| Worker type                               | `critical` | `health.enabled` | Reason                                                     |
| ----------------------------------------- | :--------: | :--------------: | ---------------------------------------------------------- |
| Burst / fire-and-forget                   |   `false`  |      `false`     | No external surface to probe; supervisor restart is enough |
| Steady consumer (queue worker)            |   `false`  |   `true` (60 s)  | Catch silent stalls; restart on failure                    |
| Critical pipeline (deposits, settlements) |   `false`  |   `true` (30 s)  | Tighter probes; faster recovery                            |

Health probes per worker run inside the worker process — set
`health.enabled: false` for workers without a credible health
signal to save the probe overhead.

## Monitoring + supervision baseline

The same supervision block fits \~90% of deployments:

```typescript
supervision: {
  strategy:    'one_for_one',
  maxRestarts: 5,
  window:      60_000,
  backoff:     { type: 'exponential', initial: 1_000, max: 30_000, factor: 2 },
},
monitoring: {
  healthCheck: { interval: 15_000, timeout: 5_000 },
  metrics:     { interval: 5_000,  retention: 3_600 },
},
logging: {
  level:    'info',
  maxSize:  '50mb',
  maxFiles: 10,
  compress: true,
},
```

Tune from here:

- **`maxRestarts: 5 / 60s`** is OK for most. Raise for noisier
  workloads, lower for crash-prone development.
- **`backoff.max: 30_000`** caps the restart delay at 30 s —
  past that, longer waits stop helping (the underlying issue
  isn't transient).
- **`healthCheck.interval: 15_000`** — every 15 s. Lower if you
  need faster failover; the cost is mostly RPC overhead.
- **`logging.maxSize: '50mb' × maxFiles: 10`** = 500 MB per app
  of log retention. Adjust for disk budget.

## Dev mode without surprises

For the dev experience:

```typescript
dev: {
  logLevel:   'debug',
  sourceMaps: true,
},
```

at the app level — picked up by `omnitron up` (which file-watches
dev stacks by default; pass `--no-watch` to disable). Plus
`watch: { directory: './apps/X' }` per app at the ecosystem
level for HMR.

Don't watch `node_modules`, `dist`, or any test-output directory
— the file watcher's default ignores cover the common ones, but
extend `watch.ignore` if needed.

## What this looks like at scale

Five apps + one gateway + Postgres + Redis + S3-compatible
storage + one custom blockchain daemon = roughly 12 supervised
processes on a single node. The daemon's footprint stays under
\~50 MB; each app brings its own \~50–150 MB depending on what it
loads. Operator memory:

| Memory budget | What runs                                                |
| ------------- | -------------------------------------------------------- |
| 1 GB          | Daemon + 3-4 light apps (dev VM)                         |
| 4 GB          | Daemon + 5 apps + Postgres + Redis (single-node dev box) |
| 8 GB          | All of the above + S3-compatible + blockchain daemons    |
| 16 GB+        | Production-like single-node test bed                     |

CPU: each app idles at near-zero; spikes track load. The
biggest CPU consumer is usually the gateway (Lua + nginx) under
heavy traffic.

## When this fan-out shape is wrong

- **Single tenant, single concern.** Don't split for the sake of
  splitting; one app with multiple `IProcessEntry` processes is
  often cleaner.
- **No shared identity domain.** If the auth domain genuinely
  differs across services, separate fully (separate daemons /
  databases / Redis instances).
- **Strict isolation requirement.** If "tenant A's data cannot
  ever sit next to tenant B's data" is a compliance requirement,
  per-tenant deployments may beat one shared deployment.
- **You need < 100ms cold deploy.** Multi-app boots take longer.
  Single binary may win.

## See also

- [Configuration](/docs/omnitron/configuration) — every field used above
- [Infrastructure](/docs/omnitron/infrastructure) — `OmnitronAppConfig` and `IServiceRequirement` deep dive
- [Cluster + Fleet](/docs/omnitron/cluster) — when you outgrow one node
- [Observability](/docs/omnitron/observability) — the supervision baseline
- [Architecture](/docs/omnitron/architecture) — where each piece sits

---

---
id: "omnitron/cli"
title: "CLI reference"
description: "Every command, every flag, every command group."
---

# CLI reference

`omnitron <command>` is the single binary. Management commands use Netron RPC
through `~/.omnitron/daemon.sock`. Local build, configuration and migration
commands execute in the CLI process; application migrations can run without a
running daemon.

Verified against `src/cli/omnitron.ts` and `src/commands/*`.

## Global options

| Option / Env                      | Effect                                                 |
| --------------------------------- | ------------------------------------------------------ |
| `--json` / `OMNITRON_OUTPUT=json` | Machine-readable JSON; suppresses spinners and styling |
| `--version`                       | Print CLI version                                      |
| `--help`                          | Per-command help                                       |

Commands that support `--json` return one `{ "ok": true, "data": ... }` result
on stdout; errors use `{ "ok": false, "error": ... }` on stderr. Machine output
is an explicit action contract. Unsupported commands reject the flag **before
loading or executing their action**, with exit `2`, code
`JSON_OUTPUT_UNSUPPORTED` and `outcome: "not-started"`; they do not perform the
operation and then complain about its output. This also applies to
`OMNITRON_OUTPUT=json`, nested commands and aliases.

A declared JSON command that returns without a result reports
`JSON_OUTPUT_CONTRACT_VIOLATION` and `outcome: "unknown"`. Its operation may have
completed: inspect state before retrying. Finite commands validate their output
and drain stdout and stderr before exiting, so large log exports remain complete
when piped into another process. Check both the JSON result and the exit status.

## Command index

| Group                                    | Commands                                                                                                                                                                                                       |
| ---------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Daemon lifecycle**                     | `up`, `down`, `ping`, `kill`                                                                                                                                                                                   |
| **Backend installation and replacement** | `backend status`, `backend install`, `backend installation`, `backend select`, `backend upgrade`, `backend abort`, `backend recover`, `backend owner-reconcile prepare`, `backend owner-reconcile acknowledge` |
| **Controller builds**                    | `release build --publish --detach`, `release builds [project]`, `release job <id> --follow`, `release stop-build <id>`                                                                                         |
| **Release admission**                    | `stack check-release <project> <stack> <id>`                                                                                                                                                                   |
| **Development SDK**                      | `sdk init`, `sdk dev`, `sdk status`, `sdk registry`                                                                                                                                                            |
| **Project**                              | `project add`, `project list`, `project remove`, `project scan`, `project topology`                                                                                                                            |
| **Topology**                             | `topology refresh`, `topology show`, `topology export`, `topology draft`, `topology plan`, `topology apply`                                                                                                    |
| **Stack**                                | `stack list`, `stack create`, `stack delete`, `stack status`, `stack progress`, `stack diagnose`, `stack prepare-release`, `stack start`, `stack stop`, `stack resume`, `stack runtime`                        |
| **App lifecycle**                        | `start`, `stop`, `restart`, `reload`                                                                                                                                                                           |
| **Information**                          | `list` (alias `ls`), `status`, `config`, `init`                                                                                                                                                                |
| **Monitoring**                           | `logs`, `monit`, `health`, `metrics`, `health-check`, `discover`                                                                                                                                               |
| **Scaling**                              | `scale`                                                                                                                                                                                                        |
| **Diagnostics**                          | `doctor`, `inspect`, `exec`, `env`                                                                                                                                                                             |
| **Database schema**                      | `migrate status`, `migrate verify`, `migrate up`, `migrate down`                                                                                                                                               |
| **Remote**                               | `remote add`, `remote remove`, `remote list`, `remote status`                                                                                                                                                  |
| **Fleet**                                | `fleet status`, `fleet health`, `fleet metrics`                                                                                                                                                                |
| **Cluster**                              | `cluster status`, `cluster step-down`                                                                                                                                                                          |
| **Secrets**                              | `secret set`, `secret get`, `secret list`, `secret delete`                                                                                                                                                     |
| **Deployment**                           | `deploy app`, `deploy build`, `rollback`                                                                                                                                                                       |
| **Infrastructure**                       | `infra up`, `infra down`, `infra status` (`ps`), `infra logs`, `infra psql`, `infra redis-cli`, `infra migrate`, `infra reset`                                                                                 |
| **Pipelines**                            | `pipeline list`, `pipeline save`, `pipeline run`, `pipeline status`, `pipeline cancel`                                                                                                                         |
| **Backup**                               | `backup create`, `backup list`, `backup restore`                                                                                                                                                               |
| **Kubernetes**                           | `k8s pods`, `k8s deploy scale`                                                                                                                                                                                 |
| **Nodes**                                | `node list`, `node add`, `node update`, `node remove`, `node check`, `node reconcile`, `node ssh-keys`                                                                                                         |
| **Access**                               | `access show`, `access users`, `access create`, `access role`, `access revoke`                                                                                                                                 |
| **SSH**                                  | `node ssh-key`, `node ssh`, `terminal list`, `terminal close`                                                                                                                                                  |
| **Webapp**                               | `webapp build`, `webapp start`, `webapp stop`, `webapp status`, `webapp open`                                                                                                                                  |
| **Tor**                                  | `tor`                                                                                                                                                                                                          |
| **Knowledge base**                       | `kb mcp`, `kb index`, `kb status`, `kb query`                                                                                                                                                                  |

## Topology workspace

`omnitron topology refresh <project> [stack]` explicitly captures bootstrap dependency
declarations in private SQLite without starting applications. It requires admin,
uses bounded separate reader processes and returns per-application failures while
retaining the last successful metadata. Graph reads remain available to viewers.
Refresh after application or dependency rebuilds; inspect source/capture times and
coverage warnings with `project topology` or `topology show`.

Use `omnitron topology refresh <project> <stack> --release <id>` to read the
declarations of a prepared candidate for an independently owned local stack.
The administrator can capture them while the previous release is running. See
[local release preparation](/docs/omnitron/releases#pin-a-local-stack).

`omnitron topology show <project> <stack>` reads configured, desired and observed
placement. `export` writes portable JSON; `draft <project> <stack> <file>` saves a
revision-protected draft; `plan` validates it; `apply <project> <stack> <plan-id>`
accepts that target. Acceptance does not change executable configuration or run a
migration. See [topology review, permissions and limits](/docs/omnitron/topology).

## Database migrations

`omnitron migrate [status|verify|up|down]` uses `omnitron.migrations.ts` by
default. `--config` selects a configuration; `--daemon` explicitly selects the
daemon database. `up --dry-run` previews, `up --to NAME` selects a prefix, and
`down --steps N` rolls back a suffix. `verify` exits with 2 when integrity is
unknown or mismatched. Direct `up` and `down` require a standalone installation. For a clustered controller schema change, use the reviewed backend schema workflow. See [migration configuration and recovery](/docs/omnitron/migrations).

## Backend replacement

`backend status --json` reports whether the running daemon supports resident
execution. `backend upgrade <sealed-dist> --check` validates a candidate and
rollback artifact. Without `--check` it replaces only a resident-mode backend.
`backend abort <checkpoint>` cancels preparation before retirement;
`backend recover <checkpoint>` refuses to preempt an unrelated live backend; it can reconcile a verified successor when startup-selection publication was interrupted.

`omnitron backend install <sealed-runtime>` copies a verified release into permanent storage without activation. `omnitron backend installation --json` reports its startup selection and unfinished activation. `omnitron backend select <sha256>` selects an installed release only while the backend and execution owner are stopped. Running installations use `backend upgrade`; it commits the selection after readiness and preserves the rollback. The stable launcher is `$OMNITRON_HOME/runtime/bin/omnitron`.

Before scheduling an owner restart, install the candidate, then run its CLI against the instance.

```sh
omnitron --json backend select <installed-sha256> --check
```

The result identifies the current and proposed releases and the migration scope. You can run this check while applications are online. Selection repeats the checks under the installation lock after the backend and owner have stopped. An independently supervised worker with cluster coordination disabled compares its complete SQLite migration registry in both releases. Controllers, enrolled workers and installations without a proven managed role compare both registries. Changed worker migrations require a separate data transition. A successful cold-selection check does not establish live owner compatibility. Use `backend upgrade --check` for that decision.

For supported independent services, `omnitron backend guardian prepare <sealed-runtime>` records a private transition plan. `omnitron backend guardian apply <plan>` applies or resumes it while retaining the execution owner and application processes. The existing backend release is restored under the updated guardian.

`omnitron backend schema prepare <sealed-runtime>` reads the master database and saves a plan for explicitly reviewed compatible additions. `omnitron backend schema apply <plan>` applies or resumes it under the shared migration lock. After rollback, `prepare <sealed-runtime> --from-plan <applied-plan>` binds the same already-applied expansion to a corrected candidate, checking the original runtimes and exact database history. `omnitron backend upgrade <sealed-dist> --schema-plan <plan>` requires that exact applied plan for an upgrade or rollback between its two sealed releases. Legacy bridges, changed definitions, foreign databases and local node schema transitions are refused.

Use the candidate's CLI for `backend upgrade --check`. It checks the owner's declared lifecycle contracts before draining or delegating; an incompatible owner requires a planned owner transition. `backend status --json` includes `execution.ownerCapabilities` when the backend reports them.

### Recover after an execution-owner crash

An unclean owner exit can leave application processes alive. A new owner cannot
adopt their native process handles, and ordinary startup refuses a same-boot
unclean receipt. Preserve the crash evidence and prepare a node-specific drain
and rollback before stopping surviving applications.

On macOS with launchd, or Linux with systemd system services,
`omnitron backend owner-reconcile prepare <worker-pids>` seals a private recovery
plan. Supply every
recorded pre-crash application, worker and descendant PID as a comma-separated
list, including non-Node processes. The plan binds process lifetimes, the host
boot, configured services, controller journal and unclean owner receipt. Linux
also records the configured cgroups; macOS records the launchd domain, native boot
session, observed descendants and their retained process groups. The command
writes only private plan/seal records; it does not open SQLite, stop services or
signal applications. Changed or legacy unsealed plans require a fresh preparation.

After the separately approved drain, `omnitron backend owner-reconcile acknowledge <absolute-plan>` requires the old owner identities to be dead, both configured
services stopped, observed ownership groups empty, every recorded lifetime gone
and execution sockets absent from the kernel. Unknown or incomplete observations,
changed configuration, journal, boot session or plan refuse recovery. Only a proven
stale owner socket may be removed. The original unclean receipt is archived; it is
never changed to claim a clean shutdown.

On macOS, this proof covers the supplied identities, observed descendants and
retained process groups. Ordinary launchd users have no cgroup-equivalent
containment: an already detached and reparented process never recorded in that
cohort cannot be discovered reliably. Include independently recorded descendants
and audit the drain; do not treat an empty process-name search as evidence.
Native `ps`, `launchctl`, `sysctl` and `lsof` observations must all succeed.

The macOS boot fence uses `kern.bootsessionuuid`, shared with process-lifetime
inspection. Historical `kern.boottime` text can drift while the same kernel keeps
running; a changed timestamp never proves a reboot. A dirty legacy receipt needs
explicit reconciliation even when its text differs. A fresh plan binds the current
UUID while preserving the complete original receipt and its hash. Old plans based
on boot-time text must be prepared again. Genuinely clean legacy receipts remain
eligible for normal startup. Legacy or malformed log-writer identities stay
unconfirmed while their PID is alive; diagnostics explain the uncertainty and
recovery proceeds only after native death is established.

A compatible replacement owner consumes the acknowledgment once under a kernel
lock before starting applications. Linux uses `flock`; macOS uses its native
`lockf`. Node retains the locked file descriptor after the helper exits; process
death releases it without deleting a lock file or trusting a stale PID. A durable
consume record also prevents replay when final acknowledgment publication fails.
These private records protect against accidental races and interrupted recovery;
they are not attestations against another process with the same filesystem UID.

A failed startup requires a fresh plan and fresh native evidence. An older runtime
without this recovery contract can be restored only after a replacement has
completed a genuinely clean shutdown. Do not promise an unconditional old-binary
rollback after another owner crash. Automatic adoption of orphan processes is not
implemented.

See [requirements, first transition and recovery](/docs/omnitron/backend-handoff).

## Local runtime updates

`omnitron runtime` owns native artifact verification. `omnitron runtime update`
owns a reviewed transition of local supervision, Console and the recorded stacks.

| Command                                     | Result                                                                                                                 |
| ------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------- |
| `omnitron runtime verify <directory>`       | Verify the sealed files, links, executable bits and host compatibility without executing the candidate                 |
| `omnitron runtime update prepare <runtime>` | Install the verified candidate and record the current installation, stack selections, infrastructure and recovery plan |
| `omnitron runtime update apply <digest>`    | Recheck and apply that exact SHA-256 plan once                                                                         |
| `omnitron runtime update status [digest]`   | Read the durable receipt and verify its candidate without connecting to the daemon                                     |
| `omnitron runtime update recover <digest>`  | Restore the previous installation recorded in the reviewed plan, or inspect its completed recovery                     |

From a clean Omnitron checkout, Xec prepares the same public operation:

```bash
export OMNITRON_DEV_CACHE=/path/to/private/development-cache
pnpm exec xec update local-sync
```

Review the returned scope, readiness conditions and recovery limitations before
applying it. `pnpm exec xec update local-sync --review <digest>` performs the
transition and may restart the listed applications. Inspect
`pnpm exec xec update local-sync --status` after an interrupted response; do not
repeat an uncertain operation. Recovery uses
`pnpm exec xec update local-sync --review <digest> --rollback`.
The cache directory must be private to the current user. Xec reuses a verified
runtime for the same commit and native Node producer; a changed checkout requires
a new preparation. In the development checkout, it also discovers the completed
local release build for that exact commit, platform, architecture and Node version.
The bytes are verified locally before use, without another build, archive copy or
CI request. An invalid selected artifact fails verification instead of silently
falling back to a different build. Preparation does not activate the runtime.

Preparation requires an idle controller and a current recovery checkpoint.
`omnitron backend status` reports backup operations and other activity that defer
it. Run preparation again after that work finishes.

The rollback backend must also support reviewed stack boot, unless every stack
already has the exact saved infrastructure-preservation review. Installing a new
candidate does not add this capability to the previous runtime. For an older
installation, first check a backend-only transition with
`pnpm exec xec update local-check <sealed-runtime>/packages/0/dist`.
After that transition is reviewed and activated, prepare `local-sync` again.
The backend-only step preserves the owner and applications; the subsequent local
plan covers their restart and retains the upgraded backend for owner rollback.
See [backend handoff](/docs/omnitron/backend-handoff) for its scope and recovery contract.

Programmatic verifiers import `verifyRuntimeArtifact` or `inspectRuntimeArtifact`
from `@omnitron-dev/omnitron/runtime-artifact`. The latter returns an
`assertUnchanged()` check for a verified filesystem snapshot. A changed snapshot
invalidates the proof; it is not a durable authorization to execute or activate.

## Daemon lifecycle

### `omnitron up`

Start the daemon in the background. Auto-detects
`omnitron.config.ts` at the current working directory or any
ancestor.

| Option                     | Effect                                                 |
| -------------------------- | ------------------------------------------------------ |
| `-c, --config <path>`      | Explicit path to `omnitron.config.ts`                  |
| `-p, --project <name>`     | Initial project name (default: auto-detect from CWD)   |
| `-f, --foreground`         | Run the daemon attached to the terminal or supervisor  |
| `--no-infra`               | Skip Docker infrastructure provisioning                |
| `--no-watch`               | Disable file watching daemon-wide                      |
| `--master`                 | First-run: configure as master in a cluster            |
| `--slave [host:port]`      | First-run: configure as slave; optional master address |
| `--webapp` / `--no-webapp` | Toggle automatic console UI start                      |

Foreground startup returns control to the daemon lifecycle, not to the CLI exit handler. The process continues serving RPC after initialization; Titan handles its shutdown. If the daemon is already running, another `up --foreground` reports that state and returns without starting a second controller.

### `omnitron down`

Stop the daemon — gracefully stops every project, stack, and
infrastructure container.

### `omnitron release preflight`

Inspect tool availability, the configured builder capacity, active and queued jobs, and support for shared-store publication. Use `--json` in a pipeline.

### `omnitron release builds`

List controller jobs, optionally restricted to one project. Use `--json` in an agent or CI client.

### `omnitron release job`

Read a single job by ID. Use `--follow` to observe phases until it settles.

### `omnitron release stop-build`

Cancel the specified queued or executing job. Other projects retain their slots.

### `omnitron stack check-release`

Check admission for a specified project, stack and release ID. This uses the same read-only method as the Console deployment dialog. The exit status is nonzero on refusal.

### `omnitron ping`

Probe whether the daemon is alive. Exit code `0` = up, non-zero =
down.

### `omnitron kill`

Force-kill the daemon process. Use only when `down` doesn't
respond.

### `omnitron service install|uninstall|status`

Register the platform's processes with the operating system. In resident mode,
the owner keeps applications running while the independent guardian supervises
the backend. In coupled mode, the daemon also owns application processes, so its
failure can interrupt them. OS supervision and application supervision have
separate responsibilities.

| Command                                                       | Effect                                                                           |
| ------------------------------------------------------------- | -------------------------------------------------------------------------------- |
| `omnitron service install`                                    | Register OS supervision; resident mode requires a stopped backend and owner      |
| `omnitron service install --prepare`                          | Prepare independent owner/backend service definitions without stopping processes |
| `omnitron service install --prepare --owner-runtime <sha256>` | Prepare an installed owner version separately from the selected guardian/backend |
| `omnitron service uninstall`                                  | Stop the supervised daemon and remove the service                                |
| `omnitron service status`                                     | Whether the OS is supervising, and what it reports                               |

Preparation defaults both roles to the selected sealed runtime. `--owner-runtime`
selects another already installed, verified owner by its full SHA-256. Its native
executable and declared capabilities must satisfy the selected backend. Install
with the same owner hash, selected backend hash and service scope after the reviewed
shutdown. Repeat `service install --prepare` while stopped to review different
installed roles. The saved plan must still belong to the same instance, execution
directory, scope and native services; its validated private environment is preserved
exactly, including secrets absent from the current shell. Invalid saved records
refuse preparation. Installation retains strict role matching under the runtime
operation lock and requires stopped processes and quiescent native services; a
missing PID alone is insufficient. It does not stop services, choose a fallback or
upgrade an old owner's capabilities automatically. Backend-only updates use `backend upgrade` and keep
the current owner.

| Platform | What is written                                                       |
| -------- | --------------------------------------------------------------------- |
| macOS    | LaunchAgent at `~/Library/LaunchAgents/dev.omnitron.daemon.plist`     |
| Linux    | systemd user unit at `~/.config/systemd/user/omnitron-daemon.service` |

**Restarts are crash-only**, so `up` and `down` keep meaning what
they meant. A graceful `omnitron down` exits 0 and stays down; a
crash or a `SIGKILL` exits abnormally and is respawned
(`KeepAlive={SuccessfulExit:false}` on launchd, `Restart=on-failure`
on systemd). Installing the service does not take `down` away from
you.

In coupled mode, the supervisor runs the same `daemon-entry.js` that `omnitron up`
forks — one daemon codepath with two launchers — and that entry
point holds a single-instance pid guard, so the two can never both
bind.

## Development SDK

From Omnitron 0.4.9, starting or restarting a source application refreshes
its existing esbuild contexts before launch. A repaired package installation is
resolved again even when application source files have not changed. CLI and RPC
report success only for confirmed online applications; inspect a failed or
in-progress launch before retrying.

| Command                                                                        | Result                                                                                                                                                                      |
| ------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `omnitron sdk init`                                                            | Prepare the consumer workspace pnpm hook and distribution lockfile                                                                                                          |
| `omnitron sdk dev [checkout]`                                                  | Select a local SDK and watch it; `--stack <name>` enables refresh of watched local applications                                                                             |
| `omnitron sdk dev [checkout] --stack <name> --start-stopped --review <digest>` | Build the SDK, start a stopped local working tree on reviewed infrastructure and continue watching. Add `--no-watch` for a single run.                                      |
| `omnitron sdk status [--stack <name>]`                                         | Verify imports and show the active source. For a stack, compare its SDK outputs and current owner generations with the recorded refresh receipts. Supports global `--json`. |
| `omnitron sdk registry`                                                        | Restore the frozen registry installation                                                                                                                                    |

Each command accepts `--project-dir <path>`. The developer selection is machine-local.
The watcher runs in the foreground of `sdk dev`; saving the selection does not
create a background watcher. Keep that command running or resume it after exit.

From Omnitron 0.4.9, `sdk status` also verifies the current Turbo build
inputs, native Node producer and output digests. It reports a current build only
when they match the saved build checkpoint. Changed sources or dependencies, an
incomplete build or an older checkpoint require `sdk dev` to rebuild; inspection
never installs, builds or restarts applications.

SDK watching uses Xec Loader for filesystem observation. From Omnitron 0.4.9, the watched directories and shared files follow the Turbo task graph; Turbo inputs decide whether a change requires a build. Declare build inputs in Turbo, including generators and shared configuration. Unsupported or unbounded watch roots produce an explicit error. Atomic saves are detected, changes are coalesced, and edits made during a build are processed afterward. Generated output and installed dependencies do not trigger a rebuild loop. Cancellation closes the watchers and waits for the active build to settle.
See [local SDK development and registry deployments](/docs/omnitron/releases#local-sdk-development-and-registry-deployments).

## Project management

A *project* is one repository with many named stacks. Register the repository
once; subdirectory configurations contribute stacks to that owner. See
[projects and stacks](/docs/omnitron/projects) for composition and safe consolidation.

| Command                                                                                    | Effect                                                                                  |
| ------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------- |
| `omnitron project add <name> <path>`                                                       | Register a repository once                                                              |
| `omnitron project configuration list <project>`                                            | List owned configuration sources                                                        |
| `omnitron project configuration import <project> <name> <path> --stack <source=target...>` | Import every source stack; optional `--registration` preserves an older namespace       |
| `omnitron project configuration detach <project> <name>`                                   | Restore an imported source as a standalone registration without restarting applications |
| `omnitron project topology` (optional project, `--stack <name>`)                           | Configured dependencies and coverage warnings; supports global `--json`                 |
| `omnitron project list` (alias `proj list`)                                                | List registered projects                                                                |
| `omnitron project remove <name>`                                                           | Unregister (does not delete files)                                                      |
| `omnitron project scan`                                                                    | Scan app bootstraps and report their infrastructure requirements                        |

Aliases: `omnitron proj` works the same as `omnitron project`.

## Stack management

A *stack* is a named environment within a project (typical
values: `dev`, `staging`, `prod`). Stacks scope deployments and
infrastructure to a subset of the project's apps.

| Command                                                                                         | Effect                                                                                                                                                                                                                                           |
| ----------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `omnitron stack list [-p project]` (alias `stacks list`)                                        | All stacks across all projects (filter with `-p, --project`)                                                                                                                                                                                     |
| `omnitron stack create <project> <stack> [-t type] [-a apps]`                                   | Create a new stack (`-t, --type local\|remote\|cluster`, default `local`; `-a, --apps` comma-separated names or `all`, default `all`)                                                                                                            |
| `omnitron stack delete <project> <stack>`                                                       | Drop a stack                                                                                                                                                                                                                                     |
| `omnitron stack status <project> <stack>`                                                       | Per-app status within the stack                                                                                                                                                                                                                  |
| `omnitron stack progress <project> [stack]`                                                     | Latest remote transfer observation per stack, node and app, including its operation ID and any release ID                                                                                                                                        |
| `omnitron stack diagnose <project> <stack>`                                                     | Read-only prerequisite checks and planned changes before start                                                                                                                                                                                   |
| `omnitron stack prepare-release <project> <stack> <release>`                                    | Fetch a missing release from the configured store, check target admission and stage archives on a local stack                                                                                                                                    |
| `omnitron stack start <project> <stack> [--release <id>]`                                       | Start the stack. An explicit release becomes the durable local selection. Later starts reuse it.                                                                                                                                                 |
| `omnitron stack start <project> <stack> --working-tree`                                         | Start a stopped local stack from its registered source and clear its release selection. Release-only stacks refuse this option.                                                                                                                  |
| `omnitron stack resume <project> <stack>`                                                       | Resume the retained release on the nodes in a confirmed cluster stop. Partial and unconfirmed results exit nonzero. Inspect each node in `--json` output.                                                                                        |
| `omnitron stack stop <project> <stack> [--keep-infrastructure] [--dry-run] [--review <digest>]` | Inspect every target with `--dry-run`. The default stops applications and containers, retaining volumes. Add `--keep-infrastructure` to stop applications while preserving containers and ownership. Use the same mode and digest for execution. |
| `omnitron stack runtime <project> <stack>`                                                      | Show runtime details (containers, ports)                                                                                                                                                                                                         |

An applications-only maintenance stop also disables automatic application resume
and preserves the selected release. A backend or owner restart does not start it
again; use an explicit `stack start`. Both stop modes share the same permissions,
review, execution fence and per-node receipts. A legacy remote node that cannot
confirm retention blocks preparation instead of receiving a destructive fallback.
See [confirmed stack stops](/docs/omnitron/projects#confirmed-stack-stops).

### Restart with preserved infrastructure

For a retained release, inspect the preservation start before execution:

```bash
omnitron stack start sample dev --release <release-id> \
  --infrastructure preserve --dry-run --json
```

After the reviewed stop, use the returned digest for that exact start:

```bash
omnitron stack start sample dev --release <release-id> \
  --infrastructure preserve --review <digest>
```

Preservation start rechecks release admission and native infrastructure evidence.
It skips provisioning, setup, migrations, static uploads and environment
provisioning; a mismatch refuses the start. A runtime or installed-path change
may invalidate the earlier digest, requiring a fresh review.

`stack stop --keep-infrastructure` durably selects preservation policy. A plain
start or autostart cannot silently replace it with reconciliation. Explicit
`--infrastructure reconcile` is a separately reviewed provisioning, setup and
migration operation. See [infrastructure ownership](/docs/omnitron/infrastructure#infrastructure-ownership)
for the responsibility boundary.

Before a first start or after changing an environment:

```bash
omnitron stack diagnose sample dev
omnitron --json stack diagnose sample dev
```

The same report appears under **Stacks → Diagnose** in Console. JSON includes
`checkedAt`, `durationMs`, `status`, structured `checks` and explicit `deferred`
coverage. Exit code `0` means prerequisite checks passed, possibly with planned
creation/downloads; `1` means blockers were found; `2` means evidence is incomplete
or the request failed. A successful prerequisite check does not certify database
migrations or application startup. No provisioning or migration is performed.

Local Docker checks include declared container ownership, specification changes,
health, image and named-volume availability, bind mounts and host port conflicts.
Remote target-host checks remain explicitly incomplete; `infra inspect` and
`node check` provide complementary evidence. Project configuration is the daemon's
loaded configuration; reload it after editing project configuration files.

## App lifecycle

These commands accept an `[app]` argument. With no app, they
operate on **all** managed apps.

`start`, `stop` and `restart` use text output in 0.4.9. They reject `--json` and
`OMNITRON_OUTPUT=json` before starting an action. Use `omnitron list --json`
for a structured observation of the resulting application state.

| Command                    | Effect                                                                                                                                    |
| -------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------- |
| `omnitron start [app]`     | Start a named app through the running daemon. A fleet-wide start can also start a stopped daemon.                                         |
| `omnitron stop [app] [-f]` | Stop the selected applications. Pass `--force` to request a forced stop.                                                                  |
| `omnitron restart [app]`   | Stop + start                                                                                                                              |
| `omnitron reload [app]`    | Reload a named supervised application one child at a time. Classic applications restart. The current bulk command invokes a full restart. |

Use a full application name such as `omnitron start project/test/api` or `omnitron stop project/test/api` to limit the request to one process group. An existing application uses its registered launch definition, including after a project reload. A short name shared by two stacks requires the operator to select the full name. When a registered handle is missing, start and restart accept a declaration from its matching project and stack. If a named start cannot reach the daemon, it exits with an error. Starting the daemon with `omnitron up` can also start its configured stacks. Use `omnitron stop all` to address every managed application. These operations require a responsive daemon. Start and stop report a connection failure with a nonzero exit code. In 0.4.9, restart can return exit code 0 after reporting an unavailable daemon; confirm the observed application state instead of relying only on that exit code. If the connection drops after sending the request, inspect `omnitron list` before retrying. Use `omnitron down` for an explicit daemon shutdown.

A named `stop` controls the current process group; it does **not** persist an
application exclusion in an enabled stack. A later stack start or boot recovery
can start that application again. Do not use a successful stop receipt as a
promise that a port or workload identity stays available for a replacement.
`stack stop` records durable intent for the **whole stack**; it is not a substitute
for excluding one application while its siblings remain active. The
[scoped owner contract](/docs/omnitron/backend-handoff#owner-compatibility) provides the
lower-level fence, but the public per-application lifecycle does not yet expose
that guarantee.

## Information

| Command                      | Effect                                                                                                                          |
| ---------------------------- | ------------------------------------------------------------------------------------------------------------------------------- |
| `omnitron list` (alias `ls`) | One row per app, then one per topology process                                                                                  |
| `omnitron status`            | Daemon-wide overview (apps, uptime, leader, infra)                                                                              |
| `omnitron config [--json]`   | Print the resolved ecosystem config                                                                                             |
| `omnitron init [--force]`    | Scaffold `omnitron.config.ts` in the CWD. Existing files require confirmation or explicit `--force`; symbolic links are refused |

Cancelling the overwrite prompt preserves the original file and exits with status `130`. An existing file in a noninteractive session requires `--force`; a new file needs no confirmation.

**Reading the child rows of `omnitron list`.** Each indented row is
one entry of the app's process topology, and two of its columns say
something a single number cannot.

A pool entry — one declaring `instances > 1` — runs several
processes and the row prints one of their pids, so it also prints
its live worker count: `transform (worker) ×2`. When that count has
drifted from what the topology declares it is shown as `×1/2` in
yellow; `doctor` reports the same drift as a finding.

`RST` shows `-`, not `0`, when nothing counts that row's restarts.
The supervisor keeps a restart count per child, and a pool is not a
supervisor child, so for a pool row the honest answer is that it is
not tracked. `0` would leave "never restarted" and "not counted"
looking identical.

`UPTIME` is the uptime of the process whose pid the row prints —
not the app's. A child that restarted an hour ago under an app that
has been up for a week reads one hour.

### `omnitron system`

```sh
omnitron system
omnitron --json system --node <registered-id>
```

The command uses the same bounded collectors as Console **System**. A named node is queried over the authenticated mesh. JSON includes the target, collection timing, completeness and the complete snapshot. Inspect `snapshot.unanswered` for per-collector failures. Exit code 1 indicates an unavailable, stale or partial reading, or a host clock too far ahead. The JSON still contains every successful measurement.

A stack start also exposes its outcome in JSON. `outcome: "confirmed"` requires a running stack, every reported application online and no missing infrastructure or nodes. A partial reply includes the affected resources and sets exit code 1. A transport timeout leaves the result unconfirmed. Inspect the stack and deployment history before repeating the request.

## Monitoring

Resident mode uses independent owner and backend guardian services. See [backend replacement](/docs/omnitron/backend-handoff#independent-os-supervision) for preparation, crash recovery, system scope and owner-failure boundaries.

### `omnitron logs [app]`

Tail logs. Omit `[app]` to read the daemon's own log.

| Option                 | Effect                                                                                                                                         |
| ---------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- |
| `-f, --follow`         | Stream new entries                                                                                                                             |
| `-n, --lines <N>`      | Show last N entries (default `50`)                                                                                                             |
| `-l, --level <lvl>`    | A minimum severity for `trace`, `debug`, `info`, `warn`, `error` or `fatal`. Use `unknown` or an application-defined level for an exact match. |
| `-g, --grep <pattern>` | Filter by message regex                                                                                                                        |
| `--file`               | Read from log files (auto-fallback when daemon offline)                                                                                        |
| `--node <name-or-id>`  | Read replicated records from one registered remote node; never falls back to local files                                                       |

Use the full application handle for production diagnostics:

```bash
omnitron logs sample/production/mail --node worker-1 -n 100
omnitron logs sample/production/mail --node worker-1 --level unknown
omnitron --json logs sample/production/api --node worker-1 --grep 'auth\.login'
```

`--node` resolves a node name or unique ID prefix through the registry and uses
the same `OmnitronLogs.streamLogs` contract as Console. `--node` and `--file`
are mutually exclusive. Omit `--node` for the local machine. JSON output records
the source and node; an empty result does not establish that no request occurred.

### `omnitron monit`

Live TUI dashboard — terminal app showing per-process CPU /
RSS / restart counts in real time. Quit with `q`.

### `omnitron health [app]`

Composite health report from `titan-health` indicators.

### `omnitron metrics [app]`

Snapshot of current CPU / memory / latency / RPC counters.

### `omnitron health-check [app]`

Checks application-reported health, declared readiness and HTTP/TCP availability.
A live process or HTTP 200 cannot override a failed application check. Without an
app name, includes every local application and declared stack infrastructure.

Use `omnitron --json health-check [app]` for the same structured report. Both
human and JSON modes exit with status 1 when any check is degraded or unhealthy;
JSON output retains the individual checks and their reasons.

### `omnitron discover`

Scan Docker + SSH on known hosts for Omnitron-managed processes
not yet in the registry. Useful when migrating in an existing
fleet.

## Scaling

### `omnitron scale <app> <count> --process <name>`

Resize one declared process pool in a running bootstrap application. The process
name is required; classic applications and singleton processes cannot be resized.
The count must be a positive safe integer. A fixed pool accepts 1 through its
configured `maxInstances` (the declared size when no maximum is configured).
An automatic pool retains its declared initial size as the minimum and uses its
configured maximum, or four times its initial size when omitted.

This changes runtime membership until restart. It does not rewrite the application
configuration; restart restores the declared size, and automatic scaling remains
active. The result reports the selected pool's observed count. For agents,
`omnitron --json scale <app> <count> --process <name>` returns the request,
observation, and `lifetime: "until-restart"`. If a dispatched request times out or
its outcome is unknown, inspect that pool before retrying.

The owner must advertise `processPoolResize@1`. An older owner refuses the request
without changing the application. Updating only the backend does not add this
capability to a running owner.

## Diagnostics

### `omnitron doctor`

One pass over the whole installation, reporting problems with the
evidence that identifies them and the command that fixes them.

It reads the database directly rather than through the daemon,
because a daemon whose schema has gone missing still answers RPCs
perfectly well.

**What it examines.** Every finding's `id` begins with the area it
came from, so this table is the same list the output uses:

| Area      | What it looks at                                                                                                                                          |
| --------- | --------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `daemon`  | Whether it answers, how fast, whether it restarted recently, whether it is aggregating metrics                                                            |
| `app`     | Managed apps and their sub-processes, and the ports they claim against the ports they answer on                                                           |
| `infra`   | Infrastructure containers, their health, their published ports, and containers detached from the stack                                                    |
| `db`      | Reachability, the tables the daemon needs, pending migrations, and a table that is bloated rather than merely large                                       |
| `build`   | Sources newer than their build output, sources never built, and a daemon older than the build it is running                                               |
| `webapp`  | Whether the console is being served and whether its proxy reaches the daemon                                                                              |
| `logs`    | One message dominating the day's errors; repetition dominating while no single message does; lines stored twice; an operation being retried without limit |
| `alerts`  | Enabled alert rules whose expression can never fire                                                                                                       |
| `metrics` | Whether any CPU or memory reading is arriving at all                                                                                                      |
| `project` | Every registered project's config, read from the file rather than from the daemon                                                                         |
| `auth`    | Which RPC methods answer without credentials, and whether the daemon is reachable off this host                                                           |
| `doctor`  | A check that failed on its own account, so a gap is never silent                                                                                          |

| Option   | Effect                              |
| -------- | ----------------------------------- |
| `--json` | Machine-readable output (see below) |

Exit code is `1` when any finding has severity `error`, so it
works as a CI gate.

**What "no problems found" means.** A check that cannot run is
reported separately from one that ran and found nothing:

```
[?] 5 area(s) not examined:
      db.pending-migrations — not checked: whether every migration
        has been applied — the database stopped answering
      ...
```

An unreachable database takes five checks down with it and a
daemon that is not answering takes seven. Without that section a
silent gap and a clean result print identically, and they point an
operator in opposite directions.

In `--json` the same distinction is two fields:

```json
{
  "ok": false,
  "complete": false,
  "worst": "error",
  "findings": [ { "id": "db.unreachable", "severity": "error", "title": "…",
                  "evidence": ["…"], "remedy": "…" } ],
  "skipped":  [ { "id": "logs.*", "reason": "not checked: … — the database stopped answering" } ]
}
```

`ok` is "nothing was found"; `complete` is "everything was
looked at". A gate that reads only `ok` cannot tell a healthy
installation from an unexamined one.

Findings carry a stable `id` — `db.unreachable`,
`app.port-unreachable`, `infra.detached`, `build.stale-sources`,
`auth.anonymous-surface`, `logs.retry-loop`,
`project.config-unloadable` and so on. A `skipped` entry names the
id its check would have reported under, or `prefix.*` where a check
reports several.

`project.config-unloadable` is worth knowing about before you need
it: the daemon refuses to start against a config it cannot load,
rather than starting with no apps, so this finding is what a failed
`omnitron up` in that project's directory would have told you. The
check reads the files rather than asking the daemon, because a
daemon holding a config it loaded hours ago cannot tell you the
file has been edited since.

### `omnitron inspect <app>`

Deep diagnostics for an app — services exposed, memory
breakdown, pending requests.

| Option                    | Effect                                                     |
| ------------------------- | ---------------------------------------------------------- |
| `--graph`                 | Render live DI dependency graph instead of memory/services |
| \`--format \<mermaid      | dot                                                        | json>\` | Graph format (default `mermaid`)   |
| `--focus <token>`         | Restrict the graph to one token + its closure              |
| \`--direction \<ancestors | descendants                                                | both>\` | Closure direction (default `both`) |

### `omnitron exec <app> <service> <method> [args...]`

Invoke an RPC method on a managed app:

```bash
omnitron exec api users findById u_42
```

Arguments are parsed as JSON when they look like JSON, otherwise
strings.

### `omnitron env <app>`

Show resolved environment variables for an app (after merging
project / stack / per-app overrides).

## Remote daemons

Register remote daemons as named aliases and address them via
`omnitron remote <alias>`.

| Command                                                  | Effect                                  |
| -------------------------------------------------------- | --------------------------------------- |
| `omnitron remote add <alias> <host> [-p port] [-t tags]` | Register a remote (default port `9700`) |
| `omnitron remote remove <alias>`                         | Unregister                              |
| `omnitron remote list`                                   | List registered remotes                 |
| `omnitron remote status <alias>`                         | Check connectivity                      |

## Fleet operations

Aggregate across all remotes (and the local daemon).

| Command                  | Effect                              |
| ------------------------ | ----------------------------------- |
| `omnitron fleet status`  | Status across the fleet             |
| `omnitron fleet health`  | Health across the fleet             |
| `omnitron fleet metrics` | Aggregated metrics across the fleet |

## Cluster (leader election)

| Command                      | Effect                                                                                                 |
| ---------------------------- | ------------------------------------------------------------------------------------------------------ |
| `omnitron cluster status`    | Confirmed authority, reason, lease freshness, epoch and recipient-fenced operations. Supports `--json` |
| `omnitron cluster step-down` | Release the current lease as an administrator                                                          |

Cluster operations require `cluster.enabled: true` in the
daemon config.

## Secrets (encrypted at rest)

| Command                             | Effect                                              |
| ----------------------------------- | --------------------------------------------------- |
| `omnitron secret set <key>`         | Store a secret using hidden terminal input or stdin |
| `omnitron secret set <key> --stdin` | Explicit stdin input for automation                 |
| `omnitron secret get <key>`         | Read a decrypted secret                             |
| `omnitron secret list`              | List all keys (values hidden)                       |
| `omnitron secret delete <key>`      | Delete a secret                                     |

The daemon stores an encrypted envelope in its SQLite state store; the former
`~/.omnitron/secrets.enc` is a legacy migration source. A running daemon owns
writes to the vault. A rejected RPC write never falls back to changing its files.

Omit the value to enter it without echo in a terminal, or feed it from a secure
source on stdin. Input is limited to 64 KiB; one final LF or CRLF is removed.
Empty input and conflicting positional/`--stdin` sources are refused. Positional
values remain compatible, but emit a warning because shell history and process
arguments can expose them. `secret get` deliberately prints the value; use it
only when that output is intended.

Keep references in project configuration, for example
`settings.appEnv.api.API_KEY = { secret: 'service.api-key' }`. References limit
which application receives the value. Do not put credentials in source files,
frontend bundles or command arguments. Encryption at rest does not remove the
need to protect the vault passphrase, state backups and the host account.

Remote deployment sends resolved configuration and migration environment through
Xec's SSH stdin channel, rather than embedding credentials in command arguments.
The existing deployment lease still fences those operations. Node configuration
is materialized in private files (0600) so applications can restart independently
of the controller; an administrator of that host can read it. Migration output
uses the common diagnostic redactor, but applications must still avoid printing
secret values themselves. Failed or interrupted migrations are not automatically
retried: inspect their ledger before deciding whether to run them again.

## Deployment

### Stack deployment and rollback

Deploy a built release with `omnitron stack start <project> <stack> --release <id>`.
Returning to a previous release uses that same workflow with the reviewed previous
ID. Admission, applicable transfer checks and readiness still apply. For a local stack, see [Pin a local stack](/docs/omnitron/releases#pin-a-local-stack) for preparation, controlled switching and rollback.

`omnitron deploy app` and `omnitron rollback` are retained refusals: they exit
nonzero without changing applications. Their former per-app path could restart
current code but could not install a requested version or restore an earlier
artifact. The corresponding RPC mutations refuse before database or process
changes as well. No rolling, blue-green or canary rollout is implied by accepting
a strategy label in a historical record.

### `omnitron deploy build <app>`

Build an application artifact without shipping or starting it. To build a release
with source commits, manifests and gates, use `omnitron release build`.

## Tor

### `omnitron tor`

Show the Tor hidden-service onion addresses for the running
stack. Top-level command; no flags.

## Infrastructure

Manage infrastructure containers (Postgres / Redis / MinIO /
custom) provisioned per app's `omnitronConfig.infrastructure`.

| Command                                                                      | Effect                                                                                                                                                                                                                                              |
| ---------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `omnitron infra up`                                                          | Provision and start all infra services                                                                                                                                                                                                              |
| `omnitron infra down`                                                        | Remove containers owned by the current standalone configuration and preserve their volumes. The legacy `--volumes` option is rejected.                                                                                                              |
| `omnitron infra status [--stack <project>/<stack>]` (alias `infra ps`)       | Inspect this host, or resolve a registered stack and show its local containers or remote per-node observations. [Reading the inventory](/docs/omnitron/infrastructure#readiness-includes-required-service-setup)                                    |
| `omnitron infra logs [service] [-f] [-n N]`                                  | View logs for one or all infra services                                                                                                                                                                                                             |
| `omnitron infra psql [database] [--stack project/stack] [-c sql]`            | Open the local stack’s PostgreSQL shell or run SQL once. The command returns the client exit status.                                                                                                                                                |
| `omnitron infra redis-cli [--stack project/stack] [--db index] [command...]` | Open the local stack’s Redis shell or execute a Redis command in the selected database.                                                                                                                                                             |
| `omnitron infra migrate [app] [--dry-run]`                                   | In a standalone ecosystem, resolve the selected applications to package migration scripts and run each package once in dependency order. Preview the packages with `--dry-run`. A failing script stops the sequence and identifies its application. |
| `omnitron infra reset --yes`                                                 | Recreate owned containers using their existing volumes.                                                                                                                                                                                             |

Direct infrastructure mutations require a standalone installation. For projects declaring named stacks, use the project and stack lifecycle to provision or remove containers. An enrolled node requires admission through the cluster lifecycle protocol. Direct container operations verify ownership and use the observed Docker ID before removal.

## Pipelines (Xec automation)

| Command                                                                  | Effect                                                                                          |
| ------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------- |
| `omnitron pipeline list`                                                 | List all pipelines                                                                              |
| `omnitron pipeline save <file> [--id <id> --expected-revision <sha256>]` | Create a JSON definition, or replace its reviewed revision without overwriting concurrent edits |
| `omnitron pipeline run <id> [--params-file <file>]`                      | Read the current revision and submit it for execution                                           |
| `omnitron pipeline status <runId> [--json] [--output]`                   | Inspect steps, attempts, status and optionally retained output                                  |
| `omnitron pipeline cancel <runId>`                                       | Request cancellation and await the owned execution                                              |

All pipeline RPC methods require `admin`. Parameters are data, not source substitutions. Failed, cancelled and interrupted runs produce a nonzero status-command exit code. See [Xec automation](/docs/omnitron/automation) for definitions, limits and recovery behavior.

## Backups

| Command                                                                                    | Effect                                                                                |
| ------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------- |
| `omnitron backup create [database]`                                                        | Back up one database, or every locally provisioned stack database with no argument    |
| `omnitron backup artifact storage-objects --project <name> --stack <name> --bucket <name>` | Capture the current object contents of one bucket                                     |
| `omnitron backup artifact tor-keys --project <name> --stack <name>`                        | Capture one stack's onion identity                                                    |
| `omnitron backup full`                                                                     | Visit stack databases, declared MinIO buckets, Tor keys and daemon state on this host |
| `omnitron backup list`                                                                     | List backups with their ID, stack, target and bucket                                  |
| `omnitron backup restore <id>`                                                             | Restore to the recorded origin                                                        |
| `omnitron backup schedule <target> <cron>`                                                 | Schedule one database, `all` databases, or a `full` pass                              |
| `omnitron backup schedules`                                                                | Show configured schedules, their latest outcome and next run                          |
| `omnitron backup unschedule <target>`                                                      | Remove a schedule                                                                     |

Use the explicit project and stack for resource backups. Infrastructure retained
while application writers are stopped remains addressable. The daemon verifies
container ownership and uses its immutable ID. The archive index records the
project, stack and bucket; a restore refuses an unavailable origin instead of
choosing another stack. Legacy archives without origin metadata require exactly
one eligible local target. Legacy object archives refer to the `storage` bucket.

```sh
omnitron backup artifact storage-objects --project acme --stack prod --bucket uploads --json
omnitron backup artifact tor-keys --project acme --stack prod
omnitron backup list
omnitron backup restore <id>
```

Creation and restore require an administrator. CLI and Console clients share the
`OmnitronBackups` Netron contract; `createArtifactBackup` accepts
`{ project, stack, kind: 'storage-objects', bucket }` or
`{ project, stack, kind: 'tor-keys' }`.

A `full` pass reports each target independently, including undeclared bucket
inventories and failures. It covers provisioned resources on the receiving daemon;
it does not automatically contact remote daemons. MinIO bucket inventory comes
from the stack's declared `buckets`. Explicit artifact capture can name another
bucket on that same provisioned service.

Object contents stream through the shared backup pipeline with bounded concurrency.
Each operation owns a private credential directory; credentials do not appear in
host command arguments. A snapshot is published only after transfer and archive
validation. Inventory changes during the copy, unsupported object names and names
that collide on a portable filesystem cause an explicit failure.

These object archives preserve current bytes, not S3 version history, policies,
object metadata or tags. A full pass is not an atomic snapshot across services.
Quiesce writers when application consistency is required. Restore overwrites the
archived keys, leaves other keys intact and requires an existing bucket. If restore
is interrupted, reconcile the destination before resuming writers.

Database dumps use a private staging file and one deadline for the local process,
compression, file writes and publication. The default database transfer budget is
30 minutes per dump or restore. Configure it on each daemon, as described in
[database backup policy](/docs/omnitron/configuration#database-backup-policy). The destination
is created exclusively; an existing file is never overwritten. File contents and
the containing directory are synchronized before a successful publication is acknowledged. If publication
or indexing cannot be confirmed, a complete destination is retained and the error
names the file and the uncertainty. Inspect that path and the backup index before
repeating the operation.

Historical `.meta.json` records are removed only after the SQLite index durably
confirms the same identity and metadata. A database error, conflicting record or
failed file removal retains the source for a later retry. This protects the index;
it does not verify that an archive can restore its application's data.

Failed requests, partial capture and an empty capture pass return a nonzero CLI
exit status. Restore and schedule mutations require a positive acknowledgement.
Read commands distinguish missing diagnostic data from an empty index and still
show uncovered stacks when no backups or schedules exist. A previously failed
scheduled pass remains a warning in an otherwise successful status query.

`omnitron backup schedules` reports the effective database transfer budget beside
the schedules and their last outcomes; `--json` returns the same diagnostic data.
An expired budget is a failed operation, never a successful partial backup.
PostgreSQL restores ignore personal `psql` startup files and stop at the first SQL
error. Neither a SQL error nor a timeout rolls back statements already applied to
the target: the restore is not a single transaction. Keep writers stopped and
inspect the destination before deciding whether to restore again or recover
another way.

A local Docker client exiting does not prove that its in-container command has
stopped. The pipeline reports unresolved cleanup instead of claiming termination.
The database budget applies to each PostgreSQL dump or restore on this daemon;
S3 transfers and Tor archives have separate limits. Per-database overrides and
verified in-container exec termination are not implemented.

Schedules persist across daemon restarts; they do not run while the daemon is down.
Active capture, restore and scheduled retention block backend handoff. Preparation
pauses schedule admission, and an aborted handoff re-arms persisted schedules.

Omnitron 0.4.9 also fences explicit `OmnitronBackups.deleteBackup`
operations. Archive removal is asynchronous, so large-file deletion does not stop
RPC or heartbeats. Restore and deletion of one archive are serialized; independent
archives remain independent. A deletion is acknowledged only after its files and
index entry are removed. On failure, inspect the reported ID before retrying:
files may already be absent while the index preserves unfinished cleanup.
Listing never finishes that deletion implicitly.

## Kubernetes

| Command                                                      | Effect               |
| ------------------------------------------------------------ | -------------------- |
| `omnitron k8s pods [namespace]`                              | List Kubernetes pods |
| `omnitron k8s deploy scale <name> <replicas> [-n namespace]` | Scale a deployment   |

## Nodes (infrastructure machines)

Register physical / virtual machines for fleet operations and
remote deploys. Different from `remote daemons` — nodes are SSH
targets that may or may not run a daemon.

| Command                                  | Effect                                                                                      |
| ---------------------------------------- | ------------------------------------------------------------------------------------------- |
| `omnitron node list` (alias `ls`)        | Registered nodes and last observed connectivity. `--json` returns full IDs in `data.nodes`. |
| `omnitron node add ...`                  | Register a node (see flags below)                                                           |
| `omnitron node update <id> ...`          | Update a node                                                                               |
| `omnitron node remove <id>` (alias `rm`) | Remove a node                                                                               |
| `omnitron node check [id]`               | Connectivity check (all nodes if omitted)                                                   |
| `omnitron node ssh-keys`                 | List available SSH keys from `~/.ssh/`                                                      |
| `omnitron node reconcile <node>`         | Inspect historical unverified metrics; administrator only                                   |

Use `omnitron --json node list` for fleet automation. Each node includes its full ID, endpoints, runtime, tags and the most recent status. A null status means the controller has no observation yet. The same command works against older controllers.

For SSH key authentication, the node registry keeps a file reference or an encrypted private key. Reading or editing a node shows **Keep stored key** for an inline key. Changing a node's host, port or SSH account requires an explicit credential choice. Existing inline keys move to the vault during startup. If the vault is unavailable, the original credential is retained for recovery and the controller reports the pending migration.

`node reconcile` is read-only unless `--restore` is supplied. It compares retained
source samples with PostgreSQL using the original timestamp, value and normalized
labels. Matching samples are reused; missing samples are restored through Titan's
durable storage transaction. The source accepts proof only after that commit.

```bash
omnitron --json node reconcile edge-1
omnitron --json node reconcile edge-1 --restore --pages 20
```

`--pages` defaults to `1` and accepts `1..2000`; a page contains at most 256 entries
and 2 MiB. JSON output includes counts, unresolved source rows and a `continuation`
object. Resume its `after`, `through` and `epoch` using `--after`, `--through` and
`--epoch`. After a failed acknowledgement, retry the same cursor. Repeating a
completed repair makes no new samples.

The source must support the reconciliation protocol. An older node with no
unverified entries answers as a no-op. For an older node with gaps, prepare a
compatible update or a reviewed maintenance worker; the command does not silently
execute SSH code. See [historical reconciliation](/docs/omnitron/cluster#reconcile-historical-metric-receipts).

### `omnitron node add` flags

| Flag                   | Required | Default                                     |
| ---------------------- | -------- | ------------------------------------------- |
| `--name <name>`        | yes      | —                                           |
| `--host <host>`        | yes      | —                                           |
| `--ssh-port <port>`    | —        | `22`                                        |
| `--ssh-user <user>`    | —        | `root`                                      |
| `--ssh-auth <method>`  | —        | `key` (or `password`)                       |
| `--ssh-key <path>`     | —        | —                                           |
| `--ssh-secret-stdin`   | —        | Hidden terminal prompt, or read from a pipe |
| \`--runtime \<node     | bun>\`   | —                                           | `node` |
| `--daemon-port <port>` | —        | `9700`                                      |
| `--tags <a,b,c>`       | —        | —                                           |

With `--ssh-secret-stdin`, an interactive terminal displays a masked prompt: **Enter**
submits the password or key passphrase, and **Ctrl+C** cancels without changing the node.
A pipe is read until EOF; one final LF/CRLF is removed, while spaces are preserved.
Input is limited to 64 KiB. Secrets are never command-line arguments.

```bash
omnitron node add --name production --host server.example.org \
  --ssh-key ~/.ssh/id_ed25519 --ssh-secret-stdin
omnitron node update production --ssh-secret-stdin
```

On `node update`, the secret belongs to the node's registered authentication method unless
`--ssh-auth` explicitly changes it. Thus an existing key-authenticated node receives a key
passphrase without needing to repeat `--ssh-auth key`.

## Webapp (console UI)

| Command                      | Effect                                                                                       |
| ---------------------------- | -------------------------------------------------------------------------------------------- |
| `omnitron webapp build`      | Build the React + Vite bundle                                                                |
| `omnitron webapp start [-f]` | Start nginx container serving static + gateway (`-f, --force` recreates a running container) |
| `omnitron webapp stop`       | Stop the webapp nginx container                                                              |
| `omnitron webapp status`     | Show webapp status                                                                           |
| `omnitron webapp open`       | Open the webapp in the system browser                                                        |

All five commands support global `--json`. Build output includes `distPath` and
`durationMs`; the builder's subprocess output is captured. Start reports observed
`running`, `healthy`, `port`, `url` and `autoStart`; stop reports `stopped` and
`autoStart`. A null `autoStart` means no saved daemon configuration existed to
update. Open waits for the system browser launcher and reports `browserOpened`
and whether it started the Console first.

Lifecycle errors distinguish `outcome: "not-started"` (for example, the daemon
is absent), `"unknown"` (the container operation failed and may have made partial
changes), and `"applied"` (the Console action completed, but a subsequent step,
such as saving autostart preference or opening a browser, failed). An applied
outcome is not a rollback. Inspect `omnitron --json webapp status` before retrying.

## Knowledge base (MCP)

| Command                                | Effect                                   |
| -------------------------------------- | ---------------------------------------- |
| `omnitron kb mcp`                      | Start MCP stdio server for AI assistants |
| `omnitron kb index [--full] [--watch]` | Reindex the knowledge base               |
| `omnitron kb status`                   | KB index health and statistics           |
| `omnitron kb query <question>`         | Test a query against the KB              |

## Exit codes

| Code | Meaning                                         |
| ---- | ----------------------------------------------- |
| `0`  | Success                                         |
| `1`  | General failure (RPC error, config error, etc.) |
| `2`  | Daemon not reachable                            |
| `3`  | Authorization failed                            |

Combined with `--json` output, this is enough to drive any CI or
agent pipeline.

## Tips

- **`omnitron --json status | jq ...`** — pipe to jq for any
  scriptable check.
- **`omnitron logs my-app -f --grep error`** — narrow follow on a
  hot pattern.
- **`omnitron exec my-app users findById '"u_42"'`** — note the
  quoted JSON string for the arg.
- **`omnitron status` before `up`** — checking what's already
  running prevents accidental double-start.
- **`omnitron --json`** in agent workflows — never parse the
  styled output; it's not stable.

## See also

- [Architecture](/docs/omnitron/architecture) — what each component does
- [Daemon](/docs/omnitron/daemon) — what `up` / `down` actually do
- [Services reference](/docs/omnitron/services-reference) — the RPC surface every
  CLI command targets
- [Console](/docs/omnitron/console) — the GUI equivalent

## Releases

`omnitron release` builds and verifies artifacts from project and runtime
commits. See [Releases and verification](/docs/omnitron/releases) for build, attestation,
artifact-store transfer and retention commands.

Use `omnitron release status <project>` to see current stack states, node
reachability and application counts beside their last recorded deployments.
`--json` carries the same evidence. Exit `2` signals an incomplete answer,
including an unreported current release identity; it must not be treated as
"nothing deployed". `release list` and `show` describe controller artifacts
and audit history, not a claim that those artifacts are currently running.

`omnitron release status <project> --inspect-installed` adds an explicit read-only
inspection of each stack's retained archives, bootstrap and reported framework
build. It does not restart applications or change their launch evidence. Archive
SHA-256 and byte count identify every matching catalog release; modifications,
ambiguous matches and unavailable node backends stay visible. JSON keeps these
results in `installed`, separate from verified launch receipts and audit history.
A matching bootstrap is a partial check, not proof of all installed or loaded code.
The operation is available to viewers and bounded by node, archive and time limits.

## Additional operational commands

| Command                                                 | Behaviour and important options                                                                                                                                                                                                                                         |
| ------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `omnitron stack account <project> <stack>`              | Runs the project's account tool on a remote stack. `--username` creates an account and stores its generated password in the vault; `--show` and `--census` inspect it. `--remove` requires the account id. The project supplies its role model and tool implementation. |
| `omnitron stack maintenance <project> <stack> [action]` | Reads the gateway maintenance lock by default. `on --minutes 15` sets a bounded lock; `off` releases it.                                                                                                                                                                |
| `omnitron remote restart <alias> <app>`                 | Restarts the named application on a registered remote daemon.                                                                                                                                                                                                           |
| `omnitron fleet rollout import <runtimes...>`           | Admits sealed Linux builds under the controller account. Identical imports reuse the immutable catalog entry.                                                                                                                                                           |
| `omnitron fleet rollout builds`                         | Lists catalog commits, architectures, Node versions and digests. Supports `--json`.                                                                                                                                                                                     |
| `omnitron fleet rollout plan <commit> [nodes...]`       | Saves a batch and starts qualification. `--stage` distributes matching catalog builds. `--apply` submits the plan after every node qualifies. `--wait` follows qualification or activation. Use `--request-id <uuid>` for idempotent submission.                        |
| `omnitron fleet rollout apply <id>`                     | Starts the reviewed managed backend update in the planned order.                                                                                                                                                                                                        |
| `omnitron fleet rollout status [id]`                    | Reads the saved plan and each server's result. Use `--wait` to follow it.                                                                                                                                                                                               |
| `omnitron fleet rollout reconcile <id>`                 | Checks an uncertain result against current node evidence.                                                                                                                                                                                                               |
| `omnitron fleet upgrade [nodes...]`                     | Builds and installs Omnitron on registered nodes. Start with `--dry-run`; `--keep` controls retained versions. A dirty working tree requires `--allow-dirty`.                                                                                                           |
| `omnitron secret generate <key>`                        | Generates a vault secret without printing its value. `--bytes` accepts 16–1024, default 32; `--encoding` accepts base64, base64url or hex.                                                                                                                              |
| `omnitron secret copy <from> <to>`                      | Copies a value between vault keys without printing it.                                                                                                                                                                                                                  |
| `omnitron secret rotate-rpcauth <passwordKey>`          | Generates a paired RPC password and bitcoind authentication value. Requires `--user-key` and `--auth-key`.                                                                                                                                                              |
| `omnitron infra inspect <project/stack>`                | Inspects remote host services without changing them. Optional repeatable selectors: `--unit`, `--path`, `--snap`, `--config`. Credential configuration keys are refused.                                                                                                |
| `omnitron audit`                                        | Lists control-plane changes, newest first. Filter with `--action`, `--resource`, `--actor` and `--before`. Continue a page with `--before-id <entry-id>`; do not combine the two cursor options. `--limit` accepts 1–500, default 50.                                   |

Account cleanup modes belong to the project's tool: `--leftovers` performs a
census, `--leftovers rehearse` previews removal in rolled-back transactions,
and `--leftovers apply` requests committed removal. Inspect the census and
project policy before applying cleanup.

## Fleet recovery and host ingress

`omnitron fleet recover <node> <receipt>` asks the selected node to recover a
bundle activation from its receipt. Use the receipt produced by the relevant
operation; recovery is not a fresh release or a command to deploy arbitrary files.

`omnitron infra ingress` manages host-owned ingress independently of application
stacks. `status` reads configured and observed generation; `recover` recovers an
interrupted publication. Manifest-driven operations use the expected generation
for compare-and-swap checks, so a stale plan cannot overwrite a newer owner state.
Consult `omnitron infra ingress --help` for the available operations and supply
the JSON manifest belonging to the intended host and owner.

For a controlled maintenance window, preserve the owner's domains, TLS and
reserved ports and use `[{ "path": "/", "deny": 503 }]` for each route's locations.
Validate and apply before changing its services or static mounts. HTTPS returns
`503`, `Retry-After: 60` and `Cache-Control: no-store`; other owners keep their
routes. Retain the previous generation and maintenance owner revision for
`infra ingress restore`. Open traffic only after private readiness checks pass.
Restoring ingress changes routing, not application processes or data. Maintenance
persists across daemon restarts until an explicit publication or restoration.

## Console access and interactive SSH

`omnitron access` and `omnitron terminal` use the same services and policy as Console.

```bash
omnitron access show
omnitron access users
omnitron access create observer --role viewer --password-stdin
omnitron access role <id> operator --expect viewer
omnitron access revoke <id>
omnitron node ssh-key edge-1
omnitron node ssh edge-1
omnitron terminal list
omnitron terminal close <id>
```

See [Console access](/docs/omnitron/access) for password input, account pagination and atomic role changes. See [SSH workspace](/docs/omnitron/terminal) for independently verified host pins, session ownership and `Ctrl+]` disconnect.

| Command                   | Result                                                               |
| ------------------------- | -------------------------------------------------------------------- |
| `omnitron access show`    | Effective RPC permissions from the running daemon                    |
| `omnitron access users`   | Safe account directory, with continuation cursor                     |
| `omnitron access create`  | Create an account with a password read from a hidden prompt or stdin |
| `omnitron access role`    | Compare-and-swap role change and session revocation                  |
| `omnitron access revoke`  | Revoke an account's sessions                                         |
| `omnitron node ssh-key`   | Inspect or explicitly trust a host fingerprint                       |
| `omnitron node ssh`       | Interactive shell through Xec                                        |
| `omnitron terminal list`  | List sessions owned by this caller                                   |
| `omnitron terminal close` | Close an owned terminal                                              |

## File workspace

`omnitron files` shares the audited Console file service. Commands include `roots`, `archive-formats`, `list`, `stat`, `read`, `save`, `mkdir`, `rename`, `trash`, `trash-list`, `restore`, `copy`, `transfers`, `pause`, `resume`, `cancel`, `upload`, `download`, `upload-status`, `upload-finish` and `upload-abort`. All support `--json`; checked mutations use an observed `--version`. For one directory, `copy --archive-format <format> --archive-output archive|directory --archive-name <name>` chooses an archive file or restored folder explicitly; `archive-formats --node <source> --to-node <destination>` probes the node pair. See [File workspace](/docs/omnitron/files) for flags, examples, ownership, limits and recovery.

Stack stop/status JSON includes a durable `stopOperation` with per-node confirmation.
A lost reply is unconfirmed, and CLI exits unsuccessfully until every required node
acknowledges the stop. See [confirmed stack stops](/docs/omnitron/projects#confirmed-stack-stops)
for retries, existing allocation bindings and upgrade prerequisites.

---

---
id: "omnitron/cluster"
title: "Cluster + Fleet"
description: "Multi-node Omnitron — leader election, state replication, fleet operations."
---

# Cluster + fleet

When you run Omnitron on more than one node, two subsystems
coordinate them: **cluster** (leader election, control-plane
consistency) and **fleet** (node inventory, cross-node queries).

This page covers both. Single-node operators can skip — none of
this runs without `cluster.enabled: true` or fleet nodes
registered.

Verified against `apps/omnitron/src/cluster/` and
`apps/omnitron/src/services/{fleet,sync,node-manager}.rpc-service.ts`.

## Two roles, two layers

| Layer      | Concept                                        | Service id                       |
| ---------- | ---------------------------------------------- | -------------------------------- |
| Cluster    | Leader / follower among a small set of daemons | `OmnitronCluster`                |
| Fleet      | Node inventory, cross-node fleet ops           | `OmnitronFleet`, `OmnitronNodes` |
| State sync | Daemon-to-daemon batch replication             | `OmnitronSync`                   |

A daemon in `role: master` is the control plane. A daemon in
`role: slave` runs autonomously and syncs back to master via
`OmnitronSync`. Optional cluster coordination observes the current database lease holder via
`OmnitronCluster`; it does not automatically transfer application ownership.

## When to use what

| Setup                                                          | Use                                                                                      |
| -------------------------------------------------------------- | ---------------------------------------------------------------------------------------- |
| Single machine, no HA needed                                   | Neither — default `role: master`, no cluster                                             |
| Multiple machines, no leader concept (each daemon independent) | Fleet — register nodes, query across them                                                |
| Multiple controllers requiring a shared authority observation  | Optional PostgreSQL lease coordination; external effects still require recipient fencing |
| Multi-region with autonomous edges syncing to a central master | Fleet + master-slave role pattern (no cluster needed)                                    |

## PostgreSQL-authoritative coordination

`LeaderElection` now uses the optional
`@omnitron-dev/titan-database/coordination` provider. A row in
`titan_database_leases` records the controller identity, a fresh incarnation for
each process lifetime, a durable decimal epoch and the expiry measured by PostgreSQL.
Migration `016_cluster_lease` creates this table through Omnitron's migration facade.
The provider does not create tables during startup.

This replaces the former peer-election implementation. `requestVote` and
`leaderHeartbeat` are no longer RPC methods. A remote caller cannot announce itself
as leader, change a term or suppress elections. All remaining cluster RPC methods
require authenticated authority.

Acquisition and renewal commit synchronously. Contenders serialize through the
same database row; releasing a lease preserves the epoch rather than deleting it.
A new owner receives a higher epoch. An expired grant cannot be renewed, and an old
owner cannot release its successor's lease. Decimal strings preserve epochs above
JavaScript's safe integer range.

PostgreSQL supplies [conflict-aware insertion](https://www.postgresql.org/docs/current/sql-insert.html),
[row locking](https://www.postgresql.org/docs/current/explicit-locking.html) and
[the database clock](https://www.postgresql.org/docs/current/functions-datetime.html).
All contenders must use the same writable database. Its recovery/failover policy
must preserve committed epochs. This is database-backed coordination, not an
independent Raft quorum or a replicated Omnitron command log.

### Authority and effect fencing

A worker admits a managed stack stop only after verifying the controller's Ed25519
signature and reading the current lease on the writable PostgreSQL primary. The
signed plan binds the enrolled cluster and node, source project and stack, owner
revision, application generations, immutable container IDs and expiry. The worker
remembers the highest accepted epoch in its durable state. It checks authority
again between application shutdown, infrastructure retirement and container removal.
A lost lease can leave a partial stop. Inspect the per-node outcome and obtain a fresh
plan from the current leader before continuing. Work admitted before a transfer can
finish. An unreachable node stays unconfirmed.

Use `PostgresLeaseStore.withLease` for effects inside the same PostgreSQL transaction.
The recipient protocol supports `stack.stop` and `stack.resume-installed`.
Recovery binds the last node stop, application generations, retained release, launch
configuration and placement to a new signed operation. Each node checks the live
lease before opening its owner and before dispatching each launch. A launch already
admitted may finish across a leadership transfer. Later launches require current
authority. A recovery plan has a five-minute admission window. New deployments and infrastructure provisioning need separate recipient
adapters. See [Resume installed applications](/docs/omnitron/projects#resume-installed-applications). Stateful failover additionally
needs a fence enforced by the data service. See the [delivery gates](/docs/omnitron/release-readiness).

The controller bounds local freshness from **before** its database request, using
monotonic time and a wall-clock expiry guard for host suspension. A delayed reply
cannot start a new full lifetime. Database failure, lost renewal or expired
confirmation makes the controller a follower. Pending requests cannot revive it
after shutdown or an operator-requested step-down.

### Enroll a controller or worker

Apply Omnitron migration `016_cluster_lease` to the shared authority database.
Configure the daemon service environment with `OMNITRON_CLUSTER_DATABASE_URL`.
Give controllers `SELECT`, `INSERT` and `UPDATE` on `titan_database_leases`.
Workers require only `SELECT`. Remote connections require `sslmode=verify-full`.
Use a database failover policy that preserves acknowledged lease writes.

Set the following fields in the instance's private `$OMNITRON_HOME/config.json`.
Use the same cluster ID across participants and a unique, stable recipient ID per
installation. Pin each target's matching `nodeId` in its stack topology. A node's
address can change independently of the identity that accepted an allocation.

```json
{
  "cluster": {
    "enabled": true,
    "clusterId": "west",
    "recipientId": "edge-01",
    "discovery": "postgres",
    "leaseTtlMs": 15000,
    "heartbeatInterval": 2000
  }
}
```

The two timing fields are optional. Each lease must cover at least three heartbeat
intervals. Controllers also need a local fleet identity for election. A worker's
durable enrollment rejects a missing configuration or a different identity on restart.
Ordinary single-controller fleets can use the default disabled mode.

## `OmnitronCluster` RPC service

| Method              | Auth                                           | Effect                                                                                 |
| ------------------- | ---------------------------------------------- | -------------------------------------------------------------------------------------- |
| `getClusterState()` | viewer/operator/admin or control-plane service | Coordination, authority reason, epoch, observed leader and lease freshness             |
| `stepDown()`        | administrator                                  | Demote locally, release this epoch, and wait one lease interval before competing again |
| `isLeader()`        | viewer/operator/admin or control-plane service | Whether this controller currently has a confirmed fresh grant                          |

```bash
omnitron cluster status
omnitron cluster step-down
```

The status includes `coordination: 'postgres-lease'`, `authority`, `reason`,
`leaseExpiresAt`, `leaseRemainingMs`, `effectFencing: 'recipient-required'` and
the recipient operations supported by this backend in `fencedOperations`.
Use `omnitron --json cluster status` in automation. `participant` distinguishes a
controller, enrolled worker and standalone daemon. Workers report their enrolled
identity. A legacy peer without the diagnostics service yields unknown participation
and a nonzero CLI exit status. The recovery preflight checks the process owner's
capabilities and retained evidence separately.
`term` is a decimal epoch string, or `null` when no authority has been observed.
An expired observation has no `leaderId`. The former `votedFor` field is removed.
CLI reports the same authority and reasons returned to Console's node diagnostics.

If a lease release cannot be confirmed, step-down reports failure; the local
controller remains demoted and the database lease expires normally. It never
claims that another controller has already taken over. Process shutdown also
invalidates pending acquisition/renewal results before releasing authority.

### Upgrade boundary

Prepare the additive schema expansion and retain a verified rollback artifact
before replacing a backend. Existing master/slave processes do not need cluster
coordination enabled. Do not mix legacy peer-election controllers and lease
controllers in an enabled control plane: disable the legacy election on all of
its participants before admitting the new coordination mode. This change alone
neither transfers running workloads nor proves a stateful failover safe.

## Config sync — `ConfigSyncService`

The daemon creates a configuration hash tracker when cluster startup succeeds on a controller.
The tracker exposes comparison and synchronization helpers. Automatic distribution
of ecosystem files between controllers is not wired into daemon startup. Provision
reviewed declarations on every controller independently.

## `OmnitronFleet` RPC service — node inventory

Distinct from clusters. A fleet is any set of registered nodes;
they don't need leader election.

| Method                       | Auth     | Effect                        |
| ---------------------------- | -------- | ----------------------------- |
| `listNodes()`                | viewer   | All registered fleet nodes    |
| `getNode({nodeId})`          | viewer   | One node by id                |
| `getSummary()`               | viewer   | Aggregate health, counts      |
| `registerNode(registration)` | operator | Add a node to the fleet       |
| `removeNode({nodeId})`       | operator | Remove                        |
| `setRole({nodeId, role})`    | operator | Promote / demote              |
| `drainNode({nodeId})`        | operator | Drain workloads off this node |

### `FleetNode`

```typescript
interface FleetNode {
  id:       string;
  hostname: string;
  address:  string;
  role:     'master' | 'slave';
  status:   'online' | 'offline' | 'draining';
  lastSeen: number;
  // ... metadata (CPU/RAM snapshots, app counts)
}
```

### Fleet liveness

The master scheduler updates its own registered row at the configured health-check interval. Remote health and connectivity come from checks against registered nodes and authenticated mesh observations. A caller cannot update an arbitrary node’s liveness by supplying its ID.

The fleet's offline-detection threshold is driven by
`DEFAULT_DAEMON_CONFIG.healthMonitor.offlineTimeoutMs`
(`90_000 ms` default).

## `OmnitronNodes` — extended node management

The fleet service holds the live registry; `OmnitronNodes` holds
the **registered inventory** (declared nodes, SSH credentials,
health history). They overlap but serve different purposes.

Key method categories:

- **Inventory**: `listNodes`, `getNode`, `addNode`,
  `updateNode`, `removeNode`.
- **Connectivity**: `checkNodeStatus`, `checkAllNodes`,
  `triggerNodeCheck`.
- **History**: `getCheckHistory`, `getUptimeBar`,
  `getNodeHealthSummaries`.
- **SSH**: `listSshKeys`.
- **Check config**: `getCheckConfig`, `setCheckConfig`
  (ping/SSH/omnitron timeouts).

### Uptime bars

`getUptimeBar({nodeId, bucketCount?, intervalMs?})` returns
historical health bucketed into time segments. The webapp uses
this for the green/yellow/red availability bars.

| Param         | Default (RPC method)         |
| ------------- | ---------------------------- |
| `bucketCount` | `60`                         |
| `intervalMs`  | `300_000` (5 min per bucket) |

Each bucket reports ping / omnitron uptime as a `0.0–1.0`
percentage. (The daemon's health-monitor worker keeps a longer
retention window — `retentionDays: 90`, `uptimeIntervalMs:
86_400_000` — but those govern check storage, not this method's
default windowing.)

## State sync — `OmnitronSync` service

The master-slave replication mechanism. Slaves locally collect
metrics, logs, traces and alert events. The master pulls batches over the
authenticated Netron connection every 15 seconds, ingests them, then acknowledges
the accepted, duplicate or explicitly discarded IDs. Failed entries remain pending.

| Method                        | Effect                                                                                     |
| ----------------------------- | ------------------------------------------------------------------------------------------ |
| `drainBuffer({limit?})`       | Slave-side: pull pending data without waiting for periodic sync                            |
| `ackDrained({ids, receipt?})` | Atomically release IDs and record their committed, discarded or unverified outcome         |
| `getSyncStatus()`             | Measured queue bytes/age, durable receipt history, contiguous progress and recorded losses |

### Connection credentials

For registered nodes, the controller always opens an Xec SSH tunnel to the daemon and presents its bearer token only to the numeric loopback listener. A reachable public TCP port alone permits a liveness probe. The built-in mesh dialer mints a five-minute `service_role` credential for each node. It renews that credential on the same Netron peer 30 seconds before expiry, coalescing concurrent calls and heartbeat reads. The SSH tunnel and running operations remain in place. The node independently checks the signed identity and its expiry on every invocation.

Credential renewal has a ten-second deadline. Failure prevents dispatch of the waiting operation and reconnects with a fresh credential; it does not replay an application mutation. A rejected signature clears the cached node signing secret, while a transport timeout does not claim the secret is wrong. Mesh status exposes `credentialExpiresAt`, `credentialRefreshedAt` and `lastCredentialError`, without credential material. CLI and Console distinguish an expired identity from a healthy authenticated connection.

Update the master before nodes when introducing per-call token expiry: older masters cannot renew an admitted peer's identity. A custom dialer can provide `refreshCredential` and `expiresAt`; non-renewable credentials require a new connection after expiry. A plain unauthenticated connection can probe liveness but cannot replicate.

Mesh authentication requires the remote daemon to confirm a `service_role` principal with a finite expiry. A ping proves reachability. Reading a release version for admission, polling an outbox and issuing commands all require the authenticated peer. Multiple registered node IDs pointing to the same daemon address leave that address unavailable for control until the registration is unambiguous.

Stack deployments use the same registered daemon host, port and optional node ID as the mesh. A registry or trust-store failure stops deployment before SSH dispatch.

For SSH bootstrap, compare the observed host fingerprint with an independently trusted copy before approving it. Automation uses the registered pin for the exact host, SSH port and user. Without a pin, Xec requires an existing matching entry in the controller account's `known_hosts`. A new machine needs an explicit trust decision before it can receive credentials. [Inspect and approve a host key.](/docs/omnitron/terminal#verify-the-host-before-connecting)

### Sync config — `ISyncConfig`

```typescript
interface ISyncConfig {
  interval?:  number;     // 30_000 ms between worker outbox maintenance runs
  batchSize?: number;     // 1_000 entries when a drain request omits its limit
}
```

Designed for **eventually consistent** replication:

- Slaves buffer enabled data categories in local SQLite WAL with `synchronous=FULL`. The configured byte budget can evict the oldest undelivered entries; these losses are recorded.
- Master pull resumes after authenticated reconnect; failed entries remain buffered.
- Connection retry uses exponential backoff; application processes continue independently.
- Idempotent batches — safe to retry.
- Conflict resolution: master timestamp wins for config; slave
  data is append-only.

### Sync flow

```mermaid
sequenceDiagram
  participant Slave
  participant Buffer as Slave buffer
  participant Master

  loop continuously
    Slave->>Buffer: append metric/log/trace/alert
  end
  loop master heartbeat (15s default)
    Master->>Slave: drainBuffer({limit: 1000})
    Slave->>Buffer: read pending entries
    Slave-->>Master: batch (entries remain pending)
    Master->>Master: ingest with deduplication
    alt confirmed outcome
      Master->>Slave: ackDrained({ids, receipt})
      Slave->>Buffer: acknowledge confirmed IDs
    else failure
      Master->>Master: retain diagnostic cause<br/>retry later
    end
  end
```

If the master is offline, slave applications continue independently. Replication
retains data up to the configured buffer budget. When the master returns, pending
data drains on subsequent sweeps; the status also preserves evidence of any loss.

### Reading connection, backlog and delivery separately

`omnitron node check <id>` and the Console node details use the same interpretation.
A connected RPC channel proves reachability, not replication. A fresh status is
required before the master reports progress. An empty queue is drained; a recent
acknowledgement with a young queue is keeping up. Old pending data is catching up,
and a queue without a recent acknowledgement is behind. Durations are measured
on the source node so host clock skew does not invent lag.

New nodes report exact UTF-8 payload bytes. `bufferSize` covers pending payloads;
`retainedBufferSize` also includes released entries awaiting retention. Neither
is the filesystem size of SQLite, its indexes or WAL. A singleton maintained by
transactional triggers plus indexed seeks supplies status without scanning the
whole queue on each poll. Legacy schemas use a measured fallback.

The optional `evidence` object survives daemon replacement and normal retention:

| Field                              | Meaning                                                                                                 |
| ---------------------------------- | ------------------------------------------------------------------------------------------------------- |
| `epoch`, `since`, `firstTrackedId` | Scope of the recorded history; no guarantee is inferred for earlier data                                |
| `producedThrough`                  | Highest local sequence generated in this buffer                                                         |
| `releasedThrough`                  | Contiguous sequence no longer pending; may include discards or evictions                                |
| `committedThrough`                 | Contiguous durable payload receipts from `firstTrackedId`; stops at a pending, unverified or lost entry |
| `committedRows`                    | Entries confirmed by an initial database commit or later payload-bound reconciliation                   |
| `discardedRows`                    | Permanently rejected entries released after recording the refusal                                       |
| `unverifiedRows`                   | Released entries without durable payload proof                                                          |
| `evictedRows`                      | Undelivered entries removed by local buffer limits                                                      |
| `lastCommittedAt`, `lastLossAt`    | Persisted timestamps for confirmed commits and known loss                                               |
| `reconciliation`                   | Later `verifiedRows`, `restoredRows`, `lastReconciledAt` and proof scope (`master` or `source`)         |

A zero `unverifiedRows` count covers this recorded epoch from `firstTrackedId`.
Older buffered rows can have `syncOutcome = NULL` because their receiver predates
receipt accounting. That absence does not establish either persistence or loss;
they are outside this reconciliation frontier and are not silently certified.

These sequence fields are strings, preserving integer precision. The frontier
before the first tracked entry means no contiguous receipt has been proven yet.
Counters apply to the stated tracking epoch. Losses remain visible even when the
queue becomes empty; a restart or retention pass must not erase them.

The master persists each receipt's outcome in the same transaction as its data.
A lost reply and a repeated batch retain that outcome: a previous discard never
becomes a committed duplicate. If recording a discard fails, the source keeps
that entry pending. Starting with migration `015_persistent_metrics`, metrics are
written through Titan's PostgreSQL adapter in that same transaction. Successful
receipts therefore prove both the payload and its ledger entry committed. The
original measurement timestamp and authenticated source node are preserved.

On updated slave daemons, Titan's SQLite adapter writes chart history and the
replication outbox in one transaction. A failed outbox write rolls both back;
retries with the same sample IDs do not enqueue duplicates. This replaces the
polling watermark that could skip delayed samples. Chart retention never removes
undelivered outbox entries.

Historical `unverifiedRows` are not automatically promoted by an upgrade. An older
in-memory destination may have received values without persisting their original
samples. Older nodes can receive new durable receipts immediately, but need their
own backend update to gain the atomic local outbox. Buffer limits and known loss
counters continue to apply; see [metrics persistence diagnostics](/docs/omnitron/observability#metric-persistence-and-confirmation).

### Reconcile historical metric receipts

Use `omnitron --json node reconcile <node>` to inspect retained, released metric
entries. Add `--restore` when repairing the identified history is intended.
Pages are bounded by entries and bytes, fix the source epoch/frontier, and return
a keyset cursor. Normal collection and replication continue independently.

A sample matches only when its metric name, application, source node, labels,
original timestamp and value match. Resampled charts are not exact source samples.
Reusing an identity with different content fails the transaction. Missing history
is restored using the same stable sample identity and Titan PostgreSQL adapter as
normal ingestion. Restoring an old point does not replace the latest gauge value.

The master's `sync_reconciliations` journal records a new `present` or `restored`
proof, payload hash, sample identity, source epoch, original ledger outcome,
receipt ID and verification time. Restored samples, proofs and summary counters
commit together. The original `sync_ingested` claim is preserved: a repair today
does not prove that the legacy receiver committed data at its original delivery.

On the source, proof must match the still-retained immutable payload and epoch.
Its journal, the decrement of `unverifiedRows` and increase of `committedRows`
commit in one SQLite WAL/FULL transaction. The acknowledgement reserves the
SQLite writer before reading the source snapshot, so concurrent live ingestion
cannot invalidate a deferred read-to-write upgrade. Original `syncOutcome` and `syncedAt`
are unchanged. Repeated pages and lost responses cannot count or restore an entry
twice. Retry the same cursor after failure; no queue reset or database restore is
needed. Other categories, pruned payloads, rejected entries, conflicting content
and another epoch cannot be certified by this metric recovery path.

The optional journal has its own checksummed `sync_reconciliation_migrations`
history, using the public Omnitron migration runner through Titan/Kysera. It adds
two tables and leaves the core registry unchanged. A reviewed worker built with
the same code can reconcile an older node alongside its running daemon, without
changing application ownership or service units. Pin and verify the worker, use
the node's existing native dependencies, and test reopening its database with the
older runtime first. The normal CLI uses RPC and never installs a worker implicitly.

Restore respects `monitoring.metrics.historyMaxAge`; already committed data missing
after retention is not silently resurrected. A proof records a past commit, not
indefinite payload retention or replica/backup availability. Proof journals retain
hashes and identities independently of chart and queue retention, without full
payloads. While any unresolved historical gap or recorded loss remains, the
contiguous frontier is conservative and can stay at its former boundary. It
advances beyond that uncertainty only when all gaps are settled and no known loss
remains. CLI and Console distinguish verified/restored counts from unresolved
history; an unreadable proof journal remains an explicit diagnostic error.

### History before delivery receipts

Data before `firstTrackedId` has no original durable delivery proof. The ordinary
`node reconcile` command deliberately stays inside the tracked frontier. A
separate retained-history audit can establish what exists now without rewriting
the original claims or moving that frontier backwards.

Use this order when recovering legacy history:

1. Preserve consistent SQLite snapshots on their originating nodes before queue
   retention removes more evidence. Inventory older snapshots and PostgreSQL
   backups; exclude validation databases and synthetic smoke-test records.
2. Verify overlapping source identities against their complete content. Union
   repeated snapshots by the original buffer sequence or log UUID. Record source
   file hashes, exact time bounds, row counts and any missing sequence ranges.
3. Inspect bounded pages against the destination. A metric must match its original
   name, value, timestamp and normalized labels. A resampled gauge is not an
   original measurement. Old SQLite logs retained milliseconds rather than their
   transmitted microsecond sequence: compare at the retained precision, including
   all structured fields and the number of equal records.
4. Restore only missing eligible records. `reconcileHistoryPage` uses Titan's
   PostgreSQL transaction adapter; payloads, page proofs and progress counters
   commit together with durable PostgreSQL settings. Keep the authenticated
   transport, native runtime and memory/page budgets explicit in the reviewed
   maintenance worker. The daemon never installs or executes one implicitly.
5. Repeat a read-only content comparison after recovery. Preserve snapshots and
   the audit report, including any unresolved range or precision limitation.

Raw files can supply additional events absent from database snapshots. Preserve
the original files on their host, including a fixed byte range for active files.
Import only records with a trustworthy absolute timestamp and unambiguous event
identity. Check both full structured content and the existing event's time and
message before inserting: a metadata mismatch is an explicit unresolved case,
not permission to add a duplicate or overwrite the old row. Keep undated lines
and incomplete multi-line records archived; a physical-line count is not a count
of lost events.

A time/message collision can be resolved only with additional identity evidence.
For example, two adjacent lifecycle records may name different `workerId` values
at the same millisecond. Compare the original lines, their hashes and all stored
fields before treating them as two events. A different metadata value alone does
not establish whether the event is missing or the same event was transformed.

Record a reviewed recovery as a new `HistoryManifest` with `resolvesGap`:
`sourceId`, `gapIndex`, `gapDigest` (from `historyGapDigest`), `evidenceDigest` and
a concise `explanation`. The reference must match an already verified source on
the same node and category, with a known positive gap count equal to the new
source's expected count. The original manifest and page proofs remain unchanged.
Unknown counts, changed fingerprints, competing recoveries and expired records
cannot close a gap. Only the final transaction with every expected row and the
correct hash chain exposes the separate recovery proof. These are reviewed
maintenance operations, not a new RPC that accepts arbitrary resolution flags.

`sync_history_sources` stores immutable manifests and progress;
`sync_history_pages` stores ordered page hashes and commit results. Their separate
checksummed migration history uses the Omnitron/Titan/Kysera runner. Pages form a
hash chain whose final digest and total row count must match the source manifest.
A lost response can replay the same page; changed data, a broken chain or a
conflicting identity fails the transaction. A partial import remains explicitly
partial. Normal ingestion, live gauges, source counters and application ownership
are unaffected.

Inspect coverage with `omnitron node check <node>` or expand **History before
receipts · coverage and gaps** in the Console node diagnostics. Both read the same
`history` DTO: source time bounds, checked/expected rows, present/restored counts,
verification time, retained timestamp precision and explicit gaps. A recovered
gap displays its explanation, verification time and separate source ID under
`resolved`; unresolved gaps remain visible. A verified
source range does not certify earlier history. An unknown number of missing rows
stays unknown; an empty queue cannot turn it into zero.

Respect the configured retention policy during repair. Metric recovery refuses
points beyond its history horizon. Missing logs outside the selected log horizon
are counted as `expiredRows` and remain in the preserved source archives; they
are not reinserted just to be pruned again. Commit proofs describe an audit at a
particular time, not permanent retention of every payload. Database backups can
independently prove existing log UUIDs and their full content without creating a
second copy in the live table.

### Reproducible native node bundles

Prepare native daemon updates beside the running version. A bundle that includes `package-lock.json` is installed with `npm ci --omit=dev`; the installer refuses any change to that lock before publishing the candidate directory. Older bundles without a lock retain their installation path, but their transitive dependency versions are not frozen. Include and validate a lock when preparing a repeatable fleet rollout.

Installation and activation are separate. Installation verifies the archive hash, installs into a fresh sibling, executes its CLI, and records the resulting dependency tree. Reusing that version verifies its complete installation receipt. It does not repair a running release in place. Native dependencies must be built and tested against the target's OS, architecture and Node runtime; a macOS frozen runtime is not a Linux bundle.

A daemon that still owns application processes needs a controlled restart for a full node update. Capture the exact running and stopped application inventory, project boot policy, current framework version, service unit, private state and replication queue first. Each application must declare whether its dependencies are independent or provide a reversible framework-binding contract. Verify both the candidate binding and restoration of the previous binding on isolated copies before maintenance. Preserve application releases, credentials and data during rollback; never restore an old replication database over entries collected after the backup.

Before activation, Omnitron reads the on-disk bootstrap declarations of previously online applications with the candidate’s isolated reader, the serving Node executable, preload arguments, working directory and private environment. A failed read refuses the transition before any process is stopped or dependency is rebound; CLI and Console receive the same upgrade failure. The environment stays on the node and is not written into inventory or recovery receipts. Identical bootstrap paths are checked once, offline applications are not imported, and classic applications remain covered by their binding and health contracts. The read has a two-minute total deadline. This validates the loader against the current application bindings; it does not replace testing the candidate bindings on isolated copies.

A declaration read does not import every worker module. Before switching an isolated or pinned application runtime, verify its complete physical npm dependency graph as well as package hashes and native addons. A dependency declared as `process`, `buffer` or `events` is an npm package even though Node has a builtin with the same name; a manifest that substitutes the builtin can pass a hash check while failing on an import such as `process/`. Keep each registration’s verifier and dependency snapshot with its artifacts, and reject incomplete snapshots before stopping applications.

Update one node at a time. Verify every previously running application's health, RPC authentication and replication progress before moving to the next node. A running daemon alone does not satisfy this check. The [backend handoff workflow](/docs/omnitron/backend-handoff) becomes available only after a separately prepared resident-owner transition; a normal bundle upgrade does not silently change process ownership or the system account.

### Rolling upgrades and database replacement

Upgrade the master and nodes independently. New masters attach a versioned receipt
with `committedIds` and `discardedIds`; old nodes ignore the extra field. New nodes
still accept old acknowledgements, recording them as unverified. A worker that only pushes to a custom receiver needs to support authenticated pull before upgrading its controller. Check for a fresh mesh status and a new receipt before advancing the rollout.

Existing pending IDs stay unchanged during the SQLite migration. Newly appended
entries use a persistent random buffer epoch and sequence on the wire. A new local
database therefore cannot collide with the previous database's IDs in the master
ledger. A receipt from another epoch is refused. Restoring a database snapshot is
not equivalent to creating a new database: restore its entire state together and
coordinate replay with the master. Do not copy tables or reset sequences manually.

SQLite evidence uses the public Omnitron migration runner, backed by Titan and
Kysera. Schema changes and migration history commit together. The migration is
additive and has no destructive automatic rollback. Older masters can acknowledge
new node batches, but their receipts remain unverified. After a node starts emitting
epoch-qualified IDs, downgrading its binary requires an explicit compatibility
plan: an older producer would revert to numeric IDs and could resend a pending
entry under a different ledger key. Backend compatibility checks cover both
master and local-node migration registries. Master migration `014_replication_receipts`
marks historical outcomes unverified because previous discards cannot be inferred
reliably. The deduplication ledger has a retention window; it is not an unlimited
exactly-once archive.

## Multi-region patterns

Use a distinct cluster ID and authority database per independent region. A
single cross-region control plane requires every participant to reach the same
writable primary within its lease budget. Independently administered nodes can
use fleet inventory and replication without enrolling in a cluster.

## Connectivity loss and database failover

A partitioned controller loses its usable authority when renewal fails or the
local deadline passes. A worker that cannot read the authoritative primary
refuses a new stop effect. A readable standby cannot authorize a stop.
After a database recovery, compare the restored lease history with the epochs
accepted by recipients. An older epoch or a conflicting incarnation at the same
epoch requires reconciliation. Keep the existing application owners and inspect
the node receipts before retrying a partial operation.

## Operations

| Goal                                       | Command                                                                |
| ------------------------------------------ | ---------------------------------------------------------------------- |
| Check cluster state                        | `omnitron cluster status`                                              |
| Release current leadership (administrator) | `omnitron cluster step-down`                                           |
| Register a node                            | `omnitron node add --name N --host H ...`                              |
| Check all nodes                            | `omnitron node check`                                                  |
| Drain a node                               | `omnitron --json` then `OmnitronFleet.drainNode` (no CLI shortcut yet) |
| Per-node connectivity                      | `omnitron node check <id>`                                             |
| Fleet status                               | `omnitron fleet status`                                                |
| Fleet health                               | `omnitron fleet health`                                                |
| Fleet metrics                              | `omnitron fleet metrics`                                               |

## Anti-patterns

- **Treating an observed leader as authorization for a later effect.** Require
  a current epoch at the database transaction or remote recipient.
- **Independent databases for controllers in one coordination namespace.** All
  contenders must use one writable authority whose recovery preserves epochs.
- **Mixing legacy peer voting with PostgreSQL leases.** Transition all enabled
  participants through a reviewed plan before admitting coordinated work.
- **Deleting expired lease rows.** This resets fencing epochs; release ownership
  without deleting its history.
- **Unbounded pool/lock waits.** Configure finite pool acquisition deadlines;
  lease operations already bound SQL statements and lock waits.
- **Treating `OmnitronSync` as low-latency RPC.** It's batch
  replication with a nominal 15 s master pull cadence. For real-time
  cross-daemon coordination, use direct RPC to the relevant
  service.
- **Editing a slave's `state.json` and expecting it to propagate.**
  Slaves are downstream of master config. Edits get overwritten on
  next sync.

## See also

- [Architecture](/docs/omnitron/architecture#cluster-subsystem) — where
  cluster fits
- [CLI](/docs/omnitron/cli#cluster-leader-election) — `omnitron cluster ...`
- [Services reference](/docs/omnitron/services-reference) — `OmnitronFleet`,
  `OmnitronSync`, `OmnitronNodes`, `OmnitronCluster`
- [Daemon](/docs/omnitron/daemon#daemon-configuration) — cluster config fields

---

---
id: "omnitron/configuration"
title: "Configuration"
description: "Every field of omnitron.config.ts, defineEcosystem(), defineSystem()."
---

# Configuration

One repository is one [project with many stacks](/docs/omnitron/projects). Use the root
configuration for shared application definitions; additional configurations can
contribute named stacks without creating additional projects.

Omnitron is driven by two files:

| File                                     | Purpose                                                 | Imports                    |
| ---------------------------------------- | ------------------------------------------------------- | -------------------------- |
| `omnitron.config.ts` (repo root)         | **Ecosystem** — what apps exist, stacks, infrastructure | `defineEcosystem({ ... })` |
| `apps/<name>/src/bootstrap.ts` (per-app) | **System** — what processes the app runs                | `defineSystem({ ... })`    |

The ecosystem exports data or a factory as `default`; the bootstrap exports its
system definition. The ecosystem loader checks the selected project directory for
`omnitron.config.ts`, then `.js`, then `.mjs`. An explicit file can be selected with
`loadEcosystemConfigFile`. The daemon reads each app's bootstrap at launch.

Verified against `apps/omnitron/src/config/` (types.ts, define-ecosystem.ts,
define-system.ts, defaults.ts).

## `defineEcosystem(config)` — `omnitron.config.ts`

```typescript
import { defineEcosystem } from '@omnitron-dev/omnitron';

export default defineEcosystem({
  project: 'my-monorepo',
  apps:    [/* IEcosystemAppEntry[] */],
  stacks:  { /* StackName → IStackConfig */ },
  infrastructure: { /* ... */ },
  gateway: { /* ... */ },
  supervision: { /* ... */ },
  monitoring:  { /* ... */ },
  logging:     { /* ... */ },
});
```

### Factories and fresh configuration reads

An async factory can separate build inputs from runtime secrets:

```typescript
import { readFile } from 'node:fs/promises';
import { defineEcosystem } from '@omnitron-dev/omnitron';
import type { EcosystemPhase } from '@omnitron-dev/omnitron/config';
import { selectedApplication } from './selected-registration.js';

export default async ({ phase }: { phase: EcosystemPhase }) => defineEcosystem({
  apps: [{
    name: 'api',
    ...selectedApplication,
    ...(phase === 'runtime' && {
      env: { API_TOKEN: (await readFile(new URL('./.local/api-token', import.meta.url), 'utf8')).trim() },
    }),
  }],
});
```

`runtime` is the default phase. The ordinary release builder supplies `release`;
it does not need to read the example's token. The example's selected application
must still declare a bootstrap, script or exact release reference. Build inputs
and imported registrations participate in [source admission](/docs/omnitron/releases).

Every read evaluates the actual file and its full native import graph in a new,
short-lived process. Changing a transitive import takes effect on the next reload;
there is no retained ESM or CommonJS cache from an earlier selection. TypeScript,
`import.meta.url`, `createRequire`, the invoking working directory and the current
environment preserve their normal meanings. Relative app paths are subsequently
anchored to the configuration directory; an explicit absolute `cwd` is retained.

The factory returns plain declarative objects, arrays and finite primitive values.
Optional `undefined` values are preserved. Functions inside the result, accessors,
class instances, cycles, symbols and bigint values are refused rather than silently
removed or coerced. Put executable lifecycle hooks in `defineSystem`, not in the
ecosystem result. A factory must not start services or rely on changing the
controller's `process.env`: its process exits after the read. Configuration remains
trusted executable code with filesystem and network access; this isolation controls
freshness and lifetime, not permissions.

```typescript
import { loadEcosystemConfig, loadEcosystemConfigFile } from '@omnitron-dev/omnitron/config';

const config = await loadEcosystemConfig(projectDirectory, 'runtime', { timeoutMs: 30_000 });
const build = await loadEcosystemConfigFile(configFile, 'release', { timeoutMs: 30_000 });
```

`EcosystemLoadOptions.timeoutMs` defaults to 60 seconds and accepts a positive value
up to ten minutes. Definition and ecosystem reads use the same child lifecycle:
the caller waits for the result and process exit, a deadline kills the reader, and
parent disconnection ends a pending asynchronous read. Output is private and
limited to 1 MiB; the returned data is limited to 16 MiB. Failures retain a bounded,
credential-redacted reason and cause. Import output does not contaminate CLI JSON.

### Top-level fields — `IEcosystemConfig`

| Field            | Type                                                            | Required | Default                                         |
| ---------------- | --------------------------------------------------------------- | :------: | ----------------------------------------------- |
| `project`        | `string`                                                        |          | —                                               |
| `apps`           | `IEcosystemAppEntry[]`                                          |     ✓    | —                                               |
| `stacks`         | `Record<StackName, IStackConfig>`                               |          | —                                               |
| `infrastructure` | `InfrastructureConfig`                                          |          | —                                               |
| `gateway`        | `{ port?, configDir?, image? }`                                 |          | port `8080`, image `openresty/openresty:alpine` |
| `supervision`    | `{ strategy, maxRestarts, window, backoff }`                    |     ✓    | one_for_one / 5 / 60 000 ms / expo              |
| `monitoring`     | `{ healthCheck, metrics }`                                      |     ✓    | 15 s / 5 s                                      |
| `logging`        | `{ level, maxSize, maxFiles, compress, databaseRetentionDays }` |     ✓    | info / 50 MB / 10 / true / 14 days              |

`StackName` is `'dev' | 'test' | 'staging' | 'prod' | string`.

### `IEcosystemAppEntry`

Each entry in `apps[]`:

```typescript
interface IEcosystemAppEntry {
  name:           string;
  bootstrap?:     string;      // bootstrap mode (preferred)
  script?:        string;      // classic mode (legacy)
  enabled?:       boolean;     // default true
  critical?:      boolean;     // daemon shuts down if this crashes past maxRestarts
  dependsOn?:     string[];    // start order
  env?:           Record<string, string>;
  cwd?:           string;      // auto-stamped to project dir
  restartPolicy?: IRestartPolicy;
  startupTimeout?: number;
  watch?:         string | IWatchConfig | false;
  readiness?:     AppReadinessConfig;
}
```

| Field            | Purpose                                                           |
| ---------------- | ----------------------------------------------------------------- |
| `name`           | Unique app id — used by every CLI command                         |
| `bootstrap`      | Source or compiled bootstrap path, relative to `cwd` or absolute  |
| `script`         | Path to a classic `main.ts` (legacy single-process apps)          |
| `enabled`        | `false` skips during `startAll`                                   |
| `critical`       | Crashing past `maxRestarts` brings the **daemon** down            |
| `dependsOn`      | Wait until these apps report healthy before starting this one     |
| `env`            | App-level env merged over process env                             |
| `cwd`            | Base for relative paths; auto-stamped to the config's directory   |
| `restartPolicy`  | Per-app override of ecosystem `supervision`                       |
| `startupTimeout` | Max ms for ready handshake before treated as failed               |
| `watch`          | Dev-mode file watching: string=directory; full config; or `false` |

An ecosystem entry describes one application. Declare replica counts on the
individual `defineSystem` process entries with `processes[].instances`; a classic
`script` entry starts one process. There is no application-wide `instances`
override. A non-unit multiplier in an existing configuration is rejected before
lifecycle changes instead of silently starting fewer processes than requested.
Runtime pool resizing uses `omnitron scale <app> <count> --process <pool>` and does
not rewrite the declared topology.

### `AppReadinessConfig`

`probes` contains 1–16 uniquely named checks. `intervalMs` (1–300 seconds,
default 5 seconds) bounds result reuse; `timeoutMs` (100 ms–10 seconds, default
3 seconds) bounds each HTTP check. HTTP probes declare `url`, optional successful
`expectedStatus`, and optional `expect` with 1–16 exact JSON scalar fields.
One heartbeat probe per classic application declares `file` relative to `cwd`
and `maxAgeMs` (1 second–24 hours). Combine multiple domain checks in that report.
Bootstrap applications report completed work through `hooks.onHealthCheck`.
See [readiness and heartbeat examples](/docs/omnitron/diagnostics#classic-services-and-completed-work).

### `IWatchConfig`

```typescript
interface IWatchConfig {
  directory: string;        // relative to cwd or absolute
  include?:  string[];      // additional directories
  ignore?:   string[];      // whole path segments; merged with defaults
  debounce?: number;        // default 300 ms
}
```

The filesystem watcher compares whole path segments: `['test']` excludes a test
directory; `['test/**']` is not a glob and matches no segment. Defaults include
`node_modules`, `dist`, `.git` and test directories. Bootstrap source changes use
the compiler's import graph, including workspace libraries. See the
[development recipe](/docs/omnitron/recipes) for both watcher modes.

Watch is an application policy within a shared owner. Installed releases are
excluded from both watchers and source compilation, even when other applications
use development mode. Their `releaseRoot` marks the installed execution boundary;
launch receipts and integrity checks still authenticate its files. Do not add
`releaseRoot` to a source entry to bypass release admission.

`watch: false` suppresses automatic rebuilding and restarting for a source entry;
an explicit restart still compiles its current TypeScript sources. Disabling watch
awaits watcher disposal and cancels queued automatic restarts. Enabling it
reconciles changes made while it was disabled, without requiring another edit.

If an application is explicitly redefined while a restart is already stopping it,
the earlier restart request is cancelled, including an identical redefinition.
The application remains stopped with its current definition; inspect it and start
it explicitly. Disabling global watch alone lets an already admitted restart
finish when the application's definition is unchanged.

Stopping and resuming an installed stack retains its verified release independently
of this watch policy. These guarantees require an updated execution owner; updating
only the controller cannot change the watcher already loaded in an older owner.

### `IStackConfig`

A stack scopes an environment within a project. Boot and automatic recovery
admit enabled stacks independently: local Docker availability does not gate
remote attachment or stacks using external infrastructure. Provisioning failures
remain attached to the affected stack, which retries with its own exponential
backoff from one to fifteen minutes. Running and degraded stacks stay under
application supervision; recovery does not restart them wholesale.

```typescript
interface IStackConfig {
  infrastructureOwnership?: 'stack' | 'controller' | 'external';
  managedOrigin?:      { project: string; stack: string };
  type:               'local' | 'remote' | 'cluster';
  watch?:             boolean;
  nodes?:             IStackNode[];
  infrastructure?:    Partial<InfrastructureConfig>;
  settings?:          IStackSettings;
  apps?:              string[] | 'all';
  portRange?:         { start: number; end: number };
  serviceOverrides?:  Record<string, IServiceOverride>;
}
```

| Field                     | Effect                                                                                                                                                          |
| ------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `type`                    | `local` — single host; `remote` — single remote host; `cluster` — multi-node                                                                                    |
| `infrastructureOwnership` | `stack` (default) owns provisioning/migrations; `controller` is a generated non-owning node allocation; `external` uses explicitly configured existing services |
| `managedOrigin`           | Required source project/stack binding for `controller` allocations; never inferred from a project basename                                                      |
| `watch`                   | Default `true` for managed source apps; disable explicitly when required                                                                                        |
| `nodes`                   | For `remote`/`cluster`: list of `IStackNode` participants                                                                                                       |
| `infrastructure`          | Per-stack override of ecosystem `infrastructure`                                                                                                                |
| `settings`                | Container prefix, port offsets, env, log level                                                                                                                  |
| `apps`                    | `'all'` or explicit list — which ecosystem apps run in this stack                                                                                               |
| `portRange`               | Auto-allocate ports for stack services within this window                                                                                                       |
| `serviceOverrides`        | Per-service override of declared infrastructure (point at external host, etc.)                                                                                  |

Non-owning modes apply only to local application allocations. They retain explicit application connection settings, reject stack-level infrastructure declarations and `serviceOverrides`, and ignore ecosystem infrastructure defaults that may belong to other stacks. Requirement discovery does not authorize a node to create replacement services or run their migrations. See [infrastructure ownership](/docs/omnitron/infrastructure#infrastructure-ownership).

### `IStackNode`

```typescript
interface IStackNode {
  host:   string;
  port?:  number;                                  // daemon port; default 9700
  role:   'app' | 'database' | 'cache' | 'gateway' | 'worker' | 'master';
  apps?:  string[];                                // default 'all'
  ssh?:   { user?; privateKey?; port? };
  label?: string;
}
```

Each remote node runs its own omnitron daemon (typically as a
**slave**). The master daemon coordinates state replication via
the `OmnitronSync` service.

### `IStackSettings`

```typescript
interface IStackSettings {
  redisDbOffset?:    number;   // stack gets non-overlapping Redis DB range
  containerPrefix?: string;    // default `${project}-${stack}`
  env?:             Record<string, string>;
  logLevel?:        'fatal'|'error'|'warn'|'info'|'debug'|'trace';
  portOffsets?:     { postgres?, redis?, minio? };
}
```

Port offsets let you run `dev` and `staging` stacks side-by-side
on the same host:

```typescript
stacks: {
  dev:     { type: 'local', settings: { redisDbOffset: 0,  portOffsets: { postgres: 0,  redis: 0,  minio: 0 } } },
  staging: { type: 'local', settings: { redisDbOffset: 5,  portOffsets: { postgres: 10, redis: 10, minio: 10 } } },
}
```

### `supervision` — restart policy

```typescript
supervision: {
  strategy:     'one_for_one' | 'one_for_all' | 'rest_for_one';
  maxRestarts:  number;
  window:       number;            // ms
  backoff:      IBackoffOptions;   // from titan-pm
}
```

| Strategy       | Behaviour                                                          |
| -------------- | ------------------------------------------------------------------ |
| `one_for_one`  | Restart only the crashed app; others continue                      |
| `one_for_all`  | Restart all apps when one crashes (atomic stack restart)           |
| `rest_for_one` | Restart the crashed app + every app started after it (cascade fix) |

Defaults: `one_for_one` / `5` restarts / `60 000` ms window /
exponential backoff `{ initial: 1 000, max: 30 000, factor: 2 }`.

### `monitoring`

```typescript
monitoring: {
  healthCheck: { interval: 15_000, timeout: 5_000 },
  metrics:     { interval: 5_000, historyMaxAge: '7d' },
}
```

| Field                   | Effect                                                                      |
| ----------------------- | --------------------------------------------------------------------------- |
| `healthCheck.interval`  | How often probes run (default 15 s)                                         |
| `healthCheck.timeout`   | Per-probe deadline (default 5 s)                                            |
| `metrics.interval`      | Sample cadence (default 5 s)                                                |
| `metrics.historyMaxAge` | Persistent history duration (`7d` by default); Titan validates the duration |
| `metrics.retention`     | Legacy numeric setting; does not control the persistent store               |

### `logging`

```typescript
logging: {
  level:    'info',            // fatal | error | warn | info | debug | trace
  maxSize:  '50mb',            // per-app file size threshold
  maxFiles: 10,                // rotated files to keep
  compress: true,              // gzip rotated files
  databaseRetentionDays: 14,   // age limit for rows in the `logs` table
}
```

`maxSize` and `maxFiles` bound the rotated **files** on disk.
`databaseRetentionDays` bounds the **table** — a different store with a
different failure mode, and until it existed the table had no bound at all: the
development host reached 13 GB of `logs`. Two weeks is long enough to
investigate an incident from the weekend and short enough that the table stops
being the largest thing on the disk.

Retention deletes rows; it does not shrink the file. Postgres keeps the freed
pages inside the table and reuses them, so a table that has been large stays
large on disk — `doctor` reports that separately as `db.bloated` and tells you
to reclaim it with `VACUUM FULL` or `pg_repack`. A `logs` table far bigger than
its live row count means retention is working and the space is simply not being
returned.

The daemon turns retention **off** when the key is absent from the config it
receives — a wrong default chosen inside the collector would delete history
nobody asked it to delete. Configs loaded the normal way carry the default
above; one assembled by hand does not.

Per-app overrides live in `IAppDefinition.observability.logging`
(see `defineSystem` below).

### `gateway`

```typescript
gateway: {
  port:      8080,
  configDir: 'infra/nginx',
  image:     'openresty/openresty:alpine',
}
```

When configured, Omnitron provisions an OpenResty/Lua gateway
container per stack — reverse proxy, maintenance mode (driven
from Redis), rate limiting, future PoW captcha.

## `defineSystem(definition)` — per-app `bootstrap.ts`

```typescript
import { defineSystem } from '@omnitron-dev/omnitron';

export default defineSystem({
  name:    'api',
  version: '1.0.0',
  processes: [
    {
      name:   'http',
      module: './modules/http.module.js',
      transports: { http: { port: 3000 } },
      health:     { enabled: true },
    },
    {
      name:   'worker',
      module: './modules/worker.module.js',
      instances: 3,
      scaling: { strategy: 'auto', maxInstances: 10, targetCPU: 70 },
    },
  ],
  observability: { metrics: true, logging: { level: 'info' } },
});
```

### Top-level fields — `IAppDefinition`

| Field            | Type                                                                                                   | Required |
| ---------------- | ------------------------------------------------------------------------------------------------------ | :------: |
| `name`           | `string`                                                                                               |     ✓    |
| `version`        | `string`                                                                                               |     ✓    |
| `processes`      | `IProcessEntry[]` — must contain ≥ 1                                                                   |     ✓    |
| `requires`       | `IAppRequirements` *(deprecated; use `omnitronConfig.infrastructure` in `config/default.json`)*        |          |
| `auth`           | App-level auth manager / JWT config                                                                    |          |
| `config`         | `{ sources?, envPrefix? }`                                                                             |          |
| `shutdown`       | `{ timeout?, priority?, drainConnections? }`                                                           |          |
| `hooks`          | `beforeCreate`, `afterCreate`, `beforeStart`, `afterStart`, `beforeStop`, `afterStop`, `onHealthCheck` |          |
| `omnitronConfig` | `OmnitronAppConfig` — declarative infra (database, redis, s3, custom services)                         |          |
| `env`            | `Record<string, string>`                                                                               |          |
| `cwd`            | `string`                                                                                               |          |
| `observability`  | `{ metrics?, tracing?, logging? }`                                                                     |          |

### `IProcessEntry` — the heart of `defineSystem`

Each entry becomes exactly one fork (or a pool of N forks when
`instances > 1`).

```typescript
interface IProcessEntry {
  name:            string;
  module:          string;     // path to the @Module file
  critical?:       boolean;
  instances?:      number;     // > 1 = PM pool with load balancing
  topology?:       { expose?: boolean; access?: string[] };
  transports?:     { http?: IHttpTransportConfig; websocket?: IWebSocketTransportConfig };
  auth?:           IAppDefinition['auth'] | false;
  health?:         { enabled?, interval?, timeout?, retries? };
  startupTimeout?: number;     // default 30_000
  scaling?:        { strategy?, maxInstances?, targetCPU?, targetMemory?, queueThreshold?, cooldownPeriod? };
  restartPolicy?:  IRestartPolicy;
  env?:            Record<string, string>;
  customRoutes?:   Array<{ method, pattern, handler }>;
  hooks?:          /* same shape as IAppDefinition.hooks */;
  observability?:  { metrics?, logging? };
}
```

#### Topology — cross-process service mesh

```typescript
processes: [
  // Worker exposes its @Service to other processes:
  { name: 'aggregator', module: './aggregator.module.js',
    topology: { expose: true } },

  // Consumer injects proxies to specific sibling services:
  { name: 'gateway', module: './gateway.module.js',
    topology: { access: ['AggregatorService'] } },
]
```

- `expose: true` — Omnitron auto-discovers `@Service` metadata
  from this process and registers a proxy in the application's private
  Netron context. Pools use load-balanced dispatch; singletons follow their
  current child process.
- `access: [...]` — list of Netron service names this process
  needs. Bootstrap injects proxies into DI under
  `createToken('topology:{ServiceName}')`.

Service names resolve within one application launch. Identical names in another
project or stack are independent; duplicate providers inside one application fail
startup. See [application service isolation](/docs/omnitron/topology#application-service-isolation).

`access` declares injected sibling proxies. Processes in one application share
its private service namespace. It does not create OS isolation or per-method
authorization between those siblings.

#### Transports

```typescript
interface IHttpTransportConfig {
  port: number;
  host?: string;
  cors?: any;
  // ...
}

interface IWebSocketTransportConfig {
  port: number;
  host?: string;
  path?: string;
  // ...
}
```

Set `transports.http.port` or `.websocket.port` to expose Netron
on those protocols from this specific process.

#### Scaling (pool mode, `instances > 1`)

```typescript
scaling: {
  strategy:        'auto' | 'fixed',
  maxInstances:    10,
  targetCPU:       70,         // % per instance
  targetMemory:    80,         // % heap
  queueThreshold:  100,        // pending RPC calls before scale-up
  cooldownPeriod:  60_000,
}
```

`auto` strategy uses `titan-pm` autoscaler — the orchestrator
adds / removes workers based on the targets. `fixed` keeps the
declared `instances` count regardless of load.

#### Per-process auth

```typescript
processes: [
  // Inherit app-level auth:
  { name: 'http', module: './http.js' },

  // Override:
  { name: 'metrics', module: './metrics.js', auth: false },  // explicitly no auth
  { name: 'admin',   module: './admin.js',
    auth: { jwt: { enabled: true, tokenCacheTtl: 60 } } },
]
```

`auth: false` explicitly disables; omitted inherits the app's
auth.

### `OmnitronAppConfig` — declarative infrastructure (per-app `config/default.json`)

Lives in `config/default.json` under the `omnitron` key (not in
`bootstrap.ts`):

```json
{
  "omnitron": {
    "database": true,
    "redis":    true,
    "s3":       { "bucket": "uploads", "quota": "10gb" },
    "services": {
      "discovery":     true,
      "notifications": true
    },
    "infrastructure": {
      "search": {
        "type": "daemon",
        "ports": { "http": 9200 },
        "env":   { "SEARCH_URL": "http://${host}:${port:http}" },
        "docker": { "image": "elasticsearch:8.11.0" }
      }
    }
  }
}
```

| Field            | Type                                                      |
| ---------------- | --------------------------------------------------------- |
| `database`       | `boolean \| { dialect?, pool?, extensions?, dedicated? }` |
| `redis`          | `boolean \| { prefix?, dedicated? }`                      |
| `s3`             | `boolean \| { bucket?, quota? }`                          |
| `services`       | `{ discovery?, notifications?, quotes? }`                 |
| `infrastructure` | `Record<string, IServiceRequirement>` — custom containers |

`services` switches on shared capabilities the daemon wires for you rather
than infrastructure it provisions:

| Flag            | What it wires                                                                                                                                                                    |
| --------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `discovery`     | Registers the app with service discovery                                                                                                                                         |
| `notifications` | Connects the app to the notifications Redis database                                                                                                                             |
| `quotes`        | Injects `quotesRedis` — host, port, password and the **quotes app's own Redis database index** — so this app can read price data at the source instead of asking quotes over RPC |

`quotes` is the one worth knowing exists: it is cross-app by design, the
database index is allocated by the daemon rather than written by you, and
nothing else gives an app a connection into another app's Redis.

Omnitron reads these, provisions missing infrastructure, and
injects resolved env vars (`DATABASE_URL`, `REDIS_URL`,
`SEARCH_URL`) into the app at startup. The app never hardcodes
infrastructure addresses.

→ Full reference: [Infrastructure](/docs/omnitron/infrastructure).

### Lifecycle hooks

App- and process-level `hooks` carry the same shape:

```typescript
hooks: {
  beforeCreate?: () => Promise<void>;
  afterCreate?:  (app: any) => Promise<void>;
  beforeStart?:  (app: any) => Promise<void>;
  afterStart?:   (app: any) => Promise<void>;
  beforeStop?:   (app: any) => Promise<void>;
  afterStop?:    () => Promise<void>;
  onHealthCheck?: (app: any) => Promise<{
    status: 'healthy' | 'degraded' | 'unhealthy';
    checks?: Array<{ name: string; status: 'pass' | 'warn' | 'fail'; message?: string }>;
  }>;
}
```

Process-level hooks override app-level. Use `beforeStart` for
schema sanity checks, `afterStart` for cache priming,
`onHealthCheck` to surface custom checks to the health probe. Return named
`checks` with actionable `message` values: arbitrary fields such as `details`
are not part of the process health contract. The daemon preserves check names
and messages for both CLI and Console. See [domain readiness](/docs/omnitron/diagnostics#domain-readiness).

### Custom routes

```typescript
processes: [{
  name: 'http',
  module: './http.module.js',
  transports: { http: { port: 3000 } },
  customRoutes: [
    { method: 'GET', pattern: '/healthz',
      handler: async () => new Response('ok', { status: 200 }) },
    { method: 'POST', pattern: '/webhooks/github',
      handler: async (req) => handleGithub(req) },
  ],
}]
```

Custom routes sit alongside the Netron-exposed services on the
same HTTP transport.

## Daemon config — `IDaemonConfig` (not project-config)

The daemon's own config — paths, ports, role, cluster, secrets,
health-monitor — is **not** in `omnitron.config.ts`. Defaults
are baked in (see [Daemon / config defaults](/docs/omnitron/daemon#daemon-configuration)).
Persist supported instance policy in `~/.omnitron/config.json` (or the selected
`OMNITRON_HOME`). CLI flags and environment variables override only the settings
they explicitly expose. These are startup settings, not live mutations.

```typescript
interface IDaemonConfig {
  socketPath:  string;     // ~/.omnitron/daemon.sock
  port:        number;     // TCP, default 9700
  host:        string;     // bind address; default 127.0.0.1
  httpPort:    number;     // default 9800
  pidFile:     string;     // ~/.omnitron/daemon.pid
  stateFile:   string;     // ~/.omnitron/state.json
  logDir:      string;     // ~/.omnitron/logs
  role:        'master' | 'slave';
  master?:     { host: string; port: number };
  sync?:       ISyncConfig;
  cluster?:    { enabled, discovery?: 'postgres', clusterId?, leaseTtlMs?, heartbeatInterval? };
  secrets?:    { provider: 'file'|'env'; path?; passphrase? };
  auth?:       { jwtSecret? };
  healthMonitor?: { intervalMs?, concurrency?, offlineTimeoutMs?, ...retentionDays? };
  applicationReadiness?: Record<string, AppReadinessConfig>; // qualified app names, HTTP fallback
  backups?: { databaseTimeoutMs?: number }; // PostgreSQL dump/restore, default 1800000
}
```

### Database backup policy

Merge this field into the daemon's existing `config.json`, preserving its other
settings and private file permissions:

```json
{
  "backups": {
    "databaseTimeoutMs": 1800000
  }
}
```

The value is a positive integer in milliseconds, at most `86400000` (24 hours).
Omission selects 30 minutes. The same budget covers each local or Docker
PostgreSQL dump and restore, including streams, compression and durable file
publication. Invalid values refuse startup before backup RPC and schedules become
available. S3 object transfers and Tor archives do not use this setting.

A running daemon retains its current policy. Apply a reviewed
[backend replacement](/docs/omnitron/backend-handoff) or planned restart to load a change;
editing the file alone does not reconfigure it. After activation,
`omnitron backup schedules --json` exposes the effective `databaseTimeoutMs`,
schedules and stack coverage. A larger budget allows a large database more time;
it does not reserve disk space or prove the archive can be restored. An expired
restore does not undo SQL already applied; see the
[backup and restore boundaries](/docs/omnitron/cli#backups).

## Runtime DTOs (returned by RPC)

These shapes appear in `omnitron status` JSON output and in
webapp data.

### `ProcessInfoDto`

```typescript
interface ProcessInfoDto {
  name:       string;
  pid:        number | null;
  status:     'stopped' | 'starting' | 'online' | 'stopping' | 'errored' | 'crashed';
  cpu:        number;
  memory:     number;
  uptime:     number;
  restarts:   number;
  instances:  number;
  port:       number | null;
  mode:       'classic' | 'bootstrap';
  critical:   boolean;
  processes?: SubProcessInfoDto[];   // bootstrap mode
}
```

### `DaemonStatusDto`

```typescript
interface DaemonStatusDto {
  version:      string;
  pid:          number;
  uptime:       number;
  apps:         ProcessInfoDto[];
  totalCpu:     number;
  totalMemory:  number;
}
```

### `AggregatedMetricsDto`, `AggregatedHealthDto`, `LogEntryDto`, `AppDiagnosticsDto`

Defined in `src/config/types.ts` — refer there for exact field
shapes. The CLI and webapp consume them directly through the
typed `DaemonClient`.

## Validation

`defineSystem()` validates synchronously at module load:

| Check                              | Throws                                                        |
| ---------------------------------- | ------------------------------------------------------------- |
| `name` missing or non-string       | `DefineSystemError`                                           |
| `processes` empty                  | `DefineSystemError('At least one process must be defined')`   |
| Process without `name`             | `DefineSystemError('Every process must have a name')`         |
| Process without `module`           | `DefineSystemError('Process X must have a module path')`      |
| Duplicate process name             | `DefineSystemError('Duplicate process name X')`               |
| HTTP port not integer 1–65535      | `DefineSystemError('Process X has invalid HTTP port N')`      |
| WebSocket port not integer 1–65535 | `DefineSystemError('Process X has invalid WebSocket port N')` |

Invalid configs fail at `import('bootstrap.js')` — the daemon
surfaces the error and marks the app `errored`.

## Loader behaviour

`loader.ts` auto-detects `omnitron.config.ts` from CWD upward
when the CLI runs. Resolution order:

1. Explicit `--config <path>` on `omnitron up`.
2. `omnitron.config.ts` in CWD.
3. `omnitron.config.ts` walking parents up to the filesystem root.

`cwd` on `IEcosystemAppEntry` is auto-stamped to the directory of
the config that declared the app — relative paths inside the app
entry resolve against the project root, not the daemon's working
directory. Manual override is rarely needed.

## Recommended layout

**my-monorepo/**

- `omnitron.config.ts` — defineEcosystem({...})
- `apps/`
  - `api/`
    - `src/bootstrap.ts` — defineSystem({...})
    - `src/modules/http.module.ts`
    - `src/modules/worker.module.ts`
    - `config/default.json` — OmnitronAppConfig
    - `dist/bootstrap.js` — compiled artefact
  - `ingest/`
- `infra/`
  - `nginx/` — gateway configs (referenced by ecosystem.gateway)
- `package.json`

## Anti-patterns

- **Putting secrets in `omnitron.config.ts`.** The file is
  committed. Use `omnitron secret set` and read via `process.env`
  inside the app.
- **Mixing `bootstrap` and `script` on the same app.** Pick one:
  `bootstrap` for new code, `script` only for legacy classic
  apps.
- **Hardcoding port numbers across stacks.** Use `portOffsets` so
  dev / staging can coexist on one host.
- **`instances > 1` without `topology.expose`.** Pool workers
  are invisible to siblings without `expose`. Either add it or
  drop the pool.
- **`critical: true` everywhere.** Then any crash brings the
  daemon down. Reserve `critical` for the one app whose absence
  truly invalidates the deployment.
- **Hand-editing `config/default.json` to add infrastructure**
  instead of using `OmnitronAppConfig`. The schema is
  validated — declarative is the right path.

## See also

- [Architecture](/docs/omnitron/architecture) — what these configs drive
- [Daemon](/docs/omnitron/daemon) — own config and lifecycle
- [Orchestrator](/docs/omnitron/orchestrator) — what `defineSystem` becomes
  at launch
- [Infrastructure](/docs/omnitron/infrastructure) — `OmnitronAppConfig` deep dive
- [CLI](/docs/omnitron/cli) — commands that read this config

## Interactive SSH policy

`daemon.terminal` accepts `enabled`, `maxSessions`, `maxSessionsPerOwner`, `idleTimeoutMs`, `absoluteTimeoutMs` and `disconnectTimeoutMs`. See the [SSH policy table and example](/docs/omnitron/terminal#session-lifetime-and-limits). This is startup policy enforced by the daemon, independent of browser settings.

## File workspace exports

`files` in `~/.omnitron/config.json` configures administrator file access per node; it is not a project ecosystem setting. Omission exposes the `/` filesystem within daemon OS permissions. Set `enabled: false` or `roots: []` to disable exports. An explicit `roots` array replaces the default with `{ id, label, path, readOnly? }` entries. See [File workspace](/docs/omnitron/files#configure-access-per-node) for configuration examples and boundaries.

---

---
id: "omnitron/console"
title: "Web console"
description: "The Omnitron webapp — React + Vite + Prism, every page, every plane."
---

# Web console

The Omnitron webapp at `apps/omnitron/webapp/` (package
`@omnitron/console`) is the visual counterpart of the CLI. It's a
React 19 + Vite + MUI SPA that talks to the daemon via the
`@omnitron-dev/prism/netron` client — the same Netron RPC surface
the CLI uses.

Verified against `apps/omnitron/webapp/src/`.

## Workspace and stack scope

Choose a project and **All stacks** to see its stack cards side by side. Choose a
stack to see its card, applications, logs, metrics, topology, containers and
deployment history. Dashboard and Dashboard Builder use the same scope for stack
data. The workspace badge counts the whole project. The footer counts the
selected scope. System-wide pages such as Audit and System omit the selector.

Opening a stack from a card or a release link preserves the header selection.
On a stack detail page, explicitly selecting another stack opens that stack's
details. Choosing **All stacks** returns to the inventory. Browser Back and
Forward restore the workspace for each history entry. Reload keeps the current
choice. Switching projects clears the stack selection.

Releases keeps the project's artifact catalog visible. The selected stack
narrows deployment observations and sets the target in the **Deploy a release**
dialog. A successful inventory refresh clears a stack that was removed or
renamed. If the refresh fails, the last selection is marked stale.

## Consistent action placement

Horizontal card and row controls put removal at the leading edge, with extra
spacing before ordinary commands. Card footers keep those ordinary commands
together at the trailing edge, and groups wrap on narrow screens. Icon-only
controls identify their action and resource for assistive technology.

Overflow menus use the shared `RowActions` component: ordinary commands first,
destructive commands last behind a separator. Keyboard navigation follows that
order and Escape returns to its trigger. Confirmation dialogs keep Cancel
before the explicit destructive confirmation. Reordering controls never removes
role checks, ownership restrictions, disabled states or confirmation steps.

## One keyboard model

Console uses three levels of focus: **Console → workspace region → control**.
The visible focus ring marks the active control; file selection is independent of
the cursor. The command palette uses the same authorized navigation as the sidebar.
Keyboard commands invoke the existing actions and preserve permission checks,
confirmation dialogs and disabled states.

| Scope        | Keys                              | Action                                                                   |
| ------------ | --------------------------------- | ------------------------------------------------------------------------ |
| Console      | Ctrl/Cmd+K                        | Find an available page or command                                        |
| Console      | Ctrl/Cmd+/                        | Open the keyboard guide                                                  |
| Regions      | F6 / Shift+F6                     | Move forward / backward between navigation, tools, tables and workspaces |
| Current view | `/`, R                            | Focus the available search; refresh the current view                     |
| Controls     | Tab / Shift+Tab                   | Move between controls; arrows operate tabs and menus                     |
| Tables       | Up/Down, Home/End, Page Up/Down   | Browse rows; Enter opens, Space selects, Escape returns to the viewport  |
| File list    | Tab / Shift+Tab                   | Enter a panel at its file list; switch A/B, including on narrow screens  |
| Files        | Escape                            | Switch between the file list and its controls                            |
| File list    | Shift+Up/Down, Space, Ctrl/Cmd+A  | Extend selection, toggle a file, select loaded entries                   |
| Files        | Enter, Left/Right, Alt+Left/Right | Open, parent/selected directory, directory history                       |
| Files        | Ctrl/Cmd+L                        | Edit the current path                                                    |
| Files        | F2, F4, F5, Delete                | Rename, open/edit, copy to the other panel, move to trash                |
| Logs         | Home                              | Jump to the latest displayed logs                                        |
| Topology     | 0, 1                              | Fit the graph, restore readable scale                                    |

Entering a file panel by Tab from the sidebar, F6 or programmatic focus lands on
its file list, just like selecting a row with the pointer. Tab then switches A/B.
Escape enters the panel tools, where ordinary Tab navigation is available; Escape
returns to the list. Clicking an input or selector intentionally preserves its
native behavior. F6 moves between workspace regions, excluding nested breadcrumbs.
It also exits SSH input; in an editor dialog it cycles that dialog's tabs, tools
and document, never background controls. Their other shortcuts and text input stay with that surface;
Console does not intercept typed commands, composing text or handled key events.
Dialogs suspend background commands. A retained, hidden Files or SSH workspace
cannot execute shortcuts for another page.

The command palette uses Prism `CommandPalette`; shared table and search behavior
comes from Prism `TableViewport` and `SearchInput`. New Console sections should use
these primitives and `data-keyboard-region` for meaningful regions. Existing
search/refresh buttons can expose `data-console-command`; specialized workspaces
register context actions with `useConsoleActions`, enabled only while visible.
Avoid installing another global keydown listener. Keep each command's callback
shared with its pointer control, and recheck permissions at the RPC boundary.

## Dependency evidence in topology

Topology combines measured process/container status with configured relationships.
`OmnitronProject.getTopology` uses the same application definitions, infrastructure
resolver, environment precedence and stack vault references as the launch builder.
Read-only diagnostics use captured bootstrap metadata and configuration files;
they never import application bootstraps or compile application source. Metadata
is preserved across backend handoffs. When no captured declaration is available,
file/environment dependencies are still shown and coverage is marked incomplete.
It extracts database, Redis, S3 and declared service endpoints plus `dependsOn`.
No separate graph configuration is required. Shared services receive links from
each consumer; external endpoints are separate nodes with **configured / unknown**
status until there is independent runtime evidence.
A local loopback endpoint can match a shared container in another project on the
same daemon when its service and published port identify exactly one provider.
Remote loopback addresses stay on their physical node; ambiguous matches remain
configured endpoints. The system view limits app-specific coverage warnings to
the applications it displays, while project views retain their stack coverage.

Select a node and open **Connections** to inspect providers, consumers and the
configuration keys establishing each relationship. Arrows point from consumer to
provider; ownership links say **Provisioned for**. Dashed dependency links describe
configuration, not measured traffic. Userinfo, URL paths, query parameters, hashes
and credentials never enter this topology response. Missing definitions or
unavailable backend support produce an incomplete-data message rather than an
apparently complete empty graph.

The system workspace combines this daemon's applications and containers with remote
stack observations across projects; a project workspace narrows that inventory. This is not a packet trace: arbitrary
connections created only inside application code are not inferred, and current
configuration may differ from a process that has not yet been restarted. A green
container confirms its reported health, not every consumer's connection to it.

The CLI returns the same evidence:

```bash
omnitron project topology
omnitron project topology sample --stack dev
omnitron --json project topology sample
```

Consumer/provider lanes leave space for links and reflow to one column on narrow
screens. Selecting a node emphasizes its connections; the inspector also provides
a textual representation. The control plane appears once as the system workspace,
not as an empty application project. Package identity distinguishes it from a user
application that happens to be named `omnitron`.

## Tech stack

| Layer          | Choice                                                                                |
| -------------- | ------------------------------------------------------------------------------------- |
| Framework      | React 19                                                                              |
| Build          | Vite 8 with `@vitejs/plugin-react-swc`                                                |
| Routing        | `react-router-dom` 7                                                                  |
| UI components  | `@mui/material` (Emotion), with select components from `@omnitron-dev/prism`          |
| RPC client     | `@omnitron-dev/prism/netron` (`createMultiBackendClient`) over HTTP + WebSocket       |
| Charts         | Prism `Chart` / `useChart` (lazy-loaded ApexCharts)                                   |
| Graphs         | `@xyflow/react` (topology, dependency graphs)                                         |
| State          | `zustand` stores in `src/stores/`                                                     |
| Code splitting | Per-page `React.lazy()`; xterm loads with SSH; Prism components use explicit subpaths |

## Filesystem layout

**webapp/**

- `index.html`
- `vite.config.ts`
- `package.json`
- `public/` — static assets shipped as-is
- `src/`
  - `main.tsx` — entrypoint
  - `app.tsx` —  root
  - `routes/` — router definition
  - `layouts/` — ConsoleLayout, AuthLayout
  - `pages/` — Lazy-loaded Console sections
  - `components/` — shared UI building blocks
  - `hooks/` — custom React hooks
  - `netron/` — Netron client and typed daemon service bindings
  - `auth/` — AuthGuard, GuestGuard, ProjectGuard, auth store
  - `stores/` — zustand state stores
  - `assets/` — icons
  - `utils/`
  - `stubs/` — fs/path stubs for Node builtins pulled in by Prism

## Route map

All routes other than `/auth/sign-in` are gated by `AuthGuard`.
A session-keyed access provider retrieves effective RPC permissions, gates page entry and hides unavailable actions. See [Console access](/docs/omnitron/access).
Routes marked **project-scoped** require an active project (via
`ProjectGuard`) — selected from the project switcher in the
sidebar.

| Path                 | Page                   | Scope   | Backing RPC                                                                                                      |
| -------------------- | ---------------------- | ------- | ---------------------------------------------------------------------------------------------------------------- |
| `/access`            | `AccessPage`           | daemon  | `OmnitronAccess`                                                                                                 |
| `/files`             | `FilesPage`            | daemon  | `OmnitronFiles`, `OmnitronNodes`                                                                                 |
| `/terminal`          | `TerminalPage`         | daemon  | `OmnitronTerminal`, `OmnitronNodes`                                                                              |
| `/auth/sign-in`      | `SignInPage`           | guest   | `OmnitronAuth.signIn`                                                                                            |
| `/`                  | `DashboardPage`        | global  | aggregate from many services                                                                                     |
| `/nodes`             | `NodesPage`            | global  | `OmnitronNodes`                                                                                                  |
| `/projects`          | `ProjectsPage`         | global  | `OmnitronProject`                                                                                                |
| `/system`            | `SystemInfoPage`       | global  | `OmnitronSystemInfo`, `OmnitronNodes.getNodeSystemSnapshot`                                                      |
| `/settings`          | `SettingsPage`         | global  | `OmnitronAuth` + config                                                                                          |
| `/logs`              | `LogsPage`             | both    | `OmnitronLogs`                                                                                                   |
| `/apps`              | `AppsListPage`         | project | `OmnitronDaemon.list`                                                                                            |
| `/apps/:name`        | `AppDetailPage`        | both    | `OmnitronDaemon.getApp` + many                                                                                   |
| `/stacks`            | `StacksPage`           | project | `OmnitronProject.listStacks`                                                                                     |
| `/stacks/:name`      | `StackDetailPage`      | project | `OmnitronProject.listStacks`                                                                                     |
| `/metrics`           | `MetricsPage`          | project | `OmnitronMetrics` (`querySeries` / `getStorageStatus`)                                                           |
| `/topology`          | `TopologyPage`         | both    | stack deployments, daemon processes and runtime containers                                                       |
| `/containers`        | `ContainersPage`       | both    | `OmnitronInfra.listContainers` for this host, `OmnitronProject.listStacks` for placement and remote observations |
| `/deployments`       | `DeploymentsPage`      | project | `OmnitronProject.getDeployProgress`, `OmnitronRelease.deployments`, `OmnitronDeploy.getHistory`                  |
| `/alerts`            | `AlertsPage`           | global  | `OmnitronAlerts`                                                                                                 |
| `/pipelines`         | `PipelinesPage`        | system  | `OmnitronPipelines`                                                                                              |
| `/traces`            | `TracesPage`           | project | `OmnitronTraces`                                                                                                 |
| `/dashboard-builder` | `DashboardBuilderPage` | both    | scoped application inventory, retained metric series and controller alerts                                       |
| `*`                  | redirect to `/`        | —       | —                                                                                                                |

## Page-by-page

### Dashboard

The landing page shows the selected project's deployments, including remote stacks,
or this daemon's applications when no project is selected. It provides:

- Online/total application and stack counts, summed process CPU and resident memory.
- A ranked resource snapshot for the six largest applications, switchable between
  CPU and memory. The graph compares readings; it does not imply a capacity limit.
- Compact application cards grouped by stack, with status, CPU, memory, uptime,
  restarts and process counts. Local application names open their detail pages.
- Links to metrics and topology from the breadcrumb actions.

Refreshes retain the last received data on partial failures and name the source
that failed. CPU may exceed 100% when processes use several cores; resident memory
has no warning threshold without a known capacity. Zero is a valid measurement;
invalid negative or non-finite readings render as unknown. Polling pauses while
the browser tab is hidden.

Pages fetch through the typed Netron client directly (the
`daemon` / `metrics` proxies from `src/netron/client.ts`) inside
`useEffect` + polling intervals, and subscribe to a shared
`zustand` realtime store (`src/stores/realtime.store.ts`) for
live pushes. There is no TanStack/React Query layer.

### Apps list & detail

`/apps` — table view of every managed app: status, instances,
CPU, memory, restart count, mode (classic vs bootstrap),
critical flag. Search matches application and stack names; the status filter and
result count share Prism’s `FilterToolbar`. Reset restores the current stack’s
complete inventory. Wide tables scroll inside their own keyboard-focusable region.

`/apps/:name` — single-app deep dive:

- Live status with restart history timeline.
- Per-process breakdown (sub-processes for bootstrap mode).
- The Metrics tab loads interactive history on demand. The application overview and categorical resource rankings load independently of the chart engine.
- Health probes summary.
- DI dependency graph (renders Mermaid via daemon's
  `getDependencyGraph`).
- Log stream with filter (level, grep).
- Resolved environment variables.
- Action buttons: Start / Stop / Restart / Reload / Scale.
- Inspect button → drills into `OmnitronDaemon.inspect`.

### Stacks

`/stacks` shows cards for the current scope, including runtime status, online
and configured applications, and node count.

`/stacks/:name` — per-stack view: which apps run here, which
infrastructure, runtime status. Start / stop the whole stack.

A stack with a confirmed signed stop offers **Resume installed apps** to operators. The detail page shows each target's receipt, confirmed applications and failure reason. After a partial recovery or a legacy receipt, **Review new stop…** opens the inventory for a fresh review. See [Resume installed applications](/docs/omnitron/projects#resume-installed-applications).

**Diagnose**, on a stack card or its detail page, opens an on-demand readiness
report. It is separate from Start: it never provisions services, starts apps,
executes container health commands or applies migrations. The progress indicator
appears immediately; results carry an observation time, individual reasons and
next actions. **Needs attention** hides verified checks. A failed refresh retains
the previous report and marks it stale.

The report distinguishes **Verified**, **Planned change**, **Blocked** and
**Not verified**. An absent PostgreSQL container can be a planned creation;
a name owned by another stack or an occupied port is a blocker. Missing images
require a pull; missing named volumes will start empty. Missing bind mounts block
startup instead of being treated as a usable empty configuration directory.

Read-only diagnosis uses saved bootstrap declarations and current static app
configuration. Missing or changed declarations remain unverified. Administrators
can use **Refresh declarations…**, which explicitly imports trusted project code
in isolated processes, and then repeat the diagnosis. This is an independent
permission from viewing the report.

The first implementation checks local Docker prerequisites. For remote stacks,
it validates configuration and placement but explicitly leaves target-host
prerequisites unverified; it never uses this workstation's Docker as evidence for
a remote node. Use Nodes diagnostics and `omnitron infra inspect project/stack`
for the complementary RPC and host-service evidence. Database credentials,
schema, migrations and successful application startup remain outside this check.

Stack and application ratios mean **running / configured** and **online /
configured deployments**, respectively. A project with one running remote stack
and one stopped local stack shows `1/2 stacks`, and six apps declared in both
stacks show `6/12 apps`. Selecting only the local stack gives `0/1` and `0/6`.
The footer and dashboard read stacks and apps from the same inventory response;
late answers cannot replace another workspace's counters. Failed observations
retain their counts with a stale marker. The workspace menu shows configured
stack totals for inactive projects rather than presenting unobserved status as live.

A local stack's release card labels its durable choice **Selected**. Open **Deploy
release…** to check a release. **Prepare release** verifies its files before a
maintenance window. If the candidate's declarations are missing or stale, an
administrator can choose **Update declarations…** for that release and stack.
The dialog checks admission again after capture. **Inspect stack** opens the
target when its current processes must stop.
After a confirmed stop, **Deploy** starts the selected code. **Use working tree…**
appears on stopped local stacks that permit source launches. It starts the stack
from its registered directory and changes the choice for future restarts.

In **Review stop…**, enable **Keep infrastructure** to stop only the
applications. The confirmation becomes **Stop applications**, and the inventory
labels retained resources **Containers kept**. With the option off,
**Stop stack** removes the listed **Containers to remove** after application exit.
Switching modes discards the previous review and obtains a new digest; confirmation
stays disabled while the replacement review is loading, incomplete or not supported
by the controller. A viewer can inspect the same inventory but cannot execute either
stop. Retained containers remain visible in runtime diagnostics even when application
status is **Stopped**. Neither mode clears the selected release, and both disable
automatic application resume until an explicit start.

The selected ID is launch intent. Current application health and any release
evidence from the owner describe actual execution. A transfer record or a
selected ID alone cannot establish that every running process loaded that release.
See [Pin a local stack](/docs/omnitron/releases#pin-a-local-stack) for the CLI and data
recovery boundaries.

### Logs

The global viewer includes local and replicated application logs. Select one exact
application handle and, optionally, a registered remote node. The default remains
the daemon's log; choosing **All Apps** removes that application filter.

**Nodes → Diagnostics → Running on this node → Logs** carries both the application
and registry node into `/logs?app=…&node=…`. This works for applications whose
production project is not registered on the local machine. The same diagnostics
show each application's health verdict and individual checks, separately from its
process status: an online process can still have an unhealthy mail queue.

Diagnostics reports RPC transport, authentication, heartbeat freshness and retry timing separately from replication backlog and acknowledgements. Read failures retain their cause instead of appearing as an empty queue or an unexamined node. Historical repairs show separate verified/restored counts; an unreadable reconciliation journal remains a warning. The same interpretation is available through `omnitron node check <id>`; see [operational diagnostics](/docs/omnitron/diagnostics#rpc-connection-and-replication).

Diagnostics separates the replication channel from the optional telemetry relay.
Every updated node exposes its relay observation: a controller can report a local
database aggregator; a worker explicitly reports **not applicable** and points to
durable replication. A missing endpoint on an older backend stays **unavailable**,
as do authentication and connectivity errors. CLI and Console interpret the same
contract; absence never becomes an invented healthy or zero-loss reading.

Live mode polls `OmnitronLogs.streamLogs`; time ranges and older pages use
`queryLogs`. Both apply the same node, application, level and label filters.
Application, node and message search remain visible. **Filters** expands labels,
levels, time ranges and the copyable CLI command; its count shows active advanced
filter groups even while collapsed. Collapsing this panel does not reset the query.
Messages occupy a separate line on narrow screens and expand with Enter, Space or
a pointer, including entries without structured metadata. Text and severity colors
follow the active light or dark theme.
Search matches literal message text in stored records. The level filter matches each selected level exactly in live and historical views. Choose `unknown` to find messages whose source supplied no recognizable severity. For the same selection on a remote node, use `omnitron logs <app> --node <node> --level unknown`. A ranked CLI filter such as `--level warn` also includes higher severities. Expand a row to inspect its source node and structured metadata.

The copy control provides `omnitron logs '<project>/<stack>/<app>' --node '<id>' -n 100`
for the selected application/node scope. Search, level and time filters are not
included in that base command; CLI `--grep` is a regular expression.

### Metrics

`/metrics` uses Prism charts for CPU and resident memory, with 5-minute,
15-minute and 1-hour windows. Buckets are 10-second, 30-second and 1-minute
averages respectively. Time labels use the browser's local timezone; byte values
use binary units (KiB, MiB, GiB). CPU is not capped at 100%.

The cards count applications in the selected project and stack, from the same
inventory as the applications page. Histories preserve metric, application and
reporting-node identity. Remote stack aliases are resolved from stack deployments
and the node registry; a matching application name on another machine is not
included. Application totals are the default; child-process lines are opt-in and
must not be added to the parent totals.

- Select a series chip with the pointer or keyboard to hide/show it.
- **Data** opens a keyboard-scrollable table of latest value, mean, peak, sample
  count and last sample time. Mean and peak describe returned buckets, not raw
  event-level extrema.
- Missing buckets break the line. An empty window and a failed query have different
  states; failed refreshes retain the previous graph with a visible warning.
- **Live** controls the 10-second poll. Manual refresh remains available. Changing
  scope or window loads that selection once even when Live is paused.
- Log ingestion is explicitly daemon-wide. The optional **Log store** panel shows
  cumulative counts for all projects, independently of the selected time window.
  Its store scan runs only while expanded, at most once per minute automatically.

Use [Dashboard Builder](#dashboard-builder) in either workspace to compose panels.
Its CPU and memory charts use the same retained observations as `/metrics`.

### Topology

The **Workspace** action opens the [target editor](/docs/omnitron/topology): versioned drafts,
placement validation, connection evidence and reviewed acceptance. Accepted targets
are separate from deployment and restart configuration; runtime migrations are not
executed by the editor.

`/topology` is a deployment diagram built from `OmnitronProject.listStacks`,
`OmnitronProject.getTopology`, `OmnitronProject.getTopologyInventory`,
`OmnitronDaemon.list` / `status` and `OmnitronInfra.listContainers`. It shows the
machines, applications, provisioned services and gateway of the selected stack.
Edges represent declared service ownership/dependencies, not observed network
traffic. An incomplete read is labelled explicitly.

The graph opens at a readable scale (100% when one card fits), rather than
shrinking every deployment into the window. Compact Prism cards show a name,
state and key readings; process lists, image identifiers and full configuration
remain in the inspector. Dependency graphs use consumer/provider lanes with a clear routing gutter;
on narrow screens they become one column. Graphs without dependency evidence
use a compact grid. The Connections inspector explains the evidence for each link.

The toolbar provides node search, graph and list views, zoom, placement lock,
fullscreen and refresh. **Fit graph** shows every node. Click the percentage to
return to a readable scale. Drag the canvas to pan, pinch to zoom, or hold
Ctrl/Command while scrolling to zoom. When the canvas has focus, arrow keys pan,
`+` and `-` zoom, `0` fits the graph and `1` restores a readable view.
The minimap has an inset around its viewport and appears when the canvas has
enough space, including when an inspector is open.

Drag a card to reposition it. A focused card moves with the arrow keys. Hold
Shift to move it farther. **Lock node positions** prevents accidental movement.
Placements persist across polling and reloads in this browser for the selected
project and stack. **Arrange nodes** clears those placements and restores
responsive layout. Searching or selecting a node brings it into view at a
readable scale. Moving a card changes only its position on the map.

Declared links use distinct ports and bend around intervening cards. Shared
dependencies use separate tracks. Connections between cards in the same column
use the free space beside them. Wide canvases place deployment groups side by
side. Narrow screens present one column. Select a node to highlight its immediate links or choose **Focus links**
to isolate them. Open **Connections** for the recorded evidence. If a card
blocks a route, its relationship remains available in this list.

Selecting a node opens an inspector beside the graph on wide windows and a modal
sheet on narrow windows. Prism tabs offer the available overview, connections, config, metrics
and logs. CPU/memory history for local apps comes from the metrics service, not
from samples invented when the drawer opens. Readings update with polling; changing
scope clears the old selection. Unknown host capacity is not rendered as a gauge.
Local lifecycle controls follow Console permissions; remote applications remain
managed through their owning stack.

### Containers

`/containers` in the Omnitron workspace shows this controller's Docker
inventory, including control-plane and unassigned containers. In a project,
**All stacks** shows local containers owned by that project and links to its
remote stacks. A local stack selection narrows the table to its registered
ownership labels. **Local total** counts the rows in that scope.

Selecting a remote or cluster stack shows read-only infrastructure grouped by
node, with the same observations as the stack detail page. Each available row
shows the service and container, with its reported ports, lifecycle, health and
setup. A node can report an empty inventory. An unavailable inventory shows its
coverage and any reported reason. Expand **Configured services** to inspect
declarations whose runtime state has yet to be observed.

Direct entry loads missing stack locations through the shared project store.
Unknown locations show a loading state; failed reads explain the failure and offer
Retry. The page never treats unavailable stack metadata as proof that no remote
stacks exist. The status bar owns continued refresh, and concurrent readers share
one request.

The inventory shows observed status, image, health, published ports and scope.
**Not reported** does not establish health. Inspect logs from a row. Start, stop,
remove and log loading show progress on the affected container and prevent
concurrent duplicate actions. After a lifecycle request, Console refreshes the
observed inventory before enabling another action. A timeout or disconnected RPC
means the acknowledgement is unavailable: the operation may already have completed.
Console shows that uncertainty and never replays the mutation automatically. If
the refresh also fails, the table retains the last inventory with a separate read
error; inspect the current state before retrying. Other failures retain their
reported reason. Lifecycle actions still obey roles and ownership. A **Read only** indicator
explains cluster or platform restrictions; focus it to inspect the reason.

### Deployments

`/deployments` follows transfer observations and recorded stack starts in the
selected workspace. **Releases** selects a build and its target stack;
**Stacks** shows configuration and observed execution; **Deployments** explains
what operations were recorded; **Audit** retains the full event trail.

The default table shows the latest recorded start per stack, including its
source, actor and release when recorded. It does not prove what is running now.
The separate legacy app table preserves older records, whose requested version
labels and success statuses did not establish an artifact deployment. Its old
mutation path is disabled in RPC as well as Console; use the release/stack
workflow for deployment or returning to a previous release.

Transfer observations follow the selected project and stack, including imported
configurations. Each row shows the node, app, last phase and diagnostic message.
**Transfer details** exposes the operation ID, observation time and any release
ID. A stale or interrupted observation shows that completion is unknown.
Earlier controllers can return unattributed observations, which have a separate
expandable list. The page requests only the views permitted for the current user.
The CLI exposes the same observations through `omnitron stack progress`.

### Alerts

Open **Alerts** in the global navigation to view alerts across all workspaces.
When the status bar shows a firing count, select it to go directly to **Active
Alerts** at `/alerts#firing`. This also works from a project workspace. The
status bar link requires permission to read the alert summary, active alerts
and rules.

Desktop and sound notifications use the active-alert read permission. The first
successful read records what is already firing. Subsequent alerts at or above
the severity selected in **Settings** can notify you from any page, including a
background tab. Revoking access or signing out discards pending reads. A newly
authorized session starts with its own list of active alerts.

The active list shows up to the 100 most recent firing alerts, with their
severity, message and firing time. An acknowledged alert remains in the list
until it resolves. If the alerts clear before you open the link, the section
shows that there are no active alerts. Acknowledging an alert and editing its
rule each require the corresponding permission.

### Pipelines

`/pipelines` — CI/CD pipeline list, definition editor, run
history, live run view.

### Traces

`/traces` — distributed trace viewer. Trace list with filters,
single-trace waterfall, service-map view. Backed by
`OmnitronTraces`.

### Nodes

`/nodes` — fleet node inventory. Per-node:

- Status, hostname, last-seen timestamp.
- Uptime bar (90 days of green/yellow/red segments — backed by
  `getUptimeBar`).
- Latest recorded health-check time refreshes with node status, independently of the slower historical aggregation. An unavailable history read retains the last evidence and shows a warning.
- SSH connectivity test.
- Per-node actions: drain, remove, edit.
- **Diagnose** reports cluster participation and the recipient operations advertised by that backend. Missing participation evidence from an older daemon is shown as unknown.

In **Nodes → Managed backend updates**, an administrator chooses the build and
server order. **Stage from controller catalog** distributes a matching sealed runtime
to each server. Preparation returns a link immediately and shows qualification
progress. Review each server's candidate, owner and rollback target before choosing
**Apply reviewed plan**. Viewers can reopen the receipt to see what completed and
where execution stopped. See [fleet handoff](/docs/omnitron/backend-handoff#upgrade-a-fleet-of-managed-backends)
for artifact staging, CLI commands and recovery.

### Projects

`/projects` — registered project list. Add / scan / remove
projects. Project switcher in sidebar mirrors this.

### System

Open `/system` to inspect the host that runs a daemon. **Host** selects the connected daemon or a registered mesh node when your role permits remote reads. The selected node and detail tab are in the URL. Each reading identifies the reporting host, collection time and duration. **Live** polls every five seconds while the tab is visible. Pause it to compare numbers or use **Refresh system snapshot** for one observation.

The controller asks the selected node for its own snapshot over the authenticated mesh. Both ends must support this read. If an older node refuses the mesh credential, the page names the node and permission to check. A viewer can select a node. The node credential can answer for its own host only.

CPU load, committed memory, the fullest mount and network throughput share a compact summary. **Overview** includes host and daemon uptime, process memory, the reported build and expandable per-core load. **Storage** lists all reported mounts by utilization with their available bytes. **Network** lists the reported external interfaces with both IP families and link speed. On updated peers the rate aggregates active external interfaces once samples are available. Older peers report their original network sampling scope.

A partial snapshot names each missing collector and its reason. An unavailable peer keeps its own last observation with a warning. Switching hosts clears the previous reading. A snapshot over 15 seconds old is marked stale. A remote clock more than 15 seconds ahead is reported separately. CLI and agents can obtain the same snapshot with `omnitron --json system --node <id>`.

### Dashboard Builder

Open `/dashboard-builder`, select a dashboard, then choose **Edit layout**. **Add panel** lets you choose application counts and resource totals, retained CPU or memory history, failures and restarts, controller runtime, or firing alerts. Set the title, width and optional exact application name. An application panel follows the selected project and stack. Controller panels identify their scope explicitly.

Use a panel’s options to edit it, move it earlier or later, or remove it. These actions work from the keyboard. Cards reflow to a single column on narrow screens. A chart’s **Data** action opens the exact observations behind it. Historical charts use retained samples with their original timestamps. Missing samples leave gaps. An unavailable source or missing permission appears on the affected panel. A failed alert request cannot turn into a zero count.

With **Live** paused, selecting another data source or history window reads it once. **Refresh** requests another observation.

**Dashboard options → Settings** changes the name, refresh interval and history window. **New dashboard** starts an empty board. **Duplicate** creates an independent layout. **Export layout** and **Import layout** transfer versioned JSON between browsers. Imports validate the layout and create a separate dashboard. Layout edits are saved in this browser. If storage refuses a write, export the session layout before closing. The document contains panel definitions and filters. Measurements and permissions come from the receiving Console.

### Settings

`/settings` — current account, change-password, system
preferences.

## Auth flow

```mermaid
sequenceDiagram
  participant U as User
  participant W as Webapp
  participant D as Daemon
  participant A as OmnitronAuth

  U->>W: visit /
  W->>W: AuthGuard checks token
  alt no token
    W->>U: redirect to /auth/sign-in
    U->>W: enter credentials
    W->>D: HTTP POST /netron/...
    D->>A: signIn(credentials)
    A-->>D: { token, sessionId, roles }
    D-->>W: response
    W->>W: store token<br/>redirect to /
  end
  W->>D: every subsequent call carries Authorization: Bearer ...
  D->>A: validateToken(token)
  A-->>D: claims | reject
```

The access token is held in `sessionStorage` under the key
`omnitron_token` (via Prism's `SessionTokenStorage`). Logout
clears it and calls `OmnitronAuth.signOut({ sessionId })`. The
`signIn`, `validateToken`, `validateSession`, `refreshSession` and `signOut` methods declare anonymous transport access. Session IDs remain bearer credentials; authenticated sign-out additionally checks ownership.

## RBAC in the UI

The webapp uses the daemon's three human roles (`viewer`, `operator`, `admin`) through `OmnitronAccess.describe()`. The response is built from the active RPC decorators and authorization rules. There is no independently maintained role matrix in the browser.

| Surface                        | Behavior                                                                                          |
| ------------------------------ | ------------------------------------------------------------------------------------------------- |
| Navigation and protected pages | Show only when the backing read method is granted                                                 |
| Mutation controls and dialogs  | Hide when the corresponding RPC capability is absent                                              |
| Access                         | Show the current account's method grants; administrator account controls require their own grants |
| SSH workspace                  | Administrator only, with live session and ownership checks on every operation                     |
| Settings                       | Current account/session preferences and password changes                                          |

`useAccess()` and `Can` consume typed `service.method` capabilities; `AccessBoundary` uses Prism's guard-only auth entry. Missing or failed verification grants nothing. The daemon independently enforces every request. See [Console access](/docs/omnitron/access) for limits and extension points.

## Real-time event flow

The webapp uses `OmnitronEvents` for live updates:

```mermaid
sequenceDiagram
  participant W as Webapp
  participant E as OmnitronEvents
  participant D as Daemon internals

  W->>E: subscribe({ channels: ['app.status', 'metric.tick', 'alert.fired'] })
  E-->>W: subscriberId
  loop while connected
    D->>E: pushEvent('app.status', { name: 'api', status: 'online' })
    E-->>W: push to socket
    W->>W: update the zustand realtime store
  end
  Note over W: last consumer unmounts or sign-out
  W->>E: unsubscribe(subscriberId)
```

The Console uses two transports for one Netron API. Typed calls such as `OmnitronLogs.queryLogs` and `OmnitronNodes.getKernelRollout` use the HTTP transport under `/netron/`. Push events use the WebSocket transport at `/ws`. The browser authenticates its Netron peer and subscribes through `OmnitronEvents`.

The webapp subscribes through a single `DaemonWsClient`
(`src/netron/ws-client.ts`) that feeds the shared realtime store.
The status bar holds a subscription throughout the authenticated Console,
including direct entry into Access, Files or Terminal. Page subscriptions share
the same connection; leaving a page does not disconnect the status bar.

**Live** means authentication and event subscription have both succeeded.
**Polling** means push is unavailable and bounded polling supplies updates.
Releasing the last consumer resets connection state immediately; a previously
connected page cannot leave a stale Live indicator behind. Sign-out and revoked
sessions close their subscriptions. These rules apply to every authorized role.

## Build / run modes

### Dev mode

```bash
cd apps/omnitron/webapp
pnpm dev                    # Vite dev server with HMR (port 9810)
```

The RPC client uses a relative `baseUrl`, so the Vite dev server
proxies the daemon for it (see `vite.config.ts`): `/netron/*` →
`http://localhost:9801` (daemon HTTP) and `/ws` → the daemon's
Netron WebSocket transport on `:9802`. The daemon must already be
running (`omnitron up`).

The dev server deliberately sits at `9810`, outside `9800`–`9803`.
It used to ask for `9802` — the daemon's WebSocket port — so it
could not start while the daemon it exists to talk to was running,
and its own `/ws` proxy pointed back at itself. Sign in with the same account you use against the
production build — the `001_initial_schema` migration seeds a single
`admin` user with password `admin` on first daemon start, and the
password must be changed through Settings → Security before external exposure. Console startup falls back to loopback while this seed remains in use. A master refuses public TCP/HTTP/WebSocket registration until the credential check confirms the seed is no longer accepted; an unavailable check also refuses publication.

### Production build

```bash
cd apps/omnitron/webapp
pnpm build                  # tsc -b && vite build → dist/
```

The compiled bundle lands in `webapp/dist/`. It is **not** served
by the daemon directly — `omnitron webapp start` launches an
`omnitron-nginx` container that mounts `dist/`, serves it on
`:9800`, and reverse-proxies `/netron/*` to the daemon's HTTP
listener (`:9801`) and `/ws` to its WebSocket transport (`:9802`).

A running nginx container reads this directory directly. Building into `dist/`
changes the live frontend immediately, even if the daemon still runs an older
build. For validation while Console is serving traffic, use a separate output:

```bash
cd apps/omnitron/webapp
pnpm typecheck
pnpm exec vite build --outDir node_modules/.console-validation
```

Activate backend and frontend together when a change adds RPC services.
`OmnitronAccess` and `OmnitronTerminal` are registered during master startup;
copying browser assets cannot activate them. Plan the daemon restart around
supervised workloads, verify the new services through CLI, then switch the
served assets. Keep the previous frontend available for rollback.

Before the restart, record each application's status and child-process state,
not only the aggregate online count. Check that native executables still exist,
that the installed dependency graph resolves the intended Titan instance, and
that immutable runtime checks pass. Long-running processes can remain healthy
after their on-disk dependencies have become incompatible with a fresh start.

Preserve the previous backend, frontend, configuration, consistent database
snapshot, lockfiles and actual dependency artifacts. A source revision or a
hash manifest alone cannot reconstruct an unarchived development-linked runtime.
Do not replace an experiment's old integrity hash to admit new dependencies;
register a distinct continuation and retain its previous results.

After activation, verify effective permissions with `omnitron access show`,
compare every previously running process with the baseline, and check application
health in addition to process status. Exercise admin, operator and viewer views,
session revocation and an independently verified SSH connection. Verify that
trust changes and terminal lifecycle events appear in Audit. Remove temporary
test accounts and close test shells after verification.

### Open the webapp

| Command                      | Effect                                                                                      |
| ---------------------------- | ------------------------------------------------------------------------------------------- |
| `omnitron webapp build`      | Compile the bundle (`vite build` → `dist/`)                                                 |
| `omnitron webapp start [-f]` | Start the `omnitron-nginx` container serving static + gateway (`-f`/`--force` recreates it) |
| `omnitron webapp stop`       | Stop and remove the nginx container                                                         |
| `omnitron webapp status`     | Report whether the console container is running / healthy                                   |
| `omnitron webapp open`       | Open the running webapp in the system browser (starts it first if stopped)                  |

All Console lifecycle commands support `--json` with one structured result and
no progress prose. A failed follow-up after successful startup is reported as
`outcome: "applied"`, rather than implying the Console was stopped again.
Use `omnitron --json webapp status` to inspect the observed container state;
see [CLI output and lifecycle outcomes](/docs/omnitron/cli#webapp-console-ui).

## Project switcher

The sidebar carries a project dropdown — switching projects
re-scopes every project-scoped page to that project's apps /
stacks / etc. Stored in the auth context; persists across
sessions.

## Empty / error states

Every page is built around three states:

- **Loading** — skeleton with shimmer (`<LoadingScreen />` from
  Prism).
- **Empty** — friendly empty state with the primary action
  ("Add your first project", "Connect a node", …).
- **Error** — error card with retry button + JSON of the underlying
  error (operators love seeing the actual error).

## Performance characteristics

- **Code-splitting** — each page is `React.lazy`'d.
- **HMR** — Vite + SWC; instant in dev.
- **Polling + push** — pages poll the daemon on an interval and
  the shared `zustand` realtime store applies pushed events; no
  TanStack/React Query cache layer.
- **WebSocket** for `OmnitronEvents` subscribers — one
  `DaemonWsClient` socket per tab.

## Adding a new page

The pattern is mechanical:

1. Add `apps/omnitron/webapp/src/pages/my-page.tsx` exporting a
   default React component.
2. Add a lazy import + `<Route path="my-page" ...>` in the router
   (`src/routes/index.tsx`), wrapping in `<ProjectRoute>` if the
   page is project-scoped.
3. Add a nav entry in `src/layouts/console-layout.tsx`.
4. Import the typed service proxy from `src/netron/client.ts`
   (e.g. `import { daemon, metrics } from 'src/netron/client'`)
   and call it from `useEffect`; for live updates, read from the
   `src/stores/realtime.store.ts` zustand store.

No separate API layer to wire — the daemon already exposes
everything as typed Netron services.

## Anti-patterns

- **Storing secrets in `localStorage`.** Use the auth token only;
  for actual secrets, the daemon's `OmnitronSecrets` is the right
  place.
- **Polling instead of subscribing.** When `OmnitronEvents`
  already emits the change, use it — saves the daemon a lot of
  RPC roundtrips.
- **Bypassing `AuthGuard`.** Every protected route should sit
  under the `<AuthGuard>` wrapper; ad-hoc auth checks per page
  drift.
- **Calling the daemon directly with `fetch`.** Use the typed
  service proxies from `src/netron/client.ts` — they carry the
  bearer token and give you end-to-end types from the daemon DTOs.

## See also

- [CLI](/docs/omnitron/cli) — the same operator surface from the terminal
- [Services reference](/docs/omnitron/services-reference) — every RPC the
  webapp consumes
- [Architecture](/docs/omnitron/architecture) — where the webapp sits
- [Prism](/docs/frontend/prism) — provides the
  `@omnitron-dev/prism/netron` RPC client the webapp uses, plus
  select UI components (the bulk of the UI is MUI)
- [netron-browser](/docs/frontend/netron/browser) — the underlying
  browser RPC transport that Prism's netron client builds on

## SSH and role administration

[SSH workspace](/docs/omnitron/terminal) provides independent terminal tabs, split view, search, paste review and a shared CLI/backend lifecycle through Xec. It is lazily loaded and stays mounted across navigation after first use.

[Access](/docs/omnitron/access) shows the running daemon’s policy to every human role. Administrators can create accounts, change roles and revoke sessions. Account changes and their audit rows commit together; account directory queries never return password hashes or second-factor secrets.

## Compact navigation and scrolling

Console pages use a single breadcrumb trail for location and navigation. A hidden level-one heading and the browser tab title retain the page name for accessibility. Parent crumbs use client-side routing, so navigating between sections preserves mounted workspaces and their SSH sessions. Page actions remain beside the trail; content headings still identify cards and subsections.

Application cards on the global dashboard open details by their full daemon
handle, without requiring a project selection. The detail page retains role
checks and links back to Overview when no project is selected.

The application shell occupies the available viewport. Its header, navigation and status bar stay within the window, while ordinary pages scroll in the main region. SSH panes and the topology canvas own their internal scrolling. The status bar progressively hides secondary details on narrow screens while retaining connection and application health. Prism theme tokens supply terminal background, text, cursor and selection colors.

### Feedback for operations

User-triggered work responds immediately at its point of action. Console's shared
`OperationStatus` shows the operation name, relevant context and elapsed waiting
time; MUI loading buttons prevent repeated submissions. After ten seconds, the
status explicitly says it is still waiting for confirmation. These are waiting
indicators, not progress estimates. Screen readers receive the operation status
without a spoken update every second; reduced-motion preferences are respected.

SSH connection/closure, file opening and saving, transfer controls, account and
role changes, stack imports, topology reads/plans/changes, automation definition
saves, release cleanup and manual node refresh use this feedback. Errors remain
in the dialog or workspace where the operation was requested. Mutations are not
automatically retried. A request timeout must not be described as proof that the
remote operation failed or was cancelled.

Background polling retains the last successful data without constantly displaying
a foreground spinner. An explicit refresh shows its own waiting state. Transfers
keep byte/file progress from the daemon; an upload waiting for publication does
not imply completion merely because all bytes have been transmitted.

The design contract for every new action is the same: immediate acknowledgement,
a truthful waiting state, a confirmed result or an actionable failure, and details
on demand. Keep primary feedback beside the initiating control; avoid duplicate
banners and dialogs. Unknown completion is a distinct state, never success or a
fabricated percentage. Keyboard focus and accessible status announcements are
part of the action, not an additional interaction mode.

CLI and agents must receive the same outcome categories and diagnostic IDs as the
Console. In HTTP RPC, stable numeric errors distinguish denied access (403), lost
identity (401), conflicts (409) and unavailable authority (503). Neither a spinner
nor an English error string is an automation contract.

### Reading an expanded log event

An expanded event continues the selected row on one surface, with a narrow level
accent and aligned context separated by thin dividers. Multiline messages retain
line breaks and long tokens wrap inside the viewport. On narrow windows the
message moves below its time, application and level, and metadata fields stack.

**Enter** or **Space** on the row expands it. **Escape** collapses the event and
returns focus to the disclosure. Application fields, event and capture times,
source stream, trace and labels remain readable without opening a JSON card.
**Raw event** exposes the complete original record, including capture sequence
and receipt metadata. **Copy JSON** copies that record and reports clipboard
failure with a selectable-text fallback.

### Tables, filters and overlays

The shared Prism `Table` and `AdminDataTable` contain horizontal scrolling inside
`TableViewport`. Focus a named table region to scroll with the keyboard; rows with
`onRowClick` also accept Enter and Space without intercepting nested links or
buttons. Pagination stays outside the scrolling rows and rearranges when its
container is narrow. `maxHeight` constrains the rows, not pagination. Console gives
wide operational tables a minimum readable width instead of squeezing their
columns into the window.

Audit is an administrator-only history of control-plane actions across projects;
the workspace selector does not narrow it. Resource, action and actor filters
are explicit. Each row shows a compact timestamp, action, outcome and actor.
Expand it with a click, Enter or Space to read exact time, resource, origin and
structured details through the same field renderer as application logs. Escape
collapses the details. Long identifiers wrap within the row.

Actor badges retain the distinction between **Local client** (`omnitron-local`,
CLI or MCP admitted through the private Unix socket), **System** (the daemon's
own action), **Service** (another control-plane principal) and **User** (a Console
account). The full stored identity stays available in details; a badge does not
invent a username or merge distinct principals.

**Load older** freezes the visible page and follows the last entry's ID. The
server orders timestamp ties by ID and compares timestamps inside PostgreSQL,
so JavaScript's lower timestamp precision cannot skip records. **Resume** or
**Refresh** returns to the newest page. A deleted cursor requires refreshing;
an unavailable history is not represented as an empty healthy trail. Structured
details are bounded and credentials are redacted on both write and read,
including nested arrays from older records. Historical `[object]` placeholders
cannot recover information that was never stored.

Access uses a compact account-action menu, a collapsible role reference and a
clearable permission search with an explicit no-match state. Permissions continue
to come from the daemon; visual rearrangement does not change policy.

Forms and menus inherit Prism’s MUI overrides. Dialogs fill small viewports using
`100dvh`; their content scrolls while action buttons remain reachable. Long menu
labels wrap, form pairs stack on small windows, and secondary account actions
return keyboard focus to their trigger when dismissed. Open a confirmation dialog
to review an operation; merely opening a menu or form does not execute it.

Workspace selection uses the same MUI menu and Prism spacing as row actions,
Select controls and context menus. Menu groups have symmetric divider spacing,
and the menu scrolls within the viewport as a single keyboard region. Arrow keys
move between items; Home/End reach its ends. **Add project** opens the existing
registration form and is offered only with `OmnitronProject.addProject`; opening
the form does not register anything.

Run all Console tests with `pnpm --dir apps/omnitron/webapp test`. This command
includes both `test/webapp` and colocated `webapp/src` tests using the same aliases
as the package suite, including real topology node connection points.

`src/theme.ts` defines Console’s light and dark surfaces through `PrismProvider`
`themeOverrides`. Component geometry and interaction remain in Prism rather than
being reimplemented in each page. For frontend-only changes, rebuild the Console
assets and reload the browser; a daemon or application restart is unnecessary.

## Automation workspace

The [Pipelines workspace](/docs/omnitron/automation) uses Prism tabs and data tables with responsive MUI editors and a run-details drawer. Step controls include local/registered-node selection, working directory, dependencies, timeout and advanced retry/environment settings. Execution requires review of the displayed source and targets. Run history loads summaries; step output is fetched only for the opened run, with stale-data and truncation indicators.

This workspace is restricted to administrators because its scripts execute with infrastructure privileges. Embedded provider sessions and delegated agent controls are specified in the [agent architecture](/docs/omnitron/agents); they are not live Console features yet.

## Watch configuration

Re-enabling watch uses the daemon’s last accepted boot or reload configuration, including a configuration selected from a registered project. It does not rediscover an unrelated configuration from the caller’s working directory. Build watchers for registered applications appear in the same watch status as classic file watchers.

Definition reads, initial builds and watch rebuilds resolve compiler source labels
and diagnostics relative to the source entry. They do not depend on the
controller process's working directory. Workspace package source is compiled
under the same resolution policy; installed packages retain their runtime
dependency identity.

## File workspace

[Files](/docs/omnitron/files) provides two independent, tabbed node locations, version-checked text editing and durable cross-node copies. Prism tabs and table viewports share the Console shell; CodeMirror loads only when a document is opened. Narrow windows retain both panels behind an A/B switch. The default export is `/`, limited by administrator authorization and the daemon OS account; node configuration can restrict roots, make them read only or disable access. Selecting one directory exposes explicit archive-file and pack-and-restore modes, with formats probed on the chosen nodes. Transfer progress distinguishes snapshots, encoding, network bytes, restoration and incomplete publication; cleanup failures retain a dedicated retry action.

## Shared API contracts

Console and CLI share the wire contracts exported by `@omnitron-dev/omnitron/dto/services`. Backup coverage, audit queries, node health, rollout progress and release build progress are declared in the common DTO layer; the daemon implementations consume the same definitions. Existing service type exports remain available for compatibility. Console typechecking does not load daemon, database or service implementations, and a compiler-graph regression test enforces that boundary.

## Publishing Console updates

From a source checkout, `pnpm --filter @omnitron/console build` typechecks and
builds into a temporary directory before publishing. `omnitron webapp build`
uses the same path. It never lets Vite empty the directory nginx is serving.

Hashed assets are admitted before the new HTML is replaced atomically, while the
mounted directory stays in place. Existing URLs cannot change their bytes.
The publisher retains at least three generations and a seven-day grace period;
cleanup only removes files recorded in its publication ledger whose bytes are
unchanged. It never prunes unrelated files. The ledger and an exclusive writer
lock sit beside `dist`, outside the web root. A killed publisher leaves a lock
with its PID for inspection; verify the process has stopped before removing it.

The runtime freezer refuses a package that contains a `webapp` directory without
`webapp/dist/index.html`. Include the complete prebuilt Console before freezing;
an intentionally headless package omits `webapp` entirely. Never alter a sealed
runtime to add missing assets: prepare a complete artifact, verify it, then
activate it through the [backend handoff](/docs/omnitron/backend-handoff) workflow and
reconcile the Console container with `omnitron webapp start`.

Only successful asset responses are cached: content-hashed URLs are immutable;
unhashed assets require revalidation after an hour. Failed responses, including
missing chunks (404), use `Cache-Control: no-store`. This prevents a temporary
missing file from keeping a browser on a failed route after recovery. Browsers
that cached such failures from older versions may require a hard reload.

Open tabs continue to use their loaded generation. A compact notification offers
**Reload** when a new build is available; it never automatically discards file
edits or terminal sessions. Notifications use the Prism header slot inside the
viewport, preserving space for the status bar and workspaces. The file editor's existing unsaved-change guard also
applies to an explicit reload. Tabs older than the retention window may need to
reload before opening a new route.

Prepare a reviewable build or publish one already verified:

```bash
node apps/omnitron/scripts/build-console.mjs --out-dir /absolute/path/to/candidate
node apps/omnitron/scripts/build-console.mjs --publish-from /absolute/path/to/candidate
```

The candidate directory must not already exist. Keep a previous build for rollback
and publish it with the same command. A Console-only update does not restart the
daemon, applications or experiment workers. Updating backend code remains a
separate [daemon maintenance operation](/docs/omnitron/daemon#updating-a-source-runtime).

Console configures the chart engine used by Prism with the line/area and bar
renderers, legends, keyboard interaction, toolbar, exports and annotations.
Other Prism consumers keep their default engine. Add further chart imports in
Console's `src/charts.ts` when introducing another chart type. MUI/Emotion chunks
preserve source initialization order; test the built app in a browser after
changing chunking rules, not only with a successful bundler exit code.

The publisher regression suite includes interrupted HTML replacement, existing
open-tab assets, immutable URL conflicts and bounded retention. Run it with
`pnpm --filter @omnitron-dev/omnitron test:console-publication`.

### Historical coverage in node diagnostics

The replication reading separates a healthy current queue from coverage before
durable receipts existed. Expand **History before receipts · coverage and gaps**
for each audited source range, checked/expected rows, existing and restored
records, verification time, retained precision and unavailable earlier history.
This uses the same evidence as `omnitron node check`; see the
[recovery procedure](/docs/omnitron/cluster#history-before-delivery-receipts).

When a reviewed recovery closes a known gap, the disclosure shows its explanation,
verification time and separate proof source. The original audit remains available;
no unresolved warning is cleared by a successful current queue reading.

### Live log tail and complete history

The live tail polls at two-second intervals with a bounded window of up to 50
matching events per poll and retains up to 2,000 rows in the browser. A filled
window is marked explicitly: a burst can exceed what the tail displays. Use a
historical time range and pagination for the complete retained database record,
including late-arriving events with earlier source timestamps. The capture/journal
and database acknowledgement indicators describe durable storage independently of
this bounded display. A healthy ingestion indicator is not a claim that every
stored row is currently on screen.

### Releases and current deployments

The **Deployment state** view on Releases and stack details combines the existing
project inventory with deployment history. It shows each stack's actual
running/configured application count and node reachability. The last recorded
deployment is dated and linked separately. The controller's stored artifact count
uses the selected project filter. Building an artifact does not start a stack.

**History: test** is a neutral historical label. **Git: no remote branch** describes
a build-time source-control observation, not a local deployment. Processes that
do not attest their current release show **Current release not reported**. Failed
reads and unreachable nodes remain unknown. Inventory refreshes reuse the shared
workspace store; changing projects discards late answers from the old workspace.

Verified launch receipts appear separately from deployment history, with per-node
coverage and concrete reasons for missing or conflicting evidence. **Inspect
installed files** starts an explicit read-only comparison with retained archives.
The action shows progress and keeps its timestamped result separate from launch
identity: modified installed files, ambiguous base releases and unsupported node
backends do not become a verified running release. Matching entry points are
labelled as partial checks, not full installation integrity.

`omnitron release status <project>` uses the same read model; add
`--inspect-installed` for the same on-demand disk comparison. See
[Releases and verification](/docs/omnitron/releases#understand-what-is-built-and-what-is-running)
for the evidence boundaries, JSON and exit codes.

---

---
id: "omnitron/daemon"
title: "Daemon"
description: "The long-running supervisor — its lifecycle, state, and RPC surface."
---

# Daemon

The backend serves the CLI, Console and mesh. In the default coupled mode it
also owns application processes. Opt-in resident execution keeps those processes
in a separate owner so the backend can be replaced without restarting them.
See [backend replacement](/docs/omnitron/backend-handoff) for the initial maintenance
window, supported environments, artifact validation and recovery.

The lifecycle below describes coupled mode. In resident mode, backend retirement
preserves the owner; an explicit shutdown still stops applications.

Verified against `src/daemon/`:

**daemon/**

- `daemon.ts` — main daemon class
- `daemon.rpc-service.ts` — Netron-facing service
- `daemon.module.ts` — Titan module wiring
- `daemon-client.ts` — Typed RPC client (used by CLI)
- `daemon-entry.ts` — Standalone entrypoint
- `daemon-scheduler.ts` — Periodic task scheduler
- `pid-manager.ts` — PID file lock + liveness sweep
- `daemon-state-store.service.ts` — Transactional SQLite state storage
- `state-store.ts` — Application snapshots and execution intent
- `execution-owner.ts` — Independent application owner
- `backend-supervisor.ts` — Backend guardian
- `backend-upgrade.ts` — Validated backend handoff

## Lifecycle

```mermaid
sequenceDiagram
  participant Init as omnitron up
  participant PM as PID manager
  participant SS as State store
  participant N as Netron bus
  participant SVC as RPC services
  participant Orch as Orchestrator
  participant Apps

  Init->>PM: acquire ~/.omnitron/daemon.pid
  alt lock held by live process
    PM-->>Init: error — already running
    Init->>Init: exit 1
  end
  Init->>SS: open durable state and project registry
  Init->>Init: register cleanup before signal handlers
  Init->>N: bind Unix socket + [TCP / HTTP]
  Init->>SVC: register all services
  Init->>Orch: read registered projects and enabled stacks
  loop per enabled stack
    Orch->>Apps: resume selected release or development configuration
  end
  Init-->>Init: ready
  Note over Init: serve indefinitely
  Note over Init: SIGTERM
  Init->>Init: cancel remaining startup steps, join in-flight work
  Init->>Orch: stopAll (coupled mode)
  Init->>SS: persist final state
  Init->>PM: release lock
  Init->>Init: exit 0
```

## Startup and node diagnostics

A master prepares its control-plane database before Titan eagerly initializes
providers. Foreground CLI, detached CLI and native supervision share this path.
A configured `OMNITRON_DATABASE_URL` uses the supplied database without creating a
local PostgreSQL container. `--no-infra` skips Docker provisioning, but still
initializes the master schema through the shared Kysera migration runner. A
slave keeps its SQLite lifecycle and does not connect to PostgreSQL.

During backend handoff, this stage only verifies schema compatibility; it does
not provision or migrate. Database or migration failures stop startup before
claiming the resident owner or publishing listeners, preserving the concrete
cause in the boot diagnostic. Prepare schema changes before replacing a backend.

`omnitron up` launches a detached daemon and releases both the child-process
and IPC event-loop references. The CLI exits after its readiness checks while
the daemon keeps running. Startup stderr is appended to the boot log reported
by the command, including failures before RPC becomes available.

The Nodes API exposes `getNodeDaemonStatus`, `getNodeDaemonHealth` and
`getNodeDaemonMetrics` for every registered node. Local requests use the same
live daemon readers as the CLI; remote requests use the authenticated mesh.
`reachable: false` includes the failure reason and no answer. It is not a
health verdict, and a remote failure never falls back to the local node's data.

### Shutdown during startup

The daemon registers cleanup before Titan installs its signal handlers. If
`SIGTERM` or `SIGINT` arrives during startup, it stops beginning new startup
phases and enabled stacks. Titan's first shutdown task waits for the current
startup operation and any in-flight health-monitor spawn or restart before
cleanup closes their resources. The existing Titan shutdown deadline still
applies: an operation that cannot finish is reported as a shutdown failure,
not successful cleanup.

An interrupted startup does not announce `daemon.started` or a replacement
backend's readiness. The startup diagnostic remains an error if an actual
operation failed while shutdown was pending. A resident owner's applications
continue running when only its backend terminates; explicit `omnitron down`
still requests application shutdown. Event-loop monitoring and the reload
signal handler are disposed with the backend.

## PID manager — `pid-manager.ts`

Owns the daemon lock file `~/.omnitron/daemon.pid`. Responsibilities:

- **Acquire** on boot: write `process.pid` to the file with an
  exclusive open. If the file exists and the PID inside is still
  alive, abort.
- **Signature verification.** The file carries a process-start
  signature (PID + start time fingerprint). A stale PID with the
  same numeric value as a recycled new process is detected and
  treated as dead.
- **Liveness sweep.** A periodic check ensures the lock owner is
  still alive; reclaims if the holder has crashed without
  releasing.
- **Release** on `SIGTERM` / `SIGINT` / normal exit.

When `omnitron up` reports "daemon already running", this is the
check that fired.

## State store — `state-store.ts`

Persists daemon intent to a **SQLite-backed** key/value row
(`DaemonStateStore`, `state_kv` key `orchestrator:persisted-state`)
co-located with the daemon's other state surfaces (PID lock,
project registry, node registry, backup index). SQLite WAL-mode
transactions remove the torn-write risk that the legacy
`fsync` + atomic-rename JSON recipe guarded against. Writes are
serialised through an internal promise chain so back-to-back
`save()` calls never interleave; callers use a synchronous API
and the actual write is awaited in the background.

Each persisted app entry records `name`, `pid`, `status`,
`mode` (`classic` | `bootstrap`), `startedAt`, `restarts`, and
`port`:

| Mode        | When                                                 | Layout                                          |
| ----------- | ---------------------------------------------------- | ----------------------------------------------- |
| `classic`   | App was launched via classic launcher (single fork)  | One PID + status per app                        |
| `bootstrap` | App was launched via the PM supervisor (per-process) | Per-process PID + status, parent app aggregates |

**Legacy migration.** On the first boot of the SQLite-backed
code, if an old `~/.omnitron/state.json` exists it is read once,
migrated into the SQLite row, and unlinked. After that the JSON
file never reappears.

State is persisted on every status transition (start, stop,
crash, restart). A crash of the daemon itself leaves the row in a
consistent state — on next boot, the daemon rehydrates and
relaunches whatever was running.

### File permissions and SQLite locks

State, recovery certificates and the log journal use owner-only directories
(`0700`) and files (`0600`) on POSIX systems. Permission checks on an open
SQLite database inspect metadata without opening or closing another descriptor.
Closing any descriptor for the same inode can cancel the process's SQLite
locks, including when the descriptor was opened only to tighten permissions
([SQLite locking guidance](https://www.sqlite.org/howtocorrupt.html#_posix_advisory_locks_canceled_by_a_separate_thread_doing_close_)).

If existing permissions need repair, Omnitron performs the no-follow,
owner-checked operation in a separate short-lived Node process and verifies the
result. Already-private databases need no subprocess. Symbolic links and
additional hard links are rejected. Equivalent Windows protection depends on
OS ACLs; POSIX modes do not establish Windows access control.

Use SQLite's online backup API for live databases. Do not copy, replace or
remove a live database's WAL/SHM files. A recovery status held in memory is not
sufficient evidence for activation: schema transitions require a current,
durably readable certificate identifying the running backend and owner.
`pnpm --filter @omnitron-dev/omnitron test:private-storage` checks cross-process
writer exclusion, repeated readers, committed visibility and file permissions
against the compiled runtime; the CI build runs it before accepting changes.

## Daemon scheduler — `daemon-scheduler.ts`

A small set of periodic jobs registered against `titan-scheduler`
and tied to the daemon's lifecycle. `registerDaemonJobs()` wires:

| Job                  | Cadence                                          | Notes                                        |
| -------------------- | ------------------------------------------------ | -------------------------------------------- |
| `metrics-collection` | `monitoring.metrics.interval` (default 5 s)      | Polls app state, records via `titan-metrics` |
| `log-rotation`       | 60 s                                             | `LogManager.checkRotation`                   |
| `session-cleanup`    | 5 min                                            | Master only (requires the auth service / PG) |
| `alert-evaluation`   | `monitoring.healthCheck.interval` (default 15 s) | Master only (requires PG)                    |
| `fleet-heartbeat`    | `monitoring.healthCheck.interval` (default 15 s) | Master only (requires PG)                    |

The master-only jobs are skipped on a standalone single-node
setup where the auth / alert / fleet services aren't wired.

## Daemon configuration

The daemon's own config (not project / ecosystem config) lives in
the `DEFAULT_DAEMON_CONFIG` object — most operators never touch
it. Values:

| Field                            | Default                             |
| -------------------------------- | ----------------------------------- |
| `socketPath`                     | `~/.omnitron/daemon.sock`           |
| `port` (TCP)                     | `9700`                              |
| `host` (TCP)                     | `127.0.0.1`                         |
| `httpPort`                       | `9800`                              |
| `pidFile`                        | `~/.omnitron/daemon.pid`            |
| `stateFile`                      | `~/.omnitron/state.json`            |
| `logDir`                         | `~/.omnitron/logs`                  |
| `role`                           | `master`                            |
| `cluster.enabled`                | `false`                             |
| `cluster.discovery`              | `postgres`                          |
| `cluster.leaseTtlMs`             | `15 000 ms`                         |
| `cluster.heartbeatInterval`      | `2 000 ms`                          |
| `secrets.provider`               | `file`                              |
| `secrets.path`                   | `~/.omnitron/secrets.enc`           |
| `healthMonitor.intervalMs`       | `60 000` (1/min)                    |
| `healthMonitor.concurrency`      | `20`                                |
| `healthMonitor.offlineTimeoutMs` | `90 000`                            |
| `healthMonitor.retentionDays`    | `90` (uptime history)               |
| `healthMonitor.uptimeIntervalMs` | `86 400 000` (24 h per bar segment) |

Override via CLI flags on `omnitron up` or via the daemon
configuration file (advanced — not part of the project ecosystem
config).

### Console request limits

The daemon shares one Netron limiter across HTTP and WebSocket. Configure `requestRateLimit` in the daemon settings. Its defaults are 3,000 requests per client and 6,000 total per 60-second window. Switching transport or opening another tab consumes the same allowance. The legacy `httpRateLimit` setting covers both listeners. If both settings exist, `requestRateLimit` takes precedence.

The Console gateway overwrites `X-Forwarded-For` and `X-Real-IP` on HTTP and WebSocket. With the default `trustProxy: false`, clients behind the same proxy share its socket-address allowance. Set `trustProxy: true` only when access to the upstream listeners is restricted to that trusted proxy. The local administrative Unix socket has its own OS access boundary. [Netron request budgets](/docs/titan/netron/transports#share-a-request-budget-across-http-and-websocket)

## RPC surface — `OmnitronDaemon` service

The daemon registers `@Service({ name: 'OmnitronDaemon' })` on
the Netron bus. Verified from
[`src/daemon/daemon.rpc-service.ts`](https://github.com/omnitron-dev/omni/tree/main/apps/omnitron/src/daemon/daemon.rpc-service.ts).

The daemon checks these roles for each request.

| Role                    | Members                                                     | Methods                                                                                                                                             |
| ----------------------- | ----------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------- |
| Control-plane read      | `viewer`, `operator`, `admin`, authenticated `service_role` | `list`, `getApp`, `status`, `getMetrics`, `getHealth`, `getLogs`, `inspect`, `inspectInstalledArtifacts`, `getDependencyGraph`, `getWatchStatus`    |
| Control-plane operation | `operator`, `admin`, authenticated `service_role`           | `getEnv` (redacted values), `attestApplicationOwner`                                                                                                |
| `operator`              | `operator`, `admin`                                         | `startApp`, `startAll`, `stopApp`, `stopAll`, `restartApp`, `restartAfterChange`, `restartAll`, `reloadApp`, `scale`, `enableWatch`, `disableWatch` |
| `admin`                 | `admin` only                                                | `shutdown`, `reloadConfig`, `setMetricsEnabled`, `exec`, `revealEnv`                                                                                |
| anonymous               | anyone with socket access                                   | `ping` (allowAnonymous)                                                                                                                             |

### Methods, by intent

#### App lifecycle (`operator`)

| Method                                                         | Returns                |
| -------------------------------------------------------------- | ---------------------- |
| `startApp({ name })`                                           | `ProcessInfoDto`       |
| `startAll()`                                                   | `ProcessInfoDto[]`     |
| `stopApp({ name, force?, timeout? })`                          | `{ success: boolean }` |
| `stopAll({ force? })`                                          | `{ count: number }`    |
| `restartApp({ name })`                                         | `ProcessInfoDto`       |
| `restartAfterChange({ name, generation, definitionRevision })` | `SourceRefreshReceipt` |
| `restartAll()`                                                 | `ProcessInfoDto[]`     |
| `reloadApp({ name })`                                          | `ProcessInfoDto`       |
| `scale({ name, process, instances })`                          | `ProcessInfoDto`       |

`scale` requires an explicit declared pool process and a positive safe integer.
It returns the existing `ProcessInfoDto`; read `processes` for the selected pool's
observed `instances` and unchanged `declaredInstances`. Resizing lasts until
restart and leaves automatic scaling enabled. The owner enforces the pool's
configured bounds and must advertise `processPoolResize@1`; an older owner
refuses before mutation. This optional capability does not prevent a backend
update when the owner remains unchanged.

`restartAfterChange` is the automatic source-refresh operation used by SDK
watch mode. Supply the canonical application name, generation and declaration
revision observed in `getWatchStatus()`. The execution owner must advertise
`sourceRefresh@1`; it refuses stale bindings, disabled watching and pinned release
applications before restarting. The response is the owner's exact
`SourceRefreshReceipt`: `{ process, source }`. `process` confirms a new online
generation; `source` carries that same generation with its `name`, `directory` and
`definitionRevision`. A later inventory read must not be substituted for this
acknowledgement. A successful build alone does not prove that an application
loaded it. Application health probes remain separate from the refresh receipt.

#### Inspection (`viewer`)

| Method                                 | Returns                                             |
| -------------------------------------- | --------------------------------------------------- |
| `list()`                               | `ProcessInfoDto[]`                                  |
| `getApp({ name })`                     | `ProcessInfoDto`                                    |
| `status()`                             | `DaemonStatusDto`                                   |
| `getMetrics({ name? })`                | `AggregatedMetricsDto`                              |
| `getHealth({ name? })`                 | `AggregatedHealthDto`                               |
| `getLogs({ name?, lines? })`           | `LogEntryDto[]`                                     |
| `inspect({ name })`                    | `AppDiagnosticsDto`                                 |
| `inspectInstalledArtifacts({ names })` | Installed artifact evidence per application         |
| `getDependencyGraph({ name })`         | Graph object                                        |
| `getWatchStatus()`                     | `{ enabled, watching, apps: WatchedApplication[] }` |

#### Owner and environment inspection

`attestApplicationOwner({ owner: { id, pid, startedAt }, epoch })` returns the
clean build stamp of the observed owner. The node checks its active controller
lease. For a legacy owner, it verifies the supervised process and sealed runtime
before checking the lease again. Normal status polling performs no disk scan.
See [application release admission](/docs/omnitron/releases#build-and-inspect).

`getEnv({ name })` returns redacted environment values. `revealEnv({ name })`
requires an administrator and releases the clear values only after the audit database
confirms the write. A node without audit storage can still return redacted values.

#### Watch control (`operator`)

| Method                   | Returns                              |
| ------------------------ | ------------------------------------ |
| `enableWatch({ apps? })` | `{ watching: WatchedApplication[] }` |
| `disableWatch()`         | `{ success: boolean }`               |

#### Admin (`admin`)

| Method                                  | Returns                |
| --------------------------------------- | ---------------------- |
| `shutdown({ force? })`                  | `{ success: boolean }` |
| `reloadConfig()`                        | `{ success: boolean }` |
| `setMetricsEnabled({ name?, enabled })` | `{ success: boolean }` |

#### Heartbeat (anonymous)

| Method   | Returns                    |
| -------- | -------------------------- |
| `ping()` | `{ uptime, version, pid }` |

#### RPC plumbing (`admin`)

| Method                                  | Returns                                        |
| --------------------------------------- | ---------------------------------------------- |
| `exec({ name, service, method, args })` | `unknown` (whatever the called method returns) |

`exec` is what `omnitron exec api users findById u_42` routes
through. It is **admin-gated** — invoking arbitrary methods on a
managed app is a privileged operation.

## Auth flow

Authentication is intentionally split between transports:

```mermaid
flowchart LR
  CLI[CLI command]
  Webapp[Webapp]
  Remote[Remote daemon]

  CLI -- Unix socket<br/>OS-level identity --> AuthBypass[implicit admin]
  Webapp -- HTTP + JWT --> JWT
  Remote -- TCP + JWT --> JWT

  JWT[Verify JWT<br/>read roles]
  JWT --> RoleCheck{Role enough<br/>for method?}
  RoleCheck -- yes --> Invoke[Run method]
  RoleCheck -- no --> Reject[403 / FORBIDDEN]
```

The Unix socket auth-bypass is **deliberate**: file mode `0o600`
means only the daemon's UID can open it. If the OS trusts you to
write to the socket, the daemon trusts the request. TCP and HTTP
always require JWT.

## Daemon client — `daemon-client.ts`

The operator API exposes the typed local client through
`@omnitron-dev/omnitron/operations`. It connects to the daemon owner's Unix
socket and has the same authority as the local CLI. Keep it on the operator
host, not in browser code.

```typescript
import { createDaemonClient } from '@omnitron-dev/omnitron/operations';

const client = createDaemonClient();
try {
  await client.ping();
  await client.startApp({ name: 'api' });
  const metrics = await client.getMetrics({ name: 'api' });
  console.log(metrics);
} finally {
  await client.disconnect();
}
```

Local and fleet TCP clients share connection lifetime handling. Concurrent
calls share one connection attempt; retained service handles reconnect after
backend replacement. Connect, reachability and teardown have separate
bounds. A lost in-flight reply rejects the call without replaying it. See
[long-lived RPC clients](/docs/omnitron/backend-handoff#long-lived-rpc-clients) before
upgrading an application that retains a client across backend replacements.

## Auto-restart and backoff

Applications inherit the ecosystem's `supervision` settings. Override them for
one app with `apps[].restartPolicy`. By default, up to five automatic restarts
may begin in a rolling 60-second window. The first retry waits 1 s. Later
retries in the same window use exponential backoff up to 30 s.
Set `maxRestarts: 0` or `enabled: false` on the app's restart policy to
disable automatic retries.

If a classic script uses its retries and fails again, the owner marks it
`crashed`, emits `app:escalated` and waits ten minutes before trying again
with a fresh budget. For Titan-supervised children, an exhausted budget
ends automatic recovery of that child. A critical child failure escalates
to its supervisor.

Use `omnitron stop <app>` during backoff or cool-down to cancel the pending
launch. The app stays stopped until a new start is requested. When ready, run
`omnitron start <app>`. For a stopped stack, use
`omnitron stack start <project> <stack>` to restore its applications together.

## Failure scenarios

| Scenario                                  | What the daemon does                                          |
| ----------------------------------------- | ------------------------------------------------------------- |
| Classic app exits during a requested stop | Mark the app stopped                                          |
| Classic app exits unexpectedly            | Record the exit and apply the configured restart policy       |
| Child process hangs (no heartbeat)        | Health probe fails; restart after grace period                |
| Daemon itself crashes                     | systemd / launchd respawns; persisted SQLite state rehydrates |
| Lock-file PID is alive but daemon hung    | `omnitron kill` force-removes the lock                        |
| Two `omnitron up` race                    | Second loses the PID lock; exits with error                   |
| Storage failure persisting state          | Logs at error; in-memory state continues; next flush retries  |
| Disk full                                 | State persistence + logs fail loudly                          |
| Cluster split (cluster.enabled)           | Both sides may elect a leader; manual `step-down` resolves    |

## Operating runbook

| Goal                                | Command                                                                                           |
| ----------------------------------- | ------------------------------------------------------------------------------------------------- |
| Is the daemon up?                   | `omnitron ping`                                                                                   |
| What's running right now?           | `omnitron status` / `omnitron --json status`                                                      |
| Force shutdown                      | `omnitron down` (graceful) or `omnitron kill` (forceful)                                          |
| Apply changed daemon startup policy | Prepare and review a [backend replacement](/docs/omnitron/backend-handoff), or schedule a restart |
| See the lock holder                 | `cat ~/.omnitron/daemon.pid`                                                                      |
| Inspect replacement/recovery state  | `omnitron backend status`                                                                         |

Do not delete `daemon-state.db` to reset a process: it also holds project and node
registrations, execution intent and the backup index. Deleting `secrets.enc`
destroys the file-provider vault. Recover these stores only from a verified backup
with a reviewed data-loss and rollback plan. Use normal application and stack
lifecycle commands for process changes.

## Anti-patterns

- **Editing the daemon state DB by hand.** Persisted state lives
  in a SQLite DB (`~/.omnitron/data/daemon-state.db`), not a
  hand-editable JSON file. The schema may evolve; poke it directly
  and you risk a broken next restart. Prefer using the CLI to
  drive state changes.
- **Running two daemons against the same `~/.omnitron/`.** They
  fight over the lock. Multiple daemons need different home
  directories.
- **Exposing the TCP port without a JWT secret.** Anyone reaching
  the port can inspect at minimum, and operate / admin with default
  roles. Configure auth before opening port `9700`.
- **Treating `kill` as routine.** Force-kill skips graceful
  shutdown — pending logs may be lost, child processes may be
  reparented to init. Use `down` first.
- **Removing the lock file while the daemon is running.** Causes
  inconsistent state where two daemons can subsequently start.

## See also

- [Architecture](/docs/omnitron/architecture) — where the daemon fits in
  the broader picture
- [Orchestrator](/docs/omnitron/orchestrator) — what the daemon hands apps off to
- [Services reference](/docs/omnitron/services-reference) — all 19 RPC services
- [CLI](/docs/omnitron/cli) — every command that talks to this daemon

## Updating a source runtime

In coupled mode the daemon directly supervises application processes. Replacing
it requires a controlled maintenance window. A resident owner can retain running
applications across a backend update. Check the reported execution mode before
choosing the [handoff procedure](/docs/omnitron/backend-handoff).

1. Record the active build, enabled stacks, application states and remote node
   process IDs. Save the daemon configuration, account database, project registry,
   runtime dependencies and builds needed for rollback. Preserve persistent data
   volumes and existing experiment registrations.
2. Build and validate the target revision separately. Use the repository's exact
   `packageManager` version and test native dependencies with the Node executable
   used by the installed service. Keep the package manager in a permanent path
   available to that service.
3. Unload the OS supervisor before replacing a watched checkout. Wait for owned
   application processes to stop. Fast-forward clean working copies and install
   from the verified lockfiles; preserve other branches and local configuration.
4. Install each source application's adapter and Titan from its frozen lockfile.
   They must resolve one physical Titan package. The daemon may use its own
   SDK. Check the active owner's capabilities when upgrading an older
   installation. Published deployments use their pinned framework artifacts.
5. Build the daemon and Console from the same revision, then restore OS supervision.
   Check each enabled stack and its child processes against the saved baseline.
   A responding daemon does not mean its applications have finished starting.
6. Reload the Console and verify [access controls](/docs/omnitron/access),
   [SSH host identity and sessions](/docs/omnitron/terminal), application health and audit
   records. Confirm remote node processes remain unchanged for a local upgrade.

Keep immutable experiment manifests intact when their recorded package trees are
unavailable. Recover the original environment or create a separately registered
experiment; changing old hashes would invalidate its reproducibility.

### Execution handoff: architecture boundary

In coupled mode the control daemon owns Titan PM supervisors, fork IPC
channels, worker threads and application routing. Fork workers terminate on parent disconnect; the parent-death watchdog
and cold-start janitor enforce that ownership. Simply disabling `stopAll` or
reusing PIDs would either lose in-process workers or create unmanaged processes.

In resident mode a separately supervised execution owner uses Titan PM to manage
application processes, threads, restart policy, log capture, metrics and the
child registry across backend replacements. Omnitron
retains project orchestration and RBAC, communicating through authenticated,
local Netron RPC. A private socket and a persistent host identity must be verified
before a controller can attach; PID existence alone is not identity evidence.

Ownership transfer requires a monotonic fencing epoch and one active writer.
Prepare the new controller, reconcile its desired configuration with the host's
actual registry, then atomically transfer the writer lease. Commands carry epoch
and operation identity; the host rejects old writers and deduplicates retries.
A controller outage leaves existing work supervised, while new mutations stop
until authorization and ownership can be verified. Telemetry has a bounded,
observable buffer and resumes with acknowledgements rather than silent loss.

Use the [backend handoff procedure](/docs/omnitron/backend-handoff) to inspect
capabilities, prepare the candidate, switch controllers and verify rollback. Migration of existing coupled workers needs one controlled restart;
only workers already owned by the execution host can survive later control-daemon
updates. Immutable application runtimes and experiment registrations must remain
separate from controller upgrades.

---

---
id: "omnitron/diagnostics"
title: "Operational diagnostics"
---

A diagnostic must say what failed, what was observed, and what can restore
service. Omnitron uses the same finding contract in commands, runtime health and
notification adapters. Capture does not mean recovery, and broker acceptance
does not mean a person received a message.

`doctor` checks the complete internal database schema, including Titan’s
`metrics_samples` table. The normal test typecheck verifies that the required
table list covers inherited module tables as well as Omnitron’s own tables.
A missing metrics table is a database finding, not a healthy empty chart.

## Investigate a failure

```bash
omnitron --json health
omnitron --json doctor
omnitron logs omnitron -n 100
omnitron inspect project/stack/application
omnitron migrate status --daemon
```

The Console's node diagnostics show the same health indicators as the CLI. The
`runtime-diagnostics` indicator retains unresolved runtime failures. Expand
**Evidence and recovery** to see causes, occurrence counts and next steps; the
log link preserves the selected node. Switching nodes clears the previous node’s
readings immediately. The dialog displays loading state until the selected node
answers, and an older request cannot overwrite a newer check.

| Failure boundary       | Evidence and response                                                                                                                              |
| ---------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------- |
| Application process    | Titan classifies operational errors and controls shutdown; Omnitron supervises process state and captures logs                                     |
| Daemon process         | Unhandled rejections, uncaught exceptions and startup failures produce structured, redacted diagnostics                                            |
| Scheduled work         | Session cleanup, metrics, alert evaluation, fleet heartbeat, log rotation and replication retain failure state until a successful run              |
| Startup dependencies   | Failed migrations, missing scheduler registration, telemetry, traces and leader election remain visible in health                                  |
| Database               | Query rejection, nested causes, driver codes, transaction/rollback failures and migration integrity remain distinguishable                         |
| Event observers        | Async regular and wildcard failures reach the caller; notification lifecycle observer failures degrade health without resending delivered messages |
| Alert delivery         | The existing alert outbox retains pending transitions and records delivery failures, including absent consumers                                    |
| Notification transport | The diagnostic adapter distinguishes queued, duplicate, unavailable consumer and failed publication                                                |

Runtime health is a bounded projection of this daemon process, not a second
persistent event store. It keeps at most 100 active diagnostic codes and counts
repeats. The daemon log is the persistent record. Evicted records are reported as
lost projection coverage; restarting clears process-local state, not the log.
Unhandled failures are not automatically declared recovered. Periodic jobs clear
their own finding only after their next successful execution.

The emergency stderr boundary uses native synchronous writes. A full disk, closed
pipe or unwritable descriptor cannot emit an unhandled stream error and terminate
the owner while it is reporting another failure. Failed writes retain their count,
last error and timestamp in `runtime-diagnostics` health, even after the original
finding resolves. These records remain in bounded memory; unavailable storage
means they are not durably recorded. The next successful output attempt clears
the current output error, while the failure count remains. Titan logger sink
health and the durable log journal report their own asynchronous failures and
commit acknowledgments; accepting a log call does not prove persistence.

The process monitor observes exceptions without replacing Titan's shutdown
policy. A fatal exception is not made safe by logging it. A healthy status also
does not prove application correctness: register domain health checks and
monitor their coverage. Older daemons without the runtime indicator appear as
unexamined coverage in `doctor`.

## Process, health and start prerequisites

`stack list`, `stack status` and Console show process lifecycle separately from
application health. **Running** means the expected processes were observed online;
it does not prove their domain checks passed. **Healthy**, **Health warning**,
**Unhealthy** and **Health unknown** describe those checks. A deliberately stopped
application has **Not running** health, rather than an invented probe failure.

Stack detail preserves the application, physical node and failed or warning
checks. A healthy replica cannot hide an unobserved replica on another node.
Missing measurements and failed owner/node reads remain unknown. Concurrent local
stack readers share one owner observation, and a process replaced during that read
cannot inherit the previous process's health. CLI and Console use the same reading.

**Start prerequisites** (`stack diagnose`) is a separate, read-only preflight:
it explains what starting would require or change. It does not start applications
and cannot establish the health of a future launch.

## Stack infrastructure ownership

`omnitron stack diagnose <project> <stack>` and Console's stack diagnosis inspect
provisioning authority before checking inferred services. For `controller` and
`external` infrastructure ownership, **Infrastructure ownership** explains who
owns provisioning/migrations and **Application connections** remains **unknown**
until endpoint reachability, credentials and schema have actually been checked.
The node does not create substitute databases or containers to make that reading
green. Conflicting infrastructure declarations or a missing controller source
binding block diagnosis with a configuration reason. See
[infrastructure ownership](/docs/omnitron/infrastructure#infrastructure-ownership).

## RPC connection and replication

Run `omnitron node check <id>` or open **Nodes → Diagnostics**. Both use the
same connection reading. A running application, an SSH-reachable machine and a
working authenticated RPC session are separate observations.

Stack and release views report **Inventory observed** or **Inventory unavailable**.
The latter means the controller could not read application ownership; it does not
prove the host or transport is offline. The topology preserves that node as
**unknown**, and `stack status` reports its inventory as **unobserved**. Use the
node diagnostics to distinguish connection failure from an unavailable process
owner; replication can remain active while application inventory is unavailable.

| Observation                      | Meaning and next step                                                                        |
| -------------------------------- | -------------------------------------------------------------------------------------------- |
| Connected directly / through SSH | Recent RPC heartbeat and a verified service credential; check replication separately         |
| Opening / authenticating         | Connection establishment is in progress                                                      |
| Disconnected / retrying          | Read the transport error, cleanup error, next retry time and attempt number                  |
| Stale heartbeat                  | Current reachability is unknown; an old successful ping is not current health                |
| Catching up                      | Delivery is active, but the oldest pending entry is at least 60 seconds old                  |
| Delivery active, age unavailable | An older node reports acknowledgements but cannot prove queue age; upgrade it to verify lag  |
| Replication behind               | Entries are waiting and no acknowledgement has arrived for 60 seconds                        |
| Replication failing              | Inspect the reported read, ingestion or acknowledgement error; queued entries remain pending |
| Replication unknown              | No recent authenticated connection or fresh replication reading is available                 |

The master normally pulls every 15 seconds. **In sync** means an empty queue.
**Keeping up** requires both a recent acknowledgement and an oldest pending entry
under 60 seconds. A fresh acknowledgement alone cannot prove this: a large old
backlog can still be catching up. Older nodes without queue-age evidence show
**delivery active, queue age unavailable**, rather than a green sync state.
The diagnostic detail includes `oldestPendingAgeMs`, node-local `lastSyncAgeMs`
(to avoid clock skew), and `deliveryRatePerSecond` over a rolling 60-second
window, including idle time. Only newly acknowledged rows contribute; duplicate
acknowledgements do not refresh progress. Database read failures propagate as
unavailable, never as an empty queue. The slave's legacy `connected` field
represents its optional push channel and is not evidence about master-driven
replication. A failed status request is shown as **unavailable**, with its reason.

The optional telemetry relay has its own explicit observation. A controller using
a local database aggregator reports **local database** without requiring a remote
transport. A stopped relay, missing destination, transport failure or dropped
entries remain visible. Workers report **not applicable** because their logs and
metrics use the durable replication outbox; inspect its acknowledgement evidence
above. This is an explicit node reply, never an inference from a missing service
error. Older nodes without this contract, failed authentication and failed reads
remain **unavailable** with a reason, without fabricated zero-loss counters.

Netron disconnect events and heartbeat failures trigger reconnect with backoff
from 5 seconds to 2 minutes. Peer and SSH tunnel cleanup run independently and
have a five-second combined deadline, so a stuck close cannot disable retries.
A replacement connection authenticates again. `authenticated` becomes true only
after that peer accepts the service credential, and resets on disconnect. Responses from the old peer cannot
overwrite its state or acknowledge new work. RPC operations are not replayed
implicitly; only explicitly retryable, idempotent operations opt into retry.

`OmnitronNodes.getMeshStatus()` includes `lastHeartbeat`, `lastError`,
`reconnectAt`, `reconnectAttempt`, `cleanupError`, `syncStatus`, `syncStatusAt`,
`lastSyncStatusError`, `lastPullError` and `lastPullAt`. These timestamps describe
observations, not guarantees between polls. Console refreshes an open diagnostic
dialog every five seconds and pauses requests while it is hidden or busy.

## Classic services and completed work

Applications launched through `script` declare `readiness` in their ecosystem
entry. Without it they remain `degraded` with `readiness_not_configured`:
`online` alone does not establish readiness. Native bootstrap applications use
`hooks.onHealthCheck`; optional HTTP probes can supplement that verdict.

```typescript
{
  name: 'service',
  script: './service.mjs',
  readiness: {
    intervalMs: 5_000,
    timeoutMs: 3_000,
    probes: [{
      name: 'service HTTP', type: 'http',
      url: 'http://127.0.0.1:4720/health',
      expect: { status: 'ok', revision: expectedRevision },
    }],
  },
}
```

An HTTP probe requires the expected status (200 by default), refuses redirects
and optionally matches exact top-level JSON fields. This checks a declared
contract, not all business operations. Use `/` without `expect` for a frontend;
use domain fields for a service. Responses are bounded to 64 KiB; diagnostics
never include response bodies or URL query strings. URL credentials are refused.

For a classic background worker, declare one `heartbeat` probe:

```typescript
readiness: {
  probes: [{ name: 'delivery', type: 'heartbeat',
    file: '.local/health/delivery.json', maxAgeMs: 60_000 }],
}
```

Report **after real completed work**, not on a timer independent of that work:

```typescript
import { writeHealthHeartbeat } from '@omnitron-dev/omnitron/health';

await writeHealthHeartbeat({
  status: 'healthy',
  checks: [{ name: 'delivery', status: 'pass', message: 'Batch committed.' }],
});
```

Failures should publish `unhealthy` with named failed checks and a recovery hint.
An initial `degraded` report may explain that the first cycle is still pending.
Await each write; the helper atomically replaces a mode-0600 file. It is a no-op
outside the supervisor when no report path is configured. There is no automatic
healthy heartbeat. Set the freshness limit to cover actual work plus its normal
idle interval; a missing, stale, malformed or future report fails readiness.

Omnitron passes `OMNITRON_HEALTH_FILE` (resolved against the application's `cwd`)
and a fresh `OMNITRON_RUN_ID` to each classic launch. Wrappers must inherit these
variables. A report from an earlier launch is rejected. Other languages can
atomically publish the same UTF-8 JSON contract:

```json
{"schemaVersion":1,"timestamp":1793320000000,"runId":"supervisor-provided-id","status":"healthy","checks":[{"name":"delivery","status":"pass","message":"Batch committed."}]}
```

`timestamp` is the Unix time in milliseconds at completion; use the actual time,
not this example. Reports accept 1–32 named checks and at most 64 KiB. Symlinks
and special files are refused. An overall failure cannot be hidden by passing
checks. Bootstrap workers use Titan's health hook instead of heartbeat files.

Measurements are demand-driven by health readers. Concurrent reads share a
probe; results are cached for `intervalMs` (default 5 seconds). A changed
configuration or launch invalidates that cache. State transitions produce
`application.readiness.changed` daemon log events. CLI `health`, declared
`health-check` probes and Console use these same measurements. After adding a
heartbeat declaration, restart its worker to receive the channel variables.

### Immutable releases

An operator can add HTTP readiness without editing an installed release. In the
node's `~/.omnitron/config.json`, `applicationReadiness` maps exact qualified app
names to the same probe contract. For example:

```json
{
  "applicationReadiness": {
    "project/production/service": {
      "probes": [{ "name": "service HTTP", "type": "http", "url": "http://127.0.0.1:4720/health", "expect": { "status": "ok" } }]
    }
  }
}
```

Merge this field into the existing daemon configuration; do not replace other
settings. A controlled daemon restart activates it. Wildcards and heartbeat
injection are refused. This is a fallback when the application has no readiness
declaration; an application-owned declaration takes precedence. Native domain
health is always preserved. A new application release should declare its own
contract, allowing this operator fallback to be removed during normal rollout.

## Container state and stack intent

The Docker indicator reads Docker and the local stack registry. Containers of an
enabled or running local stack are required: missing or stopped containers fail
health. Unhealthy/starting healthchecks remain visible. A cleanly stopped
container of a known disabled local stack is reported as intentionally inactive;
its old failing healthcheck is not treated as a current failure. This never
starts, deletes or recreates containers. A stopped container omitted from its
active stack's **current reconciler inventory** is reported as retired. A matching
stack name alone cannot prove retirement. A retired container still running
remains a warning, and missing required containers still fail.

Unknown ownership is a warning, not proof that a container is unnecessary.
Unloaded configuration, unregistered stacks and failed desired-state reads remain
explicit. Inspect project configuration, stack startup and container logs before
changing intent. Docker unavailability fails the indicator even when a cached
reconciler state claimed readiness. All simultaneous failure reasons are retained.

## Domain readiness

Process state and readiness answer different questions. `online` means the
supervised process is running. A worker with a working event loop can still be
unable to commit data or complete its business operation.

Implement `hooks.onHealthCheck(app)` in the bootstrap and return the public
`AppHealthAnswer` contract:

```typescript
import type { AppHealthAnswer } from '@omnitron-dev/omnitron';

const report: AppHealthAnswer = {
  status: 'unhealthy',
  checks: [{
    name: 'ingestion',
    status: 'fail',
    message: 'source_stale: no committed observations in the required window. Check collector errors and queue write capacity.',
  }],
};
```

Name the failed dependency or stream. Include the stable error code, the observed
condition, the effect on service and the next useful check. Return these in
`checks[].message`; a private `details` object is not forwarded as diagnostic
checks. The daemon prefixes application/process names and carries the same
messages to `omnitron health <app>` and the Console's application health view.

- Probe successful work and freshness, not connectivity alone. Redis can answer
  `PING` while rejecting writes with OOM; a collector can receive events without
  committing any of them.
- Account for every required stream. Missing results, stale cycles and unreadable
  answers must not become healthy through an empty list or a default status.
- Preserve a worker's failure verdict when adapting an older health shape. Do not
  promote it because the adapter cannot identify the cause.
- Log failure and recovery transitions with Titan's structured logger. Include
  resource identity and evidence, suppress unchanged healthy polls, and avoid
  secrets and raw request payloads.
- Restore health only after the required work succeeds. Reconnecting a source
  may leave a data window incomplete for several minutes; retain that explanation
  until the window is usable. Do not erase failure evidence or invent missing data.

For registered experiments, place diagnostic adapters outside immutable worker
artifacts. Verify the pinned artifacts before and after activation, and restart
only the affected worker. Operational recovery must not silently create a new
implementation under an existing experiment identity.

## Supervise maintenance with the same health contract

Declare infrastructure maintenance as an ecosystem application when it needs an
independent lifecycle. Use Titan Scheduler for intervals, overlap prevention and
shutdown. Return named checks through `onHealthCheck`, so the CLI and Console
report the same policy, last completed attempt, dependency capacity and recovery
action. A scheduler tick alone is not proof that its operation succeeded.

For a durable stream pipeline, a safe cleanup policy needs both an age boundary
and acknowledgement by every consumer group. Inspect group positions and pending
entries atomically with deletion; a boundary computed by a previous observation
can already be stale. Refuse cleanup when the required journal consumer is
missing. Bound work per cycle and retry by recomputing the boundary after a lost
reply. Default a reusable maintenance component to observation mode; make active
deletion an explicit deployment policy.

Capacity recovery must remain possible when normal writes are rejected. For
example, inline stream trimming attached to `XADD` cannot free memory when Redis
rejects the entire write with OOM. Test the independent cleanup under that actual
failure, as well as pending entries, unread backlog, multiple consumers and
consumer creation between observation and execution. A capacity indicator must
read the write budget and preserve the failure until useful work resumes.

Log state transitions and bounded work summaries, not every successful poll.
Show observation-only mode and blocked cleanup as degraded readiness, with the
reason data was retained. Keep maintenance separate from immutable application
artifacts when changing operational policy must not change an experiment's
registered implementation. Document the producer's own retention limits too;
a maintenance guard cannot promise stronger guarantees than the entire pipeline.

## One transport-independent contract

Import from `@omnitron-dev/omnitron/diagnostics`, or from
`@omnitron-dev/titan/diagnostics` in a standalone Titan application.

- `DiagnosticFinding`: stable `id`, severity, title, evidence and optional remedy.
- `DiagnosticEvent`: a finding plus `eventId`, timestamp, source, raised/resolved
  state, and optional correlation/resource identifiers.
- `DiagnosticSink.deliver(event)`: returns a queued/duplicate receipt or rejects.
- `describeError`: bounded, cycle-safe nested cause formatting, including
  `AggregateError` and database driver codes. `describeError(error, 0, sensitiveValues)`
  also redacts supplied credential values before truncating messages or nested causes;
  use it when a subprocess error can include credentials as positional arguments.
- `sanitizeDiagnostic`: validates the contract, allowlists fields, redacts common
  credential forms and bounds text/evidence before delivery.

Only send operational facts. Credential redaction is a defense, not permission
to include request bodies, personal data, SQL literals or arbitrary metadata.
Raw stacks, headers and payload objects are not forwarded by the adapter.

## Optional Titan notification adapter

Messenger-specific integrations are deliberately separate. The universal bridge
uses a configured `titan-notifications` transport, its queue, retry policy and
DLQ. It creates no competing delivery engine and sends nothing until explicitly
wired and invoked.

```typescript
import { NotificationsModule } from '@omnitron-dev/titan-notifications';
import {
  DiagnosticNotificationsModule, DIAGNOSTIC_NOTIFICATION_SINK,
} from '@omnitron-dev/titan-notifications/diagnostics';

const diagnosticsModule = DiagnosticNotificationsModule.forRoot({
  notifications: NotificationsModule.forRoot({ redis: process.env.REDIS_URL! }),
  channel: 'diagnostics.operational',
});
// Import diagnosticsModule in your Titan module and inject
// DIAGNOSTIC_NOTIFICATION_SINK where diagnostics are published.
```

Register a consumer before publishing. Messages use type
`operational.diagnostic` and metadata `schemaVersion: 1`. The default channel is
`diagnostics.operational`; critical/error events request urgent/high priority.
Deduplication uses the event identity and transition with a 24-hour transport
window. Reuse the same identity across retries. Delivery outside that window can
duplicate, and receivers must remain idempotent. A queued receipt acknowledges
the broker only. Consumer acknowledgement, retries and DLQ handling determine
subsequent delivery; no end-to-end exactly-once guarantee is claimed.

`DiagnosticDeliveryError` means no consumer or an unconfirmed publication. A
failed stream rejects the entire fan-out, even if other streams accepted it.
Retry with the same event identity: confirmed streams deduplicate the retry,
while failed streams can accept it. Redis failures and invalid acknowledgements
never count as duplicates. Transport causes are retained with credential
redaction, including an error observer failing alongside publication. This lets
an existing durable producer keep its pending record until all matching
streams acknowledge the event.

Explicit deduplication keys preserve punctuation through percent encoding.
When upgrading, keys previously changed by underscore substitution start a new
broker deduplication window. Consumer idempotency must cover retries that cross
that deployment boundary.

## Connect the existing Omnitron alert outbox

`createAlertDiagnosticHandler(sink)` converts the payload of
`monitoring.alertSink` into the shared contract. Call it from the exposed method
configured as your alert sink. It preserves stable identities across retries and
produces separate firing/resolved event identities.

```typescript
import { createAlertDiagnosticHandler } from '@omnitron-dev/omnitron/diagnostics';
import { DiagnosticNotificationSink } from '@omnitron-dev/titan-notifications/diagnostics';

const handleAlert = createAlertDiagnosticHandler(
  new DiagnosticNotificationSink(configuredTransport),
);
// Your exposed alert method returns handleAlert(payload).
```

The alert outbox currently delivers critical alert transitions to configured
sinks. This adapter does not automatically forward all log lines or runtime
findings. Its timestamp records receipt at the adapter; the original alert
identity remains available for correlation. Messenger routing, credentials,
recipient policy and escalation are consumer responsibilities to add separately.

## Failure-injection checklist

Test denied access, an unreachable database, a rollback failure, a missing alert
consumer, a rejected event handler and recovery after each transient failure.
Check the original cause, health state and exit code, then verify that retries do
not repeat committed effects. [Migration guarantees](/docs/omnitron/migrations) describe the
transaction and checksum boundaries used by all framework layers.

## Storage exhaustion and execution ownership

A failed background state write is observed immediately and retained by the
resident owner. It does not become an unhandled promise rejection. Owner snapshots
report pending writes, failed-write count and the current persistence error;
`execution.state-persistence` makes the same finding available in CLI and Console
health. A later successful **full** snapshot or clear can recover current state
durability. Until then, the owner's state flush rejects and clean shutdown cannot
be certified.

The `log-sinks` health indicator reports backend destination failures and byte
gaps. `execution.log-sinks` reports the resident owner's equivalent evidence.
Healthy journals continue receiving records when another destination is full.
Successful later writes do not erase earlier unconfirmed or dropped bytes, and
custom transport error text is excluded from the health response. Inspect sink
capacity and retained journals before making a historical completeness claim.

A zero backend exit code alone does not prove an intentional platform shutdown.
When the owner is gone, the supervisor requires a `clean:true` receipt matching
its observed owner PID and current host boot before exiting as stopped. A dirty,
missing or stale receipt keeps the guardian blocked with a reconciliation reason;
it never launches another owner over surviving application processes. If the
supervisor's own state file cannot be written, its private status retains that
persistence failure in memory.

---

---
id: "omnitron/files"
title: "File workspace"
description: "Navigate, edit and transfer files across Omnitron nodes through the same audited Console and CLI contract."
---

# File workspace

Open **Files** in Console. Panels **A** and **B** both open on the local daemon. Each panel can hold tabs for local and remote locations. Opening Files does not contact a remote filesystem. A node must run a compatible Omnitron daemon. After a node upgrade or policy change, use the panel’s **Refresh** button to reload its available roots and directory listing. Connection and permission failures remain visible until a successful retry; an unavailable node is not treated as an empty filesystem. File operations use the authenticated Netron mesh; no SFTP server or browser-held SSH credential is required.

**Local means the daemon host.** Upload and Download refer to the computer running your browser. Selecting a project does not restrict filesystem authority: this is an administrator workspace.

## Start with a location

1. To work locally, navigate immediately in either panel. To open another location, press **+** in that panel and choose a node. To compare two directories on the same node, choose that node again. Escape cancels without opening a tab or reading the remote filesystem.
2. A node that exports one file area opens it directly. The default **Filesystem** is `/` on the daemon host, within its operating-system permissions. If a node exports several areas, choose one; read-only areas are marked. The tab keeps its node, area and directory history. To change node or area, open another tab. If access to an area is revoked, the tab reports that and does not switch to a different area.
3. Open a directory with its name, Enter or Right. Each non-root listing includes **..**; Enter on it or Left returns to the parent. The parent row remains available in empty/filtered directories and is never a selection or mutation target. Use path breadcrumbs, Back/Forward, Parent, or the path editor for navigation. Each panel supports eight independently navigable tabs.
4. Select entries with checkboxes or Space; Shift+click selects a range. The compact navigation toolbar keeps the copy direction visible as **A → B** or **B → A**, with named icon actions for New folder, Upload and panel focus. Copy reviews the exact source and destination before starting a daemon job.
5. Open **Transfers** for phase, file/byte progress, the current path, failures and Pause/Resume/Cancel. For a selected directory, the copy dialog also offers Archive and Pack & restore. The queue distinguishes source snapshot, encoding, encoded-byte transfer, restoration and publication; transferred bytes alone do not mean the destination is ready.

On narrow windows the A/B switch shows one panel at a time without discarding the other panel. On wider windows drag the separator, or focus it and press Left/Right, to adjust widths. Lists scroll inside the panel. Directory pages contain up to 200 entries; **Load next 200 entries** extends the current list. Filtering and sorting apply to loaded entries; directories sort first. Hidden entries are available from the panel menu.

| Shortcut, while focus is inside a panel | Action                                                                    |
| --------------------------------------- | ------------------------------------------------------------------------- |
| Tab / Shift+Tab                         | Switch file panels, after entering from either keyboard or pointer        |
| Escape                                  | Switch between list and panel tools; Tab in tools follows normal controls |
| F6 / Shift+F6                           | Leave the panel for the next/previous Console region                      |
| Enter / Space                           | Open / select focused entry                                               |
| Left / Right                            | Parent directory / open the focused directory                             |
| Up / Down                               | Move row focus                                                            |
| Alt+Left / Alt+Right / Alt+Up           | Back / forward / parent                                                   |
| Ctrl/Command+L                          | Edit the root-relative path                                               |
| F2 / F4 / F5                            | Rename / edit / review copy to other panel                                |
| Delete                                  | Review moving selected entries to trash                                   |

Keyboard entry from the sidebar lands at the same list focus target as pointer entry. Clicking a field or selector keeps native input behavior; dialogs and the editor own their keys. The active cursor is shown only in its owning panel, independently of selected files. Holding navigation keys repeats row movement.

Panel menus also provide New file, New folder, Upload, Download and Restore from trash. Symlinks are displayed but not followed; navigate to the real target path. Special files, sockets and devices are not opened as regular content.

## Edit without losing another writer's change

The editor loads on demand and supports UTF-8 files up to 1 MiB, eight buffers, search/replace, undo, line numbers and JSON/JavaScript/TypeScript highlighting. Ctrl/Command+S saves a changed writable buffer. Theme, wrapping, language-label and read-only changes preserve the CodeMirror instance, cursor and undo history. The editor focuses the active document, exposes line/column and selection length, and offers a Wrap control for long lines. F6 cycles tabs, tools and document inside the dialog; Escape can minimize the editor without discarding buffers. Buffers stay in memory when the editor is minimized or you navigate to another Console section; a page unload warns about unsaved buffers. They are cleared on sign-out and are not stored in browser storage.

Saving requires the version returned by the read. A stale version is rejected, leaving the buffer intact. **Save copy** creates a different filename without overwriting. **Reload** requires confirmation before replacing the current buffer. Closing a dirty buffer requires confirmation. Renaming a file or its parent within the workspace updates open buffer locations without losing unsaved text; an already-stale version remains stale. UTF-8 BOM and uniform LF/CRLF/CR line endings are preserved. Mixed line endings are previewed read only. Binary data and larger files use transfers/download instead.

A staged regular file is synchronized before atomic publication. Existing text files retain owner/group and basic permission bits. ACLs, extended attributes, sparse layout and hard-link relationships are not preserved by text replacement or copy. Version checks serialize Omnitron writes; they are not an operating-system compare-and-swap against unrelated processes. Avoid editing a file concurrently through another privileged program.

## Configure access per node

The default `/` export does not elevate the daemon to root. Unreadable directories and files return permission errors. Configure each node independently in `~/.omnitron/config.json`, restart it through your normal controlled rollout, and inspect **Files → Node → Root**. Merge the `files` field into the existing JSON object; preserve its role, authentication and transport settings. This is daemon configuration, not a `daemon` block in a project’s `omnitron.config.ts`.

```json
{
  "files": {
    "enabled": true,
    "roots": [
      { "id": "projects", "label": "Projects", "path": "/srv/projects", "protectedPaths": ["production/shared-data"] },
      { "id": "logs", "label": "Logs", "path": "/var/log", "readOnly": true }
    ]
  }
}
```

| Configuration                                  | Effect                                                             |
| ---------------------------------------------- | ------------------------------------------------------------------ |
| `files` omitted, or `files: { enabled: true }` | Administrator access to the `/` filesystem root                    |
| `files.roots: [...]`                           | Replace the default with explicit exports, up to 32 roots          |
| `readOnly: true` on a root                     | Deny editing, uploads, copy destination, rename, trash and restore |
| `files.roots: []` or `files.enabled: false`    | Export no directories                                              |

Root IDs are stable API identifiers; labels are presentation. Paths must be absolute. Root-relative request paths reject traversal, reserved staging names, control characters and symlink components. Roots are pinned to their canonical path and filesystem identity for the daemon lifetime. Changing a pinned directory requires configuration review and a restart. These checks constrain API clients; they do not form an OS sandbox against a privileged local process racing directory replacement. Use OS identities/mount isolation for hostile local workloads.

Structural safety is enforced on each node, independently of Console. The exported
root and fixed critical host directories (including `/etc`, `/usr`, `/var`, macOS
`/System` and `/Users`, and their structural aliases/subdirectories) cannot be
renamed or moved to trash. Add `protectedPaths` to a root for application-specific
entries, using root-relative paths. A protected entry and its ancestors cannot be
relocated; ordinary children remain browsable, editable and renameable according
to the existing root permissions. This is structural protection, not a blanket
filesystem write lock or an OS sandbox. Custom paths add to the fixed safeguards.

Rename changes one name within the same parent directory. Both UI and node reject
empty/reserved names, slashes, control characters and names exceeding 255 UTF-8
bytes. The form validates before submission; direct API/CLI requests receive the
same node checks. `protectedPaths` returned with roots is an advisory UI projection,
not authorization. Older nodes need a backend update to enforce the new policy.

Temporary writes and recoverable trash live in a private `.omnitron-files` directory in the item's parent directory, on the destination filesystem. The directory must belong to the daemon user and have mode 0700. It is hidden from listings and cannot be addressed through the API. This permits a `/` export without requiring write permission at `/`, and avoids cross-filesystem atomic-renaming problems.

## Transfer protocol and recovery

`OmnitronFiles` is the human-facing service shared by Console and CLI. Every public method requires `admin`; sensitive calls revalidate the current human session independently of cached transport authorization. Content reads and write intents are audited. Required intent persistence fails closed. File contents and chunk payloads are not written to audit details.

`OmnitronFileNode` is the authenticated mesh endpoint. It requires a signed `service_role` identity with a `mesh:` subject and accepts the typed v1 file request union, never arbitrary RPC names or shell commands. The control-plane peer namespaces the actor identity; upload receipts and trash are owned by that actor. The master resolves registered node IDs through the existing `SlaveConnector`. Human browser credentials cannot call this internal endpoint. The fleet TCP transport carries the verified peer identity into the same invocation context used by service guards and audit; missing identity fails closed.

Upload admission has a caller-generated idempotency key, persisted before the remote begin request so a lost response does not allocate another session. A copy first creates a bounded manifest, then transfers one verified block at a time. The receiving node synchronizes bytes before advancing its durable acknowledgement. Duplicate frames are accepted only when their already-acknowledged contents match. SHA-256 protects each block; commit records the complete destination hash. Regular-file publication uses a no-overwrite link from private staging, so partial files are not visible under their final name. A lost commit response is reconciled against the recorded destination inode and receipt, not blindly repeated as a new copy.

The current transport uses 256 KiB base64 frames over existing Netron RPC. It provides bounded memory and explicit backpressure with roughly one-third encoding overhead. Cross-node bytes currently pass through the coordinating daemon, not the browser. Direct peer binary streams and parallel adaptive transfer lanes are future protocol optimizations, not present guarantees.

The coordinator persists manifests and progress in the existing daemon state store. Restarted jobs become **interrupted** and require explicit Resume. New jobs persist their original request, so explicit Resume can retry interrupted planning. Legacy jobs without that request require a new copy. An archive plan already saved always reuses its admitted immutable snapshot, rather than silently recapturing changed source files. Pause takes effect at the next checkpoint; the in-flight block can finish. Current authority is rechecked during planning and between transfer blocks. Cancel removes unfinished staging when its outcome is known; it never rolls back already-published files. An ambiguous commit must be reconciled before cancellation.

In ordinary **Files** mode, existing destination files are never overwritten and existing directories can receive additional copied entries. Copy is a per-file operation, not a transactional tree snapshot: source changes fail their file's version check, and earlier completed files remain. A source directory changing after manifest creation does not turn the manifest into a live mirror.

| Bound                                        | Current value                                                                 |
| -------------------------------------------- | ----------------------------------------------------------------------------- |
| Text editor                                  | 1 MiB UTF-8, 8 browser buffers                                                |
| Directory                                    | 10,000 entries; 200 per page                                                  |
| Copy manifest                                | 200 selected sources, 10,000 entries, 32 path components                      |
| File / copy payload                          | 100 GiB                                                                       |
| Concurrent coordinator jobs                  | 4                                                                             |
| Per-node archive preparation / restoration   | 2 / 2                                                                         |
| Concurrent node requests / unfinished writes | 32 / 32                                                                       |
| Retained jobs                                | 100; completed/cancelled jobs with confirmed cleanup make room                |
| Node upload/trash receipts                   | 1,000 each; unfinished writes and unrestored trash never expire automatically |
| Portable browser download                    | 32 MiB; use CLI for larger files                                              |

The coordinator acknowledges final receipts only after persisting its next cursor; this frees receipt capacity without deleting published files. Browser and CLI uploads acknowledge after receiving a successful commit. Unacknowledged completed/aborted receipts can be reclaimed after 30 days when the journal is full. Restored trash receipts can be reclaimed when full. Reaching the limit without reclaimable records fails explicitly; files are not silently purged. Restore reconciles a move that succeeded before its journal acknowledgement failed, using the original filesystem identity; an unrelated destination remains a conflict. Trash consumes disk until restored or handled by an administrator outside the workspace. Permanent purge and configurable disk quotas are not implemented yet. Do not use the workspace as a backup system.

Browser upload/download progress belongs to that page; leaving or closing it can interrupt these operations. Node-to-node transfers are daemon-owned. After a browser upload loses its publication response, inspect its displayed receipt before retrying. Do not assume a network error means no file was published.

## Copy a directory through an archive

Select exactly one directory, choose the other panel's destination, and open Copy.
The output mode is explicit:

| Mode               | Result                                                                                 |
| ------------------ | -------------------------------------------------------------------------------------- |
| **Files**          | Ordinary recursive copy; each file is published separately                             |
| **Archive**        | A new archive file at the destination, without extraction                              |
| **Pack & restore** | A source archive transferred and verified, then restored into a new destination folder |

The format selector appears after a capability probe on the chosen nodes. Supported
adapters cover TAR, gzip/bzip2/xz/Zstandard TAR, ZIP and 7-Zip when the required
codecs pass their probe. Archive availability is checked on the source;
restoration requires the destination decoder too. The current destination probe requires a working local encoder/decoder pair; decoder-only installations are not advertised. 7-Zip currently produces archive files only. Formats are never inferred from
file extensions or displayed as available merely because an executable exists.
The chosen archive filename must end in the format's extension; a restored folder
uses a plain validated name. Capabilities show their observation time and can be
retried after an unavailable node; ordinary copy remains available.

The source creates a private snapshot with a bounded manifest, verifies each file
and rechecks the tree before encoding. Only regular files and directories are
accepted: no symlinks, devices, exported roots or protected structural trees. Limits
are 10,000 entries, 32 path components and 100 GiB each for source and encoded
bytes. Source snapshot storage and encoded data consume disk until completion or
cancellation. Pause stops coordinator work at checkpoints; already admitted
snapshot/restore work on a node can finish. Active native archive tasks also block backend handoff; shutdown cancels and drains them before releasing their journal. Cancel requests cleanup and preserves
an unresolved publication for explicit recovery.

Archive files use the regular atomic file-publication receipt. Restored directories
use an exclusive destination reservation and recorded filesystem identities:
existing entries are never replaced, and the folder can become visible while its
verified contents are published. This is **not** an atomic tree rename. The queue
marks incomplete publication; Resume reconciles the same receipt rather than
starting another copy. Consumers must wait for **completed** before using that
folder. Archive utilities emit bytes only; they do not choose filesystem paths or
extract archive entries into public directories. The admitted manifest determines
paths, lengths and hashes. Restored files retain the manifest's basic permission bits. Portable archive encoding preserves executable/group/world bits while ensuring owner-read for files and owner-read/write/search for directories; ownership, ACLs and extended attributes are not transported.

If copying or cancellation has completed but source snapshot removal or destination
receipt cleanup cannot be confirmed, the result remains completed/cancelled with
**cleanup pending**. **Retry cleanup** (CLI `files resume`) retries only that cleanup. It does not repeat
the transfer or overwrite the destination. Such jobs are retained until cleanup
is confirmed.

## CLI

`omnitron files` uses the same service. All commands support the global `--json` mode. The local CLI has the trusted local socket identity; it does not inherit the browser account, so its transfer/trash ownership is separate.

```bash
omnitron --json files roots --node local
omnitron --json files list srv --node local --root system
omnitron --json files stat srv/project/config.json --root system
omnitron --json files read srv/project/config.json --root system
omnitron --json files copy srv/project --root system \
  --to-node <registered-node-id> --to-root system --to-path srv/staging
omnitron --json files archive-formats --node <source-node> --to-node <destination-node>
omnitron --json files copy srv/project --node <source-node> --root system \
  --to-node <destination-node> --to-root system --to-path srv/staging \
  --archive-format tar.gz --archive-output archive --archive-name project.tar.gz
omnitron --json files copy srv/project --node <source-node> --root system \
  --to-node <destination-node> --to-root system --to-path srv/staging \
  --archive-format tar.gz --archive-output directory --archive-name restored-project
omnitron --json files transfers
omnitron --json files pause <transfer-id>
omnitron --json files resume <transfer-id>
omnitron --json files cancel <transfer-id>
```

`files list --after <name> --revision <version>` requests the next directory page. `files save <path> --from <local-file> --version <version>` performs a checked save; `--version new` creates without replacement. A successful `omnitron --json files save` returns the file metadata with its new revision. Use that revision for the next checked edit. `files rename <path> --to <root-relative-path> --version <version>` and `files trash <path> --version <version>` require an explicit observed version; the rename destination must retain the same parent directory. Use `files trash-list --root <id>` and `files restore <trash-id> --node <id>` for recovery. `files mkdir <path> --root <id>` creates a directory.

`files upload <path> --from <local-file> --root <id>` streams verified chunks. `files download <path> --to <new-local-file> --root <id>` stages locally and refuses overwriting. After an ambiguous upload, `files upload-status <id>`, `files upload-finish <id>` and `files upload-abort <id>` inspect/reconcile its receipt on the selected `--node`. Finish cannot publish incomplete data; abort cannot discard an unresolved committing receipt.

## Typed API clients

The same authenticated service is available through `IOmnitronFilesService` from
`@omnitron-dev/omnitron/dto/services`. Obtain its `OmnitronFiles` proxy through your
existing authenticated Netron connection; human calls require the same live admin
session as Console. Do not call the internal mesh endpoint directly.

```typescript
import type {
  IOmnitronFilesService,
  FileLocation,
  FileArchiveFormat,
  FileArchiveOptions,
  FileTransfer,
} from '@omnitron-dev/omnitron/dto/services';

async function copyFolder(
  files: IOmnitronFilesService,
  source: FileLocation,
  destination: FileLocation,
  format: FileArchiveFormat,
  name: string,
  output: FileArchiveOptions['output'],
): Promise<FileTransfer> {
  const entry = await files.stat(source);
  if (entry.kind !== 'directory') throw new Error('Select one directory');
  const capabilities = await files.archiveFormats({
    nodeId: source.nodeId, destinationNodeId: destination.nodeId,
  });
  if (!capabilities.formats.some(item => item.format === format && item.outputs.includes(output))) {
    throw new Error('The selected node pair does not support this format and output');
  }
  return files.copy({
    sources: [{ ...source, version: entry.version }],
    destination,
    archive: { format, name, output },
  });
}
```

The returned job is an admission result, not a completion receipt. Read `transfers`
and use `controlTransfer` to pause, resume or cancel. Wait for **completed** and
inspect `cleanupPending` separately; `bytes === totalBytes` does not confirm
restoration or publication. Capabilities can change after observation, so admission
and execution validate the selected format again.

| Contract                  | Meaning                                                                      |
| ------------------------- | ---------------------------------------------------------------------------- |
| `FileLocation`            | Registered node, exported root and root-relative path                        |
| `FileRoot`, `FileEntry`   | Root policy and observed metadata/version; metadata does not grant authority |
| `FileDocument`            | Bounded UTF-8 content, BOM and version used for checked saves                |
| `FileArchiveCapabilities` | Observed node formats, supported output modes and limitations                |
| `FileTransfer`            | Durable phase, progress, publication evidence, errors and cleanup state      |

`FILE_ARCHIVE_FORMATS` lists protocol formats; it does not claim they are installed
on a node. `fileNameError` and `fileStructureError` support advisory UI validation;
the node independently enforces name, structural and authorization checks.

## Design references and extension points

The interaction model compares representative managers across desktop, terminal and browser environments; it is not a claim to inventory every file manager.

| Reference                                                                                                               | Applied principle                                                   |
| ----------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------- |
| [WinSCP Commander](https://winscp.net/eng/docs/ui_commander) and [transfer queue](https://winscp.net/eng/docs/ui_queue) | Independent panes, explicit source/destination, background progress |
| [Total Commander](https://www.ghisler.com/featurel.htm)                                                                 | Two-panel operations and server-to-server work                      |
| [Midnight Commander](https://source.midnight-commander.org/man/mc.html)                                                 | Productive keyboard navigation and integrated editing               |
| [Yazi](https://yazi-rs.github.io/docs/quick-start/)                                                                     | Tabs, fast navigation and progressive detail                        |
| [Directory Opus](https://docs.dopus.com/doku.php?id=basic_concepts\:the_lister)                                         | Source/destination clarity and independent views                    |
| [File Browser](https://github.com/filebrowser/filebrowser) and [Filestash](https://www.filestash.app/sftp-client.html)  | A browser workspace combining navigation, transfer and editing      |
| [VS Code Remote SSH](https://code.visualstudio.com/docs/remote/ssh)                                                     | Visible remote identity while editing                               |

Prism supplies tabs with individual close controls, table viewports, confirmation dialogs, theme and application layout. MUI supplies tables, menus, breadcrumbs, responsive dialogs and accessible controls. CodeMirror supplies the lazily loaded code-editing engine rather than a second rich-text editor. No shared Titan/Prism package is rebuilt to add the workspace.

Future work is deliberately separated from the v1 contract: direct authenticated binary transfer, persistent browser-upload reconciliation, directory comparison/synchronization, recursive indexed search, configurable trash retention/quotas, richer preview codecs and optional per-root delegated agent grants. Agent access must be an explicit attenuation of human authority, not an admin token handed to a provider SDK. See [agent architecture](/docs/omnitron/agents) and [access](/docs/omnitron/access).

Active archive snapshots and restoration tasks block backend handoff until they settle. Daemon shutdown cancels and drains their subprocesses before closing the private staging store.

---

---
id: "omnitron/infrastructure"
title: "Infrastructure"
description: "Declarative provisioning of Postgres / Redis / S3 / custom containers."
---

# Infrastructure

Omnitron treats infrastructure as **declarative dependencies of
your apps**. An app declares what it needs (a Postgres database,
a Redis instance, a custom blockchain daemon, anything). The
owning stack provisions missing services, resolves connection parameters and
injects them at app startup. A node allocation or externally managed stack uses
explicit connection settings without provisioning another copy of those services.

Use the [topology workspace](/docs/omnitron/topology) to inspect placements and bindings.
Cross-node repair, resource-aware placement and data recovery follow the
[control-plane design](/docs/omnitron/resilient-control-plane); provisioning a container alone
does not implement those guarantees.

Verified against `apps/omnitron/src/infrastructure/`:

**infrastructure/**

- `infrastructure.service.ts` — orchestrator service
- `service-resolver.ts` — picks the right provider per stack
- `container-runtime.ts` — Docker provider
- `stack-infra-manager.ts` — per-stack lifecycle
- `server-registry.ts` — bare-metal connection registry
- `config-normalizer.ts` — schema validation
- `host-ingress.service.ts` — Shared host ingress lifecycle
- `host-ingress-config.ts` — Host gateway configuration
- `infrastructure-gate.ts` — readiness barrier (block app start until deps healthy)
- `phantom-endpoint-janitor.ts` — Clean up dead resolutions
- `presets/` — built-in presets (postgres, redis, minio, openresty, tor)
- `runtime/` — Provider runtime contracts
- `types.ts` — public types

## Two layers of declaration

| Layer         | What you declare                          | Where                                            |
| ------------- | ----------------------------------------- | ------------------------------------------------ |
| **Ecosystem** | What services exist for the whole project | `omnitron.config.ts` → `infrastructure: { ... }` |
| **App**       | What this specific app needs              | `config/default.json` → `omnitron: { ... }`      |

The ecosystem layer says "we run Postgres at port 5432 with these
databases"; the app layer says "I need access to the `main`
database with these env vars". The resolver wires the two
together.

## Infrastructure ownership

Dependency discovery answers what an application needs. It does not grant every
node permission to provision that dependency or run its migrations. A stack's
`infrastructureOwnership` declares that responsibility:

| Mode              | Responsibility                                                                                                                                               |
| ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `stack` (default) | The stack merges discovered requirements with infrastructure declarations, provisions services and owns their migrations                                     |
| `controller`      | A generated local node allocation starts applications using controller-supplied connection settings; its `managedOrigin` identifies the source project/stack |
| `external`        | The stack starts applications against explicitly configured existing services; an external operator owns provisioning and migrations                         |

Generated allocations include both fields together:

```typescript
stacks: {
  deployed: {
    type: 'local',
    apps: ['api'],
    infrastructureOwnership: 'controller',
    managedOrigin: { project: 'sample', stack: 'test' },
  },
}
```

The controller remains responsible for the source stack's infrastructure. Restarting
its node allocation must not reinterpret application requirements as new
`sample/deployed` containers, substitute default database/Redis endpoints or rerun
migrations. The generated allocation retains resolved application configuration
and its explicit source binding. This governs Omnitron's provisioning and migration
stages; application-owned startup hooks still follow the application's own
configuration and must respect the same ownership policy.

Non-owning modes apply only to local application allocations, not the controller's
remote/cluster stack. They reject stack-level infrastructure declarations and
`serviceOverrides`, and ignore ecosystem defaults belonging to other stacks;
supply application connections explicitly. An owning
stack can still use per-service external overrides when only some dependencies
are external. Ownership diagnostics distinguish these cases from an unavailable
Docker daemon. External readiness remains **unknown** until endpoint reachability,
credentials and schema are verified by appropriate checks; skipping provisioning
is not proof that a database is ready.

An ownership refresh can release obsolete in-process monitors when a stack becomes
non-owning. It does not stop or remove containers, alter volumes or run migrations.
A legacy allocation with `managedOrigin` but no explicit ownership mode fails
closed until migrated. An older unbound generated configuration cannot be identified
from its name; it needs an audited source binding and compatible runtime. The new
generator does not retroactively fix running nodes.

A maintenance stop with `--keep-infrastructure` also retains a durable
preservation policy. The reviewed preservation start rechecks existing native
resources without provisioning or rerunning setup and migrations. Plain starts
and autostart cannot bypass that policy. Follow the [preservation start review](/docs/omnitron/cli#restart-with-preserved-infrastructure)
when resuming the retained release; choose separately reviewed reconciliation
only when those changes are intended.

## Readiness includes required service setup

A running container, a successful native health probe, and completed service setup
are separate facts. Omnitron retains all three. A preset with `postProvision` must
finish successfully before the infrastructure becomes ready. PostgreSQL database
creation and required extensions, and MinIO bucket creation with anonymous access
disabled, are required setup. Partial success does not allow the stack to report
ready; the diagnostic identifies the failed declaration.

Use `omnitron infra status --stack <project>/<stack>` or `omnitron stack status <project> <stack>` to see the setup state and reason. Console shows the same evidence
in Containers and stack infrastructure. The Docker health column remains the native
probe result; it can say `healthy` beside a failed setup. `pending` means setup has
not completed, `ready` confirms completion for the observed container generation,
and `failed` includes its reason. An absent setup observation is **not verified**;
an older daemon or a native-only listing cannot prove setup succeeded. When the
local daemon is unavailable, the unscoped `infra status` labels native-only data.

Without `--stack`, the inventory covers this machine. With `--stack example/dev`,
the controller resolves the public project and stack, then selects local containers
by their registered ownership labels. For a remote or cluster stack the answer
comes from its nodes. A node that reports zero services is identified separately
from one that did not report its inventory. Incomplete per-node coverage returns
a nonzero exit code and preserves rows from responding nodes. If the controller
cannot resolve the requested stack, the command reports the failure.

Failed setup is retried with bounded backoff while native health permits it. This
does not restart an otherwise healthy container. New container identity, restart
time or changed setup declarations invalidate the previous confirmation. Backend
handoff preserves observations; native polling cannot erase a failed setup. An
older backend does not enforce these new setup diagnostics after rollback.

Custom preset hooks must be idempotent and throw on any required step failure.
PostgreSQL extensions must exist in the selected image. MinIO setup retries restore
`anonymous=none` even for a bucket created by an older configuration. Setup errors
use Titan diagnostics and redact known service credentials before limiting text.
An infrastructure readiness result does not attest application schema migrations,
external services or end-to-end application health; those have separate checks.

The native regression check `node test/runtime/preset-readiness.mjs dist` (from
`apps/omnitron` after building) requires cached PostgreSQL and MinIO preset images.
It creates only private containers with no network or published ports, bounded
CPU/memory and memory-backed data, and removes those exact containers afterward.

## Ecosystem layer — `InfrastructureConfig`

```typescript
// omnitron.config.ts
export default defineEcosystem({
  // ...
  infrastructure: {
    services: {
      db:      { preset: 'postgres', config: { databases: { main: {}, ledger: {} } } },
      cache:   { preset: 'redis' },
      storage: { preset: 'minio', config: { buckets: ['uploads'] } },
      gateway: { preset: 'openresty', config: { configDir: './infra/nginx' } },
    },
  },
});
```

Two forms supported per entry:

- **Preset shorthand** — `{ preset: 'postgres', config?: {...} }`. The
  preset registry expands to a full `IServiceRequirement`.
- **Full requirement** — direct `IServiceRequirement` object for
  custom services.

### Built-in presets

| Preset      | Type     | Default ports                  | Notes                                       |
| ----------- | -------- | ------------------------------ | ------------------------------------------- |
| `postgres`  | database | `5432`                         | Configurable extensions, databases, version |
| `redis`     | cache    | `6379`                         | Configurable maxmemory policy               |
| `minio`     | storage  | `9000` (API), `9001` (console) | Buckets, quota, public/private              |
| `openresty` | gateway  | `8080`                         | Mounts `configDir`, hot-reloadable Lua      |
| `tor`       | daemon   | configurable                   | Hidden service support                      |

Presets are versioned: bumping the omnitron version may change
preset defaults, but each project pins its own ecosystem config.

### Legacy shape

```typescript
infrastructure: {
  postgres:   { ... },
  redis:      { ... },
  minio:      { ... },
  gateway:    { ... },
  tor:        { ... },
  containers: { my-service: { ... } },
}
```

Still works for backward compatibility, but the unified
`services: { ... }` map is the canonical form going forward.

## App layer — `OmnitronAppConfig`

In each app's `config/default.json`:

```json
{
  "omnitron": {
    "database": true,
    "redis":    true,
    "s3":       { "bucket": "uploads", "quota": "10gb" },
    "services": {
      "discovery":     true,
      "notifications": true
    },
    "infrastructure": {
      "search": {
        "type": "daemon",
        "ports": { "http": 9200 },
        "env":   { "SEARCH_URL": "http://${host}:${port:http}" },
        "docker": { "image": "elasticsearch:8.11.0" }
      }
    }
  }
}
```

The resolver merges this with the ecosystem-level declaration
and produces the per-app env injection.

### Shorthand booleans

| Declaration                                | Effect                                                                                             |
| ------------------------------------------ | -------------------------------------------------------------------------------------------------- |
| `"database": true`                         | Auto-create one database for this app in the shared Postgres; inject `DATABASE_URL`                |
| `"database": { "dedicated": true }`        | Provision a dedicated Postgres container                                                           |
| `"database": { "dialect": "mysql" }`       | Dialect hint (the preset architecture supports alternatives; only `postgres` ships built-in today) |
| `"redis": true`                            | Auto-assign a Redis DB index; inject `REDIS_URL`                                                   |
| `"redis": { "dedicated": true }`           | Dedicated Redis container                                                                          |
| `"s3": true`                               | Default bucket on the shared MinIO                                                                 |
| `"s3": { "bucket": "X", "quota": "10gb" }` | Named bucket with quota                                                                            |

### Custom infrastructure

For apps that need infrastructure beyond the presets, declare a
full `IServiceRequirement` under `infrastructure: { <name>: {...} }`.

## `IServiceRequirement` — the universal shape

```typescript
interface IServiceRequirement {
  description?: string;
  type?: 'database' | 'cache' | 'daemon' | 'gateway' | 'storage' | 'sidecar' | 'tool';

  // Named ports this service exposes
  ports: Record<string, number>;

  // Env vars to inject INTO THE APP (not the service container)
  env: Record<string, string>;

  // Optional health check definition
  healthCheck?: IServiceHealthCheck;

  // Other services this service depends on
  dependsOn?: string[];

  // Secrets injected via ${secret:name} templates
  secrets?: Record<string, string | SecretRef>;

  // Docker provisioning (dev/test)
  docker?: IDockerServiceConfig;

  // Bare-metal provisioning (prod)
  bareMetal?: IBareMetalServiceConfig;

  version?:        string;
  networkMode?:    string;       // 'regtest' | 'testnet' | 'mainnet' | custom
  critical?:       boolean;      // default true; app blocks on this
  startupTimeout?: number;       // default 120 000
}
```

### Env variable templating

The `env` values support three template placeholders:

| Template           | Resolves to                        |
| ------------------ | ---------------------------------- |
| `${host}`          | Provisioner-resolved hostname / IP |
| `${port:<name>}`   | Resolved port for the named port   |
| `${secret:<name>}` | Decrypted secret value at runtime  |

Example:

```json
{
  "infrastructure": {
    "search": {
      "type": "daemon",
      "ports": { "http": 9200 },
      "env": {
        "SEARCH_URL":      "http://${host}:${port:http}",
        "SEARCH_API_KEY":  "${secret:searchApiKey}"
      },
      "docker": { "image": "elasticsearch:8.11.0" }
    }
  }
}
```

The app receives a literal `SEARCH_URL=http://localhost:9200` and
`SEARCH_API_KEY=<decrypted value>` at startup.

### Network modes

`networkMode` lets one declaration provision differently per
environment:

```json
{
  "ledger-daemon": {
    "type": "daemon",
    "ports": { "rpc": 18443 },
    "env":   { "LEDGER_URL": "http://${host}:${port:rpc}" },
    "docker": {
      "variants": {
        "regtest": { "image": "ledger:latest", "command": ["--regtest"] },
        "testnet": { "image": "ledger:latest", "command": ["--testnet"] },
        "mainnet": { "image": "ledger:latest", "command": ["--mainnet"] }
      }
    }
  }
}
```

Set `networkMode` at the stack level or via env var; the
resolver picks the matching variant.

## Provisioning flow

```mermaid
sequenceDiagram
  participant A as App declaration
  participant E as Ecosystem config
  participant R as Resolver
  participant CR as Container runtime
  participant SR as Server registry
  participant G as Infrastructure gate
  participant App as App process

  A->>R: merge app + ecosystem declarations
  E->>R: stack-level overrides
  R->>R: pick provider per service (docker vs bare-metal)
  alt provider = docker
    R->>CR: provision container
    CR->>CR: pull image, create network, start container
    CR-->>R: ContainerState
  else provider = bare-metal
    R->>SR: connect to declared server
    SR-->>R: connection info
  end
  R->>R: build env map (resolve ${host} / ${port:name} / ${secret:name})
  R->>G: signal service ready
  G->>App: release start gate
  App->>App: receives env, connects to infrastructure
```

The **infrastructure gate** (`infrastructure-gate.ts`) blocks app
start until every `critical: true` service is healthy. Non-critical
services are best-effort.

## Stack-level overrides

```typescript
stacks: {
  dev: {
    type: 'local',
    serviceOverrides: {
      // All apps using 'search' connect to a dev cluster:
      search: { external: { host: '10.0.1.5', ports: { http: 9200 } } },
    },
  },
  prod: {
    type: 'cluster',
    serviceOverrides: {
      // Use specific image in prod:
      search: { docker: { image: 'elasticsearch:8.11.0-secured' } },
    },
  },
}
```

`IServiceOverride` shape:

- `external: { host, ports }` — disable provisioning, point at an
  existing server.
- `docker: { ... }` — override the Docker config.
- `bareMetal: { ... }` — override the bare-metal config.
- `disabled: true` — skip this service entirely for this stack.

Overrides can also be app-scoped using `<appName>/<serviceName>`
keys:

```typescript
serviceOverrides: {
  // Only the analytics app's search uses a custom image:
  'analytics/search': { docker: { image: 'opensearch:2.x' } },
}
```

## Docker provider

Docker is the default for `dev` / `test` stacks. The provider:

1. Pulls the image (if not local).
2. Creates the container network if missing.
3. Starts the container with declared ports / volumes / env.
4. Polls Docker's healthcheck until healthy (or `startupTimeout`).
5. Reports `ContainerState` to the resolver.

Container lifecycle is reconciled — the provider periodically
verifies declared services match running containers. Drifted
containers are restarted; orphaned ones are removed.

### `IDockerServiceConfig`

```typescript
interface IDockerServiceConfig {
  image?:        string;                              // required unless `build` given
  build?:        { context: string; dockerfile?: string; args?: Record<string,string>; tag?: string };
  portMappings?: Record<string, number>;              // host-port overrides, keyed by port name
  bindHost?:     string;                              // default '127.0.0.1'
  environment?:  Record<string, string>;              // env for the SERVICE container
  volumes?:      Record<string, string | IVolumeMount>; // name → mount path | full spec
  command?:      string[];
  entrypoint?:   string[];
  user?:         string;
  healthCheck?:  ContainerHealthCheck;
  resources?:    ResourceLimits;
  restart?:      string;                              // default 'unless-stopped'
  shmSize?:      string;
  network?:      string | IDockerNetworkConfig;
  extraHosts?:   string[];                            // e.g. 'host.docker.internal:host-gateway'
  labels?:       Record<string, string>;
  variants?:     Record<string, Partial<IDockerServiceConfig>>;   // for networkMode
}
```

Note `volumes` is a **map** (`name → path | spec`), not an array,
and container env is `environment` (not `env`). There is no
`privileged` / `capabilities` field.

### Container resource limits

`docker.resources` applies to service containers and Docker applications through
the same allocator:

```typescript
resources: {
  memory: '512m',
  memoryReservation: '256m',
  cpus: 0.5,
  shmSize: '64m',
}
```

Memory sizes use Docker's binary units. A positive RAM limit or reservation
must be at least 6 MiB; a reservation cannot exceed a nonzero hard limit. The
reservation is a soft target, not a hard cap. Zero memory or CPU removes that
limit; it does not configure swap. CPU values must be finite, nonnegative and
exactly representable in nano CPUs. A zero shared-memory size selects Docker's
default. Top-level `docker.shmSize`, when present, overrides
`docker.resources.shmSize`.

Validation runs before creating networks, volumes or containers. After creation,
Omnitron checks the actual Docker `HostConfig`, including CPU-quota overrides;
a desired-spec label alone is not proof that the limits are enforced. The same
observation participates in infrastructure reconciliation and handoff adoption.

**Existing deployments:** earlier versions could retain resource declarations
without passing them to Docker. Reconciliation now reports those containers as
requiring recreation, including stateful services. Inspect the plan, named
volumes, image pins and actual limits before a maintenance window. A backend
handoff must reject mismatched infrastructure instead of silently accepting it;
replacement container IDs are expected when recreation is required. Existing
named volumes are reused, but container replacement interrupts the service.

### `IVolumeMount`

```typescript
interface IVolumeMount {
  source:    string;     // host path (bind mount) or named volume
  target:    string;     // path inside the container
  readonly?: boolean;
}
```

Named volumes are created on demand and survive container
restarts. Bind mounts read from host paths verbatim.

## Bare-metal provider

For `prod` stacks (or wherever you don't want Docker), declare
bare-metal provisioning hints. These tell Omnitron how to
install / locate / validate the service on the target host:

```typescript
{
  type: 'database',
  ports: { tcp: 5432 },
  env: { DATABASE_URL: 'postgres://${secret:db_user}:${secret:db_pass}@${host}:${port:tcp}/main' },
  bareMetal: {
    systemdUnit:     'postgresql',
    dataDir:         '/var/lib/postgresql/data',
    validateCommand: 'pg_isready',
  },
  secrets: {
    db_user: { secret: 'db_user' },
    db_pass: { secret: 'db_pass' },
  },
}
```

For an external host you don't manage, prefer a stack
`serviceOverride` with `external: { host, ports }` (see below) —
that disables provisioning and just wires up the connection.

### `IBareMetalServiceConfig`

```typescript
interface IBareMetalServiceConfig {
  apt?:             { latest: string[]; repository?: string };
  installCommand?:  string;   // install the service executable
  systemdUnit?:     string;   // systemd service name
  configFile?:      string;   // config path on the target machine
  configTemplate?:  string;   // template (uses ${...} syntax)
  dataDir?:         string;   // data dir; omnitron ensures it exists
  user?:            string;   // user to run the service as
  bindAddress?:     string;   // bind address for remote/cluster stacks
  validateCommand?: string;   // check whether the service is installed
  variants?:        Record<string, Partial<IBareMetalServiceConfig>>; // per networkMode
}
```

There is no service-discovery layer here (no `consul` / `k8s`
provider) — bare-metal config describes install + on-disk layout.
Live external endpoints are wired via `serviceOverrides.external`.

### Tracking host packages

Declare host dependencies under the bare-metal variant that consumes them. For example, a service that uses a system Tor proxy can track the stable release for the host distribution.

```json
{
  "apt": {
    "latest": ["tor", "tor-geoipdb"],
    "repository": "https://deb.torproject.org/torproject.org"
  }
}
```

The optional HTTPS repository selects its `<VERSION_CODENAME>/main` binary channel. Configure it on the node with a scoped `Signed-By` key. Without a repository, the host's APT policy selects the candidate. A systemd dependency such as `Requires=tor@default.service` orders services and carries no package version constraint.

`omnitron infra inspect <project>/<stack>` reads local indices and reports installed and candidate versions, package holds and source mismatches. Administrators can read the same evidence under **Stacks → Diagnose → Inspect hosts** in Console. Stack provisioning refreshes the indices before resolving versions. The preview lists any new transitive dependencies. Provisioning checks the selection again and refuses removals, downgrades, changes to other installed packages or a changed dependency set. Package maintainer scripts can restart services, so review a live upgrade in a maintenance window. Package checks run on inspection and provisioning. Configure scheduled patch maintenance on the host when you need updates between deployments.

The Docker `tor` preset has a separate package lifecycle. Its Alpine 3.24 entrypoint resolves Tor from the stable branch at startup, including when the container already has a package installed. Running processes keep their current binaries until restarted. Inspect the system proxy and the onion gateway independently.

## Container state

The `OmnitronInfra` RPC service exposes:

```typescript
interface ContainerState {
  name:        string;
  image:       string;
  status:      'not_found' | 'created' | 'running' | 'paused' | 'restarting' | 'exited' | 'dead';
  containerId?: string;
  ports?:      Record<string, number>;
  health?:     'healthy' | 'unhealthy' | 'starting' | 'none';
  healthCheck?: { completedAt: string; exitCode: number };
  startedAt?:  string;
  error?:      string;
}

interface InfrastructureState {
  services:        Record<string, ContainerState>;
  ready:           boolean;
  lastReconciled?: string;
}
```

`omnitron infra status` / `omnitron infra ps` print this.

Health is a current Docker probe only while the container is `running`. A stopped,
paused or restarting container reports its lifecycle status and `health: none`;
Docker's saved probe from its previous run is not reused as a current verdict.
Exit failures, OOM kills and their reasons remain available in `error`. The same
reading feeds CLI, Console and topology.

## Reconcile loop

A periodic reconciler runs inside the daemon:

```mermaid
flowchart LR
  T((tick)) --> Read[Read declared services]
  Read --> List[List actual containers]
  List --> Diff[Diff declared vs actual]
  Diff --> Decide{action}
  Decide -- missing --> Create[Create + start]
  Decide -- stopped --> Start[Start it again]
  Decide -- changed-spec --> Recreate[Stop + remove + create]
  Decide -- unhealthy --> Confirm{Fresh failures + elapsed grace?}
  Confirm -- no --> Wait[Keep container and report reason]
  Confirm -- yes --> Delay{Recovery backoff elapsed?}
  Delay -- no --> Wait
  Delay -- yes --> Verify{Same incarnation still failing?}
  Verify -- no --> Wait
  Verify -- yes --> Rebuild[Remove + create under service lock]
  Decide -- restarting --> Yield[Yield to Docker]
  Decide -- ok --> NoOp[Skip]
  Create --> Update[Update state]
  Start --> Update
  Recreate --> Update
  Rebuild --> Update
```

The reconciler runs every 30 s, plus on demand (`omnitron infra up` /
`infra down`).

### Applying changed declarations

After rereading a project's configuration, starting its stopped or failed stack
resolves the current infrastructure declaration again. The local manager and
receiving node apply this plan through the same provisioning path. An existing
observer is reused; its first declaration is not a permanent cache. Starting an
already running stack remains idempotent and does not deploy configuration changes.

A changed image, environment, mount, command or native health check goes through
normal specification reconciliation. Only affected containers are recreated;
unchanged services keep their native identities. Named volumes remain in place.
Changing database or bucket setup invalidates its previous readiness result even
if the container specification is unchanged, and required setup must succeed again.
Container replacement can interrupt the affected service: plan that interruption
and verify data/schema compatibility before applying a changed declaration.

The complete candidate is resolved before replacing the previous definition.
Earlier health sweeps, native reconciliation and endpoint cleanup drain first.
Concurrent provisioning, stop, ownership refresh or backend handoff cannot silently
acknowledge a conflicting operation for the same stack. A failed retry retains the
observer and the accepted desired state for diagnosis and retry. Invalid resolution
leaves the previous definition intact.

Removing a declaration releases its observer; it does **not** delete that
container or its data. Explicit removal still requires the ordinary reviewed stop
path for the resources owned by the stack. Monitoring-only ownership refresh also
remains separate: it cannot change the specification of a retained container.

Run `node test/runtime/stack-infrastructure-reconfiguration.mjs dist receipt.json`
from `apps/omnitron` after building to exercise native reconfiguration, idempotent
retry and reviewed stop/start with a retained volume sentinel. The fixture uses
two isolated containers from cached `alpine:3.24`, publishes no ports, and removes
only its own containers and temporary volume after verification.

### A container you stopped comes back

This is the branch operators meet first and it is easy to mistake for
something else. **A declared service that is not running is started again
within one tick** — up to 30 seconds after you stopped it, `docker stop`
looks undone. The daemon logs `Service not running — auto-restarting`.

It is worth knowing before you try to reproduce an outage against
infrastructure: `docker stop sample-dev-redis` lasts only until the next
reconciliation tick, rather than holding the service down. The container's own `RestartCount`
stays `0` — Docker did not do it — while `StartedAt` moves, which is how you
tell this apart from a Docker restart policy. To hold a service down, stop it
through `omnitron infra down` (which is a declared intent, not a drift the
reconciler corrects), or run a throwaway container the daemon does not manage.

### When it stops trying

A failed restart enters exponential backoff: 30 seconds, doubling to a ceiling
of 15 minutes. Recovery of a running **unhealthy** container uses the same delay
and counts every recreation attempt, including a successful creation whose new
container has not become healthy. `starting`, unknown health and a successful
create call do not establish recovery. A running container observed as `healthy`
(or declaring no native probe), with no failed required setup, clears the delay.
Backoff remains visible in the daemon's diagnostic log.

Unhealthy recovery requires all of the following:

- Different completed, failed Docker probes for the same full container ID and
  `StartedAt`; rereading one cached result does not count as another failure.
- At least 60 seconds since the first valid observation of that unhealthy
  incarnation, with a new failure at or after the deadline.
- A last probe no older than 60 seconds, not dated in the future or before the
  current container start. Missing or invalid native evidence leaves the
  container in place and logs why recovery is waiting.
- An expired retry delay and a fresh native inspection immediately before
  removal. A recovered, restarted or replaced container cancels that decision.

Docker's own `restarting` state is left to its native restart policy. Automatic
health recovery and ordinary provisioning use the same per-container mutation
lock, and both drain before monitoring ownership is handed off. Only the exact
native ID with matching Omnitron allocation labels can be removed. Probe command
output is excluded from `healthCheck`; the metadata contains only completion
time and exit code, so command output cannot leak through this diagnostic field.

Backend handoff retains native failure evidence and retry delays. Importing an
older snapshot starts a new confirmation window: historical poll counts cannot
prove distinct failures. A rollback to an older backend restores that version's
recovery policy; it does not retain the newer safety guarantees.

To verify Docker's real probe timestamps and elapsed grace, run
`node test/runtime/native-health-recovery.mjs dist` from `apps/omnitron` after
building. It uses a cached `alpine:3.24` image in one disposable container with
0.25 CPU, 32 MiB memory, no network, ports or volumes. It observes actual failure
and recovery for at least 60 seconds, then removes only its own container.

## Phantom endpoint janitor

Apps that crash or are removed leave behind stale env injections
in the state store. The phantom janitor sweeps these:

- Endpoints referenced only by removed apps → marked phantom.
- Phantom endpoints older than the retention window are dropped
  from the state store.
- Underlying Docker containers are not removed unless explicitly
  requested (operator might want to keep data around).

## Secrets in infrastructure

Declared via `secrets` per service:

```json
{
  "secrets": {
    "db_password": { "secret": "production_db_password" },
    "api_key":     { "secret": "search_api_key" }
  },
  "env": {
    "DB_PASS": "${secret:db_password}",
    "API_KEY": "${secret:api_key}"
  }
}
```

The string before `:` is the logical name used in templates; the
`{ secret: 'X' }` refers to a key in the daemon's encrypted
secret store (`omnitron secret set X ...`). Secrets are resolved
**at injection time** — never written to disk.

## Health checks per service

A health check is a single discriminated `{ type, target }` pair,
not a set of nested probe blocks:

```typescript
interface IServiceHealthCheck {
  type:    'http' | 'tcp' | 'command' | 'jsonrpc';
  target:  string;        // http: URL path · tcp: port name · command: shell cmd · jsonrpc: method
  interval?:    string;   // default '30s'
  timeout?:     string;   // default '10s'
  retries?:     number;   // default 5
  startPeriod?: string;   // grace before first check; default '30s'
  jsonrpc?: { port: string; method: string; path?: string; auth?: {...} };
}
```

```json
{ "healthCheck": { "type": "command", "target": "pg_isready -U postgres", "interval": "5s", "retries": 10 } }
{ "healthCheck": { "type": "tcp",  "target": "tcp" } }
{ "healthCheck": { "type": "http", "target": "/healthz", "timeout": "10s" } }
```

Four probe types — `http`, `tcp`, `command`, and `jsonrpc` (the
last for blockchain daemons). The infrastructure gate uses these
to decide when to release the start gate for dependent apps.

## API gateway integration

When `gateway` is configured (preset or custom), Omnitron
materialises an OpenResty config from your `configDir` plus
auto-generated upstream maps. The `gateway-generator.ts` does this.

```mermaid
flowchart LR
  Apps[Per-app HTTP ports]
  Apps --> Upstream[Generated upstream map]
  ConfigDir[Operator-written nginx config<br/>references upstreams by name]
  Upstream --> Mount[Mount into OpenResty container]
  ConfigDir --> Mount
  Mount --> Gateway[OpenResty / Lua]
  Gateway --> Browser
```

This means your app process can move ports and the operator-
written nginx config doesn't change — the generated upstream map
always points at the current location.

## CLI surface

| Command                                                                      | Effect                                                                                                                                              |
| ---------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------- |
| `omnitron infra up`                                                          | Provision all infrastructure                                                                                                                        |
| `omnitron infra down`                                                        | Stop and remove owned standalone containers by their observed Docker IDs. Their volumes are retained. The legacy `--volumes` flag returns an error. |
| `omnitron infra status [--stack <project>/<stack>]` (`ps`)                   | Local host inventory or the selected stack's observed containers and setup                                                                          |
| `omnitron infra logs [service] [-f] [-n N]`                                  | Tail container logs                                                                                                                                 |
| `omnitron infra psql [database] [--stack project/stack] [-c sql]`            | Open the local stack’s PostgreSQL shell or run SQL once. The command returns the client exit status.                                                |
| `omnitron infra redis-cli [--stack project/stack] [--db index] [command...]` | Open the local stack’s Redis shell or execute a Redis command in the selected database.                                                             |
| `omnitron infra migrate [app] [--dry-run]`                                   | Resolve and run workspace package migration scripts in dependency order. Use `--dry-run` to inspect the packages.                                   |
| `omnitron infra reset --yes`                                                 | Recreate owned standalone containers and reuse their existing volumes.                                                                              |

When several stacks have the same service on one host, select the owning stack explicitly. The CLI matches the project and stack labels and opens the inspected container by ID. A missing scoped application database ends the command with an error. For a remote stack, open a terminal on its node and run the command there.

```bash
omnitron infra psql payments --stack sample/dev -c 'SELECT current_database()'
omnitron infra redis-cli --stack sample/dev --db 3 XLEN orders
```

→ Full reference: [CLI / Infrastructure](/docs/omnitron/cli#infrastructure).

## Failure modes

| Symptom                                    | Cause                                      | Fix                                                                                 |
| ------------------------------------------ | ------------------------------------------ | ----------------------------------------------------------------------------------- |
| App stuck "waiting for infrastructure"     | Critical service unhealthy or timing out   | `omnitron infra status` to check; bump `startupTimeout`                             |
| `omnitron infra up` reports port collision | Two stacks competing for the same port     | Use `portOffsets` in stack settings                                                 |
| Reconciler keeps recreating a container    | Image changed but cache stale              | `docker pull` or `omnitron infra reset`                                             |
| Phantom env vars in app                    | Old service declaration not cleaned        | Phantom janitor handles automatically; rerun after retention window                 |
| Health check passes but app can't connect  | Network firewall / Docker network mismatch | Verify the container is on the right network; check `omnitron infra logs <service>` |

## Anti-patterns

- **Hardcoding `DATABASE_URL` in app code.** The whole point of
  the subsystem is to inject it. Read `process.env.DATABASE_URL`.
- **Skipping `critical: true` on essential services.** Without it,
  the app starts before the database is ready and crashes
  hard. Default is `true`; leave it.
- **Sharing a `redis` instance between dev and staging without DB
  offsets.** Keyspaces collide. Use `portOffsets` and
  `redisDbOffset`.
- **Using container reset to clear a database.** Reset retains named volumes. Use the database’s own reviewed maintenance procedure to change its contents.
- **Embedding secrets in env templates as literals.** Use the
  `${secret:X}` template; the daemon decrypts at injection time.
- **Bypassing the resolver to write Docker compose by hand.**
  Drifts from `omnitron.config.ts` cause reconciler thrashing.

## See also

- [Configuration](/docs/omnitron/configuration) — where `infrastructure: { ... }` lives
- [CLI](/docs/omnitron/cli#infrastructure) — `omnitron infra ...`
- [Services reference](/docs/omnitron/services-reference) — `OmnitronInfra` methods
- [Architecture](/docs/omnitron/architecture#infrastructure-subsystem) — where this fits

## Readiness before provisioning

Use `omnitron stack diagnose <project> <stack>` or **Stacks → Diagnose** before
starting an environment. This is an observation, not a dry-run execution of
application hooks. It resolves the local infrastructure plan using the same
resolver as provisioning, without reserving ports or registering a new owner.

Automatically inferred database and bucket requirements survive explicit port
or credential overrides. Conflicting declarations for the same container refuse
the plan instead of silently choosing the first application's specification.
An invalid service preset fails resolution rather than falling back to a different
legacy plan. See [Console readiness reports](/docs/omnitron/console#stacks) for coverage and
[CLI exit codes](/docs/omnitron/cli#stack-management) for automation.

---

---
id: "omnitron/mcp"
title: "MCP + Knowledge base"
description: "AI agent integration via Model Context Protocol — kb + management tools."
---

# MCP + knowledge base

Omnitron ships a Model Context Protocol server that exposes
**two surfaces** to AI agents (Claude, IDE assistants, CI bots,
code-review agents):

1. **Knowledge base** — semantic + full-text search across the
   codebase, plus structured access to API surfaces, modules,
   patterns, and gotchas.
2. **Management plane** — apps, infra, monitoring, secrets,
   backups, deploys and pipelines through the implemented MCP tool catalog. CLI-only operations remain CLI-only.

Verified against `apps/omnitron/src/mcp/` and
`apps/omnitron/src/commands/kb.ts`.

The current stdio management server connects through the daemon owner’s administrative Unix socket. It is suitable only for a trusted local agent host. Filtering visible tools in that host is not server-side delegation. See [Agents and delegated operations](/docs/omnitron/agents) before embedding an agent in a multi-user Console.

## Starting the server

```bash
omnitron kb mcp
```

That's it. The server runs on stdio (the MCP wire format) and
keeps running until the client closes the stream. Agent host
configuration typically wires it as:

```jsonc
{
  "mcpServers": {
    "omnitron": {
      "command": "omnitron",
      "args":    ["kb", "mcp"]
    }
  }
}
```

The exact config varies by host (Claude Desktop, IDE plugins,
custom agents).

## What the agent sees

Two surfaces, registered conditionally on what's available:

| Tool group                                                                                                        | Requires                         | When unavailable                        |
| ----------------------------------------------------------------------------------------------------------------- | -------------------------------- | --------------------------------------- |
| `kb.*`                                                                                                            | KB indexed (`omnitron kb index`) | "Run `omnitron kb index` first"         |
| `apps.*`                                                                                                          | Daemon running                   | "Run `omnitron up` to start the daemon" |
| `infra.*`                                                                                                         | Daemon running                   | (same)                                  |
| `monitoring.*` / `health.*` / `metrics.*` / `logs.*` / `alerts.*`                                                 | Daemon running                   | (same)                                  |
| `stack.*` / `secret.*` / `backup.*` / `deploy.*` / `cluster.*` / `fleet.*` / `k8s.*` / `project.*` / `pipeline.*` | Daemon running                   | (same)                                  |

When a dependency is unavailable, the corresponding tool names remain visible as **UNAVAILABLE** stubs. Calling one returns an error explaining how to start the daemon or index the KB. Missing data is not presented as a successful empty result.

## KB tools

| Tool                | Purpose                                                                                  |
| ------------------- | ---------------------------------------------------------------------------------------- |
| `kb.query`          | Semantic + full-text hybrid search over code knowledge. Open-ended questions.            |
| `kb.get_api`        | API surface of a class / interface / type — signatures, decorators, members, inheritance |
| `kb.get_module`     | Module info: overview, specs, dependencies, dependents, gotchas                          |
| `kb.repo_map`       | Compressed architecture map (2-5K tokens). **Start here for orientation.**               |
| `kb.get_pattern`    | One named development pattern with code                                                  |
| `kb.list_patterns`  | All available patterns                                                                   |
| `kb.get_gotchas`    | Known pitfalls and critical warnings (essential before modifying unfamiliar code)        |
| `kb.search_symbols` | Search for classes / interfaces / types by name or kind                                  |
| `kb.dependencies`   | Dependency graph for a module — depends-on + dependents                                  |
| `kb.index`          | Trigger an (incremental or full) reindex from the agent                                  |
| `kb.status`         | Index health, entry counts, last-indexed timestamp                                       |

Agents typically lead with `kb.repo_map` (orient), then
`kb.query` (find), then `kb.get_api` (verify signature) before
making changes.

## Management tools — apps

| Tool           | Effect                                                                          |
| -------------- | ------------------------------------------------------------------------------- |
| `apps.list`    | Inventory of managed apps                                                       |
| `apps.start`   | Start an app by name                                                            |
| `apps.stop`    | Stop an app (graceful + optional force)                                         |
| `apps.restart` | Stop + start                                                                    |
| `apps.status`  | Daemon-wide overview                                                            |
| `apps.logs`    | Tail logs (lines + level + grep)                                                |
| `apps.scale`   | Resize the explicit `process` pool in `name` to `count` instances until restart |
| `apps.inspect` | Deep diagnostics for an app                                                     |

`apps.scale` requires all three fields: `name`, `process`, and a positive integer
`count`. It changes only an existing declared pool; classic applications and
singletons refuse. The returned `ProcessInfoDto.processes` contains the selected
pool's observed size. Configuration remains unchanged, restart restores its
initial size, and automatic scaling can subsequently adjust it. The current
owner must support `processPoolResize@1`; a backend-only update does not upgrade
that owner. If an RPC outcome is unknown, inspect the process before retrying.

## Management tools — infra

| Tool                         | Effect                                                       |
| ---------------------------- | ------------------------------------------------------------ |
| `infra.status`               | Full infrastructure state: containers, images, ports, health |
| `infra.containers`           | Managed container inventory                                  |
| `infra.connection`           | Resolved host / port / credentials for a logical service     |
| `infra.start` / `infra.stop` | Start or stop one managed container                          |
| `infra.logs`                 | Tail one container's logs                                    |
| `infra.log_stats`            | Log ingestion counters (stored, dropped, buffered)           |

> There is no `infra.psql` or `infra.redis`. Both were listed here
> and both called RPCs that do not exist; there is no service that
> executes an arbitrary SQL statement or Redis command, and adding
> one reachable by whoever the agent talks to is a decision for an
> operator rather than a gap to fill. Use `omnitron infra psql` on
> the host, where the blast radius is the person typing it.
>
> Provisioning (`infra.up` / `infra.down`) is likewise CLI-only:
> it starts and stops containers for a whole stack, which is not
> something to hand an agent by default.

## Management tools — monitoring

| Tool                  | Effect                                                                                   |
| --------------------- | ---------------------------------------------------------------------------------------- |
| `monitoring.snapshot` | Bounded metrics, alert summary and ingestion-loss evidence; partial sources are explicit |
| `metrics.series`      | Explicit apps/metric names and a bounded historical window                               |
| `alerts.active`       | Read up to 100 firing alerts without acknowledging them                                  |
| `health.check`        | Active probes for one app, or the whole platform when `app` is omitted                   |
| `metrics.get`         | Aggregate metric snapshot                                                                |
| `metrics.app`         | Per-app metrics                                                                          |
| `logs.query`          | Stored logs. Full-text filter is `search`; page with `limit` / `offset`                  |
| `logs.tail`           | The most recent entries, oldest-first — poll for near-real-time                          |
| `logs.stats`          | Exact stored record counts; may scan a large archive, so avoid frequent polling          |

## Management tools — control plane

| Tool                                                                    | Effect                                                                                         |
| ----------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------- |
| `stack.list` / `stack.status` / `stack.start` / `stack.stop`            | Stack lifecycle                                                                                |
| `project.list` / `project.scan` / `project.apps`                        | Project registry                                                                               |
| `secret.list` / `secret.get` / `secret.set`                             | Secret management                                                                              |
| `backup.create` / `backup.list` / `backup.restore` / `backup.schedules` | Database backup                                                                                |
| `deploy.history`                                                        | Retained legacy app-operation records; deploy through `stack.start` with an explicit `release` |
| `fleet.status` / `fleet.summary`                                        | Fleet nodes and counts                                                                         |
| `k8s.pods` / `k8s.scale`                                                | Kubernetes pods + scaling                                                                      |
| `pipeline.list` / `pipeline.run` / `pipeline.status`                    | Administrator Xec automation; run requires `expectedRevision` from the reviewed definition     |

Not offered, and deliberately: `stack.create` (stack creation is
`omnitron stack create`, and getting it wrong strands
infrastructure), `deploy.build`, `cluster.status`,
`fleet.health`, and the two `webapp.*` tools — the console is
managed by a CLI command that runs Docker locally, with no
service behind it.

Use the registered `tools/list` response as the live catalog. It depends on the daemon and KB services available to this process. The typed handlers call existing RPC methods rather than assuming every CLI command has a service endpoint.

See [bounded monitoring evidence](/docs/omnitron/observability#bounded-queries-and-agent-evidence) and [Xec automation](/docs/omnitron/automation) for limits, authority and cancellation behavior.

## KB index lifecycle

```bash
omnitron kb index              # incremental reindex (default)
omnitron kb index --full       # full reindex (ignore manifest cache)
omnitron kb status             # index health, entry counts, last-indexed timestamp
omnitron kb query "<question>" # one-shot query (test the index)
```

The index lives at `~/.omnitron/kb.db` (SurrealKV). Schema:

- **Symbols** — classes, interfaces, types, functions, enums
- **Modules** — packages with overview, dependencies, gotchas
- **Patterns** — canonical development recipes
- **Gotchas** — known pitfalls per module
- **Docs** — long-form markdown indexed alongside symbols

Hybrid search combines:

- Full-text BM25 on names + docs
- Semantic embeddings (when available) for "open-ended question"
  flavour

Both indexes update during `omnitron kb index`.

## Indexing strategy for a monorepo

Reindexing is incremental by default (manifest-cached), so a
plain `omnitron kb index` after a batch of edits is cheap — wire
it into a pre-commit hook or run it manually when you've changed
public surfaces. For CI / production reads:

```bash
omnitron kb index --full       # fresh, in case manifests drifted
omnitron kb status             # verify entry counts look right
```

The full reindex is idempotent and bounded — typically minutes
even on large codebases.

`OMNITRON_ROOT` env var picks the indexed root (default `cwd`).
Useful when running the MCP server from outside the project
directory.

## Tool authoring (extend MCP)

Tools live under `apps/omnitron/src/mcp/tool-groups/`. Each
group exports a factory:

```typescript
// apps/omnitron/src/mcp/tool-groups/my-tools.ts
import type { IMcpToolDef } from '../types.js';

export function createMyTools(daemonClient: any): IMcpToolDef[] {
  return [{
    name: 'mything.do',
    description: 'Do the thing. Use when X.',
    inputSchema: {
      type: 'object',
      properties: {
        target: { type: 'string', description: 'What to target' },
      },
      required: ['target'],
    },
    // Return a raw value — the bridge wraps it into the
    // MCP { content: [{ type: 'text', text }] } envelope for you.
    handler: async ({ target }) => {
      return daemonClient.someService.someMethod({ target });
    },
  }];
}
```

Register in the `kb mcp` command (`apps/omnitron/src/commands/kb.ts`):

```typescript
bridge.registerTools(createMyTools(daemonClient));
```

The tool definition interface is `IMcpToolDef` (in
`apps/omnitron/src/mcp/types.ts`); the existing group factories
(`createKbTools`, `createAppsTools`, …) all take the relevant
client/service as a loosely-typed `any`. Handlers return a raw
object or string — `McpBridge` `JSON.stringify`s it and wraps it
in the `{ content: [...] }` envelope; don't build that envelope
yourself or you'll double-encode.

The `description` field is **what the agent reads to decide
whether to call the tool**. Make it specific.

## Best practices for agent-callable tools

- **Atomic operations.** One tool, one outcome — don't bundle
  "deploy + scale + restart" into one tool.
- **Idempotent where possible.** Agents retry on transient
  failures; non-idempotent tools cause double-spend bugs.
- **Strict input schemas.** JSON schema with `required` and
  `enum` constraints — narrows the agent's failure modes.
- **Descriptive errors.** When a tool fails, return a clear
  message so the agent can recover (e.g., "App not found.
  Available: api, worker, scheduler.").
- **Read-then-write pattern.** Expose a `list` / `status` for
  every `do` — agents check before acting.
- **Cap dangerous side-effects.** Tools that destroy data
  (`infra.down --volumes`, `backup.restore`) should require
  explicit confirmation flags.

## Auth model

The MCP server **inherits the local Unix-socket trust**. The
`kb mcp` command takes no connection flags — it always builds a
local daemon client (`createDaemonClient()`) that talks to
`~/.omnitron/daemon.sock`. If the agent process can reach that
socket, it runs as admin (same model as the CLI). A different
process running as a different OS user → different daemon →
different scope.

> **There is no remote/TCP MCP transport today.** `kb mcp` has no
> `--daemon-url` or `--token` option; the server cannot point at a
> remote daemon. To drive a remote host, run the MCP server **on
> that host** (over an SSH session or inside its container) so it
> reaches the local socket there, and scope the agent's OS-level
> access accordingly.

Match agent power to its expected scope. Because local-socket
access is effectively admin, the strongest control is *where*
you run the MCP server and *which* host's daemon it can reach.

## Common agent workflows

### "What does X do?"

1. `kb.repo_map` → orient
2. `kb.query "X"` → find relevant entries
3. `kb.get_api "X"` or `kb.get_module "X"` → details
4. Agent answers from results

### "Is the platform healthy?"

1. `apps.status` → daemon overview
2. `health.check` → composite health
3. `metrics.get` → CPU/memory aggregate
4. `apps.logs` → recent error filter
5. Agent summarises

### "Fix this failing deploy"

1. `deploy.status <runId>` → what failed
2. `apps.logs <appName> --level error --grep deploy` → context
3. `kb.get_gotchas <appName>` → known issues
4. `apps.inspect <appName>` → live state
5. Agent proposes fix; operator confirms

### "Scaffold a new app"

1. `kb.get_pattern "new-app-bootstrap"` → canonical scaffold
2. `kb.get_module "<related-app>"` → similar example
3. Agent generates files; operator reviews

## Cost and latency

- **Tool calls** to the daemon are essentially free (Unix socket
  RPC).
- **KB queries** with semantic search load embedding models —
  cold start adds 1-2s; subsequent queries are sub-second.
- **The MCP server itself** is lightweight (\~50-100 MB resident).

Run it as a long-lived companion to your agent session;
restarting on every prompt adds noticeable latency.

## Anti-patterns

- **Running the MCP server next to a production daemon socket**
  without thinking. Local-socket access is admin — keep the
  server on dev/staging hosts, or gate destructive tools behind
  operator confirmation, since there is no per-token role scoping
  for the MCP surface today.
- **Skipping `kb index`** in CI before MCP-based code review.
  Stale KB makes agents confidently wrong.
- **Tools that perform multiple actions atomically.** The agent
  can't retry partial successes; the daemon ends up with
  half-applied state.
- **Tools that mutate without listing first.** Agents work better
  when they can see current state before acting.
- **Generic "execute SQL" / "run command" tools.** Wide blast
  radius, hard to audit, hard for the agent to use correctly.
  Narrow domain-specific tools win.

## See also

- [CLI / Knowledge base](/docs/omnitron/cli#knowledge-base-mcp)
- [Configuration](/docs/omnitron/configuration) — daemon config
- [Daemon / Auth flow](/docs/omnitron/daemon#auth-flow) — local Unix-socket trust model
- [Best practices](/docs/omnitron/best-practices)

---

---
id: "omnitron/migrations"
title: "Database migrations"
---

Use `@omnitron-dev/omnitron/migration` for migration definitions, execution and
inspection. `omnitron migrate` uses that same API. Applications do not need a
separate Kysera dependency or a second migration CLI.

## Start with one configuration

Install Omnitron and the driver your application uses. This PostgreSQL example
borrows a pool through a narrow adapter; it does not initialize Titan DI or start
the daemon.

```typescript title="omnitron.migrations.ts"
import { Pool } from 'pg';
import {
  defineMigrationConfig, defineSqlMigration, createPostgresMigrationDatabase,
} from '@omnitron-dev/omnitron/migration';
import { describeError } from '@omnitron-dev/omnitron/diagnostics';

export default defineMigrationConfig({
  name: 'application',
  migrations: [
    defineSqlMigration('001_widgets',
      'CREATE TABLE widgets (id integer PRIMARY KEY, label text NOT NULL)',
      'DROP TABLE widgets'),
  ],
  async connect() {
    const pool = new Pool({ connectionString: process.env.DATABASE_URL, max: 1,
      connectionTimeoutMillis: 5_000 });
    pool.on('error', error => console.error(describeError(error)));
    const db = createPostgresMigrationDatabase(pool);
    return { db, close: async () => { try { await db.destroy(); } finally { await pool.end(); } } };
  },
});
```

```bash
omnitron migrate status
omnitron migrate up --dry-run
omnitron migrate up
omnitron --json migrate verify
omnitron migrate down --steps 1
```

`--config path/to/config.ts` selects another configuration. The default command
is `status`; inspection never applies migrations. `--daemon` explicitly selects
the daemon's registry and database. It cannot be combined with `--config`.
`--to NAME` applies an inclusive prefix; `--steps N` is valid only for `down`.
Invalid or conflicting arguments fail before opening a database.

`verify` exits with 2 for changed definitions, missing files **or unknown
checksums**. A successful operation exits with 0; other failures use a nonzero
exit code and include the cause and recovery guidance. JSON output uses the
normal Omnitron `{ ok, data }` / `{ ok: false, error, diagnostic }` contract.

Direct CLI migration writes require standalone lifecycle authority. On an enrolled controller, use the [backend schema plan](/docs/omnitron/cli#backend-replacement) for daemon schema changes. For a standalone ecosystem that provides package migration scripts, `omnitron infra migrate --dry-run` lists the resolved packages in dependency order.

## Guarantees and limits

| Mechanism          | Behavior                                                                                                      |
| ------------------ | ------------------------------------------------------------------------------------------------------------- |
| Execution          | Kysera owns the single engine, including CLI and Titan compatibility APIs                                     |
| PostgreSQL locking | One pinned connection holds the advisory lock and executes migrations; a one-connection pool works            |
| Default atomicity  | Each migration body, history row and checksum commit together                                                 |
| Batch atomicity    | `transactionMode: 'batch'` commits the whole operation together; requires transactions and stopping on errors |
| Inspection         | `status`, `inspect` and dry runs do not create tables or adopt history                                        |
| Integrity          | SQL definitions have content digests; arbitrary callbacks must provide a stable `checksum`                    |
| Rollback           | Missing definitions or missing `down` handlers reject the entire plan before any rollback                     |
| Retry              | No blind retry after an ambiguous connection failure; inspect durable history first                           |
| Cleanup            | Commands close their owned connections and preserve both operation and cleanup failures                       |

New daemon PostgreSQL and local SQLite migration executions record checksums through
that same engine. The older procedural daemon definitions carry declared SHA-256
source digests, checked against their immutable source files in CI; transpilation
and installation paths do not change them. New SQL definitions use
`defineSqlMigration` to derive their digest directly from the SQL.

A fresh database must pass `omnitron --json migrate up --daemon` followed by
`omnitron --json migrate verify --daemon`. Older journal rows without checksums
remain `unverified`: neither inspection nor another `up` invents historical
integrity or replaces recorded execution times. Existing migration bodies and
history are not rewritten by adding a checksum declaration. Updating a daemon
with these declarations requires a guardian that supports the
[checked metadata transition](/docs/omnitron/backend-handoff#checksum-metadata-on-existing-migrations).
The candidate CLI reports an older guardian before changing lifecycle state;
this is separate from applying a schema migration or adopting historical hashes.

PostgreSQL and SQLite support transactional schema changes. MySQL DDL can commit
implicitly, so batch mode refuses MySQL. SQLite has no cross-process advisory
lock: concurrent writers can fail with a busy error. Serialize deployments and
handle that failure explicitly. A process that owns a driver pool must observe
its asynchronous error events as well as rejected query promises.

Batch mode takes a root database handle, not a caller-owned transaction. Plugin
completion hooks run after the batch commits. `MigrationHookError.applied` means
the database change committed even though its observer failed: do not rerun the
migration body to repair an observer.

## Adopt an existing application

Preserve the deployed journal, version names, checksum encoding and lock
identity. Do not create a second journal or infer execution from tables existing.

```typescript
import { createMigrationRunner, createMigrationHistory } from '@omnitron-dev/omnitron/migration';

const runner = createMigrationRunner(db, migrations, {
  advisoryLockKey: 'application.schema',
  history: createMigrationHistory({
    tableName: 'schema_migrations', keyColumn: 'version', keyType: 'integer',
    executedAtColumn: 'applied_at', checksumTable: 'schema_migration_checksums',
  }),
});
```

Text lock identities use PostgreSQL `hashtext`, compatible with applications
that previously used `pg_advisory_xact_lock(hashtext(...))`. Keep the identity
unchanged during a rolling deployment. New applications use an inline checksum
and a monotonic `execution_order`; existing journals must opt into their actual
layout. The adapter refuses a requested ordering column that does not exist.
PostgreSQL history lookup follows the connection’s `search_path`; an explicit
`schema` pins the journal to that schema. A configured checksum sidecar is created
only on writes; inspection never creates it. Custom `MigrationHistory` methods support existing storage constraints without
reimplementing execution, transactions or locking.

A checksum-less historical row is **unverified**, not proof of unchanged source.
After reviewing the applied artifacts, use the runner's explicit
`adoptChecksums()` or `baseline(names)` APIs. Both take the migration lock;
neither overwrites an existing digest. A baseline records execution without
running the migration body, so it requires evidence that the schema already
matches.

`createHardenedRunner` and `runMigrationCli` preserve the previous Titan journal,
lock and function-source checksum format for existing deployments. Its
trust-on-first-use and rollback-drift behavior remain compatibility policies;
new applications should use `createMigrationRunner`. Legacy error constructors
are exported as `LegacyMigrationLockError` and `LegacyMigrationChecksumError`.

## Stack-owned migrations

A database dependency does not authorize schema changes. Declare one migration
owner by its application alias; the database name and package directory can differ:

```typescript
apps: [{ name: 'api-service', cwd: 'apps/api', bootstrap: 'src/bootstrap.ts' }],
infrastructure: {
  services: {
    postgres: {
      preset: 'postgres',
      config: {
        databases: {
          api: {
            migrate: [
              {
                app: 'api-service',
                config: 'src/database/portal.migrations.ts',
                releaseConfig: 'dist/database/portal.migrations.js',
              },
              {
                app: 'api-service',
                config: 'src/database/market.migrations.ts',
                releaseConfig: 'dist/database/market.migrations.js',
              },
            ],
          },
        },
      },
    },
  },
}
```

A single binding can be an object instead of an array. The `MigrationBinding` type
is exported from `@omnitron-dev/omnitron`. Each module exports the existing
`defineMigrationConfig` shown above. Its history adapter, checksum encoding and
lock remain independent; the array specifies execution order, not a merged journal.

`config` is relative to the application's declared `cwd`. The ordinary package
build must emit `releaseConfig` inside its compiled payload. Selected local releases
and remote deployments use that exact member of the verified artifact. Missing,
changed or escaping members refuse execution; an imported alias keeps its producer's
package. There is no fallback to a source checkout or another migrator.

The same normalized plan drives local startup, remote deployment and diagnosis.
Before provisioning, the local stack validates ownership, paths and resolved
connection destinations. Before the first migration connection, every config is
imported and its definitions are resolved in an isolated child. Definition factories
must only load definitions; `connect()` owns connection creation. Execution then uses
`executeMigrationOperation` from the application's own SDK. SQL and connection
callbacks can still fail at runtime; preflight is not proof that a migration will
succeed.

A config receives the complete resolved `DATABASE_URL`, including TLS and session
options. The database path must match the declared database. Query options cannot
replace the URL's host, port, database or credentials. Conflicting owners of the same
resolved destination refuse the plan. Managed vault resolution remains part of the
stack's existing environment contract; secrets never travel as command arguments.

Migration preparation validates `settings.appEnv` against every application in the
stack before selecting migration owners. Non-migrating applications may have their
own environment overrides; an unknown application name still refuses the plan.
Each selected migrator receives only its own resolved application environment,
including managed secrets, rather than the environment of other stack members.

All requested programs are prepared before executing any. A failed migration blocks
candidate application activation, and an interrupted migration is never retried
automatically. Previously committed registries remain committed: there is no global
transaction across registries or databases. Use compatible schema changes, backups,
and `status`/`verify` before deciding how to resume.

Explicit migration owners on remote stacks must occupy one unambiguous node, processed
before consumers. Several owners spread over different nodes are refused before
provisioning: the current deployment path has no global migration barrier. A replicated
application is not silently reduced to an arbitrary migration owner.

### Existing scripts

`migrate: true` preserves the conventional `src/database/migrate.ts` →
`dist/database/migrate.js` program, with the database name as its application alias.
For this compatibility spelling, an omitted/root `cwd` retains `apps/<database>`;
another declared `cwd` takes precedence. The script receives `DATABASE_URL` and the
original producer's app-prefixed connection fields. This legacy contract cannot
preserve URL query options and refuses them rather than dropping TLS or routing
settings. `migrationDir`, which never controlled execution, is refused with a remedy
to declare an explicit binding.

Absent or false opt-in means no migration. Remote nodes no longer infer permission
from `DATABASE_URL` or the presence of a conventional script. Legacy boolean bindings
describe each node's local database; use an explicit owner for a shared database.
Non-owning node allocations never provision or migrate their controller's services.

### Preview and diagnosis

```bash
omnitron infra migrate --stack dev --dry-run
omnitron infra migrate --stack dev
```

The standalone command uses the same declared targets and ordered configs. It reads
the application's and stack's explicit environment, or the invoking environment;
it does not invent connection defaults or retrieve a managed vault. An enrolled or
remote stack requires the elected controller. If no database migration policy exists,
the older explicitly invoked `infra migrate` path still runs workspace package
`migrate` scripts in dependency order. An explicit false policy never falls through
to those scripts.

Console and CLI stack diagnosis show owners and program availability without executing
configs or opening a database connection. Database access, schema compatibility and
migration completion remain separate evidence. Runtime workers should inspect their
required schema through the read-only runner API instead of applying DDL at startup.

## Programmatic use and responsibility boundaries

`executeMigrationOperation(config, options)` is the CLI's reusable operation
layer. It owns connection cleanup and returns integrity information. Direct
runners expose `up`, `upTo`, `down`, `reset`, `inspect`, `baseline` and
`adoptChecksums`. `createMigrationRunnerWithPlugins` adds lifecycle hooks without
changing the execution engine. Use `migrationSqlClient(db)` for parameterized SQL
on the runner's current connection. `loadMigrationFiles(directory)` shares the
Node file loader with the compatibility CLI and preserves explicit exported
checksums. `migrationLockKey(databaseName)` reproduces that CLI’s historical lock
identity for startup hooks; keep the same identity across all entry points.

Kysera remains independently installable and owns persistence mechanisms.
Titan adds DI, connection lifecycle and ambient transaction context. Omnitron
adds operational defaults, commands and diagnostics. Applications own schema
changes and domain-specific data transformations. The Console consumes RPC
results; it has no migration engine and never owns database credentials.

For recovery and notification delivery, see [Operational diagnostics](/docs/omnitron/diagnostics).

---

---
id: "omnitron/observability"
title: "Observability"
description: "How logs, metrics, traces, and health flow through Omnitron."
---

# Observability

Supervised applications can emit logs, metrics, traces and health signals through Titan and Omnitron. Available signals depend on the application modules, instrumentation and configured storage. A running daemon does not imply complete instrumentation. The baseline does not require a separate Prometheus, Loki or Tempo deployment.

This page describes those pipelines end-to-end so you know which
knob to turn and which storage to inspect when something goes
wrong.

Verified against `apps/omnitron/src/{observability,monitoring,services}/`.

## The four pipelines

| Signal      | Owner                                                                        | Storage                                                                   | Exposure                                                                                      |
| ----------- | ---------------------------------------------------------------------------- | ------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------- |
| **Logs**    | Titan PM capture + `LogManager` + `LogCollectorService`                      | Private SQLite delivery journals; per-app files; PostgreSQL/SQLite `logs` | RPC: `OmnitronLogs.queryLogs` / `streamLogs`                                                  |
| **Metrics** | `MetricsBridge`, Titan storage and `OmnitronSync`                            | PostgreSQL on controllers, SQLite history and outbox on workers           | CLI and Console query `OmnitronMetrics`                                                       |
| **Traces**  | `OmnitronTraces` (`TraceCollectorService`)                                   | Postgres (`OmnitronDatabase`), write-buffered                             | RPC: `ingestSpan`, `ingestBatch`, `getTrace`, `queryTraces`, `getServiceMap`                  |
| **Health**  | `OmnitronHealth` (app/infra probes) + `OmnitronNodes` health monitor (fleet) | rolling window in state-store / node history                              | RPC: `OmnitronHealth.checkApp` / `checkAll`; `OmnitronNodes.getCheckHistory` / `getUptimeBar` |

## Logs

### Capture, persistence and replay

```mermaid
sequenceDiagram
  participant App as Fork or worker thread
  participant PM as Titan PM capture
  participant Journal as Private SQLite journal
  participant Backend as Omnitron backend
  participant DB as PostgreSQL or node SQLite
  participant Files as Application files
  App->>PM: stdout / stderr, including startup failures
  PM->>Journal: ID + source + capture time + message bytes
  Note over Journal: WAL + synchronous=FULL<br/>unfinished multiline records persist too
  Journal->>Files: Append with a durable file offset<br/>fsync
  Files-->>Journal: File projection confirmed
  Backend->>Journal: Read a bounded page
  Backend->>DB: Commit logs and node replication outbox together
  DB-->>Backend: Transaction committed
  Backend->>Journal: Confirm exact IDs and payload digests
  Note over Journal: Release only after both destinations confirm
```

Titan PM installs output capture before readiness, including for workers replaced by a pool. One UTF-8 reader handles each pipe from spawn through EOF. A failed initialization does not require a `WorkerHandle` to exist before output reaches storage. Stdout and stderr from different process instances are assembled separately. The same canonical record feeds files, the database and replication.

The local delivery files are `data/application-logs.sqlite` and `data/backend-logs.sqlite` under the configured Omnitron storage directory. They use SQLite WAL with `synchronous=FULL` and private permissions. Pending messages and finalized records survive process replacement. No oldest-first eviction is applied to unacknowledged logs. Recovery checks the writer's host boot and process start identity, not just its PID. It periodically revisits unfinished records, so a previous backend that dies after handoff does not strand them until another restart. If the original writer's lifetime cannot be verified, those bytes remain pending with an explicit diagnostic; they are not finalized while a writer may still be assembling them. On macOS, all process readers use the boot-session UUID and a birth timestamp explicitly captured in UTC. Legacy timestamps did not retain their timezone: a changed rendering or a mixed legacy/UTC identity does not prove that the writer exited. Its unfinished bytes remain pending until native death or a verified change of host boot is observed. If capture cannot write its journal, the pipe is paused and the same frame is retried. Monitor storage availability: backpressure can eventually block an application that continues writing to a full pipe.

A database response is acknowledged only after the enclosing transaction commits. PostgreSQL confirmation requires `synchronous_commit=on`, `fsync=on`, `full_page_writes=on` and a logged `logs` table. Persistent SQLite requires WAL with `FULL`/`EXTRA`, or a disk rollback journal with `EXTRA`. A slave's local rows and replication outbox share one transaction. Retries keep the original IDs; conflicting content under an existing ID is retained and reported, never overwritten.

Database and outbox failures use Titan's bounded diagnostic formatter, including the underlying connection failures of an `AggregateError` with an empty message. Common credential forms are redacted. A failed confirmation rejects the current drain and retains the same records for a later retry; it does not immediately loop during owner acknowledgment or shutdown. CLI ingestion status and Console use the same reading and show ingestion, local-journal and owner-journal failures separately. An empty error reported by an older backend is still a failure, with an explicit notice that its details are unavailable.

File projection has its own acknowledgment. Before appending, the journal records the file identity and original offset. After a crash, matching complete bytes are accepted once; a proven partial append is replayed from that offset. Unrelated or replaced bytes cause a diagnostic and remain intact. Rotation and compression are serialized with appends. File retention may remove acknowledged historical archives according to `maxSize`/`maxFiles`; it does not evict pending delivery records.

### Time and multiline messages

- A structured message keeps its supplied timestamp. PostgreSQL stores microseconds; the original timestamp also survives in capture metadata. Ordering never manufactures fractional seconds. Capture sequence and stable IDs are separate from event time.
- Unstructured messages carry the parent's time of first read from the pipe. That is capture time, not proof of when the child generated the text. The record detail identifies `reported`, `captured`, `inferred` or legacy `observed` time.
- JSON lines define explicit boundaries. Indented continuations and contiguous unstructured stderr are assembled per process/stream until the next structured boundary or 100 ms of idle time. Arbitrary plain-text output has no universal message-boundary protocol; use newline-delimited structured JSON for exact application-defined boundaries.
- A pending multiline record is journaled before it is considered accepted. An interrupted owner leaves those bytes recoverable. Physical lines above 256 KiB use linked fragments; oversized multiline records retain a continuation reference. Fragment boundaries are explicit metadata, not discarded text.
- Time-of-day-only pretty headers require a date inference. Historical undated output remains uncertain; this implementation does not retroactively invent timestamps for old files.

### Files and startup failures

Standalone apps write `logs/{app}/app.log`; project apps write `projects/{project}/{stack}/logs/{app}/app.log`. Error and fatal records also go to `error.log`. JSON files include `_omnitron` capture metadata. The daemon writes `logs/omnitron.log`. Defaults remain `level: info`, `maxSize: 50mb`, `maxFiles: 10`, `compress: true`; per-app overrides use `observability.logging`.

Native entry guards cover imports and initialization before Titan is available. They synchronously report one structured failure with UTC time, ID, component and stack to stderr, and preserve a private source file under `$OMNITRON_HOME/data/startup-failures`. The next working backend imports those files through its durable journal with an atomic import checkpoint. Repeated startup does not duplicate the row. Keep these source files as evidence; a storage failure can still leave stderr as the final available destination.

The in-memory tail is a bounded convenience view. Database history, durable pending delivery and rotated files have distinct retention and availability; a full live window is not evidence that no other records exist.

File projection during `LogManager.dispose()` has a default five-second budget,
configurable through `LogManagerConfig.shutdownProjectionTimeoutMs`. An in-flight
record finishes before the journal closes; the next record is deferred when the
budget expires. This bounds backlog draining, not a stuck operating-system I/O
call. Deferred records and append intents remain in SQLite and replay after the
next start with the same IDs and offsets. Concurrent disposal callers await the
same completion. This prevents a large derived-file backlog from indefinitely
holding the owner shutdown receipt; it does not discard or acknowledge unwritten
records. An explicit `flush()` outside disposal still waits for the file projection.

### Owner and supervisor journals

The resident execution owner owns worker pools even when the backend is replaced.
Its structured stdout includes worker replacement reasons (`max_lifetime`,
`request_count`, `memory_limit`, `memory_warning`) and process exits. Retaining
only stderr loses these events, including errors emitted by the JSON logger.

Generated launchd jobs retain both stdout and stderr. In resident mode the
owner writes to `$OMNITRON_HOME/execution/owner-service.log` and the backend
guardian to `$OMNITRON_HOME/execution/supervisor.log` (under the configured
execution directory when overridden). Programmatic unit rendering may set a
separate `stdoutLog`; otherwise both streams share `stderrLog`. Discarded,
relative and control-character paths are refused. Generated systemd units
explicitly route both streams to the journal:

```bash
journalctl -u omnitron-owner.service -u omnitron-daemon.service --since '10 minutes ago'
# For user services, add --user.
```

These native supervisor journals are separate from the application log archive
and its retention policy. Apply host journal retention or file rotation to them.
Editing an installed unit does not reconnect a running process's file descriptors.
Review the old and new definitions, preserve the original runtime and private
configuration, and schedule a controlled owner restart; it restarts the owner's
applications. A backend-only handoff leaves owner logging unchanged. Records
already discarded by an older unit cannot be reconstructed.

### `OmnitronLogs` RPC

| Method                                                                                        | Effect                                                                                                                                |
| --------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------- |
| `queryLogs({app?, nodeId?, level?, search?, labels?, traceId?, from?, to?, limit?, offset?})` | Filter & return entries (note: full-text filter is `search`, not `grep`; page with `limit`/`offset`)                                  |
| `streamLogs({app?, nodeId?, level?, search?, labels?, tail?, since?})`                        | Return the last `tail` entries (default 100) in chronological order — poll for near-real-time tailing                                 |
| `getLogStats()`                                                                               | Exact stored counts and archive range; an analytic query, not a polling health check                                                  |
| `getIngestionStats()`                                                                         | Capture capability, durable pending records/bytes, owner errors, last confirmed DB commit and last write failure; inexpensive to poll |

`streamLogs` is a poll-friendly "recent entries" call rather than
a push subscription. `since` includes its timestamp boundary; deduplicate by row ID. The response is bounded by `tail`, so use `queryLogs` to inspect complete historical windows or bursts larger than the live tail.

### CLI

```bash
omnitron logs --status              # same persistence verdict as Console Logs
omnitron --json logs --status       # structured queue, capability and commit evidence
omnitron logs                       # daemon log
omnitron logs api                   # one app
omnitron logs api -f -l warn -g err # follow + level filter + regex
omnitron logs api --file            # read directly from files (bypass daemon)
omnitron --json logs api            # JSON envelope with source and entries
omnitron logs sample/production/mail --node worker-1 -n 100
```

### Interpreting diagnostics

Console **Logs** and `omnitron logs --status` use the same reading of `OmnitronLogs.getIngestionStats()`. The status describes the connected backend; selecting a remote node in the log table does not turn a local ingestion counter into remote evidence. Inspect node replication diagnostics for the remote delivery path.

`lastCommitAt` is a successful database commit, not the newest event timestamp. `journal.pending` includes records awaiting delivery, and `journal.assembling` identifies messages still being joined. `ownerPending` is the observed owner backlog. `lastError`, `ownerError` and journal errors explain failure; a zero in an unavailable or older backend is never substituted for evidence. Counters describe this collector's lifetime, not proof that historical data was never lost.

`capture: legacy-owner` explicitly means the running owner lacks the durable protocol. A backend-only update cannot repair its old truncation, volatile event backlog or missing pipe timestamps. Updating the owner requires an approved application restart. Do not use a legacy backend with a new owner as a silent rollback: restore a matching owner/backend pair, preserving all journals.

Before replacing a legacy owner, save the exact owner/backend release pair,
supervision configuration, application inventory, restart configuration and online
SQLite backups. Prepare the candidate and rollback with the same dependency closure.
Before stopping applications, read their bootstrap declarations in isolated processes
using the candidate, the host’s actual Node executable and preload arguments, and
the daemon’s working directory. A package import check alone does not exercise
child-process loading. Framework preloads must resolve from the installed runtime
even when the daemon changes its working directory; preserve the directory seen
by application definitions.
Drain applications gracefully, stop both supervisors, replace the owner and backend
as one reviewed pair, then restore the saved inventory and watch state. An owner
restart cannot transfer existing pipes and worker threads; it needs a controlled
application maintenance window even when ordinary backend handoffs are seamless.

Accept the transition only after CLI and Console report durable capture, the database
has a confirmed commit, owner/journal errors are absent and pending records are
acknowledged. Verify application health, RPC/replication progress and immutable
experiment identities as separate checks. A running process alone is insufficient.
Rollback restores the saved release pair and supervision configuration, retaining
all current data and delivery journals; do not restore an old database snapshot over
newer writes. Historical records without original capture evidence retain that
limitation after the upgrade; their original timestamps cannot be reconstructed by
relabeling them.

The compact log rows show event time; expanded metadata includes the capture ID, process, stream, sequence, full timestamp and time basis. CLI JSON includes the same metadata. `LogEntryDto.timestamp` remains epoch milliseconds for compatibility; use metadata to inspect the original fractional precision.

## Metrics

### Flow

The daemon collects process observations through Titan. Worker samples enter local history and the replication outbox in one SQLite transaction. The controller pulls pending entries and writes the resulting history to PostgreSQL. CLI and Console read the same storage diagnostics. See the [metrics data flow](/docs/omnitron/architecture#data-flow--metrics-aggregation).

### Metric persistence and confirmation

The daemon's metric history uses Titan's SQL adapters: PostgreSQL on masters and
SQLite on updated slave daemons. Console history queries, CLI diagnostics and the
`metrics-storage` health indicator refer to that same store. No second write-only
metric table is maintained.

- A local collection call queues a measurement. `flush()` confirms that the
  selected storage completed the queued writes; `recordDurable()` returns a
  persistent commit receipt or rejects.
- A slave commits each metric and its replication outbox entry atomically. Its
  pending queue survives daemon replacement and database reconnection.
- A master commits the received metric and `sync_ingested` outcome atomically.
  A missing response leaves the same source entry pending; replay returns the
  existing outcome without writing the metric twice.
- Original sample timestamps and source-node labels survive transport. The live
  Prometheus registry is updated after commit and is not the history database.

```bash
omnitron metrics --json
omnitron health --json
omnitron node check <node>
```

The first command includes `storage.status` and the same summary/detail shown on
the Console Metrics page. It distinguishes persistent, volatile and unknown
storage, queued samples/bytes, completed writes, errors, retention failures and
known losses. Collection/write counters describe the current MetricsService instance, including
confirmed retries. Imported history and incoming replication commits have their own
evidence; replication
receipt counters retain their tracking epoch on disk. A successful RPC call or
an empty queue is not substituted for database confirmation.

Migration `015_persistent_metrics` creates Titan's `metrics_samples` table and
indexes through Omnitron's existing Kysera-backed migration lifecycle. For a
resident backend, prepare/apply the reviewed additive schema transition before
handoff; retain its exact rollback runtime and receipt. Do not run a down migration
to roll back the backend. Slave updates use the local SQLite schema initializer.
Applications do not need a direct Kysera dependency.

Old unverified receipts are retained as historical evidence. An upgrade cannot
certify them. `omnitron --json node reconcile <node>` compares retained metric
payloads with the database; `--restore` explicitly repairs missing samples and
records new commit proofs. Original claims remain unchanged. Console and CLI show
later verification/restoration separately; see [reconciliation limits and recovery](/docs/omnitron/cluster#reconcile-historical-metric-receipts).
A master update improves confirmation for new deliveries immediately; the source
polling producer changes to the atomic outbox only after its backend is updated.

Ordinary collection buffers are bounded and volatile before commit. The default
is 50,000 samples / 16 MiB; overflow is reported, never acknowledged as persisted.
Chart history uses `monitoring.metrics.historyMaxAge` (default `7d`), passed to
Titan's retention policy. For example, `historyMaxAge: '24h'` keeps one day. The
legacy numeric `monitoring.metrics.retention` field does not control SQL history.
Pruning chart history and pruning delivered
replication entries are separate operations. See the [Titan metrics contract](/docs/titan/modules/metrics)
for durable calls, database settings, query budgets and optional storage adapters.

### Producer side — `MetricsBridge`

`apps/omnitron/src/observability/metrics-bridge.ts` is the
in-daemon bridge that:

- Hosts the `MetricsService` shared by daemon services.
- Pushes daemon-internal metrics (RPC call counts, app supervision
  events, infra reconciler outcomes) to storage on the same
  cadence as app-side metrics.
- Generates daemon-level `omnitron_*` series alongside per-app
  series.

### Application metrics and worker replication

Import `TitanMetricsModule` for application-specific instruments. An in-memory application registry retains samples in that process. Configure a persistent storage adapter when the application needs its own retained history. The [Titan metrics guide](/docs/titan/modules/metrics) covers recording, querying and flushing samples.

The daemon samples the processes it supervises. A worker writes samples and replication entries in one SQLite transaction. The controller pulls entries over the authenticated mesh, commits delivery evidence and acknowledges the source. Use `omnitron --json metrics` and node diagnostics to inspect persistence and the replication backlog. Older sources may have different evidence coverage.

### Read side — `OmnitronMetrics` (from `titan-metrics`)

The daemon registers its role-protected `OmnitronMetricsRpcService` facade
over Titan's metrics service. The webapp and CLI use the same contracts.

See [titan-metrics docs](/docs/titan/modules/metrics) for the
full API — `getSnapshot`, `querySeries`, `getPrometheusText`,
`getStorageStatus` and restricted maintenance operations.

### Prometheus scrape endpoint — `MetricsServer`

`apps/omnitron/src/observability/metrics-server.ts` runs a tiny
standalone HTTP server *alongside* the RPC surface, so an
existing Prometheus can scrape the daemon directly:

- `GET /metrics` → text/plain Prometheus exposition
  (`metrics.getPrometheusText()`, after refreshing per-app gauges).
- `GET /healthz` (and `/health`) → `ok` for liveness probes.
- Anything else → 404; non-GET → 405.

Port defaults to `httpPort + 3` (≈ `9803`); it binds to
**`127.0.0.1` by default** so the aggregate counters aren't
exposed network-wide. To scrape remotely, bind `0.0.0.0` **and**
set a bearer token (`authToken`) — `/metrics` then requires
`Authorization: Bearer <token>` (constant-time compared);
`/healthz` stays public. Binding non-loopback without a token
logs a loud warning.

So the "no separate Prometheus to deploy" baseline still holds —
but if you *do* run Prometheus, this is the endpoint to point it
at, rather than the RPC read path.

### Aggregated metrics — `AggregatedMetricsDto`

`OmnitronDaemon.getMetrics({name?})` returns:

```typescript
interface AggregatedMetricsDto {
  timestamp: number;
  apps: Record<string, {
    cpu:        number;
    memory:     number;
    requests?:  number;
    errors?:    number;
    latency?:   { p50, p95, p99, mean };
  }>;
  totals: {
    cpu:    number;
    memory: number;
  };
}
```

This is the canonical "what's everyone using" snapshot — used by
the dashboard's overview tiles.

### CLI

```bash
omnitron metrics                    # all apps
omnitron metrics api                # one app
omnitron --json metrics             # NDJSON
```

For time-series queries (`OmnitronMetrics.querySeries`), use the
webapp — `OmnitronMetrics` is a daemon-hosted service, so it
isn't reachable through `omnitron exec` (which targets services
running inside a managed app).

## Traces

### Flow

```mermaid
sequenceDiagram
  participant App as App process
  participant Producer as Authorized trace producer
  participant TC as OmnitronTraces service
  participant Store as Postgres (OmnitronDatabase)
  participant UI as Webapp / CLI

  App->>Producer: Finished application span
  Producer->>TC: ingestSpan or ingestBatch
  TC->>TC: buffer span (flush at 500 or every 5s)
  TC->>Store: flush batch (INSERT)
  UI->>TC: getTrace({traceId}) | queryTraces(filter) | getServiceMap()
  TC->>Store: SQL query
  Store-->>UI: trace(s) / service map
```

### `OmnitronTraces` RPC

| Method                 | Effect                                          |
| ---------------------- | ----------------------------------------------- |
| `ingestSpan(span)`     | Push one span                                   |
| `ingestBatch({spans})` | Push many                                       |
| `getTrace({traceId})`  | One trace (all its spans)                       |
| `queryTraces(filter)`  | Filter by service / op / duration / tags / time |
| `getServiceMap()`      | Derived `service_a → service_b` call graph      |

Storage is **Postgres-backed**, not an in-memory ring:
`TraceCollectorService` write-buffers spans (flushing to
`OmnitronDatabase` whenever the buffer hits 500 spans or every
5 s), and `getTrace` / `queryTraces` / `getServiceMap` run SQL
queries against that table.

### `TraceFilter`

```typescript
interface TraceFilter {
  service?:    string;
  operation?:  string;
  minDuration?: number;        // ms
  maxDuration?: number;
  from?:       string;         // ISO timestamp
  to?:         string;
  tags?:       Record<string, string>;
  limit?:      number;
}
```

### CLI

No dedicated `omnitron traces` command. `OmnitronTraces` is a
daemon-hosted RPC service, and `omnitron exec` only reaches
services *inside a managed app* (it routes by app name) — so the
trace API is consumed by the webapp's `/traces` page rather than
from the CLI.

## Operational failures

The [diagnostics guide](/docs/omnitron/diagnostics) describes failure capture, recovery
state in CLI and Console, and optional delivery through Titan notifications.
The runtime indicator reports unresolved daemon failures independently of app
process and infrastructure health.

## Health

Two levels: **app/infra health** (on-demand probes via
`OmnitronHealth`) and **node health** (cross-machine
availability). On top of those sit **titan-health
indicators** that roll up live state for the daemon's own
`/health` summary.

### `OmnitronHealth` — active probing

`HealthCheckService` (`apps/omnitron/src/services/health-check.service.ts`,
exposed as `OmnitronHealth`) combines the orchestrator's application health with
active HTTP/TCP probes. CLI, Console and agent tools read this shared service:

- **Application health** — domain checks returned by the supervised worker.
  A failed check or failed RPC stays visible even if the process is online.
- **Declared readiness** — configured readiness measurements replace an assumed
  HTTP endpoint when supplied by the application.
- **HTTP** — GET `/health` on the application's published port; expect HTTP 200.
- **TCP** — connect to the published ports of declared infrastructure services.
- **Process liveness** — reported separately; it does not override failed checks.

Health hooks should return `{ status, checks: [{ name, status, message }] }`.
Custom `details` fields are not part of this contract: put the actionable reason
in a check's `message`. A hook that returns only a degraded or unhealthy verdict
still causes the aggregate report to fail. `omnitron --json health-check [app]`
returns the same checks with a nonzero exit code for an unhealthy or degraded
report.

```mermaid
sequenceDiagram
  participant Caller as CLI / webapp
  participant HC as OmnitronHealth (HealthCheckService)
  participant App as App process / container

  Caller->>HC: checkApp / checkApps / checkInfrastructure / checkAll
  HC->>App: HTTP GET health endpoint / TCP connect
  App-->>HC: 2xx / socket accepted
  HC-->>Caller: HealthReport { overall, checks[] }
```

### `AppHealthIndicator`

`apps/omnitron/src/monitoring/app-health.indicator.ts` is a
titan-health indicator (name `'apps'`) — **not** a network
prober. It reads `orchestrator.list()` and grades by *process
status*: `healthy` when all managed apps are online, `degraded`
when some are crashed/errored, `unhealthy` when a `critical` app
is down. It feeds the daemon's aggregated `/health` summary, not
the per-app HTTP/TCP probe path above.

### `DockerHealthIndicator`

`apps/omnitron/src/monitoring/docker-health.indicator.ts` is the
infra counterpart (name `'docker'`). It reads
`InfrastructureService.getState()` and aggregates each managed
container's reported `status` / `health` — it does not itself
run Docker healthchecks or fall back to a TCP probe; it reports
the infra service's cached state.

### `OmnitronHealth` RPC

| Method                       | Effect                      |
| ---------------------------- | --------------------------- |
| `checkApp({appName, port?})` | Probe one app on demand     |
| `checkApps()`                | Probe every app             |
| `checkInfrastructure()`      | Probe every infra container |
| `checkAll()`                 | Full platform health report |

### `OmnitronNodes` health (cross-machine)

For fleet nodes (registered in `OmnitronNodes`), the
**health-monitor** runs separately:

| Parameter              | Default                                   |
| ---------------------- | ----------------------------------------- |
| `intervalMs`           | `60_000` (1/min)                          |
| `concurrency`          | `20`                                      |
| `offlineTimeoutMs`     | `90_000`                                  |
| `pingEnabled`          | `true` (ICMP)                             |
| `pingTimeout`          | `5_000`                                   |
| `sshTimeout`           | `10_000`                                  |
| `omnitronCheckTimeout` | `15_000`                                  |
| `retentionDays`        | `90`                                      |
| `uptimeIntervalMs`     | `86_400_000` (24h per uptime bar segment) |

Per-node history is stored for 90 days and surfaced as the
green/yellow/red **uptime bars** in the webapp.

### CLI

```bash
omnitron health [app]              # daemon-level health summary
omnitron health-check [app]        # detailed HTTP/TCP probe report
omnitron node check [id]           # node connectivity probe
```

## Putting it together — what the daemon stores

The default storage directory is `~/.omnitron/`; `OMNITRON_HOME` and explicit
storage settings can relocate it. Selected files are shown below. Application
names and rotated archive names depend on the registered stack and retention
settings.

**\~/.omnitron/**

- `data/`
  - `daemon-state.db` — Local registries, execution intent and state
  - `application-logs.sqlite` — Durable application output journal
  - `backend-logs.sqlite` — Durable backend output journal
- `logs/`
  - `omnitron.log` — Backend log
  - `omnitron.error.log` — Backend errors
  - `api/` — Example standalone application
    - `app.log` — Combined application output
    - `error.log` — Error output
    - `app.log.1.gz` — Example retained archive
- `projects/`
  - `{project}/`
    - `{stack}/`
      - `logs/`
        - `{app}/` — Project application logs and rotated archives
- `execution/` — Resident owner and guardian journals
- `secrets.enc` — Encrypted file-provider secrets

Controller logs, metrics and traces persist in its configured database. Worker
history and pending replication use local SQLite storage. These query stores
are distinct from the private delivery journals; see the pipeline descriptions
above for their commit and acknowledgment semantics.

## Retention

| Signal                          | Retention default                                                                                       |
| ------------------------------- | ------------------------------------------------------------------------------------------------------- |
| Log files                       | `logging.maxFiles` × `logging.maxSize` per stream                                                       |
| Log database                    | `logging.databaseRetentionDays` (default 14 days; nonpositive disables pruning)                         |
| Unacknowledged logs             | Retained in the local journal until both destinations confirm; never evicted by file/database retention |
| Omnitron metric history         | `monitoring.metrics.historyMaxAge` (default `7d`, hourly sweep)                                         |
| Standalone Titan metric history | `retention.maxAge` (default `7d`)                                                                       |
| Traces (write buffer)           | flushed to Postgres at 500 spans or every 5 s                                                           |
| Traces (persistent)             | Postgres (`OmnitronDatabase`) — trim/retain at the DB layer                                             |
| Health history                  | `healthMonitor.retentionDays` (default 90 days)                                                         |
| Alert events                    | from `OmnitronAlerts` per-rule retention                                                                |

Tune retention to match storage budget; defaults are sensible
for small-to-medium fleets.

## Wiring telemetry from your apps

A supervised application's stdout and stderr already reach the owner log capture. To add application-specific metrics, register the metrics module and choose storage based on the required retention. This example uses a process-local registry.

```typescript
import { Module } from '@omnitron-dev/titan/decorators';
import { TitanMetricsModule } from '@omnitron-dev/titan-metrics';

@Module({
  imports: [
    TitanMetricsModule.forRoot({
      appName: 'api',
      collection: { enabled: true },
      storage: { type: 'memory' },
    }),
  ],
})
class AppModule {}
```

Daemon process metrics and replication work independently of that registry. For app-specific readiness checks, configure [Titan health](/docs/titan/modules/health).

## Anti-patterns

- **High-cardinality metric labels.** `userId` as a label
  multiplies series count by user count. Use buckets / tiers,
  not per-entity ids.
- **Treating a file tail as a persistence receipt.** `--file` supports filters and offline recovery, but a bounded tail does not prove database delivery or complete historical coverage. Check `logs --status` and the requested history window.
- **No retention on the persistent metrics store.** Disk fills;
  queries get slow. Default `'7d'` is sensible.
- **Tracing every span at 100% sampling.** Volume kills the
  daemon. Use head sampling at the producer (titan-tracing has
  built-in samplers).
- **Custom HTTP health endpoint that talks to the database on
  every probe.** Health probes run every 15 s — that's a lot of
  pointless queries. Use a cached health summary.
- **Treating `getMetrics()` as a high-frequency call.** It
  aggregates across apps; if you need real-time per-app data,
  query `OmnitronMetrics.querySeries` with the right filter.

## See also

- [`titan-metrics`](/docs/titan/modules/metrics) — producer-side detail
- [`titan-telemetry-relay`](/docs/titan/modules/telemetry-relay) — store-and-forward for offline producers
- [`titan-health`](/docs/titan/modules/health) — indicator framework
- [Services reference](/docs/omnitron/services-reference) — `OmnitronLogs`,
  `OmnitronTelemetry`, `OmnitronTraces`, `OmnitronHealth`,
  `OmnitronNodes`
- [Architecture](/docs/omnitron/architecture) — where these pipelines sit
- [CLI](/docs/omnitron/cli#monitoring) — `omnitron logs` / `metrics` / `health`

## Bounded queries and agent evidence

Log filters are validated in the collector before SQL construction, so CLI, Console and RPC consumers share the bounds: `limit` 1–1000, recent `tail` 1–10000, `offset` 0–100000, valid levels/timestamps, ordered time ranges and bounded strings/labels. These input bounds do not by themselves impose a database execution deadline. Prefer a narrow application and time window over deep offset pagination.

The MCP monitoring readers reuse the same services:

| Tool                  | Reading and budget                                                                                                                    |
| --------------------- | ------------------------------------------------------------------------------------------------------------------------------------- |
| `monitoring.snapshot` | Metrics snapshot, alert summary and log ingestion counters in parallel; no archive counts or active health probes                     |
| `metrics.series`      | 1–8 explicit applications and metric names, a positive window up to 24 hours, 1m/5m/15m/1h aggregation, at most 1000 requested points |
| `alerts.active`       | At most 100 firing alerts; does not acknowledge or silence them                                                                       |

Each source reports `available`, `unavailable` or `limited`, with collection time and duration. A failed or oversized source makes the combined answer `partial`. Each source has a 64 KiB response budget and a five-second diagnostic deadline; the underlying transport still owns RPC cancellation. An oversized source is withheld with a request to narrow the scope, rather than presented as a complete subset.

Collection time is **not** event time or evidence of freshness. Inspect source timestamps and ingestion loss. Zero dropped records since daemon startup does not prove that older data was never lost. A persisted alert summary does not prove that its evaluator is currently connected. Sampled counts must not be presented as exact population totals.

These tools currently run through the local administrative MCP connection. They do not provide delegated agent scopes, bounded database query cost, resumable subscriptions or guaranteed evidence retention. The [agent monitoring design](/docs/omnitron/agents#monitoring-as-evidence) specifies those next contracts and their failure tests.

Treat log messages, traces and tool output as untrusted data. Embedded instructions cannot authorize infrastructure changes. Read permission never implies permission to acknowledge an incident, silence a rule or run remediation.

### Auditing history before receipts

Current replication health and historical completeness are separate readings.
For data predating delivery evidence, use the [retained-history recovery
procedure](/docs/omnitron/cluster#history-before-delivery-receipts). Node diagnostics in CLI
and Console expose the same source coverage, verification time, restoration
counts and unresolved gaps. Historical imports do not update current gauges.
Records intentionally outside retention remain identifiable in their preserved
source archives; they are not reported as a transport failure.

A later recovery keeps the first audit intact and links a new committed proof to
its exact gap fingerprint. Diagnostics distinguish the resolved finding from
still unknown history. Group repeated stack frames into error reports before
counting events; a plain stack trace or a time-of-day-only header cannot be dated
from the audit time. Neighboring absolute timestamps can establish context, but
do not supply an exact event time on their own.

---

---
id: "omnitron/orchestrator"
title: "Orchestrator"
description: "The per-app launch pipeline — bootstrap, fork, watch, hand-off."
---

# Orchestrator

The orchestrator is the daemon's per-app launch pipeline. The
daemon decides **when** to start an app; the orchestrator handles
**how** — locate the bootstrap, compile TypeScript if needed,
resolve dependencies, fork the right child process(es), wire DI,
attach a file watcher in dev, supervise crashes, and tear down
cleanly.

Verified against `src/orchestrator/`:

**orchestrator/**

- `orchestrator.service.ts` — top-level service
- `app-handle.ts` — per-app supervisor state
- `bootstrap-loader.ts` — locates + loads bootstrap.{js,ts}
- `bootstrap-process.ts` — per-app bootstrap entrypoint (classic)
- `module-worker-process.ts` — per-process entrypoint (worker mode)
- `classic-launcher.ts` — fork classic-mode bootstrap
- `ts-compiler.ts` — JIT TypeScript compile
- `file-watcher.ts` — hot reload in dev
- `dependency-resolver.ts` — startup ordering by topology
- `service-router.ts` — RPC routing to apps
- `process-janitor.ts` — orphan + zombie reap
- `build-service.ts` — production bundle / tarball

## Two launch modes

The orchestrator supports two distinct app-launch shapes. The
mode is determined by the app's `defineSystem(...)` shape and the
`bootstrap.{js,ts}` exports.

### Classic mode

A single child process runs the app's `bootstrap.{js,ts}` end-to-
end. The bootstrap is responsible for creating every
`Application`, every `@Module`, and any internal sub-process
management.

| Trait           | Behaviour                                          |
| --------------- | -------------------------------------------------- |
| Forks           | 1 per app                                          |
| Memory          | All Titan modules in one process                   |
| Boot time       | Higher — loads everything                          |
| Use case        | Legacy apps, monolith-style services               |
| Crash semantics | App-wide; whole bootstrap dies, daemon restarts it |

### Module-worker mode (default for new apps)

One child process per `IProcessEntry` declared in
`defineSystem({processes: [...]})`. Each child imports **only**
that process's module file and runs `Application.create(module)`
directly.

| Trait           | Behaviour                                                       |
| --------------- | --------------------------------------------------------------- |
| Forks           | N — one per `IProcessEntry` (× `instances`)                     |
| Memory          | Per-process minimum — only what that module uses                |
| Boot time       | Lower per process; total may be similar but parallelisable      |
| Use case        | Modern multi-process apps, worker pools, isolation requirements |
| Crash semantics | Per-process; daemon restarts just the failed process            |

The orchestrator auto-selects the mode based on whether the
bootstrap exports a `defineSystem()` result with `processes: [...]`
(module-worker mode) or a classic single-app module (classic mode).

## App handle

`AppHandle` is the per-app supervisor record kept inside the
orchestrator. It carries:

```typescript
class AppHandle {
  readonly entry:      IEcosystemAppEntry;
  readonly mode:       'classic' | 'bootstrap';
  get name():          string;        // entry.name
  // Classic mode only — raw ChildProcess
  childProcess:        ChildProcess | null;
  // Bootstrap mode — PM ProcessSupervisor + per-process workers
  supervisor:          ProcessSupervisor | null;
  // Timer for classic-mode crash restart backoff (cleared on explicit stop)
  crashRestartTimer:   ReturnType<typeof setTimeout> | null;
  // Restart accounting
  restarts:            number;
  // Persisted to the daemon state store
  status: 'stopped' | 'starting' | 'online' | 'stopping' | 'errored' | 'crashed';
}
```

One `AppHandle` per app. The handle survives restarts; the
`childProcess` / `supervisor` reference is replaced on each
relaunch.

## Launch pipeline

```mermaid
sequenceDiagram
  participant D as Daemon
  participant Orch as Orchestrator
  participant BL as BootstrapLoader
  participant TSC as TS compiler
  participant DR as DepResolver
  participant CL as Launcher
  participant Child as Child process

  D->>Orch: launch(name)
  Orch->>BL: locate bootstrap path
  alt bootstrap is TypeScript (dev)
    BL->>TSC: compile to ESM
    TSC-->>BL: compiled artefact path
  end
  BL->>BL: import(bootstrap)
  BL-->>Orch: { mode, definition }
  alt mode = bootstrap (module-worker)
    Orch->>DR: resolve startup order
    DR-->>Orch: ordered processes
    loop per process
      Orch->>CL: fork module-worker-process.js
      CL-->>Child: spawn
      Child->>Child: init({bootstrapPath, processName})
      Child->>Child: Application.create(module)
      Child->>Child: onInit → onStart
      Child-->>Orch: ready handshake
    end
  else mode = classic
    Orch->>CL: fork bootstrap-process.js
    CL-->>Child: spawn
    Child->>Child: import(bootstrap)
    Child->>Child: run app
    Child-->>Orch: ready
  end
  Orch-->>D: ProcessInfoDto
```

The ready handshake is mandatory — the daemon does not consider
an app "running" until at least one child reports ready.

## Bootstrap loader — `bootstrap-loader.ts`

Locates and loads the app's bootstrap module. Resolution order:

1. Explicit path from `IEcosystemAppEntry.bootstrap`.
2. `<cwd>/dist/bootstrap.js` (production builds).
3. `<cwd>/src/bootstrap.ts` (dev mode — requires TS compiler).

When the source is TypeScript, it routes through the embedded
TS compiler. When the source is already compiled JS, it `import`s
directly.

## TS compiler — `ts-compiler.ts`

The declaration reader compiles TypeScript bootstrap definitions with esbuild so
plain Node can inspect fresh topology before starting workers. Declaration reads,
worker builds and watched rebuilds share `runtime-package-resolution.ts`:

- TypeScript workspace dependencies are compiled with their relative imports,
  decorators and constructor parameter properties. ESM-only package exports work.
- Compiled SDK and registry packages stay external, preserving their installed
  module and DI identities. Each import retains the dependency selected from its
  original package, including a library's private dependency version.
- External paths are relative to the **physical output directory**. A registered
  project may be a directory symlink; Node resolves that symlink when importing
  the output. Using its lexical depth would produce an invalid dependency path.
  ESM paths also preserve spaces, `#` and `%` through URL encoding.

Qualify a development stack through its registered path, including any symlink,
not only through the physical checkout. Verify an initial launch, a shared-library
edit that triggers a watched rebuild, and a fresh manual restart after disabling
watch. This checks the same compilation policy used by the resident owner.

> This module is `@deprecated` — superseded by `BuildService`,
> which pre-bundles every entry point. It is retained for the
> bootstrap-loader fallback and classic-launcher compatibility.

| Behaviour            | When                                                                                                                              |
| -------------------- | --------------------------------------------------------------------------------------------------------------------------------- |
| Fresh compile        | Cold start; cache miss (by source mtime)                                                                                          |
| Cached output reused | Source `mtime` unchanged since last compile                                                                                       |
| Compile errors       | Thrown — surfaced to the caller                                                                                                   |
| Source maps          | Not emitted (`sourcemap: false`)                                                                                                  |
| Output location      | Next to the source as `<name>.omnitron-compiled.mjs` (gitignored), unlinked after import in the daemon path — no global cache dir |

## Dependency resolver — `dependency-resolver.ts`

Reads the topology graph implied by each process's `topology`
declarations (`expose` / `access`). Computes a startup order such
that **providers** start before **consumers**:

```mermaid
flowchart LR
  Bootstrap[defineSystem processes]
  Bootstrap --> Build[Build topology graph]
  Build --> Sort[Topological sort]
  Sort --> Order[Startup order: providers → consumers]
```

A process declaring `topology: { access: ['SomeService'] }`
waits for the process declaring `topology: { expose: true }` on
`SomeService` to be ready. Cycles are detected and reported as
configuration errors at boot.

## Service router — `service-router.ts`

Routes RPC calls between the orchestrator and individual app
processes. When the daemon's `exec({name, service, method,
args})` arrives, the router:

1. Looks up the `AppHandle` by app name.
2. Finds the process within the app that hosts the requested
   service (via the discovered service map).
3. Proxies the call through that process's Netron transport.
4. Returns the result.

For pool-mode (`instances > 1`), the router dispatches through
the PM pool's `execute()`, which load-balances across instances
using a power-of-two-choices (P2C) policy.

## File watcher — `file-watcher.ts`

In dev mode (or when an app's `watch` is enabled), the
orchestrator runs a per-app `FileWatcher` built on Node's native
`fs.watch({ recursive: true })` (not chokidar). On file change:

```mermaid
flowchart LR
  FS[fs.watch event]
  FS --> Debounce[debounce ~300 ms]
  Debounce --> Restart[orchestrator.restartApp]
```

Heuristics:

- **Bootstrap-mode apps skip `fs.watch` entirely.** Their import
  graph is watched precisely by `BuildService` (esbuild
  `context.watch`), which knows exactly which files contribute to
  the bundle — running `fs.watch` on the same tree on top of that
  produced duplicate, staggered restart triggers (a primary cause
  of dev-mode restart storms).
- **Classic / non-bootstrap apps** get the `fs.watch` watcher,
  which on a change debounces then calls `restartApp`.
- A missing watch directory is logged at **error** (hot reload is
  silently broken otherwise), not warn.

The default debounce is **300 ms**, overridable per app via the
`watch` config; editor saves often produce multiple events, so
the watcher collapses a burst to a single restart per window.

## Classic launcher — `classic-launcher.ts`

Forks `bootstrap-process.js` (a thin wrapper) and waits for ready
handshake. Manages:

- stdio piping for log collection.
- Signal forwarding (`SIGTERM` → child).
- Crash detection via exit-code observation.
- Per-app crash-restart backoff timer.

## Module-worker entrypoint — `module-worker-process.ts`

The generic `@Process`-compatible class the PM worker-runtime
instantiates for each module-based topology entry (one per
`IProcessEntry`). The orchestrator wires these through a PM
`ProcessSupervisor` rather than forking this file directly. Each
worker:

1. Receives its `bootstrapPath` + `processName` via
   `init(bootstrapPath, processName)`.
2. Loads the bootstrap definition to find the matching
   `IProcessEntry`.
3. Imports the process's `module` file.
4. Discovers the `@Service`-decorated provider in the module's
   metadata.
5. Creates a Titan `Application` with the entry's module.
6. Runs lifecycle (`onInit` → `onStart`).
7. Exposes the service's methods over Netron RPC and reports
   ready.

A key invariant: the child imports the **single module file**
declared in `IProcessEntry.module` — not the entire bootstrap —
so an app with 10 processes loads 10 disjoint subsets of code.

## Titan identity and installed application paths

The application and its bootstrap adapter must resolve the same physical Titan installation. Two copies of the same version export different `Container` classes. If dependency injection reports an identity mismatch, check both paths in the startup diagnostic and restore the application's frozen installation.

Both coupled and resident daemons resolve the adapter from the application's current dependencies. A fresh reader runs once per application generation, so single processes and pool workers share the chosen adapter. A damaged adapter or a different Titan installation prevents launch. When only Titan is installed, the application can use an owner adapter that resolves that same Titan.

Coupled mode also accepts a plain bootstrap with neither framework package installed or declared. In that case the daemon supplies its own adapter.

This behavior is advertised as the owner capability `applicationRuntimeResolution: 1`, reported in `execution.ownerCapabilities` by `omnitron status --json` and the shared daemon status API. Older owners can retain Node package resolutions across application restarts. Replacing `node_modules` under the same absolute bootstrap path can then launch the previous adapter against the new application. Updating only the backend does not update the owner's resolver. Restore the previous coherent application SDK or prepare a qualified owner transition with an application pause and rollback.

For a development SDK update, pause only the affected source watchers and applications, perform the normal frozen dependency installation, then restart those applications through Omnitron to rebuild their entry points. Preserve other projects' watcher selections. Verify readiness before restoring the original selection. Qualification must exercise upgrade and rollback under one resident owner; a successful import from a fresh shell does not cover its cached resolution history.

For an installed application, keep each code revision and its dependencies in a distinct physical directory. Preserve the project's registration and state paths. Verify the new launch through the existing owner, repeat the ordinary application start, and test the return to the previous installation. Check job commands separately when a scheduled job launches its own interpreter. An ordinary local development stack runs its working tree.

## Process janitor — `process-janitor.ts`

Background sweep that reaps:

- **Orphans** — child processes whose `AppHandle` no longer
  exists (e.g., daemon was killed mid-spawn).
- **Zombies** — exited children whose exit status hasn't been
  collected.

Runs periodically; never aggressive enough to kill legitimate
children. The janitor's existence means a daemon restart doesn't
leave a leak of orphaned child PIDs.

## Build service — `build-service.ts`

Pre-bundles each process entry point (via esbuild) into the
app's `.omnitron-build/` directory so the daemon imports a single
compiled `bootstrap.js` instead of recompiling source per launch.
This is the runtime bundling path; it also drives the esbuild
import-graph watcher used for bootstrap-mode hot reload.

> The deployment **tarball** for `omnitron deploy build <app>` is
> a separate path: it goes through `ArtifactBuilder`
> (`src/project/artifact-builder.ts`), which runs the app's
> `pnpm build`, resolves workspace deps, and writes
> `<projectRoot>/.omnitron/artifacts/<app>-<version>.tar.gz`. The
> CLI reports the artifact path, version, size, and checksum.

## Reload semantics

`omnitron reload <app>` triggers a **graceful** restart:

- In **module-worker mode**: workers cycle one-by-one; pool size
  is maintained throughout; no request loss.
- In **classic mode**: a new bootstrap is forked side-by-side; the
  old one is signalled to drain (`SIGTERM`), the new one takes
  over once it reports ready.

Reload differs from restart: restart stops everything first,
incurring a downtime window; reload keeps capacity online
throughout.

## Watch control (operator API)

The `OmnitronDaemon` service exposes:

| Method                 | Effect                                      |
| ---------------------- | ------------------------------------------- |
| `enableWatch({apps?})` | Start watching listed apps (all if omitted) |
| `disableWatch()`       | Stop all file watchers                      |
| `getWatchStatus()`     | `{ enabled, apps: [{name, directory}] }`    |

The CLI surfaces these implicitly through the `--no-watch` flag
on `omnitron up` — at runtime, you can toggle without restart.

## Per-app observability

Each launched app reports:

- **stdio logs** → ingested by the daemon's log-collector service.
- **Process metrics** are sampled by the daemon. Worker samples reach the controller through the authenticated replication outbox.
- **Health probes** → pulled by the daemon's health scheduler.
- **Ready / shutdown** → IPC handshake on the fork channel.

The orchestrator instruments every fork so these signals are
automatic — apps don't have to wire anything explicitly beyond
loading `titan-health` / `titan-metrics` modules.

## Failure recovery

| Symptom                            | Orchestrator response                                            |
| ---------------------------------- | ---------------------------------------------------------------- |
| Child exits non-zero               | Apply restart policy; backoff; persist `restarts++`              |
| Child times out on ready handshake | Treat as start failure; backoff; retry up to `maxRestarts`       |
| Container identity mismatch        | Surface error; mark app `crashed`; no auto-retry                 |
| TS compile error                   | Surface error; if dev mode, watcher will re-attempt on next save |
| Bootstrap module not found         | Mark `crashed`; surface error from the loader                    |
| Janitor finds orphan               | `SIGKILL` the orphan; log                                        |

## Anti-patterns

- **Single huge bootstrap with many `@Module`s in classic mode.**
  Loads everything in one process. Prefer module-worker mode with
  one process per concern.
- **Poking the daemon state DB to remove a crashed app.** Use
  `omnitron stop project/stack/app` to address the affected application.
- **File watcher in production.** Wastes CPU and risks
  accidentally hot-reloading prod on a config push. Production
  ecosystems should set `watch.enabled: false`.
- **Cyclic topology declarations.** The dependency resolver
  detects cycles and refuses to start. Refactor to break the
  cycle (often: introduce a shared interface module that both
  sides import).
- **Mixing classic and worker mode in one app.** Pick one. The
  orchestrator can run both modes side-by-side, but only across
  apps — within one app, the mode is global.

## See also

- [Daemon](/docs/omnitron/daemon) — what invokes the orchestrator
- [Architecture](/docs/omnitron/architecture) — where the orchestrator fits
- [titan-pm](/docs/titan/modules/pm) — the process manager the
  orchestrator builds on
- [Services reference](/docs/omnitron/services-reference) — `OmnitronDaemon.exec`
  routes through the service router
- [CLI](/docs/omnitron/cli) — `start`, `stop`, `reload`, `restart`, `scale`

## Stop acknowledgements

An application is marked `stopped` only after its supervisor, classic process group
and topology pools confirm termination. Failed pools retain their handles for
recovery. The classic launcher uses Titan's process termination helper; a sent
signal or expired wait cannot become a successful stop.

Infrastructure teardown reports failed containers and retains stack ownership and
port reservations until removal succeeds. The Xec Docker adapter resolves a name
to an immutable container ID and verifies absence after removal. A replaced name,
unavailable daemon, or denied request is an explicit failure.

Omnitron 0.4.9 uses Xec 0.12.0 from the registry, with the exact dependency graph
recorded in its lockfile. The Docker adapter preserves the requested stop grace
period and allows time for the
transport acknowledgement. An expired command deadline does not prove that the
container stopped; inspect its state before retrying or escalating.

RPC service withdrawal is part of the acknowledgement. Failed withdrawal retains
its router for retry while process termination still runs. Destroyed pools reject
late background spawns; destruction is terminal.

The owner stores a FULL-durability SQLite fence per project/stack. It binds the stop
to exact handle generations and an operation identity. Automatic resume cannot
clear a deliberate stop. See [confirmed stack stops](/docs/omnitron/projects#confirmed-stack-stops)
for controller/node receipts, rollout prerequisites and remaining allocation limits.

---

---
id: "omnitron/overview"
title: "Overview"
description: "What Omnitron is, what it solves, and how the pieces fit together."
---

# Omnitron

> **Omnitron is opt-in.** Titan apps run perfectly well without
> it as plain Node processes — see
> [Titan vs Omnitron](/docs/foundations/titan-vs-omnitron) for
> the precise boundary. This page describes what you gain by
> adding the Omnitron supervisor on top.

**Omnitron is the supervisor, control plane, and operator surface
for Titan applications.** One long-running daemon manages many
processes across many apps across many projects, exposes a typed
RPC plane for inspection, ships a CLI and a web console, and
provisions the infrastructure your apps need.

It's itself a Titan app — supervising other Titan apps. Built on
the same primitives the apps it supervises use.

```bash
pnpm add -g @omnitron-dev/omnitron
omnitron up                          # start daemon + all configured apps
omnitron status                      # see what's running
omnitron logs --grep error           # tail filtered logs
omnitron webapp open                 # open the web console
```

## What you get

| Layer              | Responsibility                                                                                                                                           |
| ------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **CLI**            | One `omnitron` entry point for applications, stacks, deployments, infrastructure, diagnostics and backend updates.                                       |
| **Daemon**         | Typed Netron control plane with durable local state. Coupled mode owns applications directly; resident mode keeps them in an independent owner.          |
| **Orchestrator**   | Per-app launch pipeline: TS compile / module load / fork process / wire DI / start lifecycle / watch files in dev                                        |
| **Process tree**   | One OS process per `IProcessEntry` you declare; ≥ 1 instances per process; auto-restart with exponential backoff                                         |
| **Services**       | Role-protected daemon RPC for authentication, deployment, fleet inventory, secrets, infrastructure and diagnostics. Availability depends on daemon role. |
| **Webapp**         | React + Vite single-page console — same RPC surface; lives at `apps/omnitron/webapp/`                                                                    |
| **Infrastructure** | Provisions Postgres / Redis / S3-compatible / custom containers via Docker (dev) or bare-metal hooks (prod)                                              |
| **MCP**            | Built-in Model Context Protocol surface — let agents (IDE assistants, CI bots) read state and trigger operations                                         |

## When you reach for Omnitron

- **You have ≥ 2 Titan apps** and want one place to start, watch,
  inspect, and shut them down together.
- **You need a typed control plane**, not bash scripts. Every
  operator action goes through Netron RPC with auth, RBAC, and
  audit.
- **You want dev / staging / prod parity** with declarative infra
  — the same `omnitron.config.ts` boots Docker locally and
  bare-metal remote, with secrets resolved per environment.
- **You operate a fleet** — multiple nodes, leader election,
  remote daemons addressed by alias, cross-node health and
  metrics.
- **You want the console UI for free** — `omnitron webapp open`
  serves the React console talking to the same daemon.

If you only ever run **one** Titan app on **one** machine, `node
my-app.js` may be enough. Omnitron starts to pay back at two apps
or two machines.

## The mental model

```mermaid
flowchart TB
  CLI["CLI"] -->|Unix socket| RPC
  Console["Console"] -->|Netron over HTTP / WebSocket| RPC
  Agent["Agent via MCP"] -->|MCP adapter and local RPC| RPC

  subgraph Instance["Omnitron instance (one per node)"]
    RPC["Backend: Netron RPC services"]
    State[("SQLite state and registries")]
    Guardian["Resident backend guardian"]
    Owner["Application owner"]
    RPC --> State
    Guardian -->|Supervise backend| RPC
    RPC -->|Lifecycle commands| Owner
  end

  Owner -->|Supervise| AppA["Application: API"]
  Owner -->|Supervise| AppB["Application: ingestion"]
  RPC -->|Provision declared resources| Infra
  subgraph Infra["Infrastructure (Docker / bare-metal)"]
    PG[("PostgreSQL")]
    Redis[("Redis")]
    S3[("S3-compatible storage")]
  end
  AppA -. Uses .-> PG
  AppA -. Uses .-> Redis
  AppB -. Uses .-> PG
  AppB -. Uses .-> S3
```

The diagram shows **resident mode**: the owner retains applications while the
guardian replaces the backend. In default coupled mode, the backend also owns
the application lifecycle. Moving from coupled to resident mode requires an
initial application restart; later compatible backend handoffs can preserve
those processes. See [backend replacement](/docs/omnitron/backend-handoff).

Ordinary CLI invocations are short-lived RPC clients. Follow and watch commands
keep their connection or polling loop for as long as the operator needs it.

## Filesystem layout

By default, Omnitron stores its control-plane files in `~/.omnitron/`.
`OMNITRON_HOME` selects another instance directory. This is a selected layout:

**\~/.omnitron/**

- `daemon.sock` — Private management socket
- `daemon.pid` — Backend process identity
- `data/`
  - `daemon-state.db` — Registries, execution intent and persisted state
  - `application-logs.sqlite` — Application log delivery journal
  - `backend-logs.sqlite` — Backend log delivery journal
- `runtime/`
  - `releases/` — Verified, installed backend runtimes
- `execution/` — Resident owner and guardian control files
- `secrets.enc` — Encrypted file-provider secrets
- `logs/`
  - `omnitron.log` — Backend log
  - `omnitron.error.log` — Backend errors
- `projects/` — Project and stack state, including application log archives
- `backups/` — Database dumps and captured resource archives

The current state store is SQLite; `state.json` is only an import source for
older installations. Backups use `backups/` under the selected `OMNITRON_HOME`;
there is no separate backup-directory setting. Project builds, container volumes
and native service definitions have their own locations. The
[observability guide](/docs/omnitron/observability#putting-it-together--what-the-daemon-stores)
distinguishes query storage from delivery journals and log archives.

## Default endpoints

| Plane      | Endpoint                                                    | Purpose                                                                            |
| ---------- | ----------------------------------------------------------- | ---------------------------------------------------------------------------------- |
| Management | `unix://~/.omnitron/daemon.sock` (mode `0o600`, owner-only) | CLI ↔ daemon — `OmnitronDaemon` service + the rest of the built-in services        |
| TCP        | `tcp://127.0.0.1:9700` by default                           | Authenticated daemon RPC; configure the bind address for remote access             |
| Console    | `http://localhost:9800` by default                          | Browser interface; publication and backend endpoints follow the configured gateway |

The Unix socket is the **trust boundary** — owner-only file
permissions make local CLI calls implicit-auth. TCP and HTTP
require JWT.

## A minimal `omnitron.config.ts`

The single configuration file at the repo root:

```typescript
import { defineEcosystem } from '@omnitron-dev/omnitron';
import { fileURLToPath } from 'node:url';
import path from 'node:path';

const __dirname = path.dirname(fileURLToPath(import.meta.url));

export default defineEcosystem({
  apps: [
    {
      name:        'api',
      bootstrap:   path.join(__dirname, 'apps/api/dist/bootstrap.js'),
      cwd:         path.join(__dirname, 'apps/api'),
      env:         { NODE_ENV: 'production' },
      watch:       { enabled: true, paths: ['src/**/*.ts'] },
    },
    {
      name:        'worker',
      bootstrap:   path.join(__dirname, 'apps/worker/dist/bootstrap.js'),
      cwd:         path.join(__dirname, 'apps/worker'),
    },
  ],
  supervision: {
    strategy: 'one_for_one',
    maxRestarts: 5,
    window: 60_000,
    backoff: { type: 'exponential', initial: 1_000, max: 30_000, factor: 2 },
  },
});
```

Each app's `bootstrap.ts` exports a `defineSystem(...)` that
declares the **processes** (one OS process per `IProcessEntry`).

## How an app launches

```mermaid
sequenceDiagram
  participant Op as Operator (CLI)
  participant D  as Daemon
  participant Orch as Orchestrator
  participant Boot as bootstrap.js
  participant Child as Child process

  Op->>D: start api
  D->>Orch: launch(app: 'api')
  Orch->>Orch: locate bootstrap.js
  Orch->>Boot: import('bootstrap.js')
  Boot-->>Orch: defineSystem({processes: [...]})
  loop per process
    Orch->>Child: fork(module-worker-process.js)
    Child->>Child: Application.create(module)
    Child->>Child: onInit → onStart
    Child-->>Orch: ready handshake
  end
  Orch-->>D: ProcessInfoDto
  D-->>Op: { status: 'running', pid: 1234, ... }
```

Each `IProcessEntry` declared in `defineSystem(...)` becomes
exactly one fork (or a pool when `instances > 1`). The child
imports **only** that module's file — no full bootstrap re-load —
so dependencies stay process-isolated.

## Multi-project / multi-stack

A **project** owns one or more **stacks**. Each stack selects application
configuration, infrastructure and placement for an environment:

| Project      | Stack     | Applications                          |
| ------------ | --------- | ------------------------------------- |
| `monorepo-A` | `dev`     | `api`, `worker`                       |
| `monorepo-A` | `staging` | `api`, `worker`, `batch`              |
| `monorepo-A` | `prod`    | `api`, `worker`, `batch`, `scheduler` |
| `monorepo-B` | `prod`    | `gateway`, `events`                   |

Stack-aware commands (`omnitron stack status <project> <stack>`)
operate on that stack. Environments can share a node or target different
nodes; they remain configurations of the same project. See
[projects and stacks](/docs/omnitron/projects) for placement and lifecycle commands.

## Built-in observability

The supervisor captures process state and output. Additional signals depend on
application instrumentation and configured collectors:

| Signal                                 | Where                                                                                            |
| -------------------------------------- | ------------------------------------------------------------------------------------------------ |
| Application output and structured logs | Durable capture and rotated per-application files; see [logs](/docs/omnitron/observability#logs) |
| Per-process metrics                    | aggregated by daemon, exposed via RPC + Prometheus text                                          |
| Health probes                          | Application-provided checks and declared readiness probes                                        |
| Crash tracking                         | Restart count, last error, backoff state                                                         |
| Distributed traces                     | Configured application tracing and collectors                                                    |

Both the CLI and webapp read these through the daemon — there's no
separate metrics agent to deploy.

## Webapp (Omnitron Console)

The console is a React + Vite + MUI SPA that lives at
`apps/omnitron/webapp/` and talks to the daemon via the
`@omnitron-dev/netron-react` integration and browser client. It gives you:

- Project + stack view with per-app status cards
- Live log streams with filtering
- Metric charts (CPU / RSS / event-loop lag / RPC latency)
- Health probes and restart history
- Infrastructure inventory (Docker containers)
- Secret editor (against the daemon's encrypted store)
- Fleet view across remote daemons

The published package includes Console assets. `omnitron webapp start`
reconciles the serving gateway; `omnitron webapp open` opens the configured
address. `omnitron webapp build` is for a source checkout. For development, run `pnpm dev` in `apps/omnitron/webapp/`
for a Vite dev server with HMR.

→ Full webapp reference: [console](/docs/omnitron/console).

## Compatibility

| Component  | Requirement                                                                                                                                             |
| ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Node.js    | Use a supported runtime from the [compatibility reference](/docs/reference/compatibility); native dependencies are qualified for the target Node and OS |
| OS         | macOS, Linux. Windows works for CLI + non-supervisor commands; daemon assumes Unix sockets                                                              |
| Docker     | Required for `infra` commands (provisioning Postgres / Redis / etc.)                                                                                    |
| Bun / Deno | Apps may run on Bun/Deno individually, but the daemon expects Node                                                                                      |

## Where to go next

- **Architecture deep-dive:** [Architecture](/docs/omnitron/architecture) —
  every component, every plane, every data flow.
- **CLI reference:** [CLI](/docs/omnitron/cli) — every command, every flag.
- **Daemon internals:** [Daemon](/docs/omnitron/daemon) — PID, state-store,
  scheduler, lifecycle.
- **Orchestrator:** [Orchestrator](/docs/omnitron/orchestrator) —
  bootstrap, file watcher, dependency resolver.
- **RPC services:** [Services reference](/docs/omnitron/services-reference) —
  all built-in services and their methods.
- **Console UI:** [Console](/docs/omnitron/console).

---

---
id: "omnitron/projects"
title: "Projects and stacks"
description: "One monorepo owns its applications and libraries; named stacks express deployment environments without duplicating projects."
---

# Projects and stacks

Register a repository once. A project owns its applications and internal libraries;
a stack selects the applications, infrastructure and placement for one named environment.
`local`, `test`, `stage` and `prod` are ordinary stack names, not separate projects.
A stack can run on one machine or span several nodes, mixing processes, containers
and explicitly configured external services.

| Boundary           | Owns                                                                          | Does not imply                            |
| ------------------ | ----------------------------------------------------------------------------- | ----------------------------------------- |
| Project            | Repository, application definitions, internal libraries, stack configurations | One server or one environment             |
| Stack              | Application selection, service overrides, nodes, deployment policy            | A new project or a copy of source code    |
| Placement          | A particular application/service on a physical node                           | Ownership of the whole stack              |
| Service connection | A consumer's explicit destination and credentials                             | Permission to deploy or stop the provider |

Keep shared internal code in workspace packages. Fetch independently versioned libraries
from the npm registry and lock their versions. A development checkout is an explicit
local override, not a production dependency on another repository's filesystem.

## Start with one configuration

Define the applications once in the root `omnitron.config.ts`, and describe environments
in its `stacks` map. Register the root, then use the same project identity everywhere:

```bash
omnitron project add sample /srv/projects/sample
omnitron stack list --project sample
omnitron stack status sample test
omnitron topology show sample prod
```

See [configuration](/docs/omnitron/configuration), [infrastructure overrides](/docs/omnitron/infrastructure#stack-level-overrides)
and [release admission](/docs/omnitron/releases). Promoting a release does not create another project.

## Compose configurations in a large repository

An independently maintained configuration directory inside the same repository can
contribute stacks to its owner. It remains a source of configuration, not another
workspace in the project menu. Every source stack gets an explicit, unique name in
the owning project:

```bash
omnitron project configuration import sample research /srv/projects/sample/research --stack local=research
omnitron project configuration list sample
omnitron stack status sample research
```

Console exposes the same operation through **Projects → Import stacks**. The directory
must resolve inside the registered repository; symlinks cannot escape that boundary.
Map all configured and user-created source stacks. A name collision is refused.
The configuration is executable project code: importing it is an administrator operation.

A new `project add` for a directory already owned by a registered project is refused
with guidance to import its stacks. Project settings show configuration sources;
navigation remains **project → stack**.

## Consolidate an older registration without interrupting applications

If a repository subdirectory was previously registered as `sample-research`, attach
its existing runtime identity explicitly:

```bash
omnitron project configuration import sample research /srv/projects/sample/research --registration sample-research --stack local=research
```

This atomically removes the separate catalogue project and assigns its configuration
to `sample`. It does not stop, restart or rename processes. Auto-start settings, secrets,
checkpoints, log/metric identities and existing experiment registrations retain their
original namespaces. CLI stack operations resolve public names to those namespaces.
`omnitron logs sample/research/api` resolves the same application as its former name.
Low-level process diagnostics may still display the retained runtime name.

Project-wide logs and metrics include owned configuration sources; selecting one
stack narrows them to that source. Console restores a saved selection of the former
registration inside the owning project. The infrastructure map uses the same ownership.

The RPC requires `expectedRevision`; CLI and Console supply the catalogue revision they
read. A concurrent change refuses the operation instead of overwriting it. Configuration
ownership and removal of the former project row commit together in SQLite with FULL
synchronous durability. Recovery and database backup enumeration include every source.

To undo just the ownership change, without reverting newer history:

```bash
omnitron project configuration detach sample research
```

This restores the former standalone registration and preserves its runtime state.
Do not restore an old complete daemon database over newer telemetry to undo an import.
An owner with imported configurations cannot be removed until they are detached.

## Reuse a service across project boundaries

A consuming project does not import the provider's configuration or share its lifecycle.
Declare an external destination using a stack's `serviceOverrides.external`; bind
consumer credentials through the project's secret facilities. The provider remains
responsible for its deployment, data and availability. Stopping the consumer must not
stop that provider or remove its data. Use separate, least-privilege credentials for
each consuming stack, with an explicit API/protocol version and compatibility policy.

Today these connections are explicit endpoints. An external override disables local
provisioning of that service; the topology reports configuration evidence, not proof
of live traffic or health. Cross-project contract discovery, automatic provider
selection and consumer-aware rollout are future control-plane capabilities, not
permissions implied by an endpoint or a shared display name.

See the [topology workspace](/docs/omnitron/topology) and the
[resilient control-plane contract](/docs/omnitron/resilient-control-plane) for the distinction
between desired placement, observed state and admitted execution.

## Verify a standalone stack start

To keep a local stack on reviewed code while the developer checkout changes,
[prepare and select a local release](/docs/omnitron/releases#pin-a-local-stack). Record the
selection for each stack. Its working directories and persistent data remain
attached to the registered project.
A checkout path is not proof of development mode: inspect the running application's
release identity. See [development mode and a selected release](/docs/omnitron/releases#development-mode-and-a-selected-release)
for the current stack-wide selection boundary and HMR behavior.

Before starting a local stack, open **Stacks → Diagnose** or run
`omnitron stack diagnose sample local`. This read-only check shows who manages
each service and which prerequisites can be verified while the applications are stopped.

When you intend to start the stack, run `omnitron --json stack start sample local`.
Check the exit code, `data.outcome`, and the `data.online` and `data.total` counts.
A partial start returns exit code 1. If the connection drops, read the current
state before submitting another start.

```bash
omnitron stack status sample local
omnitron logs sample/local/api --lines 50
```

Open the stack’s **Applications** section in Console or read the **Apps** rows
in CLI status. **Process** and **Health** are separate columns: online processes
may have failed domain checks. The stack health summary identifies warnings,
failures and missing observations; detail retains the application and node.
**Start prerequisites** checks configuration and provisioning without starting
anything. See [process and health readings](/docs/omnitron/diagnostics#process-health-and-start-prerequisites).

Standalone deployments use the ordinary start workflow. On an enrolled HA
cluster, an operator recovering a previously stopped installation can
[resume its verified release](#resume-installed-applications).

## Confirmed stack stops

In Console, choose **Review stop…** on a stack card or its detail page. The same
read-only inventory is available to viewers with
`omnitron --json stack stop <project> <stack> --dry-run`. Each row identifies the node, runtime registration,
applications and containers. An unavailable owner or a missing source binding
appears with its reason and prevents confirmation. Opening or closing the review
does not change automatic resume or create a stop operation.

Choose whether to stop **applications and infrastructure** (the default) or
**applications only**, retaining the stack's containers. For the latter, use
`--keep-infrastructure` in both review and execution:

```bash
omnitron --json stack stop example production --keep-infrastructure --dry-run
omnitron --json stack stop example production --keep-infrastructure --review <digest>
```

Operators can confirm the reviewed inventory in Console. For automation, pass
the returned `data.digest` to the same command with `--review <digest>`.
The digest binds the chosen mode as well as the application/resource inventory.
Changing the mode, process generation, allocation or resource list requires a fresh review.
A nonzero exit status and a null digest mean the full impact could not be verified.

`omnitron stack stop <project> <stack>` prepares an ownership-checked plan on
**every participating node before sending any stop**. Each node records an operation
identity in its private SQLite journal, the current application generations, and
immutable Docker container IDs. Plans expire after ten minutes before execution.
Preparation does not stop applications or remove containers.

Once admitted, the controller durably disables automatic stack resume. The process
owner persists a separate execution fence before terminating anything. That fence
also rejects watcher restarts, scaling and automatic recovery starts. An explicit
new stack start may clear it only after the preceding stop is confirmed.

In the default mode, applications terminate before their infrastructure.
Infrastructure reconciliation and already admitted provisioning are drained before
graceful container removal. Docker volumes are retained. A failed process exit
prevents infrastructure removal on that node.

With `--keep-infrastructure`, application termination and its execution fence are
the same, but the existing containers, infrastructure ownership and remote target
inventory remain. The selected release is unchanged. Automatic application resume
is disabled in both modes: retaining containers does not enable applications again
after a backend or owner restart. Use an explicit stack start when maintenance is
complete. A stopped application stack can therefore have running infrastructure;
inspect `stack runtime` and infrastructure diagnostics separately. Other stacks
and node RPC connections remain available.

The node returns a durable receipt only after process termination, withdrawn RPC
services and the requested infrastructure outcome are confirmed: container absence
for a full stop, or the same retained container IDs for an applications-only stop.
A changed retained container inventory leaves the operation unconfirmed. Retrying an operation identity
reconciles its receipt; an old receipt cannot acknowledge a later application
generation. A timeout or lost connection remains **unconfirmed**. It does not mean
that the applications stopped, and it does not trigger a fallback that kills by
bare app name, port or directory.

```bash
omnitron stack stop example production --json
omnitron stack status example production --json
```

The response includes `stopOperation`: its identity, phase, committed intent,
per-node outcomes and diagnostic reasons. Console uses the same persisted state,
including after a page/backend reload. A failed stop retains the resources needed
for reconciliation; complete it before starting a new deployment. Every user stop
records success or failure in the audit trail.

### Resume installed applications

After an operator confirms a signed stop, the elected controller can resume the same installed applications on their original nodes. Use **Resume installed apps** in Console or run:

```bash
omnitron --json stack resume example production
omnitron --json stack status example production
```

Each node must still be bound to the source stack and retain the stopped process generations and verified release artifacts. The stop must identify each node's exact operation and must have removed no containers. A changed placement, missing owner capability, older receipt without a node operation ID or missing installation prevents recovery. Review a new stop to replace an older receipt. Use deployment when an application needs a different release or configuration.

The process owner binds the application generation, launch configuration, manifest, archive and installed tree at verified launch. A backend upgrade alone cannot reconstruct this evidence for an old owner. Classic Node applications must acknowledge readiness over IPC within 30 seconds (for example, `process.send?.("ready")`).

Preparation checks every node before the first launch. Each owner resumes its applications in dependency order. A node confirms success only after observing all of them online. The result identifies the operation, individual nodes, observed applications, time and failure reason. A lost acknowledgement triggers a read of the node's durable receipt. The controller stops dispatching on the first uncertain target. CLI returns a nonzero exit code for a partial or unconfirmed result.

Use stack status to inspect the persisted `resumeOperation` after a disconnect. An observation describes the applications at that instant. For current health, inspect the stack's live placements and application diagnostics. After a partial start, inspect the affected nodes, then review and complete a fresh stop before another recovery attempt. Recovery leaves the previously disabled automatic startup policy in place.

### Existing allocations and rollout

Generated node configurations carry `managedOrigin: { project, stack }` and
`infrastructureOwnership: 'controller'`. The source stack owns provisioning and
migrations; the generated local allocation uses explicit resolved connections and
starts applications without inventing another infrastructure stack. See
[infrastructure ownership](/docs/omnitron/infrastructure#infrastructure-ownership). Both
remote controller stop and a direct stop of that generated local stack use the
same source-bound protocol, including its infrastructure. Publication preserves
the binding and refuses to overwrite an allocation owned by another stack.
An existing unbound allocation requires an explicit, evidence-checked migration;
a new controller never guesses ownership from the project's name. Old owners and
nodes lacking this protocol must be upgraded before confirmed remote stop is
available. A recipient must explicitly confirm `keepInfrastructure: true` in its
inventory/plan and receipt for an applications-only stop. A legacy recipient that
ignores the field is refused during preparation, before any node is stopped; the
controller never silently substitutes a full stop. This is an activation
prerequisite, not an automatic compatibility mode.

The legacy `project/deployed/app` namespace still supports one source stack per
project on a node. Concurrent source stacks on the same node need the canonical
allocation model in [G09](/docs/omnitron/release-readiness); collision is refused meanwhile.
Deployment destinations are durably recorded before remote work begins. Stop merges
that journal with the current declaration and retains removed destinations across
controller restarts; obligations are cleared only after every receipt and a matching
journal revision. Even a change from a remote to a local declaration does not erase
remote obligations. Deployments made before this journal existed require an audited
initial inventory; unavailable historical sources remain explicit gaps.
Docker inventory must be readable on every participating node; an unavailable
Docker daemon is not interpreted as an empty inventory. These boundaries keep the
full lifecycle gate open until allocation migration and fleet acceptance pass.

---

---
id: "omnitron/recipes"
title: "Recipes"
description: "Step-by-step solutions for common multi-process patterns."
---

# Recipes

Copy-and-adapt patterns for the operational shapes that
recur in production. Each recipe shows the topology, the
`defineSystem` shape that materialises it, and the gotchas
worth knowing.

Generic example names throughout: substitute what fits your
domain.

## 1. Stream-based fan-in (ingest → aggregate → serve)

**When to use:** real-time data flowing in from many external
sources (exchange websockets, IoT sensors, partner APIs);
needs aggregation before storage and serving.

### Topology

```mermaid
flowchart LR
  Ext1[Exchange A WS] -->|tick| C
  Ext2[Exchange B WS] -->|tick| C
  Ext3[Exchange C WS] -->|tick| C

  subgraph App["App: market"]
    C[collector<br/>process]
    SA[stream-aggregator<br/>process]
    O1[ohlcv-aggregator #1]
    O2[ohlcv-aggregator #2]
    H[http<br/>process]
  end

  C -- Redis stream XADD --> RS[(Redis Stream:<br/>raw.ticks)]
  RS -- XREADGROUP --> SA
  SA -- VWAP + write --> DB[(Postgres)]
  SA -- publish --> Cache[(Redis cache)]

  H -- @Cron triggers --> O1
  H -- @Cron triggers --> O2
  O1 -- candle compute --> DB
  O2 -- candle compute --> DB

  Browser -- REST/RPC --> H
  H -- read --> DB
  H -- read --> Cache
```

Four `IProcessEntry`s, each with a distinct job:

- **`collector`** — one process holds every external WS connection
  and publishes raw ticks to a Redis stream. Single instance
  (you don't want N pods each opening the same exchange WS).
- **`stream-aggregator`** — consumes the stream, computes VWAP /
  per-symbol roll-ups, writes to Postgres + cache. Single
  instance with a consumer group lets you scale horizontally
  later by adding more.
- **`ohlcv-aggregator`** — CPU-bound candle generation
  (5m / 1h / 1d). Pool of N workers with P2C load balancing.
- **`http`** — public read API + `@Cron` triggers that dispatch
  work to the ohlcv pool via `topology.access`.

### `defineSystem`

```typescript
export default defineSystem({
  name: 'market',
  version: '1.0.0',
  processes: [
    {
      name: 'http',
      module: './app.module.js',
      transports: { http: { port: 3003, host: '0.0.0.0', cors: true } },
      topology: { access: ['OhlcvAggregatorWorker'] },   // ← consume sibling
      hooks: { afterCreate: wireAuth },
    },
    {
      name: 'collector',
      module: './workers/collector.module.js',
      health: { enabled: true, interval: 15_000, timeout: 10_000 },
    },
    {
      name: 'stream-aggregator',
      module: './workers/stream-aggregator.module.js',
      health: { enabled: true, interval: 15_000, timeout: 5_000 },
    },
    {
      name: 'ohlcv-aggregator',
      module: './workers/ohlcv-aggregator.module.js',
      instances: 2,
      health: { enabled: true, interval: 30_000, timeout: 5_000 },
      topology: { expose: true },                        // ← exposed for siblings
    },
  ],
  shutdown: { priority: 20, timeout: 15_000, drainConnections: true },
});
```

Drains in reverse-dependency order: http stops accepting requests
first, then ohlcv finishes in-flight candles, then aggregators
drain the stream, then collector closes WS connections.

### Why Redis streams (not pub/sub)?

- **Durability** — XADD persists; subscribers can disconnect and
  resume from the last ack'd id.
- **Consumer groups** — multiple aggregator pods can split work
  with at-least-once delivery.
- **Replay** — debug by re-running an aggregator from an earlier
  stream id.

Pub/sub is fire-and-forget — fine for cache invalidation, wrong
for raw data you can't afford to lose.

### Gotchas

- **Single collector process.** If you scale collector to `instances: 2`,
  both will dial the same exchange — you'll either trip rate limits
  or get duplicate ticks.
- **Stream backlog.** If the aggregator falls behind, XADD eventually
  fails or the stream grows unboundedly. Monitor stream length
  (`XLEN`) and set a `MAXLEN ~` trim cap on the producer.
- **Consumer-group ID conflicts on rolling restart.** Use stable
  consumer IDs so rebalancing is clean. Omnitron injects
  `OMNITRON_APP_NAME` (plus `OMNITRON_PROJECT` / `OMNITRON_STACK`
  in stack mode) into every managed process — derive the consumer
  id from those rather than a random per-boot value. (There is no
  per-instance `OMNITRON_INSTANCE_ID`; pool instances aren't
  individually addressable via env, which is another reason to
  keep stream consumers single-instance here.)

## 2. CPU-bound worker pool with P2C balancing

**When to use:** request paths that need expensive CPU work
(image transforms, video transcoding, ZK proof generation,
ML inference, candle aggregation).

### Topology

```mermaid
flowchart LR
  HTTP[http process<br/>request thread]
  HTTP -- queryInterface<br/>'topology:Worker' --> Pool
  subgraph Pool["Worker pool"]
    W1[worker #1]
    W2[worker #2]
    W3[worker #3]
  end
  Pool -- power-of-two-choices<br/>pick least-busy of 2 random --> Selected
  Selected --> Compute[CPU work]
  Compute --> HTTP
```

### `defineSystem`

```typescript
processes: [
  {
    name: 'http',
    module: './app.module.js',
    transports: { http: { port: 3000 } },
    topology: { access: ['ImageTransformWorker'] },
  },
  {
    name: 'image-transform',
    module: './workers/image-transform.module.js',
    instances: 4,                          // 4 forks — one CPU core each
    health: { enabled: true, interval: 30_000, timeout: 5_000 },
    topology: { expose: true },            // exposed to siblings
    // Optional autoscaling — pool resizes between 4 and 8:
    scaling: { strategy: 'auto', maxInstances: 8, targetCPU: 70 },
  },
]
```

The orchestrator's load balancer uses **power-of-two-choices**:
pick 2 random workers, dispatch to the less loaded one. Cheap to
compute, near-optimal balance — beats round-robin under skewed
load.

### Calling the pool

```typescript
import { createToken } from '@omnitron-dev/titan/nexus';

const TRANSFORM_POOL_TOKEN = createToken<ImageTransformWorker>('topology:ImageTransformWorker');

@Injectable()
class ImageService {
  constructor(@Inject(TRANSFORM_POOL_TOKEN) private readonly pool: ImageTransformWorker) {}

  async transform(buffer: Buffer, opts: TransformOptions) {
    return this.pool.transform(buffer, opts);   // dispatches to least-busy
  }
}
```

### Gotchas

- **Workers must be stateless across calls.** Pool dispatch
  picks any worker — if a request stores state in worker memory
  for the next request, you'll be inconsistent.
- **Heavy boot per worker.** N forks means N copies of any
  in-process module — Sharp / Tensorflow / etc. cost RAM
  proportionally. Right-size `instances`.
- **`topology.access` is mandatory.** Without it, `queryInterface`
  on the worker service returns nothing — the http process
  silently falls back to in-process work and you wonder why
  the pool sits idle.

## 3. Scheduler-triggered work dispatched to a pool

**When to use:** periodic CPU work (nightly candle generation,
report rollups, cache prewarming).

### Topology

The scheduler runs in the http process (or any always-on
process); it dispatches work to a sibling pool via `topology`.

```typescript
// In the http process (which holds the scheduler):
@Service('market@1.0.0')
class MarketService {
  constructor(@Inject(OHLCV_POOL_TOKEN) private readonly pool: OhlcvWorker) {}

  @Cron('0 */5 * * * *')                            // every 5 minutes
  async rebuild5MinCandles() {
    const symbols = await this.getActiveSymbols();
    await Promise.all(symbols.map(s => this.pool.rebuildCandle(s, '5m')));
  }

  @Cron('0 0 * * * *')                              // hourly
  async rebuild1HourCandles() {
    /* ... */
  }
}
```

`@Cron` is imported from `@omnitron-dev/titan-scheduler` (a
separate package from the core `@omnitron-dev/titan` decorators).
The 6-field form shown here includes a leading seconds field;
the standard 5-field form works too.

Pattern: **always-on processes hold the schedule; pools execute
the work.** Keeps the http event loop responsive (it just
dispatches) and the pool isolated from request traffic.

### Gotcha — `instances > 1` on the scheduler host

If the http process runs as a pool itself (`instances: 4`), the
`@Cron` fires 4× per tick. Either:

- Run the scheduler in a dedicated single-instance process, or
- Use `titan-scheduler.persistence` with distributed lock so only
  one instance fires per tick.

## 4. WebSocket realtime app alongside HTTP RPC

**When to use:** any app that serves both request/response RPC
and live push (chat, notifications, dashboards).

### `defineSystem`

```typescript
processes: [{
  name: 'http',
  module: './app.module.js',
  critical: true,
  transports: {
    http: {
      port: 3005, host: '0.0.0.0', cors: true,
      requestTimeout:    120_000,
      keepAliveTimeout:   65_000,
      headersTimeout:     70_000,
    },
    websocket: {
      port: 3006, host: '0.0.0.0', path: '/ws',
      keepAlive: { interval: 30_000, timeout: 10_000 },
    },
  },
  hooks: { afterCreate: wireAuth },
}],
```

One process serves both transports; clients can use whichever
suits their session pattern. The gateway routes `/api/realtime/`
to `:3005` and `/ws/` to `:3006`.

### Gateway snippet

```nginx
upstream realtime_backend { server ${UPSTREAM_REALTIME_HOST}:${UPSTREAM_REALTIME_PORT}; }
upstream realtime_ws      { server ${UPSTREAM_REALTIME_WS_HOST}:${UPSTREAM_REALTIME_WS_PORT}; }

location /api/realtime/ {
  proxy_pass http://realtime_backend;
}

location /ws/ {
  proxy_pass http://realtime_ws;
  proxy_http_version 1.1;
  proxy_set_header Upgrade $http_upgrade;
  proxy_set_header Connection "upgrade";
  proxy_read_timeout 7d;
}
```

`proxy_read_timeout 7d` lets long-lived connections survive
gateway idle.

### Gotchas

- **Auth on WS** runs on the **upgrade request** only. Closing
  and reopening WS after token rotation is the safest pattern —
  don't try to re-auth a live socket.
- **Sticky session** when scaling realtime to multiple pods.
  Either use sticky LB or route by user-id hash.
- **Don't share `realtime` and `http` ports across pods.** WS
  needs to land on the pod that holds the user's socket.

## 5. Multi-step deposit / settlement pipeline

**When to use:** workflows that span multiple processes /
external systems (payment processing, supply-chain steps,
multi-stage ML pipelines).

### Topology

```mermaid
flowchart LR
  External[External daemon] -->|polled| Worker
  subgraph App["App: ledger"]
    HTTP[http process<br/>RPC + scheduler]
    Worker[deposit-worker<br/>process]
  end
  Worker -- ACID transaction --> DB[(Postgres)]
  Worker -- enqueue notification --> Q[(Redis queue)]
  Q --> NotifyWorker[notification consumer]
```

### `defineSystem`

```typescript
processes: [
  {
    name: 'http',
    module: './app.module.js',
    critical: true,
    transports: { http: { port: 3004 } },
    hooks: {
      afterCreate: async (app) => {
        await wireAuth(app);
        // Eager-load wallet providers so first request doesn't pay cold start:
        await app.container.resolveAsync(WALLET_PROVIDER_A);
        await app.container.resolveAsync(WALLET_PROVIDER_B);
        // Hot-reload runtime settings (commission rate, escrow days):
        const settings = await app.container.resolveAsync(RUNTIME_SETTINGS);
        await settings.loadAndApply();
      },
    },
  },
  {
    name: 'deposit-worker',
    module: './workers/deposit-worker.module.js',
    critical: false,
    startupTimeout: 120_000,         // wallet RPC warm-up takes time
    health: { enabled: true, interval: 30_000 },
  },
],
shutdown: { priority: 15, timeout: 10_000, drainConnections: true },
```

Worker is `critical: false` — its absence doesn't take down the
daemon. Failed deposits surface via metrics and retry on next
poll.

### Why a separate worker process?

- **Crash isolation.** A bad mempool entry kills the worker,
  not the http surface.
- **Independent observability.** Worker logs are separately
  filterable; metrics tagged per worker.
- **Restart freedom.** `omnitron restart deposit-worker` cycles
  the worker without taking the API down.

## 6. Object storage with image transforms

**When to use:** any service that serves user-uploaded media,
applies on-demand transforms (resize, format conversion,
watermarking).

### Topology

```mermaid
flowchart LR
  Browser -->|GET /render/image/...| HTTP
  subgraph App["App: media"]
    HTTP[http process<br/>3002]
    TW[transform-worker pool<br/>3 instances]
  end
  HTTP -- objectService.get --> S3[(S3-compatible)]
  HTTP -- topology dispatch --> TW
  TW -- Sharp pipeline --> CDN[(transform cache)]
  CDN --> Browser
```

### Custom route alongside RPC

```typescript
processes: [{
  name: 'http',
  module: './app.module.js',
  transports: {
    http: {
      port: 3002,
      requestTimeout: 300_000,             // big uploads
      maxRequestSize: '10gb',
    },
  },
  topology: { access: ['ImageTransformWorker'] },
  customRoutes: [{
    method: 'GET',
    pattern: '/render/image/*',
    handler: async (req: Request): Promise<Response | null> => {
      const url = new URL(req.url);
      if (!url.pathname.startsWith('/render/image/')) return null;

      const { bucketId, objectName } = parsePath(url.pathname);
      const opts = transformService.parseTransformOptions(url.searchParams);

      // SAME auth gate as the RPC path — no auth bypass.
      const bucket = await bucketService.get('default', bucketId);
      if (!bucket.public) {
        const token = extractBearer(req);
        if (!token) return new Response('Unauthorized', { status: 401 });
        const authCtx = await jwtService.verify(token);
        await bucketService.assertAccess(bucket, authCtx, 'read');
      }

      const out = await transformService.render(bucketId, objectName, opts);
      return new Response(out.stream, {
        headers: {
          'content-type':   out.mime,
          'cache-control':  'public, max-age=31536000, immutable',
          'etag':           out.etag,
        },
      });
    },
  }],
}]
```

### Gotchas

- **Custom routes can bypass RPC auth.** The example above
  explicitly calls `assertAccess('read')`. Don't omit this —
  history is full of "public preview" routes that leaked
  private data.
- **`maxRequestSize: '10gb'`** is one of the only places where
  the Node defaults are unrealistic. Set it explicitly.
- **Stream the response.** Don't buffer transformed images into
  memory; pipe directly to `Response.body`.

## 7. Per-tenant data isolation via RLS

**When to use:** multi-tenant SaaS where one Postgres instance
holds data for many tenants and the application enforces
isolation.

### Pattern

```typescript
import { defineRLSSchema, filter } from '@kysera/rls';   // re-exported by @omnitron-dev/titan-database

// 1. Define the policy — policies are factory functions (filter / allow / validate),
//    and a filter returns a column-value object, not a raw sql`` fragment:
const schema = defineRLSSchema({
  users: {
    policies: [
      filter('read', ctx => ({ tenant_id: ctx.auth.tenantId })),
    ],
  },
});

// 2. Wire the RLS context wrapper
auth: {
  jwt: { enabled: true, tokenCacheTtl: 60_000 },
  invocationWrapper: createRlsInvocationWrapper(),   // sets tenantId in AsyncLocalStorage
},

// 3. Repositories pick it up automatically. Inject the Kysely connection
//    directly with @InjectConnection() (DatabaseManager has no `.connection`
//    getter — it exposes `await db.getConnection()`):
@Injectable()
class UserRepository {
  constructor(@InjectConnection() private db: Kysely<Database>) {}

  async findAll() {
    return this.db.selectFrom('users').selectAll().execute();
    // Kysera RLS plugin auto-adds `WHERE tenant_id = $current_tenant`
  }
}
```

Every backend in your fan-out uses the **same** invocation
wrapper — so the policy enforces uniformly whether the call
comes via http, websocket, or service-to-service.

### Bypassing RLS for admin operations

```typescript
@Public({ auth: { roles: ['admin'] } })
@BypassRLS()                                // explicit per-method opt-out
async listAllUsersAcrossTenants() {
  return this.db.selectFrom('users').selectAll().execute();
}
```

`@BypassRLS()` is loud by design — it appears in code review
diffs. Search for it during audit.

## 8. Hot-reloadable maintenance mode

**When to use:** scheduled downtime, emergency response, or
graceful degradation.

```mermaid
flowchart LR
  Op["Operator: set maintenance flag"] --> R[(Redis)]
  Request["Incoming request"] --> LB["Gateway Lua check"]
  LB -->|"Read flag, cached for 1 second"| R
  LB --> Flag{"Maintenance enabled?"}
  Flag -->|Yes| Page["Serve maintenance.html<br/>HTTP 503"]
  Flag -->|No| Backend["Pass through to backends"]
```

The gateway's Lua block (see [Best Practices](/docs/omnitron/best-practices#reverse-proxy-with-maintenance-mode-flag-in-redis))
reads a Redis flag on every request, cached for 1 s. Flip the
flag from any operator surface — CLI, webapp, or directly via
`redis-cli`.

```bash
# Enable maintenance
redis-cli SET platform:maintenance 1
# Disable
redis-cli DEL platform:maintenance
```

No daemon restart, no LB reconfiguration, instant flip.

## 9. Onion-routed operator console

**When to use:** sensitive operator surfaces (admin webapp,
incident dashboards) that should not have public DNS / TLS.

```typescript
infrastructure: {
  services: {
    gateway: { preset: 'openresty', config: { configDir: './infra/nginx' } },
    tor: {
      preset: 'tor',
      config: {
        hiddenServices: [
          { name: 'webapp', virtualPort: 80, target: 'host.docker.internal:9800' },
          { name: 'admin',  virtualPort: 80, target: 'host.docker.internal:8080' },
        ],
      },
    },
  },
},
```

`omnitron tor` prints the `.onion` addresses after the tor
container bootstraps (typically 30–90 s). The keys live in a
persistent volume; same address survives restart.

Defence in depth — not a substitute for RBAC, but it removes
the operator surface from public scanning entirely.

## 10. Cross-app service mesh via topology

**When to use:** apps that need to call services in other
apps directly, not just via shared infrastructure (database,
queue).

```typescript
// app: ledger
processes: [{
  name: 'http',
  module: './app.module.js',
  topology: { access: ['PriceFeed'] },        // ← from a different app
  // ...
}]

// app: market — exposes PriceFeed
processes: [{
  name: 'http',
  module: './app.module.js',
  topology: { expose: true },                  // ledger can now resolve it
}]
```

Behind the scenes Omnitron registers `PriceFeed` on the daemon's
Netron bus when `market` boots. Ledger's `http` process gets a
load-balanced proxy injected via DI under
`createToken('topology:PriceFeed')`.

No URLs, no service discovery code, no DNS — purely declarative.

## 11. Per-app database with shared infrastructure

**When to use:** apps that need data isolation without infra
sprawl.

```jsonc
// apps/ledger/config/default.json
{
  "omnitron": {
    "database": true,        // ← shared Postgres, dedicated database
    "redis":    true,
    "services": { "notifications": true }
  }
}

// apps/media/config/default.json
{
  "omnitron": {
    "database": true,
    "redis":    true,
    "s3":       true
  }
}
```

Omnitron sees both `database: true` declarations, provisions
one Postgres container, creates a `ledger` database and a `media`
database inside it, injects per-app `DATABASE_URL` with the
right database name.

Each app reads its `DATABASE_URL` from env and has no idea the
other exists. Backups can target one database without touching
the other.

## 12. Dev hot-reload with shared workspace deps

**When to use:** a monorepo whose applications and shared libraries are developed
from source together.

Use source entry points, with paths relative to the project configuration:

```typescript
// omnitron.config.ts
export default {
  project: 'media-platform',
  apps: [{ name: 'media', bootstrap: './apps/media/src/bootstrap.ts' }],
};
```

In `omnitron.stacks.json`:

```json
{ "dev": { "type": "local", "apps": "all", "watch": true } }
```

```bash
omnitron stack diagnose media-platform dev
omnitron stack start media-platform dev --working-tree
```

The bootstrap compiler and application bundler must use the same package
resolution policy. TypeScript workspace entry points belong in the compiled
import graph, including their relative imports and decorators. Compiled packages
and Node built-ins remain external. Esbuild then observes changes to the imported
workspace sources and rebuilds the affected application; no second library
watcher is necessary for source-exporting packages.

A shared library keeps ownership of its dependencies. If it imports a compiled
package declared only in that library, the runtime build must retain that exact
resolution and version. Applications should not repeat those dependencies merely
to make a bundled development entry load. Generated local builds move with the
workspace installation; release archives use their independently verified
dependency closure.

Bootstrap applications use this import graph. `watch.directory`, `watch.include`
and `watch.ignore` configure the filesystem watcher for command-based
applications; adding directories there does not extend a bootstrap application's
import graph. A package exporting only `dist` is a compiled dependency: editing
its unimported `src` cannot automatically rebuild a consumer.

Before accepting a development setup, change a shared library in an isolated
fixture and verify both a fresh definition read and an application rebuild. A
successful release build alone does not verify this source path. An older
resident owner can still use an incompatible definition compiler after a backend
upgrade. Inspect `omnitron backend status` and qualify that owner's source path;
replace it only through a reviewed transition that preserves the other workloads.

## 13. Two-database deploy per stack

**When to use:** running dev + staging on the same host without
collisions.

```typescript
stacks: {
  dev: {
    type: 'local',
    settings: {
      redisDbOffset:  0,
      portOffsets:    { postgres: 0,  redis: 0,  minio: 0  },
      containerPrefix: 'platform-dev',
    },
  },
  staging: {
    type: 'local',
    settings: {
      redisDbOffset:  5,
      portOffsets:    { postgres: 10, redis: 10, minio: 10 },
      containerPrefix: 'platform-staging',
    },
  },
}
```

Both stacks run side-by-side. `omnitron stack start <project> dev`
runs the dev stack; same for staging. They share no state.

## 14. Webhook receiver alongside RPC

**When to use:** integrating with third-party services that push
events (payment providers, git platforms, SaaS notifications).

```typescript
processes: [{
  name: 'http',
  module: './app.module.js',
  customRoutes: [{
    method: 'POST',
    pattern: '/webhooks/payments',
    handler: async (req: Request): Promise<Response> => {
      // 1. Verify signature
      const body = await req.text();
      const sig  = req.headers.get('x-signature');
      if (!verifySignature(body, sig)) return new Response('unauthorized', { status: 401 });

      // 2. Idempotency
      const id = JSON.parse(body).id;
      if (await redis.exists(`webhook:processed:${id}`)) {
        return new Response('already processed', { status: 200 });
      }
      await redis.setex(`webhook:processed:${id}`, 86_400, '1');

      // 3. Hand off to a worker (no inline DB writes here)
      await queueClient.publish('payment.event', body);
      return new Response('ok', { status: 200 });
    },
  }],
}]
```

Pattern: **acknowledge fast, work async.** Third-party webhook
senders timeout aggressively (often 5 s); responding `200 OK`
within milliseconds keeps you off their retry list.

## See also

- [Best practices](/docs/omnitron/best-practices) — patterns that appear
  across all recipes
- [Configuration](/docs/omnitron/configuration) — every option used here
- [Infrastructure](/docs/omnitron/infrastructure) — declarative provisioning
- [Orchestrator](/docs/omnitron/orchestrator) — how the orchestrator runs all this

---

---
id: "omnitron/release-readiness"
title: "Release readiness"
description: "Ordered implementation and acceptance plan for the control plane, execution, security, Console and package integrations."
---

# Release readiness

This page defines implementation and acceptance gates. It is not a live release
status report. Each gate needs code, adversarial tests, documentation, packaged
verification and an activation record. Track **implemented**, **verified**, and
**activated** separately: a passing source test does not establish that the running
fleet has received a fix. Use the candidate's qualification receipt and each
installation's status for those decisions.

The scope includes the known release blockers and the previously identified product
limitations. Do not silently defer an item by renaming it a roadmap feature. If an
external dependency prevents completion, record the exact dependency and keep the
capability unavailable or explicitly limited until its acceptance criteria pass.

## Final npm qualification

Freeze one source commit for publication. The release recipe prepares its npm
archives and qualifies that same commit. Publication requires all three checks:

1. Remote release admission uses the verified resident owner protocol and the
   exact Node target that installation will use. A frozen application owns its
   SDK; equality with the owner's Git commit is not a compatibility contract.
   Missing identity, unsupported owner protocol, an incomplete closure or a
   different target must refuse delivery. See [release admission](/docs/omnitron/releases).
2. Compiler and linter checks cover production code and all test fixtures.
   Omnitron's server tests inherit its strict backend options; Console tests use
   its browser configuration. Both belong to normal workspace typecheck.
3. Qualify the same commit through independent clean builds, all packed package
   entry points and consumers, tests with isolated services, documentation,
   and native backend transition/rollback on the fleet's Node versions. The native
   qualification also delivers an application built with the candidate SDK through
   the previous execution owner, verifies its private dependency graph, reuses the
   verified installation and restores the retained application. A matching
   controller version alone is not proof of application compatibility.
   Record CLI and Console acceptance separately from isolated fixtures.

Do not reuse a passing result from an earlier commit as proof for the selected
one. Preparation and publication do not activate the fleet. Any required live
pause needs an explicit transition, verification and rollback plan first.

For a maintainer SDK export, run the same reproducibility gate from the source repository:

```sh
pnpm check:reproducibility --revision <full-commit> --out /absolute/new/qualification
```

It runs the ordinary frozen install and complete build in two independent checkouts, then compares both the sealed runtime and every normalized public npm archive. A successful output contains `runtime/`, `sdk/framework-manifest.json`, the SDK archives and `QUALIFICATION.json`. Treat the qualification receipt as the completion marker; a partial directory is not an accepted export. Without `--revision`, the selected commit is `HEAD`.

The [maintainer release workflow](/docs/community/contributing#release-process)
retains the clean preparation build after package-consumer acceptance and uses it
as the first reproducibility build. Qualification verifies its identity and bytes,
then makes the second independent build. This keeps two clean builds across local
preparation and comparison; the standalone command above makes both from scratch.
CI and native qualification run concurrently with the comparison and retain their
own stage evidence. Publication checks the selected receipts and sealed archives
without rerunning builds or tests.

Export from these verified archives. A clean Git status does not make an existing build directory canonical: Console publication intentionally retains older hashed assets so open tabs can continue loading their chunks. Those retained generations belong to the served directory, not to a new SDK archive. Do not replace sealed preparation archives with a later pack of a working checkout.

## Invariants

- One project owns multiple environments/stacks; an allocation identifies its
  project, stack, workload, revision and node incarnation. Display names and network
  addresses do not establish ownership.
- Kysera owns database primitives and migration machinery; Titan provides reusable
  lifecycle, policy, contracts and integration; Omnitron owns infrastructure intent
  and operations. Xec owns execution and SSH; Prism owns reusable interaction.
- CLI, Console and agents use the same server contracts, permissions, operation
  journal and diagnostic reasons. Clients never decide whether an operation is safe.
- Every mutation is admitted against current authority and exact resources. A timeout
  means an unknown outcome until reconciled, not success and not permission to replay.
- Running applications, pinned experiments, data and historical evidence survive
  development and backend upgrades. Process-owner replacement requires a reviewed
  pause and rollback plan. Source builds use isolated outputs.
- Optional features remain independently importable. A basic daemon does not require
  an agent SDK, browser editor, cloud account or consensus service.

## Ordered delivery gates

| Gate                                  | Implementation and architectural boundary                                                                                                                                                                                                                                                                                                                                                                                                                        | Acceptance evidence                                                                                                                                                                                                                                                                                                                                                                                | Depends on                           |
| ------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------ |
| G00 — Baseline                        | Inventory source revisions, active artifacts, owners, application PIDs/start times, database schema, replication receipts and pinned experiment environment. Establish a repeatable isolated test environment and fault fixtures.                                                                                                                                                                                                                                | Before/after inventory, immutable rollback artifacts, no live dependency replacement during development.                                                                                                                                                                                                                                                                                           | —                                    |
| G01 — Cluster authority               | Remove anonymous election authority; bind machine identity to permitted operations. Replace volatile leadership assumptions with durable coordination, explicit failure semantics and fencing. Reject unsafe network publication with seeded administrator credentials. State exactly which database/coordination availability the cluster requires.                                                                                                             | Forged identity, replay, stale epoch, restart, concurrent controllers, partition, lost database and delayed response tests. No leadership or effect is admitted on uncertain authority.                                                                                                                                                                                                            | G00                                  |
| G02 — Stack lifecycle                 | Node-confirmed stop with exact project/stack/allocation ownership, serialization and durable operation identity. Preserve remote observation and infrastructure on partial failure. Handle retries, lost replies, unreachable nodes and legacy deployment identities without stopping another stack.                                                                                                                                                             | Multi-node tests with one unreachable node, stop failure, stale incarnation, controller restart and repeated request. CLI/Console agree on stopped, partial and unknown outcomes.                                                                                                                                                                                                                  | G01                                  |
| G03 — Xec SSH                         | Keep `ssh2` as the reviewed protocol implementation inside Xec. Applications and Omnitron use Xec, without independent SSH clients or transport policies. Audit dependency versions and upstream defects; strengthen host verification, streaming, PTY/resize, SFTP, forwarding, cancellation, deadlines and lifecycle in the owning layer. Public Xec types must install cleanly. Do not replace the SSH protocol implementation merely to remove a dependency. | Independent OpenSSH interoperability fixtures; password/key/agent authentication; host-key mismatch; Unicode/binary transfer; parallel channels; backpressure; interrupted transfer; tunnel disposal; no leaked processes, descriptors or secrets. Dependency-boundary checks distinguish direct imports from transitive lock entries. Reproduce relevant upstream reports and record mitigations. | G00; G02 before fleet activation     |
| G04 — Shared access policy            | Complete resource-scoped grants, custom roles, explicit deny, delegation ceilings, session revocation and account suspension in the shared policy layer. Separate human, service and agent identities; privileged machine credentials cannot become user delegation. Design MFA/SSO as optional identity adapters with tested enrollment and recovery.                                                                                                           | Same allow/deny matrix through HTTP, Netron, CLI and Console; cross-project isolation; live revocation; privilege escalation and stale-session tests; application integration tests.                                                                                                                                                                                                               | G01                                  |
| G05 — Release transaction             | Discover all publishable workspaces; validate exports and dependency ordering. Make dry-run read-only, prerelease tags correct, git changes scoped, tag push exact, and partial publication resumable without overwriting published versions. Repair missing publish entry points.                                                                                                                                                                               | Sandbox registry and git remotes covering interrupted publish, existing version, divergent tag, dirty tree, rerun and prerelease; package inventory has no omissions. Publication remains a separate operator action.                                                                                                                                                                              | G00                                  |
| G06 — Registry integration            | Release verified Xec/Kysera artifacts, consume registry versions in Omnitron and application manifests, and remove release-time sibling paths/vendor overrides. Preserve intentional development workspaces as an explicit mode. Keep migrations accessible through Omnitron's facade.                                                                                                                                                                           | Clean install/build/test from tarballs with sibling repositories unavailable; manifests and locks agree; runtime closure and migration checks for every application.                                                                                                                                                                                                                               | G03, G04, G05                        |
| G07 — Required quality gates          | Repair lint/type errors, obsolete fixtures and test commands that mask failures. Require lint, compiler, meaningful unit/integration/security tests, package exports, documentation and clean-install checks in CI. Isolate service fixtures and preserve the original failure on cleanup.                                                                                                                                                                       | Red tests fail CI; unavailable required fixtures fail clearly; supported Node/platform matrix passes; website builds with link/content checks.                                                                                                                                                                                                                                                     | G01–G06; extend for every later gate |
| G08 — Durable diagnostics             | Complete log/metric/audit acknowledgements, bounded queues, failure signaling and query budgets. Preserve exact timestamps/multiline boundaries, deduplication identities and explicit historical gaps. Add resumable cursors and a Titan notification adapter with durable delivery, retries and dead-letter visibility.                                                                                                                                        | Crash/timeout/replay/load tests, source-to-database reconciliation, bounded memory/storage behavior, identical diagnostic codes in CLI/Console. Sensitive effects have durable audit evidence or are refused.                                                                                                                                                                                      | G01, G02, G04                        |
| G09 — Canonical topology              | Complete environment, allocation, resource binding and route models, including Docker/bare-metal/multi-node and isolated cross-project service contracts. Extend the existing topology editor and read model; do not introduce a parallel project registry.                                                                                                                                                                                                      | Round-trip portable configuration; unambiguous ownership; dependency edges from declared and observed evidence; stale/missing evidence distinguished; accessible editor at narrow and wide sizes.                                                                                                                                                                                                  | G02, G04, G08                        |
| G10 — Durable operations and recovery | Reuse one persisted plan/admission/execution/verification lifecycle for deployment, migration, replacement and scaling. Add node-incarnation fencing, capacity reservations, failure domains, reserve pools, readiness budgets and compensation. Stateful recovery requires a provider-specific data and fencing contract.                                                                                                                                       | Controller crash at every transition; duplicate delivery; node loss; exhausted capacity; stale owner; safe stateless replacement; provider-specific stateful recovery and rollback drills. No blind replay or automatic data failover.                                                                                                                                                             | G01, G02, G08, G09                   |
| G11 — Xec automation                  | Store immutable executable revisions and inputs, surface validation/progress/output, and add durable scheduling/webhook admission. Separate restartable work from effects requiring reconciliation. Untrusted execution uses an actual isolation boundary with resource and secret budgets.                                                                                                                                                                      | CLI/Console parity; artifact tamper rejection; cancel/restart behavior; trigger deduplication; sandbox escape boundaries; bounded output and resource consumption.                                                                                                                                                                                                                                 | G03, G04, G08, G10                   |
| G12 — Agent adapters                  | Optional provider SDK adapters use the same delegated identities and reviewed operation contracts. Support analysis of logs, metrics, topology and alerts before privileged execution. Resolve subscription/API authentication from supported provider contracts, without inheriting administrator tokens.                                                                                                                                                       | Per-tool, resource, time and budget enforcement on the server; revoked grants stop subsequent effects; exact-plan approval; reproducible audit; no arbitrary terminal bypass.                                                                                                                                                                                                                      | G04, G08, G10, G11                   |
| G13 — Files and terminal              | Preserve the dual/multi-panel model while adding efficient Netron transfer streams, resumable node-to-node copies, conflict/version checks, bounded previews and safe editing. Complete SSH lifecycle feedback and explicitly authorized session recovery/reattachment.                                                                                                                                                                                          | Real-node copy/edit with permission changes, cancellation, reconnect, partial transfer and concurrent edit; path/symlink escape tests; session ownership and expiry; keyboard and mobile-size verification.                                                                                                                                                                                        | G03, G04, G08                        |
| G14 — Console product consistency     | Apply one Prism interaction model to navigation, tables, filters, menus, dialogs, logs, topology, metrics, files and SSH. Immediate acknowledgement, operation progress, retry/cancel and actionable errors share backend state. Scope keyboard commands without trapping focus; expose discoverable shortcuts. Reduce initial bundles by capability boundaries.                                                                                                 | Browser flows for administrator/operator/viewer/custom scope, keyboard-only use, focus restoration, screen-reader labels, narrow/wide viewports, reconnect and slow/failing operations; measured bundle and rendering budgets.                                                                                                                                                                     | G02, G04, G08–G13                    |
| G15 — Coordinated acceptance          | Build sealed candidates, verify migration expansion and rollback, activate backend without changing application owners where possible, then validate applications and public interfaces. Publish complete package/API/operations docs and release notes with remaining boundaries stated exactly.                                                                                                                                                                | Full clean-install suite, website build, role/SSH/files/deploy checks through real CLI and Console, stable app identities, replication and receipt reconciliation, protected experiment artifacts unchanged.                                                                                                                                                                                       | G00–G14                              |

G05 can be prepared while no active runtime changes are pending. G07 grows with each
implemented gate; it is not postponed to the end. Progress proceeds through small,
reviewable commits. Package versions become release candidates only after their
runtime, public types and packed artifacts pass the same contracts. SDK compatibility
also requires an application upgrade and rollback through the same resident owner,
including a development installation whose bootstrap path stays unchanged. Record
the owner capability `applicationRuntimeResolution: 1` separately from the backend
version. Fresh-owner startup and fresh-shell imports do not qualify that transition.

## Activation and rollback contract

Each activation record names the current and candidate digests, affected processes,
configuration/schema changes, exact commands, readiness deadline, acceptance probes,
rollback trigger and rollback commands. Take backups before the switch and verify the
rollback artifact before asking for a pause. Do not treat a backend-only handoff as
permission to replace an execution owner or restart applications.

Compatible additive schema changes stay in place during binary rollback; destructive
schema rollback requires a separate data recovery decision. Verify the supported old
and new binaries against the expanded schema. For remote lifecycle changes, keep
observation and recovery available even if a node cannot confirm an effect.

The release record must distinguish failures, waiting for domain readiness and
unavailable evidence. It must not declare success from a green process indicator,
hide a warning by weakening its threshold, or infer that an unreachable process has
stopped. Existing experiment identities and historical gaps remain intact.

### Roll out a group of changes

Select one clean source commit for the batch and wait for its CI run. Build for
each target OS, architecture and Node runtime. Record the current backend,
process owner and Console revision separately, along with application identity
and health. A backend handoff can advance the controller while the existing
owner continues running the applications.

Update one test node first. Before moving to the next node, check authenticated
RPC, advancing replication acknowledgements, delivery of a fresh log record and
application health against the baseline. On a controller that serves Console,
also verify the published asset version and exercise the affected UI. If a new
feature requires a different owner or a Console restart, document its affected
processes, stop criteria, rollback artifact and recovery budget before opening
that window. Record the last accepted revision for each component and node.

## Documentation contract

Update the owning layer's documentation together with code: Xec for execution,
Kysera for database primitives, Titan for reusable integration, Omnitron for
operations and Console, and application repositories for domain-specific behavior.
Provide a short working path first, then configuration, failure/recovery semantics,
permissions and examples. Public framework documentation uses generic sample
projects; private deployment details belong in operator evidence outside the package.

The detailed architecture remains in [Resilient control plane](/docs/omnitron/resilient-control-plane).
This plan supplies delivery order and acceptance gates, not a competing architecture.

## Current implementation evidence

CI compares release artifacts from two independent checkouts. Omnitron 0.4.9
uses registry packages from Xec 0.12.0 and Kysera 0.10.2. The committed lockfile
records their resolved versions and integrity. Qualification checks the actual
archives and installed dependency graph; an earlier release's receipt does not
qualify a changed candidate. Frozen installs resolve the packages used by each
workspace.
Run `pnpm audit --audit-level low` to check runtime and development dependencies,
and `pnpm check:packages` to verify every public export inside the packed archives.
CI and release preparation additionally install the actual archives into an independent
npm consumer and audit its production dependencies: workspace overrides do not protect
a published consumer. KB publishes its tested extraction dependency tree using standard
npm bundled dependencies. Framework admission checks every included package's identity,
files and declared dependency edges, both in the npm installation and after the portable
application graph is materialized. Undeclared bundles, altered files, missing dependencies
and archive links remain refusals.
Each target still needs its own runtime identity, launch evidence and rollout record.

G09 and G14 include a Chromium check of the production topology component and its
routing worker. Run `pnpm --filter @omnitron/console test:browser` after installing
Playwright Chromium. The fixture exercises shared services at 320, 768 and 1440 CSS
pixels. All declared links must render, panel and inspector bounds must fit, and
positions changed with the pointer or keyboard must survive reload. On a running
installation, also compare the number of rendered links with the inventory and open
a connection to inspect its evidence. See the [topology controls](/docs/omnitron/console#topology).

For the [System view](/docs/omnitron/console#system), read a fresh snapshot on each reporting
node, then compare its hostname and daemon PID with the same target selected through
the controller in CLI and Console. CI checks that a machine identity can report its
own host and cannot use the user-facing method to relay a request to a third node.

The daemon's HTTP acceptance tests exercise public liveness and readiness, authentication errors for the five detailed Health RPCs, and human and machine read access. The replication tests use a real TCP peer exposing an unprotected outbox and verify that the controller rejects it. Machine-only acknowledgement and administrative reconciliation have separate tests. WAL recovery tests include lost acknowledgements, partial replies, disk errors and capacity eviction.

G05 uses Xec's release operations for workspace discovery, immutable archive
preparation, tag verification and registry reconciliation. CI rehearses an
interrupted publish, a resume, a conflicting digest, an RC channel and an
unrelated tag against isolated Git and npm endpoints. See the
[maintainer release procedure](/docs/community/contributing#release-process).

G01 and G02 implement signed recipient admission for managed stack stops and recovery of retained application releases. PostgreSQL
identifies the current leader and SQLite retains each recipient's highest epoch.
The takeover test transfers leadership after the application step and verifies that
the former controller cannot retire infrastructure, even after recreating the
recipient. Node protocol methods require a machine identity. Leadership transfer
requires an administrator. CLI and Console show the node, epoch and receipt outcome.
Recovery acceptance transfers a real PostgreSQL lease between consecutive application launches and checks that the old controller cannot dispatch the next one. Owner tests exercise real processes and release evidence. Fleet activation and live end-to-end acceptance are separate gates. Earlier allocations
require explicit ownership evidence. Provisioning, routing and stateful recovery
still need operation-specific fencing.

G03 keeps `ssh2` inside Xec. Direct use outside Xec and protocol implementation
dependencies are different architectural concerns. Tunnel target isolation, pooled
connection ownership and acknowledged closure are implemented and verified in Xec.
Xec 0.12 exposes public SSH/SFTP contracts, custom-agent bridging, finite request deadlines, shared connection-failure observation, bounded
pending requests, binary/UTF-8 preservation, explicit unknown command/write outcomes,
and complete high-level upload/download directory operations. Incomplete transfers
reject, and a lost reply never authorizes command replay. A clean consumer check
compiles without the underlying transport or its type package and runs in Xec CI.
Reverse-listener allocation now has a deadline and cancellation, and invalidates
its dedicated connection when acknowledgement is lost. Shared commands remain
isolated. The connection budget includes in-progress handshakes, busy connections
cannot be evicted, and unpooled connections close when their last operation ends.
Protocol keepalives replace remote keepalive commands; adapter disposal is terminal.
Four upstream transport defects were reproduced in isolation: zero-size packets,
receive-window overflow, abandoned buffered-write callbacks and a shared ChaCha
nonce during nested writes. A compatible patch and an offline npm-artifact gate
cover these cases. The published Xec package carries the patched transport,
its license and checksums; direct packing of an unprepared workspace is rejected.
SFTP close acknowledgement, cleanup under backpressure and lost high-level read
callbacks are also covered by the installed-package gate, including 25 consecutive
server-initiated closures without leaking channels. Both npm and pnpm installations
preserve the reviewed patch. High-level SSH targets inherit the same Xec connection
contract as `NodeSSH`: agents, host pins, algorithms and cancellation are available
without a parallel transport API. Connection identities include the effective trust
and authentication policy; rotated key files cannot reuse the previous identity.
Timeouts retire repeated failures after active users finish, preserving other commands.
OpenSSH authentication tests exercise agent signatures, encrypted key files and
rejection of a mismatching `known_hosts` entry without password fallback.
Installed-package tests also exercise malformed packets, channel cleanup under
backpressure and simultaneous writes in the published transport. Before promotion,
complete the remaining adversarial SSH cases and transport security review. After
activation, record the running version and repeat the role, host-key and session
checks through the CLI and Console on the target nodes.

---

---
id: "omnitron/releases"
title: "Releases and verification"
description: "Build from pinned project and runtime commits, verify artifacts, attest on a stack, and retain a recovery path."
---

# Releases and verification

`omnitron release` separates building an artifact from deploying it. A release
records the **project commit** and the **Omnitron commit** used to build its
applications. The build runs in clean clones and stores its gates, artifacts
and manifest.

## Local SDK development and registry deployments

Commit exact platform versions in your pnpm catalog and the resulting registry
lockfile. With a local framework checkout, `omnitron sdk init` prepares a small
pnpm hook for the consumer repository. Commit that hook and its updated lockfile.
Stop working-tree applications before switching their installed dependencies.
From the consumer workspace:

```sh
omnitron sdk dev /work/framework --stack dev
```

The command builds the selected packages and verifies their physical resolution
across workspace importers. A private profile under `.omnitron/dev-sdk` supplies
the local overrides and pnpm lock to normal installs, test commands and IDEs.
Keep the command running while editing the platform. It watches source changes,
waits for successful builds, then refreshes affected online applications whose
watch policy is enabled in the named local stack. A Vite frontend can serve the
selected packages' built modules through its normal HMR path. Omit `--stack`
to own application reload yourself. Use `--no-watch` to install the profile
separately. Resume with `omnitron sdk dev --stack dev`. The saved selection persists,
but the watcher is a foreground process: it stops when that command exits.

For an intentionally stopped working-tree stack, first confirm that its last
stop retained infrastructure. Obtain a fresh start review on that stopped
state, then give its digest to the SDK watcher:

```sh
omnitron --json stack start example dev --working-tree --infrastructure preserve --dry-run
omnitron sdk dev /work/framework --stack dev --start-stopped --review <digest>
```

The watcher builds the SDK and requests one start for that stop. The first
confirmed application generations establish the SDK receipts. If the reply is
lost, inspect the reported state before trying again. Applications that manage
their own HMR (`watch: false`) are reported separately. Add `--no-watch` for
a single reviewed start.

`omnitron --json sdk status` reports the active origin and verified import count.
Add `--stack dev` to check existing receipts against the owner's current
generations.
After stopping source applications, `omnitron sdk registry` restores the normal
frozen installation. A release can be built while a developer uses a local SDK:
the builder installs the lockfile from the selected Git commit in a clean clone.
To enforce this input policy, set `omnitron.release.frameworkInput` to
`"registry"` in the workspace package.json. For an external deployment publish
changed platform packages first, update their catalog and lockfile together in
a registry checkout, then build and deploy the resulting project commit.

## Rebuilding an artifact

A targeted build resolves the same stack names as the daemon. It reads
`omnitron.stacks.json` from the registered directory in the clean checkout,
then applies declarations from the ecosystem file with precedence on name
collisions. Artifact selection, database migration programs and gateway
statics use that resolved map. An invalid stack file stops the build.

Register the directory containing the ecosystem configuration. The builder
clones the whole repository at the selected commit and reads this configuration
from its registered path. Starting there, it finds the nearest committed
`pnpm-lock.yaml` up to the repository root. That lockfile's directory owns
dependency installation, workspace builds and gates. A nested configuration can
share the monorepo lockfile. The manifest records its relative path in
`project.directory`.

Declare `@omnitron-dev/titan` as a runtime dependency of every bootstrap
application. During packaging, the builder resolves its public entry point
inside the installed application graph. A Titan dependency provided only by a
parent workspace fails this check before an archive is written.

Build from clean project and framework commits with the checked-in lockfiles and
their pinned package manager. The build timestamp in a clean framework package is
the commit time; the release record records when the build actually ran. A source
export without Git metadata can supply `SOURCE_DATE_EPOCH` in whole Unix seconds
for the build stamp. CI compares the SHA-256 of two independently compiled and
packed archives and verifies that changing file contents or executable permission
changes the hash.

Equal archive hashes identify delivered bytes. They do not by themselves identify
a running process, or prove that native dependencies built for macOS will run on
Linux. Use the launch receipt and the destination's runtime when checking a stack.

### Qualify a separate working copy

Install dependencies in each checkout using the Node target and the pnpm version
pinned by the project. Keep `verifyDepsBeforeRun: error`. A symlink to another
checkout's entire `node_modules` also reuses that checkout's workspace state and
links; matching lockfiles alone do not make it this checkout's installation.
Share the package manager's [content-addressable store](https://pnpm.io/symlinked-node-modules-structure)
instead. On a fresh checkout, run the project's declared checks normally:

```bash
pnpm install --frozen-lockfile
pnpm run typecheck
pnpm run test
pnpm run build
```

Use `--offline` for installation when the store already contains every required
package. A missing package is a failed prerequisite. Keep the lockfile, archive
integrity checks and allowed build scripts intact. Running an underlying test
binary directly can help diagnose a failure, but does not prove that the normal
package-manager entry point passed. Native dependencies must match the selected
Node, operating system and architecture.

Working-copy checks cover the files present during that run. For a release,
commit the intended source and use the existing builder. It creates fresh clones,
installs from the frozen lockfile and records the declared gates and artifacts.
For vendored framework archives, supply their full source revision and a local
repository containing it; the builder reproduces and verifies their bytes.
Reproduction executes the framework's declared `pnpm run build`, including any
auxiliary generation/declaration steps, with Turbo cache reads disabled. It does
not substitute a partial list of tasks. The result must match every committed
archive's SHA-512 before application installation or gates run.
A local Git repository is a supported source and does not require a push.

Current SDK package builds resolve the compiler from the workspace lockfile. For
a historical SDK, keep the build tools required by that revision on the isolated
builder’s `PATH`. If its build scripts call `npx`, include the npm CLI bundled
with the pinned Node.js distribution. Run the package’s declared build in the
same image, with the same `PATH` and network policy, before qualifying the full
release. The receipt for the full build identifies the resulting artifacts.

An isolated controller directory keeps qualification registrations and output out
of a running controller's registry. Create a fresh directory for the attempt,
leave its daemon stopped, and use the same directory for both commands:

```bash
OMNITRON_HOME=/work/qualification-instance omnitron project add example /work/example
OMNITRON_HOME=/work/qualification-instance omnitron release build example \
  --project-commit <full-project-sha> \
  --omni-commit <full-framework-sha> \
  --framework-repository /work/framework
```

Register the directory containing `omnitron.config.ts`, which can be nested in a
monorepo. Declare gates through `omnitron.release.gates` in the workspace's
`package.json` or its existing `scripts/gates.mjs`, never both. With neither,
the manifest records `not-run`; an artifact is not evidence that tests passed.
`OMNITRON_HOME` isolates controller state, not test databases, application ports
or external services. Configure those separately for the project's checks.

### Keep package-manager resolution semantics

A checked release preserves the production dependency graph installed from its
frozen lockfile. Build reuse hashes the effective pnpm override rules, their
ordering and local override archive contents. Version-qualified files, aliases,
parent-specific selectors and dependency removals therefore participate in the
same build identity without being converted to another package manager's rules.
Changing archive bytes at the same path invalidates reuse; an identical checkout
at another temporary path does not. Local override archives must stay inside the
selected workspace and retain their verified bytes.

An npm-installable source bundle is a different output. Its supported override
forms are narrower. In particular, pnpm's `parent>child` targets an immediate
child, while an npm nested override also affects deeper descendants. Omnitron
refuses an unsupported conversion rather than broadening its effect.
Version-qualified local archives also require the checked release path:

```bash
omnitron release build <project>
```

This packages the verified installed graph for the selected target. It does not
relax source, lockfile, archive or runtime admission checks. See the native
[pnpm override semantics](https://pnpm.io/settings/dependency-resolution#overrides)
and [npm bundle override semantics](https://docs.npmjs.com/cli/v11/configuring-npm/package-json#overrides).

### Keep repository identity separate from credentials

Repository addresses become public build provenance in CLI, Console and exported
manifests. Use a Git credential helper or SSH agent for authentication. Release
builds reject remote URLs containing HTTP user information, passwords, query
parameters or fragments before cloning or recording those addresses. Ordinary
SSH usernames and local repository paths remain supported.

An existing unsafe or malformed manifest is reported as **invalid manifest**,
with a safe refusal reason; it is not presented as an unfinished build. The
catalog withholds its raw metadata, including embedded producer manifests.
Admission checks the same identities without changing their pinned bytes. Restore
a verified original manifest or create a new release with a credential-free
remote; editing a selected manifest invalidates its digest.

### Independent project operations

A project team can submit a controller build from the CLI or **Releases → Build** in Console. Both submit to the same Netron service. The returned job ID identifies the work across clients. Closing the client leaves the job under controller supervision.

```bash
omnitron --json release build example --for test --detach
omnitron --json release builds example
omnitron release job <job-id> --follow
omnitron release stop-build <job-id>
```

Run `omnitron --json release preflight` to inspect installed tools, build slots and publication support. The Console build dialog reads the same capabilities and respects older controllers with one exclusive builder. Different projects can build simultaneously. A duplicate submission is refused with the existing job ID. Use that ID to follow or cancel the job. The controller starts up to two builds by default, or one on a single-CPU builder. Set `releaseBuildConcurrency` in the instance configuration to match CPU, memory, disk and test-service capacity. Further projects wait in submission order. Cancelling a queued job removes it before installation or execution. Each running job has its own source directory, logs, process groups and cancellation signal.

Queued and running jobs appear in the project's **Releases** view. The global workspace shows active jobs across projects. Operators can cancel one job. Viewers can inspect its progress. Environment overrides require administrator access. Job history retains environment names and omits their values. After a controller restart, a final manifest recovers the result. An incomplete job is identified as interrupted and is available for inspection. Use a new submission to retry it.

#### Deploy a qualified release

Build on a machine matching the execution target, including the Node executable, ABI and libc. The foreground `release build` entry point supports isolated CI builders without a working controller. A configured, shared S3-compatible release store carries qualified releases between a builder and a controller. The index is published after all bytes have arrived. Current writers address each payload by SHA-256 and commit a format 2 index using a conditional S3 PUT. Concurrent publishers with different builds under the same ID cannot overwrite the winning payload. Concurrent measurements of the same build merge against the current index. Readers verify each payload and the manifest before installing the release.

Upgrade the controllers that publish or pull from a shared store before enabling format 2 writes. Current readers also accept format 1. An unchanged legacy index stays in format 1. Updating its contents or issuing a repair publishes a verified format 2 snapshot. Already-installed applications use their local selection. The S3 endpoint must support conditional PUTs and strong ETags.

```bash
# On a controller running on the build target
omnitron --json release build example --for test --publish --detach
omnitron release job <job-id> --follow

# On the deployment controller
omnitron --json stack check-release example test <release-id>
omnitron --json stack prepare-release example test <release-id>
omnitron stack start example test --release <release-id>
omnitron stack progress example test
omnitron --json release status example
```

**Releases → Build a release** offers **Publish verified build to shared store** when the controller advertises a configured destination. Both clients submit the same job. Publication starts after all gates pass and the requested artifacts build. The final receipt includes the index SHA-256. If a controller loses the reply, inspect the job and use `release push <id>` to reconcile the completed build. An isolated builder can still run a foreground build and publish its checked directory with `release push` from a configured machine.

**Releases → Deploy** and the stack's release dialog accept a catalog selection or an exact release ID. The initial check is read-only. **Fetch and verify** transfers a missing release into the controller catalog and repeats target admission. Preparation also stages the installed archives for a local stack. Operators on different stacks requesting the same missing release share the download. An incomplete local directory stays intact for investigation. An explicit `stack start --release` can fetch from the configured store itself. Automatic resumption uses the installed selection.

During a start, the interactive CLI and Console display changing node and application observations for the selected project, stack and release. The dialog can open the stack while execution continues. A completed transfer or an unavailable progress feed does not settle the start. After deployment, inspect launch receipts and application readiness. Rollback uses the previous verified release through the same admission and start operations. Check database compatibility and retain a stack-specific recovery point when a migration changes persisted data.

Build slots and deployment leases protect different resources. On disjoint nodes, projects deploy independently. Immutable deployments wait up to five minutes for an identified writer on a shared node. Set `settings.deploymentLeaseWaitSeconds` from 0 to 1800 in the stack declaration. Zero reports contention immediately. `stack progress` and the Console deployment view show the occupied node and the stack holding it. Other acquired nodes are released while waiting. A new admission check precedes the first mutation after contention. An unresolved remote command requires inspection. All target nodes must be reachable to acquire their leases for an immutable deployment.

The control-plane design describes the ownership boundaries and failure scenarios for shared hosts. [Multi-project deployment audit and design](/docs/omnitron/resilient-control-plane#multi-project-deployment-audit-and-design)

### Inspect migration provenance

An application artifact records its owning Git package in
`artifacts[].packageSource`. The `migrations` map identifies tracked migration
files; `migrationPrograms` identifies the explicitly declared entry modules,
including registries whose filenames do not contain `migrations`.

Each declared entry records the source path, Git blob and SHA-256, plus the
compiled path and SHA-256 from the actual packed closure. The builder uses the
same database ownership and stack overrides as deployment. With `--for <stack>`,
it records that stack's declarations. Without a selected stack, it combines the
configured stacks and refuses conflicting bindings. It verifies physical files
and committed source bytes without evaluating application or migration code.

Program provenance does not prove which migrations ran in a database, nor does
it inventory every imported source file. A registry can change when adding a
migration; its fingerprint is separate from the immutable migration-file map.
The shared Kysera runner remains responsible for comparing applied migration
bodies with database history. Imported applications retain the producer's
records. Missing records in an older artifact mean **unknown**, and an empty
migration-file map does not prove the absence of migration programs.

### Inspect an archive without running it

Use the public `@omnitron-dev/omnitron/release-archive` entry point when tooling
needs to inspect compiled modules, native binaries or static assets offline:

```ts
import { inspectReleaseArtifact } from '@omnitron-dev/omnitron/release-archive';

const inspected = await inspectReleaseArtifact(releaseId, 'api', {
  files: [
    { path: 'dist/native/service' },
    { path: 'dist/native/asset.json', includeContents: true },
  ],
});
if (inspected.manifestSha256 !== trustedManifestSha256) {
  throw new Error('Release differs from the trusted selection');
}
```

Obtain `trustedManifestSha256` from the reviewed release selection, independently
of the inspected store. The result includes the consumer artifact, its original
producer and target, verified closure, and each requested file's size and SHA-256.
The default store can be overridden with `releasesRoot`.

Inspection reuses release loading, bounded extraction and complete closure
verification. It accepts a target different from the inspection machine and
never imports application code, starts an owner or records an installation.
Requested files must be physical regular members. Hashing is streamed;
`includeContents` opts into at most 4 MiB per file and 16 MiB per request.
There may be up to 64 requested files, in addition to the launch entry. Private
scratch copies are removed on success and failure. The existing full-tree
verifier's resource limits still apply.

This proves integrity against the trusted manifest, not deployment permission,
passed release gates, application behavior or migration safety. A legacy archive
without a recorded compiled launch, closure or target is refused. The shared
loader verifies all archives in the selected release, so an absent or corrupt
sibling artifact also refuses inspection.

### Match the execution target

A checked application archive contains its compiled code, production dependencies,
installed optional packages and resolved peers. Build it with the target operating
system, architecture, Node binary and libc. `omnitron release show <release>` and
**Releases → Built with** display the target. Older manifests display *Target not
recorded*. The JSON output also includes the Node executable digest.

The application owns its complete SDK and bootstrap adapter. Local and remote
installation preserve the same archived dependency graph: package bytes, peers,
optional dependencies and module identities remain those admitted by the build.
An independently updated owner supplies process supervision and the exact Node
executable; it does not replace the application's Titan or Omnitron packages.
The application SDK commit and owner commit are separate identities and may differ.

Both paths use the same bounded archive extraction and closure verification.
Before execution, they verify the archive digest and size, compiled launch entry,
private package graph, complete payload, target and allowed Node options. Checked
delivery runs offline, without installing dependencies or rebinding the graph.
A mismatch refuses the operation with its cause; rebuild for the required target
or repair the declared input instead of modifying an installed archive.

Remote admission and checked delivery use the same authenticated native owner
probe. It verifies the registered supervisor, live process, sealed runtime and
private Netron resident execution protocol 1. The probe measures the target with
the owner's actual Node executable, independently of the controller and SSH
login shell. Admission compares platform, architecture, Node version, native ABI,
executable SHA-256 and libc against each artifact's producer. A matching framework
commit cannot substitute for this evidence; different framework commits are valid.

The receiving backend supplies the extractor from its attested runtime; the
owner supplies the execution binding. Owner ID, PID and controller epoch are
checked before delivery, registration and start.
An unknown protocol, coupled daemon or changed grant refuses the operation.
These checks preserve the existing role and recipient-fencing requirements;
they are not a transaction across nodes. An older owner can run its application's
own compatible SDK without claiming newer launch-receipt capabilities.

The optional `NODE_OPTIONS=--max-old-space-size=<MiB>` setting must match the
value recorded in the archive. Additional module paths, loaders and preloads cannot
be attested. After start, the owner verifies the application entry point, runtime and
installed file tree. The launch receipt lets CLI and Console distinguish delivery
history from the release a process is actually running.

### Pin a local stack

A local stack can keep its application code separate from the developer checkout.
Its selected release identifies the archive and manifest used for future starts.
The registered project's configuration still supplies its working directories,
environment, data locations and infrastructure. A stack that has never selected
a release continues to start from its working tree.

Admission compares the configuration's source inputs with the release commit.
Relative imports may reach sibling configurations or shared files anywhere in the
same repository; those files and shipped configuration directories participate
in the check. A used symbolic link and its resolved target both participate:
changing only the link cannot silently select another configuration with identical
current bytes. Every used link must be tracked, including links in directory paths;
its resolved target must stay inside the same repository and cannot be an ignored
source input. The same rules cover links used by shipped configuration files.
Untracked or escaping links, ignored targets, imported code outside the repository,
and source inside a Git submodule without its own verified binding refuse admission.

An input-inventory error stops preparation; it does not silently reduce the set of
checked files. Ordinary ignored runtime files, such as machine-local secrets read
explicitly by configuration, remain runtime inputs whose bytes are not verified
against the release commit. This exception does not make ignored imported code or
a used symlink's ignored target admissible. A source receipt does not attest every
external file or service that trusted configuration code can read.

1. Build the project release for the local execution target. Each application
   needs a packaged entry and a complete production dependency graph. Bootstrap
   apps must resolve their own compatible Titan and Omnitron adapter. By default
   the packed bootstrap is `dist/bootstrap.js`. Set the app's `releaseEntry`
   to select a different compiled JavaScript file inside its package.
2. Run `omnitron --json stack prepare-release <project> <stack> <release>`.
   Preparation verifies the archives and stages them in a private directory.
   It can run while the stack serves traffic. A successful preparation can
   report `prepared: true`, `ok: false`, `state: "requires-stop"`. The files
   are ready, but active applications still use the previous code.
   For an independently owned local stack, `declarations: "refresh-required"`
   means the controller needs declarations from this specific candidate before
   it can review preservation. An administrator runs
   `omnitron topology refresh <project> <stack> --release <release>` while the
   current deployment serves. Inspect any per-application capture failures.
   Check admission again and run `stack start <project> <stack> --release <release> --infrastructure preserve --dry-run` to review actual resources before
   scheduling the stop. A placement that requires a stop still reports that state.
3. Record the current selection from `omnitron --json stack status <project> <stack>`
   and agree on an application maintenance window. To keep databases and other
   stack containers running, review
   `omnitron stack stop <project> <stack> --keep-infrastructure --dry-run`, then
   execute the same mode with `--review <digest>`. This stops the applications,
   disables automatic resume and preserves infrastructure ownership and the
   selected release. Confirm every stop receipt before proceeding. Omit
   `--keep-infrastructure` only when the reviewed plan includes container removal.
   Keep the previous release or working tree available for recovery.
4. Run `omnitron stack start <project> <stack> --release <release>`. Verify
   application readiness and source-specific health checks. A plain subsequent
   start, including controller recovery, selects the same manifest. Project
   reloads preserve the physical release entry.

The controller stores the selected ID and manifest SHA-256 in SQLite. The owner
keeps the applications fenced while their definitions change. If the owner loses
an acknowledgement during the switch, inspect the stack before issuing an explicit
start to finish the operation. Changed or missing release contents block launch.

Before testing controller crash recovery, check that
`execution.recovery.status` is `ready` for the new generation. Stopping or
starting an application invalidates the previous certificate. A crash before
the next certificate leaves the resident owner serving its applications and
the supervisor waiting for reconciliation. See [Recovery after a backend
crash](/docs/omnitron/backend-handoff#recovery-after-a-backend-crash).

To roll back, confirm the affected applications have stopped and run the same
start command with the previous release ID. If the original source was the
working tree, use `omnitron stack start <project> <stack> --working-tree`.
The choice affects later restarts. Database migrations, external writes and
events that arrived during downtime need their own compatibility and recovery
plan. A code rollback does not restore that data.

A local pin requires a connected independent execution owner on macOS or Linux
and the Node executable digest recorded at build time. An allocation owned by another controller
takes its release from that controller. A local gateway serving a mutable
working-tree static directory cannot use this release selection. An application
relying on a script or symlink outside the archive must package that dependency
before it can be pinned.

### Changing a local stack's applications

Use the same release preparation, reviewed stop and explicit start when a local
stack adds or removes applications. An unpinned development stack uses the same
owner contract with its registered configuration. The owner must advertise
`stoppedStackReconciliation@1`. CLI `stack prepare-release` and Console's release
admission show which retained registrations will be retired. An older owner
reports the required owner upgrade before the selected release changes.

1. Keep the previous configuration, release and data recovery plan. Prepare the
   new configuration and release, including complete producer archives for any
   imported applications. A shared worker's script hash alone does not pin the
   SDK it imports.
2. Review and complete a stop of the **entire old stack**. Its retained inventory
   includes applications already removed from the target configuration. Use
   `--keep-infrastructure` when its databases and other containers must remain.
3. Start the new release explicitly. The owner verifies the exact stopped
   generations, disposes obsolete source watches, updates retained definitions
   and retires registrations outside the target graph. It then admits the normal
   dependency-ordered startup. Logs, artifacts and infrastructure are preserved.
4. Verify each application's observed release and health. After a controller
   interruption, inspect the actual owner state and explicitly retry the selected
   start. Cold recovery reads the same SQLite selection. Boot and automatic resume
   never remove retained applications on their own.

For rollback, stop the current stack, restore the previous configuration and
explicitly select its previous release. Previously retired registrations are
created from those original sealed artifacts. A stale plan, unconfirmed process
exit, unfinished operation or failed durable write blocks the transition; it is
not treated as a successful removal.

A separate development stack can then provide HMR while a shared-services stack
keeps its sealed producer graph. Splitting an existing mixed stack still requires
one reviewed stop of all applications belonging to that old stack. This does not
provide persistent public per-application stop or source selection, and does not
make a whole-stack switch safe without reviewing the shared services it affects.

### Development mode and a selected release

A local stack can run either its registered working tree or a selected release.
A local host, a project checkout used as `cwd`, and `watch: true` do not imply
that the running code comes from that checkout. Inspect each application's
observed release identity and entry point. A packaged static web server does not
run Vite or provide HMR; editing the checkout cannot update that server.

The current persistent selection applies to the **whole stack**.
`stack start --working-tree` changes later launches for all its applications and
requires a reviewed stop of that stack. `project reread` preserves a selected
release. There is currently no durable per-application switch between working-tree
and release code. Do not replace an installed payload or redefine an owner entry
privately to simulate one: that bypasses selection and can be lost on recovery.

A reliable per-application transition requires the same persisted source selection
in startup, reread and diagnostics, together with an owner transition scoped to the
selected applications. It must preserve other process identities, survive a
controller interruption and retain the previous source for rollback. Until that
contract is implemented and qualified, a stack containing shared scientific or
production services must not be switched to development mode as a frontend fix.

### Use an application from another project

A consumer can pin one application from an independently built producer release.
The producer release must already exist in the controller release store. Obtain
its ID and exact `manifestSha256` with `omnitron --json release show <release-id>`;
Console shows the same digest under **Releases → Built with → Manifest SHA-256**.
Record both values in the consumer configuration:

```ts
{
  name: 'collector',
  releaseFrom: {
    release: producerReleaseId,
    manifestSha256: producerManifestSha256,
    app: 'feed',
  },
  env: { CAPTURE_MODE: 'continuous' },
}
```

Build the consumer through the ordinary `omnitron release build` command. It
checks the producer gates, archive size and digest, frozen entry, package source
and execution target. An old archive without those records cannot be imported.
A consumer alias changes the application name in its stack; it does not rebuild
the archive or replace the producer's SDK. The consumer carries the exact producer
manifest bytes and its archive, so it remains self-contained if the standalone
producer release is later removed from the store. Nested imports preserve every
producer reference and its independent gate results.

In **Releases → Artifacts**, expand an application's source to see its producer
repository and commit, SDK commit, target and manifest chain. CLI text and JSON
show the same per-application origin. The consumer's top-level source fields
still describe its own build. These are recorded build inputs; current launch
observations remain in **Deployment state**.

Imported applications currently use verified **local** release installation.
Prepare and select the consumer release as described above. The producer's Node
executable, ABI and platform must match the local execution owner, and its own
adapter and dependencies must remain inside the archive. Its default working
directory is the installed package. An explicit `cwd` can name external mutable
data, but must not reintroduce runtime code or dependencies from a development
checkout. A `releaseFrom` entry cannot also override `bootstrap`, `script`,
`releaseEntry`, `releaseRoot` or enable a watcher.

Preparation verifies code without executing the producer bootstrap. Prerequisite
and topology details remain unknown until its declaration is available through
the normal application lifecycle; an unresolved import is reported explicitly.
When replacing a selected release, migration checks compare the actual producer
package and its existing migration files. Missing previous provenance or changed
applied migrations block the update.

Remote deployment of an imported application is refused before provisioning or
registration changes. It needs producer-captured deployment declarations and a
compatible remote owner; a local import does not grant those capabilities.
Older local owners may run the verified package while reporting no launch
receipt. The selected release and inspected files remain useful evidence, but
neither is presented as a confirmed current process identity.

### Mount a release artifact in a local Docker service

Use `infrastructure.artifactVolumes` when a service such as a gateway needs
files shipped in an application artifact. For example, a gateway can serve the
`dist/public` directory of a pinned application named `web`:

```ts
infrastructure: {
  services: {
    gateway: { ports: {}, env: {}, docker: { image: gatewayImage } },
  },
  artifactVolumes: {
    website: { service: 'gateway', app: 'web', directory: 'dist/public', target: '/srv/site' },
  },
}
```

Declare `web` in `apps`. Set `enabled: false` when the gateway serves its
files and the stack should start no web process. The release builder includes
the referenced artifact, checks the mount declaration and records any declared
migration programs. Building the first consumer release requires no running
stack or existing consumer installation.

Before starting the stack, prepare and select the consumer release. Omnitron
verifies the installed payload and mounts its directory read-only at `target`.
Stack diagnosis in CLI and Console names the application, service and target
still awaiting selection. Topology can show the declared service before its
artifact is installed.

### Read the executing application's origin

A packaged application can use the public `@omnitron-dev/omnitron/runtime` entry
point to associate its durable output with the bytes admitted for its launch.
The normal Omnitron bootstrap supplies `RUNTIME_CONTEXT` through Titan dependency
injection before eager providers initialize. This does not add a dependency on
Omnitron to Titan itself.

`InstalledRuntimeContext` has two explicit states:

| State         | Meaning                                                                                                                         | Application behavior                                               |
| ------------- | ------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------ |
| `verified`    | The compiled entry, installed payload, SDK binding, Node executable and allowed options match the private installation receipt. | Record the producer and consumer identity with new output.         |
| `unavailable` | Development or a legacy installation has no compatible origin contract.                                                         | Report unknown provenance or refuse an operation that requires it. |

Malformed metadata, a changed payload or a mismatched runtime throws
`RuntimeOriginVerificationError`. These failures never become an unavailable
context. Verified bootstrap children must load compiled modules from the same
checked installation. Working-tree development retains its ordinary loading path.

The verified context contains `producer` (`ArtifactIdentity`), `consumer`, `entry`,
`payloadSha256`, `executableSha256` and `observedAt`. Imported artifacts retain the
original producer's release, manifest, archive, project commit and SDK commit.
The consumer records the selected release and application alias separately.
Local preparation and remote receipt generation use the same origin mapping.
When a compatible owner verifies a consumer binding, CLI and Console group its
applications under that selected release and retain each original producer in the
expanded evidence. Different producers within one consumer release are therefore
not mistaken for conflicting deployments. Missing or malformed consumer evidence
never becomes a confirmed selection; legacy launch receipts retain their original
meaning.
Remote imports remain subject to the deployment restrictions described above.

`ownerAttestation` is explicitly `null`: startup payload verification is not a
process-owner launch receipt or continuous verification of running memory. An
older owner may execute the application while its launch identity remains
unreported. The application SDK cannot add unsupported owner capabilities.

For a standalone compiled script, call `verifyInstalledRuntime(new URL(import.meta.url))`
before starting work. This verifies the actual script entry; environment variables,
a source checkout's Git HEAD and a package version are not substitutes.
Browser DTOs can use `ArtifactIdentity`, `assertArtifactIdentity` and
`parseArtifactIdentity` from `@omnitron-dev/omnitron/runtime/identity` without
importing filesystem or process APIs. Parsing validates the six identity fields;
it does not admit an artifact or prove that a caller supplied truthful metadata.

For a data collector, commit an ingestion session with this context before opening
an external stream, and preserve that session identity through downstream processing.
Do not relabel historical rows or another producer's messages with the current
aggregator's runtime. Keep absent historical evidence explicit.

### Package a project-specific release

Use the headless `@omnitron-dev/omnitron/release-archive` entry point when a
project adds its own signed manifest or native artifacts. Populate a fresh staging
directory with approved files, then generate the hashes before adding your manifest.

```ts
import { writeFile } from 'node:fs/promises';
import { join } from 'node:path';
import { releaseFileHashes, sha256File, writePortableArchive } from '@omnitron-dev/omnitron/release-archive';

const files = await releaseFileHashes(stagingDirectory);
await writeFile(join(stagingDirectory, 'release.json'), JSON.stringify({ revision, files }));
await writePortableArchive(stagingDirectory, destination, { exclusive: true });
const checksum = await sha256File(destination);
```

`releaseFileHashes` rejects symlinks. The archive writer also supports internal
relative links for dependency graphs, sorts entries, preserves executable bits
and removes filesystem timestamps. Exclusive creation protects an existing release
when two builders finish at the same time.

## Build and inspect

Register the project and commit its `pnpm-lock.yaml`. The selected project
commit determines how the controller obtains the framework.

- **Package registry.** Declare `@omnitron-dev/omnitron` in a workspace importer.
  The controller resolves the project's registry configuration, checks the original
  archives against the lockfile's SHA-512 digests, verifies their common source
  commit and checks installed package files. A sibling Omnitron checkout is
  unnecessary. Archives must contain a full `gitHead`; older packages without
  that evidence cannot establish their framework commit. Registry credentials
  belong to the package manager configuration, not the release manifest.
- **Committed framework archives.** A workspace may pin local framework tarballs
  in Git. Supply the exact source commit with `--omni-commit <sha>`.
  The release builder independently builds that commit from the selected source,
  compares package contents with the archives, checks their lockfile SHA-512
  digests and verifies the installed package files. No hosting provider or branch
  name is mandatory. A manifest written by the consuming project cannot establish
  package origin by itself.
- **Linked checkout.** For framework development, point workspace links at a
  single Omnitron checkout. The controller builds the linked packages from its
  clean clone. Mixing linked and registry framework packages fails the build.

### Read the build result

A retained manifest means packaging finished; it does not mean the release passed
its checks. CLI and Console interpret the same recorded gate and artifact outcomes.
`release build` exits **0** only when every recorded check passed and no artifact
failed, **1** for a failed check or artifact, and **2** when checks are absent or
unfinished (including `--skip-gates`, timeouts and killed checks). A failure takes
precedence over unfinished checks. Signals and earlier build exceptions retain
their ordinary nonzero exit codes.

Console marks an unsuccessful build as failed, displays the specific gate or
artifact reason, and retains its release ID, manifest, logs and individual results
for diagnosis. Successful build checks do not authorize deployment: destination,
source, runtime, role and lifecycle admission still run when selecting the release.

### Choose an SDK source

Repository location and trust are separate concerns. Omnitron uses Git rather
than a hosting provider's API to inspect source history. GitHub, GitLab, private
servers, mirrors and local repositories use the same contract. The official
package's repository is a default supplied by its metadata, not a platform
requirement.

Keep project defaults beside its existing release gates in the workspace
`package.json`:

```json
{
  "omnitron": {
    "framework": {
      "repository": "ssh://git@forge.example/platform/omni.git",
      "ref": "refs/heads/stable"
    },
    "release": {
      "gates": ["typecheck", "test"]
    }
  }
}
```

`repository` selects the actual SDK source. `ref` is optional: a full
`refs/heads/...` requires the exact commit to belong to that branch; a full
`refs/tags/...` requires it to match that tag. Omitting it does not imply `main`.
Configuration is read from the selected project commit. Selection follows one
order: a build override, committed `omnitron.framework`, then the locked
framework package's repository metadata. Legacy packages without repository
metadata use the builder package's declared repository. Linked development uses
its linked checkout unless an explicit source is selected. Invalid explicit
configuration fails instead of silently selecting another source.

A deliberate per-build override replaces that selection:

Through Console or RPC, selecting another SDK repository or supplying any
nonempty environment override requires a current administrator session and a
durable audit intent before the build starts. Source scripts and environment
settings for Git, package managers or interpreters can execute code as the
controller's OS user. The audit records environment variable names, never their
values. An empty environment object does not constitute an override.
Operators can still build registered projects using committed source settings
and the existing controller environment. The local CLI operates with the invoking
OS user's existing access; environment validation still applies.

```bash
omnitron release build sample \
  --framework-repository https://mirror.example/platform/omni.git \
  --framework-ref refs/tags/v1.0.0 \
  --omni-commit <full-commit>
```

Integrations and browser forms can validate the same request with the pure
`parseFrameworkSource` function from `@omnitron-dev/omnitron/release-source`.
It returns the `FrameworkSource` contract and rejects unknown fields, ambiguous
refs and credentials; it does not fetch code or grant permission to build.

The Console build dialog exposes the same options under **SDK source**. Ordinary
builds need no extra fields. CLI and Console report the selected repository, ref
and exact commit in build progress. Overrides are recorded with the build request, and
the resulting manifest retains the resolved repository, exact commit and any
required ref. Console disables overrides without current administrator access or
when an older controller does not report support; those controllers retain their
previous source-selection behavior.
Credentials must stay in Git's SSH agent or credential helper; URLs containing
passwords or HTTP tokens are rejected before logging the request.

Registry downloads use pnpm's own configuration, authentication and HTTP client.
The clean selected workspace supplies `pnpm-workspace.yaml` and `.npmrc`; user
configuration supplies credentials, including user-only `tokenHelper` commands.
Scope registries, TLS and proxy options follow the same precedence as the frozen
install. Download verification never executes package lifecycle or `.pnpmfile`
hooks. Original compressed bytes must match the committed SHA-512 before their
contents can establish package identity.

Run the controller in an ordinary environment without Git repository/object
redirects such as `GIT_DIR` or `GIT_OBJECT_DIRECTORY`. Source verification refuses
these overrides because they can change which repository is inspected. The
release executor inherits ambient environment keys. An internal registry request
that requires removing a key is refused before starting a worker; a partial
environment is not treated as an isolated process environment. Git credential
helpers and SSH configuration remain available.

### Build from local sources

The registered project's local Git checkout supplies its selected commit; a
reachable project hosting service is not required to clone those local inputs.
Use a local SDK repository explicitly when its commit is available only on the
build machine:

```bash
omnitron release build sample \
  --project-commit <full-project-commit> \
  --framework-repository /srv/sources/omni \
  --omni-commit <full-framework-commit>
```

The builder still creates clean clones, performs the frozen install and declared
gates, and verifies archive contents. Prepared dependencies or an accessible
registry are required; choosing local Git does not make missing packages available.
For a fully offline build, prepare those dependencies and any gate fixtures first.

Source publication is a separate, build-time observation. A local repository
can be a valid build input without being remotely published. A network failure
or unavailable ancestry is **unverified**, not evidence that a commit is
unpublished. Cached remote-tracking branches alone do not establish publication.
The stack's `release.requireOnRemote` policy decides whether confirmed remote
publication is required for deployment. When omitted or `false`, publication is
not a prerequisite; a release must still satisfy every other requirement.
An explicit `true` remains enforced, including for imported producers. Source
overrides do not skip gates, checksum checks or that policy. These states never indicate where an application
is running. Offline use also requires that the selected sources, dependencies
and declared gates are available in that environment.

In Console, **Releases → Built with** and the CLI command `omnitron release show`
identify the framework source and package count. Use `--json` on the CLI to
inspect the recorded npm versions and integrity digests. On a remote stack, a
registry release requires a verified execution owner and compatible Node target.
Its application SDK remains the exact packed graph identified by the producer;
it does not have to share the owner's SDK commit. A backend upgrade alone can
leave an older owner in charge of the applications.

Release admission obtains native evidence through the same registered SSH
credentials, host verification and private Netron endpoint as installation. It
does not infer compatibility from the public daemon version or demand that an
application use the controller's SDK. The current remote managed-owner probe
supports Linux systemd installations; local installation verifies its resident
owner and native executable on the local host. A changed owner or controller
grant invalidates the proof. An unavailable owner, incomplete target or coupled
daemon produces a refusal before any migration or application restart. Repair
the stated condition and repeat the release check; changing a build stamp does
not establish compatibility.

Pin `--project-commit` to reproduce a build. For linked sources,
`--omni-commit` selects a revision. For registry packages it checks the exact
commit already pinned by their archives.

```bash
omnitron release build sample \
  --project-commit <project-sha> \
  --omni-commit <omni-sha> \
  --for test
omnitron release list
omnitron release show <release-id>
```

`--for <stack>` includes the static bundle for that stack's gateway environment.
The deployment retains the build directories mounted by running and stopped
gateways, including older gateways that mount a build directly. A missing
`current-<project>-<stack>` link does not identify another build: the original
mount remains protected from retention cleanup. Existing browser tabs keep
access to their content-hashed assets when a newer build is published.

`--keep-source` retains the build clones after success. `--skip-gates` records
the gates as not run; it does not produce a verified release suitable for a
stack that requires those gates.

To run the workspace's existing checks as release gates, declare their script
names in the workspace `package.json`. For example, set
`"omnitron": {"release": {"gates": ["typecheck", "test"]}}` when both scripts
exist. Each runs through `pnpm run` in order. A failed or interrupted script
leaves each later gate marked `not-run`. Use one gate definition per workspace.
A project that supplies both this list and `scripts/gates.mjs` fails validation.

Read the gate results and artifact list before deployment. The manifest
describes what was built; the daemon's audit trail describes which release
a stack took.

## Understand what is built and what is running

A project owns multiple stacks. A release is a stored build that can be deployed
to several stacks; building or downloading it does not start any applications.
Keep these observations separate:

| Evidence                 | What it establishes                                                                        |
| ------------------------ | ------------------------------------------------------------------------------------------ |
| Release manifest         | Source commits, build gates and packaged artifacts                                         |
| Current stack inventory  | Observed process state, application counts and node reachability                           |
| Last recorded deployment | The latest `stack.start` audit record for that stack                                       |
| Attestation              | What probes measured, on which stack and at what time                                      |
| Git publication snapshot | The recorded source-publication result at build time; new builds query the declared origin |

In Console, select the project and open **Releases**. **Deployment state** lists
its stacks, placement, running/configured application counts and last recorded
deployment. The same view appears on release and stack details. Stack and release
links connect these views. The artifact list counts storage on the controller,
within the selected project filter. It is not a count of active deployments.
**High host load** records a build host's five-minute load average above its CPU
core count when checks finished; gate results remain separate.

For example, `dev: stopped, Apps 0/6` and `test: running, Apps 6/6` describe two
configured stacks with only the remote stack running. A historical release beside
`test` does not prove those processes still use that build after later updates.
Updated owners report verified launch receipts: the release ID, original manifest
SHA-256, artifact SHA-256, launch ID and verification time travel with each process
observation. Console shows **Launch release verified** only when every observed
running application has complete, consistent launch evidence. Missing peers,
legacy owners, conflicting manifests and mixed releases remain explicit. A receipt
identifies the artifact admitted at launch; it does not continuously inspect code
already loaded into memory. Never infer launch identity from the last audit record,
a directory name or a build-cache marker. An unreachable node or failed inventory
request remains unknown, not stopped or empty. Launch receipts trust the node,
its process owner and local filesystem; they are not signed remote attestations
against a compromised host or another process with the same OS-user privileges.

The owner also reports the application's current `generation`, independently of
its receipt's `launchId`. The shared reader requires these to match when the
owner supplies a generation. This covers thread workers, process pools and
worker-only applications that have no single main PID; `pid: null` does not mean
such an application stopped. A child replacement invalidates the current receipt.
For installed applications, automatic replacement repeats the original executable
and loader-environment verification before spawning. Once membership settles, the
owner checks the installed release again and confirms only the exact, live
supervisor children and pool workers observed throughout that check. A changed
file, failed admission, missing worker or concurrent stop keeps the reason
explicitly **unreported**. Normal worker recycling can therefore regain a verified
receipt without restarting the whole application. A partially completed resize
can confirm the actual members; readiness and the requested worker count remain
separate observations.

A stale or malformed generation cannot be rescued by a positive PID. Older
owners without a generation retain their PID-based reading; applications without
a main PID remain **unreported** until an updated owner can observe their launch.
A backend update alone cannot invent this missing evidence. For distributed
stacks, each physical replica carries its own generation and receipt; an aggregate
row never substitutes for an unavailable replica.

```bash
omnitron release status sample
omnitron --json release status sample
omnitron release status sample --inspect-installed
```

This read-only command uses the same model as Console. Exit `0` means the available
observations resolve the report; `2` means incomplete evidence, including an
unreported current release or unavailable history/inventory; `1` means command
failure. JSON separates `inventory`, `history` and per-stack `runningRelease`.
The older `list` and `show` JSON retain `deployedOn` for compatibility and identify
its meaning with `deploymentEvidence: "audit-history"`.

When launch evidence is missing or an installation may have changed, use **Inspect
installed files** on the stack's Deployment state row, or `--inspect-installed` in
the CLI. This is an explicit, read-only operation available to viewers; it never
runs during status polling and never restarts applications. The node binds its
read to fresh owner observations of the process, start time and entry point,
then hashes retained archives and the current bootstrap. The controller matches
archive SHA-256 **and byte count** against every matching project manifest.

The result separates a retained base release from installed modifications. A
changed bootstrap or different reported framework build is **modified**. Reused
artifacts can match several releases: all candidates remain visible as ambiguous.
A matching bootstrap is only a partial installation check; other files and loaded
memory have not been proven. Disk observations never create launch receipts.
Missing archives, unsupported node backends, unavailable build stamps, catalog
errors and changes during inspection remain explicit, rather than becoming a
false match. JSON adds a separate `installed` collection without changing
`runningRelease`; modifications, ambiguity or incomplete inspection also exit `2`.

Inspection is bounded: one scan per node, at most 32 application handles, eight
archives per installation, 512 MiB per archive and 2 GiB of archive input per node.
Archive subprocesses have ten-second deadlines and bounded output; the node's
disk work has a 45-second deadline. A stack inspects at most 16 nodes with four
concurrent requests and reports any nodes excluded by the time budget. Larger
installations or missing sources require a separate integrity investigation. No
file paths, environment variables or archive contents are accepted from the viewer.

The neutral **History: test** label means a recorded deployment. **Git:
publication recorded** means the manifest recorded a positive source-publication
result. **Git: unpublished source** means it recorded a negative result, including
an explicitly local source. **Git: unverified** means the builder could not
establish publication. These labels describe source evidence, not execution on
localhost, artifact-store upload status or the current availability of a Git server.

New builds query the declared origin's current branches and tags. Cached remote
refs alone cannot establish publication. Older manifests retain the weaker
remote-tracking observation made by their builder; upgrading Console does not
strengthen or rewrite those receipts. Rebuild to obtain the new observation.
The existing `requireOnRemote` stack policy continues to evaluate the recorded
result so that an upgrade does not silently invalidate pinned releases or rollback.

Imported configurations retain their runtime namespace. Deployment status joins
that namespace to the canonical project/stack and chooses the newest record.
Conflicting records or invalid timestamps remain explicit uncertainty; no extra
project registration is needed.

The browser-safe `@omnitron-dev/omnitron/release-reading` entry point exports
`readReleaseDeploymentState`, `stackDeploymentHistory` and
`readSourcePublication` for other clients. It composes the existing stack and
audit APIs without provisioning, starting, stopping or deploying anything.

## Dependency identity

For local SDK development, run `omnitron sdk init` to install or upgrade the managed pnpm hook. Relative `file:` and `link:` overrides resolve from the consumer workspace in the development profile, including transitive dependencies. `omnitron sdk registry` restores the distribution lockfile.

The local bundle carries workspace packages and local `file:` archive overrides.
It asks pnpm for the effective overrides, validates each archive's package name,
and records its version and SHA-256 in `BUNDLE.json`. The destination installs
those same bytes. A missing archive or an archive outside its source workspace
fails the build instead of silently selecting the published package. The copy is
verified again before the bundle is completed; changing an archive during packing
fails the build. Archive contents, resolution configuration and lockfiles also
participate in the application build cache, including same-version replacements.

Direct dependencies and optional dependencies use the same override as transitive
consumers. Registry version overrides are retained too. Platform-specific binaries
are installed on the destination; a macOS `node_modules` tree is never copied to
Linux. After publishing a dependency to npm, update its version and lockfile,
remove its local archive override, and verify a clean install before deployment.

### Promoting a dependency from npm

For a coordinated Xec or Kysera release, compare each exact version with the
publisher's saved SHA-512. An accepted upload, a distribution tag and a
downloadable tarball can become visible at different times. Download by package
name and exact version from a fresh cache to exercise the same index as an
install. If this read returns `ETARGET` after npm accepted the upload, keep the
original receipt and retry the read as the registry index updates.

```sh
package='@xec-sh/core@0.11.2'
scratch=$(mktemp -d)
npm view "$package" dist.integrity --prefer-online
npm pack "$package" --ignore-scripts --cache "$scratch/cache" --pack-destination "$scratch"
printf 'sha512-%s\n' "$(openssl dgst -sha512 -binary "$scratch"/*.tgz | openssl base64 -A)"
```

Compare the SHA-512 of the downloaded bytes with both the registry integrity
and the preparation receipt. Once the whole batch matches, update the
manifests and lockfile together. Keep any minimum-release-age exception
scoped to an exact, verified package version. Run two builds from separate
clean checkouts with the same Node executable, pnpm version and target.
Compare the runtime, application archives and static assets, then test
admission of the resulting archives against the target runtime. Live release
registrations continue to refer to their original digests.

### Following a remote transfer

Open **Deployments** in the project workspace or run
`omnitron --json stack progress <project> [stack]`. Each remote stack start
assigns one operation ID to provisioning and delivery across its nodes.
Expand **Transfer details** to copy that ID, the timestamp or the selected release.
Two stacks sharing a host and an application name have separate observations.
The latest report for each stack, node and application is kept in controller
memory. An earlier controller may supply reports without workspace identity.
Console keeps those behind an explicit control. Use **Audit** for recorded
starts and **Releases** for evidence of which artifact a process loaded.

## Validate linked source checkouts

Development applications can share a sibling Omnitron checkout. The release
builder reproduces that relative layout with clean clones of pinned commits.
Application tooling can use the same validation through the Node-only
`@omnitron-dev/omnitron/source-layout` entry point:

```typescript
import { readLinkLayout, planBuildRoot } from '@omnitron-dev/omnitron/source-layout';

const layout = readLinkLayout(lockfileText);
if ('refusal' in layout) throw new Error(layout.refusal);
const plan = planBuildRoot(buildRoot, projectRoot, layout);
if ('refusal' in plan) throw new Error(plan.refusal);
```

The planner rejects multiple external source roots, non-workspace targets and
layouts escaping the isolated build directory. Tooling must also check that the
installed links and package manifests exist. Published dependencies need no
sibling-checkout layout. Do not replace this validation with an allowed count of
links: adding a framework module does not change whether a layout is reproducible.

## Verify the deployed system

An attestation records what probes measured about a release on a named stack.
Build success alone does not establish that the deployed application works.

```bash
omnitron release attest <release-id> --stack test --on-node
```

`--on-node` stages the release's own probes on the stack's node and runs them
under its deployment lease. The project supplies its probe implementation.
The daemon records the results and the provenance of the probe sources.

To submit an already collected result instead:

```bash
omnitron release attest <release-id> --stack test --from result.json
```

Use one input mode: `--on-node` or `--from`. The daemon checks attestation
freshness against deployment history. A recorded probe failure remains a
failure; storing the result does not turn it into a passing gate.

## Move a release between machines

Configure `releaseStore` in the daemon configuration and keep its credential
in the vault. The transfer commands use the artifact store:

```bash
omnitron release push <release-id>
omnitron release pull <release-id>
```

A pull checks the received release before making it available locally.
`--root <dir>` changes the local release root. `push --repair` rewrites objects
when a store was found damaged; it is an explicit repair operation.

### Private S3 on a build node

One S3 bucket can serve builders on different machines. Use its HTTPS endpoint when the builders can reach it directly. For a MinIO service bound to loopback, configure the builder on the storage host with its local endpoint. On other builders, add an SSH route to the same bucket in the daemon configuration. Omnitron uses Xec to open a private forwarding socket for each transfer. The CLI, build jobs and Console use the same configuration.

```json
{
  "releaseStore": {
    "endpoint": "http://127.0.0.1:9000",
    "bucket": "platform-releases",
    "accessKey": { "secret": "release.reader.id" },
    "secretKey": { "secret": "release.reader.secret" },
    "ssh": {
      "host": "storage.example.net",
      "port": 22,
      "username": "release-forward",
      "hostFingerprint": "<verified SHA256 host fingerprint>",
      "privateKey": { "secret": "release.forward.private-key" }
    }
  }
}
```

The SSH service account only needs local forwarding to the declared numeric loopback address and port. Restrict its key to that destination. Keep shell sessions, agent forwarding, PTYs and remote forwarding disabled. An encrypted private key can name a vault entry in `ssh.passphrase`. Before enabling the route, verify the SSH server fingerprint through the node registration or another trusted channel. The S3 key controls access independently. Scope a writer to its project's objects and grant readers only the access needed to fetch verified releases.

Run `omnitron release preflight` or open **Releases → Build a release** to see the configured destination. This inspection reads configuration. Confirm actual delivery by publishing a checked release, pulling it on another host and comparing the index SHA-256. A closed SSH route aborts the transfer. Repeating the operation checks the S3 index. Loopback endpoints on different machines address separate services unless the route is configured. Upgrade all participating clients before introducing the optional SSH configuration or writing format 2 indices to a shared bucket.

For a shared store, issue independent writer credentials for each project's prefix and read-only credentials to its consumers. Grant writers conditional PUT and GET on their project objects. Keep the administrative identity on the storage host. Validate access with a synthetic release before assigning a working profile. A reader must fail to write, a builder must fail to write into another project's prefix, and an anonymous GET of a known index must fail. Compare the actual index digest and archive bytes after a pull on each participating host and after an interrupted transfer.

A forwarding-only OpenSSH account can use `AuthenticationMethods publickey`, `AllowTcpForwarding local`, `PermitOpen 127.0.0.1:<s3-port>` and `MaxSessions 0`. Disable Unix-socket and reverse forwarding, agent forwarding, X11 and PTY allocation for this account. Issue a separate key per node and make the authorized public keys readable by the account. Confirm the effective `sshd -T -C` policy and exercise an allowed and a refused destination. Keep SSH credentials and S3 credentials in their respective node vaults.

Check the S3 server's security support as well as its compatibility with conditional writes. The archived MinIO Community image from September 2025 accepts forged unsigned-trailer writes under the conditions documented in the vendor advisory. An isolated bucket and private network do not verify a request signature. [MinIO security advisory](https://github.com/minio/minio/security/advisories/GHSA-hv4r-mvr4-25vw)

## Retain recoverable artifacts

```bash
omnitron release prune --keep 5
```

Without `--yes`, prune reports what it would remove. Applied pruning protects
releases named by the latest deployment records. If that history is unavailable
or a recorded release has no identity, normal applied pruning refuses. This
protection does not attest current process identity; explicitly retain recovery
artifacts and investigate out-of-band deployments before deletion.

Retaining an artifact is only one part of recovery. A previous application
must still be compatible with the current schema. See
[Production readiness](/docs/deployment/production-readiness).

## Project tool integration

Application-specific probes and account tools belong to the application
repository. Omnitron supplies node transport, deployment coordination and
container addresses.

The container environment uses `<PROJECT>_PG_CONTAINER`,
`<PROJECT>_REDIS_CONTAINER` and `<PROJECT>_GATEWAY_CONTAINER`. The namespace
comes from the registered project name, uppercased with punctuation replaced
by underscores. Values use the stack's actual container prefix. A project
named `sample` therefore receives `SAMPLE_PG_CONTAINER`, even if its
containers use a custom prefix.

This contract lets a private application remain an integration test bed
without making its identity part of the framework.

Operator-side TypeScript tools can import `createDaemonClient`,
`ExecutionService`, `RemoteDeployer`, `buildBundle`, `archiveBundle` and
`StackInfrastructureManager` from `@omnitron-dev/omnitron/operations`.
This is a server/CLI entry point: keep it out of browser bundles and let the
daemon retain transport credentials in its vault. The regular CLI and Console
pipelines remain the interface for running and tracking deployments.

## Command reference

| Command                             | Purpose                                                                |
| ----------------------------------- | ---------------------------------------------------------------------- |
| `omnitron release build <project>`  | Build pinned project and runtime commits                               |
| `omnitron release list`             | List controller artifacts and recorded deployment history              |
| `omnitron release status <project>` | Observe stack placement, process state and separate deployment history |
| `omnitron release show <id>`        | Inspect commits, gates and artifacts                                   |
| `omnitron release attest <id>`      | Record probe results for a stack                                       |
| `omnitron release push <id>`        | Upload to the configured artifact store                                |
| `omnitron release pull <id>`        | Fetch and verify a stored release                                      |
| `omnitron release prune`            | Preview or apply retention                                             |

Use `--help` on a command for its complete option set.

## Prepare without damaging the running release

Remote application installation uses a separate directory identified by package
version and artifact digest. Reinstalling a package version does not clear the
currently running directory. A failed transfer, archive validation or target check
leaves existing application files in place. Checked releases never install
packages on arrival. The explicit legacy development bundle path has separate
installation and engine checks; it is not a fallback for a rejected checked release.

The installation receipt records the unchanged application SDK and payload,
producer and consumer identities, exact execution target and physical owner
binding. Reusing another owner's receipt is refused even when its SDK bytes
match. Preparation does not evaluate an application module. An installed entry
must pass its receipt check before bootstrap introspection or script execution;
a missing or modified receipt cannot silently turn it into a development app.

For each node, all selected artifacts must install and migrate successfully before publishing new boot definitions. If one preparation fails, no candidate on that node is activated. Earlier successful migrations may already have committed: this is not a transaction across application databases. Use backward-compatible schema changes and verified database backups.

Publication preserves definitions of applications outside the selected deployment, updates the explicit host-upgrade contract's release paths while retaining its hooks, and writes private boot files through atomic file replacement. A private on-node registration snapshot supports restoring the previous configuration after a publication/registration failure. A multi-file publication can be interrupted; the host-upgrade preflight refuses inconsistent configuration. Configuration restoration does not by itself prove running processes were rolled back; inspect application health and the deployment outcome.

Before a controlled daemon upgrade, Omnitron compares the serving inventory with the configuration that a fresh daemon would load from disk. Missing applications or stale release paths fail preflight before stopping the daemon. A forked daemon's actual Node executable is captured and reused for controlled restart, without changing the host's global Node link. Systemd upgrades preserve the captured unit's runtime binding. Application-specific health checks must pass after restoration.

New nodes without a runtime default to Node 22.23.4. Existing runtimes are not globally replaced by a deployment. Check application engine compatibility and use an explicitly scoped runtime or supervisor configuration when upgrading a host that runs other software.

---

---
id: "omnitron/resilient-control-plane"
title: "Resilient control plane"
description: "Target architecture for environments, verified promotion, placement, recovery, scaling and delegated infrastructure operations."
---

# Resilient control plane

Omnitron should turn operational intent into verified infrastructure state. A
developer, operator, security engineer or agent chooses the outcome and its allowed
constraints. The platform owns planning, admission, execution, recovery and evidence.
Adding a convenient button around an untracked shell command does not satisfy this contract.

> **Design status — 31 October 2026**
>
> This is the target architecture and implementation acceptance contract. The
> [topology workspace](/docs/omnitron/topology) implements observation, portable target drafts,
> validation and durable acceptance. Existing release verification, resident process
> ownership, Xec execution and monitoring are reusable foundations.
>
> Fleet-wide automatic placement, admitted topology migrations, delegated agent
> execution and stateful failover are not implemented by that workspace. The following
> models and policies specify their development; they are not new configuration
> options that can already be enabled in `omnitron.config.ts`.

## One lifecycle, explicit responsibility

```mermaid
flowchart LR
  Sources["Code, Console, CLI, agents"] --> Intent["Versioned target intent"]
  Intent --> Plan["Validate and plan"]
  Evidence["Inventory, health, capacity, receipts"] --> Plan
  Policy["Identity, policy, budgets"] --> Admission["Admit exact plan"]
  Plan --> Admission
  Admission --> Journal["Durable operation and reservations"]
  Journal --> Execute["Fenced node and service adapters"]
  Execute --> Verify["Readiness and effect verification"]
  Verify --> Evidence
  Evidence --> Reconcile["Explain drift or propose repair"]
  Reconcile --> Plan
```

The reconciliation loop is event-driven, with bounded periodic inventory repair.
A missed event must delay convergence, not permanently hide drift. Events wake the
controller; the authoritative state and operation journal decide what it may do.

Every mutation follows the same lifecycle. Domain adapters determine what execution
and verification mean. A process restart, PostgreSQL promotion and route change share
authorization, journaling and observability; they do not share a fictional universal rollback.

| Layer                              | Owns                                                                                                          | Must not duplicate                                           |
| ---------------------------------- | ------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------ |
| Kysera                             | Database transactions, query facilities and migration machinery                                               | Infrastructure placement or node lifecycle                   |
| Titan database                     | Application integration, database lifecycle, migration facade and provider capabilities                       | A second migration engine                                    |
| Titan core / Netron                | Contracts, principal propagation, transport, lifecycle, diagnostics and reusable dependency planning          | Project/environment policy                                   |
| Titan PM                           | Local processes, pools, supervision, readiness and in-process workflows                                       | Fleet scheduling or data failover                            |
| Optional durable workflow facility | Persisted operation transitions, effect reconciliation, cancellation and compensation contracts               | Provider-specific SSH, SQL or container management           |
| Xec                                | Local/remote execution, SSH, streaming, scripts and cancellation                                              | Deployment admission or permission policy                    |
| Omnitron                           | Environment intent, resources, placement, release admission, repair policy, adapters and operator read models | Another process supervisor, SSH stack or generic role engine |
| Prism / Console / CLI              | Accessible interaction, review, progress and evidence                                                         | Independent scheduling or authorization decisions            |
| Application                        | Workload/resource declarations, readiness, domain recovery and compatibility contracts                        | Private deployment or retry frameworks                       |

Keep optional capabilities independently importable. The basic local daemon should
not require a consensus service, agent SDK, cloud account or container runtime.
Unsupported capabilities are discoverable and cause a precise plan refusal.

## Audit of the existing foundations

These findings determine the implementation sequence rather than a parallel replacement platform.

| Existing implementation                                                     | Reuse                                                                                                                                 | Boundary to close before autonomous execution                                                                                                 |
| --------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------- |
| Release manifests, gates, `verifiedOn`, attestations and artifact checksums | Promote verified immutable artifacts                                                                                                  | Attest the complete placement/runtime combination; current on-node attestation deliberately refuses multi-node coverage it has not measured   |
| Resident owner and backend handoff                                          | Keep application processes alive while replacing the backend                                                                          | Local owner fencing is not evidence that a lost remote server stopped writing                                                                 |
| Stack deployment and node execution leases                                  | Serialize existing host deployment effects                                                                                            | Legacy `project/deployed/app` identity and per-project node configuration cannot isolate arbitrary stacks/allocations on one host             |
| Titan PM pools                                                              | Local concurrency, backoff and health                                                                                                 | Host pools do not decide fleet capacity or failure-domain placement                                                                           |
| Titan PM workflows                                                          | Stages, dependency ordering and compensation                                                                                          | Execution state is in memory; it is not a durable recovery journal                                                                            |
| Pipeline execution                                                          | Reviewed revisions, bounded output, cancellation and interrupted-run reporting                                                        | Interrupted external effects are not automatically replayable                                                                                 |
| SQLite state store and telemetry outboxes                                   | Local durability and independently acknowledged delivery                                                                              | A telemetry receipt is neither a control-plane quorum nor proof of a workload's database recovery                                             |
| Cluster lease and configuration sync                                        | PostgreSQL epochs, administrator leadership transfer and recipient-verified managed stack stops and recovery of retained applications | Each additional effect requires admission by its resource owner. Distribution of configuration and command history has separate requirements. |
| Existing roles and `@omnitron-dev/access`                                   | Shared role hierarchy and permission matching                                                                                         | Resource-scoped delegation, live budgets and per-step grants need enforcement; role names alone do not provide them                           |

Relevant source boundaries are `project/topology-control.ts`, `project/topology-state.ts`,
`project/stack-placement.ts`, `release/manifest.ts`, `services/node-deploy-lease.ts`,
`project/node-app-config.ts`, `cluster/leader-election.ts`, `cluster/cluster.rpc-service.ts`
and `packages/titan-pm/src/process-workflow.ts`.

The current remote stop path also needs a node-confirmed lifecycle contract: disconnecting
the master's peer or clearing its local status cannot count as stopping remote processes.
No automatic migration may depend on that shortcut. All accepted target edits are
isolated from executable configuration until an execution path satisfies these boundaries.

## Resource model

Use one canonical environment model. Existing stacks become its compatibility surface;
do not maintain an unrelated environment scheduler beside the stack manager.

| Resource             | Stable identity and meaning                                                                                               |
| -------------------- | ------------------------------------------------------------------------------------------------------------------------- |
| Project              | Ownership boundary; display names can change without changing identity                                                    |
| Environment          | `dev`, `test`, `stage`, `prod` or another name; release policy, quotas, bindings and availability goals                   |
| Workload             | Logical application/service, artifact reference, runtime adapter, resources, readiness and state contract                 |
| Replica / allocation | A particular workload revision placed on a node incarnation with resource reservations                                    |
| Node                 | Registered machine identity, current incarnation, verified capabilities and measured capacity; its address is replaceable |
| Node pool            | Eligible resources, failure domains, reserve policy, trust boundaries and acquisition budget                              |
| Service binding      | A logical consumer/provider relationship resolved to actual endpoints and secret references at execution                  |
| Data resource        | Storage identity, replication/backup ownership, recovery evidence, RPO and RTO                                            |
| Route                | Traffic or publication relationship, update strategy and readiness conditions                                             |
| Operation            | Durable attempt to realize a reviewed plan, with steps, receipts and recovery state                                       |

Node ID, node incarnation, workload ID, replica slot and process PID are distinct.
Reusing an IP address must not inherit a dead server's execution authority or writer
identity. A replacement node enrolls as a new incarnation and is verified before admission.

Desired, admitted and observed state are separate revisions:

- **Desired** describes the target an authorized actor accepted.
- **Admitted** identifies the exact plan, artifact set, policy revision and resource
  reservations allowed to produce effects.
- **Observed** contains evidence from an owner/runtime, its observation time and
  freshness. Missing evidence is `unknown`, not zero, healthy or stopped.

Each resource has explicit field ownership. Git-managed intent cannot be silently
rewritten by Console or an agent. Console can propose a change to it, or operate
fields assigned to runtime policy. Admission resolves conflicts rather than relying
on whichever controller wrote last.

## From a laptop to production

Development uses local source and fast feedback. Test/stage/prod consume immutable
releases under environment policy. Production promotion references the same verified
artifact identity instead of rebuilding from the developer's current working tree.
Reuse the existing [release manifest and verification](/docs/omnitron/releases) contracts.

An artifact set must identify source revisions, dependency lock, platform/architecture,
runtime compatibility, application artifacts, migration set and configuration schema.
Secrets remain versioned references. A secret rotation or policy change can invalidate
an outstanding plan without exposing values in its digest or UI.

Static frontends require an explicit choice: runtime configuration with one immutable
bundle, or separately built and attested variants within one release manifest. A test
bundle with baked-in endpoints cannot be claimed byte-identical to a different production
bundle. Multi-architecture builds similarly keep distinct checksums in the same release.

Promotion proceeds through environment-specific gates: artifact verification, capacity,
data compatibility, candidate readiness, smoke tests and traffic checks. A result from
one node cannot attest every replica. Persist the exact measurements and freshness
conditions used to admit the next environment.

Run schema migrations once under the existing Titan database/Kysera coordination
contract, not once per replica. Prefer expand/contract changes that allow old and new
application revisions to overlap. A release rollback must explicitly assess schema and
data compatibility; changing an application symlink cannot undo an irreversible migration.

## Placement and reserve capacity

Separate feasibility from ranking. A lightly loaded but incompatible machine is not
a candidate. This distinction also appears in [Nomad's scheduling model](https://developer.hashicorp.com/nomad/docs/concepts/scheduling/how-scheduling-works):
logical work and node allocations are separate, candidate constraints precede scoring,
and concurrent plans require admission conflict checks.

Feasibility checks include:

- Environment/pool permissions, node identity and incarnation, maintenance/quarantine state.
- OS, architecture, runtime version and required execution/storage/network capabilities.
- CPU, memory, disk, I/O and other declared resources, including outstanding reservations.
- Reachability of required services, data locality and storage attachment constraints.
- Failure-domain spread, anti-affinity, required co-location, ports and availability budget.
- Sufficiently fresh capacity and health evidence; missing measurements do not mean idle.

Rank only feasible candidates using a versioned, deterministic policy. Consider
effective headroom, pressure, locality, failure-domain spread, expected startup cost
and operational cost. Expose each score component and rejected constraint in the plan.
Stable tie-breaking prevents unexplained movement between equivalent nodes.

Resource accounting must distinguish managed reservations from unmanaged load. Do not
subtract both reservation and usage for the same workload twice. Reserve operating-system
headroom and account for temporary overlap during rollout. Refresh measured unmanaged
load and reject stale plans before committing reservations in the authoritative store.

A CPU percentage alone is insufficient: 90% on different core counts is not the same
capacity, and a memory-constrained service cannot be placed because CPU is idle. Distinguish
hard enforcement from advisory accounting. A node without the required isolation/resource
enforcement capability cannot host a workload whose policy requires that guarantee.

Warm reserve nodes are enrolled, authenticated, healthy, compatible and included in
inventory before an incident. Cold/unreachable machines are not immediately usable
capacity. Pool policy distinguishes emergency reserve from ordinary scaling capacity
and records when recovery consumes the reserve.

If no preferred node qualifies, a bounded emergency policy may allow temporary placement
on a different eligible node with a named expiry, resource ceiling and remediation task.
It must still satisfy security, data and minimum-availability constraints. Never silently
evict another critical service or exceed its quota to make the plan appear successful.

## Recovery without competing owners

Loss of RPC means loss of observation. A network partition may leave the original
application serving users or writing to its database. Starting a replacement single
writer from that evidence alone is unsafe. Kubernetes documents the same identity
hazard when [force-deleting an unreachable stateful instance](https://kubernetes.io/docs/tasks/run-application/force-delete-stateful-set-pod/).

The recovery sequence is explicit:

1. Detect a failed SLO or stale observation; classify process failure, host failure,
   dependency failure, control-plane failure or unknown connectivity.
2. Apply debounce, backoff and a shared failure budget. Quarantine a crashing revision
   or failed dependency instead of repeatedly moving every dependent application.
3. Obtain the authority required by the workload's state contract. Fence the previous
   writer where necessary; an expired controller lease alone does not prove that its
   process or external write stopped.
4. Select and atomically reserve a feasible replacement allocation.
5. Materialize verified artifacts and configuration. Recover/attach data only through
   the owning adapter and its declared recovery evidence.
6. Start and verify readiness, identity, dependencies and data consistency.
7. Publish routes/endpoints, drain old traffic where reachable, and confirm the change.
8. Record the observed result, release obsolete reservations and report any retained
   resources or unresolved effects. Cleanup is its own admitted action.

An old node returning from a partition is quarantined for reconciliation. It cannot
reclaim its old allocation or write authority just because it has the same address.
Avoid automatic failback oscillation; moving from an emergency placement to its final
home requires another plan and availability-budget check.

### State contracts

| Workload class                           | Required recovery evidence                                                                          |
| ---------------------------------------- | --------------------------------------------------------------------------------------------------- |
| Stateless request handler                | Verified artifact, ready replica, routing membership and duplicate-safe request behavior            |
| Queue/periodic worker                    | Lease/claim ownership, acknowledgment semantics and idempotency or deduplication for effects        |
| Single writer                            | Fencing proof accepted by the data/resource authority before another writer starts                  |
| Replicated database                      | Adapter-confirmed term/leader, replication position, quorum policy and old-primary fencing          |
| Backup-restored service                  | Verified backup/checksum, restore completion, replay position and explicit accepted data-loss bound |
| Node-local state with no recovery source | Placement blocked with a precise missing-data reason                                                |

RPO and RTO are declared objectives, not promises inferred from a container's health.
Measure achievable recovery against them. PostgreSQL, Redis, object storage and other
stateful services keep their own recovery protocols. Do not replace those protocols
with a generic volume copy or confuse monitoring replication with application-data replication.

## Scaling at the correct level

Titan PM controls local pool concurrency. Omnitron controls fleet allocations and
aggregate workload capacity. Each decision variable has one owner. The fleet policy
sets a per-allocation worker budget so independent local and global controllers cannot
multiply one another's scale decisions.

Use workload signals as well as resource pressure: latency percentiles, queue age/depth,
throughput, event-loop saturation, startup cost and error budget. Policies bound minimum
and maximum replicas, rate of change, stabilization windows, cooldown and total resource
cost. A growing queue caused by an unavailable database must not trigger unlimited workers.

Unknown/stale metrics block scale-down. Warm-up and readiness determine whether a new
replica is eligible for measurement. Multiple signals require explicit combination rules.
Kubernetes HPA provides a useful [reference for missing-metric conservatism and stabilization](https://kubernetes.io/docs/concepts/workloads/autoscaling/horizontal-pod-autoscale/);
Omnitron must expose its own decision inputs and limits rather than copying a percentage formula blindly.

Separate workload scale from node acquisition. First use suitable registered capacity;
then consume approved reserves. Provider-specific machine provisioning is optional and
requires its own cost, credential and failure-domain policy. Bare-metal, virtual and
container-based workloads remain expressible without a mandatory cloud integration.

## Durable operations and control-plane availability

An operation journal records immutable intent and every state transition. The minimal
record includes operation/step IDs, actor/delegation, target IDs and incarnations, desired
generation, artifact and argument digests, policy revision, resource reservations,
authority epoch, deadline, attempt, expected effects and recovery contract.

Persist admission, reservations, operation state and dispatch-outbox entry atomically.
A lost response is reconciled by operation ID. Retrying transport delivery may repeat a
request; the node's durable receipt makes that distinguishable from a new effect.
Do not promise exactly-once external execution: adapters must either deduplicate a
stable effect ID or determine its existing outcome before another attempt.

Useful operation states are `planned`, `admitted`, `executing`, `verifying`, `succeeded`,
`failed`, `cancelling`, `cancelled`, `compensating`, `compensated`, `unknown` and
`needs-intervention`. A deadline may produce `unknown`. Cancellation is a request until
the executor confirms what stopped. Compensation is a separate operation with its own
receipts and failures; it must not run concurrently with an unresolved effect.

Reuse and extract the existing Titan workflow mechanisms into an optional durable
execution facility as their storage/reconciliation contracts are implemented. Keep
the existing in-process PM facade compatible. Do not introduce another DAG runner in
Omnitron, another scheduler in Console, or a persistent-workflow claim backed by an in-memory map.

The authority store supports two explicit modes:

| Mode                      | Guarantee and limitation                                                                                                                                       |
| ------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Local / single controller | SQLite transactions and durable receipts on one authority; existing workloads survive controller restart, but authority-machine loss requires restore          |
| HA controller             | One authoritative transactional history, compare-and-swap admission, monotonic fencing epochs and certified failover semantics; no mutations without authority |

The HA provider can use an independently operated PostgreSQL deployment through
Titan database/Kysera when its replication/failover configuration meets that contract.
An optional established consensus-backed provider is another implementation, not a
new mandatory dependency. Do not build a second simplified consensus algorithm around
the current election class. Store capabilities must be verified, not inferred from
the product name. [etcd's documented guarantees](https://etcd.io/docs/v3.6/learning/api_guarantees/)
illustrate why authoritative reads, committed writes and asynchronous watches are different guarantees.

Execution epochs are checked and persisted by the effect owner. A valid JWT or a
controller's belief that it is leader is insufficient. State rollback on a node,
authority-store restore and node re-enrollment require explicit incarnation/epoch
reconciliation. The control store and its bootstrap path cannot depend solely on an
application that the unavailable controller must first start.

## Security and agent parity

Extend the existing role/permission system; do not add an agent-only allowlist. Resource
scope covers organization/project, environment, workload, node pool, data resource and
operation. Capabilities distinguish reading, designing, admitting, executing, cancelling,
promoting, draining, changing policy and destroying retained resources.

An agent's effective authority is the intersection of its owner's current rights,
delegation, run-specific scope, expiry, budgets and policy. Approval satisfies a required
approval condition; it does not grant a missing permission. Recheck authority at admission
and every privileged effect. Revocation prevents future effects and invokes the declared
containment policy for work already executing. See [delegated operations](/docs/omnitron/agents).

The node enrolls with a machine identity and a verified trust relationship. Bind transport
identity to sender node ID; authenticate election/control messages; scope short-lived
execution credentials to audience, resources, operation and epoch. Replace reliance on
anonymous membership claims and broadly reusable node-signing secrets before HA automation.

Plans expose effects and resource identities without secret values. Secrets are resolved
by the privileged broker at execution and redacted from logs, receipts and exports. An
arbitrary SSH/script operation carries the OS authority of its target account; it cannot
inherit a narrow “repair one service” permission merely by being wrapped in an API call.

Audit intent precedes effects and outcome follows verification. A searchable audit copy
may be temporarily unavailable if the durable authority journal records the operation;
failure of the authoritative journal blocks new effects. Export security audit to an
independent sink under retention policy. A local append-only table is not tamper-proof
against a host administrator.

## Product interaction

The Console environment workspace should keep scope, desired generation and current
health visible without duplicating large headings. Offer logical workload, physical
placement, data dependency and route views over the same resource model. Line styles
and inspector evidence distinguish declared, observed and unknown relationships.

Selecting an environment shows its verified release, availability, capacity/reserve
pressure and active operations. Editing a workload opens a compact inspector for
replicas, resource requirements, runtime, bindings and policy. Dragging a placement
creates a draft proposal; only a reviewed, admitted plan can execute it.

The review presents chosen and rejected nodes, resource deltas, affected traffic,
availability impact, data-recovery requirements and compensation limits. Execution
shows real step progress, timestamps and receipts. Unknown work remains visible with
the next safe diagnostic action. There is no fake percentage for an operation whose
duration cannot be estimated.

Use Prism/MUI focus management, responsive inspectors and keyboard commands. The same
semantic operation IDs and structured errors drive CLI JSON and agent tools. Saved
views, keyboard shortcuts and topology layout are user preferences; they do not mutate
infrastructure. Prefer incremental event updates and coalesced inventory reads over
polling the whole fleet separately from every panel.

## Distributed ingress and service publication

The model supports gateways, origin services, protection workers and publication
controllers as ordinary typed workloads and relationships. Separate a traffic edge
from a control/publication edge: an Onionbalance publisher is not an HTTP proxy in the
request path. Its [design](https://tpo.pages.torproject.net/onion-services/onionbalance/design/)
informs that adapter boundary. Tor introduction-point defenses and an HTTP proof-of-work
challenge are different mechanisms, with different metrics and policies.

Route adapters declare safe update/rollback, propagation and readiness checks. Reserve
capacity and independent failure domains apply to protection workers too. Protection
load must not consume the control plane's last resources or silently bypass origin
isolation. Each optional adapter reports the capabilities it actually implements.

## Delivery gates

| Stage                            | Required result before enabling it                                                                                                                                    |
| -------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Observation and target workspace | Node-qualified evidence; unknown states; closed schemas; concurrent-edit protection; no implicit runtime mutation                                                     |
| Durable execution                | Shared workflow kernel, persisted steps/outbox/receipts, explicit uncertainty, node-authenticated effect ownership, allocation-safe names and scoped remote lifecycle |
| Stateless placement and repair   | Reservations, fresh capacity, failure domains, readiness/routing, fenced execution, bounded reserve use and restart recovery                                          |
| Stateful recovery                | Per-service adapters; tested backups/replication; RPO/RTO evidence; old-writer fencing; refusal when evidence is insufficient                                         |
| HA authority                     | Certified store semantics, multi-controller contention tests, durable epochs, partition handling and independent bootstrap/restore                                    |
| Adaptive scaling and acquisition | Stabilized multi-signal policy, local/global budget coordination, cost limits, scale-down safety and explainable decisions                                            |

Default to observation and reviewed plans. Enable bounded automatic actions per
environment only after the corresponding adapter and policy pass their gate. An
unsupported operation remains an explicit capability refusal, not an incomplete
“successful” deployment.

The verification matrix must include lost requests/responses, duplicate delivery,
crash before/after commit, node replacement at the same address, partitions and stale
leaders, clock skew, full disks, expired/revoked grants, stale metrics, port conflicts,
concurrent reservations, bad artifacts, failing migrations, late handlers, failed
compensation and return of an old writer. Exercise the same scenarios through CLI and
Console, and prove unrelated applications remain running during control-plane upgrades.

An initial implementation is releasable when its advertised capability set is verified,
its unsupported cases are explicit, and recovery procedures reproduce the stored
evidence. Industrial quality is the strength and clarity of those guarantees, not the
number of controls or adapters exposed at once.

## Multi-project deployment audit and design

The control plane uses a project registration for ownership, a stack for the intended environment, a manifest digest for the artifact, and a node identity and owner epoch for execution. Human operators and agents use the same authenticated Netron methods. A target can host several projects. Shared infrastructure needs an explicit owner and consumers.

| Boundary           | Implemented behaviour                                                                                                                                                                                                                       | Operational implication                                                                                                                                                                                                                                       |
| ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Source and build   | A submitted build pins project and SDK revisions. The controller schedules distinct projects in bounded slots. Each job owns its logs, cancellation and build directory.                                                                    | Another team can submit while a build is running. A duplicate submission for the same project identifies its existing job.                                                                                                                                    |
| Builder capacity   | Jobs wait in FIFO order on a controller with full slots. The job record survives controller replacement.                                                                                                                                    | CLI and Console expose queued, running, stopped, failed, interrupted and completed outcomes. A missing completion receipt requires inspection.                                                                                                                |
| Artifact transport | A controller build can publish its checked artifacts to a configured S3-compatible store and record the index digest. A pull verifies files against the index and admits the manifest before exposing the release in the receiving catalog. | A builder and a deployment controller must have access to the same store. A loopback MinIO endpoint on each host can refer to different buckets.                                                                                                              |
| Target admission   | The release check and stack start share source, gate, runtime and placement checks. A frozen application supplies its own SDK.                                                                                                              | Test and production may have different placement. Diagnose missing release bytes, differing source inputs and an incompatible runtime before changing processes.                                                                                              |
| Mutation ownership | The controller obtains renewable deployment leases for affected nodes. An ambiguous remote command retains its execution fence across connection loss.                                                                                      | Distinct nodes admit independent writers. Native mutations on a shared host currently serialize. An immutable deployment can wait for an identified holder while releasing all other target leases. A timeout requires an observation of the prior operation. |
| Data and rollback  | Migration checksums, deployment history, runtime launch evidence and application readiness are separate observations.                                                                                                                       | Reapply a previous verified artifact only after checking its data compatibility. An additive migration may survive rollback. A legacy process with no attestation is shown as unreported.                                                                     |
| Supervision        | A resident owner keeps application processes alive through compatible backend handoff. Owner replacement has its own lifecycle and capability checks.                                                                                       | An application team can deploy to a compatible node while another team qualifies a platform update. Replacement of a shared owner requires its own reviewed maintenance window.                                                                               |

### Isolation on a shared host

The receiving node keeps the infrastructure declaration, observed state and service credentials for each project and stack. Current controllers request only the database, cache and storage credentials declared for the deployment. A credential request without a scope is refused on a node hosting several allocations. Container names, overlapping published ports and a shared mount source with a writable user conflict with another registered allocation. Shared read-only mounts remain usable. A verified single-scope infrastructure instance inherited at handoff can continue serving its original stack.

A build slot isolates scheduling, cancellation and the build directory. Registered project code still executes with the builder OS account. Use separate builders and test resources for mutually untrusted projects. Controller operator roles currently span registered projects. Node provisioning and writable infrastructure still use a host-level lease. Retained application recovery in an enrolled cluster has separate signed operations. Its deployment adapter requires recipient fencing before it can admit a new remote release.

### Requirements for independent operations on a shared host

The current exclusive native write lease also protects host-global operations such as provisioning an execution service or changing ingress. Shortening it safely requires the receiver to prove every mutation's write set. Application slots and credentials belong to a project and stack. A published port belongs to a bound address and protocol. A database migration belongs to a physical database and migration namespace. A shared system service or volume has one declared owner. Multiple controllers must acquire those resources in a deterministic order and revalidate ownership at the receiver before applying a mutation.

A deployment operation should retain its authenticated initiating principal, project and stack, immutable manifest and previous selection, observed owner epoch, resource set and per-phase receipts. Recovery first observes each completed effect. A worker can retry a read or a content-addressed upload. Repeating an uncertain migration, process stop or traffic switch requires reconciliation with the owning service. Clients can safely reconnect to the same operation ID.

The current build runs on the controller or in an explicitly configured native CI builder. Routing a build automatically to an enrolled Linux worker requires a declared build environment, trust policy, resource limits and a portable source revision. Development machines may be the only source of a private commit. A worker must verify the supplied Git revision and registry lock before executing project code in its build environment. Working application credentials must stay outside that environment.

### Acceptance scenarios

- Submit two projects, cancel a third queued job, and observe independent progress and exit states in CLI and Console.
- Replace the controller after a final manifest has been written and before a job result is delivered. Recover the result from the manifest. If the manifest is absent, surface interruption.
- Attempt a second write to an occupied target. Report the holder and scope without issuing another mutation. A deployment to a disjoint target can proceed.
- Drop the RPC link after a state-changing request and before its response. Retain the fence, reconnect, and inspect the recorded effect before continuing.
- Exercise forward and reverse transitions on the actual Node executable and ABI. Verify the admitted artifact, application readiness, database migration ledger and restored traffic independently.

---

---
id: "omnitron/services-reference"
title: "Services reference"
description: "Every RPC service exposed by the daemon — names, methods, roles."
---

# Services reference

The Omnitron daemon exposes the RPC services below. Some are conditional
on enabled modules or daemon configuration. All share the same auth model
(see [Daemon / Auth flow](/docs/omnitron/daemon#auth-flow)) and address the
same socket / TCP / HTTP planes.

This page is the canonical inventory: service name, registered
Netron service id, purpose, and key methods.

Re-check this page against the source with
`node scripts/check-services-reference.mjs <path-to-omni>` — it
reads `@Service`/`@Public` out of `src/**/*.rpc-service.ts` and
diffs them against the index table below.

## Quick index

| File                               | Service id (Netron)  | Methods | Role gate (typical)                                                                                                                  |
| ---------------------------------- | -------------------- | ------- | ------------------------------------------------------------------------------------------------------------------------------------ |
| `daemon/backend.rpc-service.ts`    | `OmnitronBackend`    | 2       | control-plane read (status); local Unix admin (control address)                                                                      |
| `access.rpc-service.ts`            | `OmnitronAccess`     | 5       | viewer (effective permissions) + admin (accounts)                                                                                    |
| `terminal.rpc-service.ts`          | `OmnitronTerminal`   | 9       | admin + live human session / trusted local CLI                                                                                       |
| `alert.rpc-service.ts`             | `OmnitronAlerts`     | 8       | viewer (read) + operator (mutate)                                                                                                    |
| `audit.rpc-service.ts`             | `OmnitronAudit`      | 2       | admin                                                                                                                                |
| `auth.rpc-service.ts`              | `OmnitronAuth`       | 7       | mixed (5 allowAnonymous; 2 authenticated)                                                                                            |
| `backup.rpc-service.ts`            | `OmnitronBackups`    | 12      | viewer (read) + admin (mutate)                                                                                                       |
| `deploy.rpc-service.ts`            | `OmnitronDeploy`     | 4       | viewer (history) + operator (deploy/rollback)                                                                                        |
| `discovery.rpc-service.ts`         | `OmnitronDiscovery`  | 3       | viewer                                                                                                                               |
| `event-broadcaster.rpc-service.ts` | `OmnitronEvents`     | 3       | viewer (subscribe, unsubscribe, stats)                                                                                               |
| `fleet.rpc-service.ts`             | `OmnitronFleet`      | 7       | Viewer reads and operator inventory changes                                                                                          |
| `health-check.rpc-service.ts`      | `OmnitronHealth`     | 4       | viewer                                                                                                                               |
| `infrastructure.rpc-service.ts`    | `OmnitronInfra`      | 19      | viewer (read) + operator (start/stop/remove)                                                                                         |
| `kubernetes.rpc-service.ts`        | `OmnitronKubernetes` | 9       | admin                                                                                                                                |
| `log-collector.rpc-service.ts`     | `OmnitronLogs`       | 4       | viewer                                                                                                                               |
| `node-manager.rpc-service.ts`      | `OmnitronNodes`      | 38      | viewer (read), operator (operations), admin (historical repair and managed kernel rollout)                                           |
| `pipeline.rpc-service.ts`          | `OmnitronPipelines`  | 9       | admin + live human session / trusted local CLI                                                                                       |
| `project.rpc-service.ts`           | `OmnitronProject`    | 44      | Viewer reads and reviews, operator starts/stops, administrator manages registrations, machine credentials run the recipient protocol |
| `release.rpc-service.ts`           | `OmnitronRelease`    | 12      | viewer (read), operator (build/attest), admin (prune)                                                                                |
| `secrets.rpc-service.ts`           | `OmnitronSecrets`    | 6       | admin                                                                                                                                |
| `sync.rpc-service.ts`              | `OmnitronSync`       | 5       | authenticated control plane; historical repair excludes operators                                                                    |
| `system-info.rpc-service.ts`       | `OmnitronSystemInfo` | 1       | viewer                                                                                                                               |
| `telemetry.rpc-service.ts`         | `OmnitronTelemetry`  | 1       | Authenticated relay statistics                                                                                                       |
| `trace-collector.rpc-service.ts`   | `OmnitronTraces`     | 5       | viewer (read); operator/admin (ingestion)                                                                                            |

Cluster participation is exposed on controllers, workers and standalone daemons:

| File                             | Service id (Netron) | Methods | Role gate                                       |
| -------------------------------- | ------------------- | ------- | ----------------------------------------------- |
| `cluster/cluster.rpc-service.ts` | `OmnitronCluster`   | 3       | authenticated reads and administrator step-down |

Plus the core supervisor:

| File                           | Service id       | Methods | Role gate                                                                       |
| ------------------------------ | ---------------- | ------- | ------------------------------------------------------------------------------- |
| `daemon/daemon.rpc-service.ts` | `OmnitronDaemon` | 29      | mixed — see [Daemon](/docs/omnitron/daemon#rpc-surface--omnitrondaemon-service) |

The daemon exposes its role-protected metrics facade and imports the Titan health service:

| File                                                | Service id        | Methods | Role gate                                                                                                                 |
| --------------------------------------------------- | ----------------- | ------- | ------------------------------------------------------------------------------------------------------------------------- |
| `apps/omnitron/src/services/metrics.rpc-service.ts` | `OmnitronMetrics` | 7       | Viewer/control-plane reads (including `getStorageStatus`); operator/control-plane `flush`; admin `cleanup` and `evictApp` |
| `packages/titan-health/src/health.rpc-service.ts`   | `Health@1.0.0`    | 7       | public `live` and `ready`, 5 detailed methods require control-plane read access                                           |

`OmnitronMetrics` requires authorization for every method. Use `omnitron doctor` to check which diagnostic methods a particular daemon exposes to an unauthenticated connection. It reports observed exposure as `auth.anonymous-surface`.

## Naming convention

Most service ids use the `Omnitron<Subsystem>` PascalCase shape with no
version suffix. `Health@1.0.0` is the exception, and it is not going to
be renamed: the id is Netron's own versioned form, declared by
`@omnitron-dev/titan-health` for every application that imports the
module, so it is not omnitron's to change. Query it with the suffix
included.

## What `availableServices` lists

When a `query_interface` fails, the daemon logs the ids it does have.
That list is a **superset** of this page: managed applications register
their own services into the same Netron registry. An application can
register `TransformWorker` alongside the daemon's own services.
A name in that list is not evidence of a daemon service, and its absence
from this page is not evidence of a gap.

Query services via Netron's standard interface mechanism:

```typescript
import type { IProjectRpcService } from '@omnitron-dev/omnitron/dto/services';

const peer = await netron.connect('unix://~/.omnitron/daemon.sock');
const project = await peer.queryInterface<IProjectRpcService>('OmnitronProject');
const projects = await project.listProjects();
```

The CLI and webapp do exactly this under the hood. Typed client
interfaces are published from the `@omnitron-dev/omnitron/dto/services`
subpath (`IDaemonService`, `IProjectRpcService`, `IOmnitronAuthService`,
`IOmnitronLogsService`, `IInfraService`); other services can be queried
untyped via `queryInterface(serviceId)`.

## Per-service summary

Each subsection lists the methods you can call directly. For full
parameter and return types, refer to `src/services/<name>.rpc-service.ts`
and the `src/shared/dto/` types it imports.

### `OmnitronAlerts` — `alert.rpc-service.ts`

Alert rules + delivery for operator-defined thresholds (metric
exceeds value, app crashed N times, etc.).

| Method                                        | Effect                                            |
| --------------------------------------------- | ------------------------------------------------- |
| `getRules()`                                  | All alert rules                                   |
| `createRule(...)`                             | Define a new rule                                 |
| `updateRule({id, updates})`                   | Patch a rule                                      |
| `deleteRule({id})`                            | Remove a rule                                     |
| `getEvents({ruleId?, status?, limit?})`       | Fired-alert events                                |
| `acknowledgeAlert({alertId, acknowledgedBy})` | Mark as acknowledged                              |
| `getActiveAlerts()`                           | Currently firing, unresolved alerts               |
| `getSummary()`                                | Aggregate counts (open / acknowledged / resolved) |

### `OmnitronAuth` — `auth.rpc-service.ts`

User authentication, session management, JWT issuance.

| Method                                       | Effect                                    |
| -------------------------------------------- | ----------------------------------------- |
| `signIn(credentials)`                        | Begin a session; returns JWT + session id |
| `validateToken({token})`                     | Verify a JWT; returns claims              |
| `validateSession({sessionId})`               | Verify a session id                       |
| `refreshSession({sessionId})`                | Renew session expiry                      |
| `signOut({sessionId})`                       | Drop session                              |
| `getActiveSessions()`                        | List the caller's own active sessions     |
| `changePassword({oldPassword, newPassword})` | Self-service password change              |

`signIn`, `validateToken`, `validateSession`, `refreshSession`, and
`signOut` are all `allowAnonymous` (the session id / token is the
credential). `getActiveSessions` and `changePassword` are
`@Public({ auth: true })` — any authenticated caller, operating on
its own identity (not admin-gated).

### `OmnitronBackups` — `backup.rpc-service.ts`

Database backup, restore, scheduling.

| Method                                                  | Effect                                                                              |
| ------------------------------------------------------- | ----------------------------------------------------------------------------------- |
| `createBackup({database, compress?})`                   | One-shot backup of one database                                                     |
| `createArtifactBackup({project, stack, kind, bucket?})` | Capture one stack’s `storage-objects` (explicit bucket) or `tor-keys`               |
| `createAllBackups(...)`                                 | Back up every registered database                                                   |
| `createFullBackup(...)`                                 | Full backup including non-database state                                            |
| `listBackups({database?})`                              | List available backups                                                              |
| `restoreBackup({backupId})`                             | Restore from a backup                                                               |
| `deleteBackup({backupId})`                              | Drop a backup file                                                                  |
| `setSchedule({database, cron})`                         | Configure recurring backups                                                         |
| `getSchedule({database})`                               | Read current schedule                                                               |
| `listSchedules()`                                       | Every configured schedule                                                           |
| `getBackupStatus()`                                     | Effective database transfer budget, schedule outcomes, next runs and stack coverage |
| `removeSchedule({database})`                            | Drop a schedule                                                                     |

Driven by `omnitron backup` commands. Reads (`listBackups`,
`getSchedule`, `listSchedules`, `getBackupStatus`) gate on `viewer`; every mutation
requires `admin`.

### `OmnitronDeploy` — `deploy.rpc-service.ts`

Compatibility access to legacy per-app operation records.

| Method                                              | Effect                                                                              |
| --------------------------------------------------- | ----------------------------------------------------------------------------------- |
| `deployApp({app, version, strategy?, deployedBy?})` | Refuses: use the release/stack deployment workflow                                  |
| `rollback({app, deployedBy?})`                      | Refuses before any mutation; select the previous release through the stack workflow |
| `getHistory({app?, limit?})`                        | Retained legacy operation records; labels are not artifact evidence                 |
| `listDeployableApps()`                              | Empty list: no per-app artifact executor is installed                               |

Historical strategy labels are retained as recorded requests, not evidence of an implemented rollout.

### `OmnitronDiscovery` — `discovery.rpc-service.ts`

Discovery of Omnitron-managed targets across Docker and SSH.

| Method                   | Effect               |
| ------------------------ | -------------------- |
| `discoverContainers()`   | Scan local Docker    |
| `discoverNodes({hosts})` | SSH-scan given hosts |
| `scanAll()`              | Combined scan        |

Used by `omnitron discover`.

### `OmnitronEvents` — `event-broadcaster.rpc-service.ts`

Cross-process event bus over Netron. Webapp subscribes here for
live UI updates.

| Method                       | Effect                                                |
| ---------------------------- | ----------------------------------------------------- |
| `subscribe({channels, ...})` | Register a subscriber; returns `subscriberId`         |
| `unsubscribe()`              | Drop the subscription owned by the calling connection |
| `getStats()`                 | Subscriber count                                      |

All three methods admit viewers. Publication is internal to the daemon.

### `OmnitronFleet` — `fleet.rpc-service.ts`

Cross-node fleet operations: register nodes, drain, set roles.

| Method                       | Effect                     |
| ---------------------------- | -------------------------- |
| `listNodes()`                | All registered nodes       |
| `getNode({nodeId})`          | Inspect one node           |
| `getSummary()`               | Aggregate health / counts  |
| `registerNode(registration)` | Add a node to the fleet    |
| `removeNode({nodeId})`       | Remove                     |
| `setRole({nodeId, role})`    | Promote / demote a node    |
| `drainNode({nodeId})`        | Drain workloads off a node |

Reads (`listNodes`, `getNode`, `getSummary`) gate on `viewer`;
mutations (`registerNode`, `removeNode`, `setRole`, `drainNode`)
on `operator`. The scheduler records local liveness against its own registration.

### `OmnitronHealth` — `health-check.rpc-service.ts`

Active health probes — runs HTTP / TCP / DB checks on demand.

| Method                       | Effect                        |
| ---------------------------- | ----------------------------- |
| `checkApp({appName, port?})` | Probe one app                 |
| `checkApps()`                | Probe every app               |
| `checkInfrastructure()`      | Probe Postgres / Redis / etc. |
| `checkAll()`                 | Full platform health report   |

### `OmnitronInfra` — `infrastructure.rpc-service.ts`

Provisioned infrastructure inventory, container control and host-owned ingress.

| Method                                                    | Effect                                                                           |
| --------------------------------------------------------- | -------------------------------------------------------------------------------- |
| `getState()`                                              | Full infrastructure state (containers, env, networks)                            |
| `listContainers()`                                        | All managed containers                                                           |
| `getConnectionInfo({service})`                            | Resolved host / port / creds for a logical service                               |
| `getContainerLogs({container, tail?})`                    | Tail one container's logs                                                        |
| `startContainer({container})`                             | Start a managed container                                                        |
| `stopContainer({container})`                              | Stop a managed container                                                         |
| `removeContainer({container})`                            | Remove a managed container                                                       |
| `reviewStack`                                             | Inspect an existing stack infrastructure for preservation without reconciling it |
| `provisionStack` / `inspectHostServices`                  | Reconcile or inspect infrastructure on the executing host                        |
| `configureHostIngress` / `getHostIngress`                 | Configure ingress ownership or read configured/observed state                    |
| `checkHostIngress` / `validateHostIngress`                | Inspect the proposed change and validate publication inputs                      |
| `applyHostIngress` / `removeHostIngress`                  | Publish or remove an owner’s ingress with generation checks                      |
| `restoreHostIngressOwner` / `rollbackHostIngressTakeover` | Restore an owner or reverse a takeover using the expected generation             |
| `recoverHostIngress`                                      | Recover an interrupted ingress publication                                       |

Reads gate on `viewer`; `startContainer`, `stopContainer` and
`removeContainer` on `operator`. Ingress configuration, takeover rollback and
recovery require `admin` or trusted `service_role`; check/apply/remove/owner
restore use control-plane roles. Read the generated API signature and the
method’s declared gate before composing an operational workflow.

### `OmnitronKubernetes` — `kubernetes.rpc-service.ts`

Kubernetes integration.

| Method                                          | Effect             |
| ----------------------------------------------- | ------------------ |
| `listPods({namespace?, labelSelector?})`        | List pods          |
| `getPod({name, namespace?})`                    | One pod            |
| `deletePod({name, namespace?})`                 | Delete pod         |
| `getPodLogs({name, namespace?, tail?})`         | Tail logs          |
| `listDeployments({namespace?})`                 | List deployments   |
| `scaleDeployment({name, replicas, namespace?})` | Scale              |
| `restartDeployment({name, namespace?})`         | Rolling restart    |
| `listServices({namespace?})`                    | List services      |
| `execInPod({pod, command, namespace?})`         | Run command in pod |

Driven by `omnitron k8s ...`. Every method requires `admin`.

### `OmnitronLogs` — `log-collector.rpc-service.ts`

Log query + streaming.

| Method                                                                               | Effect                                          |
| ------------------------------------------------------------------------------------ | ----------------------------------------------- |
| `queryLogs({app?, level?, search?, labels?, traceId?, from?, to?, limit?, offset?})` | Filter logs                                     |
| `getLogStats()`                                                                      | Per-app log size + rotation stats               |
| `getIngestionStats()`                                                                | Ingested / dropped totals and buffer depth      |
| `streamLogs({app?, level?, search?, tail?, since?})`                                 | Last `tail` entries, oldest-first (default 100) |

`omnitron logs` calls `queryLogs` or `streamLogs`. Note the
full-text filter is `search`, not `grep`, and `streamLogs` is a
poll — it returns a page ending at now, it does not hold a
subscription open.

### `OmnitronNodes` — `node-manager.rpc-service.ts`

Node inventory, health history, bundle activation, managed rollout and recovery include 38 public methods.

| Method category           | Methods                                                                                                              |
| ------------------------- | -------------------------------------------------------------------------------------------------------------------- |
| Inventory                 | `listNodes`, `getNode`, `addNode`, `updateNode`, `removeNode`                                                        |
| Checks                    | `checkNodeStatus`, `checkAllNodes`, `triggerNodeCheck`                                                               |
| History                   | `getCheckHistory`, `getUptimeBar`, `getNodeHealthSummaries`                                                          |
| Misc                      | `listSshKeys`, `getCheckConfig`, `setCheckConfig`, `getHistoryConfig`                                                |
| Remote daemon             | `getNodeDaemonStatus`, `getNodeDaemonHealth`, `getNodeDaemonMetrics`, `getNodeSystemSnapshot`, `getNodeClusterState` |
| Mesh and telemetry        | `getMeshStatus`, `getNodeRelayStats`, `getNodeSyncStatus`, `getNodeIndicators`                                       |
| Historical repair (admin) | `reconcileNodeMetrics({nodeId, restore?, after?, through?, epoch?})`                                                 |
| Bundle operations         | `installBundleOnNode`, `activateBundleOnNode`, `recoverBundleOnNode`                                                 |
| Upgrades                  | `planUpgrade`, `upgradeNode`, `upgradeNodes`, `getUpgradeProgress`, `cancelUpgrade`                                  |
| Managed backend rollout   | `listKernelBuilds`, `planKernelRollout`, `startKernelRollout`, `getKernelRollout`, `reconcileKernelRollout`          |

Includes the uptime-bar machinery the webapp uses to draw
green/yellow/red availability bars.

### `OmnitronPipelines` — `pipeline.rpc-service.ts`

Xec and shell automation, shared by Console, CLI and local MCP. All methods require `admin`; a human session is revalidated at the service boundary. Mutations must record an audit intent before taking effect.

| Method                                               | Effect                                                                           |
| ---------------------------------------------------- | -------------------------------------------------------------------------------- |
| `createPipeline(definition)`                         | Define a pipeline                                                                |
| `updatePipeline({id, definition, expectedRevision})` | Conditionally replace the reviewed definition, preserving its ID and run history |
| `getPipeline({id})`                                  | Read one pipeline, its revision and conditional-update capability                |
| `listPipelines()`                                    | List all                                                                         |
| `deletePipeline({id})`                               | Drop a pipeline                                                                  |
| `executePipeline({id, expectedRevision, params?})`   | Admit a reviewed revision; reject a definition changed since review              |
| `cancelRun({runId})`                                 | Request cancellation and await the owned execution                               |
| `getRunStatus({runId})`                              | One run with bounded per-step output and diagnostics                             |
| `listRuns({pipelineId?, limit?})`                    | Bounded history summaries; excludes step output and parameters                   |

See [Xec automation](/docs/omnitron/automation) for execution limits, revision review, cancellation and failure recovery. A `kind: "release"` step carries a `PipelineReleaseTarget`: registered project, stack, app, producer artifact identity and package-relative compiled entry. The verified resident owner supplies the executable. Per-attempt `PipelineReleaseExecution` records keep observed installation and owner evidence separate from the saved definition. See [pinned release steps](/docs/omnitron/automation#execute-a-pinned-release). A child process is not a sandbox for untrusted agent code.

### `OmnitronProject` — `project.rpc-service.ts`

`refreshTopologyMetadata({ project, stack? })` is admin-only: it captures bootstrap
declarations in bounded child processes and persists the last successful snapshot.
It returns `refreshed` and per-application `failures`, retaining previous metadata
on failure. Viewer `getTopology` and `getTopologyWorkspace` report declaration
source, capture time and coverage warnings without importing bootstrap code. See
[declaration coverage](/docs/omnitron/topology#declaration-coverage).

`inspectInstalledArtifacts({ project, stack })` is an explicit viewer operation. It compares retained archive hashes and byte counts with the release catalog, reports modified installed entry points and framework build metadata, and never creates launch receipts. Its node counterpart, `OmnitronDaemon.inspectInstalledArtifacts({ names })`, accepts canonical owner inventory keys and admits human readers plus authenticated cluster service identities. Both are read-only, bounded, and absent from status polling. See [release evidence](/docs/omnitron/releases#understand-what-is-built-and-what-is-running).

`diagnoseStack({ project, stack })` is a viewer operation. It returns timestamped
prerequisite checks, planned changes and explicit unverified coverage without
provisioning or importing application bootstraps. Imported configuration names
resolve to the same runtime scope as Start; the report retains the public project
and stack names. Concurrent readers share an in-flight inspection. See
[readiness before start](/docs/omnitron/console#stacks).

The project and stack registry exposes release checks, account operations, infrastructure ownership and per-node lifecycle receipts.

| Method category              | Methods                                                                                                                                                                                                          |
| ---------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Projects                     | `listProjects`, `getProject`, `scanRequirements`, `addProject`, `updateProject`, `removeProject`, `importConfiguration`, `detachConfiguration`                                                                   |
| Project apps and topology    | `getProjectApps`, `getTopology`, `getTopologyInventory`, `refreshTopologyMetadata`                                                                                                                               |
| Target workspace             | `getTopologyWorkspace`, `saveTopologyDraft`, `planTopology`, `applyTopologyPlan`                                                                                                                                 |
| Stacks                       | `listStacks`, `getStack`, `getStackStatus`, `diagnoseStack`, `inspectInstalledArtifacts`, `createStack`, `deleteStack`, `reviewStackStart`, `startStack`, `reviewStackStop`, `stopStack`, `resumeInstalledStack` |
| Infrastructure ownership     | `refreshInfrastructureOwnership`, `inspectStackHost`, `probeLeftovers`, `stackMaintenance`                                                                                                                       |
| Stack accounts               | `censusStackAccounts`, `createStackAccount`, `removeStackAccount`, `showStackAccount`                                                                                                                            |
| Recipient lifecycle protocol | `inspectNodeStackStop`, `prepareNodeStackStop`, `executeNodeStackStop`, `prepareNodeInstalledResume`, `executeNodeInstalledResume`, `getNodeInstalledResumeReceipt`                                              |
| Release inspection           | `checkRelease`, `getDeployProgress`                                                                                                                                                                              |

`updateProject({ name, path })` changes the registered location while preserving the
project name, registration history, enabled-stack settings and workspace namespace.
The directory basename is not a rename request. Stop running/starting stacks before
moving a project to another release directory. Configuration is reloaded from the
new path on the next start. A rejected path leaves the registration unchanged.
Use an immutable release directory for updates; a mutable `current` symlink can
otherwise make process definitions and installed package identities disagree.

Roles: all reads (`listProjects`, `getProject`, `scanRequirements`,
`getProjectApps`, `getTopology`, `getTopologyInventory`, `getTopologyWorkspace`, `listStacks`, `getStack`, `getStackStatus`, `diagnoseStack`, `reviewStackStop`) gate on
`viewer`; `startStack`, `stopStack` and `resumeInstalledStack` on `operator`; project CRUD
(`addProject`, `updateProject`, `removeProject`) and stack CRUD
(`createStack`, `deleteStack`) on `admin`. Target mutations (`saveTopologyDraft`,
`planTopology`, `applyTopologyPlan`) also require `admin`. They persist target intent,
not executable deployment configuration. See the [workspace contract](/docs/omnitron/topology).

`reviewStackStop({ project, stack, keepInfrastructure? })` and
`stopStack({ project, stack, review?, keepInfrastructure? })` share the same strict
boolean mode. Omitted or `false` removes the stack's containers after applications
stop; `true` retains them and their ownership. Strings, numbers and `null` are
invalid. The mode is part of the review digest, node plan and signed cluster
permit; switching it invalidates the review. A confirmed retention receipt records
`keepInfrastructure: true`, `keptContainers` and an empty `removedContainers` list.
Public `stopOperation.keepInfrastructure` carries the selected mode across page
and backend reloads. Retention preserves release selection but disables automatic
application startup, just like a full stop. It does not add an administrator-only
execution path or grant a viewer permission to mutate.

### `OmnitronSecrets` — `secrets.rpc-service.ts`

Encrypted secret CRUD.

| Method              | Effect                    |
| ------------------- | ------------------------- |
| `get({key})`        | Decrypt and read one      |
| `set({key, value})` | Encrypt and store         |
| `delete({key})`     | Remove                    |
| `list()`            | List keys (values hidden) |

Always gated by `admin` role.

### `OmnitronSync` — `sync.rpc-service.ts`

Cross-daemon state synchronisation (service-to-service, used in
cluster mode).

| Method                                                      | Effect                                                                                 |
| ----------------------------------------------------------- | -------------------------------------------------------------------------------------- |
| `drainBuffer({limit?})`                                     | Pull pending sync data                                                                 |
| `ackDrained({ids})`                                         | Acknowledge what the caller has durably taken                                          |
| `getSyncStatus()`                                           | Current sync state                                                                     |
| `readUnverifiedMetrics({after?, through?, epoch?, limit?})` | Read a bounded page of retained historical metrics; admin or service identity          |
| `acknowledgeReconciliation({proofs})`                       | Settle retained evidence against payload-bound database proofs (service identity only) |

`drainBuffer` and `readUnverifiedMetrics` accept administrators and authenticated machines. Only `service_role` can acknowledge a drained batch or settle historical evidence. Administrators inspect and repair records through `OmnitronNodes.reconcileNodeMetrics` on the controller. Inspection is the default. Pass `restore: true` for repair. The controller binds each batch to its authenticated node connection. All Console read roles and the authenticated machine can call `getSyncStatus`.

Production replication uses the master's authenticated mesh connection to pull
`drainBuffer` batches from each slave. The master ingests a batch before calling
`ackDrained`; pending entries remain available for retry until acknowledged.
Rejected entries remain pending. Permanently discarded entries are logged as
errors and acknowledged explicitly.

Inspect pending entries, the last
acknowledgement, node reachability and mesh logs when diagnosing replication.
An unavailable reading is unknown, not an empty backlog.

### `OmnitronSystemInfo` — `system-info.rpc-service.ts`

Host inventory.

| Method                     | Effect                                                                                                                                                        |
| -------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `getSnapshot({ fresh? }?)` | Host CPU, memory, mounts, external interfaces, containers and running daemon. A fresh read waits for one shared collection when the cached snapshot has aged. |

The webapp uses this for the host-info card.

### `OmnitronTelemetry` — `telemetry.rpc-service.ts`

`getRelayStats()` returns relay statistics to authorized readers and machine peers on the master. Workers use the authenticated `OmnitronSync` protocol for log and metric replication. A controller acknowledges committed data. See [replication evidence](/docs/omnitron/cluster).

### `OmnitronCluster` — `cluster/cluster.rpc-service.ts`

Controllers, workers and standalone daemons report their participation. An enrolled controller additionally reports its PostgreSQL lease.

| Method              | Effect                                                                            |
| ------------------- | --------------------------------------------------------------------------------- |
| `getClusterState()` | Authority, reason, durable epoch, lease freshness, observed leader and peer count |
| `stepDown()`        | Demote locally and release this controller's epoch                                |
| `isLeader()`        | Whether this node holds a confirmed fresh lease                                   |

All reads require viewer/operator/admin or a control-plane service identity.
Step-down requires an administrator. The former
anonymous vote and heartbeat methods have been removed. A lease observation is
not a fence for remote effects; see [cluster coordination](/docs/omnitron/cluster).

### `OmnitronTraces` — `trace-collector.rpc-service.ts`

Distributed trace ingestion + query.

| Method                 | Effect                                             |
| ---------------------- | -------------------------------------------------- |
| `ingestSpan(span)`     | Push a single span                                 |
| `ingestBatch({spans})` | Push many                                          |
| `getTrace({traceId})`  | One trace                                          |
| `queryTraces(filter)`  | Filter by service / duration / status / time range |
| `getServiceMap()`      | Derived service-call topology                      |

## Auth role recap

| Role       | Members                       | Typical capability                                                    |
| ---------- | ----------------------------- | --------------------------------------------------------------------- |
| `viewer`   | `viewer`, `operator`, `admin` | Read-only inspection                                                  |
| `operator` | `operator`, `admin`           | Lifecycle + scale                                                     |
| `admin`    | `admin` only                  | Arbitrary execution, automation, SSH, shutdown, secrets, role changes |
| anonymous  | anyone with socket access     | `ping` and public sign-in/session endpoints                           |

Methods on the same service can vary in role:

- `OmnitronAuth.signIn` is anonymous;
  `OmnitronAuth.getActiveSessions` requires any authenticated caller
  (`@Public({ auth: true })`), operating on the caller's own identity
  — not admin-gated.
- `OmnitronProject.listProjects` is viewer;
  `OmnitronProject.startStack` is operator; `OmnitronProject.deleteStack`
  is admin.

## Querying a service from your code

```typescript
import { Netron } from '@omnitron-dev/titan/netron';
import { createNullLogger } from '@omnitron-dev/titan/module/logger';
import type {
  IProjectRpcService,
  IDaemonService,
} from '@omnitron-dev/omnitron/dto/services';

// `Netron` takes a logger — pass your application's, or
// `createNullLogger()` to discard Netron's own output. `create` also
// starts it, which `connect` requires.
const netron = await Netron.create(createNullLogger());
const peer   = await netron.connect('unix://~/.omnitron/daemon.sock');

const project = await peer.queryInterface<IProjectRpcService>('OmnitronProject');
const projects = await project.listProjects();

// Services without a published interface are queried untyped:
const deploy = await peer.queryInterface('OmnitronDeploy');
const legacyOperations = await deploy.getHistory({ app: 'sample/test/api', limit: 20 });
```

The same code works against a remote daemon by swapping the URL
(`tcp://server:9700`).

## Anti-patterns

- **Calling mutating methods over the public TCP plane without
  RBAC.** Operator-level methods change app state — expose only
  behind proper auth.
- **Long-running calls on RPC.** Some methods (deploy, backup
  restore) can run for minutes — prefer fire-and-forget patterns
  (start the run; poll status) over a long-held RPC.
- **Sending unauthenticated telemetry to the daemon.** Managed workers retain replication entries in their outbox. Monitor delivery through the registered node and its persistence receipts.
- **Treating `OmnitronSync` as a public API.** It's daemon-to-
  daemon plumbing. The shape may change between releases without
  notice.

## See also

- [Daemon](/docs/omnitron/daemon) — the `OmnitronDaemon` service + auth model
- [Architecture](/docs/omnitron/architecture) — where these services live
- [CLI](/docs/omnitron/cli) — most commands map directly to one method here
- [Console](/docs/omnitron/console) — webapp uses the same services

## Release and audit services

`OmnitronRelease` provides `preflight`, `list`, `get`, `getLog`, `builds`,
`getBuild`, `build`, `stopBuild`, `prune`, `attest`, `attestOnNode` and
`deployments`. Build execution and artifact state are separate from deployment
history. See [Releases and verification](/docs/omnitron/releases) for the workflow.

`OmnitronAudit` exposes `available` and `list`, both gated to administrators.
Check availability before interpreting an empty result as proof that no
operation took place. The audit trail records who requested control-plane
changes and their outcomes.

## Console administration and SSH

`OmnitronAccess`: `describe`, `listUsers`, `createUser`, `setRole`, `revokeSessions`. See [Console access](/docs/omnitron/access) for transactions, audit and revocation.

`OmnitronTerminal`: `capabilities`, `inspectHost`, `trustHost`, `open`, `list`, `read`, `write`, `resize`, `close`. See [SSH workspace](/docs/omnitron/terminal) for ownership, host verification and bounded binary I/O. Both services are registered on a master; they are not application user-management APIs.

## File workspace services

| File                             | Service id         | Methods | Role gate                                      |
| -------------------------------- | ------------------ | ------- | ---------------------------------------------- |
| `files/files.rpc-service.ts`     | `OmnitronFiles`    | 21      | admin + live human session / trusted local CLI |
| `files/file-node.rpc-service.ts` | `OmnitronFileNode` | 1       | authenticated mesh service identity            |

`OmnitronFiles` is the master-only administrator API: `roots`, `archiveFormats`, `list`, `stat`, `readText`, `readChunk`, `saveText`, `mkdir`, `rename`, `trash`, `trashList`, `restore`, `beginUpload`, `uploadChunk`, `uploadStatus`, `finishUpload`, `acknowledgeUpload`, `abortUpload`, `copy`, `transfers`, `controlTransfer`. Each call revalidates human authority; mutations and content reads require audit intent.

`OmnitronFileNode.invoke` runs on each node and accepts the v1 typed file protocol only for authenticated control-plane service peers. It implements OS access, private staging and durable write receipts. It is not a browser or application RPC proxy. See [File workspace](/docs/omnitron/files).

The six recipient lifecycle operations accept only an authenticated machine credential.
`stopStack` and `resumeInstalledStack` coordinate them on behalf of an operator.
Nodes verify ownership and admission before applying an effect. Their durable
receipts are available to the controller after a transport interruption.

---

---
id: "omnitron/terminal"
title: "SSH workspace"
description: "Multiple SSH tabs through Xec, with pinned host keys, live authorization and bounded sessions shared with CLI."
---

# SSH workspace

Open **SSH workspace** from the sidebar, or **SSH** on a registered node. Each tab creates an independent interactive shell using the SSH identity configured for that node. The workspace supports split view, terminal search, font controls, copy selection, a local read-only mode and review before pasting line breaks or control characters.

The service runs on a master with PostgreSQL-backed authentication and audit. The daemon owns the SSH connection. Credentials stay on the server. The browser renders the terminal through lazily loaded xterm.js; the CLI uses the same `OmnitronTerminal` RPC service. SSH runs through Xec's existing `NodeSSH` implementation, exposed by `@xec-sh/core/ssh`.

## Integration boundary

The terminal service uses Xec's shell, host-key and connection-lifecycle APIs.
It must not construct a separate SSH client or duplicate authentication and
connection policies. The Xec source candidate makes its connection reference a
lifecycle-event facade; raw transport execution stays private. SSH channel and
SFTP types are exported by Xec itself. Existing terminal event subscriptions fit
this boundary; new capabilities belong in Xec before Console or another application
uses them. See the [release checklist](/docs/omnitron/release-readiness) for activation status.

## Working with tabs

The breadcrumb trail identifies the workspace without a duplicate page title. The node selector and **New SSH tab** stay visible; **Workspace actions** contains host-key verification, the equivalent CLI command and the audit trail.

The layout control reads **1 pane** by default: only the selected tab is shown. Selecting **2 panes** explicitly enables split view, showing the selected tab and another open session. Each pane is labeled with its stable tab name. The workspace shares the available space between the two sessions: side by side when the content area is at least 960 px wide, stacked otherwise. Both panes stay inside the window. Closing down to one tab resets the layout to one pane. Scrollback scrolls within each terminal, and resizing the window or sidebar updates the remote PTY dimensions. Tabs remain connected when following Console navigation, including breadcrumbs.

Each pane exposes input mode, search and a **Terminal actions** menu. **READ** disables local keyboard and paste input; it does not change the remote account's permissions. Search opens with the magnifying glass or **Ctrl/⌘ + Shift + F**. Use **Enter** for the next match, **Shift + Enter** for the previous match and **Escape** to return to the shell. The actions menu provides copy selection and text sizes from 10 to 24 px, with a 14 px reset. Every tab has its own close button. **Delete** on a focused tab requests the same action. A running session requires confirmation naming its target, including when its tab is inactive. A session already reported as closed is removed locally without another confirmation or RPC call. If the shell exits while confirmation is open, the pending action becomes local tab cleanup. Closing the active tab selects its neighbor; closing the last tab returns focus to **New SSH tab**. Tab numbers remain stable when another tab is removed.

## Connection and closure feedback

**New SSH tab** creates a pending tab immediately. It names the node and SSH
identity, shows elapsed waiting time and stays responsive while the daemon opens
the shell. There is no invented percentage or handshake stage. Switching to
another tab keeps that selection when the new connection completes. Duplicate
open requests are blocked until the attempt returns.

A rejected connection displays the reason in its tab and offers **Retry
connection**. A transport deadline instead reads **unconfirmed**: the remote
operation may still have happened. Unattached shells are subject to the daemon's
client-disconnection timeout; a failed browser request is not proof of closure.
Failed node/policy discovery has a separate retry and does not leave a permanent
loading state.

Closed panes retain their scrollback, exit reason and exit code when available.
After a read/write failure, input stops immediately and Console requests closure.
The session is shown as **State unknown** until that closure is acknowledged. A
failed manual close keeps the tab and the confirmation dialog, with a visible
reason and retry. Successful closure removes the tab. Read-only input mode does
not override a closed session or an unavailable connection.

## Verify the host before connecting

1. Register a node with its SSH hostname, port, user and credential using the [node commands](/docs/omnitron/cli#nodes-infrastructure-machines).

2. Open **Workspace actions** (the ellipsis beside the node selector), then select **Verify host key**. The daemon observes the server's key in a handshake that does not authenticate.

3. Compare the SHA256 fingerprint with an independently trusted source, such as the host's provisioning console. For an OpenSSH host, an administrator can run:

   ```bash
   ssh-keygen -lf /etc/ssh/ssh_host_ed25519_key.pub -E sha256
   ```

4. Enter the verified fingerprint to save trust, then select **New SSH tab**.

An observed fingerprint is not proof of host identity. Verification must not consist of copying a key from the same untrusted network connection. Pin replacement uses the previous pin as a compare-and-swap condition and closes sessions for that node. A pin applies to the registered host, port and username; changing that target requires verification again.

The same registered fingerprint protects automated commands, file uploads and mesh tunnelling. For an unpinned automation target, Xec can use an existing trusted `known_hosts` entry on the controller. Interactive sessions require a verified registration pin. A stack that selects a different SSH account or port must provide its own credentials.

When replacing a registered host, SSH port or account, provide or explicitly clear its credentials in the same update. If saving the new credentials fails, the replacement registration remains without active credentials. Retry the update on that registration and verify its host fingerprint before use. An already dispatched remote command can still finish after a trust change.

Pins are stored in the daemon's SQLite state store. Opening a terminal requires an exact pin match before authentication. Host-key probing and terminal opening use Xec's bounded connection and shell-request lifecycle. SSH adapter pools keep distinct host-key policies separate and preserve them during reconnect.

## Equivalent CLI workflow

```bash
omnitron node ssh-key edge-1
omnitron node ssh-key edge-1 --trust SHA256:<independently-verified-fingerprint>
omnitron node ssh edge-1
omnitron terminal list
omnitron terminal close <session-id>
```

The node selector accepts a registered name or ID. `Ctrl+]` closes the CLI terminal and restores local terminal settings. **Copy CLI command** in Console produces the same node command.

Terminal listing and closing are limited to the caller's owner identity. Local CLI sessions belong to the OS-authorized local daemon identity; browser sessions belong to the human account and its specific authenticated session. Another browser sign-in, even for the same account, cannot read or control an existing terminal.

## Session lifetime and limits

| Setting                            | Default            |
| ---------------------------------- | ------------------ |
| Enabled                            | `true` on a master |
| Active sessions across the daemon  | 32                 |
| Active sessions per owner          | 8                  |
| Idle without input                 | 15 minutes         |
| Absolute lifetime                  | 8 hours            |
| No output reads from the client    | 90 seconds         |
| Retained server output per session | 256 KiB            |
| Maximum output read                | 64 KiB             |
| Maximum input per RPC              | 8 KiB              |
| Closed-session retention           | 60 seconds         |

Configure policy in the daemon configuration:

```typescript
export default {
  daemon: {
    terminal: {
      enabled: true,
      maxSessions: 32,
      maxSessionsPerOwner: 8,
      idleTimeoutMs: 15 * 60_000,
      absoluteTimeoutMs: 8 * 60 * 60_000,
      disconnectTimeoutMs: 90_000,
    },
  },
};
```

Counts must be positive integers; the supported upper bounds are 128 daemon-wide and 32 per owner. `enabled: false` disables terminal creation. Build and restart the daemon to apply new backend code or startup configuration; a frontend rebuild alone does not register new RPC services.

Output travels as base64 bytes with an absolute byte cursor, preserving UTF-8 boundaries and terminal protocols. A slow reader receives an explicit dropped-byte count. `hasMore` allows the client to drain retained output after a shell closes. Input is sequenced and deduplicated; an ambiguous write failure closes the session instead of replaying a command.

Closing a session is idempotent after its retained server state expires, so an
old tab can still be removed. This operation still requires current administrator
authority. A live session owned by another sign-in remains inaccessible.

## Authorization and audit

SSH requires the administrator role and a current human session, or the trusted local Unix-socket identity. A machine `service_role` cannot open an interactive terminal. The daemon rechecks authorization and ownership on every method; its two-second sweep also closes revoked sessions, removed targets and expired shells.

Host trust and terminal opening require an audit write before proceeding. The audit records target identity, ownership and lifecycle events. It deliberately does not record commands, terminal output, passwords or key material. Use **Audit** or `omnitron audit --resource terminal`. An audit of session lifetime is not a shell transcript.

Use HTTPS outside loopback. xterm.js runs without automatic URL opening or OSC 52 clipboard writes. Multi-line/control-character paste requires review. **Read only** disables input for that pane; it is a local usability control, not a separate server permission. The remote shell still has the full OS rights of the registered SSH user.

## What persists

Tabs and their output survive navigation inside the current Console. They do not survive a browser reload or daemon restart. Closing the browser ends the session after the disconnect timeout. No commands are automatically replayed and no shell is automatically reattached. Use a remote terminal multiplexer when the workload itself must outlive a connection.

This workspace provides interactive PTY shells. File transfer, jump-host configuration, SSH agent forwarding, shared sessions and durable session recording are not exposed here. They require separate permission and lifecycle design before being added.

See [Console access](/docs/omnitron/access), [Xterm security guidance](https://xtermjs.org/docs/guides/security/) and [OpenSSH fingerprint inspection](https://man.openbsd.org/ssh-keygen.1).

---

---
id: "omnitron/topology"
title: "Topology workspace"
description: "Inspect physical placements and configured connections, review target changes, and keep target intent separate from runtime execution."
---

# Topology workspace

Use **Topology** to answer three questions: what is declared, where it is running,
and what evidence supports each connection. Use **Workspace** to design and review
a target placement for a project and stack.

> **Execution boundary**
>
> The workspace saves target intent. **Accept target** and `omnitron topology apply`
> do not change executable project configuration, start or stop applications, migrate
> data, or affect auto-resume. Ordinary stack deployment continues to use the project's
> configured stacks. A target is not an admitted runtime migration.
>
> Automatic node replacement, resource-aware fleet scheduling and database failover
> are specified in the [resilient control-plane design](/docs/omnitron/resilient-control-plane).
> They are not enabled by saving a target.

## Read the infrastructure map

The global map combines this daemon's actual process/container inventory with
remote stack observations across registered projects. A project or stack selection
narrows that view. Replica cards have a physical node identity: two nodes running
the same application remain separate placements.

| Evidence              | Meaning                                                               |
| --------------------- | --------------------------------------------------------------------- |
| Process observation   | The owning daemon answered with this process and its state            |
| Container observation | The owning runtime returned this container's state and health         |
| Configured endpoint   | The application launch configuration names this destination           |
| Declared dependency   | The application declares a startup dependency                         |
| Unknown observation   | This source could not be read; absence or failure has not been proved |

Connections derived from configuration do not claim live traffic. A loopback
endpoint connects only to a service on the same physical node. An ambiguous endpoint
is not arbitrarily attached to the first matching database. A container which has
stopped is not healthy just because its last health check succeeded.

Remote observations retain per-node errors and observation times. An unreachable
replica does not erase a healthy replica, become an empty inventory, or count as a
confirmed stopped application. Logical application summaries can be degraded while
the node inspector explains which placement could not be read.

Graph dragging changes the view layout only. Search, graph/list view, the connections
inspector and [keyboard navigation](/docs/omnitron/console#topology) remain available without
editing deployment intent.

## Application service isolation

Process topology is the local RPC graph inside one application launch. Each
application that declares `topology.expose` or `topology.access` receives its own
native Netron context and private Unix listener, owned by the resident process
owner. The backend's management namespace is separate.

Two projects, stacks, or instances can expose the same service name and version
without sharing a provider. For example, `sample/dev/quotes` and
`sample/test/quotes` can both expose `CollectorWorker`; each consumer resolves
its own application's worker through `createToken('topology:CollectorWorker')`.
No service renaming or extra namespace configuration is needed.

Within one application, each service name/version has one declared provider.
A second provider is a startup error; Omnitron never replaces another provider
silently. Pool proxies retain PM load balancing, and singleton proxies resolve
the current child after a restart without replacing consumers' service definitions.

The same bounded discovery checks apply to pools and single processes. An application
remains `starting` until its declared children have started and provider publication
has completed; a ready child alone cannot announce an unfinished application online. Missing
callable services, invalid metadata, and RPC failures retain the application and
provider names in the startup error. CLI inspection and Console application
details read the same recorded failure. Failed startup releases that launch's
resources; incomplete cleanup remains owned and retryable rather than being
reported as a successful stop.

Stopping or rebuilding one application closes only its listener and providers.
Backend handoff preserves these owner-held contexts. Existing sealed application
SDKs keep using their bootstrap contract and `topology.access` API. An older owner
must be upgraded to gain this isolation; the capability is reported as
`applicationTopology@1`, and an owner upgrade requires a planned application restart.

Intentional connections to another application's public services use an explicit
Netron endpoint. A bare sibling lookup never falls back to another application or
to the management socket. Unix directories use `0700` and sockets use `0600`;
this separates routing namespaces, while OS accounts and containers provide
isolation between mutually untrusted workloads.

## Declaration coverage

Open **Declarations** for the selected project to refresh the bootstrap metadata
used by the map. This administrative action reads each unique bootstrap in a
separate process and commits its `requires` and `omnitronConfig` declarations to
the daemon's SQLite store. It does not call application lifecycle hooks or start
a stack. Imports can execute trusted project code: refresh is an explicit admin
operation, while ordinary viewer reads never import bootstrap code.

```bash
# All registered configurations and stacks of one project
omnitron topology refresh sample
# One stack, using the same authorization and result as Console
omnitron --json topology refresh sample prod
omnitron --json project topology sample --stack prod
```

The read window is bounded to 30 seconds per bootstrap and 90 seconds across the
project. A failed or unread declaration keeps its previous snapshot and a visible
coverage warning; the CLI exits nonzero for partial results. Refresh after rebuilding
applications or changing their dependencies. An HTTP timeout is not a cancellation:
reload the map to inspect committed coverage before retrying.

Expand **declarations available** below the toolbar to inspect each application's
source and capture time. The JSON graph and `topology show` expose the same coverage:

| Source     | Meaning                                                                                            |
| ---------- | -------------------------------------------------------------------------------------------------- |
| `entry`    | No bootstrap; dependencies come from the configured application entry                              |
| `launch`   | Declaration captured by the execution owner when loading this application                          |
| `snapshot` | Persisted declaration from an explicit refresh or stack preparation                                |
| `memory`   | Compatibility metadata available only in a previous process cache/checkpoint                       |
| `missing`  | Bootstrap declarations unavailable; explicit configuration alone is insufficient for full coverage |

A new owner carries launch declarations across backend handoffs. Persisted snapshots
also survive a cold backend restart. The graph reports a changed or missing captured
bootstrap file without executing it. Its hash covers the selected source/compiled
bootstrap file, **not the entire imported dependency tree**. A snapshot is not a
continuous code watcher or proof of live connections. Failure, source and capture
time remain visible instead of turning an unknown dependency into an absent one.
Private declaration records stay in the daemon store; viewer responses contain
sanitized endpoint identities, dependency evidence and coverage, never environment
values or credentials.

## Design a target

1. Select a project and stack, then open **Workspace**.
2. In **Placement**, choose the enabled project applications and assign them to
   registered fleet nodes. A local target has no remote nodes; a remote target has
   one; a cluster target has one or more.
3. Inspect **Connections**. Service runtimes come from the same infrastructure
   normalization and binding rules as provisioning. Unknown statefulness remains
   unverified; the diagram does not grant permission to relocate a database.
4. **Save draft** or press Ctrl/Command+S. Saving is protected against concurrent
   edits from another Console or CLI session.
5. **Review plan** to inspect additions, removals, before/after node metadata and blocking issues. A plan is
   bound to the current configuration and draft revision and expires after ten minutes.
6. **Accept target** stores that exact plan ID. It leaves deployment and restart
   configuration unchanged. Export the portable target with **Export JSON**.

Viewers and operators can inspect the workspace. Its mutations currently require
the administrator role. The server enforces the same gate for Console and CLI.
Application definitions, environment values and SSH credentials do not travel in
the portable document or viewer stack configuration.

On a concurrent edit or changed project configuration, reload the saved target or
choose **Rebase edits**. Rebase loads the latest base while preserving your edits
locally; save and review again. It does not silently merge or apply a conflicting plan.
Closing a dirty editor asks before discarding local edits. Narrow windows use a
full-height dialog with content contained inside its scroll region.

## CLI parity

| Command                     | Result                                                        |
| --------------------------- | ------------------------------------------------------------- |
| `omnitron topology refresh` | Explicit admin declaration capture without a workload restart |
| `omnitron topology show`    | Configured target, accepted target and observation gaps       |
| `omnitron topology export`  | Portable JSON document                                        |
| `omnitron topology draft`   | Revision-protected draft save                                 |
| `omnitron topology plan`    | Validation and expiring review plan                           |
| `omnitron topology apply`   | Accept an explicit target plan ID; no runtime execution       |

```bash
omnitron topology show sample prod
omnitron --json topology show sample prod
omnitron topology export sample prod > prod.topology.json

# Edit the exported target, then save and review it.
omnitron topology draft sample prod prod.topology.json
omnitron topology plan sample prod

# Use the explicit ID returned by plan. This accepts target intent only.
omnitron topology apply sample prod <plan-id>
```

Normal `topology export` writes a portable document directly. With `--json`, the
standard CLI envelope contains that document in `data`; extract it before importing.

The JSON workspace contains `configured`, `desired`, `draft`, `plans`, `observed`,
service/dependency evidence and registered-node choices. `configured` is the current
executable project configuration. `desired` is the accepted target, or the configured
placement when none has been accepted. Use this distinction in automation.

The portable v1 document is deliberately small:

```json
{
  "version": 1,
  "type": "cluster",
  "apps": ["api", "worker"],
  "nodes": [
    { "host": "app-a.example", "port": 9700, "role": "app", "apps": ["api"] },
    { "host": "app-b.example", "port": 9700, "role": "worker", "apps": ["worker"] }
  ]
}
```

Names must identify enabled applications. Unknown fields, credentials embedded in
the document, duplicate nodes/apps, unassigned applications and invalid dependencies
are rejected. A startup `dependsOn` remains local: it cannot become cross-host
readiness merely by placing its target elsewhere. Node `port` is the daemon RPC
port, not a database, Redis or gateway port.

## Durability and limits

Drafts, accepted targets and the corresponding acceptance record use the local
daemon SQLite store. Revision comparison and acceptance share an immediate
transaction with FULL WAL durability. This is local durability, not multi-controller
consensus or a guarantee against losing the machine and its disk. Back up the daemon
state as part of control-plane recovery.

The recent plan window contains twenty plans. Repeating a retained accepted plan
returns its previous result; an evicted plan ID is refused, never re-executed. A stale,
expired or invalid unaccepted plan cannot change the target. PostgreSQL audit is an
additional searchable record; it is not the authority for whether local acceptance
committed.

The current executor supports per-node application subsets through the existing
stack configuration. An explicit `nodes[].apps` list takes precedence over role
labels; an empty list assigns no application processes. When omitted, cluster nodes
with role `database` or `cache` retain their infrastructure-only default; other
placements retain the all-applications default. The planner, observer and executor
share these rules. Infrastructure declarations
are still provisioned per declared node. This is **not** selective service placement,
replication of existing databases, or automatic movement of their volumes.

Changing a node's role label does not create a database cluster. Replacing a host in
a target does not authorize taking over a writer identity. Those operations require
the resource, execution, fencing and data-recovery contracts in the control-plane
design.

---

---
id: "reference/api-index"
title: "API index"
description: "Search public exports by symbol, package or import path, then open the relevant guide or implementation."
---

# API index

Find an exported symbol, its declared import path, the relevant guide and
its implementation. This index is generated from package manifests and
TypeScript exports, including re-exports.

An indexed symbol is an available entry point, not a promise that every
edge case has a prose example. Start with the linked guide for behaviour
and lifecycle; use your installed declarations for exact signatures.
Source links open the exact Omnitron commit used to build this index. Check
[compatibility](/docs/reference/compatibility) before using it with an older release.

---

---
id: "reference/compatibility"
title: "Compatibility and status"
description: "Runtime requirements, package boundaries, versioning, and the distinction between implemented APIs and design proposals."
---

# Compatibility and status

Use the package manifests for exact dependency ranges. The table below
describes the current source tree; pin the versions you deploy.

| Surface                     | Requirement or boundary                                                                                                                                               |
| --------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Titan and Omnitron          | Node.js 22 or newer, ESM                                                                                                                                              |
| Source workspace            | pnpm 12.16.1; use the version in the root `packageManager` field                                                                                                      |
| TypeScript applications     | Legacy decorators and decorator metadata; see the [quickstart](/docs/getting-started/quickstart)                                                                      |
| Browser RPC                 | `netron-browser`; HTTP and WebSocket, independent of the UI framework                                                                                                 |
| React integration           | `netron-react` and Prism; consult their `peerDependencies` for React and UI dependencies                                                                              |
| Other runtimes              | [Cross-runtime tests](/docs/testing/cross-runtime) cover selected packages; they do not establish daemon support on Bun or Deno                                       |
| Daemon IPC                  | Unix sockets; validate your operating system and deployment target before adopting the daemon                                                                         |
| Infrastructure provisioning | Docker for container-backed infrastructure; [bare-metal](/docs/deployment/bare-metal) has separate requirements                                                       |
| Documentation site          | A separate Lumen consumer with its own manifest and TypeScript version; use the canonical SDK bridge for unpublished source and a registry lock for released packages |

## Implemented API versus design

The module references document exported APIs. **Design RFC** callouts mark
proposed composition or application-level behaviour. The
[authorisation section](/docs/auth) includes such designs; it is not
a switch that installs a complete permissions system.

The daemon defines `admin`, `operator` and `viewer` operator roles. Access is
checked against each method's explicit role list. Control-plane credentials
use a separate service identity. See [Auth & RBAC](/docs/omnitron/auth-rbac)
for the boundaries; applications define their own end-user roles.

## Versioning

Omnitron is pre-1.0. A minor release can change a public contract.
Before upgrading:

1. Read the [changelog](/docs/community/changelog) and relevant migration guide.
2. Commit the package lockfile and build from that exact dependency graph.
3. Compile both server and client against the intended contract.
4. Exercise a real RPC call, authentication and your recovery procedure.
5. Keep the previous application artifact and a database recovery plan.

A shared type catches incompatible code during compilation. It cannot
prevent an older, already deployed client from calling a newer server.
Plan backward compatibility or a coordinated rollout for that case.

## Reporting a compatibility problem

Include the package version, Node.js version, operating system, transport,
compiler configuration and a small reproduction in an
[issue](https://github.com/omnitron-dev/omni/issues).

---

---
id: "reference/glossary"
title: "Glossary"
description: "Every term used across the stack, with cross-references."
---

# Glossary

Terms used across the stack. Cross-referenced; click through to
the deeper page.

## A

**Application** — The root Titan object created by
`Application.create(module, options)`. Owns the DI container,
the Netron transports, and lifecycle hooks.
→ [Application](/docs/titan/application)

**`@Auth(opts)`** — Method decorator from `titan-auth` that
gates an RPC call by role. `@Auth({roles: ['admin']})`.
→ [Auth decorators](/docs/titan/modules/auth#decorators)

**`AuthenticationClient`** — Browser-side auth state manager
(from `@omnitron-dev/netron-browser/auth`): stores tokens,
auto-refreshes, syncs across tabs.
→ [netron-browser auth](/docs/frontend/netron/auth)

**Autoscaler** — `titan-pm`'s built-in worker-pool scaler.
Adjusts `instances` based on CPU / queue depth / memory.
→ [titan-pm](/docs/titan/modules/pm)

## B

**Backoff** — Exponential delay between retry attempts.
`computeBackoff()` is the shared primitive.

**`BackendPool`** — Multi-backend RPC client; routes calls to
different Netron servers by service-name pattern.
→ [Multi-backend](/docs/frontend/netron/multi-backend)

**Block** — High-level Prism composite — `<AuthBlock>`,
`<DashboardBlock>`, `<DataGridBlock>`.
→ [Prism blocks](/docs/frontend/prism/blocks)

**Bootstrap** — The per-app entry that exports
`defineSystem({...})`. The orchestrator imports it to learn
about an app's process topology.
→ [Configuration](/docs/omnitron/configuration)

## C

**Cache key** — `[service, method, args]` for `useQuery`.
Args are deep-equal compared.

**`@Cacheable`** — Memoise a method's return value in `titan-cache`.

**`cuid`** — Collision-resistant URL-safe ID generator.
→ [cuid](/docs/utilities/cuid)

**Circuit breaker** — Trips after N consecutive failures;
fails-fast for the cooldown period. Half-opens with one probe.
→ [Browser client / Fluent HTTP interface](/docs/frontend/netron/browser#fluent-http-interface--caching-retry-circuit-breaking)

**Classic mode** — Per-app launch mode where one fork runs the
full bootstrap end-to-end. Compare with **module-worker mode**.

**Cluster** — Set of Omnitron daemons with leader election +
state replication. Different from **fleet** (no election).
→ [Cluster + Fleet](/docs/omnitron/cluster)

**`cnf.fp`** — JWT claim binding a short-lived access token to
a long-lived refresh token id via HMAC. Invalidates the access
token immediately when the refresh chain rotates.

**Codegen** — Translation step between server and client (e.g.,
OpenAPI → TS, protobuf → TS). This stack uses **zero** codegen.

**Container** — The DI registry; resolves and caches providers.
Built on the Nexus primitive.

**Container identity check** — Boot-time guard ensuring the
`Container` class imported by an app's modules is physically
the same class as the daemon's. Different versions cause
explicit error rather than mystery failure.

**Contextual injection** — Same DI token, different providers
selected per request context (multi-tenant, multi-environment).

**Contract** — The TypeScript service interface. The contract
between server and client.

## D

**Daemon** — The long-running Omnitron supervisor process.
Always-on; everything else (CLI, webapp) is transient.
→ [Daemon](/docs/omnitron/daemon)

**Decorator** — TypeScript metadata syntax. `@Service`,
`@Public`, `@Inject`, etc. Drives the DI + Netron grammar.
→ [Decorators catalog](/docs/titan/modules/decorators-catalog)

**`defineEcosystem(config)`** — Function in `omnitron.config.ts`
that declares the ecosystem (apps, stacks, infrastructure,
supervision).

**`defineSystem(definition)`** — Function in per-app `bootstrap.ts`
that declares the process topology (`IProcessEntry[]`).

**DI** — Dependency injection. The container resolves dependencies
declared via constructor or `@Inject(token)`.

**Discovery** — Service-discovery primitive (`titan-discovery`).
Redis-backed registry with heartbeats; multi-pod-aware.

**DLQ** — Dead-letter queue. Failed notifications land here for
manual inspection.

**DTO** — Data Transfer Object. A typed shape returned by an
RPC method.

## E

**Ecosystem** — Top-level project configuration (`omnitron.config.ts`).
Lists apps, stacks, supervision, infrastructure.

**Edge** — Runtimes designed for short-lived requests near users
(Cloudflare Workers, Deno Deploy). Not supported by Omnitron
daemon (Node-only).

**Event-bus** — Cross-service event distribution. `titan-events`
ships an in-process bus; `titan-notifications` adds cross-process
durable delivery.

## F

**Fan-out architecture** — Production shape: one identity app

- N specialist apps sharing JWT + Redis session registry.

**`@Field`** — Prism's schema-aware form field. Reads zod
schema from `<SchemaProvider>` for type / required / constraints.

**Fleet** — Inventory of remote daemons addressed by alias.
Separate from **cluster** (which adds leader election).

**`forFeature(...)`** — Module factory adding additional
per-feature config (repos, indicators, named clients) after
`forRoot`.

**`forRoot(opts)`** / **`forRootAsync({useFactory, inject})`** —
Top-level module factory. Sync vs async configuration.

**`forWorker(...)`** — `titan-notifications`-specific factory for
the worker pod (vs producer pod).

## G

**Gateway** — OpenResty/Lua reverse proxy bundled with Omnitron.
Provisioned per stack; supports Lua maintenance mode.

**Glossary** — This page.

## H

**Health probe** — Live/Ready check returning the service's
status. Powered by `titan-health` indicators.

**Hook** — Lifecycle callback (`beforeStart`, `afterStart`,
`onHealthCheck`, etc.) declared on `IAppDefinition` /
`IProcessEntry`.

## I

**Infrastructure subsystem** — Omnitron's declarative
provisioner for Postgres / Redis / S3 / custom containers.
→ [Infrastructure](/docs/omnitron/infrastructure)

**`@Inject(token)`** — DI decorator marking a parameter or
property for token-based resolution.

**`@Injectable()`** — Class decorator marking a class as
DI-instantiable.

**`IProcessEntry`** — Per-process spec in `defineSystem`.
Becomes exactly one fork (or N if `instances > 1`).

## J

**JWKS** — JSON Web Key Set. Public keys used to verify RS256
JWTs; rotated by URL.

**JWT** — JSON Web Token. The default auth credential.

## K

**Kb** — Knowledge-base framework powering `omnitron kb mcp`.
→ [kb](/docs/utilities/kb)

**Kysera** — Kysely + plugins shipping in `titan-database` —
RLS, soft-delete, timestamps, audit.

## L

**L1 / L2 cache** — Two-tier cache: L1 in-process (LRU/LFU),
L2 in Redis (durable).

**Leader election** — Simplified Raft variant for cluster mode.
5–15 s election timeout, 2 s heartbeat.

**Lifecycle** — `onInit` → `onStart` → `onStop` → `onDestroy`,
fired in dependency order.

**Logger** — pino-based structured logger; built-in module.

**Lua maintenance flag** — Gateway-level toggle in Redis; flip
to return 503 with maintenance page without restart.

## M

**MCP** — Model Context Protocol. Standard for AI agents to
discover and call tools. Omnitron's MCP server exposes \~40 tools.
→ [MCP](/docs/omnitron/mcp)

**MessagePack** — Binary serialisation format. Netron's default
wire format; preserves Date / Map / Set / BigInt / Error.
→ [msgpack](/docs/utilities/msgpack)

**Middleware** — Pluggable per-call hook. Three stages:
pre-request / post-response / error.
→ [Middleware](/docs/frontend/netron/middleware)

**Module** — A unit of composition. Class with `@Module({...})`.
Declares providers + imports + exports.

**Module-worker mode** — Per-app launch mode where each
`IProcessEntry` is a separate fork importing only that
process's module file. Compare with **classic mode**.

## N

**Nexus** — The custom DI container Titan uses. Class-token-
based; supports contextual injection.

**Netron** — Titan's RPC plane. Same `@Service` over four
transports. → [Netron](/docs/titan/netron)

**Node** — Either a Node.js process OR a fleet node (machine).
Disambiguated by context.

**`NodeInfo`** — Discovery's per-node metadata: ID, address,
services, timestamp.

## O

**`OmnitronDaemon`** — The daemon's primary RPC service.
26 methods: start/stop/restart/scale/inspect/etc.

**Operator** — Human / agent driving the platform via CLI /
console / MCP. Also: middle RBAC role
(`viewer < operator < admin`).

**Orchestrator** — Subsystem inside the daemon that handles
per-app launch (bootstrap loader, ts-compiler, file-watcher,
dependency-resolver, …).
→ [Orchestrator](/docs/omnitron/orchestrator)

## P

**P2C** — Power-of-two-choices load balancing. Pick 2 random
workers; dispatch to the less loaded. Near-optimal balance.

**PaaS** — Platform-as-a-Service. Fly.io / Railway / Render.

**`@PostConstruct`** / **`@PreDestroy`** — Lifecycle decorators
on methods.

**`@Process()`** — Marks a class as a `titan-pm` worker
process.

**`@Public()`** — Exposes a method on Netron (core);
auth-bypass marker (titan-auth); worker IPC marker (titan-pm).
Three decorators, different effects — distinguished by import.

**Pillar** — Top-level layer of the stack: Titan, Netron,
Prism, Omnitron.

**`<PrismProvider>`** — Root provider for Prism components;
sets up MUI theme, snackbar, etc.

## Q

**Query cache** — netron-react's React-aware cache built on
QueryCache. Powers `useQuery` / `useMutation`.

## R

**RBAC** — Role-Based Access Control. Three built-in roles:
viewer, operator, admin.

**Reconciler** — Infrastructure subsystem's loop: declared
services vs actual containers; create / recreate / restart /
remove.

**RLS** — Row-Level Security. Postgres feature; `titan-database`'s
Kysera plugin enforces tenant isolation at the SQL layer.

**Route** — In RPC: nothing (services own methods, not URLs).
In webapp: react-router-dom path.

## S

**Sandbox** — Per-context isolated DI scope. Used for
multi-tenant request handling.

**Schema-aware form** — Prism's `<SchemaProvider>` + `<Field>`
pattern. Schema drives input type, required, constraints.

**Service** — Class with `@Service('name@version')`. Exposes
`@Public()` methods over Netron.

**Service descriptor** — Runtime metadata about a service:
name, version, methods, schemas. Returned by
`getServiceDescriptor`.

**Session** — User auth session. ID stored in Redis under
`omni:session:{sid}`; instant revocation via deletion.

**Stack** — Named environment in a project (dev / staging /
prod). Each may have its own infra overrides.

**Stale-while-revalidate** — Cache strategy: serve stale data,
refresh in background.

## T

**TTL** — Time-to-live. Used for cache entries, session
records, discovery heartbeats.

**`titan-X`** — Independently versioned official module (e.g.,
`titan-auth`, `titan-cache`).

**Topology** — Per-process declaration of `expose` /
`access` controlling cross-process service mesh.

**Transport** — Wire format. Netron speaks four: HTTP, WS, TCP,
Unix.

**Tutorial** — The 6-step walkthrough.
→ [Tutorial](/docs/tutorial)

## U

**Unix socket** — Local IPC mechanism. Omnitron daemon's
default management plane (`~/.omnitron/daemon.sock`).

**Uptime bar** — 90-day green/yellow/red visualisation per node
in the webapp.

## V

**`@Validate(schema)`** — Decorator running a zod schema before
the method body. Bad input throws `Errors.validation`.

**Viewer** — Lowest RBAC role. Read-only: list / inspect /
status / metrics / health / logs.

## W

**WAL** — Write-Ahead Log. Used by `titan-telemetry-relay` for
durable buffering.

**Worker pool** — Multi-instance process group with P2C load
balancing; managed by `titan-pm`.

## Z

**Zod** — Schema library. Used throughout for validation,
configuration, contract types. Re-exported from
`@omnitron-dev/titan/validation`.

**Zero codegen** — The signature property of the stack: no
generated code between server and client; TypeScript compiler
is the only source of truth.

## See also

- [Packages reference](/docs/reference/packages) — every npm package
- [Architecture](/docs/foundations/architecture) — what fits where
- [Comparison](/docs/comparison) — terms in context of alternatives

---

---
id: "reference/packages"
title: "Package Index"
description: "Every package shipped from the monorepo, grouped by role."
---

# Package Index

Public packages defined in `github.com/omnitron-dev/omni`,
grouped by role and clearly labelled (server/client, required/
optional, framework-bound vs framework-agnostic). This index follows current
source; an installed registry release may expose fewer packages or entry points.
See [source-workspace installation](/docs/auth/access-policy)
and [composition boundaries](/docs/foundations/composition) when integrating unreleased APIs.

## Server-side / Backend

### Core framework

| Package                                       | What                                                                                                                                     | Required? |
| --------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------- | :-------: |
| [`@omnitron-dev/titan`](/docs/titan/overview) | Backend framework — DI, modules, lifecycle, **server-side Netron with all 4 transports (HTTP/WS/TCP/Unix) built in**, validation, errors |     ✓     |

`@omnitron-dev/titan` is **self-sufficient**. Server-side
Netron — including every transport — ships inside it as subpath
exports (`@omnitron-dev/titan/netron`,
`@omnitron-dev/titan/netron/transport/{http,websocket,tcp,unix}`).
No extra package is needed for the server.

### Backend modules (opt-in)

Independently versioned. Add only what your app uses.

| Package                                                                      | Purpose                                                       |
| ---------------------------------------------------------------------------- | ------------------------------------------------------------- |
| [`@omnitron-dev/titan-auth`](/docs/titan/modules/auth)                       | JWT authentication (HS256 / RS256 / ES256, JWKS, token cache) |
| [`@omnitron-dev/titan-cache`](/docs/titan/modules/cache)                     | Multi-tier caching (L1 LRU/LFU + L2 Redis)                    |
| [`@omnitron-dev/titan-database`](/docs/titan/modules/database)               | Kysely + migrations + RLS plugin                              |
| [`@omnitron-dev/titan-discovery`](/docs/titan/modules/discovery)             | Redis-backed service discovery + heartbeats                   |
| [`@omnitron-dev/titan-events`](/docs/titan/modules/events)                   | In-process event bus with history                             |
| [`@omnitron-dev/titan-health`](/docs/titan/modules/health)                   | Health + readiness probes (k8s-shaped)                        |
| [`@omnitron-dev/titan-lock`](/docs/titan/modules/lock)                       | Distributed Redis locks (Lua scripts, UUID ownership)         |
| [`@omnitron-dev/titan-metrics`](/docs/titan/modules/metrics)                 | Counters / gauges / histograms; Prom exposition               |
| [`@omnitron-dev/titan-notifications`](/docs/titan/modules/notifications)     | Multi-channel delivery + DLQ + rotif messaging                |
| [`@omnitron-dev/titan-pm`](/docs/titan/modules/pm)                           | Process manager / worker pools / autoscaling                  |
| [`@omnitron-dev/titan-ratelimit`](/docs/titan/modules/ratelimit)             | Token-bucket / sliding-window / fixed-window                  |
| [`@omnitron-dev/titan-redis`](/docs/titan/modules/redis)                     | Redis client (clusters, sentinel, named instances)            |
| [`@omnitron-dev/titan-storage`](/docs/titan/modules/storage)                 | Local and S3 object adapters                                  |
| [`@omnitron-dev/titan-scheduler`](/docs/titan/modules/scheduler)             | Cron / interval / timeout with persistence                    |
| [`@omnitron-dev/titan-telemetry-relay`](/docs/titan/modules/telemetry-relay) | Store-and-forward telemetry (WAL-backed)                      |

## Browser-side / Frontend

| Package                                                         | Framework                                                                                 | Required when                                              |
| --------------------------------------------------------------- | ----------------------------------------------------------------------------------------- | ---------------------------------------------------------- |
| [`@omnitron-dev/netron-browser`](/docs/frontend/netron/browser) | **Framework-agnostic** — vanilla JS, Vue, Svelte, Solid, Angular, Lit, React, Web Workers | You build a browser client that calls a Titan backend      |
| [`@omnitron-dev/netron-react`](/docs/frontend/netron/react)     | **React-only** — optional layer on top of netron-browser                                  | Your frontend uses React                                   |
| [`@omnitron-dev/prism`](/docs/frontend/prism)                   | **React-only** — MUI v9 design system                                                     | Your frontend uses React and you want pre-built components |

**Use matrix:**

| Frontend stack          | Install                                                |
| ----------------------- | ------------------------------------------------------ |
| Vanilla JS / TypeScript | `netron-browser` only                                  |
| Vue 3                   | `netron-browser` only (wrap in `reactive`/`ref`)       |
| Svelte                  | `netron-browser` only (wrap in stores)                 |
| Solid                   | `netron-browser` only (wrap in signals)                |
| Angular                 | `netron-browser` only (wrap in services / RxJS)        |
| Lit                     | `netron-browser` only                                  |
| React 19 (Prism)        | `netron-browser` + `netron-react` (+ optional `prism`) |
| Web Worker              | `netron-browser` only                                  |

## Supervisor (opt-in)

| Package                                             | What                                                                                                                             |
| --------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------- |
| [`@omnitron-dev/omnitron`](/docs/omnitron/overview) | Application supervisor + CLI + web console + MCP server. Optional. See [Titan vs Omnitron](/docs/foundations/titan-vs-omnitron). |

## Shared utilities (framework-agnostic)

Focused packages. Used internally by everything
above; reusable in non-Omnitron projects.

| Package                                                      | Purpose                                                                     |
| ------------------------------------------------------------ | --------------------------------------------------------------------------- |
| [`@omnitron-dev/access`](/docs/auth/access-policy)           | Dependency-free roles, permission catalogs and compiled checks              |
| [`@omnitron-dev/common`](/docs/utilities/common)             | Type predicates, promise helpers, object tools, data structures             |
| [`@omnitron-dev/cuid`](/docs/utilities/cuid)                 | Collision-resistant URL-safe unique IDs                                     |
| [`@omnitron-dev/eventemitter`](/docs/utilities/eventemitter) | Async event emitter with parallel/serial/reduce                             |
| [`@omnitron-dev/msgpack`](/docs/utilities/msgpack)           | Extensible MessagePack with native JS types (Date, Map, Set, BigInt, Error) |
| [`@omnitron-dev/kb`](/docs/utilities/kb)                     | Knowledge-base framework (backs `omnitron kb mcp`)                          |
| [`@omnitron-dev/testing`](/docs/testing/testing-package)     | Cross-runtime testing utilities (Node + Bun + Deno)                         |

### Shared internals

Two more packages exist and are published, but you install
neither: `titan` and `netron-browser` both depend on them, and
that is the point — each is a single definition that used to be a
pair of copies drifting apart on either side of the wire.

| Package                                                      | Purpose                                                                                                       |
| ------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------- |
| [`@omnitron-dev/netron-protocol`](/docs/reference/protocol)  | Wire packets, service definitions, identifiers and the shared error hierarchy. Dependency-free                |
| [`@omnitron-dev/netron-http-core`](/docs/reference/protocol) | Environment-neutral HTTP primitives — cache manager, retry manager, query builder, configurable request proxy |

Their versions move with the packages that consume them; pin
those, not these.

See [Protocol and HTTP primitives](/docs/reference/protocol) for these packages and
[API index](/docs/reference/api-index) for all declared entry points.

## License

MIT across every package.

## Install matrices

### Backend service only

```bash
pnpm add @omnitron-dev/titan
# + any titan-* modules you need
```

### Backend + browser client (any framework)

```bash
# Backend (server Netron + 4 transports already inside):
pnpm add @omnitron-dev/titan

# Browser (works with any framework):
pnpm add @omnitron-dev/netron-browser
```

### Backend + React frontend

```bash
# Backend:
pnpm add @omnitron-dev/titan

# React frontend:
pnpm add @omnitron-dev/netron-browser @omnitron-dev/netron-react @omnitron-dev/prism
```

### Full Omnitron-supervised platform

```bash
pnpm add @omnitron-dev/titan
pnpm add @omnitron-dev/netron-browser @omnitron-dev/netron-react
pnpm add -g @omnitron-dev/omnitron
```

## See also

- [Installation](/docs/getting-started/installation) — detailed install paths + subpath imports
- [Titan vs Omnitron](/docs/foundations/titan-vs-omnitron) — what runs without Omnitron
- [Architecture](/docs/foundations/architecture) — how the layers compose
- [Comparison](/docs/comparison) — vs alternative stacks

---

---
id: "reference/protocol"
title: "Protocol and HTTP primitives"
description: "Shared wire contracts and HTTP machinery used by the server and browser clients."
---

# Protocol and HTTP primitives

Two packages hold definitions shared by Titan and the browser client.
Most applications use the higher-level
[server](/docs/titan/netron) and [client](/docs/frontend/netron/browser)
APIs. These packages matter when building an adapter, inspecting the wire
contract or diagnosing compatibility.

## `@omnitron-dev/netron-protocol`

| Export family                                                                            | Purpose                                               |
| ---------------------------------------------------------------------------------------- | ----------------------------------------------------- |
| `Uid`, `uuid`, `MAX_UID_VALUE`                                                           | Identifier generation and its wire limit              |
| `Definition`, `Reference`                                                                | Service definitions and remote references             |
| `ServiceContract`, `ServiceMetadata`, `ArgumentInfo`, `MethodInfo`, `PropertyInfo`       | Shared service-description types                      |
| `Packet`, `createPacket`, `TYPE_*`, `StreamType`                                         | Packet construction, operation codes and stream kinds |
| `ErrorCode`, `ErrorCategory`, `ERROR_METADATA`                                           | Error taxonomy shared by the two ends                 |
| `TitanError`, `AggregateError`, `ErrorPool`                                              | Structured failures                                   |
| `createError`, `ensureError`, `isErrorCode`                                              | Error construction and narrowing                      |
| `isClientError`, `isServerError`, `isRetryableError`, `getErrorCategory`, `toHttpStatus` | Classification for callers and transports             |
| `getErrorName`, `getDefaultMessage`                                                      | Stable names and fallback descriptions                |

The package has no runtime dependencies. It is more than a set of numeric
constants: service definitions, packet shapes and the error hierarchy are
also shared. Check the [API index](/docs/reference/api-index) for the exact exports
in the current source tree.

A compatible packet does not establish authentication or validate a
business payload. Keep those concerns at their documented boundaries.
Do not assign private meanings to an existing operation code; both peers
must agree on any protocol extension.

## `@omnitron-dev/netron-http-core`

| Export                                                         | Responsibility                                      |
| -------------------------------------------------------------- | --------------------------------------------------- |
| `HttpCacheManager`                                             | Shared HTTP cache mechanics                         |
| `RetryManager`                                                 | Retries and circuit-breaker behaviour               |
| `QueryBuilder`                                                 | Query configuration                                 |
| `ConfigurableProxy`                                            | Fluent request configuration around an executor     |
| `IHttpRequestExecutor`                                         | Adapter contract for executing a configured request |
| `HttpRequestContext`, `HttpRequestHints`, `HttpRequestMessage` | Request types shared by adapters                    |
| `HttpCoreLogger`                                               | Minimal logging contract                            |

The companion types include `CacheOptions`, `CacheStats`, `RetryOptions`,
`RetryStats`, `CircuitBreakerOptions` and `QueryOptions`.
This entry point does not export a `RequestBatcher`.

Prefer the [browser client's HTTP interface](/docs/frontend/netron/browser)
unless you are implementing an executor. Define ownership for cache
invalidation, cancellation and disposal. Retry a side-effecting request
only when the operation's idempotency contract permits it.

## Upgrade server and client deliberately

Pin application dependencies and preserve the lockfile. Compile against
the intended contract, then exercise a real call and an error round trip.
Already-deployed clients do not acquire a new contract when the server
is rebuilt. See [compatibility](/docs/reference/compatibility) and
[serialization](/docs/titan/netron/serialization).

---

---
id: "testing/cross-runtime"
title: "Cross-runtime testing"
description: "One test source running unchanged on Node, Bun, and Deno."
---

# Cross-runtime testing

The platform's utility packages (`common`, `cuid`, `eventemitter`,
`msgpack`) and Titan itself target three runtimes:

| Runtime  | Test runner                    |
| -------- | ------------------------------ |
| Node 22+ | Vitest                         |
| Bun 1.x  | `bun test` (vitest-compatible) |
| Deno 2.x | `Deno.test`                    |

The same test source runs in all three when written through
`@omnitron-dev/testing`'s runtime adapter.

## Why bother

| Why                                                                                       | When                      |
| ----------------------------------------------------------------------------------------- | ------------------------- |
| **Catch runtime divergence** in TS features (decorators, top-level await, ESM resolution) | Library authors           |
| **Validate Bun-specific perf paths** without Node-only assumptions                        | Performance-critical code |
| **Run on Deno's permissioned model** to surface implicit file/network access              | Security-critical code    |

For application code (a Titan app running on Node only), one
runtime is fine. For **libraries** consumed by all three, the
cross-runtime test layer is the safety net.

## The adapter

```typescript
import { loadRuntimeAdapter, RUNTIME } from '@omnitron-dev/testing';

const t = await loadRuntimeAdapter();   // resolves to the right runtime's test() + expect()

t.test('basics', () => {
  t.expect(1 + 1).toBe(2);
});

t.test('async', async () => {
  const result = await Promise.resolve(42);
  t.expect(result).toBe(42);
});

t.test('isolation', () => {
  // Each test gets fresh state — same semantics across runtimes
});

t.describe('grouped', () => {
  t.beforeEach(() => { /* ... */ });
  t.test('child', () => { /* ... */ });
});
```

`RUNTIME` is one of `'node' | 'bun' | 'deno'` — useful for the
rare runtime-specific assertion:

```typescript
import { RUNTIME } from '@omnitron-dev/testing';

t.test('uses correct platform API', () => {
  if (RUNTIME === 'bun') {
    t.expect(Bun.version).toBeDefined();
  } else if (RUNTIME === 'node') {
    t.expect(process.versions.node).toBeDefined();
  }
});
```

## Running the suite

Same file, three commands:

```bash
# Node + Vitest:
vitest run test/cross.test.ts

# Bun:
bun test test/cross.test.ts

# Deno:
deno test --allow-read --allow-write --allow-env test/cross.test.ts
```

In CI, run all three:

```yaml
# .github/workflows/test.yml
matrix:
  include:
    - { runtime: 'node', cmd: 'vitest run' }
    - { runtime: 'bun',  cmd: 'bun test' }
    - { runtime: 'deno', cmd: 'deno test --allow-all' }
```

A test that passes on all three is portable.

## Common pitfalls

### ESM resolution

Node + Bun + Deno all support ESM, but with subtle differences:

| Feature             |   Node   |    Bun   |       Deno       |
| ------------------- | :------: | :------: | :--------------: |
| `node:` prefix      | required | optional |     required     |
| `npm:` prefix       |    n/a   |    n/a   | required for npm |
| TypeScript directly |  flagged |    yes   |        yes       |
| Top-level await     |    yes   |    yes   |        yes       |

Stick to `node:` prefixed imports for built-ins:

```typescript
import path from 'node:path';        // ✓ all three
import { readFile } from 'node:fs/promises';
```

Avoid bare `'path'` — works on Node + Bun, fails on Deno.

### File-system access

Deno requires explicit `--allow-read` / `--allow-write`. Tests
that touch the FS need:

```bash
deno test --allow-read --allow-write test/...
```

Or scope:

```bash
deno test --allow-read=./fixtures --allow-write=./tmp
```

Node + Bun have no equivalent — they always allow.

### Process / Environment

```typescript
// Works on all three (when env access is allowed):
import { env } from 'node:process';

env.NODE_ENV;                // string | undefined
env.NODE_ENV ??= 'test';     // assignment works
```

Deno needs `--allow-env` for `process.env` reads.

### Performance APIs

`performance.now()` works on all three; `process.hrtime` is
Node-only. Use `performance` for cross-runtime timing:

```typescript
const start = performance.now();
await work();
const elapsed = performance.now() - start;
```

### Crypto

```typescript
import { randomUUID } from 'node:crypto';     // ✓ all three
crypto.randomUUID();                            // also ✓ — global Web Crypto
```

Web Crypto is the universal path.

## Conditional skips

When a test genuinely can't run on a runtime:

```typescript
import { RUNTIME } from '@omnitron-dev/testing';

t.test.skipIf(RUNTIME === 'deno')('uses Node-only API', () => {
  // ... Node-specific test
});
```

Or:

```typescript
if (RUNTIME !== 'bun') t.test.skip('Bun-only feature', ...);
```

Document **why** in a comment. Skips without explanation rot.

## Suite-level configuration

### Vitest

```typescript
// vitest.config.ts
import { defineConfig } from 'vitest/config';

export default defineConfig({
  test: {
    environment: 'node',
    include:     ['**/*.test.ts'],
    pool:        'forks',
  },
});
```

### Bun

```toml
# bunfig.toml
[test]
preload = ["./test/setup.ts"]
timeout = 10000
```

### Deno

```jsonc
// deno.json
{
  "test": {
    "include": ["**/*.test.ts"]
  }
}
```

## Benchmarks across runtimes

```typescript
import { PerfTimer } from '@omnitron-dev/testing/performance';
import { RUNTIME } from '@omnitron-dev/testing';

const timer = new PerfTimer();

for (let i = 0; i < 10_000; i++) {
  timer.mark('v1:start');
  parseV1(input);
  timer.mark('v1:end');
  timer.measure('v1', 'v1:start', 'v1:end');

  timer.mark('v2:start');
  parseV2(input);
  timer.mark('v2:end');
  timer.measure('v2', 'v2:start', 'v2:end');
}

console.log(RUNTIME, {
  v1: timer.getPercentile('v1', 50),
  v2: timer.getPercentile('v2', 50),
});
```

Compare a percentile rather than a single run or a mean: one scheduling
delay moves a mean and does not move a median, and on a loaded machine
that is the difference between measuring the code and measuring the
host. Run all three runtimes and compare — Bun usually wins raw JS
work; Node wins ecosystem maturity; Deno wins startup time.

## Where the platform itself uses this

The utility packages (`common`, `cuid`, `eventemitter`,
`msgpack`) have cross-runtime test suites — `test/cross/*.test.ts`
files run on all three CI matrices.

Titan + Omnitron target Node only (Bun + Deno work for the
runtime; the daemon assumes Node's child_process model).

## Best practices

- **Default to cross-runtime** for library packages.
- **Node-only** for application code unless the app explicitly
  targets Bun/Deno.
- **Use `node:` prefixes** for built-ins everywhere.
- **Prefer Web standards** (Web Crypto, `fetch`, `performance`,
  `URL`) over Node-specific APIs when both exist.
- **Document skips** with `// skipIf bun: reason`.

## Anti-patterns

- **`process.platform === 'darwin'`** checks in tests that
  should be portable. Push platform-specific logic to the
  module under test, then assert behaviour generically.
- **Bun-only API in cross-runtime tests** without skip guard —
  hidden failure on Node.
- **`__dirname` / `__filename`.** Replace with
  `fileURLToPath(import.meta.url)`.

## See also

- [Testing overview](/docs/testing)
- [Testing package](/docs/testing/testing-package) — adapter API
- [Bun docs](https://bun.sh/docs/cli/test)
- [Deno test docs](https://docs.deno.com/runtime/manual/basics/testing/)

---

---
id: "testing/index"
title: "Overview"
description: "How to test apps built on this stack — unit, integration, cross-runtime, E2E."
---

# Testing

A pragmatic guide to testing on this stack. Covers four
levels — from unit tests of a single class to E2E tests
driving a real React app through a real Netron client.

## The pyramid

```mermaid
flowchart TB
  E2E[E2E — Playwright<br/>real browser, real daemon, real DB]
  Integration[Integration — Vitest<br/>real Application, in-memory infra]
  Module[Module — Vitest<br/>real DI container, mocked deps]
  Unit[Unit — Vitest<br/>plain classes, no DI]

  Unit --> Module --> Integration --> E2E

  style Unit fill:#e8f5e9,stroke:#43a047,color:#1b5e20
  style Module fill:#fff8e1,stroke:#fb8c00,color:#e65100
  style Integration fill:#f3e5f5,stroke:#8e24aa,color:#4a148c
  style E2E fill:#fce4ec,stroke:#d81b60,color:#880e4f
```

| Level           | What                      | Tool                                         | Speed     | When                |
| --------------- | ------------------------- | -------------------------------------------- | --------- | ------------------- |
| **Unit**        | Single class / function   | Vitest                                       | μs–ms     | Pure logic          |
| **Module**      | DI graph + mocks          | Vitest + `Application.create({ overrides })` | ms        | Service interaction |
| **Integration** | Real app, in-memory infra | Vitest + Titan testing utilities             | 10s of ms | Cross-module flow   |
| **E2E**         | Real daemon, real client  | Vitest / Playwright                          | seconds   | Full system         |

## Tooling

| Package                                                            | Role                                                                                               |
| ------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------- |
| [`vitest`](https://vitest.dev/)                                    | The runner. Native ESM, TypeScript, fast.                                                          |
| [`@omnitron-dev/testing`](/docs/testing/testing-package)           | Cross-runtime helpers + async primitives + Titan-specific test glue                                |
| [`@playwright/test`](https://playwright.dev/)                      | E2E browser automation. Used by the webapp.                                                        |
| [`@omnitron-dev/netron-react/test`](/docs/frontend/netron/testing) | `TestNetronProvider` for React components that call Netron                                         |
| Test-fixture databases                                             | Dedicated PostgreSQL databases with real application migrations; never the running daemon database |

Omnitron SQL integration suites create a database per suite on the test
PostgreSQL server and drop it after completion. `pnpm test:ci` owns the test
services for the full workspace run. A missing test server is reported as a
skip with a reason; it is not evidence that SQL behavior passed.

## What lives where

- **Unit + module tests** live next to the code: `src/foo.ts` +
  `src/__tests__/foo.test.ts`.
- **Integration tests** live in `<package>/test/` or `<app>/test/`.
- **E2E tests** live in `<app>/e2e/` (Playwright by convention).
- **Performance + load** tests in `<package>/benchmark/`.

## The four pages

| Page                                                       | What it covers                                                                       |
| ---------------------------------------------------------- | ------------------------------------------------------------------------------------ |
| [Testing package reference](/docs/testing/testing-package) | `@omnitron-dev/testing` API — async helpers, runtime adapters, mock primitives       |
| [Integration patterns](/docs/testing/integration)          | `Application.create({ overrides })`, in-memory DB, fake clocks, transaction rollback |
| [Cross-runtime testing](/docs/testing/cross-runtime)       | Same test source running on Node + Bun + Deno                                        |
| [React component testing](/docs/testing/react)             | `TestNetronProvider`, `createMockService`, Suspense, error boundaries                |

Plus the per-module testing reference:
[Titan / Testing / Modules](/docs/titan/testing/modules) —
specific mock recipes for each of the 14 modules.

## Quick reference — one test per level

### Unit

```typescript
import { describe, it, expect } from 'vitest';
import { calculateInvoiceTotal } from './invoice.js';

describe('calculateInvoiceTotal', () => {
  it('sums line items with tax', () => {
    expect(calculateInvoiceTotal([
      { qty: 2, price: '10.00', taxRate: 0.08 },
      { qty: 1, price: '5.00',  taxRate: 0.08 },
    ])).toBe('27.00');
  });
});
```

No DI, no async, no mocks — just functions and their
expectations.

### Module

```typescript
import { describe, it, expect, vi } from 'vitest';
import { Container, Scope } from '@omnitron-dev/titan/nexus';
import { UsersService } from './users.service.js';

describe('UsersService', () => {
  it('reads from the repo', async () => {
    const container = new Container();
    const fakeRepo  = { findById: vi.fn().mockResolvedValue(MOCK_USER) };

    // Container.register takes the token separately — the { provide, … }
    // form is for a module's `providers` array, not for this call.
    container.register(UserRepo, { useValue: fakeRepo });
    container.register(UsersService);          // a bare class is its own token

    const service = await container.resolveAsync(UsersService);
    const user    = await service.findById('u_42');

    expect(user).toEqual(MOCK_USER);
    expect(fakeRepo.findById).toHaveBeenCalledWith('u_42');
  });
});
```

The DI container is real; dependencies are mocks via the
override mechanism.

### Integration

```typescript
import { describe, it, expect } from 'vitest';
import { Application } from '@omnitron-dev/titan';
import { AppModule } from './app.module.js';
import { FakeMailer } from './test/fakes.js';
import { MAILER_TOKEN } from './tokens.js';

describe('user invite flow', () => {
  it('creates a user and queues a welcome email', async () => {
    const app = await Application.create({
      modules:   [AppModule],
      overrides: [{ provide: MAILER_TOKEN, useClass: FakeMailer }],
      disableGracefulShutdown: true,
    });
    await app.start();

    try {
      const users  = await app.resolve(UsersService);
      const mailer = await app.resolve(MAILER_TOKEN) as FakeMailer;

      await users.invite({ email: 'a@b.c', tier: 'pro' });

      expect(mailer.sent).toEqual([
        { to: 'a@b.c', template: 'welcome', data: { tier: 'pro' } },
      ]);
    } finally {
      await app.stop();
    }
  });
});
```

Real `Application`, real DI, real cross-module flow — only the
outermost integration (mailer) is faked.

### E2E

```typescript
// e2e/sign-in.spec.ts (Playwright)
import { test, expect } from '@playwright/test';

test('user signs in', async ({ page }) => {
  await page.goto('http://localhost:5173');
  await page.getByLabel('Email').fill('a@b.c');
  await page.getByLabel('Password').fill('correct-horse-battery');
  await page.getByRole('button', { name: 'Sign in' }).click();
  await expect(page).toHaveURL('http://localhost:5173/');
  await expect(page.getByText('a@b.c')).toBeVisible();
});
```

Real daemon, real React app, real browser. Slowest layer; run
sparingly.

## Per-module mocking

For each of the 14 Titan modules, the cheapest path to a mock
is in [Titan / Testing / Modules](/docs/titan/testing/modules).
Some quick links:

| Module                | Pattern                                                       |
| --------------------- | ------------------------------------------------------------- |
| `ConfigModule`        | `useValue` ConfigService with `get: () => ...`                |
| `LoggerModule`        | `createNullLogger()` from `@omnitron-dev/titan/module/logger` |
| `TitanDatabaseModule` | In-memory SQLite or transaction-rollback Postgres             |
| `TitanCacheModule`    | In-memory L1-only                                             |
| `TitanRedisModule`    | `ioredis-mock`                                                |
| `TitanAuthModule`     | `useValue` JWT service that returns canned claims             |
| `TitanMetricsModule`  | `'memory'` storage; assert via `recordTyped` spy              |
| `TitanHealthModule`   | One indicator returning `'healthy'`                           |

Each module ships a "Smallest mock" snippet on its page.

## Best practices

- **Test at the lowest level that proves the behaviour.** A unit
  test beats a module test beats an integration test for the
  same assertion — at 1000× speed.
- **One assertion per behavioural intent.** Multiple `expect`s
  fine; multiple intents per test makes failures hard to read.
- **Real DI for module tests.** Mocking the container masks
  wiring bugs that production will hit.
- **Real Application for integration tests.** Mocking
  lifecycle masks ordering bugs.
- **Transaction-rollback for DB tests.** Faster + isolated. No
  cleanup boilerplate.
- **`disableGracefulShutdown: true`** on test `Application.create`
  to skip signal handler installation.
- **Run tests in parallel.** Vitest does this by default — keep
  tests independent (no shared module state, no shared DB rows).

## Anti-patterns

- **Mocking the framework.** Mocking `Application` or
  `Container` means you're not testing your code — you're
  testing the mock.
- **Sleeping for async.** Use Vitest fake timers or
  `await someEvent` — `await delay(100)` is a flake factory.
- **End-to-end-only.** E2E catches integration bugs but slows
  the dev loop. Push assertions down the pyramid.
- **Test files alongside `dist/`.** Tests should run from
  source; compiled tests drift.

## See also

- [Testing package reference](/docs/testing/testing-package) — `@omnitron-dev/testing` API
- [Titan / Testing](/docs/titan/testing/overview) — framework-level guide
- [Titan / Testing / Modules](/docs/titan/testing/modules) — per-module mock recipes
- [Netron React testing](/docs/frontend/netron/testing) — `TestNetronProvider`

---

---
id: "testing/integration"
title: "Integration patterns"
description: "Real Application, fake boundaries — the recipes that scale."
---

# Integration patterns

The integration layer is where \~70% of bugs hide. Modules wire
together with real DI, but external services (DB, mailer,
payment provider, S3) are faked. This is the layer with the
highest signal-to-noise.

## The base pattern

```typescript
import { describe, beforeEach, afterEach, it, expect } from 'vitest';
import { createTestModule, type TestModule } from '@omnitron-dev/testing/titan';
import { AppModule } from '../src/app.module.js';

describe('user invite flow', () => {
  let mod: TestModule;

  beforeEach(() => {
    mod = createTestModule({
      modules: [AppModule],
      providers: [
        [MAILER_TOKEN, { useClass: FakeMailer }],
        [PAYMENT_PROVIDER, { useClass: FakeProvider }],
      ],
    });
  });

  afterEach(async () => {
    await mod.cleanup();
  });

  it('happy path', async () => {
    const container = mod.getContainer();
    const users = container.resolve(UsersService);
    const mailer = container.resolve(MAILER_TOKEN) as FakeMailer;

    await users.invite({ email: 'a@b.c' });

    expect(mailer.sent).toMatchObject([{ to: 'a@b.c', template: 'invite' }]);
  });
});
```

Pieces:

- **`createTestModule`** assembles a container from your modules,
  provider overrides and mocks. Call `createApplication()` on it when a
  test needs the full `Application` lifecycle rather than just the
  container.
- **`providers`** takes `[token, definition]` pairs and replaces
  external-boundary services with fakes. `mocks` takes
  `{ token, mock, spy? }` entries when you want a mock object rather
  than a class.
- **`getContainer().resolve(...)`** pulls anything out, including the
  fakes, so a test can inspect what was called.
- **`cleanup()`** disposes the container and anything it started.

Transaction isolation is not part of this API — arrange it in your own
`beforeEach`/`afterEach` around the connection your module resolves, or
give each worker its own database (see below).

## Fake patterns

### Fake mailer

```typescript
class FakeMailer implements IMailer {
  public sent: SentMessage[] = [];

  async send(msg: SentMessage) {
    this.sent.push(msg);
  }
}
```

Just records calls. Assert on `sent` after the action.

### Fake clock

```typescript
import { vi } from 'vitest';

beforeEach(() => vi.useFakeTimers());
afterEach(()  => vi.useRealTimers());

it('respects backoff', async () => {
  const promise = withRetry(() => failOnce());
  await vi.advanceTimersByTimeAsync(500);
  await vi.advanceTimersByTimeAsync(1_000);
  expect(await promise).toBe('success');
});
```

Use fake timers for **anything time-dependent** — backoff,
TTL, cron, debounce. Real clocks make tests slow and flaky.

### Stubbed external HTTP

```typescript
import { setupServer } from 'msw/node';
import { http, HttpResponse } from 'msw';

const server = setupServer(
  http.get('https://api.partner.example/v1/users/:id', () =>
    HttpResponse.json({ id: 'remote-1', email: 'partner@x.com' })),
);

beforeAll(() => server.listen());
afterEach(() => server.resetHandlers());
afterAll(()  => server.close());
```

[`msw`](https://mswjs.io/) intercepts at the network layer
without changing app code.

## Transaction-rollback in detail

```typescript
app = await createTestApp({
  database: 'rollback',                  // implies wrapping
  modules:  [AppModule],
});
```

Behind the scenes:

```typescript
// Pseudo:
beforeEach: await db.exec('BEGIN');
afterEach:  await db.exec('ROLLBACK');
```

- Every test starts with a clean DB state.
- No `TRUNCATE` between tests — orders of magnitude faster.
- Test isolation is real (the transaction sees only the seed
  data, then its own writes).

**Caveat**: tests that span multiple connections won't see each
other's writes (different transactions). Move those to E2E.

## Lifecycle test

```typescript
it('runs onStart hooks in dependency order', async () => {
  const calls: string[] = [];

  class AModule implements OnStart {
    async onStart() { calls.push('A'); }
  }
  class BModule implements OnStart {
    async onStart() { calls.push('B'); }
  }

  const app = await createTestApp({
    modules: [
      { provide: 'A', useClass: AModule },
      { provide: 'B', useClass: BModule, dependencies: ['A'] },
    ],
  });
  await app.start();

  expect(calls).toEqual(['A', 'B']);
  await app.dispose();
});
```

Asserts lifecycle ordering, hook firing, error propagation.

## Database integration

### In-memory SQLite

```typescript
app = await createTestApp({
  database: 'memory',                    // sqlite :memory:
  modules:  [AppModule],
});
```

Use when:

- Tests are read-heavy and Postgres-specific features aren't
  exercised.
- Speed > realism.

### Real Postgres

Use a real Postgres when RLS, triggers, advisory locks or jsonb
queries matter, or when the schema is Postgres-specific. The shared
test stack is reachable through the env helpers:

```typescript
import { TEST_POSTGRES_URL } from '@omnitron-dev/testing/env';

mod = createTestModule({
  modules: [AppModule],
  providers: [[DATABASE_URL_TOKEN, { useValue: TEST_POSTGRES_URL }]],
});
```

### Docker-managed test DB

When a suite needs its own instance rather than the shared stack:

```typescript
import { DatabaseTestManager, type DockerContainer } from '@omnitron-dev/testing/docker';

let postgres: DockerContainer;

beforeAll(async () => {
  postgres = await DatabaseTestManager.createPostgresContainer({ port: 'auto' });
}, 60_000);

afterAll(async () => {
  await postgres.cleanup();
});

beforeEach(() => {
  const port = postgres.ports.get(5432);
  mod = createTestModule({
    modules: [AppModule],
    providers: [[DATABASE_URL_TOKEN, { useValue: `postgres://test:test@localhost:${port}/test` }]],
  });
});
```

`port: 'auto'` asks the OS for a free port and is what allows several
suites to run in parallel — a fixed port makes them collide, and the
failure names a port the test never chose.

## Netron integration tests

```typescript
import { createTestModule } from '@omnitron-dev/testing/titan';
import { NetronClient } from '@omnitron-dev/netron-browser';
import type { Application } from '@omnitron-dev/titan';

describe('end-to-end users service', () => {
  let mod: TestModule;
  let app: Application;
  let client: NetronClient;

  beforeEach(async () => {
    mod = createTestModule({ modules: [AppModule] });
    app = await mod.createApplication();
    await app.start();

    // Register the transport server with `port: 0` and read back the port
    // the OS assigned — the server reports the port it bound, not the one
    // it was asked for.
    const port = app.netron.transportServers.get('http')!.port!;
    client = new NetronClient({ url: `http://localhost:${port}` });
    await client.connect();
  });

  afterEach(async () => {
    await client.disconnect();
    await mod.cleanup();
  });

  it('returns a user over the wire', async () => {
    const users = await client.service<UsersService>('users');
    const u     = await users.findById('u_42');
    expect(u.email).toBe('a@b.c');
  });
});
```

This is the highest-value integration test — actual wire format,
actual serialisation, actual auth, actual middleware. Catches a
huge class of bugs that mocked-RPC tests miss.

## Event-driven assertions

```typescript
import { waitForEvents } from '@omnitron-dev/testing';

it('fires user.created after invite', async () => {
  const container = mod.getContainer();
  const bus = container.resolve(EVENT_BUS_TOKEN);
  const users = container.resolve(UsersService);

  const eventPromise = waitForEvents(bus, ['user.created'], 2_000);
  await users.invite({ email: 'a@b.c' });

  const [user] = await eventPromise;
  expect(user.email).toBe('a@b.c');
});
```

`waitForEvents` resolves with one payload per event name, in the order
requested. Set up the wait **before** triggering the action — race-free.
`createEventSpy(bus, 'user.created')` is the alternative when you want
to inspect everything that was emitted rather than wait for the first.

## Custom routes

Custom routes alongside RPC need their own tests:

```typescript
it('serves images via /render/image/*', async () => {
  await app.start();
  const port = app.netron.getPort('http');

  const r = await fetch(`http://localhost:${port}/render/image/bucket-1/photo.jpg?width=200`);
  expect(r.status).toBe(200);
  expect(r.headers.get('content-type')).toBe('image/webp');
});
```

Hit the route with a plain `fetch`; assert on status, headers,
body.

## Speed budget

For an integration suite to stay fast:

| Tier                     | Budget per test            |
| ------------------------ | -------------------------- |
| Bare module              | <50 ms                     |
| With in-memory DB        | <100 ms                    |
| With Postgres rollback   | <200 ms                    |
| With real HTTP roundtrip | <500 ms                    |
| With Docker bootstrap    | excluded from common suite |

If you blow past those, you're probably testing too much in
one — split the test or push to E2E.

## CI considerations

- **Parallelism**: Vitest forks per-file. Tests within a file
  share a process; tests across files don't. Keep file-level
  state minimal.
- **DB connections**: limit pool size in test config; many
  parallel workers + large pool = connection exhaustion.
- **Container reuse**: in CI, prefer reusing Docker containers
  across runs (mark with a known name + don't auto-stop).
- **Snapshot updates**: gate behind explicit `pnpm test --update`
  — never auto-update in CI.

## Best practices

- **One test, one behaviour.** A failing test should point at a
  single change.
- **Real DI in integration**, not stubs. Stubs of DI mask wiring
  bugs.
- **Real Application** lifecycle, not mocked hooks.
- **Override at boundaries**: mailer, payment provider, third-
  party APIs. Don't override internal services.
- **Fake the clock**, never sleep.

## Anti-patterns

- **Asserting on log output.** Logs change; assert on observable
  state.
- **Asserting on private state.** Use public APIs to introspect.
- **Coupling test order.** Each test must work standalone.
- **Sharing app across tests** without proper reset. State leaks
  across tests are insidious.
- **Snapshot tests for non-deterministic output.** Timestamps,
  IDs, hashes — exclude or sanitise.

## See also

- [Testing overview](/docs/testing)
- [Testing package](/docs/testing/testing-package) — `createTestApp` API
- [React component testing](/docs/testing/react)
- [Cross-runtime testing](/docs/testing/cross-runtime)
- [Titan / Testing / DI overrides](/docs/titan/testing/di-overrides)

---

---
id: "testing/react"
title: "React testing"
description: "Testing React components that call Netron — TestNetronProvider patterns."
---

# React testing

Components built on `netron-react` need RPC mocks; Prism
components need a `<PrismProvider>`. This page covers both with
patterns that scale.

## TestNetronProvider + multi-backend

| Provider                                                                          | When                 |
| --------------------------------------------------------------------------------- | -------------------- |
| **`TestNetronProvider`** (from `@omnitron-dev/netron-react/test`)                 | Single-backend tests |
| **`MultiBackendProvider`** (from `@omnitron-dev/netron-react`, fed a mock client) | Multi-backend tests  |

`TestNetronProvider` builds a `NetronReactClient` substitute
(via `createTestClient`) that intercepts RPC calls and returns
the canned data from its `mocks`.

## Single-backend mock

```tsx
import { describe, it, expect } from 'vitest';
import { render, screen } from '@testing-library/react';
import { TestNetronProvider } from '@omnitron-dev/netron-react/test';
import { UserCard } from './UserCard.js';

describe('UserCard', () => {
  it('renders user email', async () => {
    render(
      <TestNetronProvider
        testConfig={{
          mocks: [
            { service: 'users', method: 'getUser', response: { id: '1', email: 'a@b.c' } },
          ],
        }}
      >
        <UserCard userId="1" />
      </TestNetronProvider>
    );

    await screen.findByText('a@b.c');
  });
});
```

Two pieces:

- **`testConfig.mocks`** — an array of `MockResponse` entries
  (`{ service, method, response?, error?, delay? }`). The test
  client's `invoke` matches on `service.method` and returns the
  canned `response` (or throws `error`).
- **`<TestNetronProvider testConfig={{ mocks }}>`** — wraps your
  component with a `NetronProvider` backed by the test client.

To branch on call arguments, build a service object with
**`createMockService<T>(implementations)`** (async methods, any
subset of the interface) and inject it as the client's service.

## Multi-backend mock

```tsx
import { MultiBackendProvider } from '@omnitron-dev/netron-react';

render(
  <MultiBackendProvider client={mockMultiBackendClient} autoConnect={false}>
    <Dashboard />
  </MultiBackendProvider>
);
```

Uses the production `<MultiBackendProvider>` directly — supply a
mock multi-backend client that satisfies the same routing
contract.

## With Prism components

Prism components need a `<PrismProvider>` for theme + snackbar
host. Wrap once at the test boundary:

```tsx
import { PrismProvider } from '@omnitron-dev/prism/core';

import { TestNetronProvider, type TestClientConfig }
  from '@omnitron-dev/netron-react/test';
import type { NetronReactClient } from '@omnitron-dev/netron-react';

function TestProviders({ children, testConfig, client }: {
  children: React.ReactNode;
  testConfig?: TestClientConfig;
  client?: NetronReactClient;
}) {
  return (
    <PrismProvider defaultSettings={{ mode: 'light' }}>
      <TestNetronProvider client={client} testConfig={testConfig}>
        {children}
      </TestNetronProvider>
    </PrismProvider>
  );
}

// In a test:
render(
  <TestProviders testConfig={{ mocks: [{ service: 'users', method: 'list', response: users }] }}>
    <UsersPage />
  </TestProviders>
);
```

For tests that use blocks (`<DataGridBlock>`, `<AuthBlock>`, …),
this wrapper is mandatory.

## Loading + error states

```tsx
it('shows skeleton while loading', async () => {
  render(
    <TestNetronProvider
      testConfig={{
        mocks: [{ service: 'users', method: 'getUser', response: user, delay: 100 }],
      }}
    >
      <UserCard userId="1" />
    </TestNetronProvider>
  );

  expect(screen.getByTestId('skeleton')).toBeInTheDocument();
  await screen.findByText('a@b.c');
  expect(screen.queryByTestId('skeleton')).not.toBeInTheDocument();
});

it('shows error on NOT_FOUND', async () => {
  render(
    <TestNetronProvider
      testConfig={{
        mocks: [{
          service: 'users',
          method:  'getUser',
          error:   new TitanError({ code: ErrorCode.NOT_FOUND, message: 'not found' }),
        }],
      }}
    >
      <UserCard userId="missing" />
    </TestNetronProvider>
  );

  await screen.findByText(/not found/i);
});
```

## Mutation assertions

```tsx
import { createTestClient, createMockService } from '@omnitron-dev/netron-react/test';

it('calls invite on submit', async () => {
  const user   = userEvent.setup();
  const invite = vi.fn(async () => ({ id: 'new', email: 'x@y.z' }));
  const usersMock = createMockService<UserService>({ invite });

  const client = createTestClient();
  client.service = (() => usersMock) as never;   // inject the mock service

  render(
    <TestProviders client={client}>
      <InviteForm />
    </TestProviders>
  );

  await user.type(screen.getByLabelText('Email'), 'x@y.z');
  await user.click(screen.getByRole('button', { name: 'Invite' }));

  await waitFor(() => {
    expect(invite).toHaveBeenCalledWith({ email: 'x@y.z' });
  });
});
```

`useMutation` exposes `isLoading` while in flight (not
`isPending`), plus `isSuccess` / `isError` / `data`.

## Subscriptions

For streaming RPCs consumed by `useSubscription`, build a
service whose method returns an async iterable with
`createMockService`, then inject it as the client's service:

```tsx
import { createTestClient, createMockService } from '@omnitron-dev/netron-react/test';

it('updates on stream events', async () => {
  const orders = createMockService<OrderService>({
    watchAll: async function* () {
      yield { type: 'created', orderId: '1' };
      yield { type: 'updated', orderId: '1', status: 'paid' };
    },
  });

  const client = createTestClient();
  client.service = (() => orders) as never;

  render(
    <TestProviders client={client}>
      <OrderStream />
    </TestProviders>
  );

  await screen.findByText('1: created');
  await screen.findByText('1: paid');
});
```

The generator yields each item in order — components using
`useSubscription` see them as they arrive.

## Suspense + error boundary

Suspense is the `suspense: true` option on `useQuery` (there is
no separate `useSuspenseQuery` hook) — it throws the in-flight
fetch to the nearest `<Suspense>` and escalates errors to the
nearest error boundary:

```tsx
it('catches render error via boundary', async () => {
  render(
    <TestProviders
      testConfig={{
        mocks: [{ service: 'users', method: 'getUser', error: new Error('boom') }],
      }}
    >
      <ErrorBoundary fallback={() => <div>caught</div>}>
        <Suspense fallback={<div>loading</div>}>
          {/* SuspenseUserCard calls useQuery([...], { suspense: true }) */}
          <SuspenseUserCard userId="1" />
        </Suspense>
      </ErrorBoundary>
    </TestProviders>
  );

  await waitFor(() => expect(screen.getByText('caught')).toBeInTheDocument());
});
```

Suspense + error boundary tests verify the "graceful degradation"
path that production needs.

## Cache control

```tsx
import { createTestClient } from '@omnitron-dev/netron-react/test';
import { NetronProvider } from '@omnitron-dev/netron-react';

const client = createTestClient({
  mocks:  [{ service: 'users', method: 'getUser', response: user }],
  config: { cache: { staleTime: 0, cacheTime: 0 } },   // disable cache
});

render(
  <NetronProvider client={client}>
    <App />
  </NetronProvider>
);
```

With `staleTime: 0`, every mount refetches — useful for
asserting "fetched N times" without cache surprises.

## Form testing pattern

For forms built with `<Field>` + `SchemaProvider`:

```tsx
import { userEvent } from '@testing-library/user-event';

it('validates email on blur', async () => {
  const user = userEvent.setup();

  render(<TestProviders><SignInForm /></TestProviders>);

  const email = screen.getByLabelText('Email');
  await user.type(email, 'not-an-email');
  await user.tab();                              // blur

  await screen.findByText(/invalid email/i);
});

it('submits on Enter', async () => {
  const onSubmit = vi.fn();
  render(<TestProviders><SignInForm onSubmit={onSubmit} /></TestProviders>);

  await userEvent.type(screen.getByLabelText('Email'),    'a@b.c');
  await userEvent.type(screen.getByLabelText('Password'), 'correct-horse{Enter}');

  await waitFor(() => {
    expect(onSubmit).toHaveBeenCalledWith({ email: 'a@b.c', password: 'correct-horse' });
  });
});
```

`@testing-library/user-event` simulates real keyboard / mouse
input — far more reliable than `fireEvent`.

## Routing tests

For components that use `react-router-dom`:

```tsx
import { MemoryRouter } from 'react-router-dom';

render(
  <MemoryRouter initialEntries={['/users/u_42']}>
    <TestProviders testConfig={{ mocks: [{ service: 'users', method: 'getUser', response: user }] }}>
      <Routes>
        <Route path="/users/:id" element={<UserPage />} />
      </Routes>
    </TestProviders>
  </MemoryRouter>
);
```

`MemoryRouter` lets you set the initial URL and inspect the
history without a real browser.

## E2E with Playwright

For browser-driven tests:

```typescript
// e2e/sign-in.spec.ts
import { test, expect } from '@playwright/test';

test.beforeEach(async ({ page }) => {
  await page.goto('http://localhost:5173');
});

test('signs in', async ({ page }) => {
  await page.getByLabel('Email').fill('a@b.c');
  await page.getByLabel('Password').fill('correct-horse');
  await page.getByRole('button', { name: 'Sign in' }).click();
  await expect(page).toHaveURL(/\/$/);
  await expect(page.getByText('a@b.c')).toBeVisible();
});
```

Playwright E2E suites live at `packages/prism/tests/e2e/` and `packages/netron-browser/tests/e2e/`.

## Best practices

- **Wrap once at the top.** Build a `TestProviders` component
  with all your providers (Prism + Netron mocks + Router) and
  reuse it.
- **Assert via the DOM**, not mock internals where possible.
  `screen.getByText(...)` beats `expect(mock).toHaveBeenCalled()`
  when both are available.
- **`userEvent`, not `fireEvent`.** Real input simulation
  catches more bugs.
- **`findBy` for async**, `getBy` for sync. Don't use `getBy`
  before an async effect resolves.
- **One test per behaviour.** If a test has two `expect`s
  asserting two different intents, split it.

## Anti-patterns

- **Mocking `useService` directly.** Use `TestNetronProvider` —
  it preserves the full hook contract (loading, error, refetch,
  etc.).
- **`act()` warnings ignored.** They mean React's batching
  surprised you; fix the test.
- **Real `fetch` in unit tests.** Use `TestNetronProvider`.
- **Shared mock state across tests.** `mockClear()` in `beforeEach`
  or recreate.

## See also

- [Testing overview](/docs/testing)
- [Testing package](/docs/testing/testing-package)
- [Integration patterns](/docs/testing/integration)
- [Netron React testing](/docs/frontend/netron/testing) — `TestNetronProvider` API
- [Prism overview](/docs/frontend/prism) — `<PrismProvider>` wiring

---

---
id: "testing/testing-package"
title: "@omnitron-dev/testing"
description: "Cross-runtime testing utilities — async helpers, runtime adapters, mock types."
---

# @omnitron-dev/testing

```bash
pnpm add -D @omnitron-dev/testing
```

Cross-runtime testing helpers. Same test source runs on
**Vitest** (Node + Bun) and **Deno test**. Provides typed mock
primitives, async helpers, runtime detection, and Titan-specific
test glue.

Verified against `packages/testing/src/`.

## What's inside

**packages/testing/src/**

- `async/` — Promise + timer + event helpers for tests
- `docker/` — Helpers to spin up Postgres / Redis in tests
- `helpers/` — Generic test helpers
- `performance/` — Benchmarks & latency utilities
- `runtime/` — Runtime adapters (vitest / jest / bun / deno)
- `titan/` — Titan-specific test utilities (Application, DI overrides)
- `env.ts` — Test environment helpers
- `errors.ts` — Error-matching helpers
- `globals.d.ts` — Vitest / Jest globals (no import)

## Runtime detection

```typescript
import { RUNTIME, loadRuntimeAdapter } from '@omnitron-dev/testing';

RUNTIME;                        // 'node' | 'bun' | 'deno'

const adapter = await loadRuntimeAdapter();
adapter.test('my test', () => { /* ... */ });
adapter.expect(actual).toBe(expected);
```

The same test code path runs unchanged on Node + Bun (via Vitest) and
Deno (via Deno's native `Deno.test`) for `describe`, `it`, `test`,
`expect`, the four hooks and `fakeTimers` — a surface test in the
package fails if the three adapters drift apart.

**Mocking is the exception**: it is `vi` on Node and Bun, `mockFn` on
Deno. If your test mocks, that part is not portable as written.

## Typed mock function

```typescript
import type { MockFunction } from '@omnitron-dev/testing';

const fetchMock: MockFunction<typeof fetch> = vi.fn();

fetchMock.mockResolvedValue(new Response('hello'));
await someCode(fetchMock);

expect(fetchMock).toHaveBeenCalledWith('/api/foo');
expect(fetchMock.mock.calls).toHaveLength(1);
expect(fetchMock.mock.lastCall()).toEqual(['/api/foo']);
```

`MockFunction<T>` preserves `T`'s type so `mock.calls[0]` is the
inferred parameter tuple, not `any[]`.

## Async helpers — `@omnitron-dev/testing/async`

Also re-exported from the package root.

### `waitFor(condition, options?)` / `waitForCondition(condition, timeout?, interval?)`

Poll until a predicate is true, or throw at the deadline:

```typescript
import { waitFor } from '@omnitron-dev/testing';

await waitFor(() => queue.size === 0, {
  timeout: 5_000,
  interval: 50,
  message: 'queue did not drain',
});
```

Defaults: 5 s timeout, 50 ms interval, message `'Condition not met'`.
`waitForCondition(fn, timeout, interval)` is the same check with
positional arguments.

### `waitForEvents(target, events, timeout?)`

Wait for several events on an emitter at once, resolving with their
payloads in the order requested:

```typescript
import { waitForEvents } from '@omnitron-dev/testing';

const [created, indexed] = await waitForEvents(bus, ['user.created', 'user.indexed'], 2_000);
expect(created.email).toBe('a@b.c');
```

### `createEventSpy(target, event)` and `EventCollector`

`createEventSpy` records every payload for one event and hands back
`{ events, clear }`. `EventCollector` does the same for several events
and adds assertions:

```typescript
import { EventCollector, createEventSpy } from '@omnitron-dev/testing';

const spy = createEventSpy(bus, 'user.created');
await service.invite({ email: 'a@b.c' });
expect(spy.events).toHaveLength(1);

const collector = new EventCollector(bus);
collector.collect('user.created').collect('user.deleted');
// …
collector.assertEmitted('user.created', 1);
collector.assertNotEmitted('user.deleted');
collector.stop();
```

### `EventListenerTracker`

Registers listeners and removes all of them in one call — the usual
cause of a leaking test suite is a listener nobody detached:

```typescript
const tracker = new EventListenerTracker();
tracker.on(emitter, 'data', handler);
afterEach(() => tracker.cleanup());
```

### `flushPromises()`

```typescript
import { flushPromises } from '@omnitron-dev/testing';

doSomethingThatScheduledMicrotasks();
await flushPromises();             // microtask queue drained
expect(someState).toBe(/* … */);
```

### `withTimeout(promise, ms)` / `retry(fn, options?)` / `delay(ms)` / `nextTick()`

```typescript
const result = await withTimeout(longRunning(), 3_000);  // throws TimeoutError
const value = await retry(() => flakyCall(), { retries: 3, delay: 100, backoff: 2 });
```

`RetryOptions` is `{ retries?, delay?, backoff?, onRetry? }`. `retries`
defaults to 3, so a call that passes an unrecognised key still runs three
times and looks like it worked — this page previously documented
`{ attempts: 3 }`, which is not read at all.

`onRetry(error, attempt)` fires between attempts, which is the hook to use when
a flaky test needs to say WHY it retried rather than only that it passed.

### `createDeferred()` / `defer()` — a promise resolved from outside

Two shapes of the same idea, and the difference is what you get back:

```typescript
const d = defer<string>();          // { promise, resolve, reject } — a plain object
emitter.once('ready', () => d.resolve('ok'));
await d.promise;

const p = createDeferred<string>(); // a DeferredPromise<T> instance
```

`defer()` returns a bare `Deferred<T>` — `{ promise, resolve, reject }`, no
class. `createDeferred()` returns a `DeferredPromise<T>`, which adds one thing:
the `isSettled` getter.

That is the whole difference, and it is worth stating plainly because the
obvious guess is wrong. A second `resolve` is ignored by BOTH — that is
ordinary Promise semantics, not something the class adds. What the class adds
is the ability to ASK whether the promise has settled without awaiting it,
which a test needs when it has to assert that something has NOT happened yet.

Use `defer()` unless you need that question answered.

Either is the right answer when a test must wait for something a callback
signals, and the wrong answer when a condition can simply be polled —
`waitFor` exists for that and does not leave a promise dangling if the event
never arrives.

### `collectEvents(target, event, condition, timeout?)`

Gathers emissions of one event until `condition(events)` returns true, then
resolves with them. `waitForEvents` above waits for one of EACH named event;
this waits for as many of ONE as the condition asks for.

### `createMockTimer()` / `MockTimerController`

Drive time forward deterministically instead of sleeping.

## Errors — `errors.ts`

The package exports its own error types, thrown by the helpers above:

```typescript
import { TestingError, TimeoutError, NotFoundError, RetryError } from '@omnitron-dev/testing';

await expect(withTimeout(hangs(), 100)).rejects.toBeInstanceOf(TimeoutError);
```

For asserting that a call rejects, use `assertRejects` from
`@omnitron-dev/testing/helpers`:

```typescript
import { assertRejects } from '@omnitron-dev/testing/helpers';

await assertRejects(service.load('missing'), /not found/);
await assertRejects(service.load('missing'), NotFoundError);
```

It accepts a string, a RegExp, or an error constructor.

## Generic helpers — `@omnitron-dev/testing/helpers`

```typescript
import { createTempDir, cleanupTempDir, suppressConsole, withFixture } from '@omnitron-dev/testing/helpers';

const dir = await createTempDir();
afterEach(() => cleanupTempDir(dir));

const restore = suppressConsole();   // silence console.* for one test
restore();

await withFixture(myFixture, async (instance) => { /* … */ });
```

## Titan-specific glue — `@omnitron-dev/testing/titan`

### `createTestModule(options)` / `testModule()`

Builds a container with your modules, providers and mocks, and can
create an `Application` from it:

```typescript
import { createTestModule } from '@omnitron-dev/testing/titan';
import { AppModule } from '../src/app.module.js';

describe('users service', () => {
  let mod: TestModule;

  beforeEach(() => {
    mod = createTestModule({
      modules: [AppModule],
      providers: [[MAILER_TOKEN, { useClass: FakeMailer }]],
      mocks: [{ token: CLOCK_TOKEN, mock: fixedClock, spy: true }],
      config: { name: 'test-app' },
    });
  });

  afterEach(async () => {
    await mod.cleanup();
  });

  it('invites a user', async () => {
    const users = mod.getContainer().resolve(UsersService);
    await users.invite({ email: 'a@b.c' });
  });
});
```

`resetMocks()`, `clearMocks()` and `restore()` manage mock state between
tests; `createApplication()` returns a started `Application` when the
test needs the full lifecycle.

`testModule()` is the same thing with a fluent builder:

```typescript
const mod = testModule()
  .withModule(AppModule)
  .withConfig({ name: 'test-app' })
  .withAutoMock()
  .build();
```

### `TestApplication`

A thinner wrapper when you want the Application itself:

```typescript
import { TestApplication } from '@omnitron-dev/testing/titan';

const app = new TestApplication({ name: 'test-app' });
await app.bootstrap(AppModule);

const users = app.get(UsersService);
await app.close();
```

### Fixtures

`TestSchemas`, `TestConfigs`, `TestRedisConfigs`, `TestModules`,
`TestData` and `TestTiming` are ready-made fixtures for the common
shapes — see `packages/testing/src/titan/test-fixtures.ts`.

## Docker helpers — `@omnitron-dev/testing/docker`

Use the database and Redis helpers when an integration test needs a real service. Each factory returns a `DockerContainer` with its assigned ports and a `cleanup()` method.

```typescript
import { RedisTestManager, DatabaseTestManager, type DockerContainer } from '@omnitron-dev/testing/docker';

let redis: DockerContainer;
let postgres: DockerContainer;

beforeAll(async () => {
  redis = await RedisTestManager.createRedisContainer({ port: 'auto' });
  postgres = await DatabaseTestManager.createPostgresContainer({ port: 'auto' });
}, 60_000);

afterAll(async () => {
  await redis.cleanup();
  await postgres.cleanup();
});
```

With `port: 'auto'`, read the assigned loopback port from `container.ports.get(6379)`. PostgreSQL and MySQL readiness probes use TCP so a temporary initialization socket cannot mark the database ready. A custom `waitFor` that specifies both `port` and `healthcheck` waits for both conditions.

Choose distinct names for parallel tests. A name collision fails before the helper can touch the existing container. Cleanup addresses the Docker ID that the manager created. You can attach tests to an existing network by name. The network stays with its owner.

In the Omnitron repository, `pnpm test:ci` starts a dedicated Compose project, runs the workspace suite, and removes that project's test resources. A failed startup, suite or teardown returns a nonzero exit status. When running two suites on the same host, assign different test ports as described in the root Compose file.

`RedisTestManager` also builds multi-node topologies —
`createRedisCluster()` and `createRedisSentinel()` — and
`DockerTestManager` is the lower-level driver the other two use.

For container commands, pass an array to preserve literal arguments, including
empty strings: `command: ['redis-server', '--save', '']`. A string deliberately
runs through `sh -c`; never interpolate untrusted values into that form. Docker
starts run asynchronously so one node cannot block another node's readiness
checks. If cluster creation fails, the helper waits for every attempted creation,
cleans all returned nodes, and removes its own network. Startup and cleanup
failures remain visible together.

## Performance helpers — `@omnitron-dev/testing/performance`

```typescript
import { PerfTimer, MemoryLeakDetector } from '@omnitron-dev/testing/performance';

const timer = new PerfTimer();
timer.mark('start');
await parse(LARGE_INPUT);
timer.mark('end');

expect(timer.measure('parse', 'start', 'end')).toBeLessThan(50);
// Repeated runs: timer.getAverage('parse'), timer.getPercentile('parse', 95)
```

A percentile over repeated runs is the useful shape here: a single
wall-clock measurement compared against a flat millisecond bound
measures the machine as much as the code.

## Env helpers — `@omnitron-dev/testing/env`

The shared endpoints for the test stack. Ports deliberately differ from
the defaults so a test run cannot reach a developer's own Redis or
Postgres:

```typescript
import {
  TEST_REDIS_URL,      // redis://localhost:16379
  TEST_POSTGRES_URL,   // postgresql://test:test@localhost__LUMEN_COLON__15432/test
  testRedisUrl,
  testPostgresUrl,
} from '@omnitron-dev/testing/env';

const url = testRedisUrl(5);            // …:16379/5
const dbUrl = testPostgresUrl('other'); // …:15432/other
```

Every value is overridable through the matching environment variable
(`TEST_REDIS_HOST`, `TEST_REDIS_PORT`, `TEST_POSTGRES_*`).

Use the shared `TEST_POSTGRES_URL` in database tests instead of repeating a
fixed port. An explicit `TEST_POSTGRES_URL` takes precedence; the existing
`TEST_DATABASE_URL` CI variable remains accepted by the shared resolver.
`testPostgresUrl(name)` changes only the database name and preserves the selected
server, credentials, and query options. Omnitron's PostgreSQL integration probes
fail CI when that test service is unavailable. Local runs report the affected
suites as skipped and explain how to start the test infrastructure.

Workspace tests use Turbo's strict environment mode. The declared `TEST_*`,
`OMNITRON_TEST_*`, `OMNITRON_HOME` and `USE_MOCK_REDIS` settings reach test
processes and affect their cache keys. Docker connection settings and temporary
directories pass through explicitly. A custom test port therefore selects the
same endpoint in a direct package test and in the full workspace command.

## Vitest configuration baseline

```typescript
// vitest.config.ts
import { defineConfig } from 'vitest/config';

export default defineConfig({
  test: {
    environment:   'node',
    setupFiles:    ['./test/setup.ts'],
    include:       ['src/**/*.test.ts', 'test/**/*.test.ts'],
    coverage:      { reporter: ['text', 'html', 'lcov'] },
    pool:          'forks',          // crash isolation
    maxWorkers:    2,               // bounded concurrency
    testTimeout:   10_000,
    hookTimeout:   30_000,
    globals:       false,             // explicit imports
  },
});
```

When tests enable globals, declare `"types": ["node", "vitest/globals"]` in
their TypeScript configuration. Package resolution supports both hoisted and
isolated dependency layouts; an explicit `node_modules/vitest/globals.d.ts`
include assumes one layout. Omnitron shares these options through
`tsconfig.vitest.json`. Keep the test `tsBuildInfoFile` separate from the build
cache so incremental checks cannot reuse diagnostics from a different file set.

## Cross-runtime tests — Node + Bun + Deno

```typescript
// test/cross.test.ts (same file)
import { loadRuntimeAdapter } from '@omnitron-dev/testing';

const t = await loadRuntimeAdapter();

t.test('runs everywhere', () => {
  t.expect(1 + 1).toBe(2);
});
```

Run with:

```bash
vitest run test/cross.test.ts            # Node
bun test test/cross.test.ts              # Bun
deno test test/cross.test.ts             # Deno
```

Same source, three runtimes, identical assertions.

## Best practices

- **Use `createTestApp`** for any test that touches multiple
  modules — it does the lifecycle right.
- **Override at boundaries**, not in the middle. Override the
  mailer (external boundary), don't override an internal
  service.
- **`database: 'memory'`** for unit-y module tests;
  `'rollback'` when behaviour depends on real Postgres
  semantics (RLS, advisory locks, jsonb queries).
- **`waitForEvent`** for async assertions; avoid `setTimeout`.
- **Use `expectThrowsAsync`** instead of `try/catch + expect.fail` —
  more readable, captures more info.

## Anti-patterns

- **Sleeping for promises to resolve.** Use `flushPromises` or
  `eventually`.
- **Shared `Application` across tests** without proper reset.
  Mutated DI state leaks.
- **Real network in unit tests.** Use mocks; reserve real for
  integration / E2E.
- **Tests that depend on order.** Vitest parallelises; an
  order-dependent test is a future flake.

## See also

- [Testing overview](/docs/testing) — the pyramid + when to use what
- [Integration patterns](/docs/testing/integration) — `Application.create` recipes
- [Cross-runtime testing](/docs/testing/cross-runtime) — Node + Bun + Deno
- [React component testing](/docs/testing/react) — `TestNetronProvider` patterns
- [common — promise helpers](/docs/utilities/common#promise-helpers) — building blocks `testing` builds on

---

---
id: "titan/application/bootstrap"
title: "Bootstrap"
description: "How Application.create + start actually run."
---

# Bootstrap

Bootstrap is the bridge between "I have a class" and "I have a
running service". Three steps, in this order.

```mermaid
sequenceDiagram
  participant Code as Your code
  participant App as Application
  participant Cont as Container
  participant Mods as Modules
  participant Prov as Providers
  participant Net as Netron / Transports

  Code->>App: Application.create({ modules: [AppModule] })
  App->>Cont: new Container()
  App->>Mods: register core + AppModule + transitive imports
  Mods->>Cont: register providers
  Cont->>Cont: analyse graph, detect cycles
  App->>Prov: eagerly init singletons
  App-->>Code: Application (state: Created)

  Code->>App: app.start()
  App->>Prov: onInit (dep order)
  App->>Prov: onStart (dep order)
  App->>Net: bind transports (if configured)
  App-->>Code: state: Started
```

## Step 1 — `Application.create(...)`

Two overloads:

```typescript
// Form 1: module-first.
Application.create(AppModule, options?)

// Form 2: options-only — matches the canonical README example.
Application.create({ modules: [AppModule], ...otherOptions })
```

Related helpers:

```typescript
import { createApp, createAndStartApp } from '@omnitron-dev/titan';

createApp           // sync new Application(options) — constructor only;
                    // core/extra modules register lazily on start()
createAndStartApp   // create + start in one call (see Step 2)
```

> **`createApp` is not `Application.create`.** The `createApp` exported
> from the package root is a thin sync wrapper over the constructor
> (`new Application(options)`). It does **not** register a main module,
> core modules, or run auto-discovery — those happen inside the async
> `Application.create` factory (or lazily during `start()`). Use
> `Application.create({ modules: [...] })` when you want the full wiring
> up front. (There is a separate `createApp` in the
> `@omnitron-dev/titan/application` Simple API that *is* bound to
> `Application.create` — but the root-barrel export is the sync one.)

What happens inside `create`:

1. **Application instance** — `new Application(options)` builds the
   instance with its container, lifecycle state machine, module
   registry, and shutdown coordinator.
2. **Main module registration** — if you passed a module class as
   the first argument, it's registered.
3. **Core modules** — unless `disableCoreModules: true`,
   `ConfigModule` and `LoggerModule` are registered.
4. **Auto-discovery** — if `autoDiscovery: true` or `scanPaths` is
   set, modules are discovered from the filesystem (off by default).
5. **Additional modules** — anything in `options.modules` is
   registered next; anything in `options.imports` is imported by
   token.
6. **Root-level providers** — if `options.providers` is set, a
   synthetic root module wraps them and registers each in the
   container.
7. **Eager singleton init** — `container.eagerlyInitialize()` runs
   so cross-module dependencies are resolvable when lifecycle hooks
   fire.

After `create` returns, the container is fully wired but **no
lifecycle hook has fired**. The state is `Created`.

## Step 2 — `app.start()`

```typescript
await app.start();
```

What happens, in order:

1. **State transition** — `Created → Starting`. The lifecycle state
   machine takes over.
2. **`onInit` phase** — every provider that implements `OnInit` (or
   has a `@PostConstruct` method) runs in **dependency order**.
3. **`onStart` phase** — every provider that implements `OnStart`
   runs in dependency order. This is where most modules open
   external connections.
4. **State transition** — `Starting → Started`.

After `start` resolves, every provider has had its `onInit` and
`onStart` called. The application is **live**.

> **Implementation note.** The phases above are the *contract* the
> framework guarantees, not a literal 1 of the call order inside
> `_doStart`. In practice the Nexus container fires most provider
> `onInit` / `@PostConstruct` hooks during the eager singleton
> initialisation that runs inside `Application.create` (and a final
> `container.initialize()` sweep at the tail of `start`), while module
> `onStart` hooks run during `start`. What you can rely on: a provider's
> `onInit` completes before its own `onStart`, and both respect
> dependency order. Don't depend on the absolute wall-clock interleaving
> of unrelated providers' phases.

A failure in any phase aborts the start, rolls back modules that
reached `onStart` (in reverse order) via their `onStop`, disposes the
container, transitions to `Failed`, and rejects with the original
error.

## Step 3 — running

Between `start` and `stop`, the application does almost nothing
itself. The kernel waits, listening for:

- A `stop()` call.
- A bound OS signal (`SIGTERM`, `SIGINT`, `SIGHUP`).
- An uncaught exception or unhandled rejection (configurable).

When one of these fires, the kernel transitions to `Stopping` and
runs the shutdown coordinator (see [Shutdown](/docs/titan/application/shutdown)).

## Useful variants

### `createAndStartApp` — when you want both calls together

```typescript
import { createAndStartApp } from '@omnitron-dev/titan';

const app = await createAndStartApp({
  name:    'my-app',
  version: '1.0.0',
  modules: [AppModule],
});
```

Signature (from the actual source):

```typescript
createAndStartApp(options?: {
  name?:    string;
  version?: string;
  config?:  any;
  modules?: IModule[];
}): Promise<IApplication>;
```

The function calls `createApp(...)`, registers any modules in the
list via `app.use()`, then awaits `app.start()`. Use when you don't
need the intermediate `Created` state.

### `titan()` — the simplest entry point

The Simple API in `@omnitron-dev/titan/application` (re-exported as
the default export of the package) wraps everything:

```typescript
import titan from '@omnitron-dev/titan/application';

// Either a module…
const app = await titan(AppModule);

// …or a bare config, which builds the module for you:
const appFromConfig = await titan({ port: 3000, redis: 'localhost:6379' });
```

Smart defaults: pretty logging in dev, JSON in prod, graceful
shutdown enabled, scan paths set. Use for prototypes and small apps;
for production stacks, use `Application.create` explicitly so the
configuration is visible.

## Bootstrap diagnostics

Set `debug: true` or `logging: { level: 'debug' }` to see each phase
in the structured log. Look for `phase` and `provider` fields to
trace lifecycle ordering.

## Anti-patterns

- **Calling `start` twice.** It is a no-op after the first
  successful start, but a reliance on idempotency hides bugs.
- **Logging from constructors.** Constructors run during container
  resolution, before `LoggerModule` has fully started. Use `onInit`
  or `onStart` instead.
- **Opening connections in constructors.** Constructors should be
  cheap. Open expensive resources in `onStart`.
- **Catching `start` errors and continuing.** A failed `start`
  leaves the app in `Failed` state. Re-throw and let the supervisor
  restart the process.

---

---
id: "titan/application/events"
title: "Application Events"
description: "The framework event bus — what fires, what to subscribe to, and what to ignore."
---

# Application Events

The kernel emits a typed set of events through the `Application`
event bus. Subscribe when you need to observe framework behaviour or
instrument the runtime.

## API

```typescript
import { ApplicationEvent } from '@omnitron-dev/titan';

app.on(ApplicationEvent.Started, (data) => { /* … */ });
app.off(ApplicationEvent.Started, handler);
app.once(ApplicationEvent.Started, (data) => { /* … */ });
app.emit('user.created', { id, email });   // user events too
```

`on` accepts a typed `ApplicationEvent` enum value (or a string for
custom events). Handlers receive `(data, meta?)` where `meta`
carries event metadata. Handler return values are ignored.

## Framework-emitted events

| Enum                                    | String value               | Payload (as emitted by the kernel)                                |
| --------------------------------------- | -------------------------- | ----------------------------------------------------------------- |
| `ApplicationEvent.Starting`             | `'starting'`               | *(none)*                                                          |
| `ApplicationEvent.Started`              | `'started'`                | *(none)*                                                          |
| `ApplicationEvent.Stopping`             | `'stopping'`               | *(none)*                                                          |
| `ApplicationEvent.Stopped`              | `'stopped'`                | *(none)*                                                          |
| `ApplicationEvent.Error`                | `'error'`                  | the raw `Error` (or `{ error }` from the bus)                     |
| `ApplicationEvent.ModuleRegistered`     | `'module:registered'`      | `{ module }`                                                      |
| `ApplicationEvent.ModuleStarted`        | `'module:started'`         | `{ module }`                                                      |
| `ApplicationEvent.ModuleStopped`        | `'module:stopped'`         | `{ module }`                                                      |
| `ApplicationEvent.ConfigChanged`        | `'config:changed'`         | `{ config }` (full reconfigure) or `{ key, value }` (`setConfig`) |
| `ApplicationEvent.HealthCheck`          | `'health:check'`           | *(reserved; not emitted by the kernel itself)*                    |
| `ApplicationEvent.Signal`               | `'signal'`                 | `{ signal }`                                                      |
| `ApplicationEvent.UncaughtException`    | `'uncaughtException'`      | `{ error }` (or `{ error, recovered: true }`)                     |
| `ApplicationEvent.UnhandledRejection`   | `'unhandledRejection'`     | `{ reason, promise }`                                             |
| `ApplicationEvent.StateSave`            | `'state:save'`             | *(none)*                                                          |
| `ApplicationEvent.ShutdownStart`        | `'shutdown:start'`         | `{ reason, details }`                                             |
| `ApplicationEvent.ShutdownComplete`     | `'shutdown:complete'`      | `{ reason, success }`                                             |
| `ApplicationEvent.ShutdownError`        | `'shutdown:error'`         | `{ reason, error }`                                               |
| `ApplicationEvent.ShutdownTaskComplete` | `'shutdown:task:complete'` | `{ task }`                                                        |
| `ApplicationEvent.ShutdownTaskError`    | `'shutdown:task:error'`    | `{ task, error }`                                                 |
| `ApplicationEvent.LifecyclePhaseEvent`  | `'lifecycle:phase'`        | `LifecyclePhaseEvent` (`{ phase, kind, taskName?, error?, … }`)   |
| `ApplicationEvent.ProcessExit`          | `'process:exit'`           | `{ code }`                                                        |
| `ApplicationEvent.Custom`               | `'custom'`                 | depends on the emitter                                            |

The full enum is exported as `ApplicationEvent` from
`@omnitron-dev/titan/application` or via the root barrel.

## Typed handlers

```typescript
app.on(ApplicationEvent.ShutdownComplete, ({ reason, success }) => {
  metrics.counter('app.shutdown', { reason, success: String(success) }).inc();
});
```

The data type is what the kernel actually emits for that event;
exact field shapes can change between minor versions, so handlers
that read deep fields should be defensive (use optional chaining).

## Custom events

Modules can emit and subscribe to their own events. The event name
is a free-form string; the payload is typed by the emitter.

```typescript
app.emit('user.created', { id, email });

app.on('user.created', (data) => { /* … */ });
```

Use sparingly. For domain events between services, prefer the
[`titan-events`](/docs/titan/modules/events) module — it gives you schema
validation, scheduled delivery, and structured handler composition.
The application event bus is for *framework-level* signalling and
ad-hoc lightweight notifications, not for business workflows.

## Common subscriptions

### Boot timing

`Started` carries no payload — read `app.uptime` (ms since start) at
the moment it fires:

```typescript
app.on(ApplicationEvent.Started, () => {
  metrics.histogram('app.boot.ms').observe(app.uptime);
});
```

### Crash signals

`Error` delivers the raw `Error` (the in-process event bus may wrap it
as `{ error }` — be defensive):

```typescript
app.on(ApplicationEvent.Error, (data) => {
  const error = data instanceof Error ? data : (data as { error?: Error }).error;
  oncall.page({ message: 'Titan lifecycle error', error });
});
```

### Config-change audit

```typescript
app.on(ApplicationEvent.ConfigChanged, (data) => {
  // `{ config }` on a full reconfigure, `{ key, value }` on setConfig().
  audit.log('config.changed', data);
});
```

### Shutdown bookkeeping

```typescript
app.on(ApplicationEvent.ShutdownComplete, ({ reason, success }) => {
  metrics.counter('app.shutdown', { reason, success: String(success) }).inc();
});
```

## Event ordering guarantees

- Framework events for the same phase fire in dependency order
  (same as the hooks themselves).
- Events emitted from inside a handler are queued; they run after
  the current handler returns.
- `emit` does **not** await handlers. For synchronous coordination,
  use the lifecycle hooks instead.

## Anti-patterns

- **Mutating application state from handlers.** Handlers are
  observers. If you find yourself starting / stopping providers
  from a handler, you are probably looking for a lifecycle hook.
- **Long-running async work in handlers.** Handlers are not
  awaited. An `async` handler that does heavy work will run, but
  the event bus will not delay subsequent events for it. For long
  work, use a scheduled job and trigger it from the handler.
- **Treating user events as a transport.** The application event
  bus is in-process and has no delivery guarantees. For
  cross-process events, use `titan-events` over redis or the
  notifications module.

[Defining modules](/docs/titan/modules-system/defining-modules) explains where to register the providers that emit and handle events.

---

---
id: "titan/application/health"
title: "Health"
description: "Liveness, readiness, indicator-based aggregation."
---

# Health

A Titan application reports three independent signals:

| Signal        | Question it answers                                 | Failure means                             |
| ------------- | --------------------------------------------------- | ----------------------------------------- |
| **Liveness**  | Is the process alive and responsive?                | Restart me                                |
| **Readiness** | Should the load balancer send me traffic right now? | Take me out of rotation, leave me running |
| **Healthy**   | Is every dependency I rely on currently working?    | Investigate; possibly degraded service    |

The kernel exposes an aggregate roll-up directly: `app.health()`
returns an `IHealthStatus` combining every registered module's health,
and `app.checkHealth(moduleName)` returns one module's status. Richer
indicator registration and HTTP probes come from the
[`titan-health`](/docs/titan/modules/health) ecosystem module.

## Reading health from inside the app

`titan-health` ships an `HealthService` and registers indicators with
the application. Inject it where you need to read aggregate health:

```typescript
import { Inject, Service } from '@omnitron-dev/titan';
import { Public } from '@omnitron-dev/titan/decorators';
import { HEALTH_SERVICE_TOKEN, type HealthService }
  from '@omnitron-dev/titan-health';

@Service({ name: 'Admin' })
class AdminService {
  constructor(@Inject(HEALTH_SERVICE_TOKEN) private readonly health: HealthService) {}

  @Public()
  async status() {
    return this.health.check();
  }
}
```

The exact shape of the return value and token names lives in the
`titan-health` package — consult its source for the canonical API.

## Indicators

An *indicator* answers "is this dependency healthy?". The aggregator
collects indicators from registered sources and combines them with a
worst-status-wins rule:

| Composition                      | Result      |
| -------------------------------- | ----------- |
| All `healthy`                    | `healthy`   |
| Any `degraded`, none `unhealthy` | `degraded`  |
| Any `unhealthy`                  | `unhealthy` |

`degraded` reports a warning; `unhealthy` reports a failed check. Readiness
decides whether the application can serve a particular workload. An operational
failure must not become a warning just because other checks pass, and a warning
must remain visible when readiness permits traffic.

## Adding an indicator

The indicator interface and registration helper live in
`titan-health`. Typically:

```typescript
import { Injectable } from '@omnitron-dev/titan';
import type { IHealthIndicator } from '@omnitron-dev/titan-health';

@Injectable()
export class StripeHealth implements IHealthIndicator {
  // Required. `registerIndicator` throws BadRequest without it —
  // it is also the key the indicator is reported under.
  readonly name = 'stripe';

  constructor(private readonly stripe: Stripe) {}

  async check() {
    try {
      await this.stripe.balance.retrieve();
      return { status: 'healthy' as const };
    } catch (e) {
      return {
        status:  'degraded' as const,
        details: { error: String(e) },
      };
    }
  }
}
```

Then register it. There are two paths, and they are not equivalent:

```typescript
// 1. Module options — the module does `new IndicatorClass()` itself.
HealthModule.forRoot({ indicators: [ClockSkewHealth] });

// 2. Through DI, for an indicator that HAS dependencies.
@Inject(HEALTH_SERVICE_TOKEN) private readonly health: IHealthService;
// …then, once your own dependencies are resolved:
this.health.registerIndicator(this.stripeHealth);
```

The `indicators` array constructs with **no arguments**
(`new IndicatorClass()`), so `StripeHealth` registered that way would
get `stripe === undefined` and fail at its first check rather than at
registration. Use it for indicators that need nothing; resolve the
others from DI and hand over the instance.

## Liveness vs readiness

The kernel separates these intentionally. Two scenarios make the
distinction important:

- **Slow startup** — your app is alive (liveness ✓) but still warming
  caches and connecting to dependencies (readiness ✗). The
  orchestrator should keep the process running but not send traffic
  yet.
- **Transient dependency failure** — your app is alive (liveness ✓)
  and could serve cached responses (readiness ✓), but Stripe is down
  (healthy ✗). The orchestrator should keep traffic flowing; the
  monitoring system should fire an alert.

If you collapse these into a single signal, the orchestrator will
restart your app (good thing → bad thing) when a dependency hiccups.

## Data origin during development

Working-tree execution and an installed immutable release provide different
origin evidence. A running process or a successful health probe does not establish
that data came from a verified artifact. Report origin separately from database,
transport and worker availability.

Make the application's readiness policy explicit. A workload that requires
verified producers should reject missing origin evidence by default. A local
development stack can explicitly allow operational readiness while retaining an
origin warning. That choice must not change persisted provenance, relabel old
records or admit unverified observations to consumers that require verified data.
Do not infer this exception merely from a project name or a missing environment
variable.

Preserve check severity through every adapter: a warning remains a warning in
process health, CLI and Console; any actual failed dependency remains a failure.
Document what an operator can use now, what evidence is missing, and what would
make the warning clear. Test both policies through the actual configuration loader
and the shared process-health adapter.

## Probe routes

When `titan-health` is loaded with its HTTP probe support, it
exposes routes that map to the three signals (typically `/healthz`,
`/readyz`, plus a detailed JSON variant). The exact path names and
status codes are configured per project — consult `titan-health`.

For Kubernetes (example):

```yaml
livenessProbe:
  httpGet: { path: /healthz, port: 3000 }
  initialDelaySeconds: 5
  periodSeconds: 10

readinessProbe:
  httpGet: { path: /readyz, port: 3000 }
  initialDelaySeconds: 0
  periodSeconds: 3
```

## Programmatic shutdown on unhealthy

```typescript
const status = await app.health();   // aggregate IHealthStatus
if (status.status === 'unhealthy' && status.modules?.['database']?.status === 'unhealthy') {
  void app.stop();   // graceful stop; supervisor restarts the process
}
```

`app.health()` returns the aggregate `IHealthStatus`
(`{ status, message?, modules?, details? }`); `app.checkHealth(name)`
returns the status for a single module. Note `HealthCheck`
(`'health:check'`) is a reserved event the kernel does not emit on its
own — drive health checks by calling `app.health()` (e.g. from a
readiness probe or an interval), not by subscribing to it. This pattern
is rare — usually you want the readiness probe to handle this
transparently — but it is available when you need it.

## Anti-patterns

- **Heavy work in indicators.** Indicators are called frequently
  (every few seconds for readiness probes). They should be cheap
  pings, not full integration tests.
- **Treating every dependency as critical.** Most dependencies
  should degrade, not fail. A cache being down does not mean you
  cannot serve requests; it means you serve them slower.
- **Ignoring liveness.** A process that has deadlocked but kept
  responding to liveness will not be restarted. Liveness should
  exercise enough of the request path to detect a stuck event
  loop.

---

---
id: "titan/application/index"
title: "Application"
description: "The Titan application kernel — what it owns, what it exposes, how to use it."
---

# Application

The `Application` object is the runtime kernel. It owns the container,
the lifecycle, the module registry, and the event bus. Every Titan
process has exactly one.

## Public surface

```typescript
import {
  Application,
  createApp,
  createAndStartApp,
  startApp,
  ApplicationToken,
  type IApplication,
  type IApplicationOptions,
} from '@omnitron-dev/titan';
```

| Symbol                 | Purpose                                                                                                                                     |
| ---------------------- | ------------------------------------------------------------------------------------------------------------------------------------------- |
| `Application`          | The orchestrator class                                                                                                                      |
| `Application.create()` | Static **async** factory; registers modules + core modules, returns a configured (not started) app                                          |
| `createApp`            | Sync `new Application(options)` — wraps the constructor only; does **not** register modules or core modules (those run lazily on `start()`) |
| `createAndStartApp()`  | Single-call bootstrap: `createApp` + `app.use(...)` per module + `app.start()`                                                              |
| `startApp`             | Sync `createApp` + `await app.start()`                                                                                                      |
| `IApplication`         | Public interface                                                                                                                            |
| `IApplicationOptions`  | Constructor options                                                                                                                         |
| `ApplicationToken`     | DI token to inject the application into a provider                                                                                          |

## The minimal app

```typescript
import { Application, Module, Injectable, Service } from '@omnitron-dev/titan';
import { Public } from '@omnitron-dev/titan/decorators';

@Injectable()
@Service({ name: 'Calculator' })
class CalculatorService {
  @Public()
  add(a: number, b: number): number {
    return a + b;
  }
}

@Module({
  providers: [CalculatorService],
})
class AppModule {}

const app = await Application.create({ modules: [AppModule] });
await app.start();
```

Three things happen, in order:

1. `Application.create` builds the container, registers `AppModule`,
   and resolves `CalculatorService` (singleton scope, default).
2. `app.start()` runs the lifecycle phases for every provider.
3. The app is **running**. Calls to `CalculatorService.add` are
   dispatched through the container.

> **Networking is not in the kernel.** Titan's core does not bind
> HTTP/WebSocket/TCP/Unix transports by itself. Netron transports are
> wired through dedicated modules (or through your own provider setup
> that constructs `Netron` and registers a transport). See
> [Netron RPC](/docs/titan/netron) for the transport-binding patterns.

## `Application.create` — two overloads

```typescript
// Form 1: module-first.
static create(module: ModuleInput, options?: IApplicationOptions): Promise<Application>;

// Form 2: options-only (matches the canonical README example).
static create(options?: CreateOptions): Promise<Application>;
```

`CreateOptions` extends `IApplicationOptions` with module wiring:

```typescript
const app = await Application.create({
  // From IApplicationOptions
  name:    'my-api',
  version: '2.3.1',
  container: undefined,
  config:    undefined,
  debug:     false,
  logging:   { level: 'info' },
  gracefulShutdownTimeout: 30_000,
  disableGracefulShutdown: false,
  disableCoreModules:      false,

  // Module wiring
  modules:    [AppModule],
  imports:    [],            // module tokens to import without registering classes
  providers:  [],            // [token, providerDef][] tuples for root-level providers

  // Auto-discovery (off by default)
  autoDiscovery: false,
  scanPaths:     ['./modules', './src/modules'],
  excludePaths:  [],
});
```

All fields are optional. The framework picks sane defaults — a
no-args call returns a usable (if empty) application.

## Core modules

When `disableCoreModules` is `false` (the default), the framework
auto-loads two modules:

- `ConfigModule` — typed configuration with file/env/object/argv
  sources. See [Configuration](/docs/titan/configuration/overview).
- `LoggerModule` — structured pino-based logging. See
  [Logging](/docs/titan/logging/overview).

Pass `disableCoreModules: true` if you provide your own config and
logger sources.

## The application as a DI participant

The application registers itself in the container under
`ApplicationToken`. Inject it where you need access to its public
API:

```typescript
import { ApplicationToken, type IApplication, Inject, Service } from '@omnitron-dev/titan';
import { Public } from '@omnitron-dev/titan/decorators';

@Service({ name: 'AdminService' })
export class AdminService {
  constructor(@Inject(ApplicationToken) private readonly app: IApplication) {}

  @Public()
  state() { return this.app.state; }

  @Public()
  uptime() { return this.app.uptime; }
}
```

Most services should not depend on `IApplication` directly — depend
on the specific service you need (`LoggerService`, `ConfigService`).
Reach for `ApplicationToken` only when you need the kernel itself
(lifecycle control, event subscription, runtime metrics).

## What the kernel exposes

Read the per-page reference for each capability:

| Capability        | Page                                           |
| ----------------- | ---------------------------------------------- |
| Bootstrap         | [Bootstrap](/docs/titan/application/bootstrap) |
| Lifecycle hooks   | [Lifecycle](/docs/titan/application/lifecycle) |
| Graceful shutdown | [Shutdown](/docs/titan/application/shutdown)   |
| Health            | [Health](/docs/titan/application/health)       |
| Events            | [Events](/docs/titan/application/events)       |

## Conventions

- **One Application per process.** A second `Application.create()`
  in the same process is supported but rarely useful — the
  container, lifecycle, and Netron registry are all process-scoped.
- **Top-level `await`.** `Application.create` and `start` are
  async. Always await them at the top level.
- **Don't catch every error.** A failure in `start()` should crash
  the process. A supervisor (Omnitron orchestrator, systemd, K8s)
  restarts it. Catching and continuing leaves a half-initialised
  process running — exactly the state Titan is designed to avoid.

→ Reference: [Omnitron App](/docs/omnitron/overview)

---

---
id: "titan/application/lifecycle"
title: "Lifecycle"
description: "OnInit, OnStart, OnStop, OnDestroy — when each fires, in what order, what to put in each."
---

# Lifecycle

Every provider in the container has the option to participate in the
application lifecycle. Four hook interfaces, fired in dependency
order, with typed contracts.

## The four hooks

| Interface   | Method        | Fires when                                                         |
| ----------- | ------------- | ------------------------------------------------------------------ |
| `OnInit`    | `onInit()`    | After the container resolves the provider; before `onStart`        |
| `OnStart`   | `onStart()`   | After all `onInit` complete; once the app enters the running state |
| `OnStop`    | `onStop()`    | When `app.stop()` runs; **reverse** dependency order               |
| `OnDestroy` | `onDestroy()` | Final cleanup phase; after `onStop`                                |

The diagram below shows the hook phases against the public
`ApplicationState` axis (`created → starting → started → stopping →
stopped`, plus `failed`). `onInit` runs within the `starting`
transition (mostly during eager init — see [Bootstrap](/docs/titan/application/bootstrap)):

```mermaid
stateDiagram-v2
  [*] --> Created: Application.create()
  Created --> Starting: app.start()
  Starting --> Started: onInit then onStart (dep order)
  Started --> Stopping: app.stop() | signal
  Stopping --> Stopped: onStop (reverse) + onDestroy
  Stopped --> [*]
  Starting --> Failed: hook throws
  Started --> Failed: uncaught
  Failed --> [*]
```

```typescript
import type { OnInit, OnStart, OnStop, OnDestroy } from '@omnitron-dev/titan/application';
```

Implement only the ones you need. A provider with no hooks is fine —
it is constructed and used; no lifecycle code runs for it.

## When to use which

- **`onInit`** — work that produces state your *constructor's
  consumers* depend on. Cache warming, schema validation, decoder
  precompilation. Runs before any service that depends on you can
  finish *its* `onInit`.
- **`onStart`** — work that opens external resources or starts
  background activity. Database connection pool, redis client,
  scheduled job registration, file watcher.
- **`onStop`** — close what `onStart` opened. Drain in-flight work.
  Cancel timers.
- **`onDestroy`** — final cleanup. Flush buffers. Write last-state
  to disk. **Do not start new work here.**

## Hook signatures

```typescript
import { Service } from '@omnitron-dev/titan';
import { Public } from '@omnitron-dev/titan/decorators';
import type { OnInit, OnStart, OnStop, OnDestroy } from '@omnitron-dev/titan/application';

@Service({ name: 'Users', version: '1.0.0' })
export class UsersService
  implements OnInit, OnStart, OnStop, OnDestroy
{
  async onInit(): Promise<void> {
    await this.cache.warm();
  }

  async onStart(): Promise<void> {
    await this.db.connect();
  }

  async onStop(): Promise<void> {
    await this.db.disconnect();
  }

  async onDestroy(): Promise<void> {
    await this.metricsBuffer.flush();
  }
}
```

All four methods may be `async`. The framework awaits each. The
signatures are `() => void | Promise<void>`.

## Decorator-marked hooks

The decorator equivalents to `OnInit` and `OnDestroy` are
`@PostConstruct` and `@PreDestroy`:

```typescript
import { Service, PostConstruct, PreDestroy } from '@omnitron-dev/titan';

@Service({ name: 'Users' })
export class UsersService {
  @PostConstruct()
  async warm() { /* runs once, after construction */ }

  @PreDestroy()
  async drain() { /* runs once, during destruction */ }
}
```

`@PostConstruct` is equivalent to `OnInit.onInit()`; `@PreDestroy`
is equivalent to `OnDestroy.onDestroy()`. For `OnStart` and `OnStop`,
implement the interface form — there are no decorator equivalents,
which is deliberate (the start/stop boundary is semantically
distinct and the interface forces the choice).

## Dependency-order guarantees

Given:

```
A   (no deps)
B   depends on A
C   depends on B
D   depends on A
```

The lifecycle order is:

| Phase       | Order                                                                                 |
| ----------- | ------------------------------------------------------------------------------------- |
| `onInit`    | `A`, then `B` and `D` in parallel, then `C`                                           |
| `onStart`   | Same as `onInit`                                                                      |
| `onStop`    | `C`, then `B` and `D` in parallel, then `A`                                           |
| `onDestroy` | Reverse-order; phased coordination — see [Shutdown](/docs/titan/application/shutdown) |

**Guarantee.** Your `onInit` will not be called until every provider
your constructor depends on has finished `onInit`. Your `onStop` will
be called before any provider your constructor depends on has its
`onStop` called.

## Failure semantics

- **Failure in `onInit` or `onStart`** — startup is aborted. Modules
  that already completed `onStart` are rolled back via their `onStop`
  in reverse order, the container is disposed, the app transitions to
  `Failed`, and `start()` rejects with the originating error (module
  failures are wrapped with the module name and the phase that failed).
- **Failure in `onStop`** — shutdown continues. The failure is
  logged with the failing provider's name; the next provider's
  `onStop` runs. A failed `onStop` should not strand resources held
  by other providers.
- **Failure in `onDestroy`** — same as `onStop`. The phased
  shutdown has a hard timeout regardless; see
  [Shutdown](/docs/titan/application/shutdown).

## Application-level lifecycle hooks

The `IApplication` interface also exposes simple registration
methods for ad-hoc hooks (not tied to a provider):

```typescript
app.onStart(async () => { /* runs during start */ });
app.onStop(async () => { /* runs during stop */ });
app.onError((err) => { /* observe lifecycle errors */ });
```

These are useful for app-level setup that does not belong to any
single module (e.g. binding a third-party SDK).

## Anti-patterns

- **Sleep loops in lifecycle hooks.** If you find yourself writing
  `while (!this.ready) await sleep(100)`, you are inverting the
  dependency. Have whatever is producing readiness be a dependency
  of this provider; the framework will sequence it for you.
- **Cross-provider state in `onInit`.** Reading another provider's
  state during `onInit` is undefined behaviour — the other provider
  may not have finished its own `onInit` yet. Use `onStart` for
  cross-provider state if you must.
- **Throwing for retryable conditions.** A thrown error aborts
  startup. If a database is briefly unavailable at boot, you want
  `onStart` to retry with backoff (use the `computeBackoff` helper
  in `utils/backoff`), not crash the process.
- **Forgetting `onStop`.** Anything you opened in `onStart` must be
  closed in `onStop`. The framework cannot release file descriptors,
  sockets, or timer handles for you.

---

---
id: "titan/application/shutdown"
title: "Graceful Shutdown"
description: "Phased shutdown with hard-exit guarantees."
---

# Graceful Shutdown

Production systems must shut down cleanly under three conditions:

1. The orchestrator sends `SIGTERM` (deploy, scale-down, restart).
2. Operator pressed Ctrl-C (`SIGINT`).
3. The process supervisor signals reload (`SIGHUP`).

Titan handles all three with a phased shutdown coordinator that
guarantees a hard exit within `gracefulShutdownTimeout`, regardless
of in-flight work.

## Hook-driven shutdown

For most providers, you do not register shutdown tasks directly —
you implement `OnStop` and/or `OnDestroy`:

```typescript
@Service({ name: 'Orders' })
export class OrdersService implements OnStop, OnDestroy {
  async onStop()    { await this.queue.drain(); }
  async onDestroy() { await this.metrics.flush(); }
}
```

The shutdown coordinator runs `onStop` in reverse dependency order,
then `onDestroy` in the same order, then issues a final hard-exit
guarantee.

## Triggering shutdown

| Source              | Effect                                         |
| ------------------- | ---------------------------------------------- |
| `await app.stop()`  | Programmatic; reason = `ShutdownReason.Manual` |
| `SIGTERM`           | Standard orchestrator stop; reason = `SIGTERM` |
| `SIGINT`            | Ctrl-C; reason = `SIGINT`                      |
| `SIGHUP`            | Reload; reason = `SIGHUP`                      |
| Uncaught exception  | reason = `UncaughtException`                   |
| Unhandled rejection | reason = `UnhandledRejection`                  |

Reasons (from `ShutdownReason` enum): `Manual`, `SIGTERM`, `SIGINT`,
`SIGHUP`, `Signal`, `UncaughtException`, `UnhandledRejection`,
`Timeout`, `Error`, `Reload`, `Upgrade`, `Maintenance`.

## `app.stop()` options

`IShutdownOptions` is `{ timeout?, force?, graceful?, signal? }`:

```typescript
await app.stop({
  timeout:  10_000,   // override gracefulShutdownTimeout for this call
  force:    false,    // true → swallow hook/task/module errors, push to Stopped
  graceful: true,     // false → a module-stop error aborts (unless force)
});
```

A direct `app.stop()` always reports `ShutdownReason.Manual` to
shutdown-task handlers and the `ShutdownComplete` event. The reason
variants (`SIGTERM`, `Maintenance`, …) come from the signal/programmatic
path that calls the internal `shutdown(reason)` entry, not from a
`reason` field on `stop()`.

## Draining Netron work

Before provider teardown, the inbound gate refuses new calls, property reads/writes and Netron tasks with `SHUTTING_DOWN`. Already admitted work stays counted until its result or error has been written to the transport. The drain has a deadline; it cannot promise completion of an operation that outlives shutdown.

If a connection disappears first, pending callers receive `TransportLostError` with the packet ID instead of waiting for the request timeout. A lost response does not reveal whether a mutation committed. Netron does not replay it automatically. Reconcile by operation ID or state, and retry only under an application-defined idempotency contract.

Stopping Netron also cancels unfinished handshakes and reconnect timers. Long-lived streams and event subscriptions belong to their original connection and must be reopened explicitly; use a durable journal or outbox when messages must survive that connection.

## Custom shutdown tasks

For tasks that don't belong to any single provider, register them
directly with the application:

```typescript
import { ShutdownPriority } from '@omnitron-dev/titan';

app.registerShutdownTask({
  id:       'audit.flush',
  name:     'Flush audit log',
  priority: ShutdownPriority.High,
  timeout:  5_000,
  handler:  async () => {
    await auditClient.flush();
  },
});

// Later, if you no longer want this task:
app.unregisterShutdownTask('audit.flush');
```

`ShutdownPriority` is a numeric enum (lower = runs first):
`First (0)`, `VeryHigh (10)`, `High (20)`, `AboveNormal (30)`,
`Normal (50)`, `BelowNormal (70)`, `Low (80)`, `VeryLow (90)`,
`Last (100)`.

## The hard-exit guarantee

The coordinator runs a watchdog timer for the whole shutdown:

```typescript
gracefulShutdownTimeout: 30_000  // default
```

If the timer fires before shutdown completes, the process exits
with a non-zero code. A misbehaving handler cannot hold the process
alive forever.

> **Why this matters.** A single `await db.disconnect()` that hangs
> because the connection is half-closed can deadlock the entire
> process. In production this means the orchestrator's restart loop
> stalls; the deploy hangs; on-call gets paged. The hard exit is
> strictly better: a forced exit is recoverable, an indefinite hang
> is not.

## Disabling auto-binding

If you embed Titan inside a larger Node.js process (test harness,
another framework, an Electron renderer), set
`disableGracefulShutdown: true`. Titan will not bind signal
handlers; you call `app.stop()` yourself.

```typescript
const app = await Application.create({
  modules: [AppModule],
  disableGracefulShutdown: true,
});
```

## Observability

The shutdown coordinator emits these events through the Application
event bus:

| Event                                   | Payload               |
| --------------------------------------- | --------------------- |
| `ApplicationEvent.ShutdownStart`        | `{ reason, details }` |
| `ApplicationEvent.ShutdownTaskComplete` | `{ task }`            |
| `ApplicationEvent.ShutdownTaskError`    | `{ task, error }`     |
| `ApplicationEvent.ShutdownComplete`     | `{ reason, success }` |
| `ApplicationEvent.ShutdownError`        | `{ reason, error }`   |

Per-phase timing flows through `ApplicationEvent.LifecyclePhaseEvent`
(a `LifecyclePhaseEvent` with `phase`, `kind`, and timing fields), which
is where duration histograms belong. Subscribe in production to record
shutdown outcomes:

```typescript
app.on(ApplicationEvent.ShutdownComplete, ({ reason, success }) => {
  metrics.counter('app.shutdown', { reason, success: String(success) }).inc();
});
```

## Anti-patterns

- **Long-running work outside `onStop`/`onDestroy`.** Anything that
  needs to clean up at shutdown should be hooked in via the lifecycle
  interfaces, not started with `setTimeout`.
- **Catching errors in shutdown tasks and continuing.** A failed
  task should fail (and let the coordinator move on or escalate),
  not silently swallow. The coordinator already isolates failures.
- **Spawning new async work in shutdown.** The framework cannot wait
  for promises you do not return. If you `setTimeout` something
  during shutdown, it will be killed on hard exit.
- **Long blocking sync code.** The coordinator can only timeout
  async work. A 30 s synchronous CPU loop will not be interrupted.

---

---
id: "titan/best-practices/error-handling"
title: "Error Handling"
description: "Throwing the right error, in the right place, for the right caller."
---

# Error Handling

Three rules and several patterns.

## Rule 1 — Throw typed errors at the boundary

Every `@Public` method should throw a `TitanError` (via the `Errors`
namespace or its subclasses) for client-visible failures. Native
`Error` becomes an `INTERNAL_ERROR` (status 500) — your client sees
an opaque server error.

```typescript
import { Errors } from '@omnitron-dev/titan/errors';

// Wrong
throw new Error('user not found');

// Right
throw Errors.notFound('user', id);
```

The framework preserves the code, status, and details across the
wire. Clients can `instanceof TitanError && e.code === ErrorCode.NOT_FOUND`
and act on the typed error.

## Rule 2 — Don't catch what you can't handle

```typescript
// Wrong — catches and silences.
try {
  return await this.upstream.fetch();
} catch (e) {
  this.logger.error({ err: e }, 'upstream failed');
  return null;
}
```

The caller now sees `null` for two different reasons (no data, or
a failure). The upstream failure becomes invisible.

Better:

```typescript
return await this.upstream.fetch();   // let the error propagate
```

Or, if you need a fallback:

```typescript
import { isOperationalError } from '@omnitron-dev/titan/utils';

try {
  return await this.upstream.fetch();
} catch (e) {
  if (isOperationalError(e)) {
    this.logger.warn({ err: e }, 'upstream unavailable, using cache');
    return await this.cache.get();
  }
  throw e;
}
```

Catch what you can handle. Re-throw what you can't.

## Rule 3 — Failure modes should be explicit

A method that can fail in different ways should signal them with
different `ErrorCode` values. Callers branch on the code, not on
parsing the message.

```typescript
import { Errors, DomainError, defineDomainCodes } from '@omnitron-dev/titan/errors';

const BillingCodes = defineDomainCodes('BILLING', {
  INSUFFICIENT_FUNDS: { status: 409, message: 'Insufficient funds' },
  ACCOUNT_FROZEN:     { status: 403, message: 'Account frozen'    },
});

@Public()
async transfer(from: string, to: string, amount: number) {
  const fromAcct = await this.repo.find(from);
  if (!fromAcct) throw Errors.notFound('account', from);

  const toAcct = await this.repo.find(to);
  if (!toAcct) throw Errors.notFound('account', to);

  if (fromAcct.balance < amount) {
    throw new DomainError({
      domainCode: BillingCodes.INSUFFICIENT_FUNDS,
      httpStatus: 409,
      message:    'Insufficient funds',
      details:    { available: fromAcct.balance, required: amount },
    });
  }

  if (fromAcct.frozen) {
    throw new DomainError({
      domainCode: BillingCodes.ACCOUNT_FROZEN,
      httpStatus: 403,
      message:    'Account frozen',
      details:    { accountId: from },
    });
  }

  return this.repo.transfer(from, to, amount);
}
```

The client checks the error code (or uses
`isDomainCode(e, BillingCodes, 'INSUFFICIENT_FUNDS')`) to
discriminate.

## Patterns

### Wrap third-party errors

External libraries throw their own error classes. Wrap them at the
boundary so the rest of your code (and your clients) sees Titan
errors:

```typescript
import { Errors } from '@omnitron-dev/titan/errors';

async findById(id: string) {
  try {
    return await this.db.users.findOne({ id });
  } catch (e) {
    if (e instanceof MongoNetworkError) {
      throw Errors.unavailable('database', String(e));
    }
    throw e;          // unknown — let the framework wrap as INTERNAL_ERROR
  }
}
```

### Use the cause field

Modern JavaScript errors support `cause`. `Errors.internal` takes the
original error as its second argument, so you can chain failures
without losing the underlying cause:

```typescript
try {
  await this.upstream.fetch();
} catch (e) {
  throw Errors.internal('upstream down', e as Error);
}
```

### Differentiate validation from business errors

```typescript
// Schema validation — input is malformed (422).
@Validate({ input: InputSchema })    // throws via the framework

// Business validation — input is well-formed but invalid in context (409).
if (input.endDate < input.startDate) {
  throw Errors.conflict('end before start', {
    startDate: input.startDate,
    endDate:   input.endDate,
  });
}
```

Both look like "user error" but mean different things.

### Centralise domain errors

Per-module error definitions reduce repetition:

```typescript
// users/errors.ts
import { Errors } from '@omnitron-dev/titan/errors';

export const usersErrors = {
  notFound:   (id: string)     => Errors.notFound('user', id),
  emailTaken: (email: string)  => Errors.alreadyExists('user', email),
  inactive:   (id: string)     => Errors.conflict('user not active', { id }),
};
```

Throw from anywhere:

```typescript
throw usersErrors.notFound(id);
```

## Logging errors

Log errors at the boundary where you decide what to do with them —
not at every level of the call stack.

```typescript
// Service — does not log; throws.
@Public()
async create(input: CreateInput) {
  if (await this.repo.findByEmail(input.email)) {
    throw Errors.alreadyExists('user', input.email);
  }
  return this.repo.create(input);
}

// Middleware / interceptor — logs the error and lets it propagate
// to the client.
class ErrorLoggingInterceptor {
  async handle(ctx, next) {
    try {
      return await next();
    } catch (e) {
      this.logger.error({
        service: ctx.service,
        method:  ctx.method,
        err:     e,
      }, 'rpc.error');
      throw e;
    }
  }
}
```

Log once per error. Multiple log lines per error muddy the
investigation.

## Anti-patterns (consolidated)

- **Generic `Error`.** No code, no status, no structure — becomes
  a 500. Always use the `Errors` namespace.
- **Catch-and-swallow.** Hides the failure and produces silent
  wrong-data bugs. Re-throw what you can't handle.
- **Catch-and-log-and-rethrow.** Doubles the log output and adds
  no value. Log at the outermost level, throw inside.
- **Polymorphic returns.** `Promise<User | Error>` makes callers
  type-check the result. Throw the error; let the framework
  signal failure.

---

---
id: "titan/best-practices/observability"
title: "Observability"
description: "Logs, metrics, traces — the three pillars and how to use each."
---

# Observability

Three pillars, three purposes:

| Pillar  | Answers                                                | Cost per event |
| ------- | ------------------------------------------------------ | -------------- |
| Logs    | "What happened?" — exact context for one event         | Highest        |
| Metrics | "How much, how often?" — aggregates over many events   | Lowest         |
| Traces  | "How did the work flow?" — relationship between events | Medium         |

Each pillar answers a question the others can't. You need all three.

## Logs

For every event you might need to investigate one day. Structured,
contextual, level-appropriate.

```typescript
this.logger.info({
  orderId:  order.id,
  userId:   order.userId,
  amount:   order.total,
  currency: order.currency,
}, 'order created');
```

Don't:

- Log in tight loops without sampling.
- Log secrets (use a `RedactionProcessor`).
- Log objects without bounded size — they end up megabytes long.

→ See [Logging](/docs/titan/logging/overview).

## Metrics

For every quantity you want to chart, alert on, or query in
aggregate.

```typescript
metrics.histogram('order.processing.ms', { tier: order.tier }, durationMs);
metrics.counter('order.created.total', { source: 'web' }, 1);
metrics.gauge('cache.size', {}, this.cache.size);
```

Three metric types:

- **Counter** — monotonically increasing. Total count of events.
- **Gauge** — point-in-time value. Current size, current connections.
- **Histogram** — distribution of values. Latencies, sizes.

Cardinality matters: every unique combination of label values
creates a separate time series. `tier=basic|pro|enterprise` is fine
(3 series); `userId=…` is not (millions of series).

→ See [titan-metrics](/docs/titan/modules/metrics).

## Traces

For finding which call to which service caused the slowness or the
failure. One trace per request, spans for sub-operations.

```typescript
import { startSpan, withTrace } from '@omnitron-dev/titan/tracing';

// startSpan(parent?) returns a child TraceContext; withTrace installs
// it for the duration of the work. The propagation layer carries the
// context — span timing/attributes/export belong to a telemetry layer
// (the relay module or an OTel sink) on top.
const span = startSpan();
return await withTrace(span, () => this.fetch());
```

Don't span every function call. Span:

- Inbound RPC calls (Netron does this automatically).
- Outbound HTTP / DB calls (your responsibility).
- Long units of work that can fail.

→ See [Tracing](/docs/titan/tracing).

## What to instrument

A canonical service exposes:

| Signal                                 | Type      | Why                     |
| -------------------------------------- | --------- | ----------------------- |
| `rpc_requests_total{method,status}`    | Counter   | Throughput, error rate  |
| `rpc_request_duration_seconds{method}` | Histogram | Latency distribution    |
| `db.query.duration_ms{table}`          | Histogram | Database tail latency   |
| `outbound.duration_ms{provider}`       | Histogram | Third-party performance |
| `cache.hit_rate{cache}`                | Gauge     | Cache effectiveness     |
| `app.active_connections{transport}`    | Gauge     | Resource utilisation    |
| `lifecycle.boot_ms`                    | Histogram | Boot SLA                |

The `rpc_requests_total` / `rpc_request_duration_seconds` pair is
auto-emitted by `titan-metrics` (note: durations are in **seconds**,
following Prometheus convention). The rest are illustrative names for
signals you wire up per service. Custom metrics are for
domain-specific quantities.

## Alerting policy

Alerts on:

- **Error rate** — `rpc_requests_total{status=error}` divided by
  `rpc_requests_total` exceeds threshold.
- **Latency** — `rpc_request_duration_seconds` p99 > SLO.
- **Resource exhaustion** — connection pool full, queue depth high.
- **Health** — readiness flipping `unhealthy`.

Don't alert on:

- Individual log lines (use logs for investigation, not alerting).
- Counter rate changes that aren't errors (info-level changes don't
  page anyone).
- Cosmetic deviations (a 5% latency increase is normal noise).

## Correlation

The three pillars correlate via shared identifiers:

- **`traceId`** — same across every log line and every span in one
  trace.
- **`service` / `method`** — labels on metrics, fields on logs.
- **`userId` / `requestId`** — fields on logs and span attributes.

In practice: a slow request in your dashboard (metric) has a
`traceId`; pull the trace (spans) to see where time was spent; pull
the logs by `traceId` to see what happened in each span.

## Anti-patterns

- **One pillar.** Logs alone: can't aggregate. Metrics alone:
  can't investigate. Traces alone: can't quantify. Use all three.
- **Logs as metrics.** Counting `grep error /var/log/app.log` is
  slow, brittle, and lossy. Emit a counter; alert on the counter.
- **Metrics as logs.** A metric with a `userId` label is a
  high-cardinality timeseries that explodes the metrics backend.
  Use logs for per-user data.
- **Tracing without sampling.** Every span ends up in the
  collector; the cost is real. Sample at scale (the relay handles
  this).

---

---
id: "titan/best-practices/performance"
title: "Performance"
description: "Where Titan apps win or lose latency and throughput."
---

# Performance

Titan's framework cost is small. Most performance problems live in
your code or your dependencies. This page covers what the framework
contributes — and what's on you.

## Where the framework spends time

Per Netron call, the framework adds:

| Step                           | Typical cost |
| ------------------------------ | ------------ |
| Transport receive + decode     | 5–15 µs      |
| Service descriptor lookup      | <1 µs        |
| Validation (typical schema)    | 1–5 µs       |
| Method dispatch                | <1 µs        |
| Result encode + transport send | 5–15 µs      |

Total framework overhead per call: roughly 15–40 µs.

For a method that does a single database query, the database is the
bottleneck — framework overhead is invisible. For a method that does
nothing (a health check), framework overhead dominates. Don't
optimise the wrong layer.

## What's on you

### Database queries

The single most common performance issue. Patterns:

- **N+1.** Loading a list and then one query per item. Use joins or
  batch loads.
- **Missing indexes.** Slow queries on large tables. Profile and
  add indexes for hot paths.
- **Over-fetching.** SELECT \* on a wide table when you need three
  columns. Project explicitly.

The framework doesn't help here. `titan-database` exposes Kysely;
write efficient queries.

### Validation overhead

Pre-compiled validators are fast (1–5 µs) but not free. For very
hot paths:

- Keep schemas small.
- Avoid expensive refinements (`z.string().refine(asyncCheck)`).
- For trusted internal calls, skip `@Validate` (TypeScript already
  enforces the type).

### Logging

Every log line costs:

- \~1 µs to format (with pino).
- \~5–10 µs to write to a transport.
- More if a processor runs.

In a tight loop, 1000 log lines / second is fine; 1 million is not.
Promote loop logging to `trace` (off in production) or sample.

### Cache

The single best win for read-heavy services. `titan-cache` with the
LRU tier:

```typescript
@Public()
@Cacheable({ keyPrefix: 'user', keyGenerator: (id) => `${id}`, ttl: 30 })
async findById(id: string) { /* … */ }
```

`@Cacheable` takes `ttl` in **seconds** (not milliseconds) and a
`keyGenerator` for the cache key. A 95% cache hit rate cuts your
database load by 20×. Worth the ceremony.

## Measuring

You can't optimise what you don't measure.

### Per-method latency

`titan-metrics` emits an `rpc_request_duration_seconds` histogram per
method by default (labelled by `method`). Look at p99 / p99.9 —
averages hide tail behaviour.

### Per-call profiling

For a specific slow method, wrap sub-operations in child spans so the
trace context propagates to each one:

```typescript
import { startSpan, withTrace } from '@omnitron-dev/titan/tracing';

@Public()
async findById(id: string) {
  const user = await withTrace(startSpan(), () => this.repo.findById(id));
  await withTrace(startSpan(), () => this.cache.set(id, user));
  return user;
}
```

`startSpan(parent?)` returns a child `TraceContext`; `withTrace`
installs it for the wrapped work. Titan's tracing layer is
propagation-only — span timing and export come from a telemetry layer
(the relay module or an OTel sink) layered on top, which is what
shows where time goes.

### Process-level profiling

Node.js has built-in CPU profiling:

```bash
node --prof src/main.js
```

Use for finding hot functions across the whole app. Combine with
`--prof-process` for human-readable output.

For production-safe profiling, use `clinic.js` or `0x`.

## Scaling vertically vs horizontally

A single Titan app can comfortably handle several thousand
calls/second on a modest box. When you need more:

- **Vertically.** Bigger machine, more workers via `titan-pm`.
- **Horizontally.** More instances behind a load balancer.

Both are fine. The framework is stateless by default
(singletons hold infrastructure, not request state); horizontal
scaling is the easy path.

## Anti-patterns

- **Premature optimisation.** Don't tune what you haven't measured.
- **Caching everything.** Adds complexity and cache-invalidation
  bugs. Cache hot paths only.
- **Threading inside Node.** Use worker threads via `titan-pm` for
  true CPU-bound work. Don't use raw `Worker` in service code.
- **One giant Singleton state.** A 1 GB cache in a singleton is
  a 1 GB heap. Use external cache (Redis tier) or bounded LRU.

## Read also

- [titan-cache](/docs/titan/modules/cache) — caching strategies.
- [titan-pm](/docs/titan/modules/pm) — worker pools for CPU work.
- [Resilience / Timeout](/docs/titan/resilience/timeout) — bounded
  external calls.

[Best practices](/docs/titan/best-practices) groups the guides on service structure, errors, observability and performance.

---

---
id: "titan/best-practices/structuring-services"
title: "Structuring Services"
description: "How to size, organise, and compose services as the codebase grows."
---

# Structuring Services

A service is the unit of behaviour. A module is the unit of
composition. Get the sizes right and the codebase scales; get them
wrong and it doesn't.

## Size a service by its bounded context

A service should map to one *bounded context* — one cohesive area of
business meaning. Not one database table, not one HTTP endpoint, not
one screen. A bounded context.

| Bad sizing                          | Better                              |
| ----------------------------------- | ----------------------------------- |
| `UsersTableService` (one table)     | `UsersService` (the user concept)   |
| `LoginEndpointService` (one route)  | `AuthService` (auth in general)     |
| `OrderListPageService` (one screen) | `OrdersService` (orders in general) |

A service that is too small drowns the codebase in coordination
boilerplate. A service that is too large grows uncomfortable to test
and refactor.

Rule of thumb: 5–15 `@Public` methods is healthy. Past 20, start
splitting.

## Calls across applications

Within one process, inject the service. Across Titan applications, expose a versioned Netron service and call its typed interface through the Netron client. Give the service method its authorization policy once. The service evaluates the same policy for browser, CLI and service callers. Grant access according to each caller’s authenticated identity.

Use a Netron transport suited to the connection. For example, a browser can use HTTP for a request and WebSocket for a subscription. Netron owns the handshake, method invocation and event framing on those connections. When integrating a provider with its own wire protocol, keep that client in the provider adapter and expose the application operation through Netron. [Netron transports](/docs/titan/netron/transports)

## Group services into modules by domain, not by layer

**Wrong** — by layer:

```
@Module({ providers: [UsersController, OrdersController, BillingController] })
class ControllersModule {}

@Module({ providers: [UsersRepository, OrdersRepository, BillingRepository] })
class RepositoriesModule {}
```

The "controllers module" has nothing in common — it bundles unrelated
features together because they share a layer name.

**Right** — by domain:

```
@Module({ providers: [UsersService, UsersRepository] })
class UsersModule {}

@Module({ providers: [OrdersService, OrdersRepository] })
class OrdersModule {}
```

Now `UsersModule` is self-contained. Refactoring it doesn't touch
unrelated code.

## Keep `@Public` methods small

A `@Public` method is a wire contract. It should:

- Take typed input (validated at the boundary).
- Delegate to private helpers for the work.
- Return a typed output.

Most logic should live in private methods or in injected helpers
(repositories, validators, calculators). The `@Public` method is a
shell.

```typescript
@Public()
@Validate({ input: CreateOrderSchema })
async create(input: CreateOrder): Promise<Order> {
  await this.validateOwnership(input.userId, input.cartId);
  const order = await this.repo.create(input);
  await this.events.emit('order.created', { orderId: order.id });
  return order;
}

private async validateOwnership(userId: string, cartId: string) { /* … */ }
```

Reading the `@Public` method gives the reader the wire contract and
the high-level flow. Helpers fill in the details.

## Avoid service-to-service calls in the same process when you can

Two services in the same process can call each other directly via
DI. This is fine — but it can mask coupling. Two patterns to prefer:

- **Shared repository.** If both `UsersService` and `OrdersService`
  need to look up users, both inject `UsersRepository`. They don't
  need `UsersService` itself.
- **Domain events.** If `OrdersService` needs to react to a user
  change, subscribe to `user.updated` rather than calling
  `UsersService` directly.

These reduce direct service-to-service dependencies and make
splitting into separate processes (later) easier.

## Use module-private providers liberally

A provider in `providers` but not `exports` is invisible to other
modules. Use this for:

- Internal repositories (`UsersRepository`).
- Validators (`UsersValidator`).
- Mappers (`UsersDtoMapper`).
- Helpers that the public API uses but no one else needs.

Smaller export surfaces mean less coupling.

## Avoid the "service sub-classes service" pattern

```typescript
class BaseService { /* shared methods */ }

@Service('users@1.0.0')
class UsersService extends BaseService { /* … */ }

@Service('orders@1.0.0')
class OrdersService extends BaseService { /* … */ }
```

Two problems:

- **Hidden coupling.** A change in `BaseService` affects every
  subclass.
- **DI confusion.** What gets injected into `BaseService`? The
  same thing or different things per subclass?

Prefer composition: extract the shared logic into a helper service
and inject it.

```typescript
@Injectable()
class CommonHelper { /* shared methods */ }

@Service('users@1.0.0')
class UsersService {
  constructor(private readonly common: CommonHelper) {}
}
```

## Anti-patterns

- **God services.** One `AppService` with 30 methods spanning every
  domain. Always splittable.
- **Anemic services.** A service that is one method delegating to
  one repository call. Inline it; the service adds no value.
- **Direct database access in `@Public` methods.** Couples wire
  contract to schema. Use a repository (even a thin one).
- **State in singleton services.** Instance state is shared across
  every call. Either move to a `Request`-scoped service or store
  in the database.

---

---
id: "titan/concepts/architecture"
title: "Architecture"
description: "Subsystems, boundaries, and what owns what at runtime."
---

# Architecture

Titan is an orchestrator over nine focused subsystems. Each subsystem
has one responsibility, one entry point, and a typed contract with the
others. Understanding the boundaries makes the framework significantly
easier to reason about.

## The nine subsystems

```mermaid
flowchart TB
  Client[Client / CLI / Console]

  subgraph App["Application Kernel"]
    direction TB
    Bootstrap["Bootstrap<br/>(create + start)"]
    Lifecycle["Lifecycle State Machine"]
    EventBus["Event Bus"]
    ModuleRegistry["Module Registry"]
    ConfigStore["Config Store"]
    HealthAggregator["Health Aggregator"]
    ShutdownCoord["Shutdown Coordinator"]
    ProcessHost["Process Host<br/>(signals, exit)"]
    ServiceExposer["Service Exposer<br/>(Netron binding)"]
  end

  subgraph DI["Nexus DI Container"]
    Providers[Providers]
    Scopes[Scopes]
    Middleware[DI Middleware]
  end

  subgraph Netron["Netron RPC"]
    Services["@Service classes"]
    Transports[Transports]
    NetMiddleware[RPC Middleware]
  end

  Client --> Transports
  Bootstrap --> ModuleRegistry --> DI
  Lifecycle --> ModuleRegistry
  ServiceExposer --> Netron
  DI --> Services
  Transports --> NetMiddleware --> Services
```

Each block in the diagram is a real type with a documented public
interface. None of them know about each other except through the
contracts declared in `src/types/`.

## Subsystem responsibilities

### Bootstrap (`Application.create`, `.start`, `.stop`)

The bootstrap layer is the single entry point. It does three things and
nothing else:

1. Builds the **Container** (see DI below).
2. Resolves and instantiates the **Module Registry** (see Modules below).
3. Hands control to the **Lifecycle** state machine.

After bootstrap completes, the `Application` object is a façade — it
holds references to the other subsystems and exposes their capabilities
through a small public surface (`use`, `resolve`, `on`, `emit`,
`config`).

→ Reference: [Application Bootstrap](/docs/titan/application/bootstrap)

### Lifecycle state machine

Drives transitions between states (`created → starting → started →
stopping → stopped`) and fires hooks in **dependency order** at each
transition. Owns timeouts, parallelism rules, and the hard-exit
guarantee.

States. The public `app.state` reports the **`ApplicationState`** enum —
six values, the coarse axis the kernel transitions through:

| `ApplicationState` | Meaning                                                  |
| ------------------ | -------------------------------------------------------- |
| `created`          | Container built, no hooks fired yet                      |
| `starting`         | Start sequence running (`onStart` hooks + module starts) |
| `started`          | Start complete; service surfaces are live                |
| `stopping`         | Stop sequence running (reverse dependency order)         |
| `stopped`          | Stop complete                                            |
| `failed`           | A phase threw; the application is in a poisoned state    |

A separate, finer-grained **`LifecycleState`** enum (`created`,
`initializing`, `initialized`, `starting`, `running`, `stopping`,
`stopped`, `shuttingDown`, `error`) is surfaced through
`getProcessMetrics().state` for diagnostics. Don't confuse the two:
`app.state` is `ApplicationState`.

→ Reference: [Lifecycle](/docs/titan/application/lifecycle), [Shutdown](/docs/titan/application/shutdown)

### Event Bus

Broadcasts framework events (`module:registered`, `config:changed`,
`lifecycle:phase`, `shutdown:complete`, etc.) to subscribers registered
via `app.on(event, handler)`. Event names come from the
`ApplicationEvent` enum; see [Application Events](/docs/titan/application/events)
for the full list and the payload each one carries.

→ Reference: [Application Events](/docs/titan/application/events)

### Module Registry

Knows about every module loaded into the application. Computes the
dependency graph between modules. Detects circular imports. Drives
ordered initialisation.

A module is a static or dynamic descriptor:

```typescript
@Module({
  imports:   [LoggerModule, ConfigModule],
  providers: [UsersService, UsersRepository],
  exports:   [UsersService],
})
export class UsersModule {}
```

→ Reference: [Modules](/docs/titan/modules-system/defining-modules), [Dynamic Modules](/docs/titan/modules-system/dynamic-modules)

### Config Store

A typed, layered, hot-reloadable configuration source. Multiple sources
are merged with deep-merge semantics; the last source wins per key.
Validators are applied per source or globally.

→ Reference: [Configuration](/docs/titan/configuration/overview)

### Health Aggregator

Aggregates health probes from registered indicators (database, redis,
disk space, custom). Surfaces a single `IHealthStatus` (`healthy |
degraded | unhealthy`) plus per-indicator detail. Used by `/healthz`,
load balancers, and the Omnitron orchestrator.

→ Reference: [Health](/docs/titan/application/health)

### Shutdown Coordinator

Phased graceful shutdown. Tasks declare a priority, a timeout, and a
critical flag; the priority is bucketed into one of three phases
(`pre-stop → stop-runtime → dispose`). The coordinator runs each phase
to completion (or timeout), then proceeds to the next. Hard exit after
the last phase.

→ Reference: [Shutdown](/docs/titan/application/shutdown)

### Process Host

Binds OS signals (`SIGTERM`, `SIGINT`, `SIGHUP`), sets up uncaught
exception / unhandled rejection handlers, and exposes runtime metrics
(`uptime`, `memoryUsage`, `cpuUsage`, `pid`). Disable with
`disableGracefulShutdown: true` if you embed Titan in a larger process.

### Service Exposer

The bridge between the DI container and Netron. When a `@Service`-marked
provider is resolved, the Service Exposer registers it with the running
Netron `LocalPeer` so its `@Public` methods become callable over the
wire.

→ Reference: [Netron Services](/docs/titan/netron/services)

## The two RPC layers

Netron has two layers of middleware that are easy to confuse:

- **DI middleware** — wraps **container resolution**. Runs at
  construction time. Used for cross-cutting object-graph concerns
  (logging instantiation, retry on construction failure, caching
  resolved instances).
- **RPC middleware** — wraps **Netron calls**. Runs per-call. Used
  for cross-cutting wire concerns (auth, rate limiting, tracing,
  serialisation tweaks).

They are independent. A typical app uses neither, one, or both
depending on what it needs.

→ Reference: [DI Middleware](/docs/titan/di/middleware), [RPC Middleware](/docs/titan/netron/middleware)

## What lives where

| Concern                            | Subpath                                        |
| ---------------------------------- | ---------------------------------------------- |
| Application + lifecycle            | `@omnitron-dev/titan/application`              |
| Lifecycle interfaces               | `@omnitron-dev/titan/lifecycle`                |
| DI container (Nexus)               | `@omnitron-dev/titan/nexus`                    |
| Decorators                         | `@omnitron-dev/titan/decorators`               |
| Validation                         | `@omnitron-dev/titan/validation`               |
| Errors                             | `@omnitron-dev/titan/errors`                   |
| Netron RPC                         | `@omnitron-dev/titan/netron`                   |
| Transports (subpaths under netron) | `…/netron/transport/{http,websocket,tcp,unix}` |
| Auth (subpath under netron)        | `…/netron/auth`                                |
| Multi-backend                      | `…/netron/multi-backend`                       |
| Config module                      | `@omnitron-dev/titan/module/config`            |
| Logger module                      | `@omnitron-dev/titan/module/logger`            |
| Tracing                            | `@omnitron-dev/titan/tracing`                  |
| Resilience helpers                 | `@omnitron-dev/titan/utils`                    |

These subpaths are stable. The root (no subpath) re-exports a
deliberately small surface — the DI decorators, `Application`,
`startApp`/`createApp`, the tokens — and **not** the rest. `@Public`,
`@Validate`, `@Timeout`, `@RateLimit` and `@Cache` are among the
eighteen decorators available only from `@omnitron-dev/titan/decorators`;
`OnStart`/`OnStop` come from `@omnitron-dev/titan/application`. Reach
for the subpath first: it is where the symbol actually lives, and it
keeps the bundle tighter.

## Boundaries between subsystems

The boundaries are not just architectural diagrams — they are concrete
rules the framework enforces:

- **The Application kernel never owns business state.** It owns
  metadata about modules, providers, and lifecycle. Business state
  lives in your services, owned by the container.
- **The container never knows about Netron.** Services that happen to
  be `@Service`-marked are also registered with Netron, but the
  container treats them as ordinary providers.
- **Netron never knows about your business logic.** It dispatches
  to methods. The methods are written in plain TypeScript; they are
  not aware they are being called over the wire.
- **Modules never directly call each other.** Modules import and
  export *providers*. Providers are what gets injected. A module
  instance is a passive descriptor, not an addressable object.

These boundaries are why Titan tests cleanly: you can replace any
subsystem in isolation. You can mock the container. You can boot
without Netron. You can bind a fake transport for end-to-end tests
without sockets.

---

---
id: "titan/concepts/design-principles"
title: "Design Principles"
description: "The seven engineering decisions that shape every API in Titan."
---

# Design Principles

Every framework has a few load-bearing decisions. Titan has seven. They
are not negotiable inside the framework — they are how the framework was
designed, why the APIs look the way they do, and what changes when you
build on top of it.

## 1. Explicitness over magic

Every dependency is declared. Every provider is registered. No
filesystem auto-discovery. No annotation scanning at startup. No
"convention over configuration" that surprises you a year later when
you grep the codebase and cannot find why a class is being instantiated.

```typescript
// Yes — explicit. The container knows about UsersService because
// AppModule said so.
@Module({ providers: [UsersService] })
export class AppModule {}

// No — implicit. Titan does not scan ./services/*.ts at boot.
```

The cost: a little more typing. The benefit: a Titan codebase is
**searchable**. Every wiring decision is a literal string in a literal
file. New team members can answer "where does this come from?" with
`grep`, not by reading the framework source.

> **Why this matters.** The most expensive bugs in mature codebases are
> ambient-state bugs — a function reads a global, the global was set
> three layers up by an init hook nobody remembers writing, the test
> doesn't trigger that hook, and the test passes. Titan's container
> makes ambient state structurally impossible: if a service needs
> something, it appears in its constructor.

## 2. Pay only for what you use

The base framework imports nothing it does not need. The 14 ecosystem
modules are independently versioned npm packages; you install the ones
you use. A Titan app with one service and no opt-in modules has no
hidden allocator, no ambient tracing, no global instrumentation.

```typescript
// A complete Titan service — no modules, no logging, no metrics.
// This works. It will not pull in pino, redis, or any other dependency.
@Service('echo@1.0.0')
export class EchoService {
  @Public()
  async ping(msg: string) { return msg; }
}

@Module({ providers: [EchoService] })
export class AppModule {}

await (await Application.create(AppModule)).start();
```

The cost: you assemble what you want. The benefit: small services stay
small. Large services do not pay the small-service tax of carrying
features they don't use.

## 3. The same primitives operate the system

There is no separate "operator API" that drifts from the runtime API.
The Omnitron CLI talks to a running Titan app over the same Netron
protocol your browser uses. The web console reads the same metrics
your services emit. The orchestrator polls the same `/healthz` your
load balancer probes.

This means three things in practice:

- **You do not learn two languages.** The wire format your frontend
  speaks is the wire format your CLI speaks.
- **You do not maintain two surfaces.** Adding an operator action is
  adding a `@Public` method on the orchestrator service.
- **Your runtime is debuggable from any client.** A Titan service can
  be inspected, called, and audited from the same code paths your
  product uses.

## 4. The contract is a TypeScript type

The wire contract between the server and any caller — browser, mobile,
service-to-service, CLI — is the **TypeScript service interface
itself**. There is no codegen step. No OpenAPI spec to keep in sync.
No protobuf compilation.

```typescript
// Server — the contract is the class.
@Service('users@1.0.0')
export class UsersService {
  @Public()
  async findById(id: string): Promise<User | null> { … }
}

// Client — the same type, imported from a shared package.
const users = await client.queryInterface<UsersService>('users@1.0.0');
const user  = await users.findById('u_42');
//    ^? User | null   — the *server's* return type
```

A signature change on the server fails the build on every caller in
the same `tsc` invocation. Drift is a compile error, not a 3 AM
production page.

## 5. Errors are values

Service methods that can fail throw typed errors (`TitanError` and
its subclasses), each carrying an HTTP-style status, a structured
`details` payload, and a stable `code`. The client receives the same
typed error with the same payload.

```typescript
// Server — typed failure.
import { Errors, ErrorCode, TitanError } from '@omnitron-dev/titan/errors';

@Public()
async findById(id: string): Promise<User> {
  const user = await this.repo.findById(id);
  if (!user) throw Errors.notFound('user', id);
  return user;
}

// Client — discriminate by code (or by class for AuthError /
// PermissionError / RateLimitError subclasses).
try {
  await users.findById('missing');
} catch (e) {
  if (e instanceof TitanError && e.code === ErrorCode.NOT_FOUND) {
    /* handle */
  }
}
```

Native `Error` instances bubbling out of your code are wrapped with
`ErrorCode.INTERNAL_ERROR` (status 500) and logged on the server
with the original stack. The client never sees an opaque 500 with no
structure.

## 6. Lifecycle is observable

Every service can declare `onInit`, `onStart`, `onStop`, `onDestroy`.
The container fires them in **dependency order** — services with more
deps initialise after their dependencies, shut down before them. A
failure in any phase aborts the corresponding transition with a typed
error. There is no "half-initialised" state where some services are
ready and some are not.

```typescript
@Service('users@1.0.0')
export class UsersService implements OnInit, OnStop {
  async onInit() { await this.cache.warm(); }
  async onStop() { await this.queue.flush(); }
}
```

Shutdown runs lifecycle hooks in reverse order with a global
`gracefulShutdownTimeout` watchdog. A hard exit guarantees no zombie
processes.

## 7. The container is the single source of truth

Every Titan-managed object is owned by the Nexus container. The
container resolves dependencies, manages scopes, applies middleware,
fires lifecycle hooks. Three corollaries:

- **No globals.** A "global service" is a singleton-scoped provider
  in the container.
- **No instance-leaking.** Tests instantiate a fresh container per
  test; nothing carries between them.
- **One place to look.** When you ask "what does this app contain?",
  the answer is "everything the container holds, in the topology
  declared by the modules."

## The shape these decisions produce

A Titan app, viewed from outside, is:

- A single `Application` object that owns everything.
- A tree of modules with declared imports / exports / providers.
- A set of services exposed over Netron.
- A set of optional ecosystem modules (config, logger, db, cache, etc.).
- Zero ambient state.

A Titan app, viewed from inside a service method, is:

- `this` is the service instance, constructor-injected with its deps.
- Method parameters are validated against the schema you declared.
- The trace context is attached to the current async scope.
- The logger reachable via `this.logger` is bound to the service name.
- Errors thrown become typed responses on the wire.

That is the whole framework. Read on for the mechanics.

---

---
id: "titan/concepts/mental-model"
title: "Mental Model"
description: "How to think about a running Titan app."
---

# Mental Model

A picture of a running Titan app, from three viewpoints:

## Viewpoint 1 — From outside the process

A Titan service, viewed from the network, is **a set of named methods
behind a versioned identifier**. The identifier is `name@semver`, the
methods are whatever is marked `@Public`, and the wire format is
Netron — choose your transport.

```
http://localhost:3000  → service "users@1.0.0":
                          findById(id: string): Promise<User|null>
                          create(input: CreateInput): Promise<User>
                          remove(id: string): Promise<void>
```

A client does not care which DI container instantiated the service,
which module declared it, or which lifecycle phase is running. It cares
about the wire contract. **The wire contract is the TypeScript
interface.**

## Viewpoint 2 — From inside a service method

Inside a method body, your service is an ordinary TypeScript class:

```typescript
@Service('orders@1.0.0')
export class OrdersService {
  constructor(
    private readonly db:    Database,    // injected
    private readonly cache: CacheService, // injected
    private readonly logger: Logger,     // injected
  ) {}

  @Public()
  async findById(@Validate(IdSchema) id: string): Promise<Order> {
    this.logger.debug('findById', { id });
    return this.cache.getOrSet(`order:${id}`, () => this.db.findOrder(id));
  }
}
```

What is *not* visible from the method body — but is real:

- The container instantiated this object exactly once (default scope:
  Singleton) and reuses it for every call.
- `this.logger` is bound to the service name, so every log line
  carries `service: 'orders'` automatically.
- The `id` parameter has been validated against `IdSchema` *before*
  the method body runs.
- A trace context is attached to the current async scope; calls from
  this method to other services propagate it.
- Whatever this method throws reaches the client **as its own class**.
  A `TitanError` arrives as a `TitanError` with its `code`, `message`
  and details; a plain `Error` arrives as an `Error`, and a `TypeError`
  as a `TypeError` — the serializer registers std errors alongside
  `TitanError`, so class identity survives the wire. There is no
  `NetronError` class and no `InternalError` wrapper.

  This matters for what you put in a message: **the message crosses
  verbatim**, so `throw new Error(\`bad password for \${user}\`)`sends
  that string to the caller. Stack traces are the part that is stripped
  by default (see [Serialization](../netron/serialization.md)); the
  message is not. Throw a`TitanError\` with a deliberate message for
  anything a client should see, and log the detail separately.

You write the method as if these were not there. The framework wires
them in.

## Viewpoint 3 — From the lifecycle's perspective

The lifecycle state machine sees the application as **a topologically
ordered set of providers**:

```
Container has these providers:
  ConfigService     (no deps)
  LoggerService     (depends on: ConfigService)
  Database          (depends on: ConfigService, LoggerService)
  CacheService      (depends on: ConfigService)
  OrdersService     (depends on: Database, CacheService, LoggerService)
  PaymentsService   (depends on: Database, OrdersService, LoggerService)

Topological order for onInit:
  ConfigService → LoggerService → Database → CacheService → OrdersService → PaymentsService

Topological order for onStop (reverse):
  PaymentsService → OrdersService → CacheService → Database → LoggerService → ConfigService
```

The framework guarantees:

- A provider's `onInit` is called only after every dependency's
  `onInit` has resolved.
- A provider's `onStop` is called before any dependency's `onStop`.
- A failure during `onInit` aborts the entire startup; partially-
  initialised providers have their `onStop` called in cleanup.
- A failure during `onStop` does not abort shutdown; the failure is
  logged, and the next provider's `onStop` runs.

This is the same guarantee modern DI frameworks (Spring, Guice, Dagger)
provide. The lifecycle state implementation is available in the
+[source repository](https://github.com/omnitron-dev/omni/blob/main/packages/titan/src/application/_internal/lifecycle-state.ts).
+That source file is not a supported package import; use the public
+[application and lifecycle APIs](/docs/reference/api-index).

## Three things to internalise

### 1. The container is the world

When you wonder "where does X come from?", the answer is "the
container resolved it from a provider declared in some module." Always.
There are no globals. There are no singletons stored on `Object` or
imported from a `state.ts` file. If you find yourself reaching for
module-level state, you are stepping outside Titan's model.

### 2. Methods are just methods

A `@Public` method is no different from a private method except that
Netron can dispatch to it. You can call `this.findById(...)` from
inside the same class; it is a normal method call. You can call
`other.findById(...)` from another service in the same process; it is
a normal method call. Only when the call crosses the wire does Netron
get involved — and Netron's involvement is invisible to the method
body.

This is why testing Titan services is easy: instantiate the class with
mock dependencies, call the methods directly. No "test client" needed
unless you are testing the wire layer itself.

### 3. The framework gets out of the way

Once an `Application` has started, the framework does almost nothing
during a request. Netron decodes a packet, looks up a service in a
map, validates parameters, calls the method, encodes the result. The
container is not consulted on every call (services are singletons by
default; their instances are already in memory). The lifecycle is not
consulted (it has finished `onStart` and is waiting for `onStop`).

You are running a TypeScript method on a Node.js process. The
framework is the surrounding scaffolding, not a per-call interpreter.

## What this means for performance

The hot path of a Titan service is:

1. Transport receives bytes.
2. msgpack deserialises into a request.
3. Service descriptor lookup (one Map lookup).
4. Parameter validation (pre-compiled Zod validator).
5. Method invocation.
6. Result serialisation.
7. Transport sends bytes.

Steps 3–6 are framework code. They are O(1) per call. The DI container
is not on the hot path. The lifecycle is not on the hot path. The
event bus is not on the hot path.

When you tune Titan for throughput, you are tuning your method body,
your validation schemas, and your transport configuration. The
framework itself is approximately invisible at runtime.

See [application bootstrap](/docs/titan/application/bootstrap) for the order in which Titan builds the container and starts transports.

---

---
id: "titan/configuration/hot-reload"
title: "Configuration Hot Reload"
description: "Change config without restarting — when, how, and the gotchas."
---

# Configuration Hot Reload

The `ConfigWatcherService` watches file sources for changes. On a
change it triggers `ConfigService.reload()`, which re-loads **all**
sources, optionally re-validates against the schema, and notifies
subscribers registered via `ConfigService.onChange()`.

The model is deliberately simple, and it pays to know its shape before
you build on it:

- Subscription is the `onChange(listener)` callback — there is **no**
  `config:changed` event on the application bus.
- The watcher uses Node's `fs.watch`; there are **no** polling or
  debounce options.
- A file change reloads and re-validates the **whole** config and
  fires **one** event — there is no per-leaf-key diffing.

The rest of this page walks through each of those in detail, with
source references into
[`packages/titan/src/modules/config/`](https://github.com/omnitron-dev/omni/tree/main/packages/titan/src/modules/config).

## When hot reload is the right tool

It saves a restart cycle. Useful for:

- **Feature flags.** Toggle a feature without redeploying.
- **Rate limit thresholds.** Tighten or loosen during an incident.
- **Log levels.** Bump to `debug` to investigate a live issue.
- **Cache TTLs.** Adjust without invalidating.

It is the **wrong** tool for:

- **Database connection strings.** A live connection pool can't be
  swapped without dropping in-flight calls.
- **Service identities.** Renaming `users@1.0.0` mid-flight breaks
  every connected client.
- **Schema changes.** The framework expects the schema to be
  stable between boot and shutdown.

For "wrong tool" cases, restart the process.

## Enabling

File watching is **opt-in**. Turn it on with `watchForChanges`:

```typescript
ConfigModule.forRoot({
  schema: AppConfigSchema,
  sources: [...],
  watchForChanges: true,
})
```

Omit it (or set it `false`) to disable watching. On `onInit`,
`ConfigService` only attaches the watcher when both `watchForChanges`
is truthy and a watcher service is present; the watcher's change
handler is what calls `reload()` (the watcher wiring in `ConfigService.initialize`, and
`ConfigService.handleConfigChange`). Only `file` sources are watched — `ConfigWatcherService`
filters the source list to `type === 'file'` and ignores the rest
(`ConfigWatcherService.watch`).

## Subscribing to changes

Register a listener with `ConfigService.onChange()`. It adds the
listener to an internal `Set` and returns an unsubscribe function that
removes it (`ConfigService.onChange`) — call it in `onDestroy` to
avoid leaks. There is **no** `config:changed` event on the application
bus; the callback is the only subscription path.

```typescript
@Service('cache@1.0.0')
class CacheService implements OnInit, OnDestroy {
  private unsubscribe?: () => void;

  constructor(private readonly config: ConfigService) {}

  async onInit() {
    this.applyTtl(this.config.get<number>('cache.ttlMs'));

    this.unsubscribe = this.config.onChange((event) => {
      // A file reload emits a single whole-config event
      // (event.path === ''); re-read the key you care about.
      this.applyTtl(this.config.get<number>('cache.ttlMs'));
    });
  }

  async onDestroy() {
    this.unsubscribe?.();
  }

  private applyTtl(ms: number) {
    this.cache.setDefaultTtl(ms);
  }
}
```

The listener receives an `IConfigChangeEvent`:

```typescript
interface IConfigChangeEvent {
  path:      string;   // dotted path; '' for a whole-config reload
  oldValue:  any;
  newValue:  any;
  source:    string;   // e.g. 'reload' or 'runtime'
  timestamp: Date;     // a Date, not epoch ms
}
```

## What gets emitted

A file change triggers a full `reload()`, which fires **one** event
with `path: ''`, `source: 'reload'`, and `oldValue` / `newValue`
holding the entire previous / new config object — it does **not**
diff and emit per leaf key (in `ConfigService.reload`). A
programmatic `config.set(path, value)` is the other emit path: it
fires a targeted event for that `path` with `source: 'runtime'`,
carrying the old and new value at that path (`ConfigService.set`).

So inside an `onChange` handler, branch on `event.source` (or on
`event.path === ''`) and re-read the specific keys you care about from
`ConfigService` rather than relying on a per-key payload from a file
reload.

## Validation on reload

`reload()` only re-validates when `validateOnStartup` is set and a
schema is present (`validateOnStartup` in `ConfigService`). In that case a reload
that fails validation is **rejected** — `reload()` restores the
previous config and throws, so the running app keeps its
last-known-good values (the validation branch of `ConfigService.reload`). The reload is
triggered from the watcher's change handler (`handleConfigChange`),
which `await`s `reload()` inside a `try/catch` and logs the failure
rather than propagating it (`ConfigService.handleConfigChange`).

This means a typo in a config file does not crash the running app.
You see the error in the logs, fix the file, and the next change
applies.

> Note: if `validateOnStartup` is **not** enabled, a reload swaps in
> the new config unconditionally. Enable `validateOnStartup` if you
> want bad reloads rejected.

## Atomicity

`reload()` re-loads and merges the *whole config*, runs validation,
then swaps the result in atomically. There is no intermediate state
where half the keys are updated and half are not. A single
whole-config `onChange` event fires after the swap.

## Watch mechanism

The watcher resolves each file source's path and opens a Node
`fs.watch` on it, de-duplicating already-watched paths
(`ConfigWatcherService.watch`). It reacts to `fs.watch` events
whose `eventType` is `'change'`, calling back into `ConfigService` to
reload. There are no polling-interval or debounce options — the change
event triggers a reload directly, so a rapid burst of writes can drive
several reloads. `unwatch()` closes every `FSWatcher` and clears the
tracked set (`ConfigWatcherService.unwatch`); `ConfigService`
calls it on both `onStop` and `onDestroy`. On filesystems where
`fs.watch` is unreliable (some NFS mounts), prefer a restart-on-change
deployment or a `remote` source instead.

## Multi-pod considerations

Hot reload changes config **only on the pod where the file was
modified**. For multi-pod deployments:

- Use a config tool that pushes to all pods (e.g. K8s ConfigMap +
  pod restart, or a remote source that polls).
- Or restart pods on config change — losing the hot-reload benefit
  but gaining cross-pod consistency.

The `remote` source type (experimental) is built for the
push-to-all-pods case; it polls a central server.

## Anti-patterns

- **Treating the `onChange` callback as transactional.** If your
  handler fails partway, you have inconsistent state. Make handlers
  idempotent.
- **Restarting a pool on every reload event.** A file reload fires a
  whole-config event even when the key you care about didn't change.
  Compare the value you read against the current one before acting.
- **Mutable singletons reading config in hot paths.** Read once at
  `onInit` and update inside `onChange`. Reading on every call pays
  the lookup cost forever.
- **Forgetting to unsubscribe.** `onChange` returns a disposer; call
  it in `onDestroy`.

→ Back to [Configuration Overview](/docs/titan/configuration/overview).

---

---
id: "titan/configuration/overview"
title: "Configuration"
description: "Layered, validated, hot-reloadable configuration for Titan apps."
---

# Configuration

`ConfigModule` is one of the two core modules auto-loaded with every
Titan app (the other is `LoggerModule`). Disable with
`disableCoreModules: true` if you need to provide your own.

This is the entry point. Detail in:

- [Sources](/docs/titan/configuration/sources) — files, env, argv, objects, remote.
- [Validation](/docs/titan/configuration/validation) — schema-checked config at boot.
- [Hot Reload](/docs/titan/configuration/hot-reload) — `ConfigWatcherService` and the `onChange` callback.

## The mental model

Configuration is **layered**: multiple sources merge into a single
typed object. Later sources override earlier ones. Optionally
validated against a Zod schema.

```typescript
import { ConfigModule } from '@omnitron-dev/titan/module/config';

@Module({
  imports: [
    ConfigModule.forRoot({
      schema: AppConfigSchema,
      sources: [
        { type: 'file', path: 'config/default.yaml' },
        { type: 'env',  prefix: 'APP_' },
      ],
      validateOnStartup: true,
      watchForChanges:   true,
    }),
  ],
})
class AppModule {}
```

## Reading config

### Through `ConfigService`

```typescript
import { Inject, Service } from '@omnitron-dev/titan';
import { ConfigService, CONFIG_SERVICE_TOKEN } from '@omnitron-dev/titan/module/config';

@Service({ name: 'Users' })
class UsersService {
  constructor(@Inject(CONFIG_SERVICE_TOKEN) private readonly config: ConfigService) {}

  @Public()
  async findById(id: string) {
    const ttl = this.config.get<number>('cache.ttlMs', 60_000);
    // …
  }
}
```

`get<T>(path, default?)` returns the typed value at the dotted path.
The default is returned if the key is missing.

### Through decorators

```typescript
import { Config, InjectConfig } from '@omnitron-dev/titan/module/config';

@Service({ name: 'Users' })
class UsersService {
  @Config('cache.ttlMs', 60_000)
  private readonly cacheTtl!: number;

  @InjectConfig()
  private readonly fullConfig!: AppConfig;
}
```

`@Config(path, default?)` injects a specific value. `@InjectConfig()`
injects the full config object.

Additional config decorators:

| Decorator                 | Effect                                                  |
| ------------------------- | ------------------------------------------------------- |
| `@ConfigSchema(schema)`   | Class-level Zod schema for this class's config subtree  |
| `@Configuration(prefix?)` | Bind a class to a config prefix (auto-populates fields) |
| `@ConfigWatch(path)`      | Method runs when the watched path changes               |
| `@ConfigDefaults({...})`  | Provide defaults at class level                         |
| `@ConfigProvider(name)`   | Mark a class as a custom config provider                |

## Why validate at boot

A typed schema for your config means:

- **Misconfiguration crashes at startup**, not at the first call
  that needs the bad value.
- **Type safety in code** — `config.get('database.url')` returns the
  schema's type, not `string | undefined`.
- **Documentation lives in the schema** — readers know what config
  the app accepts by reading the Zod schema.

## When to use what

| Approach                  | When                                               |
| ------------------------- | -------------------------------------------------- |
| `ConfigService.get(path)` | Reading multiple unrelated keys; dynamic key paths |
| `@Config(path, default?)` | A class needs one or two specific values           |
| `@Configuration(prefix)`  | A class is a *typed view* of a config subtree      |
| Compile-time constants    | Compile-time values that don't change per env      |

For values that change per environment, **always** use config —
hardcoded `localhost:5432` in code locks you to one deployment.

## Read on

- [Sources](/docs/titan/configuration/sources) — every source type and what it accepts.
- [Validation](/docs/titan/configuration/validation) — schema patterns and error
  handling.
- [Hot Reload](/docs/titan/configuration/hot-reload) — `ConfigWatcherService` and the
  `ConfigService.onChange()` callback.

---

---
id: "titan/configuration/sources"
title: "Configuration Sources"
description: "Files, env vars, argv, objects, remote — every way to get config in."
---

# Configuration Sources

A source produces a partial config object. Multiple sources merge;
the union is what your app sees.

The shapes below are the actual `ConfigSource` union from
`@omnitron-dev/titan/module/config`:

```typescript
type ConfigSource =
  | IFileConfigSource
  | IEnvironmentConfigSource
  | IArgvConfigSource
  | IObjectConfigSource
  | IRemoteConfigSource;
```

## Five built-in source types

### File — JSON / YAML / TOML / INI / .env

```typescript
{
  type:     'file',
  path:     'config/default.yaml',
  format?:  'json' | 'yaml' | 'toml' | 'ini' | 'env',  // auto-detected if omitted
  encoding?: 'utf8',
  transform?: (data) => data,
  optional?: false,
}
```

Reads a file on startup. Format is auto-detected from extension if
`format` is not specified.

> The `format` type accepts `'toml' | 'ini'`, but the current
> `ConfigLoaderService` only parses `json`, `yaml`, `env`, and
> `properties` — passing `toml`/`ini` throws at load time. Use one
> of the supported formats until loader support lands.

#### What `optional` forgives

`optional: true` means the file may be ABSENT. It does not mean the file may
be WRONG.

A missing path returns `{}` and the boot continues, which is the point of the
option — a git-ignored `config/local.yaml` that most machines do not have. But
a file that EXISTS and cannot be used — unparseable content, an unsupported
`format`, a path that cannot be read — throws, `optional` or not.

The line is drawn where the loader already draws it: the existence check. A
typo in an optional YAML file used to be skipped as readily as a missing one,
so the application booted on defaults with nothing to indicate that the file it
was told to read had been ignored. That is the failure `optional` is least able
to afford, because the operator who wrote the file believes it is in effect.

The two cases are told apart in code — the loader marks the error `malformed`
— rather than by matching the text of a message, so this does not change if the
wording does.

### Environment variables

```typescript
{
  type:      'env',
  prefix?:   'APP_',
  separator?: '__',                                   // for nested paths
  transform?: 'lowercase' | 'uppercase' | 'camelCase' | (key, value) => any,
}
```

Env vars matching `prefix` become config keys. `APP_DATABASE__URL`
becomes `database.url` when `separator: '__'`.

### Argv

```typescript
{
  type:    'argv',
  prefix?: '--app-',
}
```

Reads command-line flags.

### Inline object

```typescript
{
  type: 'object',
  data: { port: 3000, debug: true },
}
```

Note `data` (not `value`). A literal object; useful for defaults
that don't warrant a file.

### Remote HTTP

```typescript
{
  type:    'remote',
  url:     'https://config.internal/my-app',
  headers?: { Authorization: 'Bearer …' },
  timeout?: 5_000,
  retry?:   3,
}
```

Fetches config from a URL. Periodic re-fetch is implementation-
defined; check the `ConfigLoaderService` source for current
behaviour.

## Source ordering

```mermaid
flowchart LR
  S1[file: default.yaml] --> Merge[deep merge]
  S2[file: local.yaml] --> Merge
  S3[env: APP_*] --> Merge
  S4[argv: --app-*] --> Merge
  Merge --> Validate{schema?}
  Validate -- valid --> Frozen[(frozen config)]
  Validate -- invalid --> Throw[/throw TitanError BAD_REQUEST/]
```

Sources merge in the order listed (deep merge — later wins per
leaf key):

```typescript
sources: [
  { type: 'object', data: { db: { url: 'localhost', pool: { max: 10 } } } },
  { type: 'env',    prefix: 'APP_' },        // APP_DB__URL=prod-db
]
// Result: { db: { url: 'prod-db', pool: { max: 10 } } }
```

You can also assign explicit `priority` on the base `IConfigSource`
when natural order is awkward.

## Conventional source layout

A common pattern:

```typescript
ConfigModule.forRoot({
  schema: AppConfigSchema,
  sources: [
    { type: 'file', path: 'config/default.yaml' },
    { type: 'file', path: 'config/local.yaml', optional: true },
    { type: 'env',  prefix: 'APP_' },
  ],
})
```

- `default.yaml` — checked into git, sane defaults.
- `local.yaml` — git-ignored, developer overrides.
- `APP_*` env vars — production secrets, dynamic infra params.

Environment-specific files (`production.yaml`, etc.) can be loaded
by adding an extra source whose `path` is computed at startup from
your app's environment detection.

## Custom sources

Provide your own loader by extending `ConfigLoaderService` or
implementing `IConfigSource` plus a registration shim. The source
interface and loader are in
`@omnitron-dev/titan/module/config/config-loader.service.ts`.

## Anti-patterns

- **Secrets in files committed to git.** `config/default.yaml`
  should not contain production secrets. Use env vars or a secret
  manager.
- **Dynamic config paths from user input.** A config-injection
  vulnerability. Use only static paths or values from trusted
  sources.
- **Big monolithic config.** A 500-line `default.yaml` is hard to
  reason about. Split into per-domain files and load them
  individually.

---

---
id: "titan/configuration/validation"
title: "Configuration Validation"
description: "Schema-checked config — crash at boot, not at 3 AM."
---

# Configuration Validation

A typed schema for your config is the cheapest reliability win you
can take. Misconfiguration becomes a startup failure instead of a
runtime crash hours later.

## Defining a schema

```typescript
import { z } from '@omnitron-dev/titan/validation';

export const AppConfigSchema = z.object({
  port:        z.number().int().min(1).max(65535),
  environment: z.enum(['development', 'staging', 'production']),
  database: z.object({
    url:  z.string().url(),
    pool: z.object({
      min: z.number().int().nonnegative().default(2),
      max: z.number().int().positive().default(20),
    }),
  }),
  cache: z.object({
    tier:  z.enum(['memory', 'redis']).default('memory'),
    ttlMs: z.number().int().positive().default(60_000),
  }).prefault({}),
  features: z.object({
    enableBilling: z.boolean().default(false),
  }).prefault({}),
});

export type AppConfig = z.infer<typeof AppConfigSchema>;
```

> **.default({}) on a nested object drops the inner defaults**
>
> In zod 4 `.default(v)` is an **output** default: when the key is
> absent, `v` is returned as-is, without being parsed. So
> `z.object({ tier: …default('memory') }).default({})` yields `{}` — the
> inner defaults never run, and `config.cache.tier` is `undefined` while
> the schema reads as though it cannot be. TypeScript rejects it (the
> default must satisfy the output type), which is worth not casting away.
>
> `.prefault({})` is the **input** default: the value is parsed, so the
> inner defaults apply and you get `{ tier: 'memory', ttlMs: 60000 }`.
> Writing the object out in full — `.default({ tier: 'memory', ttlMs:
> 60_000 })` — works too, at the cost of stating every default twice.

Wire it in. **`validateOnStartup` is required** — it has no default, so
a schema on its own is parsed by nothing and an invalid config loads in
silence:

```typescript
ConfigModule.forRoot({
  schema: AppConfigSchema,
  validateOnStartup: true,
  sources: [...],
})
```

## What happens at boot

1. Sources load and merge into a candidate config object.
2. The candidate is parsed against the schema.
3. On success: the typed result is frozen and exposed via
   `ConfigService`.
4. On failure: `ConfigService.initialize()` throws
   `Errors.badRequest('Configuration validation failed', { errors })`,
   which surfaces as a `TitanError` with code `BAD_REQUEST`. There is
   no `ConfigValidationError` class.

Every invalid field is carried in `details.errors`, one entry per Zod
issue:

```typescript
try {
  await app.start();
} catch (err) {
  if (err instanceof TitanError && err.code === ErrorCode.BAD_REQUEST) {
    // [{ path, message, expected, received }, …]
    console.error(err.details.errors);
  }
}
```

```
[
  { path: 'port',          message: 'Expected number, received string', expected: 'number', received: 'string' },
  { path: 'database.url',  message: 'Required',                          expected: 'string', received: 'undefined' },
  { path: 'cache.tier',    message: "Invalid enum value. Expected 'memory' | 'redis', received 'redos'" }
]
```

`path` is the Zod issue path joined with `.`, so a nested field reads
`database.url`. Use this to fail fast in CI: a config that doesn't
validate locally won't validate in production either.

This runs only when both `validateOnStartup` and `schema` are set, as
above.

## Defaults

Schema-level defaults (`z.number().default(60_000)`) apply when no
source supplies the value. This is the right place for "sensible
default for development":

```typescript
cache: z.object({
  ttlMs: z.number().int().positive().default(60_000),
})
```

Don't put production defaults here — they should come from
`production.yaml` so they're explicit.

## Coercion

`@omnitron-dev/titan/validation` exposes Zod with coercion enabled
for env-var sources. A `z.number()` field can read its value from
the string `"3000"` and coerce. Same for booleans, dates.

For stricter parsing (the source must provide the right type), use
`z.number().strict()` etc.

## Per-section validation

You can validate just a slice of the config without loading the
whole schema:

```typescript
const cacheConfig = this.config.getTyped<CacheConfig>(CacheConfigSchema, 'cache');
//      ^? CacheConfig (z.infer<typeof CacheConfigSchema>)
```

`getTyped<T>(schema, path?)` parses the value at `path` (or the whole
config when `path` is omitted) and throws if it doesn't match.

Useful in dynamic modules whose options are a subtree of the global
config.

## Branded types for safety

For values that should never accidentally interconvert (e.g. URLs vs
plain strings), use Zod's `.brand`:

```typescript
const DatabaseUrl = z.string().url().brand<'DatabaseUrl'>();
type DatabaseUrl = z.infer<typeof DatabaseUrl>;

const Schema = z.object({
  database: z.object({ url: DatabaseUrl }),
});
```

`DatabaseUrl` and `string` are now distinct types; you cannot pass a
plain string where a `DatabaseUrl` is expected. Catches "I forgot to
read the right config key" bugs at the type level.

## Anti-patterns

- **`z.unknown()` everywhere.** Defeats the point. Use specific
  types; if a value can be one of several shapes, use a union.
- **Validating in code instead of the schema.** A runtime check
  in `onInit` is fine for state-derived invariants (the database
  schema matches my code), but not for config shape — that's
  what the schema is for.
- **Optional vs default for production values.** Use defaults for
  development conveniences. Production values should be required
  so a missing one fails the boot.

---

---
id: "titan/decorators/index"
title: "Decorators"
description: "Every decorator Titan ships — what it does, when to use it, what to import."
---

# Decorators

Titan's decorators come from two locations:

| Import path                      | Surface                                                                                                                                                                                                                                                                  |
| -------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `@omnitron-dev/titan`            | Core set: `Service`, `Injectable`, `Inject`, `Optional`, `Singleton`, `Transient`, `Module`, `PostConstruct`, `PreDestroy`                                                                                                                                               |
| `@omnitron-dev/titan/decorators` | Full set: all of the above plus `Public`, `Auth`, `RateLimit`, `Cache`, `Scoped`, `Request`, `Global`, `Memoize`, `Retry`, `Deprecated`, `Validate`, `Contract`, `NoValidation`, `ValidateInput`, `ValidateOutput`, `contract`, `Timeout`, `Retryable`, `Log`, `Monitor` |

For day-to-day code, import the core set from the main entry. Reach
for the `/decorators` subpath when you need the extras.

## Class-level

### `@Module(options)`

Marks a class as a module. Without it, the class is invisible to the
module registry.

```typescript
@Module({
  imports:   [LoggerModule, ConfigModule],
  providers: [UsersService, UsersRepository],
  exports:   [UsersService],
})
export class UsersModule {}
```

→ [Defining Modules](/docs/titan/modules-system/defining-modules)

### `@Injectable(options?)`

Marks a class as DI-resolvable. The container can construct it; its
constructor parameters are resolved as dependencies.

```typescript
@Injectable()
export class UsersRepository {
  constructor(private readonly db: Database) {}
}
```

`@Service` implies `@Injectable()`. Use `@Injectable()` for
non-service helpers (repositories, mappers, internal utilities).

→ [Providers](/docs/titan/di/providers)

### `@Service(options?)`

Marks a class as a Netron RPC service.

```typescript
// String form — name + optional version
@Service('users@1.0.0')
class UsersService {}

// Object form
@Service({ name: 'users', version: '1.0.0' })
class UsersService {}

// Just a name (version becomes empty / unversioned)
@Service('users')
class UsersService {}

// Defaults — derived from class name
@Service()
class UsersService {}
```

The qualified identifier is `name@version`. The name must match
`/^[a-zA-Z0-9]+(\.[a-zA-Z0-9]+)*$/` — Latin letters, numbers, and
dots only (dots for namespacing). The version must be valid semver
if provided.

`@Service` implies `@Injectable()`.

→ [Netron Services](/docs/titan/netron/services)

### Scope shorthands

```typescript
import { Singleton, Transient, Scoped, Request } from '@omnitron-dev/titan/decorators';

@Singleton()
class CacheService {}

@Transient()
class RequestBuilder {}

@Request()
class RequestContext {}
```

`Singleton` and `Transient` are also re-exported from the main entry;
`Scoped` and `Request` live in `/decorators`.

→ [Scopes](/docs/titan/di/scopes)

### `@Global()`

Marks a module as global — its exports are visible to every module
in the application without explicit imports.

```typescript
@Global()
@Module({
  providers: [LoggerService],
  exports:   [LoggerService],
})
class LoggerModule {}
```

Use sparingly. Reserved for framework infrastructure.

## Method-level — RPC

### `@Public(options?)`

Exposes a method (or property) over Netron. The options object can
carry `auth`, `rateLimit`, `cache`, `audit`, `prefetch`, `transports`,
`readonly` configuration. Methods without `@Public` are visible to
the process but not RPC-callable.

```typescript
@Service('users@1.0.0')
class UsersService {
  // Plain exposure.
  @Public()
  async findById(id: string) { /* … */ }

  // Inline configuration of everything @Public knows about.
  @Public({
    auth:      { roles: ['admin'] },
    rateLimit: { limit: 100, window: 60_000 },
    cache:     { ttl: 30_000 },
    transports: ['ws', 'tcp'],
  })
  async heavyMethod(query: string) { /* … */ }

  private internalHelper() { /* not exposed */ }
}
```

`@Public` is **opt-in**. The default is internal — methods that
look fine inside the process should not silently become RPC-callable
without an explicit decision.

A member's `transports` list narrows the service's own list; it cannot widen
it. Omitted means unrestricted and `[]` denies all remote transports. The same
rule covers methods and properties, HTTP and persistent transports. See
[transport boundaries](/docs/titan/netron/transports#configuring-transports).
`@Public({ transports: [...] })` and a separate `@Auth(...)` compose in either
decorator order; transport options never erase the authorization requirements.

### `@Auth(config)`

Configures authentication and authorisation for a method. Use
alongside `@Public()` for cleaner separation of concerns. The shape
of `AuthConfig`:

```typescript
@Public()
@Auth({
  roles:       ['admin'],          // ANY role grants access
  permissions: ['users:read'],     // ALL permissions required
  scopes:      ['user.read'],      // ALL OAuth2 scopes required
  policies:    ['policy.name'],    // or { all: [...] } / { any: [...] } / expression
  allowAnonymous: false,
  inherit:        true,
  override:       false,
})
async findById(id: string) { /* … */ }
```

The policy framework (see `BuiltInPolicies` in
`@omnitron-dev/titan/netron/auth`) provides reusable
`requireRole`, `requireAnyRole`, `requireAllRoles`,
`requirePermission`, and similar policies.

→ [Netron Authentication](/docs/titan/netron/authentication)

### `@RateLimit(config)`

Configures rate limiting. The basic shape:

```typescript
@Public()
@RateLimit({ limit: 100, window: 60_000 })   // 100 requests per minute
async searchDocuments(query: string) { /* … */ }
```

Advanced (tier-based) shape:

```typescript
@Public()
@RateLimit({
  defaultTier: { name: 'free',    limit: 10,  burst: 20  },
  tiers:       { premium:        { limit: 100, burst: 150 } },
  window:      60_000,
})
async searchDocuments(query: string) { /* … */ }
```

The actual `RateLimitConfig` lives in
`@omnitron-dev/titan/netron/auth/rate-limiter.ts`.

This decorator RECORDS the limit: it writes metadata that Netron's
authorization layer reads. A second decorator of the same name, exported from
`@omnitron-dev/titan-ratelimit`, WRAPS the method and enforces the limit
itself. They are told apart by the import path and by their options — `window`
here, `windowMs` there — so a stack that mixes them compiles and does something
other than what it reads like.

→ [titan-ratelimit](/docs/titan/modules/ratelimit)

### `@Cache(config)`

Caches the method result. The shape:

```typescript
@Public()
@Cache({
  ttl:          30_000,                          // ms
  keyGenerator: (args) => `user:${args[0]}`,     // optional; default uses args hash
  invalidateOn: ['user:updated'],                // events that bust this cache
  maxSize:      1_000,
})
async findById(id: string) { /* … */ }
```

→ [titan-cache](/docs/titan/modules/cache)

### `@Validate(schema, options?)` and `@Contract(contract)`

Validate input (and optionally output) against Zod schemas.

```typescript
import { z } from '@omnitron-dev/titan/validation';
import { Validate, ValidateInput, Contract } from '@omnitron-dev/titan/decorators';

@Public()
@Validate({ input: CreateUserSchema })
async create(input: z.infer<typeof CreateUserSchema>) {
  // input is parsed and trusted here.
}

// Input-only shorthand — equivalent to @Validate({ input: schema })
@Public()
@ValidateInput(CreateUserSchema)
async createAlt(input: z.infer<typeof CreateUserSchema>) { /* … */ }

@Public()
@Contract(FindByIdContract)
async findById(input: { id: string }) { /* input + output validated */ }
```

`@Validate` is a **method** decorator. It takes a `MethodContract`
options object (`{ input?, output?, errors?, stream?, options? }`)
of Zod schemas — not a bare schema, and not a parameter decorator.

Other validation decorators: `ValidateInput`, `ValidateOutput`,
`NoValidation`, `Contracts` (class-level), `contract` (builder).

→ [Validation](/docs/titan/validation/overview)

## Method-level — utilities

### `@Memoize()`

Per-instance memoisation of method results, keyed by `JSON.stringify`
of arguments. In-memory only; lives as long as the instance.

```typescript
@Memoize()
private computeHash(input: string) { /* … */ }
```

### `@Retry({ attempts?, delay? })`

Retries the method on failure with a **fixed** delay between
attempts. `attempts` defaults to 3, `delay` to 1000 ms.

```typescript
@Retry({ attempts: 3, delay: 200 })
async fetchUpstream() { /* … */ }
```

For exponential backoff and classifier-driven decisions, use the
`computeBackoff` helper in `@omnitron-dev/titan/utils` with your own
retry loop. See [Resilience / Retry](/docs/titan/resilience/retry).

### `@Deprecated({ message?, version? })`

Marks the method as deprecated. Metadata only; the framework does
not log warnings.

```typescript
@Deprecated({ message: 'Use findById instead', version: '2.0.0' })
async getUser(id: string) { /* … */ }
```

### `@Timeout({ ms })`, `@Retryable(...)`, `@Log({...})`, `@Monitor({...})`

Additional utility interceptors in
`@omnitron-dev/titan/decorators/utility.ts`. Less commonly used; see
the source for current signatures.

## Parameter-level

### `@Inject(token)`

Specifies the DI token for this parameter. Required for symbol
tokens and multi-tokens; optional for class tokens (auto-detected
from constructor metadata).

```typescript
constructor(
  @Inject(LOGGER_TOKEN) private readonly logger: ILogger,
  private readonly db: Database,                            // class token; @Inject not needed
) {}
```

### `@Optional()`

Makes the dependency optional. If no provider is registered, the
parameter receives `undefined`.

```typescript
constructor(
  @Optional() @Inject(METRICS_TOKEN) private readonly metrics?: IMetrics,
) {}
```

### `@InjectAll(token)`, `@InjectMany(token)`

Inject all providers registered against a multi-token.

### `@Value(path, default?)`, `@InjectEnv(key, default?)`, `@InjectConfig(path)`

Inject from configuration sources. `@Value` resolves through a
config tree; `@InjectEnv` reads `process.env`; `@InjectConfig` reads
through `ConfigService` (see [Configuration](/docs/titan/configuration/overview)).

### `@Lazy(tokenFactory)`

Inject a token resolved lazily on first use. Used to break circular
dependencies.

→ [Circular Dependencies](/docs/titan/di/circular-dependencies)

## Lifecycle decorators

### `@PostConstruct()`, `@PreDestroy()`

Mark methods that run during the `onInit` / `onDestroy` lifecycle
phases.

```typescript
@PostConstruct()
async warm() { /* … */ }

@PreDestroy()
async drain() { /* … */ }
```

Equivalent to implementing `OnInit` / `OnDestroy` interfaces. See
[Lifecycle Decorators](/docs/titan/decorators/lifecycle).

## Decorator order

When stacking decorators, **outermost-first** in declaration order;
execution at call time is **inside-out**:

```typescript
@Public()                                              //  ← outer
@Auth({ roles: ['admin'] })                            //
@RateLimit({ limit: 10, window: 60_000 })              //
@Validate({ input: InputSchema })                      //  ← inner, runs first
async dangerousOp(input: Input) { /* … */ }
```

→ See [Method Traits](/docs/titan/decorators/method-traits) for stacking guidance.

## Anti-patterns

- **Decorating private methods with `@Public`.** Private methods
  are not meant to be called from outside; making them RPC-callable
  is a security risk.
- **Inline configuration vs decorators.** `@Public({ auth: …,
  rateLimit: …, cache: … })` is equivalent to stacking `@Public()
  @Auth(…) @RateLimit(…) @Cache(…)`. Pick one style per project
  and stick to it.
- **Using `@Memoize` for cross-instance caching.** Memoisation is
  per-instance. For shared caching across calls and clients, use
  `@Cache`.
- **Decorating constructors.** Decorators on constructors do
  nothing in Titan — the framework reads class-level decorators,
  not constructor-level. Move the metadata to the class.

---

---
id: "titan/decorators/lifecycle"
title: "Lifecycle Decorators"
description: "PostConstruct, PreDestroy — and when to prefer interfaces."
---

# Lifecycle Decorators

Lifecycle work can be declared two ways: implementing the
`OnInit` / `OnStart` / `OnStop` / `OnDestroy` interfaces, or
decorating methods with `@PostConstruct` / `@PreDestroy`.

The framework treats them identically. Choose by intent.

## `@PostConstruct()`

Runs after the container constructs the instance, in the dependency-
ordered `onInit` phase. Equivalent to implementing
`OnInit.onInit()`.

```typescript
import { Service, PostConstruct } from '@omnitron-dev/titan';

@Service('users@1.0.0')
class UsersService {
  @PostConstruct()
  async warm() {
    await this.cache.warm();
  }
}
```

Multiple `@PostConstruct` methods on one class run in declaration
order.

## `@PreDestroy()`

Runs during shutdown, in the `onDestroy` phase. Equivalent to
implementing `OnDestroy.onDestroy()`.

```typescript
import { Service, PreDestroy } from '@omnitron-dev/titan';

@Service('users@1.0.0')
class UsersService {
  @PreDestroy()
  async drain() {
    await this.queue.drain();
  }
}
```

Multiple `@PreDestroy` methods on one class run in declaration order.
(It is the disposal order *across instances/modules* that is reversed —
dependents are torn down before their dependencies — not the multiple
`@PreDestroy` methods within a single class.)

## When to prefer the interface form

The interface form (`implements OnInit`) is recommended for:

- **Single-purpose lifecycle work** — one method per phase, with a
  signature the type system enforces.
- **Public API** — the class declares lifecycle participation in
  its type signature, making it explicit to readers.
- **Tooling support** — IDEs and the framework can detect the
  interface and offer better autocomplete / quick fixes.

```typescript
import { Service, type OnInit, type OnDestroy } from '@omnitron-dev/titan';

@Service('users@1.0.0')
class UsersService implements OnInit, OnDestroy {
  async onInit()    { await this.cache.warm(); }
  async onDestroy() { await this.queue.drain(); }
}
```

## When to prefer decorators

The decorator form is useful when:

- **Multiple lifecycle methods of the same kind** — you want two
  things to happen at `onInit`, separated for readability:

  ```typescript
  @PostConstruct()
  async warmCache() { await this.cache.warm(); }

  @PostConstruct()
  async loadDictionary() { await this.dict.load(); }
  ```

- **Mixin patterns** — a base class wants lifecycle behaviour
  without forcing subclasses to implement an interface.

## Mapping to lifecycle phases

| Decorator        | Equivalent interface    | Phase       |
| ---------------- | ----------------------- | ----------- |
| `@PostConstruct` | `OnInit.onInit()`       | `onInit`    |
| —                | `OnStart.onStart()`     | `onStart`   |
| —                | `OnStop.onStop()`       | `onStop`    |
| `@PreDestroy`    | `OnDestroy.onDestroy()` | `onDestroy` |

There is no decorator equivalent for `onStart` / `onStop` because
the distinction between `onInit`/`onStart` and `onStop`/`onDestroy`
is semantically important — the framework wants you to choose by
phase, and the interface forces the choice.

For `onStart` / `onStop` work, implement the interface.

## Anti-patterns

- **Mixing decorators and interfaces on the same class.** For the
  init phase the two are *not* additive: if a class has any
  `@PostConstruct` method, the container runs those and **skips**
  `onInit()`. Don't rely on both firing — pick one style per class.
- **Lifecycle decorators on private methods.** They work — but the
  intent is unclear. Lifecycle work is part of the class's public
  contract, even if the method is private.
- **Lifecycle work in regular methods.** Calling `await
  this.warmCache()` from a constructor or from a deferred
  `setImmediate` bypasses the framework's ordering guarantees. Use
  the proper hook.

---

---
id: "titan/decorators/method-traits"
title: "Method Traits"
description: "Composing decorators on a single method — order, semantics, common stacks."
---

# Method Traits

A "method trait" is a cross-cutting concern applied via decorator.
Most methods have one trait or none; some methods stack several.

## Two ways to express the same thing

`@Public(options)` accepts the same configuration that `@Auth`,
`@RateLimit`, `@Cache`, and `@Audit` set individually. The two styles
are equivalent.

### Inline (single decorator):

```typescript
@Public({
  auth:      { roles: ['admin'] },
  rateLimit: { limit: 10, window: 60_000 },
  cache:     { ttl: 30_000 },
})
async dangerousOp(input: Input) { /* … */ }
```

### Composed (stacked decorators):

```typescript
@Public()
@Auth({ roles: ['admin'] })
@RateLimit({ limit: 10, window: 60_000 })
@Cache({ ttl: 30_000 })
async dangerousOp(input: Input) { /* … */ }
```

Pick a style per project. Inline is terser; composed reads top-down
and groups easily.

## The order rule (when composing)

Decorators evaluate **outside-in** at declaration; they execute
**inside-out** at call time.

```typescript
@Public()                                              //  ← outer, processes last
@Auth({ roles: ['admin'] })                            //
@RateLimit({ limit: 10, window: 60_000 })              //
@Validate({ input: InputSchema })                      //  ← inner, processes first
async dangerousOp(input: Input) { /* … */ }
```

`@Validate` runs first, then `@RateLimit`, then `@Auth`, then `@Public`
finalises the dispatch. The runtime order is critical for stacks like
"validate before rate-limit" so malformed requests don't consume rate
budget.

## Common stacks

### Public read endpoint

```typescript
@Public({
  cache:     { ttl: 30_000 },
  rateLimit: { limit: 100, window: 60_000 },
})
@Validate({ input: IdSchema })
async findById(id: string) { /* … */ }
```

- Cache outermost — cache hits skip everything else.
- Rate-limit before validation.

### Authenticated write endpoint

```typescript
@Public({
  auth:      { scopes: ['orders:write'] },
  rateLimit: { limit: 10, window: 60_000 },
})
@Validate({ input: CreateOrderSchema })
async create(input: z.infer<typeof CreateOrderSchema>) { /* … */ }
```

- Rate limit applies to *all* callers (authenticated or not).
- Auth enforces a scope.

### Admin-only endpoint

```typescript
@Public({ auth: { roles: ['admin'] } })
@Validate({ input: InputSchema })
async resetDatabase(input: Input) { /* … */ }
```

No rate limit — admin operations are deliberate and infrequent.
No cache — admin operations should always run.

### Internal method (not RPC)

```typescript
// No @Public — not exposed.
@Retry({ attempts: 3, delay: 100 })
async fetchFromUpstream(id: string) { /* … */ }
```

Internal methods use `@Retry`, `@Memoize`, `@Timeout` (from
`/decorators/utility.ts`), etc. — but not `@Public`, `@Auth`, or
`@RateLimit`, which only make sense at the network edge.

## Combining lifecycle decorators

`@PostConstruct` and `@PreDestroy` mark methods for lifecycle phases.
They do **not** stack with method traits — the marked method is
executed during the lifecycle, not as a wrapper for a regular call.

```typescript
// Wrong — @Memoize on a lifecycle hook.
@PostConstruct()
@Memoize()
async warm() { /* … */ }
```

If you need memoisation in lifecycle work, extract the memoised
helper:

```typescript
@PostConstruct()
async warm() {
  this.dictionary = await this.loadDictionary();
}

@Memoize()
private async loadDictionary() { /* … */ }
```

## Custom traits via `createMethodInterceptor`

If you need a trait the built-ins don't cover, write your own:

```typescript
import { createMethodInterceptor } from '@omnitron-dev/titan/decorators';

const AuditLog = createMethodInterceptor<{ resource: string }>(
  'AuditLog',
  async (originalMethod, args, context) => {
    const result = await originalMethod(...args);
    auditLog.append({
      resource: context.options?.resource,
      method:   context.propertyKey,
      args,
      at: Date.now(),
    });
    return result;
  },
);
```

Use:

```typescript
@Public()
@AuditLog({ resource: 'orders' })
async create(input: Input) { /* … */ }
```

Custom interceptors stack inside-out with the built-ins.

## Class-level vs method-level

Many traits also support class-level application — they apply to
every `@Public` method on the class:

```typescript
@Service('orders@1.0.0')
@Auth({ scopes: ['orders:*'] })                          // class-level
@RateLimit({ limit: 100, window: 60_000 })               // class-level
class OrdersService {
  @Public() async list() { /* inherits class-level traits */ }

  @Public()
  @Auth({ scopes: ['orders:write'] })                    // overrides class-level
  async create(input: CreateInput) { /* … */ }
}
```

Method-level decorators **override** class-level ones for that
method.

## Anti-patterns

- **Cache outside an auth-gated method.** Cache hits skip the auth
  check — anyone who can warm the cache can read its contents. Put
  `@Auth` outside `@Cache`, or cache per-user with a `keyGenerator`
  that includes the user identity.
- **Rate limit inside auth.** Unauthenticated abusers hit your auth
  backend at full rate. Rate-limit *before* auth (or apply
  rate-limit class-level so it covers all methods).
- **Validate after rate limit on cheap endpoints.** Malformed
  requests should be rejected before consuming rate budget —
  `@Validate` belongs inside (executed first).
- **Stacking similar decorators.** `@Cache` and `@Memoize` together
  cache the same call twice. Pick one — `@Cache` for shared,
  `@Memoize` for per-instance.

→ Back to [Decorators](/docs/titan/decorators).

---

---
id: "titan/di/circular-dependencies"
title: "Circular Dependencies"
description: "Detect, diagnose, and break cycles in the DI graph."
---

# Circular Dependencies

A circular dependency is a cycle in the constructor graph: A's
constructor needs B, B's constructor needs A. The container cannot
construct either without the other being already constructed. Nexus
detects this and throws `CircularDependencyError`.

```typescript
import { CircularDependencyError, forwardRef } from '@omnitron-dev/titan/nexus';
```

## What you see

```
CircularDependencyError: Circular dependency detected: UsersService
  The dependency chain forms a cycle:
  UsersService -> SessionService -> AuthService -> UsersService
```

The message (built by Nexus's actionable-error formatter) names every
node in the cycle, in order, and lists fixes — extract a shared
service, use lazy injection, switch to property injection, or
`forwardRef()`.

## Why cycles are bad

Beyond "the container cannot do its job":

- **Construction order is undefined.** If both A and B could be
  constructed in some order, that order leaks into runtime
  semantics. Tests pass under one order, fail under another.
- **Refactoring is fragile.** A method moves from A to B and the
  cycle suddenly closes. The build breaks and nobody knows why.
- **Architectural smell.** Two modules that mutually depend
  usually have a shared concept they both need to know about.
  Extracting it is almost always the right move.

## Three ways to break a cycle

### 1. Extract the shared concept

Most cycles hide a third concept that should be its own module.

```typescript
// Before — cycle:
//   UsersService → AuthService
//   AuthService  → UsersService

// After — extract:
class UsersDirectory { /* read-only user lookup */ }

class UsersService { constructor(private dir: UsersDirectory, private auth: AuthService) {} }
class AuthService  { constructor(private dir: UsersDirectory) {} }
// Now AuthService no longer depends on UsersService.
```

This is the cleanest fix. The third concept is usually a
"directory", "registry", "context", or "core" of the domain.

### 2. Invert the dependency

If A and B genuinely need each other at runtime, restructure so the
constructor dependency goes one way:

```typescript
// Before:
class A { constructor(private b: B) {} foo() { this.b.bar(); } }
class B { constructor(private a: A) {} bar() { this.a.foo(); } }

// After: B publishes a callback, A injects nothing from B.
interface BarHandler { bar(): void; }

class A {
  private barHandler?: BarHandler;
  registerBarHandler(h: BarHandler) { this.barHandler = h; }
  foo() { this.barHandler?.bar(); }
}

class B {
  constructor(private a: A) { a.registerBarHandler(this); }
  bar() { /* … */ }
}
```

### 3. Lazy resolution with `forwardRef`

`forwardRef` defers reference resolution until first use, breaking
the construction-time cycle:

```typescript
import { forwardRef } from '@omnitron-dev/titan/nexus';
import { Inject } from '@omnitron-dev/titan';

class UsersService {
  constructor(
    @Inject(forwardRef(() => AuthService)) private auth: AuthService,
  ) {}
}
```

The container resolves `AuthService` when the field is first
*accessed*, not when `UsersService` is constructed. Use sparingly
— it hides the wiring; future readers will struggle to trace the
graph.

Lazy tokens (`createLazyToken`) serve the same purpose with a
different API.

## Cycles between modules vs cycles between providers

| Error                      | Where                         | Detection                    |
| -------------------------- | ----------------------------- | ---------------------------- |
| Module import cycle        | `@Module({ imports: [...] })` | Module discovery (boot time) |
| Provider constructor cycle | Constructor signatures        | Container graph analysis     |

Both throw at boot. Both name the cycle. Both have the same three
fixes.

## Avoiding cycles in design

### Layer your modules

Sort modules into layers: core (no domain deps), domain (depends
on core), feature (depends on domain). A cycle implies a layer
violation.

| Layer   | Example modules                                  | Dependency direction  |
| ------- | ------------------------------------------------ | --------------------- |
| Feature | `BillingModule`, `UsersModule`, `OrdersModule`   | Domain and core       |
| Domain  | `AuthModule`                                     | Core                  |
| Core    | `LoggerModule`, `ConfigModule`, `DatabaseModule` | Shared infrastructure |

Lower layers can be imported by higher layers, never the reverse.

### Domain types in their own module

If both `UsersModule` and `OrdersModule` need a `User` type, put
the type in a `UsersTypesModule` (or a shared `domain/users.types`
file that exports types but no providers). Both modules import the
types without depending on the implementation.

## Diagnosing a cycle in a large graph

When the cycle path is long, use [DevTools](/docs/titan/di/devtools) to
render the full container graph and visually locate the cycle.

---

---
id: "titan/di/contextual-injection"
title: "Contextual Injection"
description: "Resolve different providers based on runtime context — current user, environment, feature flags, tenant."
---

# Contextual Injection

Contextual injection lets a single token resolve to **different
providers** depending on runtime context — the current user, the
active tenant, the environment, or a feature flag. Same dependency,
different implementation, chosen per request or per scope.

The whole subsystem lives in
[`packages/titan/src/nexus/context.ts`](https://github.com/omnitron-dev/omni/blob/main/packages/titan/src/nexus/context.ts).
It has four moving parts:

- **`ContextProvider`** — a key/value bag of context values
  (`get`/`set`/`has`/…), with parent-chained inheritance.
- **`ContextManager`** — owns the active `ContextProvider`
  (isolated per async flow via `AsyncLocalStorage`) and the registry
  of resolution strategies.
- **`ResolutionStrategy`** — `{ name, applies(token, ctx), select(providers, ctx) }`;
  given a candidate array of providers and a `ResolutionContext`, it
  picks one.
- **`ContextAwareProvider`** — `{ provide(ctx), canProvide?(ctx) }`;
  a provider whose value is computed from the resolution context.

```typescript
import {
  ContextKeys,
  InjectContext,
  TenantStrategy,
  ContextManager,
  createContextKey,
  getContextManager,
  RoleBasedStrategy,
  EnvironmentStrategy,
  FeatureFlagStrategy,
  DefaultContextProvider,
  createContextAwareProvider,
  type ContextKey,
  type ContextProvider,
  type ResolutionStrategy,
  type ContextAwareProvider,
} from '@omnitron-dev/titan/nexus';
```

## When you need it

Three scenarios that all reduce to "this dependency depends on who
is asking":

1. **Multi-tenancy.** Tenant A uses a different storage backend
   than tenant B.
2. **Feature flags.** New users get the new payment processor;
   old users get the legacy one.
3. **Per-role behaviour.** Admins see audit-aware queries; regular
   users see plain ones.

Without contextual injection, you would write
`if (tenant === 'X') { useA() } else { useB() }` in every consumer.
Contextual injection moves the decision into the container.

## How resolution threads context

```mermaid
flowchart TD
  Req[Incoming request]
  Set["Middleware: ctx.set(KEY, value)<br/>on the current ContextProvider"]
  Run["runWithContext(ctx, handler)<br/>(AsyncLocalStorage frame)"]
  RC["createResolutionContext(container, scope)<br/>→ folds context values into metadata"]
  Sel["selectProvider(token, providers, rc)<br/>→ runs strategies in order"]
  Resolved[Chosen provider]
  Req --> Set --> Run --> RC --> Sel --> Resolved
```

Two facts about this flow are worth pinning down up front:

1. **Context values live on a `ContextProvider`, not on the
   `ContextManager`.** `ContextManager.getCurrentContext()` returns the
   active `ContextProvider`, and `get`/`set`/`has`/`delete`/`clear`/
   `keys`/`toObject`/`createChild` are all methods on *that* object
   (the `ContextProvider` interface in `nexus/context.ts`).
2. **Strategy selection is an explicit call.** A strategy never runs
   "automatically" on a bare `container.resolve(token)`. The code that
   has several candidate providers for a token calls
   `ContextManager.selectProvider(token, providers, resolutionContext)`,
   which walks the registered strategies and returns one provider
   (`ContextManager.selectProvider`). The `ResolutionContext.metadata` it reads is
   built by `createResolutionContext`, which flattens the current
   `ContextProvider` into a plain object via `toObject()`
   (`createResolutionContext`).

The four building blocks:

- **`ContextProvider`** — the per-flow key/value bag.
- **`ContextKey<T>`** — typed key for one piece of context.
- **`ResolutionStrategy`** — maps `(candidate providers, context) →
  one provider`.
- **`ContextAwareProvider`** — a provider whose value is computed from
  the resolution context.

## Built-in strategies

| Strategy              | What it reads                |
| --------------------- | ---------------------------- |
| `TenantStrategy`      | Tenant id from context       |
| `RoleBasedStrategy`   | Current user's role          |
| `EnvironmentStrategy` | NODE_ENV / config-driven env |
| `FeatureFlagStrategy` | Flag query                   |

These compose with the `ContextManager` to drive resolution.

## Defining a context key

```typescript
import { createContextKey } from '@omnitron-dev/titan/nexus';

const TENANT_ID = createContextKey<string>('tenant.id');
const USER_ROLE = createContextKey<string>('user.role');
```

`ContextKeys` is a registry of pre-defined keys the framework uses
(see source for the canonical set).

## Setting context per request

The `ContextManager` owns the strategy registry and the active
context. The key/value accessors —
`get`/`set`/`has`/`delete`/`clear`/`keys`/`toObject`/`createChild` —
live on the **`ContextProvider`** returned by `getCurrentContext()`,
so request middleware reaches through the manager to set values:

```typescript
import { ContextManager, ContextKeys } from '@omnitron-dev/titan/nexus';

@Injectable()
class TenantMiddleware {
  constructor(private readonly contextManager: ContextManager) {}

  async handle(ctx: any, next: () => Promise<any>) {
    const tenantId = ctx.headers.get('x-tenant-id');
    // Set on the active ContextProvider, not on the manager itself.
    this.contextManager.getCurrentContext().set(ContextKeys.Tenant, {
      id: tenantId,
      name: tenantId,
    });
    return next();
  }
}
```

To bind a context to one async flow — so concurrent requests don't
see each other's values — wrap the request handler in
`runWithContext`. The active `ContextProvider` is stored in an
`AsyncLocalStorage` frame, so every `getCurrentContext()` call inside
the callback (including across `await`s) sees the same provider
(`ContextManager.runWithContext`):

```typescript
const requestContext = contextManager.createScopedContext();
requestContext.set(ContextKeys.Tenant, { id: 'acme', name: 'Acme' });

await contextManager.runWithContext(requestContext, async () => {
  // Everything here resolves against the `acme` tenant context.
  await handleRequest();
});
```

> The Nexus `Container` holds its own `ContextManager` and a root
> `ContextProvider`. `container.getContext()` returns that provider and
> `container.withContext(fn)` runs `fn` inside the container's context
> frame (`Container.getContext` / `Container.withContext`) — convenient when you are wiring
> context outside an HTTP layer.

## End-to-end: context-driven provider selection

Here is the full loop, mirroring the integration test in
`test/nexus/context/context.spec.ts`. We populate a request-scoped
context, run a handler inside it, build a `ResolutionContext`, and let
the strategies pick the right provider from a candidate array:

```typescript
import {
  ContextManager,
  ContextKeys,
  Scope,
} from '@omnitron-dev/titan/nexus';

const manager = new ContextManager(); // default strategies pre-registered

// 1. Build the request context and fill it with the facts strategies read.
const requestContext = manager.createScopedContext();
requestContext.set(ContextKeys.User, { id: '1', name: 'Ada', roles: ['admin'] });
requestContext.set(ContextKeys.Tenant, { id: 'tenant1', name: 'Acme' });
requestContext.set(ContextKeys.Environment, 'production');
requestContext.set(ContextKeys.Features, ['newUI']);

// 2. The candidate providers for one logical token. Each carries the
//    discriminator its strategy looks at (environment / tenant / feature /
//    requiredRole). These are plain descriptors, not container registrations.
const providers = [
  { environment: 'development', value: 'dev-service' },
  { environment: 'production', value: 'prod-service' },
];

// 3. Run the handler inside the context, then select.
const chosen = await manager.runWithContext(requestContext, async () => {
  const resolutionContext = manager.createResolutionContext(
    /* container */ undefined,
    Scope.Request,
  );
  // resolutionContext.metadata now contains { user, tenant, environment,
  // features, ... } flattened from the current ContextProvider.

  return manager.selectProvider(SOME_TOKEN, providers, resolutionContext);
});

chosen.value; // 'prod-service' — EnvironmentStrategy matched 'production'
```

Walking `ContextManager.selectProvider`:

- `selectProvider` short-circuits when there are 0 or 1 candidates.
- Otherwise it iterates strategies **in registration order**
  (`environment`, then `feature-flag`, then `tenant`, then
  `role-based` by default) and calls `applies(token, ctx)` on each.
- The first strategy whose `applies` returns `true` runs
  `select(providers, ctx)`; if that returns a truthy provider, it
  wins. Here `EnvironmentStrategy.applies` is true (metadata has
  `environment`) and its `select` matches `provider.environment ===
  'production'`.
- If no strategy applies (or none selects), it falls back to
  `providers[0]`.

Because `environment` is registered first, it takes precedence over
`tenant` when both could match — registration order is the tie-break.

## Building a context-aware provider

`createContextAwareProvider` is an identity helper over the
`ContextAwareProvider` interface — `{ provide(context), canProvide?(context) }`
(`createContextAwareProvider` in `nexus/context.ts`). It returns its argument unchanged; its only job
is to give you the typed shape. The `provide` callback receives the
active `ResolutionContext` and returns the value (or a Promise of it);
the optional `canProvide` lets a provider opt out of a context:

```typescript
import { createContextAwareProvider } from '@omnitron-dev/titan/nexus';

const storageProvider = createContextAwareProvider<IStorage>({
  provide(context) {
    const tenant = context.metadata?.['tenant'];
    return tenant?.tier === 'enterprise' ? new S3Storage() : new LocalStorage();
  },
  canProvide(context) {
    return context.metadata?.['tenant'] !== undefined;
  },
});

const storage = await storageProvider.provide(resolutionContext);
```

All the branching lives inside `provide`; read whatever you need from
`context.metadata`, which `createResolutionContext` populated from the
current `ContextProvider` via `toObject()` — so a key set as
`ContextKeys.Tenant` (name `'tenant'`) is readable at
`context.metadata['tenant']`.

This is a **distinct** mechanism from `ResolutionStrategy`. A
context-aware provider computes *one* value from the context;
strategies *choose between several* pre-built providers. Use a
context-aware provider when the value is cheap to compute inline; use
strategies (+ `selectProvider`) when you have several concrete
implementations to pick from.

## Composition with scopes

Contextual providers usually live in `Request` scope — the context
varies per request, so the resolution should too. A `Singleton`
contextual provider would resolve once and cache the result,
defeating the purpose.

| Scope       | Behaviour with contextual                                     |
| ----------- | ------------------------------------------------------------- |
| `Singleton` | Strategy runs once on first resolve; result cached forever    |
| `Request`   | Strategy runs once per request; result cached for the request |
| `Transient` | Strategy runs every resolve                                   |

`Request` is almost always the right answer. (This assumes you have
wired `selectProvider`/`provide` into a factory registered at that
scope — the scope governs how often that factory re-runs, and the
factory is where the strategy or context-aware `provide` is invoked.)

## `@InjectContext` decorator

`@InjectContext(key)` is a **parameter** decorator that marks a
constructor (or method) parameter to receive a single context
**value**, identified by its `ContextKey` — it does not inject the
`ContextManager`:

```typescript
import { InjectContext, ContextKeys } from '@omnitron-dev/titan/nexus';

@Service({ name: 'users' })
class UsersService {
  constructor(
    @InjectContext(ContextKeys.Tenant) private readonly tenant: { id: string; name: string },
  ) {}

  @Public()
  async whoAmI() {
    return this.tenant?.id;
  }
}
```

Under the hood the decorator only records metadata: it writes the key
into a `context:inject` map keyed by parameter index, via
`Reflect.defineMetadata('context:inject', …)` (the `InjectContext` decorator).
The integrating runtime reads that metadata and supplies
`getCurrentContext().get(key)` for the marked parameter. (Plain
`container.resolve()` does not interpret `context:inject` on its own —
it is consumed by the service layer that builds the instance.)

Use it sparingly. Pulling context into business code blurs the line
that contextual injection is meant to draw: ideally a service is
handed the right *implementation* and never inspects the context
itself.

## Custom strategies

Implement `ResolutionStrategy` — `{ name, applies(token, ctx), select(providers, ctx) }`
— and register it with `contextManager.registerStrategy(...)`:

```typescript
import type { ResolutionStrategy } from '@omnitron-dev/titan/nexus';

const PaymentTierStrategy: ResolutionStrategy = {
  name: 'payment-tier',
  applies: (_token, ctx) => ctx.metadata?.['user']?.tier !== undefined,
  select: (providers, ctx) => {
    const tier = ctx.metadata?.['user']?.tier;     // 'free' | 'premium' | …
    return providers.find((p) => p.tier === tier) ?? providers[0];
  },
};
```

`select` receives the candidate provider **array** and the
`ResolutionContext`, and returns the chosen provider. `applies` gates
whether the strategy runs at all for a given token/context.

`registerStrategy` keys strategies by `name`, so registering a
strategy whose name matches a built-in one replaces it; use a fresh
name to add alongside the defaults, or `unregisterStrategy('environment')`
to drop a built-in. Newly registered strategies are appended, so they
are consulted *after* the four defaults — order matters because the
first strategy that both `applies` and returns a provider wins
(strategy registration and `selectProvider`).

## Anti-patterns

- **Contextual for static decisions.** If "which storage" is
  decided at boot, use a regular provider (or
  `EnvironmentStrategy` at registration time). Contextual adds
  runtime cost.
- **Reading context manually inside services.** The point of
  contextual injection is to push the decision into a strategy or
  context-aware provider, not into business logic.
- **Forgetting to set context.** A strategy that reads `tenant` from a
  context that was never set has nothing to match on, so
  `selectProvider` falls back to `providers[0]`. Make sure middleware
  that sets context runs (inside `runWithContext`) before selection
  happens.
- **Strategy with side effects.** Strategies are queried during
  resolution; they should be pure functions of the context.

---

---
id: "titan/di/devtools"
title: "DevTools"
description: "Inspect the container at runtime — dependency graph, performance, state snapshots."
---

# Container Introspection

> **History.** An experimental `DevToolsServer` / `DevToolsPlugin`
> debugger (with its own wire protocol) once lived in
> `src/nexus/devtools.ts`. It had zero consumers and was **removed**
> (NX-5); `NEXUS_FEATURES.DEVTOOLS` is `false`. There is no
> `@omnitron-dev/titan/nexus/devtools` subpath. What survived — and
> what this page documents — is the dependency-graph export plus the
> lifecycle-observer system, both of which ship from the main
> `@omnitron-dev/titan/nexus` barrel.

You can still introspect a running container: which providers are
registered, what depends on what, how long resolutions take, what the
in-memory graph looks like.

## Two things you can inspect

1. **The dependency graph** — `Container.exportGraph()` returns the
   provider/dependency graph, renderable as DOT, Mermaid, or JSON.
2. **Lifecycle events** — the observer system surfaces resolution
   timings, memory, and an audit trail.

## The dependency graph — `Container.exportGraph()`

The container exposes `exportGraph(options?)` directly. It walks the
container's registrations and returns a stable `DependencyGraph`
(defined in `src/nexus/dependency-graph.ts`):

```typescript
import { exportToMermaid, exportToDot, exportToJson } from '@omnitron-dev/titan/nexus';

const graph = container.exportGraph();        // { includeParent?: boolean }

interface DependencyGraph {
  nodes: Array<{ id: string; label?: string; type?: string }>;
  edges: Array<{ from: string; to: string; type?: 'dependency' | 'parent' }>;
  roots?: string[];
  leaves?: string[];
}

console.log(exportToMermaid(graph));          // also: exportToDot, exportToJson
```

The same export backs `omnitron inspect <app> --graph`. The three
formatters (`exportToDot` / `exportToMermaid` / `exportToJson`) are
exported from the nexus barrel, so you can render the graph with
Graphviz, Mermaid, or pipe the JSON anywhere.

## Observers

DevTools build on Nexus's lifecycle observation system. Three
built-in observers ship in `nexus/lifecycle.ts`:

| Observer              | Purpose                              |
| --------------------- | ------------------------------------ |
| `AuditObserver`       | Audit every provider lifecycle event |
| `MemoryObserver`      | Track memory growth per provider     |
| `PerformanceObserver` | Track resolution timing              |

Attach observers via `LifecycleManager`:

```typescript
import {
  LifecycleManager,
  AuditObserver,
  PerformanceObserver,
  type LifecycleEvent,
  type LifecycleEventData,
  type LifecycleObserver,
} from '@omnitron-dev/titan/nexus';

const manager = new LifecycleManager();
manager.addObserver(new AuditObserver());
manager.addObserver(new PerformanceObserver());

manager.on(LifecycleEvent.BeforeResolve, (data: LifecycleEventData) => {
  // …
});
```

The resolution events are `BeforeResolve` (`'resolve:before'`),
`AfterResolve` (`'resolve:after'`), and `ResolveFailed`
(`'resolve:failed'`) — see the full `LifecycleEvent` map in
`src/nexus/lifecycle.ts` for container, instance, module, cache,
scope, and middleware events.

## Production guidance

Enable observers only when:

- Diagnosing a specific issue (slow boot, suspected leak,
  mystery resolution).
- Running in a controlled environment with extra headroom.
- For a bounded period, then disable.

Observers add per-resolution work and retain event data, so the
overhead is non-zero. Long-running production processes with verbose
observers attached will slowly accumulate memory — attach them for a
bounded window, then remove them.

## Anti-patterns

- **Leaving verbose observers on in production by default.**
  Diagnostic tooling, not a runtime requirement.
- **Using introspection data to drive runtime decisions.** The graph
  and observer data are for *humans diagnosing*, not for code reading.
  If your application needs to know about its own DI graph at runtime,
  you have probably reinvented configuration in a worse form.

→ Back to [DI Overview](/docs/titan/di/overview).

---

---
id: "titan/di/middleware"
title: "DI Middleware"
description: "Wrap container resolution itself — retry, caching, rate-limit, circuit-breaker, logging, validation, transactions."
---

# DI Middleware

DI middleware wraps **container resolution**. It runs at construction
time, not at call time. Use it to add cross-cutting behaviour to how
providers are constructed, not to how their methods are called.

```typescript
import {
  type Middleware,
  type MiddlewareFunction,
  type MiddlewareNext,
  type MiddlewareResult,
  createMiddleware,
  composeMiddleware,
  MiddlewarePipeline,
  // Built-ins
  RetryMiddleware,
  LoggingMiddleware,
  CachingMiddleware,
  RateLimitMiddleware,
  ValidationMiddleware,
  TransactionMiddleware,
  CircuitBreakerMiddleware,
} from '@omnitron-dev/titan/nexus';
```

> **DI middleware ≠ Netron middleware.** This page covers wrapping
> the resolution of providers. For wrapping per-call RPC dispatch,
> see [Netron Middleware](/docs/titan/netron/middleware).

## The `Middleware` interface

```typescript
interface Middleware<T = unknown> {
  name:     string;
  execute:  (context: MiddlewareContext<T>, next: () => T | Promise<T>) => T | Promise<T>;
  priority?: number;
  condition?: (context: MiddlewareContext<T>) => boolean;
  onError?:   (error: Error, context: MiddlewareContext<T>) => void;
}
```

`context` carries:

- `token` — the token being resolved.
- `scope` — current resolution scope.
- `parent` — the resolution that triggered this one (depth +
  cycle tracking).

`next()` resolves the provider; call exactly once and return the
value.

## Built-in middleware

```mermaid
flowchart LR
  Resolve[container.resolve]
  Resolve --> M1[Logging]
  M1 --> M2[Retry]
  M2 --> M3[Caching]
  M3 --> M4[CircuitBreaker]
  M4 --> M5[Provider executes]
```

| Middleware                 | Purpose                                              |
| -------------------------- | ---------------------------------------------------- |
| `RetryMiddleware`          | Retry construction if it throws                      |
| `LoggingMiddleware`        | Log every resolution with timing                     |
| `CachingMiddleware`        | Cache resolved instances by token + context          |
| `RateLimitMiddleware`      | Reject resolutions exceeding a threshold             |
| `ValidationMiddleware`     | Validate resolved instance against a schema          |
| `TransactionMiddleware`    | Wrap resolution in a transaction (DB-aware contexts) |
| `CircuitBreakerMiddleware` | Open a circuit after consecutive resolution failures |

These come in two flavours. `LoggingMiddleware`, `CachingMiddleware`,
`RetryMiddleware`, `ValidationMiddleware`, and `TransactionMiddleware`
are **pre-built middleware objects** (`export const … = createMiddleware(…)`)
— add them directly. `CircuitBreakerMiddleware` and
`RateLimitMiddleware` are **classes** you instantiate with `new`
(e.g. `new RateLimitMiddleware(100, 60000)`).

## Applying middleware

Middleware is added one at a time via `container.addMiddleware()`,
which returns the container for chaining:

```typescript
const container = createContainer();

container.addMiddleware(LoggingMiddleware);
container.addMiddleware(CachingMiddleware);
container.addMiddleware(new CircuitBreakerMiddleware({ threshold: 5 }));
container.addMiddleware(new RateLimitMiddleware(100, 60000)); // 100/min
```

Execution order is governed by each middleware's `priority` field
(higher priority runs first); the container sorts on insertion. There
is no array-position ordering — set `priority` explicitly when order
matters.

## Custom middleware via `createMiddleware`

```typescript
import { createMiddleware, getTokenName } from '@omnitron-dev/titan/nexus';

const InstrumentationMiddleware = createMiddleware({
  name:    'instrumentation',
  execute: async (ctx, next) => {
    const t0 = performance.now();
    try {
      const instance = await next();
      // `ctx.token` is a ServiceIdentifier: a class, a string, a symbol or a
      // Token. Only two of those have `.name`, so use the exported helper.
      metrics.histogram('di.resolve.ms', { token: getTokenName(ctx.token) })
        .observe(performance.now() - t0);
      return instance;
    } catch (e) {
      metrics.counter('di.resolve.errors', { token: getTokenName(ctx.token) }).inc();
      throw e;
    }
  },
});

container.addMiddleware(InstrumentationMiddleware);
```

## Composing middleware

`composeMiddleware(...middlewares)` takes middleware as rest arguments
and produces a single middleware that chains them. Useful for grouping
related middleware as a single unit, then adding the composite with one
`addMiddleware()` call.

`MiddlewarePipeline` is the class that drives execution; the
container creates one internally, but you can construct one
yourself if you need to test middleware in isolation.

## Conditional execution

Middleware can opt out per resolution via `condition`:

```typescript
const ExpensiveMiddleware: Middleware = {
  name: 'expensive',
  condition: (ctx) => ctx.token === MY_TOKEN,
  execute: async (ctx, next) => { /* … */ return next(); },
};
```

When `condition(ctx)` returns false the middleware is skipped for that
resolution and `next()` runs directly.

## Per-token middleware

Some middleware suites support per-token application. Check the
specific built-in for whether it accepts a `tokens` filter, and the
provider definition for a `middleware:` field. The exact surface
varies and is documented in the source for each middleware.

## When to write DI middleware

Custom DI middleware is *rare*. The built-ins cover most cases.
Write your own when you need:

- **Project-specific telemetry** — emit to a metrics backend the
  built-ins don't know about.
- **Cross-cutting validation** — assert that every resolved
  provider matches a schema.
- **Tracing instrumentation** — attach a span to every resolution.
- **Test-only middleware** — capture the resolution graph for a
  test assertion.

Most application code does not need DI middleware at all.

## Anti-patterns

- **Putting business logic in DI middleware.** It runs at
  construction time, not per call. Method-call concerns belong in
  Netron middleware.
- **Side effects in middleware.** Middleware should observe and
  decorate, not mutate the application. A middleware that
  registers more providers is a smell.
- **Async work that should be in `onStart`.** Heavy setup belongs
  in lifecycle hooks where the framework can sequence it.

---

---
id: "titan/di/multi-injection"
title: "Multi-injection"
description: "One token, many providers — the canonical pattern for plugins, validators, middleware chains."
---

# Multi-injection

Multi-injection is one token with many providers. Resolving the token
returns an array of all of them. It is the foundation for every plugin
or registry pattern in Titan.

## The basic pattern

```typescript
import { createToken } from '@omnitron-dev/titan/nexus';

interface IValidator {
  name: string;
  validate(input: unknown): boolean;
}

const VALIDATORS = createToken<IValidator[]>('Validators');

@Injectable()
class EmailValidator implements IValidator {
  name = 'email';
  validate(input: unknown) { return typeof input === 'string' && input.includes('@'); }
}

@Injectable()
class PasswordValidator implements IValidator {
  name = 'password';
  validate(input: unknown) { return typeof input === 'string' && input.length >= 8; }
}

@Module({
  providers: [
    EmailValidator,
    PasswordValidator,
    { provide: VALIDATORS, useExisting: EmailValidator,    multi: true },
    { provide: VALIDATORS, useExisting: PasswordValidator, multi: true },
  ],
  exports: [VALIDATORS],
})
class ValidationModule {}
```

A consumer injects the array:

```typescript
@Service('signup@1.0.0')
class SignupService {
  constructor(@Inject(VALIDATORS) private readonly validators: IValidator[]) {}

  @Public()
  async signup(input: unknown) {
    for (const v of this.validators) {
      if (!v.validate(input)) throw Errors.validation(`${v.name} failed`);
    }
    // …
  }
}
```

The array contains every provider registered against `VALIDATORS`,
in the order they were registered. New validators in new modules
slot in without changing the consumer.

> Titan also ships a dedicated `@InjectAll(token)` decorator
> (`@omnitron-dev/titan`) that explicitly resolves *all* providers
> bound to a token into an array — use it when you want the intent to
> be unambiguous at the injection site.

## Why this matters

The alternative — a registry the consumer maintains — couples consumer
and contributor. Adding a new validator means editing the consumer.
With multi-injection, the consumer depends on a token; contributors
depend on the token. The two never meet.

This is the core of every extensibility seam in Titan:

- **Health indicators** — modules contribute, the aggregator
  consumes.
- **DI middleware** — modules contribute, the container consumes.
- **Netron RPC middleware** — modules contribute, Netron consumes.
- **Lifecycle phase tasks** — modules contribute, the coordinator
  consumes.
- **Custom event handlers** — modules contribute, the bus consumes.

All of them use multi-injection underneath.

## Cross-module multi-tokens

Multi-tokens compose across modules. If module A registers two
providers and module B registers one, a consumer that imports both
gets all three:

```typescript
@Module({
  providers: [
    { provide: VALIDATORS, useExisting: EmailValidator,    multi: true },
    { provide: VALIDATORS, useExisting: PasswordValidator, multi: true },
  ],
  exports: [VALIDATORS],
})
class CoreValidatorsModule {}

@Module({
  providers: [
    { provide: VALIDATORS, useExisting: PhoneNumberValidator, multi: true },
  ],
  exports: [VALIDATORS],
})
class ExtraValidatorsModule {}

@Module({
  imports: [CoreValidatorsModule, ExtraValidatorsModule],
  providers: [SignupService],
})
class AppModule {}

// SignupService gets [EmailValidator, PasswordValidator, PhoneNumberValidator]
```

The accumulation is the runtime composition of `imports`. Drop
`ExtraValidatorsModule` from the root, and `SignupService` sees only
the core validators — no other code change required.

## Order

Providers register in the order their modules are discovered, which
follows the `imports` graph. Within a module, registration follows
the order in `providers`. The result is deterministic but tied to
import order, so if order matters (middleware chains, for example),
make it explicit:

```typescript
interface IMiddleware {
  priority: number;        // explicit ordering
  handle(ctx: any, next: () => any): any;
}

@Service('orders@1.0.0')
class OrdersService {
  constructor(@Inject(MIDDLEWARE) private readonly middleware: IMiddleware[]) {
    this.middleware.sort((a, b) => a.priority - b.priority);
  }
}
```

## Optional empty arrays

If no provider is registered, resolving a multi-token returns an empty
array — not `undefined`, not throws. This means consumers can write
code that handles the no-contributor case naturally:

```typescript
const validators = container.resolve(VALIDATORS);   // [] if nothing registered
for (const v of validators) v.validate(input);       // loop is a no-op
```

## Mixing multi and single

A token is either multi or single. You cannot register one provider
without `multi: true` and another with `multi: true` against the same
token — the framework will throw at registration time.

If you need the "first one wins" pattern, use a regular single token
plus `useExisting` to alias to the chosen impl. If you need the "all
of them" pattern, use multi.

## A real example — health indicators

The `titan-health` ecosystem module uses multi-injection so any
module can contribute an indicator without changing the consumer:

```typescript
import { Injectable } from '@omnitron-dev/titan';
import { createToken } from '@omnitron-dev/titan/nexus';

const HEALTH_INDICATOR = createToken<IHealthIndicator[]>('HealthIndicator');

@Injectable()
class StripeHealth implements IHealthIndicator {
  async check() { /* … */ }
}

@Module({
  providers: [
    StripeHealth,
    { provide: HEALTH_INDICATOR, useExisting: StripeHealth, multi: true },
  ],
  exports: [HEALTH_INDICATOR],
})
class PaymentsModule {}
```

Adding `PaymentsModule` to the root module adds the indicator to
the health aggregator. Removing the module removes the indicator.
No other code changes.

The exact token name and shape are owned by `titan-health`; the
pattern shown above is the general multi-injection recipe.

## Anti-patterns

- **Forgetting `multi: true`.** Without it, the second registration
  overwrites the first. Symptom: only one provider shows up in the
  array.
- **Mutating the resolved array.** The container does not defensively
  copy. Mutating the array mutates the cached resolution. Treat
  resolved arrays as read-only.
- **Order-dependent behaviour without explicit ordering.** If your
  consumer assumes a specific order, encode that order in the
  providers themselves (`priority` field, sort step) — do not rely
  on import-order coincidence.

---

---
id: "titan/di/overview"
title: "Dependency Injection"
description: "Nexus — the IoC container that owns every Titan-managed object."
---

# Dependency Injection

The DI container is the single source of truth for every object in a
Titan application. It is named **Nexus** and ships in the
`@omnitron-dev/titan/nexus` subpath.

```typescript
import { Container, createContainer } from '@omnitron-dev/titan/nexus';
```

This page is the entry point. Mechanics in the per-page references.

## What a DI container does

Three jobs:

1. **Registration** — you tell the container "this token maps to
   this provider".
2. **Resolution** — you ask the container for a token; it constructs
   (or returns the cached instance of) whatever the provider says.
3. **Lifetime** — the container decides when a constructed instance
   is reused, scoped, or disposed.

In a Titan app, you almost never call `container.resolve` yourself.
You declare a class with constructor dependencies; the container
walks the constructor signature, resolves each dependency, and
injects them.

```typescript
@Service({ name: 'users' })
export class UsersService {
  // The container resolves Database and LoggerService from the
  // module's import graph and injects them here.
  constructor(
    private readonly db:     Database,
    private readonly logger: LoggerService,
  ) {}
}
```

## What Nexus brings beyond a basic DI container

```mermaid
flowchart LR
  subgraph Core
    Reg[Registration]
    Res[Resolution]
    Sc[Scoping]
    Lc[Lifecycle]
  end
  subgraph Advanced
    Mt[Multi-tokens]
    Cx[Contextual injection]
    Mw[DI middleware pipeline]
    Ld[Lazy / Async / Conditional]
  end
  subgraph Diagnostics
    Dt[Dependency-graph export]
    Lc2[Lifecycle observers]
  end
  Core --> Advanced
  Advanced --> Diagnostics
```

> The experimental Service Mesh, Distributed Tracing, and DevTools
> debugger modules were removed (`NEXUS_FEATURES.SERVICE_MESH`,
> `DISTRIBUTED_TRACING`, and `DEVTOOLS` are all `false`). The
> dependency-graph export and lifecycle observers survive as stable
> diagnostics — see [DevTools](/docs/titan/di/devtools).

- **Multi-tokens** — a single token can hold multiple providers
  (`createMultiToken<T[]>('Validators')`).
- **Contextual injection** — `ContextManager` + strategies
  (`RoleBasedStrategy`, `EnvironmentStrategy`, `FeatureFlagStrategy`,
  `TenantStrategy`) resolve different providers per request /
  tenant / feature flag.
- **DI middleware** — `RetryMiddleware`, `CachingMiddleware`,
  `LoggingMiddleware`, `RateLimitMiddleware`,
  `CircuitBreakerMiddleware`, `ValidationMiddleware`,
  `TransactionMiddleware` wrap resolution itself.
- **Cross-platform** — Node, Bun, Deno, browser. Runtime detection
  via `Runtime`, `detectRuntime()`, `isBun()`, `isNode()`, `isDeno()`,
  `isBrowser()`.
- **Lifecycle hooks** — `LifecycleManager`, `LifecycleEvent`,
  `LifecycleObserver`, plus built-in observers (`AuditObserver`,
  `MemoryObserver`, `PerformanceObserver`).

You do not need to learn a new mental model — Nexus is "DI like you
already know" — but the deeper pages cover the parts that go beyond
the basics.

## A complete example

```typescript
import {
  Container,
  createContainer,
  createToken,
  Scope,
} from '@omnitron-dev/titan/nexus';

interface ILogger {
  info(message: string): void;
}

const LOGGER = createToken<ILogger>('Logger');

class ConsoleLogger implements ILogger {
  info(message: string) { console.log(message); }
}

class UsersService {
  constructor(private readonly logger: ILogger) {}
  list() { this.logger.info('listing users'); return []; }
}

const container = createContainer();

container.register(LOGGER, {
  useClass: ConsoleLogger,
  scope:    Scope.Singleton,
});

container.register(UsersService, {
  useClass: UsersService,
  inject:   [LOGGER],
  scope:    Scope.Singleton,
});

const users = container.resolve(UsersService);
users.list();
```

In a Titan app, you do not write `container.register` calls —
`@Module`, `@Service`, and `@Injectable` translate to the same
registrations behind the scenes.

## What the container guarantees

| Guarantee                  | Explanation                                           |
| -------------------------- | ----------------------------------------------------- |
| **Topological resolution** | Dependencies are constructed before dependents        |
| **Cycle detection**        | Throws `CircularDependencyError` at resolution time   |
| **Scope correctness**      | A `Singleton` returns the same instance every time    |
| **Disposal**               | `container.dispose()` runs lifecycle teardown         |
| **Type safety**            | Tokens are typed; `resolve(LOGGER)` returns `ILogger` |
| **Async resolution**       | `resolveAsync<T>(token)` for async providers          |

## Error types exposed

The Nexus error hierarchy (`NexusError` → specialised subclasses):

| Error class                  | When                                             |
| ---------------------------- | ------------------------------------------------ |
| `RegistrationError`          | Invalid provider registration                    |
| `ResolutionError`            | Generic resolution failure                       |
| `DependencyNotFoundError`    | Provider for token not registered                |
| `CircularDependencyError`    | Constructor cycle detected                       |
| `ScopeMismatchError`         | Narrower scope captured in wider scope           |
| `NotInjectableError`         | Class missing the `@Injectable` decoration       |
| `DuplicateRegistrationError` | Two providers for the same single token          |
| `InvalidProviderError`       | Provider definition malformed                    |
| `InitializationError`        | Provider construction threw                      |
| `AsyncResolutionError`       | Sync `resolve()` on an async provider            |
| `ContainerDisposedError`     | Resolve after dispose                            |
| `DisposalError`              | Error during teardown                            |
| `NexusAggregateError`        | Multiple errors batched (e.g. parallel disposal) |
| `ModuleError`                | Module-level configuration error                 |

## Read the deep pages

| Topic                                                         | When to read                                        |
| ------------------------------------------------------------- | --------------------------------------------------- |
| [Providers](/docs/titan/di/providers)                         | The provider types and when to use each             |
| [Scopes](/docs/titan/di/scopes)                               | Transient / Singleton / Scoped / Request            |
| [Tokens](/docs/titan/di/tokens)                               | Class, symbol, multi-, lazy, async, optional tokens |
| [Multi-injection](/docs/titan/di/multi-injection)             | Plugin patterns, middleware chains                  |
| [Contextual injection](/docs/titan/di/contextual-injection)   | Per-request / per-tenant / per-environment          |
| [Middleware](/docs/titan/di/middleware)                       | Wrapping resolution itself                          |
| [Circular Dependencies](/docs/titan/di/circular-dependencies) | Diagnosing and fixing cycles                        |
| [DevTools](/docs/titan/di/devtools)                           | Inspecting the container at runtime                 |

---

---
id: "titan/di/providers"
title: "Providers"
description: "The provider types Nexus understands and the helpers that build them."
---

# Providers

A provider tells the container *how* to produce a value for a token.
Nexus exposes four core shapes plus async / conditional / multi
variants — and helper functions that produce them.

```typescript
import {
  createValueProvider,
  createFactoryProvider,
  createClassProvider,
  createTokenProvider,
  createMultiProvider,
  createConditionalProvider,
  isAsyncProvider,
  isMultiProvider,
  isConstructor,
  hasScope,
} from '@omnitron-dev/titan/nexus';
```

## 1. ClassProvider — `useClass`

The default. The container calls `new` on the class, resolving its
constructor dependencies first.

```typescript
container.register(LOGGER, {
  useClass: ConsoleLogger,
  scope:    Scope.Singleton,
  inject:   [],                // optional; auto-detected from metadata
  multi:    false,             // single token
});
```

A `ClassProvider` also accepts `condition` (a context predicate) and
`fallback` (a provider to use when the condition fails). `async` and
`timeout`/`retry` belong to `FactoryProvider`, not `ClassProvider`.

In Titan, this is what `@Module({ providers: [ConsoleLogger] })`
becomes. The bare-class form is shorthand:

```typescript
@Module({ providers: [ConsoleLogger] })   // same as { useClass: ConsoleLogger }
```

Use when: the value is a class you control and the container can
construct it.

## 2. ValueProvider — `useValue`

A pre-built value. The container hands out the same value every time.

```typescript
container.register(API_KEY, { useValue: 'sk_live_…' });
container.register(CONFIG,  { useValue: { tier: 'redis-lru', max: 10_000 } });
```

Or via helper — note the helpers take the token and return a
`[token, provider]` tuple ready to spread into `register`:

```typescript
const [token, provider] = createValueProvider(API_KEY, 'sk_live_…');
container.register(token, provider);
```

Use when: the value is data, an external object, or a constant.

## 3. FactoryProvider — `useFactory`

A function called once (per scope) to produce the value. The factory
can declare its own dependencies via `inject`.

```typescript
container.register(DB_POOL, {
  useFactory: (config: ConfigService) => {
    return createPool({
      url: config.get('database.url'),
      max: config.get('database.pool.max'),
    });
  },
  inject: [ConfigService],
  scope:  Scope.Singleton,
});
```

The factory may return a Promise; declare with `async: true` or use
`resolveAsync` on the caller side:

```typescript
container.register(REDIS_CLIENT, {
  useFactory: async (config: ConfigService) => {
    const client = createRedisClient({ url: config.get('redis.url') });
    await client.connect();
    return client;
  },
  inject: [ConfigService],
  async:  true,
  scope:  Scope.Singleton,
});

const redis = await container.resolveAsync(REDIS_CLIENT);
```

Use when: the value requires async setup, complex construction
logic, or a runtime decision.

## 4. TokenProvider — `useToken`

Aliases one token to another. Resolving the alias returns whatever
the other token resolves to.

```typescript
container.register(LOGGER, {
  useClass: ConsoleLogger,
  scope:    Scope.Singleton,
});

container.register(AUDIT_LOGGER, {
  useToken: LOGGER,           // same instance as LOGGER
});
```

Use when: you want two tokens to refer to the same object —
feature flags that gate which implementation, abstraction wrappers,
gradual migration between two interfaces.

## 5. MultiProvider — `multi: true` (or `createMultiProvider`)

Multiple providers register against the same multi-token. Resolving
returns an array.

```typescript
import { createMultiToken } from '@omnitron-dev/titan/nexus';

const VALIDATORS = createMultiToken<IValidator>('Validators');

container.register(VALIDATORS, { useClass: EmailValidator, multi: true });
container.register(VALIDATORS, { useClass: PasswordValidator, multi: true });

const validators = container.resolve(VALIDATORS);  // IValidator[]
```

See [Multi-injection](/docs/titan/di/multi-injection) for the full pattern.

## Conditional providers — `createConditionalProvider`

```typescript
import { createConditionalProvider } from '@omnitron-dev/titan/nexus';

// createConditionalProvider(token, provider, condition, fallback?)
// returns a [token, provider] tuple with `condition`/`fallback` attached.
const [token, provider] = createConditionalProvider(
  BILLING_SERVICE,
  { useClass: StripeBillingService, scope: Scope.Singleton },
  (ctx) => ctx.metadata?.billingProvider === 'stripe',
  { useClass: NoopBillingService },
);

container.register(token, provider);
```

The `condition` predicate runs at resolution time against the
`ResolutionContext`; if it returns false the `fallback` provider is
used. (`condition` and `fallback` are first-class fields on every
provider shape — see `types.ts`.)

## Provider options summary

| Option      | Default           | Effect                                                  |
| ----------- | ----------------- | ------------------------------------------------------- |
| `scope`     | `Scope.Singleton` | Lifetime — see [Scopes](/docs/titan/di/scopes)          |
| `inject`    | `[]`              | Tokens for `useFactory` arguments or constructor params |
| `multi`     | `false`           | Multi-injection — multiple providers per token          |
| `async`     | `false`           | `FactoryProvider` only — provider produces a Promise    |
| `condition` | —                 | Context predicate; falls back to `fallback` when false  |
| `fallback`  | —                 | Provider used when `condition` returns false            |

## Provider predicates

Helper predicates for introspection:

| Function                    | What it checks                            |
| --------------------------- | ----------------------------------------- |
| `isConstructor(value)`      | Is the value a class constructor?         |
| `isAsyncProvider(provider)` | Does the provider produce a Promise?      |
| `isMultiProvider(provider)` | Is it registered as `multi: true`?        |
| `hasScope(provider)`        | Does the provider have an explicit scope? |

## Choosing between provider types

| If you have …                                      | Use                         |
| -------------------------------------------------- | --------------------------- |
| A class you can `new` directly                     | `useClass`                  |
| A pre-built object, literal, or constant           | `useValue`                  |
| A function that produces the value (sync or async) | `useFactory`                |
| One token that should mirror another               | `useToken`                  |
| Multiple independent contributions to one consumer | `multi: true`               |
| Conditional based on context / config              | `createConditionalProvider` |

## Anti-patterns

- **`useFactory` for trivial construction.** If the factory is just
  `() => new MyClass()`, use `useClass`.
- **`useValue` for mutable shared state.** A `useValue` provider
  hands out the *same reference* every time. If consumers mutate
  it, the state is global. Wrap mutable state in a class.
- **Async providers in hot paths.** Async resolution awaits a
  Promise on every call (per scope). Prefer eager async setup in
  `onStart` lifecycle hooks.

---

---
id: "titan/di/scopes"
title: "Scopes"
description: "Transient, Singleton, Scoped, Request — how Nexus decides when to reuse an instance."
---

# Scopes

A scope is the container's answer to "when do I make a new instance?"
Nexus has four:

```typescript
import { Scope } from '@omnitron-dev/titan/nexus';

Scope.Transient   // 'transient'
Scope.Singleton   // 'singleton'  (default)
Scope.Scoped      // 'scoped'
Scope.Request     // 'request'
```

```mermaid
flowchart LR
  Resolve[container.resolve]
  Resolve --> CheckScope{Scope?}
  CheckScope -->|Singleton| GlobalCache[(Global instance cache)]
  CheckScope -->|Transient| New[Construct fresh]
  CheckScope -->|Scoped| ChildCache[(Child-container cache)]
  CheckScope -->|Request| ReqCache[(Per-request cache)]
  GlobalCache --> Return
  New --> Return
  ChildCache --> Return
  ReqCache --> Return
```

| Scope       | New instance per …                | Use for                                     |
| ----------- | --------------------------------- | ------------------------------------------- |
| `Singleton` | Container (one for the whole app) | Stateless services, infra (logger, db pool) |
| `Transient` | `resolve()` call (every time)     | Lightweight value objects, builders         |
| `Scoped`    | Child container scope             | Per-feature state, request-aware factories  |
| `Request`   | Request scope                     | Per-request data — current user, request id |

`Singleton` is the default.

## Singleton

One instance for the lifetime of the container. The container caches
the result of the first `resolve()` and returns it on every
subsequent call.

```typescript
container.register(LOGGER, {
  useClass: ConsoleLogger,
  scope:    Scope.Singleton,        // default; can omit
});
```

**Use for**: stateless services, services that hold shared
infrastructure (pool, client, registry), services with expensive
construction.

**Avoid for**: services that hold per-request state (current user,
transaction handle). Use `Request` instead.

## Transient

A new instance every time the container resolves the token.

```typescript
container.register(REQUEST_BUILDER, {
  useClass: RequestBuilder,
  scope:    Scope.Transient,
});

const a = container.resolve(REQUEST_BUILDER);
const b = container.resolve(REQUEST_BUILDER);
a === b;  // false
```

**Use for**: lightweight value objects (builders, formatters,
short-lived calculators) where instantiation is cheap.

**Avoid for**: services that allocate resources (file handles,
sockets, large caches). Each instance leaks unless explicitly
disposed.

## Scoped

One instance per child container scope. Used to isolate state for a
feature or for a request handler.

```typescript
container.register(FEATURE_STATE, {
  useClass: FeatureState,
  scope:    Scope.Scoped,
});
```

In day-to-day Titan code, `Scoped` overlaps with module-level
encapsulation; you can usually achieve the same isolation by
declaring the provider in the module that owns the scope and not
exporting it.

## Request

One instance per request scope. The container creates a child scope
per request (typically per Netron call); providers in `Request`
scope live exactly as long as that scope.

```typescript
container.register(REQUEST_CONTEXT, {
  useClass: RequestContext,
  scope:    Scope.Request,
});
```

**Use for**:

- Holding the current user (set by auth middleware, read by
  business logic).
- Carrying trace context across handlers within one request.
- Per-request transaction handles in DB modules.

**Avoid for**: shared infrastructure. A `Request`-scoped database
pool would create a new connection per request — exactly what the
pool is meant to prevent.

## Mixing scopes — the "narrower-into-wider" rule

A wider-scoped provider **must not** depend on a narrower-scoped
one. This is a `ScopeMismatchError`:

```typescript
@Service({ name: 'users' })                 // Singleton (wider)
class UsersService {
  constructor(private ctx: RequestContext) {}   // Request (narrower) — invalid
}
```

Why: a singleton lives for the whole app. If it captures a
request-scoped object, that object outlives its scope. Subsequent
requests see stale data.

The fix: inject a *factory* that resolves the narrower scope on
demand, or use `Lazy` (see [Tokens](/docs/titan/di/tokens)).

## Lifetime by scope

| Scope       | Created                  | Disposed                        |
| ----------- | ------------------------ | ------------------------------- |
| `Singleton` | First resolve            | `container.dispose()`           |
| `Transient` | Every resolve            | Never (caller's responsibility) |
| `Scoped`    | First resolve in scope   | Child-scope disposal            |
| `Request`   | First resolve in request | End of request                  |

`Transient` instances are not tracked by the container. If they
hold resources, the caller must dispose them.

## Anti-patterns

- **`Singleton` for per-request state.** Easy mistake; usually
  fixed by wrapping the state in a service that injects a
  `Request`-scoped context.
- **`Transient` for resource-holding services.** Each instance
  leaks. Use `Singleton` or explicit pooling.
- **`Request` for stateless calculators.** Wasteful — a new
  instance per request, with no upside.

---

---
id: "titan/di/tokens"
title: "Tokens"
description: "Class, symbol, multi-, lazy, async, optional, config, stream — the token family."
---

# Tokens

A token is the key the container uses to look up a provider. Nexus
ships a rich family of token constructors for different use cases.

```typescript
import {
  createToken,
  createMultiToken,
  createLazyToken,
  createOptionalToken,
  createAsyncToken,
  createConfigToken,
  createScopedToken,
  createStreamToken,
  isToken,
  isMultiToken,
  isOptionalToken,
  tokenFromClass,
  getTokenName,
} from '@omnitron-dev/titan/nexus';
```

## Token interface

Every token implements:

```typescript
interface Token<T = any> {
  readonly id:       symbol;
  readonly name:     string;
  readonly metadata: TokenMetadata;
  readonly type?:    T;
  toString(): string;
}
```

The phantom `type?: T` carries the value type for compile-time
checking; tokens are matched by reference equality at runtime
(identity = `id` symbol).

## Token kinds

### Class token — the class itself

```typescript
@Service({ name: 'users' })
class UsersService {}

container.resolve(UsersService);    // class is the token
```

When you have a concrete class, the class **is** the token. This is
the idiomatic form for service classes.

### Symbol token — `createToken<T>(name)`

```typescript
interface ILogger { info(msg: string): void; }
const LOGGER = createToken<ILogger>('Logger');

container.register(LOGGER, { useClass: ConsoleLogger });
const logger = container.resolve(LOGGER);  // ILogger
```

The name is both a debug label **and** the registry key: `createToken`
caches by name in a process-global registry, so two
`createToken('Logger')` calls (with no metadata) return the **same**
token instance. Tokens are still matched by `symbol` identity at
runtime — the registry just guarantees that identity is shared for a
given name.

Use for: interfaces, configuration bundles, third-party types.

### Multi-token — `createMultiToken<T>(name)`

```typescript
const VALIDATORS = createMultiToken<IValidator>('Validators');

container.register(VALIDATORS, { useClass: EmailValidator,    multi: true });
container.register(VALIDATORS, { useClass: PasswordValidator, multi: true });

container.resolve(VALIDATORS);  // IValidator[]
```

Use for: plugin registration, middleware chains, validators. See
[Multi-injection](/docs/titan/di/multi-injection).

### Optional token — `createOptionalToken<T>(name)`

```typescript
const METRICS = createOptionalToken<IMetrics>('Metrics');

@Service({ name: 'users' })
class UsersService {
  constructor(@Inject(METRICS) private readonly metrics?: IMetrics) {}
  // metrics is undefined if no provider was registered.
}
```

`isOptionalToken(token)` distinguishes these at runtime.

### Lazy token — `createLazyToken<T>(name, metadata?)`

```typescript
const AUTH = createLazyToken<AuthService>('Auth');
```

Tags the token as lazy (`isLazy: true`); pair it with
`container.resolveLazy(AUTH)` to defer construction until first use.
Useful for breaking circular import graphs. See
[Circular Dependencies](/docs/titan/di/circular-dependencies).

### Async token — `createAsyncToken<T>(name)`

```typescript
const REMOTE = createAsyncToken<RemoteService>('Remote');

container.register(REMOTE, {
  useFactory: async () => connectRemote(),
  async:      true,
});

const remote = await container.resolveAsync(REMOTE);
```

### Config token — `createConfigToken<T>(name, { validate?, defaults? })`

A typed token specifically for configuration bundles. The
`@omnitron-dev/titan/module/config` module uses these internally.

### Stream token — `createStreamToken<T>(name)`

A token for streaming values (async iterables). Used in advanced
patterns where the provider yields data continuously.

### Scoped token — `createScopedToken<T>(name, scope)`

Bake a default scope into the token; providers registered against it
inherit the scope unless explicitly overridden.

## Class → token helper

```typescript
const token = tokenFromClass(UsersService);
// Equivalent to using UsersService directly as the token,
// but returns the canonical Token<UsersService> object.
```

Useful when you need to pass the token around (logs, error messages)
without re-deriving it from the class.

## Injecting tokens

In a class, use `@Inject(TOKEN)` for non-class tokens:

```typescript
import { Inject } from '@omnitron-dev/titan';

@Service({ name: 'users' })
class UsersService {
  constructor(
    @Inject(LOGGER)     private readonly logger:     ILogger,
    @Inject(VALIDATORS) private readonly validators: IValidator[],
    private readonly db: Database,        // class token; no @Inject needed
  ) {}
}
```

Class tokens do not need `@Inject` — Titan uses TypeScript's emitted
constructor metadata. Symbol-based tokens always need `@Inject`
because TypeScript cannot emit them.

## Token uniqueness

Tokens are matched by `symbol` identity at runtime, and `createToken`
backs that identity with a process-global, name-keyed registry: two
`createToken('Logger')` calls (with no metadata) return the **same**
token. Passing metadata bypasses the cache and yields a fresh,
non-cached token.

For clarity, still declare a shared token once in a shared module and
import it everywhere rather than relying on the by-name registry to
reconcile independently-created tokens.

## Token naming conventions

| Convention        | Example                            |
| ----------------- | ---------------------------------- |
| `SCREAMING_SNAKE` | `LOGGER`, `API_KEY`, `DB_POOL`     |
| Descriptive name  | `'Logger'`, `'DatabasePool'`       |
| Suffix `_TOKEN`   | `LOGGER_TOKEN` (ecosystem modules) |
| Suffix `_OPTIONS` | `LOGGER_OPTIONS`, `CACHE_OPTIONS`  |

Ecosystem modules (`titan-cache`, `titan-redis`, etc.) use the
`_TOKEN` suffix consistently for their exported tokens.

## Anti-patterns

- **String tokens.** Some DI frameworks accept `'logger'` as a
  token; Nexus does not. Use `createToken` for typed identity.
- **Tokens declared in the implementation file.** A consumer that
  injects `LOGGER` should not have to import the implementation
  just to get the token. Place tokens in a separate file (or a
  `*.tokens.ts` barrel) that the implementation also imports.
- **Re-creating tokens at consumption sites.** While `createToken`
  reconciles bare names through its global registry, re-deriving a
  token at each call site is brittle (a typo, or any metadata
  argument, breaks identity). Always import the same token instance.

---

---
id: "titan/errors/classification"
title: "Error Classification"
description: "Categorise errors for telemetry, retry, and alerting decisions."
---

# Error Classification

Different kinds of failure deserve different responses. A 404 is not
a 500. A timeout is not an auth failure. Titan provides classifier
helpers in two places.

## `ErrorCode`-driven classification

The fastest classification uses the error code itself. For a known
`ErrorCode`:

```typescript
import {
  ErrorCode,
  ErrorCategory,
  getErrorCategory,
  isClientError,
  isServerError,
  isRetryableError,
  getErrorName,
  getDefaultMessage,
} from '@omnitron-dev/titan/errors';

const code = ErrorCode.SERVICE_UNAVAILABLE;

getErrorCategory(code);   // ErrorCategory.SERVER
isClientError(code);      // false  — 5xx
isServerError(code);      // true
isRetryableError(code);   // true   — 503 is retryable
getErrorName(code);       // 'SERVICE_UNAVAILABLE'
getDefaultMessage(code);  // 'The service is temporarily unavailable'
```

These helpers operate on the code alone — no need for the full error
instance. Useful in middleware and telemetry where you only have the
code.

## `ErrorCategory` enum

```typescript
import { ErrorCategory } from '@omnitron-dev/titan/errors';

ErrorCategory.CLIENT       // 4xx
ErrorCategory.SERVER       // 5xx
ErrorCategory.AUTH         // 401, 403, 407
ErrorCategory.VALIDATION   // 422
ErrorCategory.RATE_LIMIT   // 429
ErrorCategory.SUCCESS      // 2xx
ErrorCategory.CUSTOM       // 600+
```

The mapping from `ErrorCode` to `ErrorCategory` is the
implementation's responsibility (`getErrorCategory(code)`); the
enum is the vocabulary.

## `isOperationalError(error)` — for retry decisions

Operational errors are those caused by external systems (network,
disk, database, third-party API) — distinct from programming bugs.
They are typically retryable.

```typescript
import { isOperationalError } from '@omnitron-dev/titan/utils';

if (isOperationalError(e)) {
  // Retry, log, alert as transient.
} else {
  // Programming error — log with full stack and fix the bug.
}
```

The classifier recognises:

- Postgres `SQLSTATE` codes in the operational set (connection
  failure, timeout, serialization, etc.).
- Network error codes (`ECONNREFUSED`, `ETIMEDOUT`, `EHOSTUNREACH`,
  `ENETUNREACH`, …).
- Specific framework errors with retry hints.

You can extend the operational set with
`createOperationalErrorRecorder` — see the source in
`utils/error-classification.ts`.

## Putting it together

A typical retry / telemetry pattern:

```typescript
import { isOperationalError } from '@omnitron-dev/titan/utils';
import { TitanError, isRetryableError } from '@omnitron-dev/titan/errors';

async function callWithRetry<T>(fn: () => Promise<T>, maxAttempts = 3): Promise<T> {
  let lastError: unknown;
  for (let attempt = 0; attempt < maxAttempts; attempt++) {
    try {
      return await fn();
    } catch (e) {
      lastError = e;
      const retryable =
        isOperationalError(e) ||
        (e instanceof TitanError && isRetryableError(e.code));
      if (!retryable) throw e;
      await sleep(computeBackoff({ attempt, baseMs: 100, maxMs: 5_000 }));
    }
  }
  throw lastError;
}
```

For ready-made resilience, use `retry()` and `CircuitBreaker` from
`@omnitron-dev/titan/utils/resilience.ts` — see
[Resilience](/docs/titan/resilience/overview).

## Telemetry partitioning

```typescript
catch (e) {
  const code =
    e instanceof TitanError ? e.code : ErrorCode.INTERNAL_ERROR;
  metrics.counter('rpc.errors', {
    code:    String(code),
    category: getErrorCategory(code),
  }).inc();
  throw e;
}
```

Now your dashboards can show "client errors" separately from "server
errors" — the former are usually client bugs; the latter usually
indicate something wrong on your side.

## Alerting policy

```typescript
app.on(ApplicationEvent.Error, ({ error }) => {
  const code = error instanceof TitanError ? error.code : ErrorCode.INTERNAL_ERROR;

  if (isServerError(code) || isRetryableError(code)) {
    oncall.page({ error });
  }
  if (code === ErrorCode.UNAUTHORIZED || code === ErrorCode.FORBIDDEN) {
    securityFeed.report(error);
  }
});
```

## Anti-patterns

- **Retrying based on `instanceof Error`.** Catches everything,
  including programming bugs (which will fail again) and validation
  errors (which won't change). Classify before retrying.
- **String-matching error messages.** Brittle. If the message
  changes (i18n, library upgrade), classification breaks silently.
  Use codes.
- **Treating all 5xx as transient.** A 500 is usually a bug, not a
  transient failure. Be specific about which codes warrant retry
  (`isRetryableError(code)` is the safe default).

→ Back to [Errors Overview](/docs/titan/errors/overview).

---

---
id: "titan/errors/factories"
title: "Error Factories"
description: "The Errors / NetronErrors / HttpErrors / AuthErrors namespaces."
---

# Error Factories

A factory is a function that produces a properly-typed `TitanError`
for a common pattern. Titan groups factories into four namespaces.

## `Errors` — the general namespace

```typescript
import { Errors } from '@omnitron-dev/titan/errors';
```

| Factory                                       | Status | Code                  |
| --------------------------------------------- | ------ | --------------------- |
| `Errors.badRequest(message?, details?)`       | 400    | `BAD_REQUEST`         |
| `Errors.unauthorized(message?, details?)`     | 401    | `UNAUTHORIZED`        |
| `Errors.forbidden(message?, details?)`        | 403    | `FORBIDDEN`           |
| `Errors.notFound(resource, id?)`              | 404    | `NOT_FOUND`           |
| `Errors.conflict(message, details?)`          | 409    | `CONFLICT`            |
| `Errors.alreadyExists(resource, identifier?)` | 409    | `CONFLICT`            |
| `Errors.validation(fields, options?)`         | 422    | `VALIDATION_ERROR`    |
| `Errors.tooManyRequests(retryAfter?)`         | 429    | `TOO_MANY_REQUESTS`   |
| `Errors.timeout(operation, timeoutMs)`        | 408    | `REQUEST_TIMEOUT`     |
| `Errors.internal(message?, cause?)`           | 500    | `INTERNAL_ERROR`      |
| `Errors.notImplemented(feature)`              | 501    | `NOT_IMPLEMENTED`     |
| `Errors.unavailable(service, reason?)`        | 503    | `SERVICE_UNAVAILABLE` |
| `Errors.invalidCredentials(message?)`         | 401    | `UNAUTHORIZED`        |
| `Errors.permissionDenied(permission)`         | 403    | `FORBIDDEN`           |
| `Errors.create(code, message, details?)`      | varies | any                   |

`Errors.validation(fields, options?)` takes an array of
`{ field, message, code? }` (delegating to
`ValidationError.fromFieldErrors`) — not a message string.
`Errors.tooManyRequests` returns a `RateLimitError` subclass.

The catalogue is exhaustive — check the source for the latest
methods. Use `Errors.create(code, message, details)` for codes that
don't have a dedicated factory.

## `NetronErrors` — RPC-specific

```typescript
import { NetronErrors } from '@omnitron-dev/titan/errors';
```

Wraps transport-layer failures. Used internally by Netron; you
usually catch these on the client rather than throwing them on the
server.

| Factory                                                                    | What it represents                           |
| -------------------------------------------------------------------------- | -------------------------------------------- |
| `NetronErrors.serviceNotFound(serviceId)`                                  | No such service                              |
| `NetronErrors.methodNotFound(serviceId, method)`                           | No such method on the service                |
| `NetronErrors.connectionFailed(transport, address, cause?)`                | Could not establish the connection           |
| `NetronErrors.connectionTimeout(transport, address)`                       | Connection attempt timed out                 |
| `NetronErrors.connectionClosed(transport, reason?)`                        | Connection closed (clean shutdown)           |
| `NetronErrors.transportLost(transport, peerId, packetId?, reason?)`        | Connection vanished mid-RPC (result unknown) |
| `NetronErrors.rpcTimeout(serviceId, method, timeoutMs)`                    | RPC call exceeded its deadline               |
| `NetronErrors.invalidRequest(reason, details?)`                            | Malformed RPC request                        |
| `NetronErrors.peerNotFound(peerId)` / `peerDisconnected(...)`              | Peer-level failures                          |
| `NetronErrors.streamClosed(streamId, reason?)` / `streamBackpressure(...)` | Stream-level failures                        |
| `NetronErrors.serializeEncode(value, cause?)` / `serializeDecode(...)`     | (De)serialization failures                   |

Each returns the matching `NetronError` subclass
(`ServiceNotFoundError`, `TransportError`, `TransportLostError`,
`RpcError`, `PeerError`, `StreamError`, `SerializationError`).

## `HttpErrors` — when you need an `HttpError` subclass

```typescript
import { HttpErrors } from '@omnitron-dev/titan/errors';
```

Like `Errors`, but produces `HttpError` instances (or its
subclasses) so `instanceof HttpError` works:

```typescript
import { HttpErrors, HttpError } from '@omnitron-dev/titan/errors';

throw HttpErrors.notFound('User not found', { userId });

catch (e) {
  if (e instanceof HttpError) {
    console.log(e.httpStatus);  // typed as number
  }
}
```

## `AuthErrors` — auth-specific subclasses

```typescript
import { AuthErrors } from '@omnitron-dev/titan/errors';
```

Produces `AuthError`, `PermissionError`, or `RateLimitError`
instances. Use when class-based dispatch matters on the client.

## Conversion helpers

### `toTitanError(error)`

Convert any thrown value (a native `Error`, a string, an object) into
a `TitanError`:

```typescript
try {
  await someThirdPartyCall();
} catch (e) {
  throw toTitanError(e);
}
```

Useful at boundaries where you do not control what's thrown.

### `ensureError(value)`

Coerce a value into a `TitanError`. Same as `toTitanError` but with
a slightly different policy for non-error inputs.

### `assert(condition, errorOrMessage, details?)`

Throw a `TitanError` if the condition is falsy. Type-narrows on the
truthy branch:

```typescript
import { assert, Errors } from '@omnitron-dev/titan/errors';

function process(input: Input | null) {
  assert(input !== null, Errors.badRequest('input required'));
  // input is `Input` here, not `Input | null`.
}
```

### `assertDefined(value, message)`

Specialised `assert` for null/undefined:

```typescript
import { assertDefined } from '@omnitron-dev/titan/errors';

const user = await this.repo.findById(id);
assertDefined(user, 'user not found');
// `user` is `User`, not `User | undefined`.
```

### `assertType(value, predicate, message)`

Specialised `assert` with a type predicate.

## Custom project factories

For domain-specific errors that recur across modules, wrap the
generic factories in a per-domain helper:

```typescript
// users/errors.ts
import { Errors } from '@omnitron-dev/titan/errors';

export const usersErrors = {
  notFound:   (id: string)    => Errors.notFound('user', id),
  emailTaken: (email: string) => Errors.alreadyExists('user', email),
  inactive:   (id: string)    => Errors.conflict('user inactive', { userId: id }),
};
```

```typescript
// users/users.service.ts
import { usersErrors } from './errors.js';

throw usersErrors.notFound(id);
```

Worth doing when the same error appears in multiple methods.

---

---
id: "titan/errors/hierarchy"
title: "Error Hierarchy"
description: "The classes the framework exports — and the codes that classify what they are."
---

# Error Hierarchy

Titan distinguishes between **classes** (the JavaScript types you
can `instanceof`) and **codes** (the `ErrorCode` enum values that
classify the underlying error). Most errors are instances of
`TitanError` with a specific `ErrorCode`; a few have dedicated
subclasses.

## Class hierarchy

```mermaid
classDiagram
  Error <|-- TitanError
  TitanError <|-- HttpError
  HttpError <|-- AuthError
  HttpError <|-- PermissionError
  HttpError <|-- RateLimitError
  TitanError <|-- AggregateError
  TitanError <|-- DomainError

  class TitanError {
    +code: ErrorCode | number
    +category: ErrorCategory
    +httpStatus: number
    +details: object
    +context?: ErrorContext
    +correlationId?: string
    +traceId?: string
  }
  class HttpError { +httpStatus }
  class AuthError { 401 }
  class PermissionError { 403 }
  class RateLimitError { 429 }
  class AggregateError { +errors }
  class DomainError { +domainCode }
```

That's the complete class tree.

Errors like "not found" (404), "conflict" (409), "validation" (422)
are **not** separate classes — they are instances of `TitanError`
with `code: ErrorCode.NOT_FOUND`, `ErrorCode.CONFLICT`,
`ErrorCode.VALIDATION_ERROR`. Use the `Errors` namespace to throw
them; use `e.code === ErrorCode.X` to discriminate on the client.

## The narrow subclasses — when class identity matters

Three subclasses are dedicated because they need extra fields beyond
what `TitanError` carries:

### `AuthError` (401)

Authentication failure. Carries auth-specific context (provider,
reason, hints for the client about how to re-authenticate).

```typescript
import { AuthError } from '@omnitron-dev/titan/errors';

// Constructor is positional: (message?, details?, options?)
throw new AuthError('session expired', { reason: 'token_expired', refreshable: true });

// Or use a static factory:
throw AuthError.tokenExpired();
throw AuthError.bearerTokenRequired('api');
throw AuthError.invalidToken('signature mismatch');
```

Catch by `instanceof AuthError` when you want auth-specific
handling (e.g. redirect to login). The third `options` argument
carries `{ authType, realm }` for the `WWW-Authenticate` header.

### `PermissionError` (403)

Authorisation failure — caller is known but lacks the right scope or
role. Carries the missing capability for diagnostics.

```typescript
import { PermissionError } from '@omnitron-dev/titan/errors';

// Constructor is positional: (message?, details?, options?)
throw new PermissionError('users:write scope required', undefined, {
  requiredPermission: 'users:write',
  userPermissions:    ['users:read'],
});

// Or use the static factory:
throw PermissionError.insufficientPermissions('users:write', ['users:read']);
```

### `RateLimitError` (429)

Throttling. Carries the retry-after hint:

```typescript
import { RateLimitError } from '@omnitron-dev/titan/errors';

// Constructor is positional: (message?, details?, options?)
throw new RateLimitError('too many requests', undefined, { retryAfter: 60 }); // seconds

// Or, most commonly, via the factory:
import { Errors } from '@omnitron-dev/titan/errors';
throw Errors.tooManyRequests(60);
```

`options` accepts `{ limit, remaining, resetTime, retryAfter }`;
`getRateLimitHeaders()` renders the matching `X-RateLimit-*` /
`Retry-After` response headers.

### `AggregateError`

Multiple errors batched into one. Useful when a batch operation
partially fails:

```typescript
import { AggregateError, toTitanError, type TitanError } from '@omnitron-dev/titan/errors';

const errors: TitanError[] = [];
for (const item of batch) {
  try {
    await this.processItem(item);
  } catch (e) {
    errors.push(toTitanError(e));
  }
}
if (errors.length > 0) {
  // Constructor is positional: (errors[], options?). Code is MULTIPLE_ERRORS (600),
  // message is auto-generated ("N errors occurred"); pass { deduplicate: true } to
  // collapse identical code+message pairs.
  throw new AggregateError(errors, { deduplicate: true });
}
```

The batched errors are exposed on `.errors`, and a one-line
`.summary`. `TitanError.aggregate(errors, opts?)` is an equivalent
static shortcut.

### `DomainError`

Base for project-specific error classes. Use the `defineDomainCodes`
helper to declare a typed code namespace, then create errors from it:

```typescript
import { defineDomainCodes, DomainError } from '@omnitron-dev/titan/errors';

// Code config keys are `status` (HTTP) + `message` (+ optional `retryable`).
const BillingCodes = defineDomainCodes('BILLING', {
  CARD_DECLINED:        { status: 402, message: 'Payment declined' },
  INSUFFICIENT_BALANCE: { status: 409, message: 'Insufficient balance' },
  FRAUD_DETECTED:       { status: 403, message: 'Fraudulent activity detected' },
});

// BillingCodes.CARD_DECLINED is the string 'BILLING_CARD_DECLINED'.
// DomainError takes { domainCode, httpStatus?, message, details? }.
throw new DomainError({
  domainCode: BillingCodes.CARD_DECLINED,
  httpStatus: 402,
  message:    'Card 4242 declined',
  details:    { last4: '4242', reason: 'do_not_honor' },
});
```

The framework provides `createDomainErrorFactory`,
`createSimpleDomainFactory`, `isDomainCode`, `isDomainError`,
`getDomainCode` helpers for richer typing — see the source for
current signatures.

## The wide use of `ErrorCode`

For everything *not* in the class tree above, use the appropriate
`ErrorCode`:

| Status | Code                  | Throw via                                                          |
| ------ | --------------------- | ------------------------------------------------------------------ |
| 400    | `BAD_REQUEST`         | `Errors.badRequest(...)`                                           |
| 401    | `UNAUTHORIZED`        | `Errors.unauthorized(...)` or `new AuthError(...)`                 |
| 403    | `FORBIDDEN`           | `Errors.forbidden(...)` or `new PermissionError(...)`              |
| 404    | `NOT_FOUND`           | `Errors.notFound(resource, id?)`                                   |
| 409    | `CONFLICT`            | `Errors.conflict(...)` / `Errors.alreadyExists(...)`               |
| 422    | `VALIDATION_ERROR`    | `Errors.validation(fields, opts?)`                                 |
| 429    | `TOO_MANY_REQUESTS`   | `Errors.tooManyRequests(retryAfter?)` or `new RateLimitError(...)` |
| 500    | `INTERNAL_ERROR`      | `Errors.internal(...)` (rare; framework usually wraps)             |
| 503    | `SERVICE_UNAVAILABLE` | `Errors.unavailable(...)`                                          |

The `NetronErrors` namespace covers transport-specific failures:
`serviceNotFound`, `methodNotFound`, `connectionFailed`,
`connectionTimeout`, `connectionClosed`, `transportLost`,
`peerNotFound`, `rpcTimeout`, `invalidRequest`, `streamClosed`,
`serializeEncode`/`serializeDecode`. They produce the matching
`NetronError` subclass (`ServiceNotFoundError`, `TransportError`,
`RpcError`, …) with the appropriate code.

## Discrimination on the client

Two ways, both supported:

```typescript
import { TitanError, ErrorCode, AuthError, PermissionError, RateLimitError }
  from '@omnitron-dev/titan/errors';

catch (e) {
  // By class (works for the four subclasses above + DomainError)
  if (e instanceof AuthError)        return redirectToLogin();
  if (e instanceof PermissionError)  return showForbiddenScreen();
  if (e instanceof RateLimitError)   return backoff(e.details.retryAfter);

  // By code (for everything else)
  if (e instanceof TitanError) {
    switch (e.code) {
      case ErrorCode.NOT_FOUND:        return show404();
      case ErrorCode.CONFLICT:         return showConflict(e.details);
      case ErrorCode.VALIDATION_ERROR: return showFormErrors(e.details);
    }
  }

  // Unknown — re-throw
  throw e;
}
```

Use class-based dispatch when you have a subclass. Use code-based
dispatch for the long tail of `TitanError` variants.

---

---
id: "titan/errors/overview"
title: "Errors"
description: "A typed error system built on TitanError, HTTP status codes, and ErrorCode-driven factories."
---

# Errors

Errors in Titan are **values** — instances of `TitanError` (or one of
its narrow subclasses) carrying a code, an HTTP-style status, a
message, and a structured `details` object. They serialise across
Netron preserving their identity.

This page is the entry point. Detail in:

- [Hierarchy](/docs/titan/errors/hierarchy) — every error class.
- [Factories](/docs/titan/errors/factories) — the `Errors`, `NetronErrors`,
  `HttpErrors`, `AuthErrors` namespaces.
- [Classification](/docs/titan/errors/classification) — partitioning errors for
  telemetry and resilience.

## The base class — `TitanError`

```typescript
class TitanError extends Error {
  readonly code:        ErrorCode | number;
  readonly category:    ErrorCategory;
  readonly httpStatus:  number;
  readonly details:     Record<string, any>;
  readonly context:     ErrorContext;
  readonly timestamp:   number;
  readonly requestId?:     string;
  readonly correlationId?: string;
  readonly spanId?:     string;
  readonly traceId?:    string;
  // …plus isRetryable() / getRetryStrategy() / withContext() /
  //    withDetails() / toJSON() helpers
}
```

Three reasons it inherits from native `Error`:

- `instanceof Error` keeps third-party tooling happy.
- The `stack` property is preserved.
- Async stack traces in modern Node attach naturally.

Constructed directly when no factory fits:

```typescript
import { TitanError, ErrorCode } from '@omnitron-dev/titan/errors';

throw new TitanError({
  code:    ErrorCode.CONFLICT,
  message: 'Cannot delete user with active orders',
  details: { userId, orderCount: 5 },
});
```

## `ErrorCode` — the universal vocabulary

`ErrorCode` is the enum the framework uses to classify errors. Each
code maps to:

- An HTTP status (400, 401, 404, 422, 429, 500, 503, …).
- An `ErrorCategory` (client error vs server error vs transient).
- A default message.
- A retry policy hint.

```typescript
import { ErrorCode } from '@omnitron-dev/titan/errors';

ErrorCode.BAD_REQUEST          // 400
ErrorCode.UNAUTHORIZED         // 401
ErrorCode.FORBIDDEN            // 403
ErrorCode.NOT_FOUND            // 404
ErrorCode.CONFLICT             // 409
ErrorCode.VALIDATION_ERROR     // 422
ErrorCode.RATE_LIMITED         // 429
ErrorCode.INTERNAL_ERROR       // 500
ErrorCode.SERVICE_UNAVAILABLE  // 503
// …more
```

## Throwing the right error

The idiomatic way to throw is through the `Errors` namespace —
short factory methods that produce a properly-typed `TitanError`:

```typescript
import { Errors } from '@omnitron-dev/titan/errors';

@Public()
async findById(id: string) {
  const user = await this.repo.findById(id);
  if (!user) throw Errors.notFound('user', id);
  return user;
}

@Public()
async create(input: CreateInput) {
  if (await this.repo.findByEmail(input.email)) {
    throw Errors.alreadyExists('user', input.email);
  }
  return this.repo.create(input);
}
```

The `Errors` namespace has methods for every common case:

| Factory                               | Status | Code                  |
| ------------------------------------- | ------ | --------------------- |
| `Errors.badRequest(msg, …)`           | 400    | `BAD_REQUEST`         |
| `Errors.unauthorized(...)`            | 401    | `UNAUTHORIZED`        |
| `Errors.forbidden(...)`               | 403    | `FORBIDDEN`           |
| `Errors.notFound(resource, id?)`      | 404    | `NOT_FOUND`           |
| `Errors.conflict(msg, …)`             | 409    | `CONFLICT`            |
| `Errors.alreadyExists(resource, id?)` | 409    | `CONFLICT`            |
| `Errors.validation(fields, opts?)`    | 422    | `VALIDATION_ERROR`    |
| `Errors.tooManyRequests(retryAfter?)` | 429    | `TOO_MANY_REQUESTS`   |
| `Errors.internal(msg, ...)`           | 500    | `INTERNAL_ERROR`      |
| `Errors.unavailable(...)`             | 503    | `SERVICE_UNAVAILABLE` |

Other namespaces:

- **`NetronErrors`** — transport-specific (`serviceNotFound`,
  `methodNotFound`, `connectionFailed`, `rpcTimeout`, …).
- **`HttpErrors`** — when you want a `HttpError` subclass
  specifically.
- **`AuthErrors`** — for `AuthError` / `PermissionError` /
  `RateLimitError` subclasses.

See [Factories](/docs/titan/errors/factories) for the full surface.

## The HTTP status code is universal

Status codes are the bridge between every transport and every
framework. Even if you serve over WebSocket, your error carries an
HTTP-style status code so the client (or any middleware) can route
on it.

## On the client

Netron preserves the error code, status, and details across the
wire. The client receives a `TitanError` (or matching subclass) with
the same payload:

```typescript
import { TitanError, ErrorCode } from '@omnitron-dev/titan/errors';

try {
  await users.findById('missing');
} catch (e) {
  if (e instanceof TitanError && e.code === ErrorCode.NOT_FOUND) {
    // e.details.resource === 'user', e.details.id === 'missing'
  }
}
```

For class-based discrimination, check against the narrower error
classes (`HttpError`, `AuthError`, `PermissionError`,
`RateLimitError`, `AggregateError`, `DomainError`) — see
[Hierarchy](/docs/titan/errors/hierarchy).

## Internal vs domain errors

Two categories the framework distinguishes:

- **Domain errors** — instances of `TitanError` (or its subclasses)
  thrown from your business logic. Client-visible. The client
  receives the typed error with intact code, status, and details.
- **Internal errors** — native `Error` instances bubbling out of
  your service code. The framework wraps them as
  `INTERNAL_ERROR` (status 500), logs the original with the full
  stack, and sends a generic message to the client. The client
  never sees your stack trace or internal class names.

**Always throw `TitanError` (via a factory) for client-visible
failures.** Plain `throw new Error('user not found')` becomes a 500
to the caller, with no structure they can act on.

## Errors in lifecycle hooks

A lifecycle hook that throws aborts the corresponding phase. The
error is logged with the failing provider's name and re-thrown from
`Application.start()` (for `onInit` / `onStart`) or logged and
isolated (for `onStop` / `onDestroy`).

For retryable conditions in `onStart`, use the `computeBackoff`
helper in `@omnitron-dev/titan/utils/backoff` with your own retry
loop. Do not let a transient failure crash the whole boot.

## Helper functions

- `createError(options)` — build a `TitanError` from
  `{ code, message, details, ... }`.
- `isErrorCode(error, code)` — type-narrow check for a specific
  code.
- `ensureError(value)` — coerce any thrown value into a
  `TitanError`. Useful for adapters that catch unknown values.
- `toTitanError(error)` — convert a native `Error` (or another
  error class) into a `TitanError`.
- `assert(condition, errorOrMessage)` — throw a `TitanError` if the
  condition is falsy. Type-narrows the value.

---

---
id: "titan/identifiers"
title: "Identifiers"
description: "Which ID generator to reach for, and why the default is time-ordered."
---

# Identifiers

`@omnitron-dev/titan/utils` ships eight ID generators. They differ in what they
guarantee, and the differences matter most where the ID becomes a database key.

```typescript
import { generateUuidV7 } from '@omnitron-dev/titan/utils';
```

## `generateUuidV7()` — the default for stored rows

An RFC 9562 UUIDv7: a 48-bit millisecond timestamp, then a 12-bit counter, then
74 bits of randomness. Two properties follow, and both are worth having.

**It sorts by creation time as a string.** `01a07bd1-66a7-7c7a-…` — the leading
hex is the timestamp, so lexical order is chronological order. A B-tree keyed
on it appends to one side instead of writing into random pages, which is the
difference between a hot leaf page and a scattered index. A UUIDv4 primary key
gives up that locality for nothing.

**It is monotonic within a process.** Same millisecond, the counter increments
(4096 per ms). On counter overflow the timestamp advances by one instead of
repeating — so ordering holds even under a burst faster than the clock.

Measured, not inferred: 20 000 consecutive calls come out already sorted, with
no duplicates.

The randomness comes from `node:crypto`, drawn from a 640-byte pool refilled
every \~64 IDs, so a burst pays one syscall rather than sixty-four.

`generateUuid()` is an alias for this one. Prefer the explicit name — `Uuid`
alone reads as "v4" to most people, and the version is the whole point.

## The rest

| Function                     | Shape                       | For                                     |
| ---------------------------- | --------------------------- | --------------------------------------- |
| `generateResolutionId()`     | `res_` + 32 hex (from a v7) | one DI resolution — time-ordered too    |
| `generateConnectionId()`     | `conn_` + 16 hex            | a transport connection                  |
| `generateTraceId()`          | 32 hex                      | W3C Trace Context trace-id              |
| `generateSpanId()`           | 16 hex                      | W3C Trace Context span-id               |
| `generateLockValue()`        | 32 hex                      | the fencing value of a distributed lock |
| `generatePrefixedId(prefix)` | `<prefix>_` + 16 hex        | anything else that wants a readable tag |

The trace and span widths are not a style choice: W3C Trace Context fixes them
at 16 and 8 bytes, so a value of any other length is rejected by every consumer
of the header.

`generateLockValue()` is random rather than time-ordered on purpose. A lock
value is compared for equality when releasing — the point is that nobody can
guess or reconstruct it, and ordering would be an invitation to try.

## Choosing

Use `generateUuidV7()` for anything that lands in a table. Use a prefixed hex ID
for things that live in memory or in a log line, where a human reading it
benefits from seeing what kind of thing it is. Use the trace and span helpers
only for their protocol.

For URL-safe IDs shorter than a UUID — a share link, a public handle — see
[`@omnitron-dev/cuid`](/docs/utilities/cuid): 16 base36 characters against a
UUID's 36, with no dashes to escape or lose in a double-click.

What you give up is the ordering, and it is worth being exact about why. A cuid
DOES embed a base36 timestamp — but behind a random leading letter, so
consecutive IDs do not sort chronologically. Measured: 5 000 in a row come out
unsorted, while 20 000 UUIDv7s come out already sorted.

You also give up the *format*: a `uuid` column in Postgres, and every tool that
recognises the shape.

So: cuid where the ID is read or typed by a person, `generateUuidV7()` where it
is a key.

---

---
id: "titan/logging/child-loggers"
title: "Child Loggers"
description: "Bind context to a logger instance — service, request, trace."
---

# Child Loggers

A child logger is a logger with extra context bound to it. Every log
line written through the child carries the bound fields automatically.

This is the right way to add per-call or per-request fields without
repeating them at every call site.

## Creating a child

```typescript
const child = this.logger.child({ requestId: 'r_42', userId: 'u_91' });

child.info('processing');
// → {"level":"info","requestId":"r_42","userId":"u_91","msg":"processing", …}

// Object-first when you have per-call fields:
child.error({ reason: 'db timeout' }, 'oops');
// → {"level":"error","requestId":"r_42","userId":"u_91","reason":"db timeout","msg":"oops", …}
```

The child carries the bound fields forever. Pass it down to
helpers, into other services, into queue handlers — every log line
through it has the same context.

## Per-service binding (automatic)

Every `LoggerService` injected into a `@Service` class is
automatically a child bound to `service: <ClassName>`:

```typescript
@Service('users@1.0.0')
class UsersService {
  constructor(private readonly logger: LoggerService) {}
  // logger.info('hello') → {"service":"UsersService","msg":"hello"}
}
```

You don't need to set this; the framework wires it.

## Per-call binding via context

For Netron calls, a request-scoped logger is available in the
`NetronContext`. It is already bound to:

| Field       | From                                |
| ----------- | ----------------------------------- |
| `service`   | The service identifier              |
| `method`    | The method being called             |
| `traceId`   | The trace context                   |
| `spanId`    | The trace context                   |
| `requestId` | Generated per call                  |
| `userId`    | The auth context (if authenticated) |

Use it via `@Context()`:

```typescript
@Public()
async findById(id: string, @Context() ctx: NetronContext) {
  ctx.logger.debug({ id }, 'looking up user');
  return this.repo.findById(id);
}
```

Every log line from this method carries the request context — no
manual passing.

## Across async boundaries

Child loggers cross async boundaries by *passing the reference*. They
do not propagate via `AsyncLocalStorage` automatically (unlike trace
context). If you spawn a background task, pass the logger:

```typescript
@Public()
async findById(id: string, @Context() ctx: NetronContext) {
  const logger = ctx.logger;        // capture
  void this.warmCache(id, logger);  // pass into the task
  return this.repo.findById(id);
}

private async warmCache(id: string, logger: ILogger) {
  logger.debug({ id }, 'warming cache');
  // …
}
```

For propagation through deep call chains where passing is awkward,
use the trace integration — log lines automatically carry `traceId`
and `spanId` from the active trace context.

## Don't pass the parent logger

A common mistake:

```typescript
// Wrong — uses the unbound parent.
@Public()
async findById(id: string) {
  this.logger.info({ id }, 'looking up');   // no requestId, no traceId
}
```

Inject `@Context()` and use `ctx.logger` instead. The parent
`this.logger` has no per-call context.

## Performance

Creating a child is cheap (it's a shallow clone of the parent's
bound fields). Logging through a child is the same cost as logging
through the parent.

A common pattern in tight loops:

```typescript
// Bind once outside the loop.
const itemLogger = ctx.logger.child({ batch: 'b_42' });

for (const item of items) {
  itemLogger.debug({ id: item.id }, 'processing item');
}
```

The child is created once; each log is normal cost.

## Anti-patterns

- **Recreating the same child per call.** If you find yourself
  writing `this.logger.child({ service: 'X' })` everywhere, that's
  what `@Context()` already gives you.
- **Mutating bound fields after binding.** Child loggers' fields
  are frozen at creation. To change context, create a new child.
- **Logging via `console.log`.** Bypasses the framework — no
  context, no level, no transport routing. Always use the
  injected logger.

→ Back to [Logging Overview](/docs/titan/logging/overview).

---

---
id: "titan/logging/overview"
title: "Logging"
description: "Structured pino-based logging with child loggers, transports, and processors."
---

# Logging

`LoggerModule` is one of the two core modules auto-loaded with every
Titan app (the other is `ConfigModule`). Disable with
`disableCoreModules: true` if you need to provide your own.

```typescript
import {
  LoggerModule,
  LoggerService,
  ConsoleTransport,
  RedactionProcessor,
  Logger,
  Log,
  Monitor,
  LOGGER_TOKEN,
  LOGGER_SERVICE_TOKEN,
  type ILogger,
  type ILoggerOptions,
  type ILoggerModuleOptions,
  type ITransport,
  type ILogProcessor,
} from '@omnitron-dev/titan/module/logger';
```

## The `ILogger` interface

```typescript
interface ILogger {
  // Six levels (Pino-compatible). Each is overloaded: object-first
  // (Pino v9 style) or message-first.
  trace(obj: object, msg?: string, ...args: any[]): void;
  trace(msg: string, ...args: any[]): void;
  debug(obj: object, msg?: string, ...args: any[]): void;
  debug(msg: string, ...args: any[]): void;
  info(obj: object, msg?: string, ...args: any[]): void;
  info(msg: string, ...args: any[]): void;
  warn(obj: object, msg?: string, ...args: any[]): void;
  warn(msg: string, ...args: any[]): void;
  error(obj: object, msg?: string, ...args: any[]): void;
  error(msg: string, ...args: any[]): void;
  fatal(obj: object, msg?: string, ...args: any[]): void;
  fatal(msg: string, ...args: any[]): void;

  // Child logger with bound context
  child(bindings: object): ILogger;

  // Timing + level control
  time(label?: string): () => void;       // returns a stop() that logs elapsed ms
  isLevelEnabled(level: LogLevel): boolean;
  setLevel(level: LogLevel): void;
  getLevel(): LogLevel;
}
```

> **Six levels.** `ILogger` exposes the full Pino set —
> `trace`, `debug`, `info`, `warn`, `error`, `fatal`. `LogLevel` is
> `'fatal' | 'error' | 'warn' | 'info' | 'debug' | 'trace' | 'silent'`.

> **Object-first signatures.** Following Pino v9, pass the structured
> data **first**, then the message: `logger.info({ userId }, 'login')`.
> A bare-string call also works: `logger.info('login')`.

## The minimal usage

```typescript
import { LOGGER_TOKEN, type ILogger } from '@omnitron-dev/titan/module/logger';

@Service({ name: 'users' })
class UsersService {
  // LOGGER_TOKEN yields an ILogger — the thing with the level methods.
  // `LoggerService` is the module facade and has no `.info()`; from it you
  // reach a logger through `.logger` or `.child({ … })`.
  constructor(@Inject(LOGGER_TOKEN) private readonly logger: ILogger) {}

  @Public()
  async findById(id: string) {
    this.logger.info({ id }, 'findById');   // object first, message second
    return this.repo.findById(id);
  }
}
```

Output (JSON, with pino formatting):

```json
{"level":"info","time":"2026-05-15T20:00:00.000Z","service":"users","msg":"findById","id":"u_42"}
```

## Why a structured logger

Three reasons:

1. **Searchability.** A log line is a JSON object with named
   fields. `level=error service=users method=findById userId=u_42`
   is queryable; a free-text string is not.
2. **Context propagation.** A child logger inherits its parent's
   context. A request-scoped logger automatically tags every log
   with the trace ID, the user ID, the call ID.
3. **Performance.** Pino is asynchronous and avoids JSON
   serialisation in the hot path.

## Setting the level

```typescript
LoggerModule.forRoot({
  level: process.env.LOG_LEVEL ?? 'info',
})
```

Per-context level overrides through child loggers (`child` accepts
an options bag that includes `level` on pino).

## Output format (JSON, or pretty in dev)

By default the logger writes **structured JSON**, one record per line —
what log shippers (Loki, ELK, Datadog) expect. Keep this in production.

For human-friendly local output, set `prettyPrint` (or the `pretty`
alias, or the `logger.prettyPrint` ConfigService key). The logger then
uses [`pino-pretty`](https://github.com/pinojs/pino-pretty) — a declared
dependency — as its stdout destination, producing colorised,
human-readable lines. pino-pretty writes to fd 1 directly:

```typescript
LoggerModule.forRoot({
  prettyPrint: true,   // colorised human-readable output for dev
})
```

```yaml
# or via ConfigService
logger:
  prettyPrint: true
```

`LoggerService` chooses pretty output when `prettyPrint === true`, when
the `pretty` alias is true, or when the environment is `development` and
neither flag is explicitly `false`; otherwise it stays on structured
JSON (the `prettyPrint` branch of `LoggerService.initialize`,
`packages/titan/src/modules/logger/logger.service.ts`).
Leave it off (JSON) for production so log shippers can parse each line.

You can still pipe JSON output through `pino-pretty` externally if you
prefer (`node dist/main.js | npx pino-pretty`), since the logger's JSON
is `pino-pretty`-compatible.

## Decorators

```typescript
import { Logger, Log, Monitor } from '@omnitron-dev/titan/module/logger';

@Service({ name: 'users' })
class UsersService {
  @Logger() private readonly logger!: ILogger;          // property injection

  @Public()
  @Log()                                                 // auto-log entry/exit
  async findById(id: string) { /* … */ }

  @Public()
  @Monitor()                                             // track performance
  async heavyMethod(input: Input) { /* … */ }
}
```

## Integration with tracing

If a trace context is active, every log line in the same async
scope can carry `traceId` and `spanId`. Wire this up via a
`ILogProcessor` that reads `currentTrace()`:

```typescript
import { currentTrace } from '@omnitron-dev/titan/tracing';
import type { ILogProcessor } from '@omnitron-dev/titan/module/logger';

const TraceContextProcessor: ILogProcessor = {
  process(record) {
    const trace = currentTrace();
    if (trace) {
      record.traceId = trace.traceId;
      record.spanId  = trace.spanId;
    }
    return record;
  },
};

LoggerModule.forRoot({ processors: [TraceContextProcessor] });
```

Correlate logs across services by `traceId` in your log aggregator.

## Helpers

- `createNullLogger()` — returns an `ILogger` that discards
  everything. Useful in tests.
- `isLogger(value)` — type guard.

## Tokens

| Token                     | Purpose                     |
| ------------------------- | --------------------------- |
| `LOGGER_TOKEN`            | The default `ILogger`       |
| `LOGGER_SERVICE_TOKEN`    | The `LoggerService` wrapper |
| `LOGGER_OPTIONS_TOKEN`    | Resolved options bundle     |
| `LOGGER_TRANSPORTS_TOKEN` | Registered transports       |
| `LOGGER_PROCESSORS_TOKEN` | Registered processors       |

## Read on

- [Transports](/docs/titan/logging/transports) — pluggable destinations.
- [Processors](/docs/titan/logging/processors) — transform / filter pipeline.
- [Child Loggers](/docs/titan/logging/child-loggers) — bound context.

---

---
id: "titan/logging/processors"
title: "Log Processors"
description: "The ILogProcessor pipeline (transform / drop, child-inherited) and how it relates to Pino's native redact."
---

# Log Processors

Read this first: **processors are a functional, hot-path pipeline.**
Every registered `ILogProcessor` runs on every log line — through
Pino's `logMethod` hook, which is inherited by the root logger **and**
all child loggers. Each processor either **transforms** the record
(returns the modified object) or **drops** the log entirely (returns
`null`/`undefined`). Redaction has two working paths: Pino's native
`redact` option (recommended for simple path redaction) and a
`RedactionProcessor` (for arbitrary transform/drop logic).

## How the pipeline runs

`LoggerService` installs the pipeline as Pino's `logMethod` hook when it
builds the root logger
(the `logMethod` hook in `LoggerService.initialize`,
`packages/titan/src/modules/logger/logger.service.ts`). Because
the hook lives on the root logger, **every child logger inherits it** —
processors apply everywhere, not just on the root.

For each log call the service builds the record the processors see,
runs each processor in order, and then either drops the log or forwards
the (possibly transformed) fields back to Pino
(`LoggerService.runProcessors`). The record shape is:

```typescript
{
  level: '<string label>',   // e.g. 'info' — pino-managed, re-emitted by pino
  time:  1715800000000,      // ms epoch — pino-managed, re-emitted by pino
  ...childBindings,          // bindings from base / setContext / child()
  ...perCallFields,          // the object you passed: logger.info({ id }, …)
  msg:   'the message',      // present when a message was passed
  // err: <Error>            // present only for a top-level Error argument
}
```

After the pipeline, `level`/`time`/`msg` are stripped before the call
because they are **pino-managed** — pino re-emits its own values
(in `runProcessors`). A top-level `Error` is preserved as the
original object so Pino's native error serialiser still applies
(also in `runProcessors`).

**Fast path.** With no processors registered the hook is a single
length check and a passthrough (the `processors.length === 0` early return in the hook), so the
common case keeps Pino's native overhead — zero added cost.

**Dynamic registration.** The processor list is read **by reference on
every log**, so `LoggerService.addProcessor()` after init takes effect
on the next log line (`LoggerService.addProcessor`, with the list consumed inside
`runProcessors`). You can also register at startup via
`LoggerModule.forRoot({ processors })`.

```typescript
import type { ILogProcessor } from '@omnitron-dev/titan/module/logger';

// Transform: enrich every line with a static field
const RegionProcessor: ILogProcessor = {
  process(record) {
    record.region = process.env.REGION ?? 'unknown';
    return record;            // return the (modified) record to keep it
  },
};

// Drop: suppress health-check noise entirely
const DropHealthChecks: ILogProcessor = {
  process(record) {
    if (record.path === '/healthz') return null;  // null/undefined → log dropped
    return record;
  },
};

LoggerModule.forRoot({ processors: [RegionProcessor, DropHealthChecks] });
```

## The `ILogProcessor` interface

```typescript
interface ILogProcessor {
  process(log: any): any | null;
}
```

A processor is a single `process(log)` method that returns a
transformed log object, or `null`/`undefined` to drop the record
(the `ILogProcessor` interface in `logger.types.ts`). Processors run in registration order; the
first one to return `null`/`undefined` short-circuits the rest and
emits nothing (`runProcessors`).

## Redaction

Redaction has two working paths — pick by need:

- **`redact` option (recommended for simple path redaction).** Pino's
  own redaction, configured on the module option, runs inside Pino's
  serialisation on the hot path. It is pino-optimised and supports
  `*` wildcards and bracket notation. See below.
- **A processor (for arbitrary transform/drop logic).** Use the
  `ILogProcessor` pipeline when you need conditional redaction,
  cross-field logic, computed values, or dropping. The bundled
  `RedactionProcessor` (below) is the simple case.

### `redact` (Pino native)

Set the `redact` module option. `LoggerService` forwards it straight to
Pino when it builds the root logger (`redact` in `LoggerService.initialize`):

```typescript
LoggerModule.forRoot({
  redact: ['password', 'token', 'apiKey', 'headers.authorization', '*.creditCard'],
})
```

This is Pino's own redaction, so:

- It uses **Pino path syntax**, including `*` wildcards and bracket
  notation (`headers["x-api-key"]`).
- Matched values are replaced with `[Redacted]` by default; Pino also
  supports censoring to a custom value or removing keys entirely via
  the object form (`{ paths, censor, remove }`).
- It runs inside Pino's serialisation, on the actual hot path, for
  every log line — including child loggers.

You can also supply the list through `ConfigService` under the
`logger.redact` key, which `LoggerService` reads at startup
(in `LoggerService.getConfiguration`). **Set the list per app** — Titan redacts
nothing by default (`redact: config.redact || []` in
`LoggerService.initialize`), because what counts as sensitive is
project-specific.

> **Redaction is defence-in-depth, not a primary control.** It catches
> a field that slips through; it is not a licence to log secrets.
> Don't put credentials into a log call in the first place.

### `RedactionProcessor`

`RedactionProcessor` is exported and implements `ILogProcessor`. Its
constructor takes a **positional** `string[]` of dotted paths (not an
options object), and `process()` walks each path and replaces the
matched leaf with the literal string `'[REDACTED]'`. Paths are literal
dotted segments only — **no `*` wildcard support** in this class
(source: `RedactionProcessor` in `logger.module.ts`):

```typescript
import { RedactionProcessor } from '@omnitron-dev/titan/module/logger';

const p = new RedactionProcessor(['password', 'token', 'headers.authorization']);
p.process({ password: 'hunter2' }); // → { password: '[REDACTED]' }

// Registered as a processor, it now redacts every line on the log path:
LoggerModule.forRoot({ processors: [p] });
```

Registered via `forRoot({ processors })`, it **does** redact your logs —
it runs on the hot path like any processor. Prefer Pino's `redact`
option for plain path redaction (it is pino-optimised and supports
wildcards / custom censor); reach for the processor when you need the
processor pipeline's extra power (conditional logic, drop).

## Other shaping mechanisms

- **To enrich** every line with static fields, you can also use `base`
  (`forRoot({ base: { region, version } })`) or `setContext(...)`,
  both of which flow into Pino's bindings — lower-ceremony than a
  processor when the field is constant.
- **To add per-request context** (trace ID, user ID), bind a **child
  logger** — see [Child Loggers](/docs/titan/logging/child-loggers). This is the
  idiomatic way to attach per-scope fields.
- **To filter by level**, set the level (`forRoot({ level })` or
  `setLevel`); Pino drops below-threshold records before serialising —
  cheaper than a dropping processor for a pure level cut.

## Pretty output

Human-readable dev output is a Pino concern, not a processor one. See
[Logging Overview](/docs/titan/logging/overview) for the module's pretty-print handling
(via pino-pretty) and how JSON-vs-pretty is selected.

## Anti-patterns

- **A dropping processor where a level cut would do.** To suppress a
  whole level, set `level` — Pino drops below-threshold records before
  serialising, cheaper than running a processor that returns `null`.
- **Heavy work in `process()`.** It runs on every log line (after the
  level filter). Keep it cheap; offload expensive shipping to a
  [transport](/docs/titan/logging/transports), which runs off the hot path.
- **Redaction as the only privacy guard.** It is defence-in-depth.
  Don't log secrets to begin with.
- **A processor for a constant field.** A static value belongs on
  `base` or `setContext`; a per-scope value belongs on a child logger,
  not a global transform.

---

---
id: "titan/logging/transports"
title: "Log Transports"
description: "Where log records go — async stdout, extra destination streams, and the async ITransport sink."
---

# Log Transports

Titan's logger is Pino. Every log line goes to stdout (async, or
through `multistream` when there are extra sinks). On top of that you
have **two** ways to fan records elsewhere:

- **`destinations`** — raw Pino streams. Each receives the serialised
  JSON line as a `Buffer`/chunk and is plugged straight into Pino's
  `multistream`.
- **`transports`** (`ITransport`) — objects whose `write(log)` receives
  each **fully-serialised record parsed back to an object**, delivered
  **off** Pino's hot path and **isolated** in try/catch. `flush()` is
  awaited by `LoggerService.flush()`.

Both are functional. This page covers the write path, then `destinations`,
then `transports`, and finally when to use which.

## The real write path: async Pino streams

When the module starts (`LoggerModule.forRoot(...)`), `LoggerService`
builds the root Pino logger in one of these shapes:

- **No `destinations`, no `transports`, JSON output** — Pino writes to
  an **async stdout destination** (`pino.destination({ dest: 1, sync:
  false })`). Log calls do not block the event loop on the `write()`
  syscall. This is the default for almost every production app.
- **Pretty output, no extra sinks** — Pino writes through a `pino-pretty`
  stream straight to stdout (see [Output format](/docs/titan/logging/overview)).
- **`destinations` and/or `transports` configured** — Pino writes
  through `multistream` over the stdout stream **plus** every
  `destinations` stream **plus** (when any transport is registered) a
  single fan-out stream that drives the `ITransport` sinks. Each
  user-supplied `destinations` stream is wrapped in an async forwarder
  so a slow or fully-synchronous consumer can't stall Pino's hot path
  or its sibling streams.

Source: the stream-selection branch in `LoggerService.initialize`
(`packages/titan/src/modules/logger/logger.service.ts`) and the
`wrapAsyncStream` forwarder in the same file.

### Fanning out to extra streams with `destinations`

`destinations` is the raw-stream way to send the same JSON log lines to
somewhere besides stdout — a file, a socket, a second sink. Each entry
is either a raw `WritableStream` or `{ stream, level }`, and it receives
Pino-formatted JSON as a chunk:

```typescript
import { createWriteStream } from 'node:fs';

LoggerModule.forRoot({
  destinations: [
    { stream: createWriteStream('/var/log/myapp/app.log'), level: 'info' },
  ],
})
```

The stdout stream is always included as the first stream; your entries
are added alongside it. Use `pino.destination()` or
`fs.createWriteStream()` for file targets. The option is typed on
`ILoggerModuleOptions.destinations` (`ILoggerModuleOptions.destinations` in `logger.types.ts`).

### Async stdout and shutdown

When stdout is the async destination, buffered lines can be lost if the
process is hard-killed. To recover what it can on a clean exit,
`LoggerService` installs a process-wide flush hook on `beforeExit`,
`SIGTERM`, and `SIGINT` that calls `flushSync()` on every registered
async destination (`LoggerService.installFlushOnExit`). `SIGKILL` still loses
unflushed data — there is no user-space recovery for that.

The flush hook is installed **once per process** and tracks every
`LoggerService` instance's destination in a shared `Set`, so repeated
initialisations (tests, hot-reload) neither skip flushing nor
double-flush.

## The `ITransport` interface

```typescript
interface ITransport {
  name: string;
  write(log: any): void | Promise<void>;
  flush?(): Promise<void>;
}
```

Only `name` and `write` are required; `flush` is optional. There is no
`dispose()` method, no `filter` hook, and no `bufferSize`/`flushEveryMs`
options. (Definition: the `ITransport` interface in `logger.types.ts`.)

### How `write()` is driven

A registered transport's `write()` **is called for every log line**,
with these guarantees (`LoggerService.createTransportFanout`):

- **It receives the fully-serialised record, parsed back to an object.**
  The fan-out stream sits in Pino's `multistream`, so it gets exactly
  what Pino emits — JSON, which the service `JSON.parse`s before calling
  `write(record)`. (If a line somehow isn't valid JSON, the raw string
  is passed instead.)
- **Post-processor, post-level-filter.** Records arrive after the
  [processor pipeline](/docs/titan/logging/processors) has transformed/dropped them,
  and only for logs that passed the level filter.
- **Off the hot path.** Delivery is deferred a macrotask
  (`setImmediate`), so a slow or synchronous `transport.write` can
  never block Pino's hot path (the `setImmediate` in `createTransportFanout`).
- **Error-isolated.** Each transport runs in its own try/catch, and
  rejected promises are swallowed — a throwing or rejecting transport
  can never break logging or affect sibling transports
  (the per-transport try/catch in `createTransportFanout`).

```typescript
import type { ITransport } from '@omnitron-dev/titan/module/logger';

class HttpSinkTransport implements ITransport {
  name = 'http-sink';
  private buf: any[] = [];
  constructor(private readonly url: string) {}

  write(record: any) {
    // record is a parsed object: { level, time, msg, ...fields }
    this.buf.push(record);
    if (this.buf.length >= 100) return this.drain();
  }

  async flush() { await this.drain(); }   // awaited by LoggerService.flush()

  private async drain() {
    if (this.buf.length === 0) return;
    const batch = this.buf.splice(0);
    await fetch(this.url, { method: 'POST', body: JSON.stringify(batch) });
  }
}

LoggerModule.forRoot({
  transports: [new HttpSinkTransport('https://logs.example/ingest')],
})
```

### `flush()`

`LoggerService.flush()` awaits `flush()` across every registered
transport that defines one (`LoggerService.flush`). Call it on
shutdown to drain anything a transport has buffered.

### Caveat: configure transports at `forRoot`

The fan-out stream that drives transports is created **at init**, and
only when at least one transport was registered at that point — it is
the multistream branch that wires it in (the multistream branch of `LoggerService.initialize`).
Consequences:

- **Configure transports at `forRoot`** (or via the constructor) for
  guaranteed delivery. The live `this.transports` list is read on every
  write, so transports added at init all receive records.
- **`addTransport()` after init** is honoured **only if the logger was
  initialised with at least one transport** (that's what created the
  fan-out stream; the live list it reads then includes the new one). If
  the logger started with **zero** transports, there is no fan-out
  stream, so a later `addTransport()` will collect the transport (and
  its `flush()` still runs) but its `write()` won't receive records.
  When in doubt, register transports up front.

## Built-in transport

### `ConsoleTransport`

The one bundled example transport. Its constructor takes an
**optional** `ILogger` (positional — there is no options object, and no
`pretty`/`stderr`/`filter` option), and `write(log)` forwards to
`logger?.info({ log }, 'Console transport log')`. With no logger it is a
no-op. (Source: `ConsoleTransport` in `logger.module.ts`.)

```typescript
import { ConsoleTransport } from '@omnitron-dev/titan/module/logger';

// Registered at forRoot, its write() receives every record off the hot path:
LoggerModule.forRoot({ transports: [new ConsoleTransport(myLogger)] });
```

It is a reference implementation more than a production sink — for
routine output you don't need a transport at all (stdout is the default,
and `destinations` covers extra raw streams). There is **no
`FileTransport`** and no built-in rotation: use `destinations` with a
rotating write stream, or (the common production choice) write JSON to
stdout and let an OS-level shipper handle files and rotation.

## `destinations` vs `transports`: which to use

Both fan records out; they differ in what they receive and how they run:

|                | `destinations`                                  | `transports` (`ITransport`)                                  |
| -------------- | ----------------------------------------------- | ------------------------------------------------------------ |
| Receives       | raw serialised JSON chunk (`Buffer`)            | parsed record **object**                                     |
| Wiring         | a Pino `multistream` branch                     | a fan-out stream → `write(record)`                           |
| Scheduling     | Pino's multistream (user streams async-wrapped) | always deferred a macrotask                                  |
| Error handling | a throwing `_write` surfaces as a stream error  | isolated per-transport try/catch                             |
| Lifecycle      | `flushSync()` on exit (async stdout only)       | `flush()` awaited by `LoggerService.flush()`                 |
| Dynamic add    | n/a (set at `forRoot`)                          | `addTransport()` works **iff** ≥1 transport at init          |
| Best for       | high-throughput raw byte sinks (files, sockets) | structured sinks needing the parsed object + error isolation |

Rule of thumb: reach for **`destinations`** when you want the raw JSON
bytes on a stream (file rotation, a socket, an OS shipper); reach for
**`transports`** when you want the parsed record object, want delivery
isolated from the hot path and from sibling sinks, or need a `flush()`
lifecycle hook.

## Anti-patterns

- **Adding the first transport after init.** With zero transports at
  `forRoot`, no fan-out stream exists, so a later `addTransport()`
  won't deliver `write()`. Register at least one transport up front.
- **Heavy synchronous work in `write()`.** Delivery is off the hot path,
  but a transport that blocks still ties up the event loop on its
  macrotask. Batch and ship asynchronously (see `flush()`).
- **Synchronous file writes in a `destinations` stream.** They block
  the event loop. Prefer an async stream plus an OS-level log shipper.
- **Writing structured records as text.** A sink that does `String(log)`
  defeats structured logging. Keep the JSON / object intact.
- **Relying on a built-in `FileTransport` / rotation.** There isn't one.
  Use `destinations` with a rotating stream, or ship stdout.

---

---
id: "titan/migrations/auth-1-to-2"
title: "Auth v1 → v2 Migration"
---

# Migration: Auth v1 → v2

> **Design RFC**
>
> This migration plan accompanies the planned authorisation stack documented under [Authentication & Authorisation](/docs/auth). The unified permission engine, the `pv` claim, and the RLS bridge are target shapes — not changes the current Omnitron build has actually rolled out. Treat the steps below as the upgrade contract you'd hand to a platform team adopting that stack.

The Auth surface gained a unified permission engine, decorator
façades, per-user overrides, live `permVer` propagation, and a
stricter RLS tiering. The wire-level JWT shape grew one new
claim (`pv`); everything else is back-compat or codemod-able.

This page lists the call-site changes for an existing app
upgrading from the pre-unification surface.

## TL;DR

| Before                                                                            | After                                                                                                            |
| --------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------- |
| `@Public({auth:{roles:['admin','superadmin'], permissions:['admin.users.ban']}})` | `@RequirePlatformPermission('admin.users.ban', { roles: ['admin','superadmin'] })`                               |
| `@Public({auth:{permissions:['admin.users.list']}})`                              | `@RequirePlatformPermission('admin.users.list')`                                                                 |
| Service-method `this.requirePermission(actor, orgId, 'shops.create')`             | (gradually) `@RequireOrgPermission('org.shops.create')` on the RPC; keep service-layer guard as defence in depth |
| Frontend `user.platformRole === 'admin'`                                          | `usePermission('admin.x.y')` / `<PermissionGuard permission="admin.x.y">`                                        |
| Frontend `<RoleGuard roles={['admin','superadmin']}>`                             | `<PermissionGuard anyOf={['admin.users.list', 'admin.audit.view', …]}>`                                          |
| Inline `ADMIN_ROLES = ['admin','superadmin']` in three files                      | `import { ADMIN_ROLES, isAdminRole } from 'src/auth/admin-roles'`                                                |

## Permission strings

Permission keys did not change. The set GREW (new entries:
`admin.users.permissions`, `admin.orgs.*`,
`admin.security.impersonate`, the entire `org.*` registry); old
keys still work. The matcher is now consistent across all
scopes — wildcards (`admin.*`) and hierarchical prefix grants
(`admin.users` grants `admin.users.ban`) work identically on
platform and org.

## Decorator façades

The `@RequirePlatformPermission` and `@RequireOrgPermission`
decorators compile to:

| Decorator                                       | Underlying                                                    |
| ----------------------------------------------- | ------------------------------------------------------------- |
| `@RequirePlatformPermission(p)`                 | `@Public({auth:{permissions:[p]}})`                           |
| `@RequirePlatformPermission(p, { roles })`      | `@Public({auth:{permissions:[p], roles}})`                    |
| `@RequireOrgPermission(p)`                      | `@Public({auth:{policies:[org-permission:p:default]}})`       |
| `@RequireOrgPermission(p, { argIndex, field })` | `@Public({auth:{policies:[org-permission:p:argIdx.field]}})`  |
| `@RequireAnyOrgPermission([a, b])`              | `@Public({auth:{policies:[org-permission:any:a,b:default]}})` |
| `@RequirePlatformOrOrgPermission(plat, org)`    | OR composite (platform                                        |   | org) |

Migration is mechanical. The pre-built codemod (in
`scripts/migrate-auth-decorators.mjs`) handles both shapes —
or just grep for `@Public({ auth: { ` and rewrite by hand;
the platform variant is by far the most common.

## `permVer` JWT claim

The access token gained one claim:

```ts
{
  sub: '<userId>',
  roles: [...],
  permissions: [...],
  pv: 7,                  // ← NEW. Snapshot of user's permission-version counter at mint time.
  // ... existing claims
}
```

Verifiers that **don't** check `pv` continue to work; the
field is ignored. Verifiers that consume the
`createRlsInvocationWrapper` from `@yourorg/auth-utils` get
`pv` enforcement out of the box once they pass a `permVerRedis`
client.

A client-side handler should treat
`PERMISSION_VERSION_STALE` (401) the same as `TOKEN_EXPIRED` —
trigger the refresh flow, retry the call. The canonical
implementation adds three lines to the existing refresh-on-401
logic in your Netron RPC wrapper.

## Per-user permission overrides

New surface; existing apps have no migration to do — adopting
overrides is purely additive. The relevant pieces:

- DB: `users.customPermissions: jsonb` (migration 045).
- Service: `UserPermissionService.{grant, revoke, replaceAll, list}`.
- RPC: `adminListUserPermissions` / `adminGrantUserPermission`
  / `adminRevokeUserPermission` / `adminSetUserPermissions`
  on `UsersService`, gated on `admin.users.permissions`.
- Identical org-scope shape on `employees.customPermissions`
  via `EmployeePermissionService`.

See [Per-user overrides](/docs/auth/overrides) for the
runtime story.

## RLS plugin split

The pre-fix single `rlsPlugin({ allowUnfilteredQueries: true,
requireContext: false })` is replaced by two plugin instances:

```ts
const strict = rlsPlugin({ schema, bypassRoles: ['superadmin'], requireContext: true,  allowUnfilteredQueries: false });
const permissive = rlsPlugin({ schema, bypassRoles: ['superadmin'], requireContext: false, allowUnfilteredQueries: true });

for (const t of STRICT_RLS_TABLES) registerTablePlugins(t, [...existing, strict]);
for (const t of PERMISSIVE_RLS_TABLES) registerTablePlugins(t, [...existing, permissive]);
```

If your app has background workers or scripts that hit STRICT
tables (`audit_logs`, `platform_roles`, `users`, `user_sessions`,
`organization_roles`) without an authenticated context, they
MUST enter a `'system'` role context — the previous silent
bypass via missing context is gone.

See [RLS bridge](/docs/auth/rls-bridge) for the bridge
contract and the `'system'` escape hatch.

## Frontend

The presentational gates split into two primitives:

```tsx
// Capability-based (preferred for new surfaces)
<PermissionGuard permission="admin.users.ban">…</PermissionGuard>

// Any-of (for routes accessible to multiple roles via different perms)
<PermissionGuard anyOf={['admin.users.list', 'admin.audit.view']}>…</PermissionGuard>

// Role-set (kept for the rare legitimate role-set semantics)
<RoleGuard roles={['admin', 'superadmin']}>…</RoleGuard>
```

The `/admin` section gate was the most impactful migration —
previously a hard-coded `['admin','superadmin']` role-set that
locked out `moderator` and `security`; now `PermissionGuard`
with the common admin-perm any-of opens the section to every
tier that holds at least one admin capability.

## DTOs — removed actor fields

Every `admin*` RPC DTO had its `adminId: string` field removed.
Same for `viewerId` on `getProfile` / `getProfileByUsername` /
`getFollowers` / `getFollowing`. The server derives the actor
from the verified JWT via `actorId()` — there is no longer a
way for a client to claim to be acting as a different admin.

If your client sent these fields, the new DTOs reject them at
the zod boundary; just delete the field from the call.

## Quick sanity check

After upgrading, every backend should:

1. Boot through omnitron without errors.
2. Return `HTTP 200` on `/health`.
3. Log an entry on the FIRST authenticated call that says
   either nothing (happy path) OR `auth.session.fail_open`
   if Redis is unreachable (the MED-2 telemetry tag).

A `PERMISSION_VERSION_STALE` (401) the moment a role changes is
the new normal — the client refresh-token flow handles it
transparently.

---

---
id: "titan/migrations/from-express"
title: "From Express"
description: "Migrating a plain-Node Express service to Titan."
---

# From Express

Express services tend to share a recognisable shape: an app
object, a stack of middleware, a router file or two, ad-hoc
process-level config via `process.env`, and `console.log` for
observability. None of this is wrong — Titan just trades the
manual wiring for an opinionated container.

The point of this page is not to lecture you about why a
framework is better; it is to show you the **smallest concrete
mapping** between what you wrote in Express and what its
equivalent looks like in Titan, so you can decide whether the
trade is worth it.

## At a glance

| Express                                 | Titan                                                   |
| --------------------------------------- | ------------------------------------------------------- |
| `express()` app                         | `Application.create({ modules: [...] })`                |
| `app.use(mw)` middleware                | Netron RPC middleware via the HTTP `MiddlewarePipeline` |
| `app.get('/path', handler)`             | `@Service` method with `@Public()`                      |
| `req.body` / `req.query` / `req.params` | Plain method arguments                                  |
| `req.headers`                           | RPC `ExecutionContext` (auth, tracing)                  |
| `process.env`                           | `ConfigModule.forRoot({ schema, sources })`             |
| `console.log` / `winston` / `pino`      | Built-in `LoggerService` (pino under the hood)          |
| `helmet` / `cors` / `compression`       | Transport-level options or per-method middleware        |
| `joi` / `ajv` / `class-validator`       | Zod schemas + `@Validate(schema)`                       |
| `error-handler` middleware              | `TitanError` subclasses (auto-serialised)               |
| `node-cron`                             | `titan-scheduler` `@Cron(...)` decorator                |
| `prom-client`                           | `titan-metrics` counters / gauges / histograms          |
| `winston-loki` / file rotation          | Logger transports (configured declaratively)            |
| `bull` / `bullmq` queues                | `titan-notifications` (rotif) or `titan-scheduler`      |
| Manual `redis` connection               | `titan-redis` (clusters, sentinel, named instances)     |
| Manual graceful shutdown handlers       | Built-in lifecycle (`OnStop` / `OnDestroy`)             |

## A side-by-side concrete example

Suppose you have a small REST endpoint that creates a user.

```typescript
// Express — index.ts
import express from 'express';
import { z } from 'zod';

const app = express();
app.use(express.json());

const CreateUserSchema = z.object({ email: z.string().email() });

app.post('/users', async (req, res, next) => {
  try {
    const input = CreateUserSchema.parse(req.body);
    const user  = await repo.create(input);
    res.status(201).json(user);
  } catch (e) {
    next(e);
  }
});

app.use((err, _req, res, _next) => {
  if (err.name === 'ZodError') return res.status(400).json({ error: err.errors });
  console.error(err);
  res.status(500).json({ error: 'internal' });
});

app.listen(3000);
```

```typescript
// Titan — equivalent
import { Application, Module, Service, Injectable, Inject }
  from '@omnitron-dev/titan';
import { Public, Validate } from '@omnitron-dev/titan/decorators';
import { z } from '@omnitron-dev/titan/validation';

const CreateUserSchema = z.object({ email: z.string().email() });
type  CreateUser       = z.infer<typeof CreateUserSchema>;

@Injectable()
class UserRepo { /* ... */ }

@Service('users@1.0.0')
class UsersService {
  constructor(private readonly repo: UserRepo) {}

  @Public()
  @Validate({ input: CreateUserSchema })
  async create(input: CreateUser) {
    return this.repo.create(input);
  }
}

@Module({ providers: [UserRepo, UsersService] })
class UsersModule {}

const app = await Application.create({ modules: [UsersModule] });
await app.start();
```

Things you stop writing yourself:

- **Body parsing.** The transport layer hands you typed arguments.
- **Per-handler try/catch.** Throw a typed error; the framework
  serialises it on the wire.
- **Status-code mapping.** `Errors.notFound(...)` / `Errors.validation(...)`
  carry their own HTTP-equivalent codes.
- **Listen / port management.** Configured declaratively on the
  Netron transport.

## Mapping middleware

### Authentication

```typescript
// Express
app.use((req, res, next) => {
  const token = req.headers.authorization?.replace(/^Bearer\s+/, '');
  if (!token) return res.status(401).end();
  req.user = verifyJwt(token);
  next();
});
```

```typescript
// Titan — use titan-auth
import { TitanAuthModule } from '@omnitron-dev/titan-auth';
import { Public, Auth } from '@omnitron-dev/titan/decorators';

@Module({
  imports: [TitanAuthModule.forRoot({ jwtSecret: env.JWT_SECRET })],
})
class AppModule {}

// In your service — @Auth declares the requirement; the verified
// AuthContext is carried on the RPC ExecutionContext, not a parameter.
@Service('users@1.0.0')
class UsersService {
  @Public()
  @Auth({ roles: ['user'] })
  async me() { /* … */ }
}
```

### CORS / Helmet / Compression

These belong to the HTTP **transport**, not the service. Configure
on the Netron HTTP transport directly:

```typescript
import { HttpTransport } from '@omnitron-dev/titan/netron/transport/http';

netron.registerTransport('http', () => new HttpTransport());
netron.registerTransportServer('http', {
  name: 'api-http',
  options: {
    port: 3000,
    cors: { origin: 'https://example.com' },
    // compression, helmet-style headers, etc.
  },
});
```

### Logging requests

Express:

```typescript
app.use(morgan('combined'));
```

Titan: enable the built-in logger, then register a request-logging
middleware function on the HTTP server's `MiddlewarePipeline`:

```typescript
LoggerModule.forRoot({ level: 'info' });

// A middleware is a plain (ctx, next) function, registered on the
// pipeline — there is no netron.use().
const requestLogging: MiddlewareFunction = async (ctx, next) => {
  const t0 = performance.now();
  await next();
  logger.info({ method: ctx.methodName, ms: performance.now() - t0 });
};
pipeline.use(requestLogging);
```

→ See [Netron / Middleware](/docs/titan/netron/middleware) for the
full middleware contract.

## Configuration

Replace ad-hoc `process.env` reads with a validated schema. You
get type-safety and a single source of truth.

```typescript
// Express — typical
const PORT      = parseInt(process.env.PORT ?? '3000', 10);
const REDIS_URL = process.env.REDIS_URL ?? 'redis://localhost:6379';
```

```typescript
// Titan
import { ConfigModule } from '@omnitron-dev/titan/module/config';
import { z } from 'zod';

const AppConfigSchema = z.object({
  port:     z.coerce.number().default(3000),
  redisUrl: z.string().url(),
});

ConfigModule.forRoot({
  schema:  AppConfigSchema,
  sources: [{ type: 'env', prefix: 'APP_' }],
});
```

→ [`ConfigModule`](/docs/titan/modules/config)

## Logging

Drop `console.log` and `winston`:

```typescript
// Express
console.log(`[${new Date().toISOString()}] user created`, { id });
```

```typescript
// Titan
@Injectable()
class UsersService {
  constructor(private readonly logger: LoggerService) {}
  async create(input) {
    const user = await this.repo.create(input);
    this.logger.info({ id: user.id }, 'user created');
    return user;
  }
}
```

→ [`LoggerModule`](/docs/titan/modules/logger)

## Graceful shutdown

Express requires manual `SIGTERM` / `SIGINT` handlers and
`server.close()` orchestration. Titan handles this automatically:

```typescript
// Express
const server = app.listen(3000);
process.on('SIGTERM', () => server.close(() => process.exit(0)));
```

```typescript
// Titan
@Injectable()
class FlushBuffersOnShutdown implements OnStop {
  async onStop() { await this.bufferedWriter.flush(); }
}
```

The `Application` registers signal handlers, fires
`OnStop` → `OnDestroy` in reverse dependency order, and exits
cleanly.

→ [Application / Shutdown](/docs/titan/application/shutdown)

## Background work (cron, queues)

| Express setup                   | Titan equivalent                       |
| ------------------------------- | -------------------------------------- |
| `node-cron` + manual handler    | `titan-scheduler` `@Cron('0 * * * *')` |
| `bullmq` queue + worker process | `titan-notifications` rotif backbone   |
| Manual setInterval              | `@Interval(60_000)`                    |
| `setTimeout` + bookkeeping      | `@Timeout(5_000)`                      |

→ [`SchedulerModule`](/docs/titan/modules/scheduler)
and [`NotificationsModule`](/docs/titan/modules/notifications).

## Migration order — recommended

1. **Bootstrap.** Wrap your `express()` app inside an Application
   shell that runs alongside it — gradually move endpoints over.
2. **Config + logger.** Replace `process.env` reads and
   `console.log` calls. Wins are immediate (type-safety + structured
   logs).
3. **One endpoint.** Pick a low-risk endpoint, convert it to a
   `@Service` with `@Public()`. Front it with a thin Express proxy
   if needed:
   ```typescript
   app.post('/users', async (req, res) => {
     try { res.json(await usersService.create(req.body)); }
     catch (e) { /* map TitanError → status */ }
   });
   ```
4. **Cross-cutting concerns.** Replace middleware (auth, rate-limit,
   metrics) with Titan modules one at a time.
5. **Background work.** Migrate cron / queues to the scheduler /
   notifications modules.
6. **Cut the cord.** Once endpoints are all converted, drop the
   Express proxy and use Netron HTTP transport directly.

## What you gain

- **Validation, errors, logging, metrics as first-class** — not
  bolted-on.
- **Transport flexibility.** Same service works over HTTP, WS,
  TCP, Unix without rewriting handlers.
- **Real DI.** Easier tests; cleaner separation of concerns;
  contextual overrides for multi-tenant scenarios.
- **Operability.** Built-in graceful shutdown, health probes,
  metrics endpoint, traceable IDs.

## What you give up

- **The "one file, one app" feeling.** Titan applications are
  structured. If your service is < 200 lines of Express, the
  ergonomic gain is modest.
- **Direct control over the request/response cycle.** You will
  rarely need it, but if you do (e.g., streaming a multipart
  upload byte-for-byte), you reach for transport-level escapes.
- **Decorator-heavy code style.** If your team is decorator-averse,
  Titan will feel heavy.

## See also

- [Recipes / Webhook receiver](/docs/titan/recipes/webhook-receiver) —
  closest direct analogue to a small Express service
- [Concepts / Mental Model](/docs/titan/concepts/mental-model)
- [`titan-metrics`](/docs/titan/modules/metrics) — counters, gauges and histograms for services migrating from `prom-client`

---

---
id: "titan/migrations/from-nestjs"
title: "From NestJS"
description: "Concept-by-concept mapping from NestJS to Titan."
---

# From NestJS

NestJS and Titan share a decorator-driven, container-centric
approach — most concepts map directly. The genuine deltas are
where Titan made different decisions; this page calls them out
explicitly so you don't fight the framework.

## At a glance

| NestJS                                  | Titan                                                      | Note                               |
| --------------------------------------- | ---------------------------------------------------------- | ---------------------------------- |
| `@Module()`                             | `@Module()`                                                | Same shape                         |
| `@Injectable()`                         | `@Injectable()`                                            | Same                               |
| `@Controller()`                         | `@Service({ name, version })`                              | RPC contract, not HTTP route       |
| `@Get()` / `@Post()` etc.               | `@Public()`                                                | Transport-agnostic exposure        |
| `@Inject(TOKEN)`                        | `@Inject(TOKEN)`                                           | Same                               |
| `@Optional()`                           | `@Optional()`                                              | Same                               |
| `@Global()`                             | `@Global()` + `global: true` on DynamicModule              | Both available                     |
| `OnModuleInit`                          | `OnInit`                                                   | Slightly different name            |
| `OnModuleDestroy`                       | `OnDestroy`                                                | Slightly different name            |
| `OnApplicationBootstrap`                | `OnStart`                                                  | Slightly different name            |
| `OnApplicationShutdown`                 | `OnStop`                                                   | Plus `OnDestroy` for finalisation  |
| `ModuleRef.get()`                       | `app.resolve(Token)` or `container.resolve(Token)`         | Same idea                          |
| `DynamicModule`                         | `DynamicModule`                                            | Same shape                         |
| `forRoot / forRootAsync`                | `forRoot / forRootAsync`                                   | Same patterns                      |
| `HttpException` / `RpcException`        | `TitanError` and subclasses                                | Different hierarchy                |
| `ConfigModule` (`@nestjs/config`)       | `ConfigModule` (built-in)                                  | Similar, multi-source merge        |
| `Logger`                                | `LoggerService` (built-in pino-based)                      | 4 levels                           |
| `ScheduleModule` (`@nestjs/schedule`)   | `SchedulerModule` (`titan-scheduler`)                      | Same `@Cron` / `@Interval` pattern |
| `CacheModule` (`@nestjs/cache-manager`) | `TitanCacheModule` (`titan-cache`)                         | Native multi-tier (L1/L2)          |
| `ThrottlerModule` (`@nestjs/throttler`) | `TitanRateLimitModule` (`titan-ratelimit`)                 | Three algorithms                   |
| `BullModule` / `BullMQModule`           | `SchedulerModule` (cron) + `NotificationsModule` (rotif)   | Different decomposition            |
| Guards (`@UseGuards`)                   | `@Auth(...)` or middleware via Netron                      | Auth-specific surface              |
| Interceptors                            | RPC middleware (per-call) + DI middleware (per-resolution) | Two distinct layers                |
| Pipes (`@UsePipes`)                     | `@Validate(schema)` / `@Contract(...)`                     | Zod schemas, not class-validator   |
| Filters (`@UseFilters`)                 | Error class hierarchy + middleware                         | Errors are values, not events      |

## The biggest delta — controllers vs services

The largest mental shift: NestJS controllers are HTTP-aware (`@Get`,
`@Post`, `@Param`, `@Body`). Titan's `@Service` is **transport-
agnostic**. The same service class is reachable over HTTP, WebSocket,
TCP, and Unix — the framework dispatches to the same method body
regardless.

```typescript
// NestJS
@Controller('users')
class UsersController {
  @Get(':id')
  async findById(@Param('id') id: string) { return this.repo.find(id); }

  @Post()
  async create(@Body() input: CreateUserDto) { return this.repo.create(input); }
}

// Titan
@Service('users@1.0.0')
class UsersService {
  @Public()
  async findById(id: string) { return this.repo.find(id); }

  @Public()
  @Validate({ input: CreateUserSchema })
  async create(input: CreateUser) { return this.repo.create(input); }
}
```

URLs and parameter mapping are no longer part of the contract.
Clients call `users.findById('u_42')` directly — the wire format
is the framework's concern.

→ See [Netron / Services](/docs/titan/netron/services) for the full
contract semantics.

## Lifecycle hooks

| NestJS interface             | Titan interface | Fires when                             |
| ---------------------------- | --------------- | -------------------------------------- |
| `OnModuleInit`               | `OnInit`        | After construction, before start       |
| `OnApplicationBootstrap`     | `OnStart`       | Once all `onInit` complete             |
| `OnApplicationShutdown(sig)` | `OnStop`        | On `app.stop()` (in reverse dep order) |
| `OnModuleDestroy`            | `OnDestroy`     | Final cleanup phase                    |

Map your existing hooks accordingly. The `signal` argument on
`OnApplicationShutdown` becomes a `ShutdownReason` — but the
`OnStop` interface method takes no argument. If you need the
reason, register a shutdown handler via `app.onStop((reason,
details) => …)`, which receives it.

→ [Application / Lifecycle](/docs/titan/application/lifecycle)

## Validation — class-validator → Zod

NestJS commonly uses `class-validator` + `class-transformer`:

```typescript
// NestJS — DTO with decorators
class CreateUserDto {
  @IsEmail()      email: string;
  @MinLength(1)   @MaxLength(120) name: string;
}

@Post()
async create(@Body() input: CreateUserDto) { /* … */ }
```

Titan uses Zod (re-exported from `@omnitron-dev/titan/validation`):

```typescript
// Titan — schema-first
const CreateUserSchema = z.object({
  email: z.string().email(),
  name:  z.string().min(1).max(120),
});

@Public()
@Validate({ input: CreateUserSchema })
async create(input: z.infer<typeof CreateUserSchema>) { /* … */ }
```

The schema **is** the type. No separate DTO class; no decorator
soup; the `z.infer` gives you exact compile-time types.

→ [Validation / Contracts](/docs/titan/validation/contracts)

## Errors — exception filters → typed errors

NestJS catches via filters:

```typescript
// NestJS
@Catch(NotFoundException)
class NotFoundFilter implements ExceptionFilter { /* … */ }

throw new NotFoundException('user not found');
```

Titan: errors are values; the framework serialises them across the
wire with their class intact.

```typescript
// Titan
import { Errors, ErrorCode, TitanError } from '@omnitron-dev/titan/errors';

throw Errors.notFound('user', id);

// On the client:
try { await users.findById('missing'); }
catch (e) {
  if (e instanceof TitanError && e.code === ErrorCode.NOT_FOUND) { /* handle */ }
}
```

No filter classes needed. The error class is the wire contract.

→ [Errors](/docs/titan/errors/overview)

## DI scopes

| NestJS scope      | Titan equivalent (Nexus) |
| ----------------- | ------------------------ |
| `Scope.DEFAULT`   | `Scope.Singleton`        |
| `Scope.REQUEST`   | `Scope.Request`          |
| `Scope.TRANSIENT` | `Scope.Transient`        |
| (no analogue)     | `Scope.Scoped`           |

Plus Titan's [contextual injection](/docs/titan/di/contextual-injection) —
one token, multiple providers, chosen per request context. This is
the closest Titan has to NestJS's request-scoped providers but
covers more (per-tenant, per-feature-flag, per-environment).

## Guards / middleware

NestJS:

```typescript
@UseGuards(JwtAuthGuard)
@Get(':id')
async findById(@Param('id') id: string) { /* … */ }
```

Titan:

```typescript
@Public()
@Auth({ roles: ['user'] })
async findById(id: string) { /* … */ }
```

Or register RPC middleware globally. Middleware is a plain
`(ctx, next)` function registered on the HTTP server's
`MiddlewarePipeline` (there is no `netron.use()`); the HTTP server
already wires auth — app bootstrap adds the rest:

```typescript
pipeline.use(authMiddlewareFn);
pipeline.use(rateLimitMiddlewareFn);
```

→ [Netron / Middleware](/docs/titan/netron/middleware) and
[Netron / Authentication](/docs/titan/netron/authentication)

## Module helpers

NestJS has helpers like `Test.createTestingModule()` for tests.
Titan overrides a provider by registering it **last** — pass the
`providers` tuple array to `Application.create`; entries are wired
after the modules' own providers, so the last registration for a
token wins:

```typescript
// NestJS
const module = await Test.createTestingModule({
  providers: [UsersService],
}).overrideProvider(Database).useClass(FakeDatabase).compile();

// Titan — providers are [token, providerDefinition] tuples
const app = await Application.create({
  modules:    [UsersModule],
  providers:  [[Database, { useClass: FakeDatabase }]],
  disableGracefulShutdown: true,
});
```

→ [Testing / DI Overrides](/docs/titan/testing/di-overrides)

## Migration order — recommended

1. **Bootstrap.** Move the entry-point into `Application.create`;
   bridge legacy providers via `providers: [[TOKEN, { useValue: ... }]]`.
2. **One module.** Pick a low-stakes feature module; convert
   controller → `@Service`, DTOs → Zod, guards → `@Auth`.
3. **Lifecycle.** Rename `OnModuleInit` → `OnInit`, etc.
4. **Errors.** Replace `HttpException` instances with the `Errors`
   namespace.
5. **Config / logging.** Drop `@nestjs/config` and `Logger` —
   built-in equivalents are auto-loaded.
6. **Repeat module-by-module.** Strangler-fig — no big bang.

## What you gain

- **Transport-agnostic RPC.** Same service over HTTP / WS / TCP /
  Unix without duplicating handlers.
- **Native time-series metrics.** No prom-client; built-in storage
  backends.
- **End-to-end typed errors.** Errors travel as classes, not
  status codes.
- **Smaller dependency footprint.** Ecosystem modules are
  individually versioned; you install what you use.

## What's different (not better-or-worse, just different)

- **Validation is Zod, not class-validator.** If your codebase
  is heavily class-decorated DTOs, the rewrite is real work.
- **No `@Get('/path')` semantics.** RPC-first means thinking in
  service methods, not URL routes.
- **`OnApplicationShutdown(signal)` → `OnStop` (no argument);
  the `ShutdownReason` is delivered to handlers registered via
  `app.onStop((reason, details) => …)`** — slightly different API.

## See also

- [Concepts / Design Principles](/docs/titan/concepts/design-principles)
- [Recipes / API service stack](/docs/titan/recipes/api-service) — the
  Titan equivalent of the canonical NestJS REST API

---

---
id: "titan/migrations/from-prom-client"
title: "From prom-client"
description: "Replacing prom-client with titan-metrics."
---

# From prom-client

[`prom-client`](https://github.com/siimon/prom-client) is the
de-facto Prometheus client for Node. It is excellent at what it
does — instrument code, expose `/metrics`, hand off to a real
Prometheus server. `titan-metrics` covers the same use case while
also persisting samples to a storage backend (memory / SQLite /
Postgres), letting you query time-series **inside** your app and
ship cross-pod aggregates without standing up Prometheus first.

This page is the surgical recipe for moving from one to the other.

## At a glance

| `prom-client`                             | `titan-metrics`                                         |
| ----------------------------------------- | ------------------------------------------------------- |
| `new Registry()`                          | `MetricsService` (DI-provided singleton)                |
| `new Counter({ name, help, labelNames })` | `metrics.recordTyped('counter', name, labels, 1)`       |
| `new Gauge(...)`                          | `metrics.recordTyped('gauge', name, labels, v)`         |
| `new Histogram({ buckets })`              | `metrics.recordTyped('histogram', name, labels, v)`     |
| `register.metrics()`                      | `metrics.getPrometheusText()`                           |
| `collectDefaultMetrics()`                 | `collection: { process: true, system: true }`           |
| Custom registry per app                   | `appName` option (tags every sample)                    |
| (no built-in storage)                     | `storage: { type: 'memory' \| 'sqlite' \| 'postgres' }` |
| (manual cleanup)                          | `retention: { maxAge: '7d' }` (automatic)               |
| (manual exposition route)                 | `MetricsRpcService` over Netron (or roll your own)      |

## A side-by-side concrete example

Suppose you currently instrument an HTTP handler like this:

```typescript
// prom-client
import { Counter, Histogram, register, collectDefaultMetrics }
  from 'prom-client';

collectDefaultMetrics();

const httpRequests = new Counter({
  name: 'http_requests_total',
  help: 'HTTP requests',
  labelNames: ['route', 'status'],
});

const httpDuration = new Histogram({
  name: 'http_request_duration_ms',
  help: 'HTTP latency (ms)',
  labelNames: ['route'],
  buckets:    [5, 25, 100, 500, 2500],
});

app.post('/orders', async (req, res) => {
  const end = httpDuration.startTimer({ route: '/orders' });
  try {
    const order = await orders.create(req.body);
    res.json(order);
    httpRequests.inc({ route: '/orders', status: '200' });
  } catch (e) {
    httpRequests.inc({ route: '/orders', status: '500' });
    throw e;
  } finally { end(); }
});

app.get('/metrics', async (_req, res) => {
  res.type(register.contentType).end(await register.metrics());
});
```

The Titan equivalent uses the `@Metrics` decorator for automatic
RPC instrumentation, and `recordTyped` for your own counters /
histograms:

```typescript
// titan-metrics
import { Module, Service, Inject } from '@omnitron-dev/titan';
import { Public } from '@omnitron-dev/titan/decorators';
import { TitanMetricsModule, MetricsService, METRICS_SERVICE_TOKEN, Metrics }
  from '@omnitron-dev/titan-metrics';

@Service('orders@1.0.0')
class OrdersService {
  constructor(
    @Inject(METRICS_SERVICE_TOKEN) private readonly metrics: MetricsService,
  ) {}

  // @Metrics() auto-records rpc_requests_total,
  // rpc_request_duration_seconds and rpc_errors_total for this method.
  // The optional string only overrides the `method` label.
  @Public()
  @Metrics()
  async create(input: CreateOrder) {
    const order = await this.repo.create(input);
    // Custom metrics use recordTyped — there is no counter/histogram
    // option on @Metrics.
    this.metrics.recordTyped('counter', 'orders_processed_total', { status: 'ok' }, 1);
    return order;
  }
}

@Module({
  imports: [
    TitanMetricsModule.forRoot({
      appName:    'orders-api',
      collection: { enabled: true, process: true, system: true, rpc: true },
      storage:    { type: 'memory' },
    }),
  ],
  providers: [OrdersService],
})
class AppModule {}
```

> **Note**
>
> `@Metrics()` only records once the instance is linked to the
> `MetricsService` via `attachMetricsService(instance, service)` —
> `forRoot()` does not do this for you. The imperative `recordTyped`
> path (using the injected service) works with no extra wiring.

If you need the raw Prometheus text:

```typescript
const text = await metrics.getPrometheusText();
// Serve from any HTTP route, or expose via the RPC service.
```

## Step-by-step migration

### 1. Install the module, keep `prom-client` for now

```bash
pnpm add @omnitron-dev/titan-metrics
```

Don't uninstall `prom-client` yet — you'll cut it last.

### 2. Boot the module

```typescript
TitanMetricsModule.forRoot({
  appName:    'my-service',
  collection: { enabled: true, process: true, system: true, rpc: true },
  storage:    { type: 'memory' },  // try sqlite/postgres later
})
```

`collection.process: true` replaces `collectDefaultMetrics()` —
RSS, heap, CPU, uptime. (Event-loop lag is not built in; register
a gauge yourself if you need it.)

### 3. Migrate one metric at a time

For each existing counter / gauge / histogram, write the Titan
equivalent:

```typescript
// Before
const userCreated = new Counter({
  name: 'users_created_total',
  help: 'Users created',
  labelNames: ['source'],
});
userCreated.inc({ source: 'web' });

// After
metrics.recordTyped('counter', 'users_created_total', { source: 'web' }, 1);
```

`recordTyped` is the canonical API — it keeps the Prometheus
registry and storage backend synchronised in a single call.

### 4. Migrate the exposition route

```typescript
// Before
app.get('/metrics', async (_req, res) => {
  res.type(register.contentType).end(await register.metrics());
});

// After (HTTP)
app.get('/metrics', async (_req, res) => {
  res.type('text/plain; version=0.0.4').end(await metrics.getPrometheusText());
});

// After (via Netron RPC — what the Omnitron console reads)
// MetricsRpcService (Netron service name 'OmnitronMetrics') is NOT
// auto-registered by the module — expose it yourself, constructing it
// with the injected MetricsService.
```

### 5. Drop `prom-client`

Once every `Counter` / `Gauge` / `Histogram` is gone:

```bash
pnpm remove prom-client
```

## Naming conventions

`prom-client` uses `snake_case`. **`titan-metrics` does not
normalise metric names** — `getPrometheusText()` emits the name
exactly as you registered it. Prometheus metric names may only
contain `[a-zA-Z0-9_:]`, so a dotted name like `users.created.total`
would be emitted verbatim and produce **invalid** exposition.

Use `snake_case` at the call site:

| What you record          | Prometheus exposition    |
| ------------------------ | ------------------------ |
| `users_created_total`    | `users_created_total`    |
| `http_request_duration`  | `http_request_duration`  |
| `orders_processed_total` | `orders_processed_total` |

The built-in collectors already follow this (`rpc_requests_total`,
`heap_used_bytes`, …); match it so your custom metrics scrape cleanly.

## Labels

Same advice as `prom-client`: keep label cardinality small.

- **Good:** `tier`, `region`, `status`, `route`, `version`.
- **Bad:** `userId`, `requestId`, `email` — every value creates a
  fresh time series.

`titan-metrics` will happily accept high-cardinality labels and
your storage backend will fill up. Discipline lives in the call
sites.

## Histograms

`prom-client` lets you set buckets per-histogram. `titan-metrics`
applies one **registry-wide** bucket set instead: `recordTyped('histogram',
…)` records against the registry's configured buckets (a default set
unless the registry is constructed with a custom `buckets` array) —
there is no per-call bucket argument.

Five-to-ten buckets is usually right. Each bucket is a separate
time-series, so over-bucketing has real cost.

## Default metrics

| `prom-client`                         | `titan-metrics`                                                                                                              |
| ------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------- |
| `collectDefaultMetrics()`             | `collection: { process: true }`                                                                                              |
| `collectDefaultMetrics({ register })` | (registry is module-managed; nothing to pass)                                                                                |
| heap / RSS / CPU / uptime             | Included via `collection.process`; sample at `collection.interval` (default 5s). Event-loop lag / GC / FDs are not built in. |

## Persistence — the bit `prom-client` doesn't do

`titan-metrics` flushes samples to the storage backend on a fixed
5s cadence. This unlocks queries from inside your app:

```typescript
const series = await metrics.querySeries({
  names:    ['orders_process_ms'],   // array; filter by metric name(s)
  from:     Date.now() - 3_600_000,
  to:       Date.now(),
  interval: '1m',                    // bucket size as a duration string
});
```

Use cases:

- **Operator console** without Prometheus.
- **Auto-degradation logic** that reads its own metrics.
- **Cross-pod aggregation** by pointing `storage: { type: 'postgres' }`
  at a shared connection.

If you don't need these, the `'memory'` backend is essentially
free — the ring buffer caps RAM use, the exposition path is
identical to a plain registry, and you keep the option of turning
persistence on later.

## What's different

- **No registry per-instance.** The container provides one
  `MetricsService` per app; multi-registry scenarios are rare and
  handled by `appName` tags or separate processes.
- **`recordTyped` instead of typed builders.** Less verbose,
  fewer pre-declared objects, but you lose the compile-time
  guarantee that the label set matches the declaration.
- **`@Metrics()` decorator** auto-instruments a Netron method with
  the fixed `rpc_requests_total` / `rpc_request_duration_seconds` /
  `rpc_errors_total` set. It takes only an optional method-label
  string — for custom metrics you still use `recordTyped`.

## See also

- [`titan-metrics`](/docs/titan/modules/metrics) — full reference
- [Best Practices / Observability](/docs/titan/best-practices/observability)
- [`titan-telemetry-relay`](/docs/titan/modules/telemetry-relay) —
  store-and-forward shipping for offline/edge deployments

---

---
id: "titan/migrations/index"
title: "Migration guides"
description: "Move an existing service onto Titan — one concern at a time."
---

# Migration guides

Coming from another framework or a hand-rolled Node stack? These
guides cover the most common starting points and show what the
equivalent piece looks like in Titan.

The advice across all of them: **migrate one concern at a time**.
Don't try to flip the whole codebase to Titan in one PR. Start
with one of:

- A new feature module — write it Titan-style; everything else
  stays where it is.
- One existing concern (auth, metrics, scheduling) — swap just
  that, leave business logic untouched.
- The bootstrap layer — move the app entry point into Titan's
  `Application.create`; keep your existing services as
  `@Injectable` providers.

## Guides

| Source                                                              | What you keep, what you replace                                                      |
| ------------------------------------------------------------------- | ------------------------------------------------------------------------------------ |
| [From NestJS](/docs/titan/migrations/from-nestjs)                   | Most concepts map 1; the deltas are decorator names, `Netron` for RPC, and lifecycle |
| [From plain Express / Fastify](/docs/titan/migrations/from-express) | Add structure: module system, DI container, lifecycle, typed RPC                     |
| [From prom-client](/docs/titan/migrations/from-prom-client)         | Replace the metrics layer alone — keep everything else                               |

## Common patterns regardless of source

### Strangler fig — module by module

The safest migration strategy is the **strangler fig** pattern:

```mermaid
flowchart LR
  subgraph T0["Today"]
    Old1[Legacy: auth]
    Old2[Legacy: orders]
    Old3[Legacy: payments]
    Old4[Legacy: notifications]
  end

  subgraph W1["Week 1"]
    New1[Titan: auth]
    Old2b[Legacy: orders]
    Old3b[Legacy: payments]
    Old4b[Legacy: notifications]
  end

  subgraph W4["Week 4"]
    New1c[Titan: auth]
    New2c[Titan: orders]
    Old3c[Legacy: payments]
    Old4c[Legacy: notifications]
  end

  subgraph W12["Week 12"]
    New1d[Titan: auth]
    New2d[Titan: orders]
    New3d[Titan: payments]
    New4d[Titan: notifications]
  end

  T0 --> W1 --> W4 --> W12
```

At every checkpoint, the app boots, tests pass, prod stays up.

### Boot-time bridging

If your legacy app has its own bootstrap, wrap it with Titan's
`Application.create({ providers: [...] })`:

```typescript
import { Application } from '@omnitron-dev/titan';
import { existingService } from './legacy/services.js';

const app = await Application.create({
  providers: [
    [EXISTING_SERVICE_TOKEN, { useValue: existingService }],
  ],
  modules: [
    NewFeatureModule,                          // new code lives here
  ],
});

await app.start();
```

Legacy code keeps working; new Titan modules can `@Inject` into
it; you migrate at your own pace.

### Two-way bridges

A Titan service can call a legacy function; a legacy function can
call a Titan service via the container. Use this to migrate
incrementally without duplicating logic.

```typescript
// In a Titan service:
constructor(@Inject(LEGACY_PAYMENT_TOKEN) private readonly pay: any) {}

// Or from legacy code:
const titanService = container.resolve(NewService);
await titanService.doSomething();
```

## Testing during migration

Run the new code under the same integration tests as the legacy
code. If you can substitute implementations in DI, you can:

1. Run tests against the legacy implementation — green.
2. Switch the DI binding to the Titan implementation — re-run
   tests — green.
3. Delete the legacy code.

This catches behavioural drift immediately.

## See also

- [Best Practices / Structuring services](/docs/titan/best-practices/structuring-services) — where Titan-style services land
- [Modules system / Defining modules](/docs/titan/modules-system/defining-modules) — the new unit of composition
- [Testing](/docs/titan/testing/overview) — how to verify the migration

---

---
id: "titan/modules/auth"
title: "titan-auth"
---

# titan-auth

<span class="module-badge">official · @omnitron-dev/titan-auth · stable</span>

> **Info**
>
> This page covers the JWT primitives (issuance, verification,
> rotation, caching). For the **authorisation** surface
> (permissions, roles, ABAC, per-user overrides, RLS bridge,
> audit), start at the [Authentication & Authorisation](/docs/auth)
> section — `titan-auth` is the foundation it builds on.
>
> For migrating from a pre-`pv`-claim deployment, see the
> [Auth v1 → v2 migration guide](/docs/titan/migrations/auth-1-to-2).

JWT authentication with HS256 / RS256 / ES256 support, remote JWKS,
multi-key rotation via the `kid` header, an LRU token cache, signed-URL
service, and Bearer/API-key middleware. Single self-contained module
that issues, verifies, caches, and rotates.

```bash
pnpm add @omnitron-dev/titan-auth
```

## When you need it

- **Inbound API auth.** Validate Bearer tokens or API keys on every
  RPC call without re-decoding on the hot path.
- **Internal service-to-service.** Use the service-key path for
  trusted machine clients.
- **Signed URLs.** Time-bounded, signed download / webhook URLs that
  don't require server-side session lookup.
- **Zero-downtime key rotation.** Roll signing keys without forcing
  re-login via the `kid` header registry.

## Quickstart

### HS256 (symmetric — simplest)

```typescript
import { TitanAuthModule } from '@omnitron-dev/titan-auth';

@Module({
  imports: [
    TitanAuthModule.forRoot({
      algorithm:    'HS256',
      jwtSecret:    env.JWT_SECRET,
      issuer:       'my-app',
      audience:     'my-api',
      cacheEnabled: true,
      cacheMaxSize: 1_000,
      cacheTTL:     300_000,           // 5 min
    }),
  ],
})
class AppModule {}
```

### RS256 / ES256 with JWKS (external IDP)

```typescript
TitanAuthModule.forRoot({
  algorithm: 'RS256',
  jwksUrl:   'https://auth.example.com/.well-known/jwks.json',
  issuer:    'https://auth.example.com',
  audience:  'my-api',
})
```

The JWKS is lazy-fetched on first verify and cached.

### Async configuration

```typescript
TitanAuthModule.forRootAsync({
  imports:    [ConfigModule],
  useFactory: (config: ConfigService) => ({
    algorithm: 'HS256',
    jwtSecret: config.get('auth.jwtSecret'),
    issuer:    config.get('auth.issuer'),
  }),
  inject: [ConfigService],
})
```

## `IAuthModuleOptions`

| Option             | Type                                                                | Default     |
| ------------------ | ------------------------------------------------------------------- | ----------- |
| `algorithm`        | `'HS256' \| 'RS256' \| 'ES256'`                                     | `'HS256'`   |
| `jwtSecret`        | `string`                                                            | —           |
| `jwksUrl`          | `string`                                                            | —           |
| `issuer`           | `string`                                                            | —           |
| `audience`         | `string`                                                            | —           |
| `serviceKey`       | `string` — service-level API key                                    | —           |
| `anonKey`          | `string` — anonymous API key                                        | —           |
| `verificationKeys` | `Record<string, string>` — kid → secret registry for HS256 rotation | —           |
| `requireKid`       | `boolean` — reject HS256 tokens without `kid` header                | `false`     |
| `defaultTenantId`  | `string`                                                            | `'default'` |
| `cacheEnabled`     | `boolean`                                                           | `true`      |
| `cacheMaxSize`     | `number`                                                            | `1_000`     |
| `cacheTTL`         | `number` (ms)                                                       | `300_000`   |
| `urlSigningKey`    | `string` (falls back to `jwtSecret`)                                | —           |
| `isGlobal`         | `boolean`                                                           | `false`     |

## `JWTService`

The injected service for verification, context creation, and signed
URL issuing.

```typescript
import { JWT_SERVICE_TOKEN, type IJWTService } from '@omnitron-dev/titan-auth';

@Service({ name: 'admin' })
class AdminService {
  constructor(@Inject(JWT_SERVICE_TOKEN) private readonly jwt: IJWTService) {}

  @Public()
  async whoIs(token: string) {
    return this.jwt.createContext(token);
  }
}
```

### Token verification

| Method                 | Returns                 | Notes                                                         |
| ---------------------- | ----------------------- | ------------------------------------------------------------- |
| `verify(token)`        | `Promise<IJWTPayload>`  | Cache-first; throws `TokenExpiredError` / `InvalidTokenError` |
| `createContext(token)` | `Promise<IAuthContext>` | Convenience — verify + wrap into auth context                 |
| `clearCache()`         | `void`                  | Drop every cached token                                       |
| `getCacheStats()`      | `ITokenCacheStats`      | `{ size, maxSize, hits, misses, hitRate }`                    |

### Signed URLs

| Method                                         | Returns                        |
| ---------------------------------------------- | ------------------------------ |
| `createSignedToken(payload, expiresInSeconds)` | `Promise<string>`              |
| `verifySignedToken(token)`                     | `Promise<ISignedTokenPayload>` |

```typescript
const token = await this.jwt.createSignedToken(
  { resource: `download/${fileId}`, userId },
  3600,                              // 1 hour expiry
);

const url = `https://api.example.com/files/${fileId}?token=${token}`;
```

## `AuthMiddleware`

Extracts credentials from an incoming request, validates them, and
returns an `IAuthContext`.

```typescript
import { AUTH_MIDDLEWARE_TOKEN, type IAuthMiddleware } from '@omnitron-dev/titan-auth';

@Service({ name: 'users' })
class UsersService {
  constructor(@Inject(AUTH_MIDDLEWARE_TOKEN) private readonly auth: IAuthMiddleware) {}

  @Public()
  async whoami(request: IRequestLike) {
    return this.auth.authenticate(request);   // anonymous fallback if no credentials
  }

  @Public()
  async listMine(request: IRequestLike) {
    return this.auth.authenticateRequired(request);   // throws UnauthorizedError if anonymous
  }
}
```

| Method                          | Returns                      | Notes                                                     |
| ------------------------------- | ---------------------------- | --------------------------------------------------------- |
| `authenticate(request)`         | `Promise<IAuthContext>`      | Falls back to anonymous if no credentials present         |
| `authenticateRequired(request)` | `Promise<IAuthContext>`      | Throws `UnauthorizedError` if no valid credentials        |
| `extractToken(request)`         | `string \| null`             | Pulls Bearer token from headers                           |
| `validateApiKey(apiKey)`        | `{ valid, type?, context? }` | Constant-time comparison — defends against timing attacks |

The auth chain: Bearer token (JWT) → service-key API key → anon-key
API key → anonymous context.

## Decorators

```typescript
import {
  RequireAuth,
  RequireServiceAuth,
  RequireAdminAuth,
  RequireRole,
  Public,
} from '@omnitron-dev/titan-auth';
```

### `@RequireAuth(options?)`

```typescript
@Public()
@RequireAuth({ roles: ['admin'], allowAnonymous: false })
async deleteUser(request: IRequestLike, userId: string) {
  // this.__authContext__ is set by the decorator after successful auth
}
```

| Option           | Type       | Default       |
| ---------------- | ---------- | ------------- |
| `roles`          | `string[]` | —             |
| `allowAnonymous` | `boolean`  | `false`       |
| `message`        | `string`   | (default 401) |

The decorator looks up the middleware on the instance under
`__authMiddleware__`; inject it there. After successful auth, the
resolved context is stored on the instance as `__authContext__`.

### Shorthands

| Decorator                   | Equivalent                                           |
| --------------------------- | ---------------------------------------------------- |
| `@RequireServiceAuth()`     | `@RequireAuth({ roles: ['service_role'] })`          |
| `@RequireAdminAuth()`       | `@RequireAuth({ roles: ['admin', 'service_role'] })` |
| `@RequireRole(roles, msg?)` | `@RequireAuth({ roles, message: msg })`              |
| `@Public()`                 | Skip auth on this method or class                    |

## Multi-key rotation (HS256)

Rotate signing secrets without forcing re-login by registering
multiple keys keyed by `kid`:

```typescript
TitanAuthModule.forRoot({
  algorithm: 'HS256',
  jwtSecret: env.JWT_SECRET_CURRENT,                 // for new tokens
  verificationKeys: {
    '2025-q1': env.JWT_SECRET_OLD_Q1,               // accept tokens signed with old key
    '2025-q2': env.JWT_SECRET_CURRENT,
  },
  requireKid: false,                                  // accept tokens without kid (legacy)
})
```

Resolution order on verify:

1. RS256/ES256 via JWKS if `jwksUrl` is configured.
2. HS256 with `kid` header → look up `verificationKeys`. Unknown
   `kid` rejects.
3. HS256 without `kid` → legacy single-secret path, unless
   `requireKid: true`.

After every active token has expired, drop the retired key from
`verificationKeys` to complete the rotation.

## Auth flow

```mermaid
flowchart LR
  Req[Incoming request]
  Req --> Mw[AuthMiddleware.authenticate]
  Mw --> Bearer{Bearer token?}
  Bearer -- yes --> Verify[JWTService.verify]
  Verify -- valid --> Ctx[IAuthContext]
  Verify -- invalid --> Err1[UnauthorizedError]
  Bearer -- no --> ApiKey{API key?}
  ApiKey -- service --> CtxSvc[service-role context]
  ApiKey -- anon --> CtxAnon[anonymous context]
  ApiKey -- none --> AnonFallback{authenticate or required?}
  AnonFallback -- authenticate --> CtxAnon
  AnonFallback -- required --> Err1
```

## Tokens

| Token                      | Purpose                                          |
| -------------------------- | ------------------------------------------------ |
| `JWT_SERVICE_TOKEN`        | `JWTService`                                     |
| `AUTH_MIDDLEWARE_TOKEN`    | `AuthMiddleware`                                 |
| `SIGNED_URL_SERVICE_TOKEN` | `JWTService` (same instance, narrower interface) |
| `AUTH_OPTIONS_TOKEN`       | Resolved options bundle                          |

## Lifecycle

Neither service implements explicit lifecycle hooks. JWKS is
lazy-loaded; the token cache is in-memory and trims itself via LRU
eviction.

## Anti-patterns

- **Long `cacheTTL` with frequent token revocation.** Revocation
  bypasses the cache only after `cacheTTL` expires. For high-churn
  revocation lists, lower `cacheTTL` or call `clearCache()` on
  revocation events.
- **Reading `jwtSecret` from a checked-in config.** Always inject
  via env or a secret manager.
- **Skipping `issuer` and `audience` claims.** Without them, a token
  issued for one tenant validates against another. Always pin both.
- **Using `serviceKey` from a browser.** The service key bypasses
  user-scoped policies. Reserve it for trusted machine clients.

## See also

- [Netron Authentication](/docs/titan/netron/authentication) — the policy
  framework (`BuiltInPolicies`, `PolicyEngine`) for fine-grained
  authorisation
- [`@RequireAuth`](#requireauthoptions) — vs `@Public({ auth: …  })`
  (the inline form on `@Public`)

## Shared sessions across backends

The shipped [shared-session preset](/docs/titan/modules/shared-session) configures JWT claim
validation, session-registry checks and an optional canonical-store fallback.
Read it before composing a multi-backend authentication system.

## Portable role and permission policy

Use [`@omnitron-dev/access`](/docs/auth/access-policy) for shared browser/server
policy without installing JWT infrastructure. `titan-auth/access` remains a
compatibility facade. The JWT module and shared-session preset are optional adapters.

---

---
id: "titan/modules/cache"
title: "titan-cache"
---

# titan-cache

<span class="module-badge">official · @omnitron-dev/titan-cache · stable</span>

Multi-tier (L1/L2) caching with multiple eviction strategies, tag-based
invalidation, optional compression, atomic `getOrSet`, partition views,
and warming strategies. Single-tier in-memory by default; pluggable L2
(typically Redis) for cross-pod sharing.

```bash
pnpm add @omnitron-dev/titan-cache
```

## When you need it

Three workloads it solves cleanly:

1. **Read-heavy services** — a per-process LRU in front of expensive
   lookups. Cuts database load 10-50× on typical request patterns.
2. **Cross-pod cache coherence** — L1 in each pod for low latency, L2
   in Redis for shared invalidation. Tag-based eviction propagates
   to every pod through L2.
3. **Idempotency tokens** — `getOrSet` is atomic from the caller's
   perspective; a thundering herd of identical requests collapses to
   one factory invocation.

## Quickstart

### Single-tier in-memory

```typescript
import { TitanCacheModule } from '@omnitron-dev/titan-cache';

@Module({
  imports: [
    TitanCacheModule.forRoot({
      maxSize:        5_000,
      defaultTtl:     600,                    // seconds
      evictionPolicy: 'lru',
      enableStats:    true,
    }),
  ],
})
class AppModule {}
```

### Multi-tier (L1 in-memory + L2 Redis)

```typescript
@Module({
  imports: [
    TitanRedisModule.forRoot({ host: 'redis-host' }),
    TitanCacheModule.forRoot({
      multiTier: true,
      l1: { maxSize: 1_000, ttl: 60 },        // hot cache, short TTL
      l2: { client: redisAdapter, ttl: 3600, prefix: 'cache:' },
    }),
  ],
})
class AppModule {}
```

A read hits L1; a miss falls through to L2 and backfills L1. A write
populates both tiers. Tag-based invalidation propagates through L2.

### Async configuration

```typescript
TitanCacheModule.forRootAsync({
  useFactory: (config: ConfigService) => ({
    multiTier: config.get('cache.multiTier'),
    l1: { maxSize: config.get('cache.l1.size'), ttl: config.get('cache.l1.ttl') },
    l2: { client: redisAdapter, ttl: config.get('cache.l2.ttl') },
  }),
  inject: [ConfigService],
})
```

## `ICacheModuleOptions`

| Option                 | Type                                                 | Default  |
| ---------------------- | ---------------------------------------------------- | -------- |
| `multiTier`            | `boolean`                                            | `false`  |
| `l1`                   | `{ maxSize, ttl }`                                   | —        |
| `l2`                   | `{ client, ttl, prefix }`                            | —        |
| `defaultTtl`           | `number` (seconds)                                   | `300`    |
| `maxSize`              | `number`                                             | `1_000`  |
| `evictionPolicy`       | `'lru' \| 'lfu' \| 'fifo' \| 'random' \| 'ttl'`      | `'lru'`  |
| `enableStats`          | `boolean`                                            | `true`   |
| `statsInterval`        | `number` (ms)                                        | —        |
| `partitions`           | `ICachePartition[]`                                  | —        |
| `warmingStrategies`    | `ICacheWarmingStrategy[]`                            | —        |
| `useWeakRef`           | `boolean`                                            | `false`  |
| `compressionThreshold` | `number` (bytes)                                     | `1_024`  |
| `compressionAlgorithm` | `'none' \| 'gzip' \| 'deflate' \| 'brotli' \| 'lz4'` | `'none'` |
| `ttlCleanupInterval`   | `number` (ms — wheel-timer bucket size)              | —        |
| `wheelTimerBuckets`    | `number`                                             | —        |
| `staleWhileRevalidate` | `number` (seconds — SWR window)                      | —        |
| `backgroundRefresh`    | `boolean`                                            | —        |
| `isGlobal`             | `boolean`                                            | `false`  |

## `CacheService`

The top-level service exposed via DI. Resolves named caches; creates
ad-hoc ones at runtime; emits / observes lifecycle events.

```typescript
import { CACHE_SERVICE_TOKEN, type ICacheService } from '@omnitron-dev/titan-cache';

@Service({ name: 'users' })
class UsersService {
  constructor(@Inject(CACHE_SERVICE_TOKEN) private readonly cache: ICacheService) {}
}
```

| Method                                         | Purpose                                  |
| ---------------------------------------------- | ---------------------------------------- |
| `getCache<T>(name?)`                           | Get cache by name (default: `'default'`) |
| `createCache<T>(name, options?)`               | Create a new named cache                 |
| `getOrCreateCache<T>(name, options?)`          | Atomic get-or-create                     |
| `listCaches()`                                 | All cache names                          |
| `deleteCache(name)`                            | Remove a cache                           |
| `getGlobalStats()`                             | Aggregated stats across every cache      |
| `on(event, listener)` / `off(event, listener)` | Subscribe / unsubscribe                  |
| `dispose()`                                    | Async cleanup of all caches              |

## `ICache<T>` — the per-cache surface

```typescript
const users = this.cache.getCache<User>('users');
const user  = await users.getOrSet(`u:${id}`, () => this.repo.findById(id), { ttl: 60 });
```

| Method                                       | Purpose                                               |
| -------------------------------------------- | ----------------------------------------------------- |
| `get(key, options?)`                         | Read; returns `undefined` on miss                     |
| `set(key, value, options?)`                  | Write with optional TTL / tags / compression          |
| `has(key)`                                   | Existence check — **see TOCTOU note below**           |
| `getOrSet(key, factory, options?)`           | Atomic read-through; preferred over `has + get + set` |
| `delete(key)`                                | Remove a key                                          |
| `clear(pattern?)`                            | Drop all or by string / RegExp pattern                |
| `invalidateByTags(tags[])`                   | Drop every entry tagged with any of these             |
| `getMany(keys[])` / `setMany(map, options?)` | Batch read / write                                    |
| `getStats()` / `resetStats()`                | Per-cache stats                                       |
| `getEntry(key)`                              | Read value + metadata (created, hits, tags)           |
| `keys(pattern?)` / `size()`                  | Enumerate / count                                     |
| `warm(strategies?)`                          | Preload from configured warming strategies            |
| `partition(name)`                            | Get a partition view (isolated key namespace)         |
| `dispose()`                                  | Async per-cache cleanup                               |

### TOCTOU note on `has()`

From the source:

> There is a TOCTOU window between `has(key) === true` and a subsequent
> `get(key)` — the entry can expire or be evicted in between, so
> `get()` may return `undefined` even when `has()` just returned `true`.
> If you need to *use* the value, prefer `getOrSet(key, factory)`
> (atomic from the caller's perspective). Reserve `has()` for fast-path
> cardinality checks where the value isn't read afterwards.

## Multi-tier API — `IMultiTierCache<T>` extends `ICache<T>`

```mermaid
flowchart LR
  Caller[caller]
  Caller --> L1{L1 hit?}
  L1 -- yes --> Return[return]
  L1 -- no --> L2{L2 hit?}
  L2 -- yes --> Backfill[backfill L1] --> Return
  L2 -- no --> Factory[run factory] --> WriteBoth[write L1+L2] --> Return
```

Additional methods on a multi-tier cache:

| Method                | Effect                                       |
| --------------------- | -------------------------------------------- |
| `getL1()` / `getL2()` | Access per-tier cache for advanced ops       |
| `promote(key)`        | Force entry into L1 (preload after eviction) |
| `demote(key)`         | Force entry out of L1 (keep in L2 only)      |
| `sync()`              | Pull recently-changed L2 entries into L1     |
| `getTierStats(tier)`  | Stats per tier — useful for hit-ratio tuning |

## Decorators

### `@Cacheable(options)` — read-through cache

```typescript
import { Cacheable, CacheKey } from '@omnitron-dev/titan-cache';

@Public()
@Cacheable({ cacheName: 'users', keyPrefix: 'u', ttl: 300, tags: ['users'] })
async findById(@CacheKey() id: string): Promise<User> {
  return this.repo.findById(id);
}
```

| Field           | Type                                                |
| --------------- | --------------------------------------------------- |
| `cacheName?`    | `string` — which cache to use                       |
| `keyPrefix?`    | `string`                                            |
| `keyGenerator?` | `(...args) => string`                               |
| `ttl?`          | `number` (seconds)                                  |
| `tags?`         | `string[] \| ((...args) => string[])`               |
| `condition?`    | `(...args) => boolean` — skip caching if `false`    |
| `unless?`       | `(result) => boolean` — don't cache certain results |
| `compress?`     | `boolean` — force compression regardless of size    |

### `@CacheInvalidate(options)` — bust on mutation

```typescript
@CacheInvalidate({ cacheName: 'users', tags: ['users'] })
async update(id: string, patch: Partial<User>): Promise<User> { /* … */ }
```

| Field               | Type                                                      |
| ------------------- | --------------------------------------------------------- |
| `cacheName?`        | `string`                                                  |
| `keyPattern?`       | `string` — supports `{0}`, `{1}` placeholder substitution |
| `tags?`             | `string[] \| ((...args) => string[])`                     |
| `allEntries?`       | `boolean` — clear the whole cache                         |
| `beforeInvocation?` | `boolean` — invalidate before the method (default: after) |
| `keyGenerator?`     | `(...args) => string[]`                                   |

### `@CachePut(options)` — write-through cache

Extends `@Cacheable` options; runs the method first and writes the
returned value into the cache. Used when the method itself is the
canonical source.

### `@CacheKey()` — parameter marker

When **any** parameter is marked `@CacheKey()`, only marked parameters
contribute to the auto-generated key. Useful when the method has
auxiliary args (logger, span) you don't want in the key.

### Property injection helpers

| Decorator               | Effect                                                                               |
| ----------------------- | ------------------------------------------------------------------------------------ |
| `@InjectCacheService()` | Property decorator — marks the property as the cache service for decorator discovery |
| `@InjectCacheLogger()`  | Property decorator — marks the property as logger for error reporting                |

## Tags-based invalidation

Tags are the right primitive for "drop everything related to user `u_42`":

```typescript
@Cacheable({ cacheName: 'users', tags: (id) => [`user:${id}`] })
async findById(id: string) { /* … */ }

@CacheInvalidate({ cacheName: 'users', tags: (id) => [`user:${id}`] })
async updateProfile(id: string, patch: ProfilePatch) { /* … */ }
```

`invalidateByTags(['user:u_42'])` drops every entry tagged
`user:u_42` regardless of key. In multi-tier deployments, tag
invalidation goes through L2 and reaches every pod.

## Compression

For payloads above `compressionThreshold` (default 1 KB), the cache
optionally compresses with the configured `compressionAlgorithm`.
`'none'` is the default — turn it on only when payloads are
demonstrably large and CPU has budget.

```typescript
TitanCacheModule.forRoot({
  compressionAlgorithm: 'lz4',     // fast, modest ratio
  compressionThreshold: 4_096,
})
```

Set `compress: true` on a specific `@Cacheable` to force compression
regardless of size.

## Warming

Pre-populate hot keys at boot, on schedule, or on demand:

```typescript
TitanCacheModule.forRoot({
  warmingStrategies: [
    { trigger: 'onBoot', loader: async () => loadHotKeys() },
    { trigger: 'cron',   schedule: '*/5 * * * *', loader: async () => loadRecent() },
  ],
})

// Or trigger manually
await this.cache.getCache('users').warm();
```

## Partitions

Slice one cache into namespaced views — useful for per-tenant
isolation without separate caches:

```typescript
const users = this.cache.getCache('users');
const tenantUsers = users.partition(tenantId);
await tenantUsers.get('u_42');     // isolated from other tenants
```

## Stats

```typescript
const stats = users.getStats();
// { hits, misses, hitRate, size, bytes, evictions, ... }
```

Per-cache and global. Reset with `resetStats()` after a warmup or
deployment.

## Anti-patterns

- **Caching everything.** Adds invalidation surface for no benefit on
  cold paths. Cache only what you've measured to be hot.
- **`has + get` pattern.** Race condition (TOCTOU). Use `getOrSet`.
- **Skipping `tags` and relying on `clear()`.** `clear()` is a
  carpet bomb — tags target only what you actually want to drop.
- **Putting `Request`-scoped keys in a `Singleton` cache.** Per-user
  data needs per-user keys (`user:${id}`), not per-process keys.
- **Forgetting `defaultTtl`.** Unbounded entries grow until the
  eviction policy kicks in. Always set a TTL appropriate to your
  data's volatility.

## Tokens

| Token                 | Purpose                                        |
| --------------------- | ---------------------------------------------- |
| `CACHE_SERVICE_TOKEN` | Resolve `ICacheService`                        |
| `CACHE_DEFAULT_TOKEN` | Resolve the default `ICache` instance directly |
| `CACHE_OPTIONS_TOKEN` | Resolved options bundle                        |
| `getCacheToken(name)` | Token for a named cache (advanced injection)   |

## Lifecycle

`CacheService` does not implement explicit `OnInit` / `OnDestroy` —
cleanup runs through its async `dispose()`, which the framework calls
during container teardown. If you create caches dynamically, call
`deleteCache(name)` when done.

## Inter-module dependencies

- Uses `@omnitron-dev/eventemitter` for cache lifecycle events.
- Optional L2 typically uses an adapter built on
  [`titan-redis`](/docs/titan/modules/redis); the cache module itself does not
  hard-depend on Redis.
- Compresses with Node's `zlib` (built-in) or a pluggable adapter.

## See also

- [`titan-redis`](/docs/titan/modules/redis) — typical L2 backing
- [`titan-lock`](/docs/titan/modules/lock) — prevent thundering-herd writes outside `getOrSet`
- [Best Practices / Performance](/docs/titan/best-practices/performance)

---

---
id: "titan/modules/community"
title: "Community Modules"
---

# Community Modules

<span class="module-badge">community · @your-scope/titan-yourmodule</span>

No community-maintained Titan modules are registered yet. This page
is the canonical home for them as the ecosystem grows.

## What counts as a community module

A community module is a third-party package that:

1. Follows the Titan module conventions — exports a class with
   `.forRoot(...)` (and ideally `.forRootAsync(...)`).
2. Depends on `@omnitron-dev/titan` and (optionally)
   `@omnitron-dev/titan-redis` or other ecosystem modules.
3. Is published to a public registry (npm, GitHub Packages, etc.).
4. Is maintained outside the `omnitron-dev` organisation.

## Adopting a community module — what to check

```mermaid
flowchart LR
  Find[Find a candidate]
  Find --> Audit{Read the source}
  Audit -->|forRoot signature documented| OK1[OK]
  Audit -->|undocumented| Reject1[Reject]
  Audit --> Lifecycle{Honours onInit/onStop?}
  Lifecycle -->|yes| OK2[OK]
  Lifecycle -->|no| Reject2[Reject — resource leaks likely]
  Audit --> Pinning{Pins exact titan core?}
  Pinning -->|no \^/~ range| OK3[OK]
  Pinning -->|exact| Yellow[Yellow — needs updates]
  OK1 --> Try[Try in dev]
  OK2 --> Try
  OK3 --> Try
  Yellow --> Try
  Try --> Decide{Use in prod?}
```

Specifically:

- **Lifecycle compliance.** Does it implement `OnInit` / `OnStart` /
  `OnStop` / `OnDestroy` properly? Lifecycle violations leak
  resources at shutdown.
- **Token isolation.** Does it export typed tokens for all its
  services? Modules that resolve by string keys are fragile.
- **Configuration validation.** Does the `forRoot` options interface
  have a Zod schema, or does it crash at first use on bad
  configuration?
- **Test coverage.** Does the package ship with a test suite that
  exercises the public API?
- **Maintenance.** When was the last commit? Does the maintainer
  respond to issues?

## Publishing your own

Quick summary:

1. **Package convention.** Name it `<your-scope>/titan-<purpose>`
   (e.g. `@acme/titan-stripe`).
2. **Module class.** Export a `XxxModule` class with at least
   `.forRoot(options)`. Add `.forRootAsync(...)` if your
   configuration depends on other providers.
3. **README.** Document the `forRoot` options, the exported
   services and tokens, and a minimal quickstart.
4. **License.** MIT or Apache 2.0 are the friendliest defaults.
5. **Mention us.** Open a PR against `omnitron-dev/omni` adding
   your module to this page. Include the package URL, a one-line
   purpose statement, and a maintenance commitment.

→ **Full step-by-step guide:** [Authoring a module](/docs/titan/modules-system/authoring-modules)
covers the canonical package skeleton, every conventional file,
options + tokens + service + module class + barrel + README, plus
the conventions checklist that distinguishes a polished module
from a thrown-together one.

## Future entries

Once community modules are registered, each gets its own page in
this section with a `<span class="module-badge">community</span>`
chip. The reader knows at a glance the module is third-party and
should be audited before adoption.

For the canonical Titan ecosystem, see the [Official
modules](/docs/titan/modules).

---

---
id: "titan/modules/config"
title: "ConfigModule"
---

# ConfigModule

<span class="module-badge">built-in · @omnitron-dev/titan · /module/config · stable</span>

Layered, validated, hot-reloadable configuration. **Auto-loaded** as
a core module by every Titan application (unless `disableCoreModules:
true`). No extra install required — ships inside `@omnitron-dev/titan`.

## When you need it

Every non-trivial backend has configuration that varies by
environment, secret, deploy target. ConfigModule provides:

- **Layered sources** — file defaults under environment overrides
  under env-var overrides under command-line overrides.
- **Schema validation at boot** — misconfiguration fails immediately
  rather than at the first dependent service call.
- **Hot reload** — change file-based config without restarting (use
  for log levels, feature flags, anything safe to swap live).
- **Typed injection** — `@Config('database.url')` returns the
  schema's type, not `string | undefined`.
- **Atomic reload** — if reload's validation fails, the old config
  stays in place; no half-updated state.

## Quickstart

```typescript
import { z } from '@omnitron-dev/titan/validation';
import { ConfigModule } from '@omnitron-dev/titan/module/config';

const AppConfigSchema = z.object({
  port:     z.number().int().min(1).max(65535),
  database: z.object({ url: z.string().url() }),
  cache:    z.object({ ttlMs: z.number().int().positive().default(60_000) }),
});

@Module({
  imports: [
    ConfigModule.forRoot({
      schema:  AppConfigSchema,
      sources: [
        { type: 'file', path: 'config/default.yaml' },
        { type: 'file', path: 'config/local.yaml', optional: true },
        { type: 'env',  prefix: 'APP_' },
      ],
      validateOnStartup: true,
      watchForChanges:   true,
    }),
  ],
})
class AppModule {}
```

## `IConfigModuleOptions`

| Option              | Type                                         |
| ------------------- | -------------------------------------------- |
| `sources`           | `ConfigSource[]` (priority — last wins)      |
| `schema`            | `AnyZodSchema`                               |
| `environment`       | `string` (default: `process.env.NODE_ENV`)   |
| `validateOnStartup` | `boolean`                                    |
| `watchForChanges`   | `boolean`                                    |
| `cache`             | `{ enabled, ttl? }` — TTL in ms              |
| `strict`            | `boolean` — throw on missing required values |
| `logger`            | `any`                                        |
| `global`            | `boolean`                                    |

> **Priority order matters.** Sources are processed first-to-last;
> later sources override earlier ones during deep merge. Put
> committed defaults first, dev overrides second, environment-
> sourced secrets last.

## `ConfigSource` union

The five source types each have an explicit shape:

```typescript
type ConfigSource =
  | { type: 'file';   path, format?, encoding?, transform?, optional? }
  | { type: 'env';    prefix?, separator?, transform? }
  | { type: 'argv';   prefix? }
  | { type: 'object'; data }                                  // ← `data`, not `value`
  | { type: 'remote'; url, headers?, timeout?, retry? };   // `retry` is a number (attempt count)
```

Files auto-detect format by extension (`json`, `yaml`, `toml`,
`ini`, `env`). Env vars become nested paths via `separator`
(default `'__'`).

→ See [Configuration / Sources](/docs/titan/configuration/sources) for
the full per-source reference.

## Services

| Class                    | Purpose                                                                                   |
| ------------------------ | ----------------------------------------------------------------------------------------- |
| `ConfigService`          | Main API — `get`, `set`, `has`, `getAll`, `getTyped`, `onChange`, `reload`, `validate`, … |
| `ConfigLoaderService`    | Loads from sources at startup                                                             |
| `ConfigValidatorService` | Runs the schema(s)                                                                        |
| `ConfigWatcherService`   | Hot-reload watcher (when `watchForChanges: true`)                                         |

### `ConfigService` API

```typescript
import { Inject, Service } from '@omnitron-dev/titan';
import { ConfigService, CONFIG_SERVICE_TOKEN } from '@omnitron-dev/titan/module/config';

@Service('users@1.0.0')
class UsersService {
  constructor(@Inject(CONFIG_SERVICE_TOKEN) private readonly config: ConfigService) {}

  @Public()
  async findById(id: string) {
    const ttl = this.config.get<number>('cache.ttlMs', 60_000);
    return this.cache.getOrSet(`u:${id}`, () => this.repo.findById(id), { ttl });
  }
}
```

| Method                        | Purpose                                                       |
| ----------------------------- | ------------------------------------------------------------- |
| `get<T>(path, defaultValue?)` | Read a typed value at the dotted path                         |
| `set(path, value)`            | Write at runtime (not persisted)                              |
| `has(path)`                   | Existence check                                               |
| `getAll()`                    | Snapshot of the full config object                            |
| `getTyped<T>(schema, path?)`  | Validated read — runs the schema against `path`-subtree       |
| `validate(schema?)`           | Validate against a schema (returns `IConfigValidationResult`) |
| `reload()`                    | Force re-load all sources; rolls back on validation fail      |
| `onChange(listener)`          | Subscribe to changes; returns `unsubscribe` function          |
| `getMetadata()`               | Source attribution, loaded-at timestamps                      |
| `environment` (getter)        | The configured environment string                             |

## Decorators

```typescript
import {
  Config, InjectConfig, ConfigSchema, Configuration,
  ConfigWatch, ConfigDefaults, ConfigProvider,
} from '@omnitron-dev/titan/module/config';
```

| Decorator                 | Effect                                                 |
| ------------------------- | ------------------------------------------------------ |
| `@Config(path, default?)` | Inject a single value                                  |
| `@InjectConfig()`         | Inject the full `ConfigService`                        |
| `@ConfigSchema(schema)`   | Class-level Zod schema for this class's config subtree |
| `@Configuration(prefix?)` | Bind a class as a typed view of a config subtree       |
| `@ConfigWatch(path)`      | Method runs when the watched path changes              |
| `@ConfigDefaults({...})`  | Provide class-level defaults                           |
| `@ConfigProvider(name)`   | Custom config provider class                           |

```typescript
@Service('users@1.0.0')
class UsersService {
  @Config('cache.ttlMs', 60_000)
  private readonly cacheTtl!: number;

  @InjectConfig()
  private readonly config!: ConfigService;

  @ConfigWatch('cache.ttlMs')
  async onCacheTtlChange(newTtl: number) {
    this.cache.setDefaultTtl(newTtl);
  }
}
```

### Bound configuration class

```typescript
@Configuration('cache')
class CacheConfig {
  @Config('ttlMs', 60_000)    ttlMs!: number;
  @Config('maxItems', 10_000) maxItems!: number;
}

// Inject the typed view directly:
constructor(private readonly cache: CacheConfig) {}
```

`@Configuration` is the most ergonomic pattern for a service that
needs many config values — the typed class doubles as documentation
of what config it consumes.

## Tokens

| Token                            | Purpose                         |
| -------------------------------- | ------------------------------- |
| `CONFIG_SERVICE_TOKEN`           | Inject `ConfigService`          |
| `CONFIG_LOADER_SERVICE_TOKEN`    | Inject `ConfigLoaderService`    |
| `CONFIG_VALIDATOR_SERVICE_TOKEN` | Inject `ConfigValidatorService` |
| `CONFIG_WATCHER_SERVICE_TOKEN`   | Inject `ConfigWatcherService`   |
| `CONFIG_OPTIONS_TOKEN`           | Resolved options bundle         |
| `CONFIG_SCHEMA_TOKEN`            | Global schema                   |

## Source merge + validation pipeline

```mermaid
flowchart LR
  S1[file: default.yaml] --> Merge[deep merge<br/>last source wins]
  S2[file: local.yaml]   --> Merge
  S3[env: APP_*]         --> Merge
  S4[argv: --app-*]      --> Merge
  Merge --> Validate{schema?}
  Validate -- valid --> Frozen[(config in service)]
  Validate -- invalid --> Throw[/throw badRequest at boot/]
  Frozen --> Watch{watchForChanges?}
  Watch -- yes --> Watcher[file watchers fire reload]
  Watcher --> Reload[reload sources<br/>+ revalidate]
  Reload -- valid --> Frozen
  Reload -- invalid --> Rollback[restore previous config<br/>throw badRequest]
```

### Atomic reload semantics

`reload()` is **atomic**: when validation fails after a reload, the
service rolls back to the previous config snapshot and throws. You
never see a half-updated config inside `ConfigService.get` mid-call.

```typescript
try {
  await config.reload();
} catch (e) {
  // Validation failed; the previous config is still active.
  // Log and alert; the running app is not broken.
  logger.error({ err: e }, 'config reload validation failed');
}
```

## Multi-environment scaffold

A common production layout for layered config:

**config/**

- `default.yaml` — committed, shared defaults
- `development.yaml` — committed, dev overrides
- `staging.yaml` — committed, staging overrides
- `production.yaml` — committed, prod-safe non-secrets
- `local.yaml` — git-ignored, per-developer

Wire it up:

```typescript
const env = process.env.NODE_ENV ?? 'development';

ConfigModule.forRoot({
  schema:      AppConfigSchema,
  environment: env,
  sources: [
    { type: 'file', path: 'config/default.yaml' },
    { type: 'file', path: `config/${env}.yaml`,  optional: false },
    { type: 'file', path: 'config/local.yaml',    optional: true },
    { type: 'env',  prefix: 'APP_' },
    { type: 'argv', prefix: 'app-' },
  ],
  validateOnStartup: true,
  watchForChanges:   env !== 'production',
});
```

Priority (low → high): defaults → env-specific → local-dev →
process env → CLI argv. Secrets always come from `env` or `argv`,
never from any committed file.

## Remote config source

The `'remote'` source fetches from an HTTP endpoint at boot:

```typescript
sources: [
  { type: 'file', path: 'config/default.yaml' },
  { type: 'remote',
    url:     'https://config-server.internal/api/configs/my-api',
    headers: { authorization: `Bearer ${process.env.CONFIG_TOKEN}` },
    timeout: 5_000,
  },
  { type: 'env', prefix: 'APP_' },
],
```

The loader retries automatically on transient failures:

- Up to **3 attempts** total per remote source.
- Exponential backoff with ±30% jitter (200 ms base, capped at 2 s).
- Only **retryable** failures are retried (5xx, 408/429, network
  errnos, request-timeout `AbortError`); a 4xx or a 2xx-with-bad-body
  fast-fails without retrying.
- After exhaustion, throws — boot aborts.

> The retry policy (attempt count and backoff) is **fixed in the
> loader**, not configurable per-source. `IRemoteConfigSource` carries
> an optional `retry?: number` field, but the current loader does not
> read it.

For consumers that may be partitioned from the config server,
combine a remote source with a file fallback:

```typescript
sources: [
  { type: 'file',   path: 'config/default.yaml' },
  { type: 'remote', url, optional: true },     // skipped on failure
],
```

When `optional: true`, a remote-source failure is logged and the
boot continues with file defaults only.

`optional` means something narrower for a **file** source: the file may be
absent, but a file that exists and cannot be parsed or read still throws. See
[what `optional` forgives](/docs/titan/configuration/sources#what-optional-forgives).

## Validation error UX

Schema failure on boot raises `Errors.badRequest('Configuration
validation failed', { errors: [...] })` where each error is:

```typescript
interface ConfigValidationError {
  path:     string;       // e.g., 'database.url'
  message:  string;       // human-readable
  expected?: string;
  received?: string;
}
```

For operator-friendly boot diagnostics, catch and format:

```typescript
try {
  await Application.create({ modules: [AppModule] });
} catch (e) {
  if (e?.code === 'BAD_REQUEST' && e.details?.errors) {
    console.error('Configuration is invalid:');
    for (const err of e.details.errors) {
      console.error(`  ${err.path}: ${err.message}`
        + (err.expected ? ` (expected ${err.expected}, got ${err.received})` : ''));
    }
    process.exit(1);
  }
  throw e;
}
```

## Deep-merge edge cases

The merger is recursive but predictable:

| Case                              | Behaviour                                              |
| --------------------------------- | ------------------------------------------------------ |
| Object × Object                   | Recursive merge, last-write-wins per key               |
| Array × Array                     | **Replace**, not concat (avoids accumulating defaults) |
| Primitive × undefined             | Keep the primitive                                     |
| Primitive × null                  | `null` wins — explicit "remove" semantics              |
| Object × Primitive                | Primitive replaces the object (type change is allowed) |
| Forbidden keys (`__proto__` etc.) | Silently skipped — prototype-pollution guard           |

The forbidden-key guard is non-configurable; the keys are
`__proto__`, `constructor`, `prototype`. Any source that tries to
set them is ignored.

## Cache semantics

```typescript
ConfigModule.forRoot({
  // ...
  cache: { enabled: true, ttl: 5_000 },
});
```

When enabled, `get(path)` results are cached for `ttl` ms — useful
when many components read the same value on every request. The
cache is cleared on every successful `reload()` and on `set()`
to the affected path.

Skip the cache for paths that change often via `set()` — the
clear-on-write hits the entire cache, not just the changed path,
so per-write thrashing can defeat the optimisation.

## Strict mode

```typescript
ConfigModule.forRoot({ strict: true });
```

When `strict: true`, `get(path)` without a `defaultValue` throws
on a missing path instead of returning `undefined`. Combines well
with `validateOnStartup: true` to make every missing required value
loud at boot rather than silent at first-use.

## Conventional layout

```typescript
ConfigModule.forRoot({
  schema: AppConfigSchema,
  sources: [
    { type: 'file', path: 'config/default.yaml' },                  // committed defaults
    { type: 'file', path: 'config/local.yaml', optional: true },    // git-ignored dev overrides
    { type: 'env',  prefix: 'APP_' },                               // production secrets / per-pod overrides
  ],
  validateOnStartup: true,
  watchForChanges:   process.env.NODE_ENV !== 'production',
})
```

## Watcher cascade

When `watchForChanges: true`, the `ConfigWatcherService` opens an
fs-watch on every file source. On change:

```mermaid
sequenceDiagram
  participant FS as fs.watch
  participant W as ConfigWatcherService
  participant S as ConfigService
  participant L as Change listeners

  FS-->>W: change event
  W->>S: reload()
  S->>S: load sources, validate
  alt validation passes
    S->>S: replace config
    S->>L: notifyChangeListeners({path, oldValue, newValue, source: 'reload'})
    L->>L: @ConfigWatch handlers fire
  else validation fails
    S->>S: rollback to oldConfig
    S->>S: throw badRequest
  end
```

Watch is debounced internally — a burst of changes (e.g., editor
save with multiple write events) coalesces to one reload.

## Lifecycle

| Hook        | What it does                                    |
| ----------- | ----------------------------------------------- |
| `onInit`    | Load all sources in order; deep-merge; validate |
| `onStart`   | If `watchForChanges`, start file watchers       |
| `onStop`    | `watcher.unwatch()`                             |
| `onDestroy` | `watcher.unwatch()` again (idempotent)          |

If validation fails in `onInit`, the app aborts boot with the
underlying error wrapped — no partial start.

## Anti-patterns

- **Secrets in committed files.** `default.yaml` should not
  contain production secrets. Use env vars or a secret manager.
- **No schema.** Without a schema, misconfiguration surfaces at the
  first dependent service call — typically in production, in a
  rare code path. Schema-validate at boot.
- **Hot-reloading connection strings.** A live database URL swap
  cannot rebind an open connection pool. Reserve hot-reload for
  log levels, feature flags, and similar safe-to-swap values.
- **Reading `process.env` in services.** Defeats the layered
  source merge and breaks tests. Read through `ConfigService` /
  `@Config`.
- **Mutating `getAll()` output.** It's a snapshot — mutations
  don't propagate and surprise the next reader. Use `set(path,
  value)` for runtime writes.
- **`watchForChanges: true` in containers where files don't
  change.** The fs watcher idle-runs forever doing nothing.
  Disable in production unless you actually deploy config to the
  pod's filesystem.
- **Treating `set()` as persistent.** It modifies the in-memory
  config only. The next `reload()` reverts it. For persistent
  changes, edit the source file.

## See also

- [Configuration / Overview](/docs/titan/configuration/overview) —
  conceptual guide
- [Configuration / Sources](/docs/titan/configuration/sources) — per-
  source shape and merge order
- [Configuration / Validation](/docs/titan/configuration/validation) —
  schema patterns and error handling
- [Configuration / Hot Reload](/docs/titan/configuration/hot-reload) —
  `config:changed` event flow
- [Options patterns](/docs/titan/modules/options-patterns) — wiring other modules
  through `ConfigService` via `forRootAsync`
- [Tokens & RPC reference](/docs/titan/modules/tokens-reference) — token inventory

---

---
id: "titan/modules/database"
title: "titan-database"
---

# titan-database

<span class="module-badge">official · @omnitron-dev/titan-database · stable</span>

Kysely-based typed query builder with decorator-driven repository
configuration, declarative migrations, row-level security, plugin
system (soft-delete / timestamps / audit), multi-dialect (Postgres /
MySQL / SQLite), and `AsyncLocalStorage`-based transaction context so
nested calls participate in the active transaction without parameter
threading.

```bash
pnpm add @omnitron-dev/titan-database
```

## Quickstart

### Single connection

```typescript
import { TitanDatabaseModule } from '@omnitron-dev/titan-database';

@Module({
  imports: [
    TitanDatabaseModule.forRoot({
      connection: {
        dialect:      'postgres',
        connection:   env.DATABASE_URL,
        pool:         { min: 2, max: 20 },
        migrationsPath: './migrations',
        coerceBigint: true,
      },
    }),
  ],
})
class AppModule {}
```

### SQLite

Install the optional `better-sqlite3` peer when using the SQLite dialect.
`DatabaseModule` opens SQLite on the first connection. For a dedicated local
journal, the `@omnitron-dev/titan-database/sqlite` entry point exposes the same
checked driver directly.

```ts
import { openSqliteDatabase } from '@omnitron-dev/titan-database/sqlite';

const database = openSqliteDatabase(':memory:');
try {
  const rows = database.prepare('select 1 as ready').all();
  console.log(rows);
} finally {
  database.close();
}
```

The native binding requires Node-API 10. Omnitron needs this capability for its
durable state and logs. On an incompatible Node runtime, opening SQLite raises
a JavaScript error before loading the binding. PostgreSQL and MySQL users can
use their own drivers independently.

### Multiple named connections

```typescript
TitanDatabaseModule.forRoot({
  connections: {
    primary: { dialect: 'postgres', connection: env.PRIMARY_URL },
    analytics: { dialect: 'postgres', connection: env.ANALYTICS_URL },
  },
})
```

### `forFeature` — repository registration

```typescript
@Module({
  imports: [
    TitanDatabaseModule.forRoot({ connection: { dialect: 'postgres', connection: env.DATABASE_URL } }),
    TitanDatabaseModule.forFeature([UsersRepository, OrdersRepository]),
  ],
  providers: [UsersService],
})
class UsersModule {}
```

`forFeature` registers repository classes, wires them into the
container under per-repo tokens, and ensures decorator-driven plugins
(soft-delete, timestamps, audit) bind correctly at boot.

### Async configuration

```typescript
TitanDatabaseModule.forRootAsync({
  imports:    [ConfigModule],
  useFactory: (config: ConfigService) => ({
    connection: {
      dialect:    'postgres',
      connection: config.get('database.url'),
      pool:       config.get('database.pool'),
    },
  }),
  inject: [ConfigService],
})
```

## Read-only component schema readiness

Use `@omnitron-dev/titan-database/readiness` when a component must verify the
PostgreSQL schema it consumes before accepting traffic. This optional entry
exports `requirePostgresSchema`, `SchemaReadinessError`, `requiredColumn`,
`nullableColumn` and their contract types. It borrows an existing pool, performs
catalog checks in a **read-only, repeatable-read transaction**, then releases its
connection. Importing it does not create a pool, load a PostgreSQL driver, build a
DI container or run migrations.

Omnitron applications can import the identical API through
`@omnitron-dev/omnitron/readiness`; they do not need to depend on Kysera or reach
into Titan internals. Choose the public entry for the layer your application uses.

### Declare what this component requires

The application owns its driver and pool. This example uses `pg`; install that
driver when it is not already an application dependency.

```typescript
import { Pool } from 'pg';
import {
  requirePostgresSchema,
  requiredColumn,
  nullableColumn,
  type PostgresSchemaRequirement,
} from '@omnitron-dev/omnitron/readiness';

const notificationSchema = {
  name: 'notification-worker',
  schema: 'public',
  tables: {
    notification_jobs: {
      insert: true,
      columns: {
        id: requiredColumn('uuid'),
        payload: requiredColumn('jsonb'),
        state: {
          ...requiredColumn('text'),
          default: true,
          defaultExpression: "'pending'::text",
        },
        processed_at: nullableColumn('timestamptz'),
      },
      unique: [['id']],
    },
  },
} satisfies PostgresSchemaRequirement;

export async function openWorkerPool(connectionString: string): Promise<Pool> {
  const pool = new Pool({ connectionString, connectionTimeoutMillis: 2_000 });
  try {
    await requirePostgresSchema(pool, notificationSchema, {
      statementTimeoutMs: 10_000,
      lockTimeoutMs: 1_000,
    });
    return pool;
  } catch (error) {
    await pool.end();
    throw error;
  }
}
```

Call this during the component's normal startup, after the
[stack-owned migration phase](/docs/omnitron/migrations#stack-owned-migrations)
and before serving requests or consuming work. Continue using the returned pool;
the application closes it on shutdown. The checker itself leaves a borrowed pool
open. Avoid starting the component's own copy of a shared migration registry:
a later, unrelated migration must not force an older compatible worker to become
a schema writer.

### Compatibility contract

`PostgresSchemaRequirement` names a component, an optional schema and the tables
and routines it needs. Without `schema`, PostgreSQL's `current_schema()` on the
borrowed connection selects the namespace. Declare the required subset rather
than an entire historical database snapshot.

| Contract                                      | What is checked                                                                                                                                           |
| --------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `PostgresTableRequirement.columns`            | Required physical table and exact declared column shapes; views and foreign tables do not satisfy it                                                      |
| `PostgresColumnRequirement.type` / `nullable` | PostgreSQL catalog type name and exact nullability; helpers set the nullability flag                                                                      |
| `typeSchema`                                  | Type namespace; defaults to `pg_catalog`, and can name a schema containing an enum or domain                                                              |
| `formattedType`                               | Optional exact `format_type` spelling, such as `numeric(12,2)` or `character varying(80)`                                                                 |
| `default` / `defaultExpression`               | Presence of a default, or its exact `pg_get_expr` representation when its meaning matters                                                                 |
| `generated`                                   | `false` requires a writable column; a string requires the exact generated expression                                                                      |
| `insert: true`                                | Declared columns must be writable unless an expression is explicitly required; extra non-null columns must provide a default, identity or generated value |
| `unique`                                      | Ordered whole-table unique key columns, enforced immediately by a valid unique index; primary/unique constraints and standalone indexes share this check  |
| `foreignKeys`                                 | Ordered local/referenced columns, referenced table in the checked schema, and exact delete/update actions; defaults are `no action`                       |

Compatible extra tables and nullable/defaulted columns are allowed. A read-only
projection can omit the `generated` distinction; insert contracts default to
non-generated declared columns.

A standalone `CREATE UNIQUE INDEX` can enforce the same key as a primary/unique
constraint and can support a foreign key. Readiness inspects that enforcing index
in PostgreSQL's catalog: it must be unique, valid, ready, live and immediate, with
no predicate or expression. `INCLUDE` columns carry payload and are excluded from
the ordered key. Partial, invalid, unfinished, dropping or deferrable indexes do
not satisfy `unique`. Foreign keys are checked independently. This does not
require changing migrations to turn an existing valid index into a constraint.
See [PostgreSQL index metadata](https://www.postgresql.org/docs/current/catalog-pg-index.html).

These checks do not prove arbitrary check constraints, index performance, query
permissions, row-level policies or every possible PostgreSQL invariant.

### Routines and triggers

`routines` binds each ordinary scalar function to its exact ordered
`argumentTypes` (no arguments by default), SHA-256 of `pg_proc.prosrc`, language
(`sql` or `plpgsql`), volatility (`i`, `s` or `v`) and invoker security. Variadic,
default-argument and set-returning functions cannot satisfy this contract. An
unrelated overload does not count. Optional `returnType` uses `format_type`:
for example, `bigint`, whereas a column's catalog `type` is `int8`.

Table `triggers` bind the function name, reviewed body hash and PostgreSQL
`tgtype` bitmask. The trigger must be enabled for ordinary execution, apply
without a `WHEN` or column filter, have no trigger arguments and invoke an
invoker-security PL/pgSQL function in the checked schema returning `trigger`.
The `immutable` shorthand declares the row-level `BEFORE UPDATE OR DELETE`
trigger shape (`type: 27`); its reviewed function body determines the actual
rejection behavior.

Derive `bodySha256` from the exact function body supplied by reviewed migration
source. Do not derive an expected hash from the live function being checked:
that would accept the drift this contract is intended to detect. The checker
compares properties and bytes; it does not analyze arbitrary SQL semantics.

For routines and trigger functions, `configuration` is the exact set of
`pg_proc.proconfig` settings, empty by default. `schemaSearchPath: true` additionally
requires `pg_catalog, <checked schema>, pg_temp` using PostgreSQL identifier
quoting. Other intentional settings must still be declared. This binds name
resolution to the component's namespace without trusting a caller's mutable
search path; it is not a general SQL-security proof.

### Deadlines and failure reporting

The optional third argument is `PostgresReadinessOptions`:

| Option               | Default   | Boundary                  |
| -------------------- | --------- | ------------------------- |
| `statementTimeoutMs` | 10,000 ms | Each PostgreSQL statement |
| `lockTimeoutMs`      | 1,000 ms  | Each PostgreSQL lock wait |

Both accept positive integer milliseconds within PostgreSQL's supported range.
The settings are transaction-local and are restored afterward. They are **not**
a total-operation deadline. Set a finite pool acquisition timeout separately,
such as `pg`'s `connectionTimeoutMillis`; the pool owner controls acquisition.

A schema mismatch throws `SchemaReadinessError` with stable
`code: 'database_schema_incompatible'`, the component name and individual
`violations`. Authentication, connection and PostgreSQL timeout failures retain
their own errors. Report that actual failure through the application's normal
health/diagnostic path; do not translate an unavailable database into “schema is
compatible.” The check applies no DDL, writes no migration journal and never
repairs the schema automatically.

## `DatabaseModuleOptions`

| Option        | Type                                                             |
| ------------- | ---------------------------------------------------------------- |
| `connection`  | `DatabaseConnection` — single connection                         |
| `connections` | `Record<string, DatabaseConnection>` — multiple named            |
| `kysera`      | `{ core?, repository?, plugins? }` — Kysera integration config   |
| `plugins`     | `PluginsConfiguration` — global soft-delete / timestamps / audit |

## `DatabaseConnection`

| Field             | Type                                                                         |
| ----------------- | ---------------------------------------------------------------------------- |
| `name?`           | `string`                                                                     |
| `dialect`         | `'postgres' \| 'mysql' \| 'sqlite'`                                          |
| `connection`      | `string \| ConnectionConfig` — URL or config object                          |
| `pool?`           | `{ min, max, acquireTimeoutMillis, idleTimeoutMillis }`                      |
| `debug?`          | `boolean`                                                                    |
| `plugins?`        | `string[]` — plugin names                                                    |
| `migrationsPath?` | `string`                                                                     |
| `seedsPath?`      | `string`                                                                     |
| `coerceBigint?`   | `boolean` — parse PG `BIGINT` as JS `number` if safe (default `true` for PG) |

## Repository pattern — `TransactionAwareRepository<DB, Table>`

```typescript
import { Repository, SoftDelete, Timestamps, Audit } from '@omnitron-dev/titan-database';
import { TransactionAwareRepository } from '@omnitron-dev/titan-database';

interface Database {
  users: UsersTable;
}

@Repository('users')
@SoftDelete({ column: 'deleted_at' })
@Timestamps({ createdAt: 'created_at', updatedAt: 'updated_at' })
@Audit({ table: 'audit_logs', captureOldValues: true })
export class UsersRepository extends TransactionAwareRepository<Database, 'users'> {
  async findByEmail(email: string) {
    return this.executor.selectFrom('users')
      .where('email', '=', email)
      .selectAll()
      .executeTakeFirst();
  }
}
```

### Inherited methods

| Method                                                  | Purpose                                              |
| ------------------------------------------------------- | ---------------------------------------------------- |
| `findById(id)`                                          | Fetch by primary key (`null` if missing)             |
| `findByIds(ids[])`                                      | Fetch many by primary key                            |
| `create(data)` / `createMany(data[])`                   | Insert one / many (returns inserted rows)            |
| `update(id, data)`                                      | Update by id (returns row or `null`)                 |
| `delete(id)`                                            | Hard delete by id (returns `boolean`)                |
| `softDelete(id)` / `restore(id)`                        | Soft-delete / restore (requires `@SoftDelete`)       |
| `list(options?)`                                        | Offset-paginated list → `OffsetPaginatedResult`      |
| `findWhere(where, options?)`                            | Dynamic `WHERE` (via `@kysera/repository` operators) |
| `findOneWhere(where)` / `countWhere(where)`             | Single match / count by `WHERE`                      |
| `updateWhere(where, data)` / `deleteWhere(where)`       | Bulk update / delete by `WHERE` (returns count)      |
| `exists(id)` / `count()`                                | Existence check / row count                          |
| `upsert(data, options)` / `upsertMany(data[], options)` | Insert-or-update                                     |

### Protected accessors

| Member             | Purpose                                        |
| ------------------ | ---------------------------------------------- |
| `executor`         | Current executor — transaction-aware           |
| `inTransaction`    | `true` if running inside a transaction context |
| `transaction`      | Current `Transaction<DB>` if any               |
| `hasSoftDelete`    | Plugin flag (set by `@SoftDelete`)             |
| `softDeleteColumn` | Column name (default `'deletedAt'`)            |

## `DatabaseManager`

```typescript
import { DATABASE_MANAGER, type IDatabaseManager, runInTransaction } from '@omnitron-dev/titan-database';

interface AppDB {
  users: { id: number; email: string };
}

@Service({ name: 'reports' })
class ReportsService {
  constructor(@Inject(DATABASE_MANAGER) private readonly db: IDatabaseManager) {}

  @Public()
  async summary() {
    // getConnection() is async, and takes your schema as its type argument.
    // Without one it is Kysely<unknown>, which accepts no table name.
    const conn = await this.db.getConnection<AppDB>();
    return runInTransaction(conn, async () => {
      const users = await conn.selectFrom('users').selectAll().execute();
      return users;
    });
  }
}
```

| Method                                        | Purpose                                                   |
| --------------------------------------------- | --------------------------------------------------------- |
| `getConnection(name?)`                        | `async` — named or default plugin-aware `Kysely<unknown>` |
| `getExecutor(name?, plugins?)`                | `async` — plugin-aware Kysera executor                    |
| `getPool(name?)` / `getPoolMetrics(name?)`    | Raw driver pool / pool stats                              |
| `isConnected(name?)` / `getConnectionNames()` | Connection state / registered names                       |
| `close(name?)` / `closeAll()`                 | Async cleanup of one / every connection (on shutdown)     |

Transactions are run through the standalone `runInTransaction(conn, fn, options?)`
helper (below), **not** a method on the manager — pass it a connection from
`getConnection()`.

## Transaction context

The module uses `AsyncLocalStorage` to track the active transaction
per request. Repository calls inside `runInTransaction` automatically
use the transaction; no plumbing required.

```typescript
import {
  runInTransaction,
  getExecutor,
  getCurrentTransaction,
  isInTransactionContext,
} from '@omnitron-dev/titan-database';

await runInTransaction(db, async () => {
  await this.usersRepo.create({ email: 'ada@example.com' });   // uses the transaction
  await this.auditRepo.create({ kind: 'user.created' });        // same transaction
}, { name: 'signup' });
```

### Helpers

| Function                             | Purpose                                                  |
| ------------------------------------ | -------------------------------------------------------- |
| `runInTransaction(db, fn, options?)` | Open a new transaction and run `fn` inside it            |
| `getExecutor(db)`                    | Return either current transaction or the base connection |
| `getCurrentTransaction()`            | Current transaction or `undefined`                       |
| `isInTransactionContext()`           | Boolean check                                            |
| `getTransactionContext()`            | Full context: depth, started-at, name, connection name   |

## Decorators

### Repository / plugins

```typescript
import { Repository, SoftDelete, Timestamps, Audit, Migration }
  from '@omnitron-dev/titan-database';
```

| Decorator                                                                                     | Effect                                                                                                                                                                      |
| --------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `@Repository(table \| config)`                                                                | Mark a class as a repository; config: `{ table, connection?, softDelete?, timestamps?, audit?, schema?, validate?, plugins? }` (RLS is configured separately via `@Policy`) |
| `@SoftDelete({ column?, includeDeleted?, tables? })`                                          | Soft-delete behaviour — column defaults to `'deletedAt'`                                                                                                                    |
| `@Timestamps({ createdAt?, updatedAt? })`                                                     | Auto-managed timestamp columns                                                                                                                                              |
| `@Audit({ table?, captureOldValues?, captureNewValues? })`                                    | Row-level audit logging                                                                                                                                                     |
| `@Migration({ version, description?, dependencies?, connection?, transactional?, timeout? })` | Mark a class as a migration                                                                                                                                                 |

### Row-level security

```typescript
import { Policy, Allow, Deny, Filter, BypassRLS } from '@omnitron-dev/titan-database';
```

| Decorator                                         | Effect                                     |
| ------------------------------------------------- | ------------------------------------------ |
| `@Policy({ table?, skipFor?, defaultPolicy? })`   | Class-level RLS configuration              |
| `@Allow({ operations: [...], priority?, name? })` | Allow rule for the listed operations       |
| `@Deny({ operations: [...], priority?, name? })`  | Deny rule (evaluated before allow)         |
| `@Filter({ operations?, name? })`                 | Method returns a WHERE-clause predicate    |
| `@BypassRLS()`                                    | Skip RLS (requires admin / system context) |

Example:

```typescript
@Repository('orders')
@Policy({ skipFor: ['admin'] })
class OrdersRepository extends TransactionAwareRepository<Database, 'orders'> {
  @Filter({ operations: ['select'] })
  tenantFilter(ctx: ExecutionContext) {
    return { tenant_id: ctx.tenantId };
  }

  @Allow({ operations: ['insert', 'update'] })
  ownerWrite(ctx: ExecutionContext, row: Row) {
    return row.user_id === ctx.auth.userId;
  }
}
```

### Injection

| Decorator                      | Effect                       |
| ------------------------------ | ---------------------------- |
| `@InjectConnection(name?)`     | Inject a named connection    |
| `@InjectDatabaseManager()`     | Inject the `DatabaseManager` |
| `@InjectRepository(RepoClass)` | Inject a repository instance |

## Migrations

```typescript
import { Migration } from '@omnitron-dev/titan-database';

@Migration({ version: '20260101_001', description: 'create users table' })
export class CreateUsersTable {
  async up(db: Kysely<any>) {
    await db.schema.createTable('users')
      .addColumn('id', 'uuid', (c) => c.primaryKey())
      .addColumn('email', 'text', (c) => c.notNull().unique())
      .addColumn('created_at', 'timestamptz', (c) => c.defaultTo('now()'))
      .execute();
  }

  async down(db: Kysely<any>) {
    await db.schema.dropTable('users').execute();
  }
}
```

`@Migration` records class metadata. The module does not discover or execute
migrations automatically at DI startup; `migrationsPath` does not schedule them.
Choose an explicit registry and deployment command. Omnitron applications use
[`@omnitron-dev/omnitron/migration`](/docs/omnitron/migrations); standalone
Titan applications use `@omnitron-dev/titan-database/migration`.

### One execution engine

Kysera owns execution, connection pinning, locking, history and checksums.
Titan's narrow migration entry point re-exports that engine and adds driver
adapters. There is no second transaction or journal implementation.

`createHardenedRunner` preserves the original Titan journal (`migrations` plus
`migration_checksums`), advisory lock and function-source checksum format.
It defaults to per-migration transactions and fails promptly on lock contention;
set `lockTimeoutMs` to wait. Body, journal and checksum commit together. Legacy
checksums are backfilled on writes under its compatibility policy; read-only
inspection reports missing checksums as unverified. An explicit `checksum`
overrides function-source hashing.

For new applications, Omnitron's `createMigrationRunner` uses explicit defaults
and an inline checksum journal with execution order. `defineSqlMigration` hashes
SQL contents; `transactionMode: 'batch'` makes an entire operation atomic where
the dialect supports it. Preserve existing history and lock identities when
adopting the engine. See [guarantees and migration examples](/docs/omnitron/migrations).

### Transaction context

`runInTransaction` and `getExecutor` propagate only the transaction belonging to
the requested database. Nested work uses Kysera DAL savepoints; a second database
gets an independent transaction. Whole-transaction retries use Kysera Infra and
are opt-in. A failed rollback preserves both causes. Cross-database work is not
a distributed transaction: callers must design compensation explicitly.

## What comes through from `@kysera/*`

This package re-exports about sixty symbols from the `@kysera` family, so they
are importable from `@omnitron-dev/titan-database` and a reader has no reason
to guess that a second package is involved. They are listed here by origin
rather than described — their own documentation is authoritative, and copying
it here would rot.

| From                 | What                                                                                                                                                                                                       |
| -------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `@kysera/core`       | error mapping (`parseDatabaseError`, `ErrorCodes`, `UniqueConstraintError`, `ForeignKeyError`, `NotNullError`), pagination (`paginate`, `paginateCursor`, `applyOffset`, `applyDateRange`, `executeCount`) |
| `@kysera/repository` | `upsert`, `upsertMany`, `atomicStatusTransition`, `ContextAwareRepository` (also exported as `BaseRepository`), and the `WHERE`-operator helpers                                                           |
| `@kysera/infra`      | `withRetry`, `withTransactionRetry`, `CircuitBreaker`, `isTransientError`, `isSerializationError`, `HealthMonitor`, `checkDatabaseHealth`                                                                  |
| `@kysera/rls`        | `defineRLSSchema`, `allow`, `deny`, `filter`, `rlsPlugin`                                                                                                                                                  |
| `@kysera/dialects`   | `getAdapter`, `createDialectAdapter`, and the per-dialect adapters                                                                                                                                         |
| plugins              | `softDeletePlugin`, `timestampsPlugin`, `auditPlugin`                                                                                                                                                      |
| `kysely`             | `Kysely`, `Transaction`, `Selectable`, `Insertable`, `Updateable` (types)                                                                                                                                  |

Three of kysera's error classes are renamed on the way through:
`DatabaseError`, `NotFoundError` and `BadRequestError` are exported as
`KyseraDatabaseError`, `KyseraNotFoundError` and `KyseraBadRequestError`. The
unprefixed names are NOT exported from this package at all — reach for
`NotFoundError` here and the import fails rather than resolving to something
that merely looks right, which is the intended outcome: those names belong to
your application's own error hierarchy or to Titan's, and a database driver's
version of them is a different type.

## Plugin lifecycle ordering

```mermaid
flowchart LR
  Call[repo.create]
  Call --> Plugin1[Timestamps<br/>fill created_at / updated_at]
  Plugin1 --> Plugin2[Audit<br/>capture new values]
  Plugin2 --> SD[SoftDelete<br/>no-op on insert]
  SD --> SQL[INSERT INTO users …]
  SQL --> Done[Return inserted row]
```

For deletes, soft-delete intercepts first: it issues an UPDATE that
sets `deleted_at` instead of a DELETE; the audit plugin captures the
before/after.

## Tokens

| Token                               | Purpose                                  |
| ----------------------------------- | ---------------------------------------- |
| `DATABASE_MANAGER`                  | `DatabaseManager`                        |
| `DATABASE_MODULE_OPTIONS`           | Resolved options bundle                  |
| `DATABASE_CONNECTION`               | Default `Kysely<unknown>`                |
| `DATABASE_HEALTH_INDICATOR`         | Health indicator for `titan-health`      |
| `getDatabaseConnectionToken(name?)` | Token for a named connection             |
| `getRepositoryToken(RepoClass)`     | Token for a specific repository instance |

## Lifecycle

`TitanDatabaseModule` implements:

- `async onStop(app)` — `manager.closeAll()` to release every
  connection. Crucial during dev with file watchers — without this,
  rapid restarts exhaust PG connection slots.

## Adding your own kysera plugins

Plugins go on the CONNECTION, through `kysera.plugins`. They are
applied when the executor is built, so every query on that
connection passes through them — including hand-written Kysely,
which is how applications that do not use the base repository
class get covered:

```typescript
TitanDatabaseModule.forRootAsync({
  useFactory: () => ({
    connection: { /* … */ },
    kysera: {
      plugins: [rlsPlugin({ schema, bypassRoles }), myPlugin()],
    },
  }),
});
```

`kysera.plugins` and the built-in `plugins` block are MERGED, not
alternatives — configuring one of them does not silently drop the
other.

> **Removed: registerTablePlugins**
>
> There used to be a per-table registry documented here, recommended
> for exactly the case above. Plugins registered through it were
> always skipped: `DATABASE_CONNECTION` already resolves to a kysera
> executor, and the reader correctly declines to wrap a connection
> that is already plugin-aware — so the registry's own reader was
> unreachable by construction. It was deleted rather than fixed,
> because `kysera.plugins` is the path that works. If you are
> following an older copy of this page, move the plugin list into
> the module options as above.

## Anti-patterns

- **Manual transaction threading.** Don't pass `tx` as a parameter
  through every layer. Use `runInTransaction`; repository calls
  pick up the transaction from the async context.
- **Naked DELETE on soft-delete tables.** Use the repo's `delete()`
  — the plugin transforms it into an UPDATE setting `deleted_at`.
- **Forgetting `migrationsPath`.** Without it, the migration runner
  has nothing to discover.
- **Using `@BypassRLS` casually.** It exists for system flows
  (cron-driven cleanups, admin scripts) — every use case needs a
  written justification.
- **Sharing one connection across very different workloads.** OLTP
  and analytics traffic on one pool starves each other. Use
  multiple named connections.

## Inter-module dependencies

- Uses [`@kysera/*`](https://github.com/kysera) — core / repository
  / rls / soft-delete / timestamps / audit / executor / infra /
  migrations.
- Peer deps on the drivers you use: `pg`, `mysql2`, `better-sqlite3`.
- Optional health indicator integrates with [`titan-health`](/docs/titan/modules/health).

## See also

- [`titan-health`](/docs/titan/modules/health) — `DatabaseHealthIndicator` exported by this module
- [Best Practices / Performance](/docs/titan/best-practices/performance) — N+1, projection, indexing

## PostgreSQL coordination leases

The optional `@omnitron-dev/titan-database/coordination` entry point exports
`PostgresLeaseStore`, `LeaseLostError`, `POSTGRES_LEASE_SCHEMA_V1`, `LeaseIdentity`,
`LeaseGrant` and `LeaseStore`. It imports Kysely without starting Titan DI, Redis or
an election service. Apply its schema through the migration facade owned by your
application; constructing a store never mutates schema.

```typescript
import { randomUUID } from 'node:crypto';
import type { Kysely } from '@omnitron-dev/titan-database';
import { PostgresLeaseStore } from '@omnitron-dev/titan-database/coordination';

interface BillingDatabase {
  jobs: { id: string; status: 'pending' | 'claimed' };
}

// Retain one incarnation for this controller's lifetime.
const incarnation = randomUUID();

async function claimJob(db: Kysely<BillingDatabase>, instanceId: string, jobId: string) {
  const leases = new PostgresLeaseStore(db);
  const grant = await leases.acquire({ key: 'billing/reconcile', owner: instanceId, incarnation }, 15_000);
  if (!grant) return false;
  try {
    return await leases.withLease(grant, async trx => {
      // All protected database effects must use this transaction.
      const result = await trx.updateTable('jobs')
        .set({ status: 'claimed' })
        .where('id', '=', jobId)
        .where('status', '=', 'pending')
        .executeTakeFirst();
      return result.numUpdatedRows === 1n;
    });
  } finally {
    await leases.release(grant);
  }
}
```

Use one incarnation for a controller lifetime. Repeating an acquisition with the
same identity reconciles a lost reply while the epoch is valid. `renew` returns
`null` after expiry or replacement; it never grants a new epoch. `read` reports only
an unexpired database observation. Release retains the row and epoch.

All contenders need the same writable PostgreSQL authority; its failover must
preserve committed epochs. Mutations request synchronous commit. Lock waits are
bounded to two seconds and SQL statements to five seconds; configure a finite
connection-acquisition timeout on the pool too. Fenced callbacks must be bounded
and perform database work through the supplied transaction. External I/O is not
rolled back or fenced by a SQL transaction: its recipient needs an epoch-aware
admission contract. Never delete lease rows to reset ownership.

---

---
id: "titan/modules/decorators-catalog"
title: "Decorators catalog"
description: "Every decorator exported by every module — what it does, where it comes from."
---

# Decorators catalog

Titan and its ecosystem ship a lot of decorators. This page is the
single-source-of-truth lookup: what each decorator does, which
package to import it from, and which decorators share a name
across packages (so you don't accidentally import the wrong one).

Every entry below is checked against the packages' real exports by a
script, not by hand — a hand check is true on the day it is made and
decays silently afterwards:

```bash
node scripts/check-titan-inventories.mjs ../omni
```

It exits non-zero if this page names a decorator no package exports, or
if the set the root re-exports stops being exactly the nine listed
above.

## Quick lookup by purpose

| You want to…                                    | Reach for                                                      |
| ----------------------------------------------- | -------------------------------------------------------------- |
| Mark a class as DI-injectable                   | `@Injectable()` *(core)*                                       |
| Expose a class as RPC service                   | `@Service('name@version')` *(core)*                            |
| Expose a method on the wire                     | `@Public()` *(core)*                                           |
| Validate a method's args / return               | `@Validate(schema)` *(core)*                                   |
| Require authenticated caller                    | `@RequireAuth()` *(titan-auth)*                                |
| Skip auth for one method                        | `@Public()` *(titan-auth — opts out of auth)*                  |
| Memoise an idempotent method                    | `@Cacheable()` *(titan-cache)*                                 |
| Throttle calls per key                          | `@RateLimit({ key, limit })` *(titan-ratelimit)*               |
| Run at a cron schedule                          | `@Cron('0 * * * *')` *(titan-scheduler)*                       |
| Run on an interval                              | `@Interval(60_000)` *(titan-scheduler)*                        |
| Subscribe to a typed event                      | `@OnEvent('user.created')` *(titan-events)*                    |
| Auto-emit an event from a method                | `@EmitEvent({ event: 'user.created' })` *(titan-events)*       |
| Wrap a method in a distributed lock             | `@WithDistributedLock(key, ttl)` *(titan-lock)*                |
| Instrument a method with RPC counters + latency | `@Metrics(metricName?)` *(titan-metrics)*                      |
| Define a worker process class                   | `@Process()` *(titan-pm)*                                      |
| Define a supervised hierarchy                   | `@Supervisor()` + `@Child()` *(titan-pm)*                      |
| Bind a class to a config subtree                | `@Configuration('cache')` *(config built-in)*                  |
| Read one config value                           | `@Config('cache.ttlMs', 60_000)` *(config built-in)*           |
| Apply RLS to a repository                       | `@Policy(...)` + `@Allow`/`@Deny`/`@Filter` *(titan-database)* |
| Soft-delete a repository's rows                 | `@SoftDelete()` *(titan-database)*                             |
| Auto-add `created_at`/`updated_at`              | `@Timestamps()` *(titan-database)*                             |
| Audit row changes                               | `@Audit()` *(titan-database)*                                  |

## Core decorators

<span class="module-badge">built-in · @omnitron-dev/titan</span>

Imported from `@omnitron-dev/titan/decorators`, which exports all of
them. The package **root** re-exports exactly nine — `Service`,
`Injectable`, `Inject`, `Optional`, `Singleton`, `Transient`,
`Module`, `PostConstruct`, `PreDestroy` — and nothing else, so
`@Public`, `@Validate`, `@Timeout`, `@RateLimit`, `@Cache` and the
rest below must be imported from `@omnitron-dev/titan/decorators`.
Use that subpath consistently in application code.

### DI

| Decorator            | Targets              | Effect                                        |
| -------------------- | -------------------- | --------------------------------------------- |
| `@Injectable(opts?)` | class                | Marks a class as a DI provider                |
| `@Service(name)`     | class                | Marks a class as a Netron RPC service         |
| `@Inject(token)`     | parameter / property | Resolves a token from the container           |
| `@Optional()`        | parameter / property | Allow undefined if the token isn't registered |
| `@Global()`          | class (on Module)    | Makes the module's exports visible everywhere |

### RPC surface

| Decorator                      | Targets           | Effect                                          |
| ------------------------------ | ----------------- | ----------------------------------------------- |
| `@Public(opts?)`               | method / property | Exposes the method on the wire                  |
| `@Contract(class)`             | class             | Binds a Zod-based input/output contract         |
| `@Validate({input, output})`   | method            | Runs Zod schemas around the call                |
| `@NoValidation()`              | method            | Opts out of class-level validation              |
| `@WithValidationOptions(opts)` | class             | Sets default validation behaviour for the class |
| `@ValidateInput(schema)`       | method            | Input-only validation                           |
| `@ValidateOutput(schema)`      | method            | Output-only validation                          |

> The core `@Public()` exposes; the *titan-auth* `@Public()`
> further marks the same method as auth-bypass. They compose:
> `@Public() @RequireAuth()` for an exposed-but-protected method
> is the common pattern.

Core also ships three lightweight **composable annotations** meant
to sit alongside `@Public()` — they write metadata that transports /
modules read, rather than implementing the behaviour themselves:

| Decorator           | Targets | Effect                                       |
| ------------------- | ------- | -------------------------------------------- |
| `@Auth(opts?)`      | method  | Marks the method as requiring authentication |
| `@RateLimit(opts?)` | method  | Attaches rate-limit metadata to the method   |
| `@Cache(opts?)`     | method  | Attaches cache metadata to the method        |

> These share names with the satellite-package decorators
> (`titan-ratelimit.@RateLimit`, `titan-pm.@Cache`, etc.). The core
> ones are bare metadata annotations; see the collision table below.

### Lifecycle

| Decorator          | Targets | Effect                                  |
| ------------------ | ------- | --------------------------------------- |
| `@PostConstruct()` | method  | Run after construction, before `onInit` |
| `@PreDestroy()`    | method  | Run during `onDestroy`                  |

For full hooks (`onInit`, `onStart`, `onStop`, `onDestroy`)
implement the interfaces instead — they're more discoverable than
decorators. See [Lifecycle reference](/docs/titan/modules/lifecycle-reference).

### Method utilities

Most utility decorators are method interceptors that compose
freely with other decorators.

| Decorator                           | Effect                                                      |
| ----------------------------------- | ----------------------------------------------------------- |
| `@Memoize`                          | Cache the result of a function in-memory by arg fingerprint |
| `@Retry({ attempts, delay })`       | Retry on throw with linear delay                            |
| `@Retryable({...})`                 | More configurable retry (backoff curve, predicate)          |
| `@Timeout({ ms })`                  | Reject if the method doesn't resolve within `ms`            |
| `@Log({...})`                       | Log entry / exit / error                                    |
| `@Monitor({...})`                   | Emit metric samples around the call                         |
| `@Deprecated({ message, version })` | Warn callers; metadata for tooling                          |

> The core `@Timeout` clashes with `titan-scheduler.@Timeout` —
> distinguish by import path (`@omnitron-dev/titan/decorators` vs
> `@omnitron-dev/titan-scheduler`).

## Built-in module decorators

### `config`

<span class="module-badge">built-in · @omnitron-dev/titan/module/config</span>

| Decorator                 | Targets  | Effect                                             |
| ------------------------- | -------- | -------------------------------------------------- |
| `@Config(path, default?)` | property | Inject one config value at a dotted path           |
| `@InjectConfig()`         | property | Inject the whole `ConfigService`                   |
| `@Configuration(prefix?)` | class    | Bind the class as a typed view of a config subtree |
| `@ConfigSchema(schema)`   | class    | Class-level Zod schema for the bound subtree       |
| `@ConfigWatch(path)`      | method   | Method runs when the watched path changes          |
| `@ConfigDefaults({...})`  | class    | Provide class-level defaults                       |
| `@ConfigProvider(name)`   | class    | Custom config provider class                       |

```typescript
@Configuration('cache')
class CacheConfig {
  @Config('ttlMs', 60_000)  ttlMs!: number;
  @Config('maxItems', 1024) maxItems!: number;
}
```

### `logger`

<span class="module-badge">built-in · @omnitron-dev/titan/module/logger</span>

The logger module exports no decorators — instances are injected
via token / interface (`LOGGER_SERVICE_TOKEN`, `LOGGER_TOKEN`).
See [`logger`](/docs/titan/modules/logger).

## Official module decorators

### `titan-auth`

<span class="module-badge">official · @omnitron-dev/titan-auth</span>

| Decorator                         | Targets        | Effect                                             |
| --------------------------------- | -------------- | -------------------------------------------------- |
| `@RequireAuth(opts?)`             | method         | Caller must present a valid JWT                    |
| `@RequireServiceAuth()`           | method         | Caller must hold the `service` role                |
| `@RequireAdminAuth()`             | method         | Caller must hold the `admin` role                  |
| `@RequireRole(roles[], message?)` | method         | Caller must hold at least one of the listed roles  |
| `@Public()`                       | class / method | Auth bypass — explicitly mark as anonymous-allowed |

> **Two `@Public()` decorators exist.** The core one exposes the
> method on Netron. The titan-auth one opts it out of authentication.
> Import path determines which: `@omnitron-dev/titan/decorators`
> (expose) vs `@omnitron-dev/titan-auth` (auth bypass).

### `titan-cache`

<span class="module-badge">official · @omnitron-dev/titan-cache</span>

| Decorator                 | Targets   | Effect                                              |
| ------------------------- | --------- | --------------------------------------------------- |
| `@Cacheable(opts?)`       | method    | Memoise return value; key derived from args         |
| `@CachePut(opts?)`        | method    | Always run the method, but update cache with result |
| `@CacheInvalidate(opts?)` | method    | Remove cache entry / key pattern after method runs  |
| `@CacheKey()`             | parameter | Mark one parameter as the cache key seed            |
| `@InjectCacheService()`   | property  | Inject the `ICacheService` directly                 |
| `@InjectCacheLogger()`    | property  | Inject the cache-scoped logger                      |

### `titan-database`

<span class="module-badge">official · @omnitron-dev/titan-database</span>

| Decorator                   | Targets   | Effect                                             |
| --------------------------- | --------- | -------------------------------------------------- |
| `@InjectConnection(name?)`  | parameter | Inject a named database connection                 |
| `@InjectDatabaseManager()`  | parameter | Inject the global `DatabaseManager`                |
| `@SoftDelete(config?)`      | class     | Apply Kysera soft-delete plugin                    |
| `@Timestamps(config?)`      | class     | Apply Kysera timestamps plugin                     |
| `@Audit(config?)`           | class     | Apply Kysera audit plugin                          |
| `@Policy(config?)`          | class     | Declare RLS policy for the repository              |
| `@Allow(rule)`              | method    | RLS rule that ALLOWS access                        |
| `@Deny(rule)`               | method    | RLS rule that DENIES access                        |
| `@Filter(filter?)`          | method    | RLS row-level filter                               |
| `@BypassRLS()`              | method    | Skip RLS for an admin-only operation               |
| `@AutoTransactional(opts?)` | method    | Run the method in a transaction; commit on success |

### `titan-discovery`

<span class="module-badge">official · @omnitron-dev/titan-discovery</span>

`titan-discovery` does not export decorators — it integrates
transparently with Netron via the module's `enableNetronIntegration`
option. Service announce / lookup is automatic.

### `titan-events`

<span class="module-badge">official · @omnitron-dev/titan-events</span>

| Decorator                                      | Targets | Effect                                                             |
| ---------------------------------------------- | ------- | ------------------------------------------------------------------ |
| `@OnEvent(event, opts?)`                       | method  | Subscribe to a named event                                         |
| `@OnceEvent(event, opts?)`                     | method  | Subscribe once, then unsubscribe                                   |
| `@OnAnyEvent(opts?)`                           | method  | Subscribe to every event (firehose — use with care)                |
| `@OnModuleEvent(module, event, opts?)`         | method  | Subscribe across module boundaries                                 |
| `@EmitEvent({ event, mapResult?, mapError? })` | method  | Emits `<event>.success` after a return, `<event>.error` on a throw |
| `@ScheduleEvent(event, opts?)`                 | method  | Emit on a schedule (cron-like, via this module's scheduler)        |
| `@BatchEvents(event, opts?)`                   | method  | Batch handler — invoked once per batch                             |
| `@EventEmitter(opts?)`                         | class   | Marks the class as a typed event source                            |

### `titan-health`

<span class="module-badge">official · @omnitron-dev/titan-health</span>

`titan-health` doesn't export decorators — indicators are
registered as classes via `forFeature([...])`.

### `titan-lock`

<span class="module-badge">official · @omnitron-dev/titan-lock</span>

| Decorator                           | Targets | Effect                                           |
| ----------------------------------- | ------- | ------------------------------------------------ |
| `@WithDistributedLock(key, ttlMs?)` | method  | Acquire a distributed Redis lock around the call |
| `@Lock(key, ttlMs?)`                | method  | Alias of `@WithDistributedLock`                  |

The decorator releases the lock in `finally` — even on throw.

### `titan-metrics`

<span class="module-badge">official · @omnitron-dev/titan-metrics</span>

| Decorator               | Targets | Effect                                                                                                                     |
| ----------------------- | ------- | -------------------------------------------------------------------------------------------------------------------------- |
| `@Metrics(metricName?)` | method  | Auto-track `rpc_requests_total`, `rpc_request_duration_seconds`, `rpc_errors_total` around the call (optional name prefix) |

### `titan-notifications`

<span class="module-badge">official · @omnitron-dev/titan-notifications</span>

| Decorator                         | Targets | Effect                                                   |
| --------------------------------- | ------- | -------------------------------------------------------- |
| `@OnNotification(pattern, opts?)` | method  | Worker-side handler for notifications matching a pattern |

### `titan-pm`

<span class="module-badge">official · @omnitron-dev/titan-pm</span>

| Decorator                    | Targets         | Effect                                          |
| ---------------------------- | --------------- | ----------------------------------------------- |
| `@Process(opts?)`            | class           | Marks the class as a spawnable worker process   |
| `@Public(opts?)`             | method/property | Expose method to the process supervisor / IPC   |
| `@RateLimit(opts)`           | method          | Worker-runtime rate limiter (pm-internal)       |
| `@Cache(opts?)`              | method          | Worker-runtime cache (pm-internal)              |
| `@Validate(opts)`            | method          | Worker-runtime validation (pm-internal)         |
| `@Trace()`                   | method          | Add tracing span around method                  |
| `@Metric(name?)`             | method          | Emit a metric per call                          |
| `@Supervisor(opts?)`         | class           | Class describing a supervised hierarchy         |
| `@Child(opts?)`              | property        | Declare a child within a supervisor             |
| `@Workflow()`                | class           | Workflow orchestration class                    |
| `@Stage(opts?)`              | method          | A stage within a workflow                       |
| `@Compensate(stageName)`     | method          | Compensating action if a stage fails            |
| `@Actor(opts?)`              | class           | Actor-style stateful entity                     |
| `@CircuitBreaker(opts)`      | method          | Circuit-breaker around external calls           |
| `@Idempotent({ key, ttl? })` | method          | Idempotency key-based deduplication             |
| `@HealthCheck(opts?)`        | method          | Marks a method as the process's health check    |
| `@OnShutdown(opts?)`         | method          | Runs the method during graceful worker shutdown |

> `titan-pm` re-exports several decorator names (`@Public`,
> `@RateLimit`, `@Cache`, `@Validate`) for its **worker runtime**.
> They operate within the worker process, not on the supervisor's
> Netron surface. Don't confuse them with the same-named
> decorators in `@omnitron-dev/titan` / `titan-ratelimit` /
> `titan-cache`.

> Self-healing configuration is currently unimplemented. Use the documented
> restart policy for automatic process recovery.

### `titan-ratelimit`

<span class="module-badge">official · @omnitron-dev/titan-ratelimit</span>

| Decorator           | Targets | Effect                                                      |
| ------------------- | ------- | ----------------------------------------------------------- |
| `@RateLimit(opts?)` | method  | Apply rate limit to the method (uses key derived from opts) |
| `@Throttle(rps)`    | method  | Convenience for "N requests per second"                     |

### `titan-redis`

<span class="module-badge">official · @omnitron-dev/titan-redis</span>

`titan-redis` ships only injection decorators — `@InjectRedis()`
for the default client and `@InjectRedisManager()` for the manager
(plus `@RedisCache` / `@RedisLock` / `@RedisRateLimit` helpers). For
token-based injection use `REDIS_MANAGER` or
`getRedisClientToken(name)`. See [`titan-redis`](/docs/titan/modules/redis).

### `titan-scheduler`

<span class="module-badge">official · @omnitron-dev/titan-scheduler</span>

| Decorator                  | Targets | Effect                                                        |
| -------------------------- | ------- | ------------------------------------------------------------- |
| `@Cron(expression, opts?)` | method  | Run on a cron schedule                                        |
| `@Interval(ms, opts?)`     | method  | Run every `ms` milliseconds                                   |
| `@Timeout(ms, opts?)`      | method  | Run once after `ms` milliseconds                              |
| `@Schedulable()`           | class   | Marks the class for scheduler discovery (when not auto-found) |

> `titan-scheduler.@Timeout` schedules a **one-shot deferred call**.
> Core `@Timeout` *aborts* a method if it doesn't return in time.
> Different decorators with the same name — distinguish by import.

### `titan-telemetry-relay`

<span class="module-badge">official · @omnitron-dev/titan-telemetry-relay</span>

No decorators — instantiate `TelemetryRelayService` and wire it
in directly.

## Name collisions — which one to import

When the same decorator name lives in multiple packages, import
path resolves the ambiguity. Quick table:

| Name         | Available in                                                                                     | Default expectation                        |
| ------------ | ------------------------------------------------------------------------------------------------ | ------------------------------------------ |
| `@Public`    | `@omnitron-dev/titan/decorators` (expose) + `titan-auth` (auth bypass) + `titan-pm` (worker IPC) | Pick by purpose; compose if needed         |
| `@Validate`  | `@omnitron-dev/titan/decorators` + `titan-pm`                                                    | Core for Netron methods; pm for workers    |
| `@Timeout`   | `@omnitron-dev/titan/decorators` (abort) + `titan-scheduler` (one-shot)                          | Read the verb — abort vs defer             |
| `@RateLimit` | `@omnitron-dev/titan/decorators` (composable) + `titan-ratelimit` + `titan-pm`                   | Use titan-ratelimit for service methods    |
| `@Cache`     | `@omnitron-dev/titan/decorators` (composable) + `titan-pm` (worker)                              | Compare with `@Cacheable` from titan-cache |

When in doubt, the import path is authoritative.

## Composability rules

Decorators apply **outside-in** — outermost runs last. The framework
order normally arranges:

```typescript
@Public()                  // Netron exposure (outermost)
@RequireAuth()             // auth check
@RateLimit({ ... })        // rate limit
@Validate({ input })       // input validation
@Metrics()                 // instrumentation
@WithDistributedLock(key)  // lock
async create(input) {/*…*/}
```

The runtime order at call-time:

- enter Public → enter auth → enter rate limit → enter validate →
  enter metrics → enter lock → **method body** → exit lock → exit
  metrics → exit validate → exit rate limit → exit auth → exit Public

This is the *only* order that makes sense:

- Exposure outside auth (auth runs after the request enters the surface).
- Rate limit before validate (cheaper to reject early).
- Metrics around the lock (so you measure lock contention).

## Anti-patterns

- **`@Cacheable` on a non-idempotent method.** Cache returns the
  same value forever; the side-effect runs once. Use `@CachePut`
  or `@CacheInvalidate` if you have side-effects.
- **`@RateLimit` without a per-caller key.** All callers share one
  bucket. Include the user / IP / api-key in the key template.
- **`@WithDistributedLock` without a TTL.** A crashed holder pins
  the lock forever. Always set a TTL ≥ the longest credible work
  time.
- **`@OnEvent` handler that throws.** Throws don't propagate
  through the bus by default; the bus may log and continue. Wrap
  in try/catch if you need to ensure recovery.
- **Two `@Public()` decorators on the same method.** Each from a
  different package — they don't conflict, but the duplication is
  noise. Pick one (the one you import explicitly) or import a
  re-export alias.

## See also

- [Decorators / Index](/docs/titan/decorators) — framework-level guide
- [Decorators / Lifecycle](/docs/titan/decorators/lifecycle)
- [Decorators / Method traits](/docs/titan/decorators/method-traits)
- [Tokens & RPC reference](/docs/titan/modules/tokens-reference) — companion lookup for DI tokens
- [Options patterns](/docs/titan/modules/options-patterns) — module configuration recipes

---

---
id: "titan/modules/discovery"
title: "titan-discovery"
---

# titan-discovery

<span class="module-badge">official · @omnitron-dev/titan-discovery · stable</span>

Redis-backed service discovery with TTL-based heartbeats, pub/sub
event broadcasting, automatic Netron integration so clients can
resolve services without hard-coded URLs, atomic Lua-script
registration to avoid races, and exponential-backoff retry on
heartbeat failures.

```bash
pnpm add @omnitron-dev/titan-discovery @omnitron-dev/titan-redis
```

## When you need it

- **Dynamic service topology.** Pods come and go (autoscaling,
  rolling deploys); discovery keeps the routing table current.
- **Service-to-service resolution.** Caller queries
  `users@1.0.0` without knowing which pods host it.
- **Health-aware routing.** Unhealthy nodes drop out of the registry
  automatically when heartbeats stop and the TTL expires.

## Quickstart

```typescript
import { DiscoveryModule } from '@omnitron-dev/titan-discovery';
import { TitanRedisModule } from '@omnitron-dev/titan-redis';

@Module({
  imports: [
    TitanRedisModule.forRoot({ config: { url: env.REDIS_URL } }),
    DiscoveryModule.forRoot({
      heartbeatInterval:       5_000,
      heartbeatTTL:            15_000,
      pubSubEnabled:           true,
      pubSubChannel:           'titan:discovery:events',
      clientMode:              false,
      redisPrefix:             'titan:discovery',
      maxRetries:              3,
      retryDelay:              1_000,
      enableNetronIntegration: true,
    }),
  ],
})
class AppModule {}
```

## `DiscoveryModuleOptions`

`forRoot()` accepts `DiscoveryModuleOptions`, which extends `DiscoveryOptions`
(the heartbeat / pub-sub fields) with the three module-wiring fields
(`redisUrl`, `redisOptions`, `enableNetronIntegration`).

| Option                    | Type                                  | Default                    |
| ------------------------- | ------------------------------------- | -------------------------- |
| `heartbeatInterval`       | `number` (ms)                         | `5_000`                    |
| `heartbeatTTL`            | `number` (ms)                         | `15_000`                   |
| `pubSubEnabled`           | `boolean`                             | `false`                    |
| `pubSubChannel`           | `string`                              | `'titan:discovery:events'` |
| `clientMode`              | `boolean` (no register, only consume) | `false`                    |
| `redisPrefix`             | `string`                              | `'titan:discovery'`        |
| `maxRetries`              | `number`                              | `3`                        |
| `retryDelay`              | `number` (ms)                         | `1_000`                    |
| `redisUrl`                | `string`                              | —                          |
| `redisOptions`            | `any`                                 | —                          |
| `enableNetronIntegration` | `boolean`                             | `true`                     |

> `heartbeatTTL` must be ≥ `3 × heartbeatInterval`. A missed-
> heartbeat ratio of 3 strikes balance between premature ejection
> (`TTL < 2×`) and slow recovery (`TTL > 5×`).

## `DiscoveryService` — the API

```typescript
import { DISCOVERY_SERVICE_TOKEN, type IDiscoveryService }
  from '@omnitron-dev/titan-discovery';

@Service({ name: 'admin@1.0.0' })
class AdminService {
  constructor(@Inject(DISCOVERY_SERVICE_TOKEN) private readonly disco: IDiscoveryService) {}

  @Public()
  async listInstances() {
    return this.disco.findNodesByService('users', '1.0.0');
  }
}
```

### Registration

| Method                                    | Purpose                                |
| ----------------------------------------- | -------------------------------------- |
| `registerNode(nodeId, address, services)` | Manually register this node            |
| `deregisterNode(nodeId)`                  | Manually deregister                    |
| `registerService(service)`                | Announce a new service this node hosts |
| `unregisterService(serviceName)`          | Drop a service from this node          |
| `updateNodeAddress(nodeId, address)`      | Change the reachable URL               |
| `updateNodeServices(nodeId, services)`    | Replace the service list               |
| `updateAddress(address)`                  | Self-update (uses `this.nodeId`)       |
| `updateServices(services)`                | Self-update services                   |

### Discovery

| Method                                      | Returns                               |
| ------------------------------------------- | ------------------------------------- |
| `getActiveNodes()`                          | `Promise<NodeInfo[]>`                 |
| `findNodesByService(serviceName, version?)` | `Promise<NodeInfo[]>`                 |
| `findNodes()`                               | `Promise<NodeInfo[]>` (filtered)      |
| `getAllNodes()`                             | `Promise<NodeInfo[]>` (incl. expired) |
| `isNodeActive(nodeId)`                      | `Promise<boolean>`                    |
| `nodeExists(nodeId)`                        | `Promise<boolean>`                    |
| `getNodeInfo(nodeId)`                       | `Promise<NodeInfo \| null>`           |

### Events

```typescript
this.disco.onEvent((event) => {
  // { type: 'NODE_REGISTERED' | 'NODE_UPDATED' | 'NODE_DEREGISTERED',
  //   nodeId, address, services, timestamp }
});
this.disco.offEvent(handler);
```

When `pubSubEnabled: true`, registration / update events propagate
**immediately** via Redis pub/sub. With pub/sub disabled, peers see
changes only on their next `getActiveNodes` / `findNodesByService`
call — fine for periodic sync, too slow for fast failover.

## Redis key layout

The module owns four namespaces under `redisPrefix` (default
`titan:discovery`):

| Key                                  | Type   | Purpose                                             |
| ------------------------------------ | ------ | --------------------------------------------------- |
| `titan:discovery:nodes:{nodeId}`     | Hash   | Node metadata: address, services, timestamp         |
| `titan:discovery:heartbeat:{nodeId}` | String | TTL-keyed marker — its existence means "node alive" |
| `titan:discovery:index:nodes`        | Set    | Index of all known nodeIds (cleaned on expiry)      |
| `titan:discovery:events` (pub/sub)   | —      | Notification channel for `pubSubEnabled: true`      |

A node is considered active iff its `heartbeat:{nodeId}` key still
exists. The Lua script that publishes a heartbeat refreshes both
the node hash and the heartbeat marker atomically with the same
TTL — no half-state.

## How it works

```mermaid
sequenceDiagram
  participant N as Node
  participant R as Redis
  participant C as Other clients

  N->>R: EVAL REGISTER_HEARTBEAT_SCRIPT<br/>HSET nodes:{id}, SETEX heartbeat:{id}, SADD index:nodes
  R-->>N: OK
  N->>R: PUBLISH titan:discovery:events<br/>{ type: 'NODE_REGISTERED' }
  R-->>C: event
  loop every heartbeatInterval (5s)
    N->>R: EVAL REGISTER_HEARTBEAT_SCRIPT (refresh TTL)
    alt Redis available
      R-->>N: OK
      Note over N: emit 'NODE_UPDATED' if pubSubEnabled
    else transient failure
      N->>N: retry with exponential backoff:<br/>retryDelay × 2^(attempt-1)
      Note over N: up to maxRetries
    end
  end
  Note over N: Process crashes
  R-->>R: heartbeat:{id} TTL expires after 15s
  Note over C: detected on next findNodes()<br/>or via pub/sub eviction event
```

The Lua script makes registration atomic:
`HSET nodes:{id}` + `SETEX heartbeat:{id}` + `SADD index:nodes`
succeed together or not at all. **No partial state in the
registry.**

## Heartbeat retry behaviour

When a heartbeat fails (Redis hiccup, network blip), the service
retries with exponential backoff:

| Attempt | Delay before retry                |
| ------- | --------------------------------- |
| 1       | `retryDelay × 1` (1 s by default) |
| 2       | `retryDelay × 2` (2 s)            |
| 3       | `retryDelay × 4` (4 s)            |
| ...     | `retryDelay × 2^(attempt-1)`      |

After `maxRetries` exhaustions, the heartbeat is dropped and a
warning is logged. The next periodic tick (5 s later by default)
attempts again — so a sustained Redis outage causes node ejection
once the TTL elapses without a successful refresh.

## TTL tuning guide

| Scenario                          | `heartbeatInterval` | `heartbeatTTL` | Why                                            |
| --------------------------------- | :-----------------: | :------------: | ---------------------------------------------- |
| Default (general use)             |         5 s         |      15 s      | Detect a dead pod within 15 s                  |
| Fast failover (e.g., trading)     |         1 s         |       3 s      | 3 s ejection; \~3× Redis ops/s/pod             |
| Quiet IoT fleet                   |         30 s        |      120 s     | Tolerates slow networks; node loss takes 2 min |
| Aggressive autoscaling            |         2 s         |       6 s      | Scale-down decisions reflect topology in 6 s   |
| Sluggish managed Redis (high p99) |         10 s        |      60 s      | Survives Redis latency spikes without ejection |

Rules of thumb:

- **`heartbeatTTL ≥ 3 × heartbeatInterval`** — anything tighter
  ejects healthy nodes during a single Redis hiccup.
- **`heartbeatTTL ≤ 10 × heartbeatInterval`** — anything looser
  is barely better than no heartbeat at all.
- **Pods × `1 / heartbeatInterval` = baseline Redis ops/s.** With
  100 pods at 5 s interval, that's 20 ops/s — trivial. With 100
  pods at 1 s interval, it's 100 ops/s — still cheap but plan
  Redis sizing.

## Multi-region topology

```mermaid
flowchart LR
  subgraph "Region: EU-west"
    P1[pod-eu-1]
    P2[pod-eu-2]
    R_EU[(Redis EU)]
    P1 --> R_EU
    P2 --> R_EU
  end
  subgraph "Region: US-east"
    P3[pod-us-1]
    P4[pod-us-2]
    R_US[(Redis US)]
    P3 --> R_US
    P4 --> R_US
  end
  R_EU -. cross-region replication .-> R_US
```

For multi-region, two patterns work:

1. **One Redis per region, no cross-replication.** Each region
   has its own discovery registry. Clients within a region see
   only local services. Cross-region calls go through an explicit
   ingress, not discovery. Simpler, lower latency, smaller blast
   radius.
2. **Cross-region replicated Redis (or managed Redis Global).**
   Every region sees every node. Watch for clock skew —
   `timestamp` on `NodeInfo` becomes lossy across regions; use
   `findNodesByService` not "newest first" sorting.

Pattern (1) is almost always the right answer unless you have a
strong reason to want a single global view.

## Sentinel / cluster mode

`titan-discovery` consumes whatever `Redis` client
`titan-redis` provides — sentinel and cluster modes work
transparently. The Lua script and pub/sub channel route correctly
in cluster mode because all four keys/channels share the same
prefix (`titan:discovery:*`) and therefore the same hash slot when
hashtags are used.

If you operate in cluster mode, configure `redisPrefix` with a
hash-tag wrapper:

```typescript
DiscoveryModule.forRoot({
  redisPrefix: '{titan:discovery}',     // Force all keys to one slot
});
```

This guarantees the Lua script's multi-key atomicity holds in
cluster mode.

## Netron integration

When `enableNetronIntegration: true` (default), the module
registers a `NetronDiscoveryIntegration` service that subscribes to
two Netron events:

| Netron event                    | Discovery action                               |
| ------------------------------- | ---------------------------------------------- |
| `NETRON_EVENT_SERVICE_EXPOSE`   | `discovery.registerService({ name, version })` |
| `NETRON_EVENT_SERVICE_UNEXPOSE` | `discovery.unregisterService(name)`            |

This means **you don't call `registerService` manually** in normal
apps — exposing a `@Service` via Netron auto-announces it.

```typescript
@Service('users@1.0.0')
class UsersService {
  @Public() async findById(id: string) { /* … */ }
}

// In your app:
await app.expose(usersService);
// → Netron emits NETRON_EVENT_SERVICE_EXPOSE
// → titan-discovery registers `users@1.0.0` on this node
// → other peers' findNodesByService('users','1.0.0') sees this node
```

The integration also re-registers existing services when itself
starts up (`registerExisting: true`, default).

## Client mode

For pods that only consume the registry (don't host services —
gateways, admin UIs, schedulers):

```typescript
DiscoveryModule.forRoot({
  clientMode:              true,
  enableNetronIntegration: true,
})
```

In client mode:

- No heartbeat publishes; the pod never appears in the registry.
- The pub/sub subscriber still runs — events are received as normal.
- Lookups (`findNodesByService`, `getActiveNodes`) work as normal.
- `registerNode` / `registerService` raise no error but do nothing
  observable to peers.

## Failure scenarios — what happens, what to alert on

| Scenario                            | Discovery behaviour                                                   | What to alert on                       |
| ----------------------------------- | --------------------------------------------------------------------- | -------------------------------------- |
| Pod crashes                         | Heartbeat stops; TTL expires after `heartbeatTTL`; eventually evicted | Eviction in `cleanupInactiveNodes` log |
| Pod hangs (no heartbeat work loop)  | Same as crash                                                         | Process-level liveness probe           |
| Redis briefly unavailable           | Heartbeat retries with backoff; usually recovers                      | Sustained `heartbeat failed` warnings  |
| Redis sustained outage > TTL        | All pods get evicted from each other's views                          | Critical pager event                   |
| Cluster split (network partition)   | Each side ejects the other; recovers when partition heals             | Sudden registry-size drop              |
| Pub/sub channel mis-configured      | Events don't propagate; discovery still works via polling             | `pubSubEnabled` mismatch in config     |
| Many pods, slow `heartbeatInterval` | Stale registry; new pods take long to be seen                         | Set `heartbeatInterval` < `5_000`      |

## Tokens

| Token                                |
| ------------------------------------ |
| `DISCOVERY_SERVICE_TOKEN`            |
| `DISCOVERY_OPTIONS_TOKEN`            |
| `REDIS_TOKEN` (re-exported)          |
| `NETRON_DISCOVERY_INTEGRATION_TOKEN` |

> `LOGGER_TOKEN` re-export from `discovery/types.ts` is **internal-
> only** to avoid circular deps. Always import `LOGGER_TOKEN` from
> `@omnitron-dev/titan/module/logger`.

## Lifecycle

`DiscoveryModule` implements:

- `async onStart(app)` — `DiscoveryService.onStart()` opens
  pub/sub if enabled, registers this node, starts heartbeat;
  `NetronDiscoveryIntegration` binds to Netron events.
- `async onStop(app)` — `DiscoveryService.onStop()` stops heartbeat,
  deregisters this node (best-effort), closes pub/sub.

Service-level hooks (run in dependency order from the module hooks):

- `onInit()` — resolve dependencies, prepare scripts.
- `onStart()` — open pub/sub, register node, start heartbeat timer.
- `onStop()` — graceful shutdown: stop timer, attempt deregister,
  unsubscribe.
- `onDestroy()` — final cleanup.

In `clientMode: true`, `onStart` still opens pub/sub (so events
arrive) but skips registration and heartbeat.

## Performance notes

- **Heartbeat cost.** One Lua script EVAL per pod per
  `heartbeatInterval`. At default 5 s, 100 pods → 20 EVAL/s
  cluster-wide. Each script touches three keys.
- **Lookup cost.** `findNodesByService` reads `index:nodes` (SMEMBERS)
  then `getNodeInfo` for each (HGETALL). Cache the result client-
  side if you call it on the hot path — the registry doesn't
  change every request.
- **Pub/sub cost.** Each registration/update event is one PUBLISH
  - N SUBSCRIBE callbacks where N is the number of subscribed
    pods. Negligible at hundreds of pods.

## Anti-patterns

- **Long `heartbeatInterval`.** Slow node-loss detection. Pick
  intervals such that `heartbeatTTL = 3 × heartbeatInterval`.
- **Putting business state in `services`.** The `ServiceInfo`
  list is identity / version metadata. Don't pack request counts,
  feature flags, or per-request metadata into it.
- **Treating discovery as authoritative for routing.** Discovery
  tells you *who exists*; the load balancer / multi-backend client
  decides *which to call*. Pair with
  [Netron multi-backend](/docs/titan/netron/multi-backend).
- **Manual `registerNode` from app code.** The Netron integration
  handles it. Manual registration is for unusual topologies (sidecar
  registrants, gateway-mediated nodes).
- **Sharing Redis with high-cardinality unrelated keyspaces.**
  Discovery touches a small fixed set of keys. If your app's main
  Redis is hot, dedicate a separate DB index for discovery to keep
  the keyspace pristine.
- **`heartbeatTTL` set to less than 2 × interval.** Single Redis
  hiccup ejects healthy nodes — exactly the time your app needs
  the registry most.

## See also

- [Netron / Multi-backend](/docs/titan/netron/multi-backend) — pairs with discovery
- [`titan-redis`](/docs/titan/modules/redis) — backing store
- [`titan-health`](/docs/titan/modules/health) — health drives heartbeat
- [Module map](/docs/titan/modules/module-map#where-redis-sits) — Redis DB split recommendation
- [Observability matrix](/docs/titan/modules/observability-matrix#titan-discovery) — what discovery emits and logs

---

---
id: "titan/modules/errors-catalog"
title: "Errors catalog"
description: "What every module throws, error codes, and handling patterns."
---

# Errors catalog

Per-module catalog of the errors a Titan service may emit, the
codes they carry on the wire, and the recommended handling
pattern. Useful when designing error-handling middleware or
writing retry / fallback logic.

All claims here are verified against module source (greps in
`packages/titan-*/src` for `Errors.X(`, `throw new XError`, and
class declarations).

## Two flavours of error

| Source                                            | Shape                                                   | Wire-friendly |
| ------------------------------------------------- | ------------------------------------------------------- | :-----------: |
| `Errors.X(...)` from `@omnitron-dev/titan/errors` | `TitanError` with `code: ErrorCode` enum                |       ✓       |
| Dedicated subclass per module                     | `class FooError extends Error` (sometimes `TitanError`) |     varies    |

The framework prefers the `Errors` namespace — these errors
serialise across Netron transports with the class name and code
preserved. Bespoke subclasses are reserved for cases where the
extra fields (e.g., `retryAfter` on `RateLimitExceededError`) make
the API clearer.

## `ErrorCode` cheat sheet

The most commonly thrown codes across all modules:

| Code                  | Numeric | When used                              |
| --------------------- | :-----: | -------------------------------------- |
| `BAD_REQUEST`         |   400   | Caller-supplied input invalid          |
| `UNAUTHORIZED`        |   401   | Missing or invalid credentials         |
| `FORBIDDEN`           |   403   | Authenticated but disallowed           |
| `NOT_FOUND`           |   404   | Target identifier doesn't exist        |
| `CONFLICT`            |   409   | Resource state contradicts the request |
| `TOO_MANY_REQUESTS`   |   429   | Rate limit / quota exceeded            |
| `INTERNAL_ERROR`      |   500   | Unexpected; investigate                |
| `NOT_IMPLEMENTED`     |   501   | Feature flag off / variant missing     |
| `SERVICE_UNAVAILABLE` |   503   | Transient — downstream dep failed      |
| `REQUEST_TIMEOUT`     |   408   | Operation exceeded configured timeout  |
| `VALIDATION_ERROR`    |   422   | Schema validation failed               |

These behave like HTTP status codes on the wire but are
transport-agnostic — the same codes ride TCP / Unix / WS / HTTP.

## Aggregate count per module

What each module raises most often:

<span class="module-badge">official · @omnitron-dev/titan-*</span>

| Module                  | Top errors raised                                                                       | Count\* |
| ----------------------- | --------------------------------------------------------------------------------------- | :-----: |
| `titan-pm`              | conflict (19), timeout (9), notFound (9), internal (5), badRequest (5), unavailable (3) |    50   |
| `titan-database`        | badRequest (8), notFound (6), unavailable (4), timeout (1)                              |    19   |
| `titan-scheduler`       | notFound (7), badRequest (3), timeout (2), conflict (2)                                 |    14   |
| `titan-events`          | timeout (3), badRequest (3), notFound (2), internal (2), tooManyRequests (1)            |    11   |
| `titan-redis`           | notFound (5), tooManyRequests (2), timeout (2), conflict (1), badRequest (1)            |    11   |
| `titan-health`          | internal (6), notFound (1), conflict (1), badRequest (1)                                |    9    |
| `titan-discovery`       | conflict (1), badRequest (1)                                                            |    2    |
| `titan-notifications`   | notFound (1)                                                                            |    1    |
| `titan-auth`            | — (uses bespoke subclasses below)                                                       |    0    |
| `titan-cache`           | — (uses bespoke subclasses below)                                                       |    0    |
| `titan-lock`            | — (uses FailureTracker windows + log)                                                   |    0    |
| `titan-metrics`         | — (records and continues, doesn't throw)                                                |    0    |
| `titan-ratelimit`       | — (bespoke `RateLimitExceededError`)                                                    |    0    |
| `titan-telemetry-relay` | — (relays observability; doesn't throw user-visible)                                    |    0    |

\*Counts are static `throw Errors.X(...)` callsites in source; per-
request error rate depends entirely on input.

## Bespoke error classes per module

Some modules export dedicated `Error` subclasses for cases where
extra fields aid programmatic handling.

### `titan-auth`

<span class="module-badge">official · @omnitron-dev/titan-auth</span>

| Class                      | Parent              | When thrown                                             |
| -------------------------- | ------------------- | ------------------------------------------------------- |
| `InvalidTokenError`        | `Error`             | JWT verification fails (bad signature, malformed)       |
| `TokenExpiredError`        | `InvalidTokenError` | Token's `exp` is in the past                            |
| `UnauthorizedError`        | `Error`             | Middleware ran but no credentials presented             |
| `SessionRevokedAuthError`† | `Error`             | Shared-session preset detected revocation               |
| `InvalidTokenClaimError`†  | `Error`             | Token decoded but claims don't match the expected shape |

† `SessionRevokedAuthError` and `InvalidTokenClaimError` are exported
from the `@omnitron-dev/titan-auth/preset/shared-session` subpath, not
the package root. The other three are root exports.

```typescript
import { InvalidTokenError, TokenExpiredError, UnauthorizedError }
  from '@omnitron-dev/titan-auth';

try {
  await jwt.verify(rawToken);
} catch (e) {
  if (e instanceof TokenExpiredError)   return refresh();
  if (e instanceof InvalidTokenError)   return reject(401);
  throw e;
}
```

### `titan-database`

<span class="module-badge">official · @omnitron-dev/titan-database</span>

| Class                    | When thrown                                                      |
| ------------------------ | ---------------------------------------------------------------- |
| `MigrationLockError`     | Migrations can't acquire the advisory lock (another pod running) |
| `MigrationChecksumError` | A migration file's checksum doesn't match what was applied       |

Both signal "don't proceed with the deploy" — the operator must
investigate before retry.

### `titan-pm`

<span class="module-badge">official · @omnitron-dev/titan-pm</span>

| Class                   | When thrown                              |
| ----------------------- | ---------------------------------------- |
| `PoolBackpressureError` | Pool queue is full; caller must back off |

> `PoolBackpressureError` is **not re-exported** from the
> `@omnitron-dev/titan-pm` barrel, so you can't `instanceof` it from
> the package root — narrow on `error.name === 'PoolBackpressureError'`
> instead.

```typescript
try {
  await pool.exec('compress', payload);
} catch (e) {
  if (e instanceof Error && e.name === 'PoolBackpressureError') {
    await sleep(100);
    return retry();
  }
  throw e;
}
```

### `titan-ratelimit`

<span class="module-badge">official · @omnitron-dev/titan-ratelimit</span>

| Class                    | When thrown                                     |
| ------------------------ | ----------------------------------------------- |
| `RateLimitExceededError` | `enforce(key)` rejected — caller exceeded limit |

Carries an `IRateLimitResult` on its `.result` property (the fields
are nested under `error.result`, not on the error directly):

| `error.result` field | Type               | Meaning                                |
| -------------------- | ------------------ | -------------------------------------- |
| `retryAfter`         | `number` (seconds) | Suggested back-off before next attempt |
| `remaining`          | `number`           | Remaining capacity at error time       |
| `limit`              | `number`           | Maximum requests allowed in the window |
| `resetAt`            | `number`           | Unix ms when the window resets         |

Surface this to clients as a 429 with a `Retry-After` header
(`error.result.retryAfter`).

## Common patterns per error kind

### `notFound` (404 / `NOT_FOUND`)

The most-thrown family. Typical handlers:

```typescript
import { TitanError, ErrorCode } from '@omnitron-dev/titan/errors';

try {
  return await users.findById(id);
} catch (e) {
  if (e instanceof TitanError && e.code === ErrorCode.NOT_FOUND) {
    return null;                                          // null-on-miss
  }
  throw e;
}
```

Modules that throw it: `titan-pm` (process not found), `titan-
scheduler` (job not found), `titan-redis` (key namespace not
configured), `titan-database` (connection name missing), `titan-
health` (indicator name missing), `titan-discovery` (node id),
`titan-events` (history entry), `titan-notifications`
(template / channel).

### `conflict` (409 / `CONFLICT`)

State-change refused because something already exists or is in an
incompatible state. Not retryable without resolving the conflict.

Typical sources: `titan-pm` (process already running), `titan-
discovery` (lifecycle violation — `Cannot start a stopped
DiscoveryService`), `titan-health` (indicator name collision),
`titan-scheduler` (job already registered), `titan-redis` (named
client already exists).

### `timeout` (408 / `REQUEST_TIMEOUT`)

Operation exceeded its configured timeout. **Sometimes retryable**
(if the work is idempotent) — usually surface to caller.

Sources: `titan-pm` (`exec()` timeout, health-check timeout),
`titan-events` (handler timeout), `titan-scheduler` (job timeout),
`titan-redis` (script timeout).

### `unavailable` (503 / `SERVICE_UNAVAILABLE`)

Transient dependency failure. Retry with exponential backoff.

Sources: `titan-database` (pool exhausted / connection lost),
`titan-pm` (worker pool empty for class).

### `tooManyRequests` (429 / `TOO_MANY_REQUESTS`)

Rate / quota exceeded. Always honour `retryAfter` if provided.

Sources: `titan-events` (subscription quota), `titan-redis`
(connection backoff).

### `internal` (500 / `INTERNAL_ERROR`)

Unexpected — usually a bug or environmental anomaly. Log loudly,
surface as 500.

Sources: `titan-health` (indicator output parsing — e.g., `df` /
`wmic` output), `titan-pm` (supervisor internal state),
`titan-events` (handler dispatch internal failure).

### `badRequest` (400 / `BAD_REQUEST`)

Caller-side mistake. Never retry; fix the caller.

Sources: every module — most commonly `titan-database` (invalid
dialect, bad schema name), `titan-pm` (bad spawn options), `titan-
scheduler` (invalid cron expression).

## Handling errors generically

The `TitanError` class is the unifying shape:

```typescript
import { TitanError, ErrorCode, Errors } from '@omnitron-dev/titan/errors';

try {
  return await someService.doWork(input);
} catch (e) {
  if (!(e instanceof TitanError)) throw e;       // not a framework error

  switch (e.code) {
    case ErrorCode.NOT_FOUND:           return null;
    case ErrorCode.UNAUTHORIZED:        return reject(401);
    case ErrorCode.FORBIDDEN:           return reject(403);
    case ErrorCode.TOO_MANY_REQUESTS:   return scheduleRetry(e);
    case ErrorCode.REQUEST_TIMEOUT:     return idempotent ? retry() : reject(408);
    case ErrorCode.SERVICE_UNAVAILABLE: return retryWithBackoff();
    default:                            throw e;
  }
}
```

Per-call error metadata travels in `e.details`:

```typescript
catch (e) {
  if (e instanceof TitanError && e.code === ErrorCode.NOT_FOUND) {
    logger.warn({ resource: e.details?.resource }, 'lookup miss');
  }
}
```

## Throwing your own

In your own services, prefer the `Errors` namespace over bespoke
subclasses unless you need extra fields:

```typescript
import { Errors } from '@omnitron-dev/titan/errors';

if (!user) throw Errors.notFound('user', id);
if (user.locked) throw Errors.conflict('user account is locked');
if (input.amount <= 0) throw Errors.badRequest('amount must be > 0');
```

Cross-the-wire serialisation, class identity, and code-based
routing all work out of the box.

## Retryability matrix

A field guide for your retry middleware:

| Code                  | Retryable? | Backoff             |
| --------------------- | :--------: | ------------------- |
| `BAD_REQUEST`         |      ✗     | —                   |
| `UNAUTHORIZED`        |      ✗     | —                   |
| `FORBIDDEN`           |      ✗     | —                   |
| `NOT_FOUND`           |      ✗     | —                   |
| `CONFLICT`            |      ✗     | resolve state first |
| `VALIDATION_ERROR`    |      ✗     | —                   |
| `TOO_MANY_REQUESTS`   |      ✓     | honour `retryAfter` |
| `REQUEST_TIMEOUT`     |      ⚠     | only if idempotent  |
| `SERVICE_UNAVAILABLE` |      ✓     | exponential         |
| `INTERNAL_ERROR`      |      ⚠     | with budget         |
| `NOT_IMPLEMENTED`     |      ✗     | —                   |

`@omnitron-dev/titan/utils` ships pre-built
[retry](/docs/titan/resilience/retry) and
[circuit-breaker](/docs/titan/resilience/circuit-breaker) primitives
(`retry()`, `CircuitBreaker`, `withTimeout()`) that already encode
this matrix — wire one of those rather than hand-rolling.

## Anti-patterns

- **Catching `Error` and swallowing.** Loses the typed error
  identity. Always narrow to `TitanError` and inspect `code`.
- **Mapping every error to 500.** The whole point of typed codes
  is that the client gets actionable status. Forward `code`.
- **Throwing `Errors.internal` for user mistakes.** A bad input
  is `badRequest` or `validation`, not `internal`. `internal`
  should mean *the developer is surprised*.
- **Logging the full token in `InvalidTokenError`.** Log `kid` /
  `code` only; the token may be re-usable elsewhere.
- **Treating `MigrationLockError` as transient.** It usually means
  another deploy is mid-flight; resolve manually, don't retry.

## See also

- [Errors / Overview](/docs/titan/errors/overview) — framework-level guide
- [Errors / Hierarchy](/docs/titan/errors/hierarchy) — the class graph
- [Errors / Factories](/docs/titan/errors/factories) — full `Errors.*` reference
- [Errors / Classification](/docs/titan/errors/classification) — retryable vs not
- [Resilience / Retry](/docs/titan/resilience/retry) — automatic handling
- [Observability matrix](/docs/titan/modules/observability-matrix) — how errors surface as events

---

---
id: "titan/modules/events"
title: "titan-events"
---

# titan-events

<span class="module-badge">official · @omnitron-dev/titan-events · stable</span>

Typed event bus with wildcard / namespaced patterns, decorator-based
listeners, async / serial / reduce emit modes, event history with
replay, transactions, schema validation, delayed and cron-scheduled
events, and rich metrics.

```bash
pnpm add @omnitron-dev/titan-events
```

## When you need it

- **In-process domain events.** Decouple services that should react
  to each other (e.g., `user.created` → send welcome email →
  update analytics) without direct calls.
- **Wildcard fan-out.** Subscribe to `order.*` to capture every
  order-lifecycle event with one handler.
- **Cross-module orchestration.** A workflow listens to multiple
  modules' events without becoming a hard dependency of any of them.
- **Pub/sub-style replay.** A late-joining consumer pulls the
  recent event history and catches up.

> For cross-process delivery use [`titan-notifications`](/docs/titan/modules/notifications)
> over Rotif or a real message broker. `titan-events` is in-process.

## Quickstart

```typescript
import { EventsModule } from '@omnitron-dev/titan-events';

@Module({
  imports: [
    EventsModule.forRoot({
      wildcard:     true,
      delimiter:    '.',
      maxListeners: 100,
      history:      { enabled: true, maxSize: 1_000 },
      metrics:      { enabled: true, slowThreshold: 100 },
      concurrency:  10,
    }),
  ],
})
class AppModule {}
```

```typescript
import { OnEvent, EVENTS_SERVICE_TOKEN, EventsService }
  from '@omnitron-dev/titan-events';

@Service({ name: 'users' })
class UsersService {
  constructor(@Inject(EVENTS_SERVICE_TOKEN) private readonly events: EventsService) {}

  @Public()
  async create(input: CreateInput) {
    const user = await this.repo.create(input);
    await this.events.emitAsync('user.created', { id: user.id, email: user.email });
    return user;
  }
}

@Injectable()
class WelcomeMailer {
  @OnEvent({ event: 'user.created', async: true })
  async onCreated(payload: { id: string; email: string }) {
    await this.mailer.send(payload.email, 'Welcome');
  }
}
```

## `IEventsModuleOptions`

| Option              | Type                                                | Default                                   |
| ------------------- | --------------------------------------------------- | ----------------------------------------- |
| `wildcard`          | `boolean` — enable `*` / `**` patterns              | `true`                                    |
| `delimiter`         | `string` — namespace separator                      | `'.'`                                     |
| `maxListeners`      | `number`                                            | `100`                                     |
| `verboseMemoryLeak` | `boolean`                                           | —                                         |
| `history`           | `{ enabled, maxSize?, ttl? }`                       | `maxSize: 1_000`                          |
| `metrics`           | `{ enabled, slowThreshold?, sampleRate? }`          | `slowThreshold: 100ms`, `sampleRate: 1.0` |
| `onError`           | `(error, event, data) => void`                      | —                                         |
| `schemas`           | `Record<string, ZodSchema>` — event payload schemas | —                                         |
| `concurrency`       | `number` — max concurrent async handlers            | `10`                                      |
| `isGlobal`          | `boolean`                                           | `true`                                    |

Also exported: `forRootAsync({ useFactory, inject? })`.

## `EventsService` — the API

### Emit modes

```typescript
// Sync — fire-and-forget; returns true if any listener accepted
this.events.emit('user.created', payload);

// Async — concurrent listeners; resolves when all handlers complete
const results = await this.events.emitAsync('user.created', payload);

// Serial — listeners run one after another; resolves with their results
const results = await this.events.emitSerial('user.created', payload);

// Reduce — pipe payload through listeners, each transforming the previous result
const final = await this.events.emitReduce('user.transform', input, initialValue);
```

| Method                                                  | Returns                                  |
| ------------------------------------------------------- | ---------------------------------------- |
| `emit<T>(event, data?, options?)`                       | `boolean`                                |
| `emitAsync<T>(event, data?, options?)`                  | `Promise<any[]>` — handler return values |
| `emitSerial<T>(event, data?, options?)`                 | `Promise<any[]>`                         |
| `emitReduce<T, R>(event, data, initialValue, options?)` | `Promise<R>`                             |

`emitAsync` propagates failures from regular and wildcard handlers, including
synchronous throws. Multiple failed groups produce `AggregateError`. Handler
error policies still apply. Await and handle this promise at the operation
boundary; event delivery does not roll back committed domain effects.

### Subscribe / unsubscribe

| Method                                       | Returns                |
| -------------------------------------------- | ---------------------- |
| `subscribe(event, handler, options?)`        | `IEventSubscription`   |
| `once(event, handler, options?)`             | `IEventSubscription`   |
| `subscribeMany(events[], handler, options?)` | `IEventSubscription[]` |
| `subscribeAll(handler, options?)`            | `IEventSubscription`   |
| `unsubscribe(event, handler?)`               | `void`                 |
| `unsubscribeAll()`                           | `void`                 |
| `waitFor<T>(event, timeout?, filter?)`       | `Promise<T>`           |

### Introspection / patterns

| Method                                       | Purpose                              |
| -------------------------------------------- | ------------------------------------ |
| `getListenerCount(event)`                    | Number of listeners on an event      |
| `getEventNames()`                            | All known event names                |
| `hasListeners(event)`                        | Existence check                      |
| `getStatistics(event?)`                      | Per-event or whole-bus statistics    |
| `getEventsInNamespace(namespace)`            | Events under `namespace.*`           |
| `filterEventsByPattern(pattern)`             | Match events against a glob          |
| `getListenersByPattern(pattern)`             | Listeners matching a pattern         |
| `emitToPattern<T>(pattern, data?, options?)` | Emit to every event matching pattern |
| `removeListenersByPattern(pattern)`          | Bulk unsubscribe by pattern          |

### Transactions

```typescript
const tx = this.events.beginTransaction();

tx.emit('order.created', order);
tx.emit('audit.log',     { kind: 'order.created', orderId: order.id });

if (everythingOk) {
  await tx.commit();   // every emit fires
} else {
  tx.rollback();       // nothing fired
}
```

Useful for "either all of these events fire or none" semantics
within a single request scope.

## Decorators

### `@OnEvent(options)`

```typescript
@OnEvent({
  event:         'user.*',
  async:         true,
  priority:      10,
  timeout:       5_000,
  filter:        (data) => data.tenantId === 'enterprise',
  transform:     (data) => normalizeUser(data),
  errorBoundary: true,
  onError:       (err, data) => log.warn('handler failed', { err, data }),
})
async onUserChange(data: UserEvent) { /* … */ }
```

### `@OnceEvent(options)` — fire once then unsubscribe

```typescript
@OnceEvent({ event: 'app.ready' })
async warmCaches() { /* … */ }
```

### `@OnAnyEvent()` — wildcard catch-all

### `@EmitEvent(options)` — emit when a method runs

```typescript
@EmitEvent({
  event:     'user.created',
  mapResult: (user) => ({ id: user.id, email: user.email }),
  mapError:  (err)  => ({ reason: err.message }),
})
async create(input: CreateInput) { /* … */ }
```

The interceptor runs **after** the method: on success it emits
`<event>.success` (payload = `mapResult(result)` if provided, else the
raw result); on throw it emits `<event>.error` (payload =
`mapError(error)` if provided, else the error) and re-throws. Options
are `event`, `mapResult?`, `mapError?` only.

Two things about it surprise people, and both are silent.

**Subscribers must use the suffixed names.** `event: 'user.created'` never
emits `user.created`; subscribing to that bare name receives nothing.

**The emitter is found by convention, not injected.** The decorator reads
`this.__eventEmitter__`, then `this.eventEmitter`. On an instance with
neither it runs the method and emits nothing — no warning, no error. A service
that stores the emitter as `events`, for example, will not emit through
this decorator. Expose it as `eventEmitter` or `__eventEmitter__`; the
property name is part of the contract.

### `@ScheduleEvent(options)`

```typescript
@ScheduleEvent({ event: 'user.welcome-followup', delay: 24 * 60 * 60 * 1000 })   // 24h delay
async welcomeFollowup() { /* … */ }

@ScheduleEvent({ event: 'reports.daily', cron: '0 9 * * *' })
async dailyReport() { /* … */ }
```

Options: `event` (required), plus one of `delay` (ms), `cron`, or
`at` (a `Date`).

### `@BatchEvents(options)`

Coalesce many emits into one delivery — useful when a tight loop
would otherwise fire thousands of identical events. Options: `event`,
`maxSize`, and `maxWait` (ms) are all required.

### `@OnModuleEvent({ event, filter? })` — listen to framework module events

### `@EventEmitter(namespace?)` — class-level emitter binding

## Wildcard semantics

With `wildcard: true`:

```typescript
@OnEvent({ event: 'user.*' })       // user.created, user.deleted (one segment)
@OnEvent({ event: 'user.**' })      // any nested suffix
@OnEvent({ event: 'order.*.shipped' })  // order.{id}.shipped
```

Pattern resolution is cached, so wildcards are O(1) per emit at
runtime.

## Other services exposed

| Service                         | Token                            | Purpose                   |
| ------------------------------- | -------------------------------- | ------------------------- |
| `EventsService`                 | `EVENTS_SERVICE_TOKEN`           | Main API (above)          |
| `EventBusService`               | `EVENT_BUS_SERVICE_TOKEN`        | Inter-module bus          |
| `EventHistoryService`           | `EVENT_HISTORY_SERVICE_TOKEN`    | Query + replay history    |
| `EventSchedulerService`         | `EVENT_SCHEDULER_SERVICE_TOKEN`  | Delayed / cron scheduling |
| `EventValidationService`        | `EVENT_VALIDATION_SERVICE_TOKEN` | Schema validation         |
| `EVENT_EMITTER_TOKEN`           | —                                | Raw underlying emitter    |
| `EVENT_METADATA_SERVICE_TOKEN`  | —                                | Metadata propagation      |
| `EVENT_DISCOVERY_SERVICE_TOKEN` | —                                | Decorator discovery       |

`EventValidationService.registerSchema(name, schema)` accepts Zod schemas,
JSON Schema and custom `ISchemaValidator` implementations. Pass Zod objects
directly; the service adapts their parse results to its validation contract.
`validateData(name, payload)` returns a boolean; `validate(name, payload)`
returns the validation result, including field errors.

## Lifecycle

`EventsService` implements `onInit`, `onStart`, `onStop`,
`onDestroy`. The module honours them via the standard Titan lifecycle.

```mermaid
flowchart LR
  Producer[Service.emit] --> Bus[EventsService]
  Bus --> Pattern{matching?}
  Pattern -- exact --> H1[Handler A]
  Pattern -- wildcard --> H2[Handler B]
  Pattern -- catch-all --> H3[Handler C]
  Bus --> History[EventHistoryService<br/>if history.enabled]
  Bus --> Metrics[Stats]
  Bus --> Sched[EventSchedulerService<br/>for delayed/cron]
```

## Anti-patterns

- **Treating in-process events as durable.** They aren't. Restart →
  in-flight handlers gone. For at-least-once delivery, use
  `titan-notifications` over Rotif.
- **Heavy work in sync `emit`.** Sync emit blocks the publisher.
  Use `emitAsync` for I/O-bound handlers; reserve sync for
  metrics/audit-style side effects.
- **Schema-less events at scale.** A typo in an event name silently
  drops the listener. Use the `schemas` option for production-
  critical events.
- **`subscribeAll` for analytics.** Captures every event in the
  process — easy to leave running and blow the listener budget.
  Prefer scoped patterns.

## See also

- [`titan-notifications`](/docs/titan/modules/notifications) — for cross-process
  durable delivery
- [`titan-scheduler`](/docs/titan/modules/scheduler) — for full cron/interval
  semantics (events module has lighter-weight scheduled emission)

---

---
id: "titan/modules/health"
title: "titan-health"
---

# titan-health

<span class="module-badge">official · @omnitron-dev/titan-health · stable</span>

Centralised health-check system with built-in indicators (memory,
event loop, disk, database, Redis), Kubernetes-style probes (live,
ready, full), TTL caching, timeout-guarded execution, and a Netron
RPC endpoint for remote inspection.

```bash
pnpm add @omnitron-dev/titan-health
```

## When you need it

- **Kubernetes liveness / readiness probes.** Distinguish "process
  is alive" from "ready to accept traffic" from "every dependency
  is up".
- **Operator dashboards.** A single endpoint returns per-indicator
  status the web console / CLI can render.
- **Auto-degradation triggers.** Subscribe to `health:changed` and
  remove the pod from rotation on unhealthy.

## Quickstart

```typescript
import { TitanHealthModule } from '@omnitron-dev/titan-health';

@Module({
  imports: [
    TitanHealthModule.forRoot({
      enableMemoryIndicator:    true,
      enableEventLoopIndicator: true,
      enableDiskIndicator:      true,
      enableDatabaseIndicator:  true,
      databaseConnection:       db,             // any object with execute/raw/query
      enableRedisIndicator:     true,
      redisClient:              redis,          // any object with .ping()
      memoryThresholds:         { heapDegradedThreshold: 0.8, heapUnhealthyThreshold: 0.95 },
      eventLoopThresholds:      { lagDegradedThreshold: 50, lagUnhealthyThreshold: 200 },
      timeout:                  5_000,
      enableCaching:            true,
      cacheTtl:                 1_000,
      enableRpcService:         true,
      version:                  '1.0.0',
    }),
  ],
})
class AppModule {}
```

Async configuration via `forRootAsync({ useFactory, inject?, imports?, isGlobal? })`.

## `HealthModuleOptions`

| Option                     | Type                                                                      | Default |
| -------------------------- | ------------------------------------------------------------------------- | ------- |
| `enableMemoryIndicator`    | `boolean`                                                                 | `true`  |
| `enableEventLoopIndicator` | `boolean`                                                                 | `true`  |
| `enableDiskIndicator`      | `boolean`                                                                 | `false` |
| `enableDatabaseIndicator`  | `boolean`                                                                 | `false` |
| `databaseConnection`       | `{ execute?, raw?, query? }`                                              | —       |
| `enableRedisIndicator`     | `boolean`                                                                 | `false` |
| `redisClient`              | `{ ping(), info?, status?, isReady? }`                                    | —       |
| `timeout`                  | `number` (ms)                                                             | `5_000` |
| `enableCaching`            | `boolean`                                                                 | `false` |
| `cacheTtl`                 | `number` (ms)                                                             | `1_000` |
| `enableRpcService`         | `boolean` — expose Netron RPC                                             | `true`  |
| `detailRpcRoles`           | Nonempty roles for detailed Netron calls, declared at module registration | unset   |
| `version`                  | `string`                                                                  | —       |
| `isGlobal`                 | `boolean`                                                                 | `false` |
| `indicators`               | `Array<new (...args) => IHealthIndicator>` — custom indicator classes     | —       |
| `memoryThresholds`         | `{ heapDegradedThreshold, heapUnhealthyThreshold }` (0..1 ratio)          | —       |
| `eventLoopThresholds`      | `{ lagDegradedThreshold, lagUnhealthyThreshold }` (ms)                    | —       |
| `diskThresholds`           | `DiskThresholds`                                                          | —       |
| `databaseOptions`          | `DatabaseIndicatorOptions`                                                | —       |
| `redisOptions`             | `RedisIndicatorOptions`                                                   | —       |

## `HealthService` — the API

```typescript
import { HEALTH_SERVICE_TOKEN, type HealthService } from '@omnitron-dev/titan-health';

@Service({ name: 'admin' })
class AdminService {
  constructor(@Inject(HEALTH_SERVICE_TOKEN) private readonly health: HealthService) {}

  @Public()
  async status() {
    return this.health.check();
  }
}
```

### Aggregate health

| Method              | Returns                                    | Use case                        |
| ------------------- | ------------------------------------------ | ------------------------------- |
| `check()`           | `Promise<HealthCheckResult>` (full report) | `/healthz` full JSON            |
| `isHealthy()`       | `Promise<boolean>`                         | Generic check                   |
| `isAlive()`         | `Promise<boolean>`                         | Kubernetes **liveness** probe   |
| `isReady()`         | `Promise<boolean>`                         | Kubernetes **readiness** probe  |
| `checkOne(name)`    | `Promise<HealthIndicatorResult>`           | Per-indicator inspection        |
| `getUptime()`       | `number` (ms)                              | Operational metrics             |
| `invalidateCache()` | `void`                                     | Force re-check (skip TTL cache) |

### Indicator registry

| Method                         | Purpose                     |
| ------------------------------ | --------------------------- |
| `registerIndicator(indicator)` | Add an indicator at runtime |
| `unregisterIndicator(name)`    | Remove                      |
| `getIndicators()`              | All registered names        |
| `getIndicator(name)`           | Lookup by name              |
| `hasIndicator(name)`           | Existence check             |
| `getIndicatorCount()`          | Count                       |
| `clearIndicators()`            | Remove all                  |

## Built-in indicators

| Class                       | What it checks                                                   |
| --------------------------- | ---------------------------------------------------------------- |
| `MemoryHealthIndicator`     | Heap usage vs `heapDegradedThreshold` / `heapUnhealthyThreshold` |
| `EventLoopHealthIndicator`  | Event loop lag (ms)                                              |
| `HighResEventLoopIndicator` | High-resolution event loop lag (sub-ms)                          |
| `DiskHealthIndicator`       | Free disk space                                                  |
| `DatabaseHealthIndicator`   | Database connectivity via `execute`/`raw`/`query` shape          |
| `RedisHealthIndicator`      | Redis connectivity via `.ping()`                                 |
| `CompositeHealthIndicator`  | Combine multiple indicators with worst-status wins               |

## Custom indicators

Implement the indicator interface:

```typescript
import { Injectable } from '@omnitron-dev/titan';
import type { IHealthIndicator, HealthIndicatorResult }
  from '@omnitron-dev/titan-health';

@Injectable()
class StripeHealthIndicator implements IHealthIndicator {
  name = 'stripe';

  constructor(private readonly stripe: Stripe) {}

  async check(): Promise<HealthIndicatorResult> {
    try {
      await this.stripe.balance.retrieve();
      return { status: 'healthy' };
    } catch (e) {
      return { status: 'degraded', message: String(e) };
    }
  }
}
```

Register via the `indicators: [StripeHealthIndicator]` option on
`forRoot`, or call `health.registerIndicator(...)` at runtime.

## Aggregation rule

```mermaid
flowchart LR
  M[memory]      --> Agg
  L[event-loop]  --> Agg
  D[database]    --> Agg
  R[redis]       --> Agg
  C[custom]      --> Agg
  Agg{worst wins}
  Agg -- all healthy --> H[healthy]
  Agg -- any degraded --> Deg[degraded]
  Agg -- any unhealthy --> U[unhealthy]
```

- All `healthy` → `healthy`
- Any `degraded`, none `unhealthy` → `degraded`
- Any `unhealthy` → `unhealthy`

## Kubernetes probes — typical wiring

```yaml
livenessProbe:
  httpGet:    { path: /healthz, port: 3000 }
  initialDelaySeconds: 5
  periodSeconds:       10

readinessProbe:
  httpGet:    { path: /readyz, port: 3000 }
  initialDelaySeconds: 0
  periodSeconds:       3
```

`/healthz` → `isAlive()`; `/readyz` → `isReady()`. Detailed JSON
report typically served under `/healthz/full` or via the Netron RPC
endpoint.

## `HealthRpcService`

When `enableRpcService: true` (default), a Netron service is
auto-registered:

```typescript
const healthSvc = await client.queryInterface<HealthRpcService>('Health@1.0.0');

await healthSvc.check();   // HealthResponse
await healthSvc.live();    // LivenessResponse
await healthSvc.ready();   // ReadinessResponse
```

Pass `detailRpcRoles: ['viewer', 'operator', 'admin']` to `forRoot` or return it from the `forRootAsync` factory to protect `check`, `checkIndicator`, `listIndicators`, `uptime` and `isHealthy`. With this policy the anonymous `live` and `ready` probes contain only status and timestamp. Each module registration has its own policy. When the option is omitted, the module keeps its existing public diagnostics. Omnitron sets its control-plane read roles explicitly, including its authenticated service identity.

## TTL caching

Set `enableCaching: true` + `cacheTtl: 1000` to coalesce repeated
checks within a 1-second window. Useful when many components ask
for health simultaneously (probe + dashboard + auto-degrade
listener). Call `invalidateCache()` to force re-check.

## Tokens

| Token                               |
| ----------------------------------- |
| `HEALTH_SERVICE_TOKEN`              |
| `HEALTH_MODULE_OPTIONS_TOKEN`       |
| `HEALTH_RPC_SERVICE_TOKEN`          |
| `MEMORY_HEALTH_INDICATOR_TOKEN`     |
| `EVENT_LOOP_HEALTH_INDICATOR_TOKEN` |
| `DISK_HEALTH_INDICATOR_TOKEN`       |
| `DATABASE_HEALTH_INDICATOR_TOKEN`   |
| `REDIS_HEALTH_INDICATOR_TOKEN`      |

## Lifecycle

The module manages indicator setup directly; `HealthService` itself
is not strictly lifecycle-bound. The RPC service registers as part
of standard Netron exposure.

## Anti-patterns

- **Heavy probes.** Indicators run frequently (every few seconds).
  Keep them to cheap pings (`SELECT 1`, `PING`, `process.memoryUsage`).
- **Treating every dependency as critical.** A cache being slow does
  not block reads. Use `degraded` for non-fatal hiccups.
- **No `timeout`.** An indicator that hangs blocks the whole probe.
  Configure a `timeout` ≤ probe period.
- **Collapsing liveness and readiness.** Slow boot → liveness fails
  → pod restarts before warming completes. Always separate the two.

## See also

- [Application / Health](/docs/titan/application/health) — conceptual guide
- [`titan-database`](/docs/titan/modules/database) — provides `DatabaseHealthIndicator`
- [`titan-redis`](/docs/titan/modules/redis) — its `RedisHealthIndicator` is
  superseded by this module's version when both are loaded

---

---
id: "titan/modules/index"
title: "Modules"
---

# Titan Modules

The Titan ecosystem has three module tiers, each rendered with a
origin badge so you can tell at a glance where a module comes
from and what its guarantees are.

| Tier          | Origin    | Where it lives                                    | Guarantees                                            |
| ------------- | --------- | ------------------------------------------------- | ----------------------------------------------------- |
| **Built-in**  | Built-in  | `@omnitron-dev/titan/module/*`                    | Ships with the framework; same release cadence        |
| **Official**  | Official  | `packages/titan-*` in the `omnitron-dev` monorepo | Maintained by the Omnitron team; independent versions |
| **Community** | Community | Third-party npm packages                          | Audit before adopting; no maintenance commitment      |

Every module page in this section starts with a `<span class="module-badge"></span>`
chip that reflects its tier.

> Want a bird's-eye view of how the 16 modules interlock — who
> depends on whom, which combinations form common stacks, when to
> prefer one over a similar one? Read the **[Module map](/docs/titan/modules/module-map)**.

## Built-in modules

<span class="module-badge">built-in · @omnitron-dev/titan · /module/*</span>

Config and Logger ship inside `@omnitron-dev/titan` and are **auto-loaded**
unless you pass `disableCoreModules: true`. RLS guards are imported explicitly.
No extra Titan package is needed for these entry points.

| Module                                       | Subpath          | Purpose                                                    |
| -------------------------------------------- | ---------------- | ---------------------------------------------------------- |
| [`ConfigModule`](/docs/titan/modules/config) | `/module/config` | Layered, validated, hot-reloadable configuration           |
| [`LoggerModule`](/docs/titan/modules/logger) | `/module/logger` | Structured pino-based logging with transports + processors |
| [RLS guards](/docs/titan/modules/rls)        | `/module/rls`    | Context guard decorators; no module registration           |

## Official modules (`titan-*` packages)

<span class="module-badge">official · @omnitron-dev/titan-*</span>

Fourteen independent packages in the `omnitron-dev` monorepo. Each is
versioned separately; adopting one does not pin you to others.

| Module                                                         | Package                               | Purpose                                                   |
| -------------------------------------------------------------- | ------------------------------------- | --------------------------------------------------------- |
| [`TitanAuthModule`](/docs/titan/modules/auth)                  | `@omnitron-dev/titan-auth`            | JWT auth (HS256 / RS256 / ES256), JWKS, token cache       |
| [`TitanCacheModule`](/docs/titan/modules/cache)                | `@omnitron-dev/titan-cache`           | Multi-tier (L1/L2) caching with LRU / LFU, Redis backing  |
| [`TitanDatabaseModule`](/docs/titan/modules/database)          | `@omnitron-dev/titan-database`        | Kysely + migrations + RLS, multi-dialect                  |
| [`DiscoveryModule`](/docs/titan/modules/discovery)             | `@omnitron-dev/titan-discovery`       | Redis-backed service discovery + Netron integration       |
| [`EventsModule`](/docs/titan/modules/events)                   | `@omnitron-dev/titan-events`          | Typed event bus, wildcard, scheduling, history            |
| [`TitanHealthModule`](/docs/titan/modules/health)              | `@omnitron-dev/titan-health`          | Health indicators (memory, event-loop, disk, db, redis)   |
| [`TitanLockModule`](/docs/titan/modules/lock)                  | `@omnitron-dev/titan-lock`            | Distributed Redis locks with UUID ownership + Lua scripts |
| [`TitanMetricsModule`](/docs/titan/modules/metrics)            | `@omnitron-dev/titan-metrics`         | Counters / gauges / histograms, pluggable storage         |
| [`NotificationsModule`](/docs/titan/modules/notifications)     | `@omnitron-dev/titan-notifications`   | Multi-channel delivery + templates + DLQ                  |
| [`ProcessManagerModule`](/docs/titan/modules/pm)               | `@omnitron-dev/titan-pm`              | Process supervision, worker pools, IPC                    |
| [`TitanRateLimitModule`](/docs/titan/modules/ratelimit)        | `@omnitron-dev/titan-ratelimit`       | Token-bucket, sliding-window, fixed-window strategies     |
| [`TitanRedisModule`](/docs/titan/modules/redis)                | `@omnitron-dev/titan-redis`           | Redis client (clustering, sentinel, TLS, named instances) |
| [`SchedulerModule`](/docs/titan/modules/scheduler)             | `@omnitron-dev/titan-scheduler`       | Cron + interval + timeout with persistence                |
| [`TelemetryRelayService`](/docs/titan/modules/telemetry-relay) | `@omnitron-dev/titan-telemetry-relay` | Store-and-forward telemetry pipeline (no module)          |

## Community modules

<span class="module-badge">community · @your-scope/titan-*</span>

No third-party modules registered yet. See [Community
Modules](/docs/titan/modules/community) for what counts, how to adopt safely, and
how to publish your own.

## Composition

Modules compose through the standard `imports` chain — order does
not matter, the container resolves the dependency graph:

```typescript
@Module({
  imports: [
    // Built-in (auto-loaded; shown explicitly here for illustration)
    ConfigModule.forRoot({ schema: AppConfigSchema, sources: [/* … */] }),
    LoggerModule.forRoot({ level: 'info' }),

    // Official
    TitanRedisModule.forRoot({ config: { url: env.REDIS_URL } }),
    TitanAuthModule.forRoot({ jwtSecret: env.JWT_SECRET }),
    TitanCacheModule.forRoot({ multiTier: true, l2: { client: redisClient, ttl: 60 } }),
    TitanDatabaseModule.forRoot({ connection: { dialect: 'postgres', connection: env.DATABASE_URL } }),
    SchedulerModule.forRoot({ persistence: { provider: redisPersistenceProvider } }),
    TitanMetricsModule.forRoot({ storage: { type: 'memory' } }),
  ],
  providers: [/* your services */],
})
export class AppModule {}
```

## Redis dependency among official modules

Five official modules depend on a Redis client provided by
`TitanRedisModule`:

- `titan-cache` (when L2 tier enabled)
- `titan-lock`
- `titan-discovery`
- `titan-ratelimit` (when `storageType: 'redis'`)
- `titan-notifications` (rate limiter, preference store, rotif
  messaging)

Register `TitanRedisModule.forRoot(...)` first; the rest pick up the
shared client automatically.

## Choosing modules

Most backends start with a small set:

- **Web API service**: built-in config + logger; auth + database +
  cache + health + metrics.
- **Worker service**: built-in config + logger; database + lock +
  scheduler + metrics + health.
- **Notification gateway**: built-in config + logger;
  notifications + ratelimit + metrics.
- **Multi-instance app**: add discovery to the above.

→ Start with [`ConfigModule`](/docs/titan/modules/config) for built-in config or
jump to [`TitanAuthModule`](/docs/titan/modules/auth) for the most-used official
module.

→ Want to build your own? Read
[Authoring a module](/docs/titan/modules-system/authoring-modules) — the
step-by-step recipe every official `titan-*` package follows.

---

---
id: "titan/modules/lifecycle-reference"
title: "Lifecycle reference"
description: "What every module does in onInit / onStart / onStop / onDestroy."
---

# Lifecycle reference

Knowing **when** during boot/shutdown each module runs which hook
is the difference between a clean restart and a half-drained
buffer. This page documents the lifecycle behaviour of every
official module — verified against source — so you can predict
where a `console.log('foo')` in your `onStart` will run.

## The four phases

Recap of [Application lifecycle](/docs/titan/application/lifecycle):

| Phase       | Fires when                                        |
| ----------- | ------------------------------------------------- |
| `onInit`    | After construction, before any `onStart` fires    |
| `onStart`   | Once every `onInit` finished; in dependency order |
| `onStop`    | On `app.stop()`; in **reverse** dependency order  |
| `onDestroy` | Final cleanup phase after `onStop`                |

Modules are not obliged to implement all four — most use just one
or two. The table below shows what each official module actually
does.

## Lifecycle behaviour by module

<span class="module-badge">official · @omnitron-dev/titan-*</span>

### `titan-auth`

| Hook        | What it does |
| ----------- | ------------ |
| `onInit`    | —            |
| `onStart`   | —            |
| `onStop`    | —            |
| `onDestroy` | —            |

`titan-auth` is **lifecycle-passive**. The JWT service has no
long-lived resources to set up or tear down beyond the JWKS HTTP
client it opens lazily on first use.

### `titan-cache`

| Hook        | What it does                                                 |
| ----------- | ------------------------------------------------------------ |
| `onInit`    | —                                                            |
| `onStart`   | —                                                            |
| `onStop`    | L2 adapter (Redis) is released when its owning client closes |
| `onDestroy` | —                                                            |

L1 (in-memory LRU/LFU) lives in JS heap. L2 (Redis) is owned by
`titan-redis` and shuts down with that module.

### `titan-database`

| Hook        | What it does                                    |
| ----------- | ----------------------------------------------- |
| `onInit`    | —                                               |
| `onStart`   | Connection pool warmed up on first query (lazy) |
| `onStop`    | **Drains and closes** the connection pool       |
| `onDestroy` | —                                               |

The `onStop` matters: a clean drain prevents in-flight queries
from being killed mid-transaction.

### `titan-discovery`

Implemented on both the module and the service.

| Hook        | Module (`DiscoveryModule`) | Service (`DiscoveryService`)                         |
| ----------- | -------------------------- | ---------------------------------------------------- |
| `onInit`    | —                          | Resolves Redis client; prepares scripts              |
| `onStart`   | Triggers service start     | Opens pub/sub; registers this node; starts heartbeat |
| `onStop`    | Triggers service stop      | Stops heartbeat; deregisters; closes pub/sub         |
| `onDestroy` | —                          | Final cleanup                                        |

In `clientMode: true`, registration is skipped but pub/sub still
opens on `onStart` so the node receives events.

### `titan-events`

Each sub-service has its own `onInit` / `onDestroy`; the
orchestrator `EventsService` adds `onStart` / `onStop`.

| Hook        | What happens                                                                 |
| ----------- | ---------------------------------------------------------------------------- |
| `onInit`    | History / metadata / scheduler / validation services initialise their stores |
| `onStart`   | Bus subscribes; persisted scheduled jobs are re-armed                        |
| `onStop`    | Drain in-flight emissions; flush history                                     |
| `onDestroy` | Release all listener references                                              |

### `titan-health`

| Hook        | What it does |
| ----------- | ------------ |
| `onInit`    | —            |
| `onStart`   | —            |
| `onStop`    | —            |
| `onDestroy` | —            |

`titan-health` is **lifecycle-passive**: the module registers its
indicators eagerly at config time (inside `forRoot` / `forFeature`),
not in a lifecycle hook. Checks are *pull-driven* — Kubernetes probes
invoke `check()` on demand. No background loop runs.

### `titan-lock`

| Hook        | What it does                                               |
| ----------- | ---------------------------------------------------------- |
| `onInit`    | —                                                          |
| `onStart`   | —                                                          |
| `onStop`    | Best-effort: releases locks currently held by this process |
| `onDestroy` | —                                                          |

> Releasing locks on `onStop` is best-effort because the process
> may already be in a degraded state. Always set a lock TTL so
> abandoned locks self-expire.

### `titan-metrics`

Uses **explicit** `start()` / `stop()` methods rather than the
lifecycle interface, called from module setup.

| Method    | What it does                                                                |
| --------- | --------------------------------------------------------------------------- |
| `start()` | Starts periodic collection (process, system, RPC); kicks off flush interval |
| `stop()`  | Drains the buffer to storage; stops collection; calls `cleanup()`           |

The module calls `start()` during initialisation and `stop()`
during shutdown — you don't have to wire it up by hand.

### `titan-notifications`

| Hook        | What it does                                                          |
| ----------- | --------------------------------------------------------------------- |
| `onInit`    | Builds channel registry; resolves transports                          |
| `onStart`   | Producer connects to messaging transport; worker subscribes to queues |
| `onStop`    | Stop consuming new messages; drain in-flight delivery                 |
| `onDestroy` | Disconnect transport; release channel resources                       |

`forWorker(...)` registrations subscribe to the messaging transport
on `onStart` — the producer (registered via `forRoot`) opens its
publisher connection in the same phase.

### `titan-pm`

| Hook        | What it does                                                                             |
| ----------- | ---------------------------------------------------------------------------------------- |
| `onInit`    | Initialise registry / spawner / health checker / metrics                                 |
| `onStart`   | Start any auto-start processes from config                                               |
| `onStop`    | Send `SIGTERM` to every child; wait for the configured grace period; `SIGKILL` survivors |
| `onDestroy` | Final reap of any stragglers                                                             |

Crash-supervised children stop in the **opposite** order they
started — gives you a chance to drain a worker before its
dependencies disappear.

### `titan-ratelimit`

Uses an explicit `destroy()` method called during module shutdown.

| Method      | What it does                                                                       |
| ----------- | ---------------------------------------------------------------------------------- |
| `destroy()` | Stops the periodic cleanup timer on in-memory storage; closes Redis storage client |

### `titan-redis`

| Hook              | What it does                                                          |
| ----------------- | --------------------------------------------------------------------- |
| `onModuleInit`    | Establishes the connection(s) configured via `forRoot` / `forFeature` |
| `onModuleDestroy` | Disconnects all named clients gracefully (`closeAllClients()`)        |

`titan-redis` uses the `onModuleInit` / `onModuleDestroy` hook names
(on `RedisManager`) — functionally equivalent to `onInit` / `onDestroy`
in newer modules. There is no `onStop` hook; cleanup runs in
`onModuleDestroy`.

### `titan-scheduler`

| Hook        | What it does                                              |
| ----------- | --------------------------------------------------------- |
| `onInit`    | Resolve persistence backend; rebuild registry from store  |
| `onStart`   | Schedule all registered jobs; arm cron timers / intervals |
| `onStop`    | Cancel pending timers; flush in-flight to persistence     |
| `onDestroy` | Release listeners; close persistence backend              |

The split matters: `onInit` rebuilds the *registry*, `onStart`
actually *arms* the timers. Don't expect jobs to fire between
those two.

### `titan-telemetry-relay`

Not a module — instantiated directly. Lifecycle is manual:

| Method    | What it does                                                   |
| --------- | -------------------------------------------------------------- |
| `start()` | Open WAL; start flush interval; start aggregator subscriptions |
| `stop()`  | Stop flush; drain WAL; close aggregator                        |

When integrated into an app via DI, wrap it in a thin lifecycle
adapter:

```typescript
@Injectable()
class TelemetryRelayAdapter implements OnStart, OnStop {
  constructor(private readonly relay: TelemetryRelayService) {}
  async onStart() { await this.relay.start(); }
  async onStop()  { await this.relay.stop();  }
}
```

## Built-in modules

<span class="module-badge">built-in · @omnitron-dev/titan · config + logger</span>

### `config`

| Hook        | What it does                                                       |
| ----------- | ------------------------------------------------------------------ |
| `onInit`    | Load all `sources` in order; deep-merge; validate against `schema` |
| `onStart`   | If `watchForChanges: true`, start file watchers                    |
| `onStop`    | Stop file watchers                                                 |
| `onDestroy` | Release schema reference                                           |

If validation fails in `onInit`, the app aborts boot with the
underlying Zod error — no partial start.

### `logger`

| Hook        | What it does                                                            |
| ----------- | ----------------------------------------------------------------------- |
| `onInit`    | Open transports (file, network, console)                                |
| `onStart`   | —                                                                       |
| `onStop`    | **Flush** all transports before draining (critical: lossy without this) |
| `onDestroy` | Close transport handles                                                 |

The flush in `onStop` is why graceful shutdown matters for
observability — `SIGKILL` skips it and you lose the last few
seconds of logs.

## Putting it together — full boot/shutdown timeline

```mermaid
sequenceDiagram
  participant App
  participant Config
  participant Logger
  participant Redis
  participant DB as Database
  participant Lock
  participant Disc as Discovery
  participant Cache
  participant RL as RateLimit
  participant Sched as Scheduler
  participant Notif as Notifications
  participant Metr as Metrics
  participant User as Your services

  App->>Config: onInit (load + validate)
  App->>Logger: onInit (open transports)
  App->>Redis: onModuleInit (connect)
  App->>DB: (no onInit)
  App->>Disc: onInit
  App->>Cache: (no onInit)
  App->>Notif: onInit
  App->>Sched: onInit (rebuild registry)
  App->>Metr: start()
  App->>User: onInit

  Note over App: ─── onInit complete ───

  App->>Config: onStart (watchers)
  App->>Disc: onStart (register node + heartbeat)
  App->>Notif: onStart (subscribe)
  App->>Sched: onStart (arm timers)
  App->>User: onStart

  Note over App: ─── App ready ───
  Note over App: ─── app.stop() called ───

  App->>User: onStop (reverse order)
  App->>Sched: onStop (cancel timers + flush)
  App->>Notif: onStop (drain delivery)
  App->>Disc: onStop (deregister)
  App->>Lock: onStop (release best-effort)
  App->>DB: onStop (drain pool)
  App->>Config: onStop (stop watchers)
  App->>Metr: stop() (drain buffer)
  App->>Logger: onStop (FLUSH transports)

  Note over App: ─── onDestroy phase ───
  App->>User: onDestroy
  App->>Sched: onDestroy
  App->>Notif: onDestroy
  App->>Disc: onDestroy
  App->>RL: destroy()
  App->>Redis: onModuleDestroy (disconnect)
  App->>Logger: onDestroy
  App->>Config: onDestroy
```

## Practical rules

- **Long-running background work belongs in `onStart`, not
  `onInit`.** Hooks at `onInit` should be fast — they block every
  later module from initialising.
- **Resources opened in `onInit` should close in `onDestroy`.**
  Resources started in `onStart` should stop in `onStop`. Mixing
  them causes subtle leaks.
- **`onStop` runs in reverse dependency order.** Your services
  drain before the modules they depend on. Don't rely on Redis
  inside an `onStop` if your service's `onStop` was already
  followed by Redis disconnecting.
- **Don't throw in `onStop` / `onDestroy`.** A throw aborts the
  remaining cleanup chain. Catch + log, let the process exit.
- **Heavy CPU work in `onStart` blocks readiness.** If you must
  precompute, do it in a background task started from `onStart`
  and gate readiness on a separate flag.

## See also

- [Application / Lifecycle](/docs/titan/application/lifecycle) — framework-level explanation
- [Application / Shutdown](/docs/titan/application/shutdown) — signals, timeouts, graceful drain
- [Module map](/docs/titan/modules/module-map) — dependency order determines lifecycle order
- [Options patterns](/docs/titan/modules/options-patterns) — when `onInit` factories run

---

---
id: "titan/modules/lock"
title: "titan-lock"
---

# titan-lock

<span class="module-badge">official · @omnitron-dev/titan-lock · stable</span>

Distributed locks over Redis with UUIDv7 ownership, Lua-script-based
atomic compare-and-delete release, TTL leasing with auto-extend,
configurable retries with exponential backoff, and per-operation
failure tracking to suppress log spam under sustained contention.

Requires [`titan-redis`](/docs/titan/modules/redis) (peer dependency).

```bash
pnpm add @omnitron-dev/titan-lock @omnitron-dev/titan-redis
```

## When you need it

- **Exactly-once jobs across a fleet** — multiple pods running the
  same scheduler; only one should execute a given run.
- **Critical sections across processes** — write a payment, drain a
  queue, allocate a unique resource.
- **Coordinated migrations** — only one pod runs the schema upgrade.

## Quickstart

```typescript
import { TitanLockModule } from '@omnitron-dev/titan-lock';
import { TitanRedisModule } from '@omnitron-dev/titan-redis';

@Module({
  imports: [
    TitanRedisModule.forRoot({ config: { url: env.REDIS_URL } }),
    TitanLockModule.forRoot({
      defaultTtl:        30_000,        // ms
      keyPrefix:         'lock',
      defaultRetries:    3,
      defaultRetryDelay: 100,            // ms — exponential backoff applied per attempt
    }),
  ],
})
class AppModule {}
```

Async config:

```typescript
TitanLockModule.forRootAsync({
  imports:    [ConfigModule],
  useFactory: (config: ConfigService) => ({
    defaultTtl:        config.get('locks.ttl'),
    redisClientName:   config.get('locks.redisNamespace'),
  }),
  inject: [ConfigService],
})
```

## `ILockModuleOptions`

| Option              | Type          | Default  |
| ------------------- | ------------- | -------- |
| `defaultTtl`        | `number` (ms) | `30_000` |
| `keyPrefix`         | `string`      | `'lock'` |
| `defaultRetries`    | `number`      | `3`      |
| `defaultRetryDelay` | `number` (ms) | `100`    |
| `redisClientName`   | `string`      | —        |
| `isGlobal`          | `boolean`     | `false`  |

## `DistributedLockService` — the API

```typescript
import { DistributedLockService, LOCK_SERVICE_TOKEN } from '@omnitron-dev/titan-lock';

@Service({ name: 'billing' })
class BillingService {
  constructor(@Inject(LOCK_SERVICE_TOKEN) private readonly locks: DistributedLockService) {}

  @Public()
  async charge(invoiceId: string) {
    return this.locks.withLock(`invoice:${invoiceId}`, async () => {
      return this.processCharge(invoiceId);
    }, { ttl: 30_000, retries: 3 });
  }
}
```

### Low-level — explicit acquire / release

| Method                           | Returns          | Purpose                                        |
| -------------------------------- | ---------------- | ---------------------------------------------- |
| `acquireLock(key, ttlMs)`        | `string \| null` | Lock ID (UUIDv7) on success; `null` if held    |
| `releaseLock(key, lockId)`       | `boolean`        | `true` if released; `false` if not owner       |
| `extendLock(key, lockId, ttlMs)` | `boolean`        | `true` if extended; `false` if not owner       |
| `isLocked(key)`                  | `boolean`        | Is the key currently held by anyone?           |
| `getLockTtl(key)`                | `number` (ms)    | Remaining TTL; `-1` no expiry, `-2` not exists |

```typescript
const lockId = await this.locks.acquireLock('jobs:cleanup', 60_000);
if (!lockId) return; // someone else has it

try {
  await doWork();

  // Halfway through, the work is taking longer than expected — extend.
  await this.locks.extendLock('jobs:cleanup', lockId, 60_000);

  await finishWork();
} finally {
  await this.locks.releaseLock('jobs:cleanup', lockId);
}
```

### High-level — `withLock`

```typescript
async withLock<T>(
  key: string,
  fn: () => Promise<T>,
  options?: {
    ttl?:                number;     // ms (default: module defaultTtl)
    retries?:            number;     // attempts (default: module defaultRetries)
    retryDelay?:         number;     // ms (default: module defaultRetryDelay)
    exponentialBackoff?: boolean;    // default: true — delay × (attempt + 1)
    skipOnLockFailure?:  boolean;    // default: false — return undefined instead of throwing
    autoRenew?:          boolean;    // default: true — extend the lock while fn runs so it can't expire mid-execution
    renewIntervalMs?:    number;     // default: floor(ttl / 3)
  },
): Promise<T>
```

```typescript
// Skip if someone else has the lock (idempotent scheduled task)
const result = await this.locks.withLock('cleanup', () => cleanup(), {
  ttl:               60_000,
  retries:           5,
  retryDelay:        200,
  skipOnLockFailure: true,
});
if (result === undefined) {
  this.logger.info('skipped — another instance is running cleanup');
}
```

By default (`autoRenew: true`) `withLock` runs a watchdog that
extends the lock every `ttl / 3` while `fn` is running, so a function
that outlives its TTL cannot let the lock expire and be re-acquired
by another node mid-execution. If a renewal ever finds the lock no
longer owned, it logs an error and stops renewing.

## `@WithDistributedLock` decorator

Wraps a method with auto acquire / release. Returns `undefined` if
another instance holds the lock — perfect for scheduled jobs that
should run on exactly one pod per fleet.

```typescript
import { Cron, CronExpression } from '@omnitron-dev/titan-scheduler';
import { WithDistributedLock } from '@omnitron-dev/titan-lock';

@Injectable()
class ScheduledTasksService {
  constructor(
    @Inject(LOCK_SERVICE_TOKEN)   private readonly __lockService__: IDistributedLockService,
    @Inject(LOGGER_SERVICE_TOKEN) private readonly loggerModule:    ILoggerModule,
  ) {}

  @Cron(CronExpression.EVERY_30_SECONDS)
  @WithDistributedLock('jobs:poll-queue', 25_000)
  async pollQueue() {
    // Exactly one pod runs this per 30s window.
  }
}
```

> **Wiring requirement.** The decorator looks up the lock service on
> the instance under the property name `__lockService__`. Inject it
> there. Optionally inject `loggerModule` or `logger` for debug
> output; the decorator falls back to console otherwise.

`Lock` is exported as an alias for `WithDistributedLock`.

## Lock semantics

```mermaid
sequenceDiagram
  participant A as Pod A
  participant B as Pod B
  participant R as Redis

  A->>R: SET lock:job <uuid-A> NX PX 30000
  R-->>A: OK   (acquired)
  B->>R: SET lock:job <uuid-B> NX PX 30000
  R-->>B: nil  (held)
  B->>B: retry with backoff
  A->>R: EVAL release-script lock:job uuid-A
  R-->>A: 1    (deleted)
  B->>R: SET lock:job <uuid-B> NX PX 30000
  R-->>B: OK   (acquired)
```

### Embedded Lua scripts

`releaseLock` and `extendLock` use Lua to make the
"check-then-delete" / "check-then-extend" atomic:

```lua
-- release: only delete if I own the lock
if redis.call("get", KEYS[1]) == ARGV[1] then
  return redis.call("del", KEYS[1])
else
  return 0
end
```

```lua
-- extend: only refresh TTL if I own the lock (uses pexpire — ms)
if redis.call("get", KEYS[1]) == ARGV[1] then
  return redis.call("pexpire", KEYS[1], ARGV[2])
else
  return 0
end
```

The lock ID is a UUIDv7 generated per acquire. Ownership cannot be
forged. If a holder dies before releasing, the TTL expires the key
and another acquirer takes over.

## Failure tracking

Each operation has its own `FailureTracker` (one for `acquireLock`,
one for `releaseLock`, one for `extendLock`, and one shared by the
status reads `isLocked` / `getLockTtl`), keyed per lock name
(`acquire:<key>`, `release:<key>`, …). The tracker categorises a
stream of failures (first / continuing / persistent / suppress /
recovery) and only logs at the matching level. Outcome: a single
noisy lock under contention does not flood your logs with thousands
of identical warnings.

## Tokens

| Token                              |
| ---------------------------------- |
| `LOCK_SERVICE_TOKEN`               |
| `LOCK_OPTIONS_TOKEN`               |
| `DEFAULT_LOCK_PREFIX` (= `'lock'`) |

## Anti-patterns

- **Using locks where idempotency would do.** A lock is coordination
  state. If your work is idempotent (write with a unique constraint,
  upsert, etc.), prefer that over a lock.
- **Long-held locks across slow I/O.** A 60 s lock around a network
  call is a 60 s window for staleness. Either extend periodically
  (`extendLock`) or break the work into smaller transactions.
- **Forgetting `try/finally` with manual acquire.** A crash between
  `acquireLock` and `releaseLock` leaves the lock held until TTL.
  Use `withLock` whenever you can.
- **TTL too short for the work.** Lock expires mid-job; another
  process picks it up; both run; you've broken mutual exclusion.
  Pick TTL = (max expected duration × 2) and extend if needed.
- **TTL too long.** A crashed holder leaves work blocked until the
  TTL expires. Keep TTL small; extend on heartbeat.

## See also

- [`titan-scheduler`](/docs/titan/modules/scheduler) — pair with `@WithDistributedLock`
- [`titan-redis`](/docs/titan/modules/redis) — the underlying client
- [Resilience / Circuit Breaker](/docs/titan/resilience/circuit-breaker) —
  pair with locks when the locked work touches a flaky backend

---

---
id: "titan/modules/logger"
title: "LoggerModule"
---

# LoggerModule

<span class="module-badge">built-in · @omnitron-dev/titan · /module/logger · stable</span>

Structured pino-based logging with six levels, child loggers with
bound context, a processor pipeline (transform / drop), pino-native
redaction, raw-stream and `ITransport` fan-out, optional pretty-print,
automatic per-service binding, decorator-based property injection, and
method-level auto-instrumentation. **Auto-loaded** as a core module
by every Titan application (unless `disableCoreModules: true`).
No extra install required — ships inside `@omnitron-dev/titan`.

By default the logger writes **structured JSON**, one record per line,
to an async stdout destination (or a pino `multistream` over stdout
plus any extra `destinations`/`transports` you supply). Set
`prettyPrint` for colorised human-readable output in dev. It is built
on [pino](https://getpino.io); its JSON is what log shippers (Loki,
ELK, Datadog) expect.

## When you need it

- **Every backend service.** A structured logger is non-negotiable
  for production.
- **Per-call context propagation.** Trace IDs, request IDs, user
  IDs attached to every log line automatically via child loggers.
- **Multiple destinations.** Async stdout for ingestion, plus extra
  `destinations` raw streams and/or `ITransport` sinks (file, socket,
  ClickHouse / Loki / OTLP sink).
- **Transform / drop pipeline.** Enrich every line, or drop noisy
  records, via the `processors` pipeline (runs on the hot path,
  inherited by child loggers).
- **Selective redaction.** Strip auth headers and PII via pino's
  native `redact` (or a `RedactionProcessor`) before they hit output.
- **Pretty dev output.** `prettyPrint` for colorised human-readable
  lines locally; structured JSON in production.

## Quickstart

```typescript
import { createWriteStream } from 'node:fs';
import { LoggerModule, RedactionProcessor } from '@omnitron-dev/titan/module/logger';

@Module({
  imports: [
    LoggerModule.forRoot({
      level: 'info',
      // Colorised human-readable output for dev; leave off (JSON) in prod.
      prettyPrint: process.env.NODE_ENV !== 'production',
      // pino-native redaction — pino-optimised, hot-path, supports
      // path syntax + `*` wildcards. The simplest path for plain redaction.
      redact: ['password', 'token', 'headers.authorization'],
      // Processor pipeline — runs on every line (child loggers too). Use it
      // for transform/drop logic; RedactionProcessor is the redaction case.
      processors: [new RedactionProcessor(['ssn', 'creditCard'])],
      // Raw-stream fan-out besides stdout (stdout is the first stream).
      destinations: [
        { stream: createWriteStream('/var/log/myapp/app.log'), level: 'info' },
      ],
      // Object-receiving sinks, off the hot path + error-isolated.
      // transports: [new HttpSinkTransport('https://logs.example/ingest')],
    }),
  ],
})
class AppModule {}
```

> **Pretty-print, processors, and transports all work.** Set
> `prettyPrint` for colorised dev output (via pino-pretty); leave it off
> for structured JSON in production. `processors` run on every line and
> can transform or drop records; `transports` receive each parsed record
> off the hot path. The `RedactionProcessor` above genuinely redacts —
> though for plain path redaction the `redact` option is simpler and
> pino-optimised. See [Processors](#redaction-and-enrichment) and
> [Extra sinks](#extra-sinks-destinations-and-transports) below.

## The `ILogger` interface

Pino-style, **object-first** signatures (the bound object comes first,
then an optional message). Six levels, plus level introspection and a
timing helper:

```typescript
interface ILogger {
  // Six levels — object-first (obj, msg?, ...args) or (msg, ...args)
  trace(obj: object, msg?: string, ...args: any[]): void;
  trace(msg: string, ...args: any[]): void;
  debug(obj: object, msg?: string, ...args: any[]): void;
  debug(msg: string, ...args: any[]): void;
  info(obj: object,  msg?: string, ...args: any[]): void;
  info(msg: string,  ...args: any[]): void;
  warn(obj: object,  msg?: string, ...args: any[]): void;
  warn(msg: string,  ...args: any[]): void;
  error(obj: object, msg?: string, ...args: any[]): void;
  error(msg: string, ...args: any[]): void;
  fatal(obj: object, msg?: string, ...args: any[]): void;
  fatal(msg: string, ...args: any[]): void;

  // Child logger with bound context — inherits the parent's pino
  // config (level, redact, destinations) and adds the supplied
  // bindings to every log line through it
  child(bindings: object): ILogger;

  // Timing helper — returns a stop() that logs the elapsed duration
  time(label?: string): () => void;

  // Level introspection / control
  isLevelEnabled(level: LogLevel): boolean;
  setLevel(level: LogLevel): void;
  getLevel(): LogLevel;
}
```

> **Object-first calls.** Because signatures are Pino-style, pass
> structured context as the first argument and the message second:
> `logger.error({ err, userId }, 'repo failed')`. The single-arg
> string form `logger.info('done')` also works.

## Usage in a service

Inject the **`ILogger`** via `LOGGER_TOKEN` (the convenient alias the
module derives from the root logger) — or use the `@Logger()` property
decorator (below). The DI-injected `LoggerService` is the *module*
(`ILoggerModule`: `create` / `child` / `setLevel` / …), not an
`ILogger`; reach a logger via `loggerService.logger` or
`loggerService.create(name)`.

```typescript
import { Service, Inject } from '@omnitron-dev/titan';
import { Public } from '@omnitron-dev/titan/decorators';
import { LOGGER_TOKEN, type ILogger } from '@omnitron-dev/titan/module/logger';

@Service({ name: 'users' })
class UsersService {
  constructor(@Inject(LOGGER_TOKEN) private readonly logger: ILogger) {}

  @Public()
  async findById(id: string) {
    this.logger.info({ id }, 'findById');         // object first, message second
    try {
      return await this.repo.findById(id);
    } catch (e) {
      this.logger.error({ id, err: e }, 'repo failed');
      throw e;
    }
  }
}
```

For a per-service tag, bind it with a child logger
(`this.logger.child({ service: 'UsersService' })`) — there is no
automatic class-name binding.

## Per-request child loggers

For request-scoped context (trace IDs, user IDs):

```typescript
@Public()
async findById(id: string, @Context() ctx: NetronContext) {
  const log = this.logger.child({
    traceId:   ctx.traceId,
    userId:    ctx.auth?.userId,
    requestId: ctx.requestId,
  });

  log.info({ id }, 'findById');
  return this.repo.findById(id);
}
```

→ See [Logging / Child Loggers](/docs/titan/logging/child-loggers) for
the full reference.

## Decorators

```typescript
import { Logger, Log, Monitor } from '@omnitron-dev/titan/module/logger';
```

### `@Logger()` — property injection

```typescript
@Service({ name: 'users' })
class UsersService {
  @Logger() private readonly logger!: ILogger;

  @Public()
  async findById(id: string) {
    this.logger.info({ id }, 'findById');
  }
}
```

### `@Log()` — method auto-logging

```typescript
@Public()
@Log()                                  // logs entry + exit at debug
async findById(id: string) { /* … */ }

@Public()
@Log({ level: 'info', includeArgs: true, includeResult: false })
async create(input: CreateInput) { /* … */ }
```

Options: `level` (`'trace'|'debug'|'info'|'warn'|'error'`),
`includeArgs`, `includeResult`, `message`. The method's instance must
expose a logger (a `logger` / `_logger` / `log` property) or the
decorator silently no-ops.

### `@Monitor()` — performance instrumentation

```typescript
@Public()
@Monitor()                              // logs duration + outcome
async heavyComputation(input: Input) { /* … */ }
```

## Services and helpers

| Symbol               | Purpose                                                                                                                                                                                   |
| -------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `LoggerService`      | DI-injected logger manager (`ILoggerModule`): `create` / `child` / `setLevel` / `logger` getter                                                                                           |
| `ConsoleTransport`   | Bundled example `ITransport`; ctor takes an optional `ILogger` (positional). Registered via `transports`, its `write()` receives each parsed record off the hot path                      |
| `RedactionProcessor` | Built-in `ILogProcessor`; redacts dotted paths (positional `string[]`). Runs on the hot path like any processor; for plain path redaction the `redact` option is simpler + pino-optimised |
| `createNullLogger()` | Returns an `ILogger` that discards everything (tests)                                                                                                                                     |
| `isLogger(value)`    | Type guard                                                                                                                                                                                |

## Extra sinks: `destinations` and `transports`

Two complementary ways to fan records out beyond stdout.

**`destinations` — raw Pino streams.** Each receives the serialised
JSON line as a chunk, plugged straight into Pino's `multistream`.
stdout is always the first stream, and each user stream is async-wrapped
so a slow consumer can't stall the hot path
(the stream-selection branch of `LoggerService.initialize`, plus
`wrapAsyncStream`):

```typescript
import { Writable } from 'node:stream';

class OtlpSink extends Writable {
  constructor(private readonly endpoint: string) { super(); }
  _write(chunk: Buffer, _enc: BufferEncoding, cb: (e?: Error | null) => void) {
    fetch(this.endpoint, { method: 'POST', body: chunk }).then(() => cb(), cb);
  }
}

LoggerModule.forRoot({
  destinations: [
    { stream: new OtlpSink('https://otel.internal/v1/logs'), level: 'info' },
  ],
})
```

**`transports` — object-receiving sinks.** A registered `ITransport`'s
`write(record)` **is called for every log line**, receiving the
fully-serialised record **parsed back to an object**, after the
processor pipeline and only for logs that passed the level filter.
Delivery is deferred a macrotask (off Pino's hot path) and each
transport is isolated in try/catch, so a slow or throwing transport
can't block or break logging. `flush()` is awaited by
`LoggerService.flush()` (`LoggerService.createTransportFanout` and `LoggerService.flush`):

```typescript
import type { ITransport } from '@omnitron-dev/titan/module/logger';

class HttpSinkTransport implements ITransport {
  name = 'http-sink';
  private buf: any[] = [];
  constructor(private readonly url: string) {}
  write(record: any) {                 // record is a parsed object
    this.buf.push(record);
  }
  async flush() {                      // awaited on shutdown
    if (this.buf.length === 0) return;
    const batch = this.buf.splice(0);
    await fetch(this.url, { method: 'POST', body: JSON.stringify(batch) });
  }
}

LoggerModule.forRoot({
  transports: [new HttpSinkTransport('https://logs.example/ingest')],
})
```

> **Configure transports at `forRoot`.** The fan-out stream that drives
> `ITransport.write()` is created at init, and only when at least one
> transport is registered then (the multistream branch of `LoggerService.initialize`). So
> `addTransport()` after init delivers records **only if** the logger
> started with ≥1 transport; with zero transports at init there is no
> fan-out stream, so a later `addTransport()`'s `write()` won't receive
> records (its `flush()` still runs). Register transports up front for
> guaranteed delivery.

Use **`destinations`** for raw byte sinks (files, sockets, an OS
shipper) and **`transports`** when you want the parsed record object,
hot-path isolation, or a `flush()` lifecycle. There is **no
`FileTransport`** and no built-in rotation — use a rotating
`destinations` stream, or ship stdout with an OS-level log shipper.

→ See [Logging / Transports](/docs/titan/logging/transports).

## Delivery failures and bounded output

Every raw destination, including stdout, has an independent failure boundary.
An `ENOSPC`, synchronous write exception, callback error or emitted stream error
cannot abort delivery to healthy destinations or escape the logging call. JSON
stdout uses Pino's asynchronous destination even when a durable journal is also
configured. Each forwarding queue is bounded to 1 MiB; stdout's native buffer is
separately bounded to 1 MiB. Capacity overflow is counted, rather than allowing an
unavailable output to consume unlimited process memory.

`LoggerService.getSinkHealth()` returns per-sink pending bytes, failed writes,
unconfirmed bytes, dropped bytes and the current error. Retry attempts for future
records back off from one to thirty seconds. A write that does not acknowledge
within thirty seconds remains uncertain; no further writes are admitted to that
underlying sink until the original operation settles. A closed destination must
be replaced by its owner. `ITransport` calls have a five-second observation budget;
a timed-out call remains in flight and is not duplicated. Other transports still
receive the record independently.

The guard does not replay records whose partial delivery is unknown. A recovered
sink can accept future records while retaining its historical gap counters.
`flush()` waits for healthy destinations and aggregates delivery failures; it
rejects when any sink has an unresolved or historical unconfirmed/drop gap. This
is delivery evidence, not a claim that every writable stream performs `fsync`.
A synchronous durable journal destination is invoked before the logging call
returns; failure isolation still lets the application continue when that sink
fails. Inspect delivery evidence and await `flush()` where confirmation is
required. Shutdown code must not interpret a rejected flush as successful
durability.

## Redaction and enrichment

Redaction has two working paths. For **plain path redaction**, prefer
**pino's native `redact` option** — `LoggerService` forwards it straight
to pino when it builds the root logger (`redact: config.redact || []` in `LoggerService.initialize`). It
runs inside pino's serialisation, on the hot path, for every line
including child loggers; it is pino-optimised and supports pino path
syntax (`*` wildcards, bracket notation) and the object form for a
custom censor:

```typescript
LoggerModule.forRoot({
  redact: ['password', 'token', 'apiKey', 'headers.authorization', '*.creditCard'],
})
```

The list can also come from `ConfigService` under `logger.redact`
(read at startup in `getConfiguration`). Titan redacts nothing by
default (`redact: config.redact || []`) — set the list per app.

For **arbitrary transform/drop logic** (conditional redaction,
cross-field rules, computed values, or dropping a record), use the
`processors` pipeline — it runs on the hot path and is inherited by
child loggers. The bundled `RedactionProcessor` covers the simple
dotted-path case and genuinely redacts when registered:

```typescript
import { RedactionProcessor } from '@omnitron-dev/titan/module/logger';

LoggerModule.forRoot({ processors: [new RedactionProcessor(['ssn', 'creditCard'])] });
```

For the other shaping needs:

- **Static fields on every line** → `base`
  (`forRoot({ base: { region, version } })`) or `setContext(...)`, both
  of which flow into pino's bindings (lower-ceremony than a processor).
- **Per-request context** (trace ID, user ID) → a **child logger** —
  see [Per-request child loggers](#per-request-child-loggers). This is
  the idiomatic way to attach per-scope fields.
- **Transform or drop per line** → a `processors` entry. Each runs on
  every log call and returns the modified record, or `null`/`undefined`
  to drop it (`LoggerService.runProcessors`).
- **Filtering by level** → set the level (`forRoot({ level })` or
  `setLevel`); pino drops below-threshold records before serialising
  (cheaper than a dropping processor for a pure level cut).

→ See [Logging / Processors](/docs/titan/logging/processors).

## Pipeline

```mermaid
flowchart LR
  Code[Service.logger.info]
  Code --> Proc[processors<br/>transform / drop]
  Proc -->|dropped| X[discarded]
  Proc --> Bind[LoggerService<br/>base + child bindings]
  Bind --> Pino[pino instance<br/>redact + serialise]
  Pino --> Out1[stdout async / pino-pretty]
  Pino --> Out2[destinations: file / socket]
  Pino --> Fan[transport fan-out<br/>off hot path]
  Fan --> T1[ITransport.write parsed record]
```

The `processors` pipeline runs first (via pino's `logMethod` hook,
inherited by child loggers) and can drop a record. The fan-out is
pino's `multistream` over stdout — JSON, or pino-pretty when
`prettyPrint` is set — plus any `destinations` streams and a fan-out
stream that drives `ITransport` sinks off the hot path. Redaction
happens inside pino's serialisation (`redact`) and/or as a processor.

## Tokens

| Token                     | Purpose                 |
| ------------------------- | ----------------------- |
| `LOGGER_TOKEN`            | Default `ILogger`       |
| `LOGGER_SERVICE_TOKEN`    | `LoggerService` wrapper |
| `LOGGER_OPTIONS_TOKEN`    | Resolved options        |
| `LOGGER_TRANSPORTS_TOKEN` | Registered transports   |
| `LOGGER_PROCESSORS_TOKEN` | Registered processors   |

## Setting the level from config / at runtime

When ConfigModule is loaded, the logger reads its initial settings from
the `logger.*` namespace **once at construction** — `logger.level`,
`logger.prettyPrint`, `logger.redact`, `logger.base`, and so on
(`LoggerService.getConfiguration`):

```yaml
logger:
  level: debug
```

There is **no live `config:changed` subscription** — editing the file
at runtime does not re-level an already-constructed logger. To change
the level on a running process, call `setLevel()` imperatively on the
`LoggerService` (or any `ILogger`):

```typescript
loggerService.setLevel('debug');   // or: logger.setLevel('debug')
```

## Anti-patterns

- **`console.log` in services.** Bypasses the framework — no
  per-service context, no level, no JSON formatting. Always use
  the injected logger.
- **Logging in tight loops.** A `debug` log inside a 100 K-iteration
  loop floods the output even if it's filtered out. Hoist the
  log outside the loop, or raise the level.
- **PII in logs without redaction.** Set the `redact` option (pino
  native, simplest for path redaction) — or a `RedactionProcessor` /
  custom processor for conditional logic — for anything shipping off-host.
- **Adding the first `ITransport` after init.** The fan-out stream is
  created at `forRoot` only when ≥1 transport is registered then; a
  later `addTransport()` on a logger that started with none won't
  deliver `write()`. Register transports up front.
- **Heavy work in a `processor.process()`.** It runs on every line
  (after the level filter). Keep it cheap; offload expensive shipping
  to an `ITransport`, which runs off the hot path.
- **Synchronous `destinations` streams blocking the event loop.** Each
  user stream is async-wrapped, but a stream that does a blocking
  syscall in `_write` still stalls; keep extra sinks async / non-blocking.

## See also

- [Logging / Overview](/docs/titan/logging/overview) — conceptual guide
- [Logging / Transports](/docs/titan/logging/transports) — async stdout, `destinations`, `ITransport` sinks
- [Logging / Processors](/docs/titan/logging/processors) — transform / drop pipeline + redaction
- [Logging / Child Loggers](/docs/titan/logging/child-loggers) — bound context
- [Tracing](/docs/titan/tracing) — automatic correlation with log lines

---

---
id: "titan/modules/mailer"
title: "titan-mailer"
---

# titan-mailer

<span class="module-badge">official · @omnitron-dev/titan-mailer · stable</span>

Transactional email for Titan: lifecycle-managed Nodemailer SMTP pools, typed templates,
a concrete email notification channel, and a durable outbox consumer using Titan Notifications.

## Application module

```ts
import { MailerModule, MailerService } from '@omnitron-dev/titan-mailer';

const mailModule = MailerModule.forRoot({
  smtp: process.env.SMTP_URL!, // smtps://...:465 or smtp://...:587 with STARTTLS
  from: 'Research <noreply@example.org>',
  pool: { maxConnections: 3 },
  maxPending: 1000,
});
// Import mailModule in your Titan @Module; inject MailerService or MAILER.
```

`forRootAsync({ imports, inject, useFactory })` resolves configuration through Nexus.
`verifyOnStart: true` performs SMTP authentication during provider creation. Otherwise
call `verify()` from a readiness probe; it sends no email. Titan disposal calls
`onDestroy()`, which stops new submissions, drains accepted operations and closes the pool.
SMTP timeouts and `maxPending` bound waiting work. The caller owns admission/retry policy.

`send()` accepts **one envelope recipient**, explicit UTF-8 plaintext, optional HTML,
memory-backed attachments and a stable Message-ID. It returns SMTP acceptance, not inbox
delivery. Fan-out creates a separate durable delivery for each recipient, avoiding partial
recipient success and accidental address disclosure. Arbitrary SMTP envelopes, filesystem
paths, URL attachments, raw MIME and security overrides are not forwarded.

TLS certificate verification and TLS 1.2+ are mandatory. Cleartext is only available with
`allowInsecureLocalhost: true` and a loopback host. SMTP URL query options are rejected.
Public errors contain a classification, never credentials, server transcripts or bodies.
`MailDeliveryError.retryable` distinguishes temporary failures from invalid/configuration/
permanent failures. Nodemailer's hidden pool requeue is disabled: one queue owns retries.

## Templates

```ts
import { MailTemplates, escapeMailHtml } from '@omnitron-dev/titan-mailer';
const templates = new MailTemplates<{ welcome: { name: string } }>({
  welcome: ({ name }) => ({
    subject: 'Welcome',
    text: `Hello, ${name}`,
    html: `<p>Hello, ${escapeMailHtml(name)}</p>`,
  }),
});
const rendered = templates.render('welcome', { name: 'Ada' });
```

The registry is renderer-independent: Handlebars, MJML or a function may return the same
`RenderedMail` contract. Render trusted templates with escaped data, then persist that
result in the application's business transaction. Rendering never makes a network request.
Plaintext is required; the module does not invent it by stripping arbitrary scientific HTML.

## Redis notification workers

Use `NotificationsModule.forRoot({ redis })`, inject `NOTIFICATIONS_TRANSPORT`, and create
`MailDeliveryWorker` from `@omnitron-dev/titan-mailer/notifications` with that bus,
a `MailOutbox`, `MailerService`, and `{ channel, group }`. Start it in the supervised
worker process and stop it **before** destroying its database, mailer and transport.
The worker does not own injected dependencies; the Titan container disposes them.

`MailOutbox` is the application's durable transaction boundary:

- `pending(limit)` lists due opaque IDs, without message bodies or credentials.
- `deliver(id, send)` locks/claims one due row. Completed or cancelled IDs are harmless.
- SMTP acceptance and deletion/completion are recorded in the same database transaction.
- A failed attempt retains the message with backoff, or records an explicit terminal failure.
- Expired/replaced tokens cancel their queued mail. Database failure rejects; Redis is not ACKed.

Only an ID travels through Redis. Polling reconciliation republishes due IDs after lost or
trimmed wake-ups; it is bounded by `batchSize`, `intervalMs` and `maxConcurrent`. Concurrent
consumers share a group. Worker health covers transport connectivity and failed reconciliation/
delivery. Failed delivery policy and administrative re-drive remain in the durable store.

### PostgreSQL delivery store

`@omnitron-dev/titan-mailer/postgres` exports `PostgresMailOutbox`, the database port,
and `POSTGRES_MAIL_OUTBOX_SCHEMA`. Apply the schema in your application migration and
pass an existing pool/transaction executor; the adapter never creates a second connection pool.
An existing Titan database transaction can supply this executor. Applications with an older
mail schema must migrate its columns explicitly before enabling the worker.

```typescript
const outbox = new PostgresMailOutbox(database, {
  messageIdDomain: 'example.org',
  maxAttempts: 6,
  retrySeconds: 30,
  maxRetrySeconds: 3600,
});
```

Insert the recipient, subject, `body` (plain text), optional HTML and expiry into `mail_outbox`
inside the **same transaction as the business operation**. The application can add foreign
keys to cancel queued messages when an account or one-time credential is deleted. Do not
store SMTP credentials in the outbox.

`status(id)` distinguishes queued, retrying, accepted, failed and expired. `failed(limit)`
and `health()` return bounded operational information without addresses or message content.
`health().oldestDueSeconds` identifies an overdue queue even when Redis reports connected.
After repairing a terminal failure, `retry(id)` resets its retry budget, retains the stable
Message-ID and cumulative attempt count, and refuses completed or expired messages.
`pruneReceipts(days)` removes only old receipts with no remaining queued payload.
No failed payload is silently dropped. Keep database backups and monitor failed/overdue mail.

The delivery contract is **at least once**, assuming durable storage, eventual recovery and retry of retained failures. An SMTP server may accept a message before a worker
loses its database transaction. A stable Message-ID aids correlation but cannot make SMTP
and SQL atomic, and mail clients need not deduplicate it. Do not call this exactly once.
Use a dedicated Redis database/ACL, bounded streams and retention for durable delivery events.

`MailerEmailChannel` adapts the service to the standard notification channel registry.
It shares, and does not close, its injected MailerService. Critical security messages use
the durable outbox path, not optional marketing notification preferences.

## Delivery diagnostics

Use the `PostgresMailOutbox` option `onDelivery` to forward committed outcomes to
Titan Logger and therefore Omnitron's Console and CLI:

```ts
const outbox = new PostgresMailOutbox(database, {
  messageIdDomain: 'example.org',
  onDelivery: (event) => {
    const level = event.state === 'failed' ? 'error'
      : event.state === 'accepted' ? 'info' : 'warn';
    logger[level](
      { event: 'mail.delivery', ...event },
      `mail.delivery ${event.state} delivery=${event.deliveryId}`,
    );
  },
});
```

`MailDeliveryEvent` contains `deliveryId`, `state`, `attempts` in the current retry
cycle and `errorKind`. It has no recipient, body, credential or server response.
The callback runs only after the transaction commits. Async callbacks are supported;
a throwing or rejected observer cannot change a committed outcome or trigger another
send. These are best-effort operational logs, not a second durable event queue:
`status(id)` and the SQL receipt remain authoritative if the process exits before logging.

`MailDeliveryWorker.onError(stage, code)` reports `outbox_read_failed`,
`notification_publish_failed`, `outbox_delivery_failed` or `notification_ack_failed`.
It never passes the raw exception. A recorded SMTP rejection belongs to the outbox
outcome; it is not an uncommitted worker error. Report queue counts and
`oldestDueSeconds` in an application's standard health checks as well as transport
connectivity; a running process or connected Redis alone is insufficient.

For authentication mail, record the request's `queued` or `skipped` decision in the
application **after commit**, with its opaque delivery ID. Keep the public recovery
response uniform for missing, disabled and existing accounts. Use a keyed, domain-separated
subject reference for operator correlation, not a raw email or unkeyed email hash.
An unknown account produces no mail; `queued` means SQL committed; `accepted` means
SMTP accepted the message; none of these establishes arrival in the recipient's inbox.

## Verification

`pnpm test` uses a real local SMTP server and independent MIME parser, tests STARTTLS
refusal, reply/header injection, mixed content, rejected recipients, queue capacity and
actual Titan DI disposal. Worker contract tests cover lost wake-ups, failed commits, poison
envelopes and draining. An independent consumer also exercises the complete PostgreSQL → Redis → SMTP path
with competing workers and checks the resulting Mailpit messages.

Design references: [NestJS Mailer](https://nest-modules.github.io/mailer/docs/configuration/),
[Nodemailer SMTP](https://nodemailer.com/smtp), [pool lifecycle](https://nodemailer.com/smtp/pooled).

---

---
id: "titan/modules/metrics"
title: "titan-metrics"
---

# titan-metrics

<span class="module-badge">official · @omnitron-dev/titan-metrics · stable</span>

Pure Titan-native counters, gauges, and histograms with a Prometheus
exposition registry, pluggable storage backends (memory / PostgreSQL
/ SQLite), automatic process / system / RPC collection, retention,
re-enqueue-on-failure semantics, ghost-app filtering, and optional
sync hooks for external systems.

```bash
pnpm add @omnitron-dev/titan-metrics
```

> **No `prom-client` dependency.** This module ships its own
> Prometheus exposition format generator and time-series storage.
> Scrape with any Prometheus-compatible tool; persist locally with
> SQLite; aggregate across pods with PostgreSQL.

## When you need it

- **Dashboards.** Counters for throughput, histograms for latency,
  gauges for resource use.
- **Time-series queries inside your app.** Persist to SQLite or
  Postgres and query directly without setting up Prometheus.
- **Prometheus exposition.** Generate the standard `/metrics`
  text format from the registry.
- **Operator console without external observability stack.** The
  persistent Titan storage and the `OmnitronMetrics` facade let the
  CLI and webapp read history and database confirmation directly.

## Quickstart

```typescript
import { TitanMetricsModule } from '@omnitron-dev/titan-metrics';

@Module({
  imports: [
    TitanMetricsModule.forRoot({
      appName:    'my-api',
      collection: {
        enabled:  true,
        interval: 5_000,                 // sample every 5s
        process:  true,                  // CPU, RSS, heap
        system:   true,                  // load, free mem
        rpc:      true,                  // Netron call metrics
        custom:   true,
      },
      storage:   { type: 'memory', batchSize: 200, flushInterval: 5_000 },
      retention: { maxAge: '7d', cleanupInterval: 3_600_000 },
    }),
  ],
})
class AppModule {}
```

Async config via `forRootAsync({ useFactory, inject? })`.

## `IMetricsModuleOptions`

| Option       | Type                                                                       | Default                    |
| ------------ | -------------------------------------------------------------------------- | -------------------------- |
| `appName`    | `string` — required; tags every metric                                     | —                          |
| `collection` | `{ enabled?, interval?, process?, system?, rpc?, custom? }`                | enabled, 5s, all on        |
| `storage`    | `{ type: 'memory' \| 'postgres' \| 'sqlite', batchSize?, flushInterval? }` | `'memory'`, `200`, `5_000` |
| `retention`  | `{ maxAge?, cleanupInterval? }`                                            | `'7d'`, `1h`               |
| `sync`       | `{ enabled?, onFlush?: (batch) => Promise<void> }`                         | —                          |
| `isGlobal`   | `boolean`                                                                  | `false`                    |

## `MetricsService` — the API

```typescript
import { MetricsService, METRICS_SERVICE_TOKEN }
  from '@omnitron-dev/titan-metrics';

@Service('users@1.0.0')
class UsersService {
  constructor(@Inject(METRICS_SERVICE_TOKEN) private readonly metrics: MetricsService) {}

  @Public()
  async create(input: CreateInput) {
    this.metrics.recordTyped('counter', 'users.created.total',
      { source: input.source }, 1);

    const t0 = performance.now();
    try {
      return await this.repo.create(input);
    } finally {
      this.metrics.recordTyped('histogram', 'users.create.ms',
        { source: input.source }, performance.now() - t0);
    }
  }
}
```

### Recording API

| Method                                   | Purpose                                            |
| ---------------------------------------- | -------------------------------------------------- |
| `record(sample)`                         | Record a fully-formed sample                       |
| `recordBatch(samples[])`                 | Queue many samples at once                         |
| `recordDurable(samples[])`               | Await a persistent commit receipt                  |
| `recordTyped(type, name, labels, value)` | **Preferred** — keeps registry + storage in sync   |
| `getRegistry()`                          | Direct access to the in-memory Prometheus registry |

The `recordTyped()` method is the canonical entry point — it
updates the Prometheus registry and queues history together. The queue
is volatile until its write commits; use `recordDurable` to await that boundary. Lower-level `record` is for cases where you've
already built a sample object.

### Query API

| Method                | Purpose                                         |
| --------------------- | ----------------------------------------------- |
| `getSnapshot()`       | Point-in-time per-app snapshot (ghost-filtered) |
| `querySeries(filter)` | Query time-series with bucketing                |
| `getPrometheusText()` | Standard Prometheus exposition format           |
| `evictApp(app)`       | Drop all metrics for a given app tag            |

### Lifecycle

| Method               | Purpose                                                   |
| -------------------- | --------------------------------------------------------- |
| `start()`            | Start periodic collection + flushing + cleanup            |
| `stop()`             | Drain buffers, stop collection                            |
| `flush()`            | Await accepted samples; preserve failed batches for retry |
| `getStorageStatus()` | Inspect queue, commit evidence and errors                 |
| `cleanup()`          | Apply retention policy                                    |

## Why `recordTyped` matters (T#74)

The unified entry point exists because pre-T#74, callers had two
incompatible ways to write metrics:

1. **`record(sample)`** — wrote to BOTH registry and storage, but
   the registry routed by previously-registered type, defaulting
   to **gauge**. Wrong for counters and histograms unless someone
   else pre-registered the metric name.
2. **`getRegistry().counter(...)` etc** — wrote to ONLY the
   registry. These metrics appeared in `/metrics` (Prometheus) but
   were **invisible** to `querySeries()` — every dashboard query
   for them returned empty.

`recordTyped` takes an explicit type, writes to the registry via
the correct type-specific method, and pushes the sample to the
storage buffer in the same call. Use it unless you have a specific
reason not to.

## `MetricsQueryFilter` — the shape of time-series queries

```typescript
interface MetricsQueryFilter {
  names?:    string[];                          // ['orders.process.ms']
  apps?:     string[];                          // ['orders-api']
  from?:     string | number;                   // ISO string or ms
  to?:       string | number;
  labels?:   Record<string, string>;            // { region: 'eu-west' }
  interval?: string;                            // '1m' | '5m' | '1h' for bucketing
  limit?:    number;
}
```

Returns `MetricsTimeSeries[]`:

```typescript
interface MetricsTimeSeries {
  name:   string;
  app:    string;
  labels: Record<string, string>;
  points: Array<{ timestamp: number; value: number }>;
}
```

A typical dashboard query:

```typescript
const series = await metrics.querySeries({
  names:    ['orders.process.ms'],
  apps:     ['orders-api'],
  from:     Date.now() - 3_600_000,           // last hour
  to:       Date.now(),
  interval: '1m',                              // 60 points
});
```

Pagination uses `limit` + repeated calls with adjusted `from` —
there's no opaque cursor.

## `MetricsSnapshot` — what `getSnapshot()` returns

```typescript
interface MetricsSnapshot {
  timestamp: number;
  apps: Record<string, {
    cpu:        number;
    memory:     number;
    requests:   number;
    errors:     number;
    instances:  number;
    status:     string;
    latency?:   { p50: number; p95: number; p99: number; mean: number };
  }>;
  totals: {
    cpu:        number;
    memory:     number;
    apps:       number;
    onlineApps: number;
  };
}
```

The snapshot powers the omnitron-console dashboard. It includes
**per-app aggregates** plus cluster-wide totals.

### Ghost-app filtering

A subtle but important property: `getSnapshot()` filters out apps
whose freshest `app_status` sample is older than
`max(3 × collection.interval, 30_000)` ms. Without this filter,
the ring buffer would keep showing apps the orchestrator no longer
reports (renamed, removed, dev-mode reload, etc.) as "offline" —
inflating `totals.apps` with ghosts.

The staleness multiplier is fixed at 3× (the smallest value that
survives normal flush jitter). Override the floor by configuring
a longer `collection.interval`.

## Storage and commit boundaries

| Backend    | Setup                                                                           | What survives restart                          |
| ---------- | ------------------------------------------------------------------------------- | ---------------------------------------------- |
| `memory`   | Default; no SQL runtime dependency                                              | Nothing; a bounded ring serves recent readings |
| `sqlite`   | A caller-owned Kysely connection; WAL + FULL/EXTRA, or rollback journal + EXTRA | Committed rows in the database file            |
| `postgres` | A caller-owned Kysely connection and a registered schema migration              | Committed rows in the logged table             |

Omnitron selects PostgreSQL on masters and SQLite on updated slave daemons. Its
`monitoring.metrics.historyMaxAge` setting supplies the retention duration (default
`7d`); standalone Titan applications use `retention.maxAge` directly.
Standalone Titan applications opt in by registering `METRICS_STORAGE_TOKEN`
with the appropriate adapter. Selecting a storage type does not create a database
connection. `forRootAsync` accepts a `storage: { useFactory, inject }` provider;
it refuses a persistent configuration without one instead of silently using memory.
Kysely is an optional peer dependency; install the driver for your chosen database.

`PostgresMetricsStorage.write(samples)` and `SQLiteMetricsStorage.write(samples)`
resolve after the transaction commits. They have no internal queue or flush timer.
Errors reject the call. All INSERT chunks and an optional `afterWrite(trx, samples)`
outbox callback share that transaction. The callback receives newly inserted samples,
so a retry with the same IDs cannot append the outbox twice. `writeInTransaction`
joins a caller-owned transaction; only the outer commit confirms persistence.
Initialize the SQLite adapter before joining another transaction.

### Apply the schema once

Titan owns the immutable `METRICS_POSTGRES_SCHEMA_V1` and `METRICS_SQLITE_SCHEMA_V1`
DDL. PostgreSQL schema changes run through the application's migration framework,
not opportunistically from a write:

```typescript
import { METRICS_POSTGRES_SCHEMA_V1 } from '@omnitron-dev/titan-metrics';
import { defineSqlMigration } from '@omnitron-dev/titan-database/migration';

export const migration = defineSqlMigration(
  '001_metrics',
  METRICS_POSTGRES_SCHEMA_V1.join(';\n'),
  'DROP TABLE metrics_samples',
);
```

Omnitron registers this as `015_persistent_metrics` through its public migration
API. Application developers using Omnitron do not need to import Kysera directly.
Existing `metrics_raw` tables are not adopted or erased: the new contract uses
`metrics_samples`, epoch milliseconds, stable sample IDs and indexed series identity.
SQLite initialization creates that table idempotently; use `initializeSchema: false`
when your migrations own its creation. Setup errors propagate and remain retryable.

### Await a durable write

```typescript
import { MetricsService, PostgresMetricsStorage } from '@omnitron-dev/titan-metrics';

// db is your existing, migrated Kysely connection.
const metrics = new MetricsService(
  { appName: 'api', collection: { enabled: false }, storage: { type: 'postgres' } },
  new PostgresMetricsStorage(db),
);
const receipt = await metrics.recordDurable([{
  id: 'measurement-42', // preserve across caller retries after an uncertain response
  name: 'queue_depth', value: 12, timestamp: Date.now(), labels: { app: 'api' },
}]);
// receipt: { durability: 'persistent', samples: 1, committedAt: ISO timestamp }
```

An ID identifies one immutable sample. Reusing it with different data rejects the
write. `recordDurable` rejects memory or undeclared durability. PostgreSQL requires
`fsync=on`, `full_page_writes=on` and a logged table; writes set transaction-local
`synchronous_commit=on`. SQLite requires WAL with `synchronous=FULL` or `EXTRA`,
or a disk rollback journal with `synchronous=EXTRA`. OFF/MEMORY journaling is
rejected. The adapter verifies the settings before initialization and inside every
write transaction; it never changes the caller's PRAGMAs. This follows
[SQLite's durability matrix](https://sqlite.org/pragma.html#pragma_synchronous).
An in-memory SQLite database is reported as volatile. Database commit does not certify a backup or availability of replicas.

### Buffered collection and diagnostics

`record`, `recordBatch` and `recordTyped` are synchronous collection calls, **not
commit acknowledgements**. The service assigns IDs before queuing. `flush()` awaits
all samples accepted before the call, including an in-flight writer; `stop()` stops
collection and awaits the final flush. A failed write keeps its batch for retry.
Until commit, this queue is volatile. Use the awaited API when the caller needs a
persistence result.

The queue defaults to 50,000 samples and 16 MiB of serialized UTF-8 data, including
writes in flight (`storage.maxBufferSize`, `storage.maxBufferBytes`). Overflow
rejects new samples from persistence and increments `droppedSamples`; it cannot be
mistaken for a committed write. Background errors are caught, exposed by
`getStorageStatus()` and sent to the optional `onError(error, operation)` callback.
Explicit writes, flush and cleanup still reject. Counters describe this process;
the database history survives its replacement.

Status includes pending sample/byte counts, capacity, oldest enqueue time, last
completed write, failed writes, known losses, write/retention/callback errors and
storage durability. Omnitron exposes the same interpretation in `omnitron metrics --json`, daemon health and the Console Metrics page. A disconnected status poll
must not leave a green persistence claim on screen.

`sync.onFlush` is a post-write notification, not a transactional outbox. Its failure
is reported separately and never retries an already committed storage write. Use
`afterWrite` with the supplied transaction for reliable enqueueing.

### Bounded history queries

Names, apps and labels filter in SQL before per-series limits. Node labels remain
separate series. `limit` is 1–2000 points per series (default 120); SQL adapters
default to the last 24 hours when `from` is omitted. Time buckets average gauges
and keep the last `*_total`/`*_count` value. Bounds cover the whole requested window,
not just its first rows. Limits apply to the newest points.

Results are capped at 100,000 points and 1,024 series by default. Adapter options
`maxQueryPoints` and `maxQuerySeries` can lower these budgets; the series cap can be
raised to 4,096. PostgreSQL queries have a 10-second statement timeout by default.
An exceeded budget rejects with an instruction to narrow the query; it does not
silently truncate history. Labels are returned once per series. Samples allow up
to 64 string labels and 8 KiB of serialized labels. Keep cardinality low.

## `@Metrics` decorator

```typescript
import { Metrics } from '@omnitron-dev/titan-metrics';

@Public()
@Metrics()                      // metric label defaults to `ClassName.process`
async process(order: Order) { /* … */ }

@Public()
@Metrics('orders.process')      // optional explicit method label
async processNamed(order: Order) { /* … */ }
```

The decorator takes a single optional `metricName` string — the value
used as the `method` label (defaults to `ClassName.methodName`). It
auto-instruments the wrapped method, recording three registry metrics
on every call:

| Registry metric                | Type      | Labels             |
| ------------------------------ | --------- | ------------------ |
| `rpc_requests_total`           | counter   | `method`, `status` |
| `rpc_request_duration_seconds` | histogram | `method`           |
| `rpc_errors_total`             | counter   | `method`, `error`  |

> **Wiring required.** The decorator reads its `MetricsService` from a
> `WeakMap` keyed by the instance — call `attachMetricsService(instance,
> service)` during DI wiring, or the decorator silently records nothing.
> It writes to the registry only (Prometheus exposition); these metrics
> are not pushed to the storage buffer.

## Prometheus exposition

```typescript
const text = await this.metrics.getPrometheusText();
// # HELP users_created_total Number of users created
// # TYPE users_created_total counter
// users_created_total{source="web"} 42
// ...
```

Serve this on a `/metrics` route from a tiny HTTP handler and point
Prometheus at it. No `prom-client` needed.

## RPC surface — `OmnitronMetrics`

Expose `MetricsRpcService` explicitly on Netron when needed. The standalone service is defined in
[`packages/titan-metrics/src/rpc-service.ts`](https://github.com/omnitron-dev/omni/tree/main/packages/titan-metrics/src/rpc-service.ts).

| Method                | Returns                | Auth             |
| --------------------- | ---------------------- | ---------------- |
| `getSnapshot()`       | `MetricsSnapshot`      | `allowAnonymous` |
| `querySeries(filter)` | `MetricsTimeSeries[]`  | `allowAnonymous` |
| `getPrometheusText()` | `string`               | `allowAnonymous` |
| `getStorageStatus()`  | `MetricsStorageStatus` | requires auth    |
| `cleanup()`           | `{ cleaned: true }`    | requires auth    |
| `flush()`             | `{ flushed: true }`    | requires auth    |
| `evictApp({ app })`   | `{ evicted: true }`    | requires auth    |

The standalone Titan RPC service retains its anonymous snapshot/history/scrape
methods for existing integrations. The Omnitron facade requires viewer, operator,
admin or control-plane roles for reads, operator/control-plane authority for flush,
and administrator authority for cleanup and eviction. The dedicated Prometheus
HTTP endpoint follows its own configured authentication policy.

```typescript
const metricsClient = await peer.queryInterface<MetricsRpcService>('OmnitronMetrics');
const snap = await metricsClient.getSnapshot();
```

## Pipeline at runtime

```mermaid
flowchart LR
  Code[Service.recordTyped]
  Code --> Reg[Prometheus registry<br/>in-memory]
  Code --> Buf[Sample buffer<br/>batchSize=200]
  Buf -- auto when full<br/>or flushInterval --> Flush{flush}
  Flush -- success --> Storage[(Storage backend)]
  Flush -- failure --> Buf
  Reg -- /metrics --> Prom[Prometheus scrape]
  Storage -- querySeries --> Dashboard[Console / dashboards]
  Storage -- retention sweep --> Cleanup[Drop > maxAge]
  Storage -- ghost-filter --> Snap[MetricsSnapshot]
```

Key invariants:

- **One bounded queue.** Failed batches retain their identities until retry;
  overflow is reported as loss.
- **One awaited flush.** Concurrent callers wait for writes they depend on;
  a busy writer is never treated as a completed flush.
- **Cleanup runs hourly.** `cleanupInterval` default 3 600 000 ms;
  applies `maxAgeMs` to the storage backend.

## Retention tuning

The default retention is 7 days with an hourly sweep. Duration values must be
positive and use `s`, `m`, `h` or `d`. SQL cleanup deletes at most 10,000 rows per
transaction, yielding between chunks and stopping a pass after its time budget;
the next sweep resumes. Errors remain visible in storage diagnostics.

Estimate capacity from measured row/index/WAL growth. At 1,000 samples/s, one day
is 86.4 million samples and seven days is 604.8 million; this is substantial storage.
Reduce collection frequency, label cardinality or retention to fit the workload.
An hourly deletion policy is not partitioning or downsampling. Replication outbox
retention is independent: pruning chart history cannot delete pending deliveries.

## Sizing & cardinality control

The single biggest operational risk is **label cardinality**.
Every unique combination of label values creates a separate
time-series in storage. At 1 metric × 10 labels × 100 values
each, that's 10^10 series — far past anything healthy.

Recipes:

- **Bucket continuous values.** Don't label by exact `amount`;
  label by `tier` derived from amount (`free|pro|enterprise`).
- **Drop request IDs.** They're unique per request — never use as
  labels.
- **Quantise time.** If you must label by a timestamp, round to
  the hour or day, not the second.
- **One metric, many labels** beats **many metrics, few labels**
  for cardinality budget — but be sane about it.

Healthy cardinality targets:

| Setup               | Total series target |
| ------------------- | ------------------- |
| Single small app    | < 10 000            |
| Multi-pod cluster   | < 100 000           |
| Aggregator backbone | < 1 000 000         |

Monitor `metrics.getRegistry()` size as a meta-metric; alert when
it grows faster than your service count.

## Tokens

| Token                   | Type                    |
| ----------------------- | ----------------------- |
| `METRICS_SERVICE_TOKEN` | `IMetricsService`       |
| `METRICS_OPTIONS_TOKEN` | `IMetricsModuleOptions` |
| `METRICS_STORAGE_TOKEN` | `IMetricsStorage`       |

## Lifecycle

`MetricsService.start()` is invoked at module init and:

- Starts the collector (process / system / RPC samples on
  `collection.interval`, default 5 s).
- Schedules flush on `storage.flushInterval` (default 5 s).
- Schedules cleanup on `retention.cleanupInterval` (default 1 h).

`stop()` does the reverse in order:

- Stops the collector.
- Clears flush + cleanup timers.
- Drains the collector buffer.
- Calls `flush()` one last time and awaits active `recordDurable` writes.

Titan DI calls `onDestroy()` when it disposes the service, awaiting `stop()` before
its storage dependencies are disposed. When constructing `MetricsService` yourself,
call `stop()` before closing your database connection. A failed flush rejects and
remains visible in diagnostics; forced process termination cannot persist an
uncommitted in-memory buffer.

Both timers `unref()` so they don't pin the event loop alive.

## Anti-patterns

- **Per-user labels.** A label per `userId` creates a new time
  series per user, blowing up cardinality. Keep labels low-
  cardinality (`tier`, `region`, `status`).
- **Counters that decrement.** Counters are monotonically
  increasing. Use a gauge for values that go up *and* down.
- **Histograms with too many buckets.** Each bucket is a separate
  time series. Five to ten buckets per histogram is usually right.
- **Forgetting `appName`.** Without it, metrics from different
  services pile together. Always set it per application.
- **Confusing collection with commit.** `record` and `recordTyped` both queue
  history. Use `recordDurable` when the caller needs database confirmation.
- **Disabling retention.** Leaving samples to accumulate
  indefinitely fills disk on Postgres / SQLite. Default `'7d'`
  is sensible.
- **Anonymous RPC `cleanup` / `flush` / `evictApp`.** These
  mutate state. The defaults require auth — don't relax this in
  production.

## See also

- [Best Practices / Observability](/docs/titan/best-practices/observability)
  — logs, metrics, traces working together
- [`titan-telemetry-relay`](/docs/titan/modules/telemetry-relay) — ship metrics
  off-host via store-and-forward
- [`titan-health`](/docs/titan/modules/health) — register a health indicator that
  surfaces `MetricsService` connectivity
- [Tokens & RPC reference](/docs/titan/modules/tokens-reference) — full token
  inventory
- [Migrations / From prom-client](/docs/titan/migrations/from-prom-client) —
  surgical migration path

---

---
id: "titan/modules/module-map"
title: "Module map"
description: "How the 16 modules relate, depend on one another, and combine."
---

# Module map

A bird's-eye view of every module in the Titan ecosystem — who
depends on whom, who can substitute for whom, and which
combinations show up in real backends.

## Dependency graph

```mermaid
flowchart TB
  subgraph Builtin["Built-in (ship with @omnitron-dev/titan)"]
    Config[config]
    Logger[logger]
  end

  subgraph Foundations["Foundation modules"]
    Redis[titan-redis]
    Events[titan-events]
    Database[titan-database]
  end

  subgraph Observability["Observability"]
    Health[titan-health]
    Metrics[titan-metrics]
    Telemetry[titan-telemetry-relay]
  end

  subgraph Coordination["Coordination & runtime"]
    Lock[titan-lock]
    Scheduler[titan-scheduler]
    PM[titan-pm]
    Discovery[titan-discovery]
  end

  subgraph Surface["Surface modules"]
    Auth[titan-auth]
    Cache[titan-cache]
    RateLimit[titan-ratelimit]
    Notifications[titan-notifications]
  end

  Lock --> Redis
  Discovery --> Redis
  Cache -. L2 tier .-> Redis
  RateLimit -. redis storage .-> Redis
  Notifications --> Redis
  Scheduler -. redis persistence .-> Redis

  Health -. db indicator .-> Database
  Health -. redis indicator .-> Redis

  Notifications -. preference store .-> Database
  Telemetry -. ships .-> Metrics

  Auth -. token cache .-> Cache

  classDef builtin    fill:#e8f5e9,stroke:#43a047,color:#1b5e20
  classDef found      fill:#e3f2fd,stroke:#1976d2,color:#0d47a1
  classDef obs        fill:#fff8e1,stroke:#fb8c00,color:#e65100
  classDef coord      fill:#f3e5f5,stroke:#8e24aa,color:#4a148c
  classDef surf       fill:#fce4ec,stroke:#d81b60,color:#880e4f

  class Config,Logger          builtin
  class Redis,Events,Database  found
  class Health,Metrics,Telemetry obs
  class Lock,Scheduler,PM,Discovery coord
  class Auth,Cache,RateLimit,Notifications surf
```

Solid arrows are **hard dependencies** — you can't run the
dependent module without the foundation. Dotted arrows are
**soft / optional** — feature flagged at config time
(`storageType: 'redis'`, `multiTier: true`, `enableDatabaseIndicator: true`).

## Install footprint

The numbers below count peer modules a typical adoption pulls in
(transitive Redis client, postgres driver, etc. are omitted):

| You install             | What else lights up                                       |
| ----------------------- | --------------------------------------------------------- |
| `titan-events`          | Standalone                                                |
| `titan-redis`           | Standalone                                                |
| `titan-database`        | Standalone (+ db driver)                                  |
| `titan-cache`           | Standalone (`+ titan-redis` if `multiTier: true`)         |
| `titan-lock`            | `+ titan-redis`                                           |
| `titan-discovery`       | `+ titan-redis`                                           |
| `titan-ratelimit`       | Standalone (`+ titan-redis` if `storageType: 'redis'`)    |
| `titan-notifications`   | `+ titan-redis` (`+ titan-database` for preference store) |
| `titan-scheduler`       | Standalone (`+ titan-redis` for distributed persistence)  |
| `titan-auth`            | Standalone (`+ titan-cache` for token cache)              |
| `titan-health`          | Standalone (indicators light up per `enableXxxIndicator`) |
| `titan-metrics`         | Standalone (`+` storage driver if persisting)             |
| `titan-telemetry-relay` | Standalone (paired with `titan-metrics` at remote end)    |
| `titan-pm`              | Standalone                                                |

The framework deliberately keeps modules independently versionable —
you only pull in what you actually use.

## Common stacks

### Web API service

<span class="module-badge">built-in · @omnitron-dev/titan · config + logger</span>
<span class="module-badge">official · titan-redis + titan-database + titan-cache + titan-auth + titan-health + titan-metrics</span>

```mermaid
flowchart LR
  C[config + logger] --> R[titan-redis]
  R --> Ca[titan-cache]
  R --> A[titan-auth]
  D[titan-database] --> H[titan-health]
  R --> H
  C --> M[titan-metrics]
```

Common starting set for any REST / RPC backend.

### Worker fleet

<span class="module-badge">built-in · @omnitron-dev/titan · config + logger</span>
<span class="module-badge">official · titan-redis + titan-database + titan-lock + titan-scheduler + titan-metrics + titan-health</span>

```mermaid
flowchart LR
  C[config + logger] --> R[titan-redis]
  R --> L[titan-lock]
  R --> S[titan-scheduler]
  D[titan-database] --> H[titan-health]
  C --> M[titan-metrics]
```

`titan-lock` is what stops two workers running the same job.

### Notification gateway

<span class="module-badge">built-in · @omnitron-dev/titan · config + logger</span>
<span class="module-badge">official · titan-redis + titan-notifications + titan-ratelimit + titan-metrics</span>

```mermaid
flowchart LR
  C[config + logger] --> R[titan-redis]
  R --> N[titan-notifications]
  R --> RL[titan-ratelimit]
  C --> M[titan-metrics]
```

`titan-ratelimit` shields the upstream provider; `titan-metrics`
records throughput / failures by channel.

### Multi-instance app

Add **`titan-discovery`** to any of the above when you have more
than one pod and want to address services without hard-coded URLs.

### Edge / offline node

<span class="module-badge">official · titan-telemetry-relay · stable</span>

Add `titan-telemetry-relay` when the node may be disconnected from
the central metrics endpoint — it buffers samples to SQLite and
flushes when connectivity returns.

## Similar modules — when which?

| Need                              | Reach for                                                      | Why                                               |
| --------------------------------- | -------------------------------------------------------------- | ------------------------------------------------- |
| Sync in-process pub/sub           | [`titan-events`](/docs/titan/modules/events)                   | Fire-and-forget, intra-process, typed channels    |
| Reliable cross-pod queue          | [`titan-notifications`](/docs/titan/modules/notifications)     | Built on rotif; survives crashes, has DLQ + retry |
| Periodic work (cron / interval)   | [`titan-scheduler`](/docs/titan/modules/scheduler)             | Per-job persistence + missed-fire handling        |
| Memoize a function call           | [`titan-cache`](/docs/titan/modules/cache)                     | `@Cacheable(...)` decorator; multi-tier (L1+L2)   |
| Talk to Redis directly            | [`titan-redis`](/docs/titan/modules/redis)                     | Just a typed client; no service abstraction       |
| Throttle calls                    | [`titan-ratelimit`](/docs/titan/modules/ratelimit)             | Token-bucket / sliding-window / fixed-window      |
| Coordinate across instances       | [`titan-lock`](/docs/titan/modules/lock)                       | Redis-backed mutex with UUID ownership            |
| Spawn / supervise child processes | [`titan-pm`](/docs/titan/modules/pm)                           | Worker pools, IPC, crash supervision              |
| Find sibling services             | [`titan-discovery`](/docs/titan/modules/discovery)             | Heartbeat-based service registry                  |
| Count / measure                   | [`titan-metrics`](/docs/titan/modules/metrics)                 | Counters, gauges, histograms; Prom exposition     |
| Ship metrics off-host             | [`titan-telemetry-relay`](/docs/titan/modules/telemetry-relay) | Store-and-forward; survives offline windows       |
| Check it's alive                  | [`titan-health`](/docs/titan/modules/health)                   | k8s probes + custom indicators                    |
| Authenticate RPC callers          | [`titan-auth`](/docs/titan/modules/auth)                       | JWT (HS256/RS256/ES256), JWKS, token cache        |
| Query a database                  | [`titan-database`](/docs/titan/modules/database)               | Kysely + migrations + RLS                         |
| Validated config                  | [`config`](/docs/titan/modules/config) (built-in)              | Layered, validated, hot-reloadable                |
| Structured logging                | [`logger`](/docs/titan/modules/logger) (built-in)              | pino under the hood; transports + processors      |

## Replacement / overlap matrix

| Module                  | Closest overlap                    | When to prefer                                    |
| ----------------------- | ---------------------------------- | ------------------------------------------------- |
| `titan-events`          | `titan-notifications`              | In-process / no persistence / lowest latency      |
| `titan-notifications`   | `titan-events` + `titan-scheduler` | Cross-process, retry, DLQ, channel templating     |
| `titan-cache`           | bare `titan-redis`                 | Caching semantics (`@Cacheable`, TTL, multi-tier) |
| `titan-scheduler`       | `setInterval` / `node-cron`        | Persistence, missed-fire, observability           |
| `titan-lock`            | bare `SETNX` on Redis              | Safe ownership (UUID) + Lua-script atomicity      |
| `titan-discovery`       | env-var URL config                 | Dynamic topology, autoscaling, rolling deploys    |
| `titan-metrics`         | `prom-client`                      | Native persistence + cross-pod aggregation        |
| `titan-telemetry-relay` | direct Prom scrape                 | Edge / offline / store-and-forward                |
| `titan-health`          | rolling-your-own `/healthz`        | Standard k8s probe shapes + indicator registry    |
| `titan-auth`            | manual JWT verification            | JWKS rotation, token cache, RBAC primitives       |

## Where Redis sits

Five official modules use Redis as their primary backend. They
share the same `Redis` client provided by `titan-redis`:

```mermaid
flowchart LR
  R[(Redis)]
  R --- L[titan-lock]
  R --- D[titan-discovery]
  R --- Ca[titan-cache L2]
  R --- RL[titan-ratelimit]
  R --- N[titan-notifications]
```

Register `TitanRedisModule.forRoot(...)` once at app boot; all
five pick up the connection automatically.

```typescript
@Module({
  imports: [
    TitanRedisModule.forRoot({ config: { url: env.REDIS_URL } }),
    TitanLockModule.forRoot(),
    DiscoveryModule.forRoot(),
    TitanCacheModule.forRoot({ multiTier: true, l2: { /* picked up */ } }),
    TitanRateLimitModule.forRoot({ storageType: 'redis' }),
    NotificationsModule.forRoot({ /* … */ }),
  ],
})
class AppModule {}
```

**Recommended Redis-DB split** (avoid keyspace collisions):

| DB | Purpose                                    |
| -- | ------------------------------------------ |
| 0  | App data (cache, sessions, business state) |
| 1  | Queues / pub-sub (notifications, events)   |
| 2  | Rate limits                                |
| 3  | Locks (`titan-lock`)                       |
| 4  | Discovery registry                         |

Configure per-module by passing a different `db:` index in the
options, or by registering multiple named Redis clients via
`TitanRedisModule.forFeature(['cache', 'queue', …])` (or the
`clients: [...]` option on `forRoot`).

## Lifecycle order — first to last

The container resolves lifecycle in dependency order:

```mermaid
flowchart TB
  Start([App start])
  Start --> Config[ConfigModule]
  Config --> Logger[LoggerModule]
  Logger --> Redis[TitanRedisModule]
  Logger --> Database[TitanDatabaseModule]
  Redis --> Lock[TitanLockModule]
  Redis --> Discovery[DiscoveryModule]
  Redis --> Cache[TitanCacheModule]
  Redis --> RateLimit[TitanRateLimitModule]
  Redis --> Notif[NotificationsModule]
  Database --> Health[TitanHealthModule]
  Redis --> Health
  Database --> Scheduler[TitanSchedulerModule]
  Cache --> Auth[TitanAuthModule]
  Logger --> Metrics[TitanMetricsModule]
  Metrics --> Telemetry[TelemetryRelay]
  Metrics --> User[Your services]
  Auth --> User
  Health --> Ready([App ready])
  User --> Ready
```

On shutdown, the reverse order is followed — user services drain
first, then surface modules, then foundations, then logger / config
last. The exact reverse ensures dependants always stop before
their dependencies.

## Picking the right tier

| Question                                         | Tier                                                               |
| ------------------------------------------------ | ------------------------------------------------------------------ |
| Always available, smallest install?              | **Built-in**                                                       |
| Battle-tested, official, individually versioned? | **Official**                                                       |
| Niche need, no official equivalent yet?          | **Community** (audit first)                                        |
| Want to publish your own?                        | [Authoring a module](/docs/titan/modules-system/authoring-modules) |

## See also

- [Modules index](/docs/titan/modules) — flat list by tier
- [Authoring a module](/docs/titan/modules-system/authoring-modules) —
  conventions every official module follows
- [Recipes](/docs/titan/recipes) — concrete combinations that solve
  real problems

---

---
id: "titan/modules/notifications"
title: "titan-notifications"
---

# titan-notifications

<span class="module-badge">official · @omnitron-dev/titan-notifications · stable</span>

Multi-channel notification delivery (in-app, webhook, plus extensible
email / SMS / push abstractions) with a template engine, Redis-backed
rate limiting, per-user preferences, channel routing, scheduled and
broadcast send, and a separate **worker mode** for consuming
notification events from Rotif.

```bash
pnpm add @omnitron-dev/titan-notifications @omnitron-dev/titan-events @omnitron-dev/titan-redis
```

For SMTP email, use [titan-mailer](/docs/titan/modules/mailer): a concrete Nodemailer channel,
strict TLS, typed templates, bounded pools and a durable outbox worker over this
module's Redis transport. Business-state changes and queued email are committed
together; only opaque delivery IDs travel through Redis. SMTP acceptance is not
an exactly-once or inbox-delivery guarantee.

## When you need it

- **Multi-channel delivery.** Same payload, different channels per
  user (email + push for some, in-app only for others).
- **Templated messages.** Render once, send via any channel.
- **Preference-respecting workflow.** Users opt out of certain
  channels; rate limits stop you spamming.
- **Producer/worker split.** API pod publishes, worker pod consumes
  and delivers — `forRoot` and `forWorker` cover both ends.

## Quickstart — producer (the app that emits notifications)

```typescript
import { NotificationsModule } from '@omnitron-dev/titan-notifications';

@Module({
  imports: [
    TitanRedisModule.forRoot({ config: { url: env.REDIS_URL } }),
    NotificationsModule.forRoot({
      redis:           env.REDIS_URL,
      enableInApp:     true,
      enableWebhook:   true,
      defaultChannels: ['inapp', 'email'],
      templates:       { enabled: true, cacheEnabled: true },
      rateLimiterConfig: {
        defaultLimits: { perMinute: 10, perHour: 100, perDay: 1_000 },
        channelLimits: { sms: { perMinute: 1, perHour: 10 } },
      },
    }),
  ],
})
class AppModule {}
```

## Quickstart — worker (the pod that consumes and delivers)

```typescript
import { NotificationsModule } from '@omnitron-dev/titan-notifications';

@Module({
  imports: [
    NotificationsModule.forWorker({
      targetResolver:   NOTIFICATION_TARGET_RESOLVER,
      persister:        NOTIFICATION_PERSISTER,
      realtimeSignaler: NOTIFICATION_REALTIME_SIGNALER,
      // There is no `concurrency` — the worker runs one XREADGROUP loop and
      // takes a batch per read. `readCount` sizes the batch.
      workerOptions:    { readCount: 100, blockTimeoutMs: 5_000 },
    }),
  ],
  providers: [
    { provide: NOTIFICATION_TARGET_RESOLVER, useClass: MyUserResolver },
    { provide: NOTIFICATION_PERSISTER,        useClass: MyPersister },
    { provide: NOTIFICATION_REALTIME_SIGNALER,useClass: MyRealtimeSignaler },
  ],
})
class WorkerModule {}
```

The worker consumes events emitted by the producer (typically via
Rotif), looks up channel targets, persists the notification, and
signals realtime channels (WebSocket / SSE).

## `NotificationsModuleOptions`

| Option                  | Type                                                                     |
| ----------------------- | ------------------------------------------------------------------------ |
| `transport`             | `{ useTransport?: MessagingTransport, rotif?: RotifTransportOptions }`   |
| `redis`                 | `RedisOptions \| string`                                                 |
| `rateLimiter`           | `IRateLimiter`                                                           |
| `preferenceStore`       | `IPreferenceStore`                                                       |
| `channelRouter`         | `IChannelRouter`                                                         |
| `defaultChannels`       | `string[]`                                                               |
| `channels`              | `NotificationChannel[]`                                                  |
| `enableInApp`           | `boolean` (default `true`; needs Redis)                                  |
| `enableWebhook`         | `boolean` (default `true`)                                               |
| `inAppConfig`           | `{ keyPrefix?, defaultTTL?, maxNotificationsPerUser?, enableRealtime? }` |
| `webhookConfig`         | `{ timeout?, retries?, signatureSecret?, signatureHeader? }`             |
| `templates`             | `{ enabled?, cacheEnabled?, cacheTTL? }`                                 |
| `rateLimiterConfig`     | `{ keyPrefix?, defaultLimits, channelLimits?, enableBurstDetection? }`   |
| `preferenceStoreConfig` | `{ keyPrefix?, defaultPreferences? }`                                    |
| `isGlobal`              | `boolean`                                                                |

Also: `forRootAsync({ useFactory, inject?, useExisting?, useClass?, imports?, isGlobal? })`.

## `NotificationsService` — the producer API

```typescript
import { NotificationsService, NOTIFICATIONS_SERVICE }
  from '@omnitron-dev/titan-notifications';

@Service({ name: 'notify' })
class NotifyService {
  constructor(@Inject(NOTIFICATIONS_SERVICE) private readonly notify: NotificationsService) {}

  @Public()
  async welcome(user: User) {
    await this.notify.send(
      { id: user.id, email: user.email },
      { type: 'info', title: 'Welcome', message: `Hello, ${user.name}` },
      { channels: ['email', 'inapp'] },
    );
  }
}
```

| Method                                       | Returns                    |
| -------------------------------------------- | -------------------------- |
| `send(recipient, payload, options?)`         | `Promise<SendResult>`      |
| `broadcast(recipients[], payload, options?)` | `Promise<BroadcastResult>` |
| `schedule(recipient, payload, scheduledAt)`  | `Promise<ScheduleResult>`  |

`NotificationPayload` requires `type`, `title` and `message`. There is
no `template` field on it — see below.

The service:

1. Checks rate limits per channel per recipient.
2. Checks preferences (does the user opt in to this channel?).
3. Routes to the right channel via `IChannelRouter` — only if one is
   registered; the injection is `@Optional()`, and without it the
   `channels` you pass in the options are used as given.
4. Hands off to the registered channel implementations.
5. Records the send against the rate limiter.

### Templates are opt-in and caller-driven

`send()` does not render templates. `TemplateEngine` is a separate
service exported under `NOTIFICATIONS_TEMPLATE_ENGINE`; nothing inside
the package calls it. You apply it yourself and pass the result on:

```typescript
import {
  NOTIFICATIONS_TEMPLATE_ENGINE, NOTIFICATIONS_SERVICE,
  TemplateEngine, NotificationsService,
} from '@omnitron-dev/titan-notifications';

@Service({ name: 'notify' })
class NotifyService {
  constructor(
    @Inject(NOTIFICATIONS_SERVICE) private readonly notify: NotificationsService,
    @Inject(NOTIFICATIONS_TEMPLATE_ENGINE) private readonly templates: TemplateEngine,
  ) {}

  @Public()
  async welcome(user: User) {
    const payload = this.templates.applyToPayload(
      'welcome',
      { type: 'info', title: '', message: '' },
      { name: user.name },
    );
    await this.notify.send({ id: user.id, email: user.email }, payload);
  }
}
```

`applyToPayload` overlays the rendered `title`/`message` onto the
payload and stashes the full render under `data._rendered`. The
`templates` block in the module options configures that engine
(caching, TTL) — it does not switch on rendering inside `send()`.

### `NotificationRecipient.id` is the only required field, and it is load-bearing

```typescript
interface NotificationRecipient {
  id: string;              // required
  email?: string; phone?: string; pushTokens?: string[];
  webhookUrl?: string; locale?: string;
}
```

Every step above keys off `id`: the rate-limit bucket
(`checkLimit(recipient.id, …)`), the preference lookup, the realtime
channel the delivery is published to
(`notifications.{type}.{recipient.id}`), and the in-app channel's own
`canDeliver` check, which is literally `!!recipient.id`.

That makes a misspelt key — `userId`, say — quiet rather than loud: the
send resolves, in-app delivery is skipped for failing `canDeliver`, and
everything else is published to `notifications.welcome.undefined`,
where no subscriber is listening. Nothing throws. TypeScript is the
only thing that catches it, so do not widen this argument to `any`.

## Channels

Out-of-the-box implementations + extensible bases:

| Channel class                                             | Purpose                                        |
| --------------------------------------------------------- | ---------------------------------------------- |
| `InAppChannel`                                            | In-app inbox; Redis-backed list per user       |
| `WebhookChannel`                                          | HMAC-signed HTTP POST to a URL                 |
| `AbstractEmailChannel`                                    | Base — extend for SMTP / Sendgrid / SES / etc. |
| `AbstractSMSChannel`                                      | Base — extend for Twilio / Vonage / etc.       |
| `AbstractPushChannel`                                     | Base — extend for FCM / APNS / etc.            |
| `MockEmailChannel` / `MockSMSChannel` / `MockPushChannel` | Test doubles                                   |

A concrete channel typically implements just `deliver(recipient,
rendered, options)` plus optional `validate()`.

## Templates

```typescript
NotificationsModule.forRoot({
  templates: { enabled: true, cacheEnabled: true, cacheTTL: 60_000 },
})

// Use:
await this.notify.send(
  { userId, email },
  { template: 'welcome', data: { name: 'Ada' } },
);
```

`TemplateEngine` supports variable substitution via `{{name}}`
placeholders. Templates can be passed at boot or registered at
runtime. `DEFAULT_TEMPLATES` is exported as a starting set.

## Rate limiting and preferences

`RedisRateLimiter` enforces per-minute / per-hour / per-day limits
globally or per-channel; `RedisPreferenceStore` tracks per-user
opt-in/out per channel. Wired automatically when `redis:` is set.

```typescript
rateLimiterConfig: {
  defaultLimits:        { perMinute: 10, perHour: 100, perDay: 1_000, burstLimit: 5 },
  channelLimits:        { sms: { perMinute: 1, perHour: 10 } },
  enableBurstDetection: true,
}

preferenceStoreConfig: {
  defaultPreferences: { channels: { email: true, sms: false, push: true } },
}
```

## Worker model

```mermaid
flowchart LR
  Api[API pod<br/>NotificationsService.send]
  Api -- emits notification event --> Rotif[(Rotif / queue)]
  Worker[Worker pod<br/>NotificationsModule.forWorker]
  Rotif --> Worker
  Worker --> Resolver[TargetResolver]
  Worker --> Persister[Persister<br/>store the notification]
  Worker --> Channel[Channel.deliver]
  Worker --> Realtime[RealtimeSignaler<br/>push to WS / SSE]
```

Producer/worker split lets you scale delivery independently from the
request path and isolate flaky external providers (email, SMS) from
the API tier.

## `@OnNotification` decorator

```typescript
import { OnNotification } from '@omnitron-dev/titan-notifications';

@Injectable()
class AuditListener {
  @OnNotification('notifications.notification.sent')
  async onSent(notification: any) { /* … */ }

  @OnNotification('notifications.notification.failed', { maxRetries: 5, exactlyOnce: true })
  async onFailed(notification: any) { /* … */ }
}
```

`OnNotification(pattern, options?)` takes the pattern as the **first
positional argument**. `options` extends `TransportSubscribeOptions`
(`groupName`, `consumerName`, `maxRetries`, `retryDelay`,
`retryStrategy`, `autoAck`, `prefetchCount`, …) plus an `exactlyOnce`
flag. Patterns are the transport topic names — the lifecycle topics are
exported as `NOTIFICATIONS_EVENTS` (e.g.
`NOTIFICATIONS_EVENTS.NOTIFICATION_SENT === 'notifications.notification.sent'`).

Subscribe to delivery lifecycle events for observability, audit,
metrics.

## Health indicator

`NotificationsHealthIndicator` is exported and registers
automatically with [`titan-health`](/docs/titan/modules/health) if both modules
are loaded.

Observer failures also appear in the health report. Async lifecycle listeners
are awaited; failures are logged and retained until the same event succeeds.
An observer failure does not resend a notification that already reached its
transport. Without an Events module, lifecycle emission is disabled and reported
as such in the health details.

## Operational diagnostics

`@omnitron-dev/titan-notifications/diagnostics` provides
`DiagnosticNotificationSink` and `DiagnosticNotificationsModule`. They use the
configured transport's queue, deduplication, retry and DLQ mechanisms. A missing
consumer rejects delivery; broker acceptance is not recipient acknowledgement.
See the [contract, module wiring and outbox bridge](/docs/omnitron/diagnostics).

## Tokens (selection)

| Token                            |
| -------------------------------- |
| `NOTIFICATIONS_SERVICE`          |
| `NOTIFICATIONS_TRANSPORT`        |
| `NOTIFICATIONS_MODULE_OPTIONS`   |
| `NOTIFICATIONS_HEALTH`           |
| `NOTIFICATIONS_RATE_LIMITER`     |
| `NOTIFICATIONS_PREFERENCE_STORE` |
| `NOTIFICATIONS_CHANNEL_ROUTER`   |
| `NOTIFICATIONS_EVENT_EMITTER`    |
| `NOTIFICATIONS_CHANNEL_REGISTRY` |
| `NOTIFICATIONS_TEMPLATE_ENGINE`  |
| `NOTIFICATION_TARGET_RESOLVER`   |
| `NOTIFICATION_PERSISTER`         |
| `NOTIFICATION_REALTIME_SIGNALER` |

## Lifecycle

The service is registered as a Titan provider; standard lifecycle
hooks apply. The worker mode (`forWorker`) starts a Rotif consumer
during the application `onStart` phase and stops it on `onStop`.

## Anti-patterns

- **Sending from the request path.** Heavy channels (SMS / email)
  can take seconds. Send via the worker pattern, not directly in
  the request handler.
- **Skipping preferences.** Sending to users who opted out is a
  compliance risk in many jurisdictions. Always wire
  `preferenceStore`.
- **Per-user templates.** Templates are content; per-user data goes
  in the `data` payload. Pre-render with the engine, not in code.
- **No rate limits.** Without limits, a runaway loop can spam every
  user in your database in minutes.

## See also

- [`titan-events`](/docs/titan/modules/events) — in-process events; this module
  bridges them to multi-channel delivery
- [`titan-redis`](/docs/titan/modules/redis) — backs rate limiter, preference
  store, and Rotif
- [`titan-ratelimit`](/docs/titan/modules/ratelimit) — pluggable rate limiter
  for the producer side

---

---
id: "titan/modules/observability-matrix"
title: "Observability matrix"
description: "What every module logs, raises as events, and writes as metrics."
---

# Observability matrix

Operating a Titan app means knowing *what to grep for* and *what
to alert on*. This page lists — per module — the events emitted,
notable log keys, and metric/health signals you can rely on.

All claims here are verified against module source. If a row is
empty, the module is deliberately quiet at that layer.

## At-a-glance

<span class="module-badge">official · @omnitron-dev/titan-*</span>

The **Metrics surface** column distinguishes metrics actively *pushed*
into `titan-metrics` from stats you read on demand (*pull* — a
`getStats()` / `getMetrics()` / `stats()` accessor).

| Module                  |              Emits events              |        Notable logs        |             Metrics surface            |    Health indicator    |
| ----------------------- | :------------------------------------: | :------------------------: | :------------------------------------: | :--------------------: |
| `titan-auth`            |                    —                   |     JWT verify failures    |                    —                   |            —           |
| `titan-cache`           |                    —                   |    L2 fallback / errors    |  hit/miss/evict *(pull: `getStats()`)* |            —           |
| `titan-database`        |                    —                   |     slow query, retries    |                    —                   | yes (via titan-health) |
| `titan-discovery`       |            `discovery:event`           | register, heartbeat, dereg |                    —                   |            —           |
| `titan-events`          |                  proxy                 |       dispatch errors      |                    —                   |            —           |
| `titan-health`          |                    —                   |     indicator failures     |                    —                   |      n/a (itself)      |
| `titan-lock`            |                    —                   |   failure-tracker windows  |                    —                   |            —           |
| `titan-metrics`         |                    —                   |       quiet by design      |        process/system collectors       |            —           |
| `titan-notifications`   | `notifications.*` *(via titan-events)* |         send + DLQ         |                    —                   |           yes          |
| `titan-pm`              |             many *(below)*             |   spawn / crash / restart  |              process stats             | yes (via titan-health) |
| `titan-ratelimit`       |                    —                   |       reject reasons       |    allow/deny *(pull: `getStats()`)*   |            —           |
| `titan-redis`           |                    —                   |     reconnect / errors     |                    —                   | yes (via titan-health) |
| `titan-scheduler`       |                    —                   |     job start / finish     | runs/failures *(pull: `getMetrics()`)* |            —           |
| `titan-telemetry-relay` |                    —                   |         WAL events         |             pull: `stats()`            |            —           |

<span class="module-badge">built-in · @omnitron-dev/titan</span>

| Module   |  Change notifications |     Notable logs    | Pushes metrics |
| -------- | :-------------------: | :-----------------: | :------------: |
| `config` | `onChange()` callback | reload success/fail |        —       |
| `logger` |           —           | (the logger itself) |        —       |

## Per-module reference

### `titan-pm` — the most chatty module

<span class="module-badge">official · @omnitron-dev/titan-pm</span>

Process manager is the loudest module by design — every supervised
process / pool / worker raises events you can subscribe to.

| Event                     | Args                       | Meaning                                            |
| ------------------------- | -------------------------- | -------------------------------------------------- |
| `process:spawn`           | `processInfo`              | Child process started                              |
| `process:ready`           | `processInfo`              | Child responded to ready handshake                 |
| `process:stop`            | `processInfo`              | Child exited cleanly                               |
| `process:crash`           | `processInfo, error`       | Child died unexpectedly                            |
| `child:started`           | `name`                     | Supervised child entry started                     |
| `child:stopped`           | `name`                     | Supervised child stopped                           |
| `child:start-failed`      | `name, error`              | Initial start raised before ready                  |
| `child:crash`             | `name, error`              | Supervised child crashed mid-run                   |
| `child:restart`           | `name, count`              | Child restarted; `count` = total restarts          |
| `pool:initialized`        | `{ size, class }`          | Worker pool warmed up                              |
| `pool:scaled`             | `{ from, to, class }`      | Pool grew / shrank                                 |
| `pool:drained`            | `{ class }`                | Pool finished draining queued work                 |
| `pool:destroyed`          | `{ class }`                | Pool released                                      |
| `pool:memory`             | `{ workerId, rssMB }`      | Memory limit exceeded                              |
| `worker:spawned`          | `{ workerId, class }`      | Pool worker forked                                 |
| `worker:shutdown`         | `{ workerId, class }`      | Pool worker reaped                                 |
| `worker:unhealthy`        | `{ workerId }`             | Worker missed a ping                               |
| `worker:unresponsive`     | `{ workerId }`             | Worker still silent after retries                  |
| `worker:replaced`         | `{ workerId }`             | Unresponsive worker replaced                       |
| `request:queued`          | `{ poolClass, queueSize }` | Pool saturated; work queued                        |
| `circuitbreaker:open`     | —                          | Breaker tripped                                    |
| `circuitbreaker:halfopen` | —                          | Probe window started                               |
| `circuitbreaker:close`    | —                          | Breaker reset                                      |
| `escalate`                | `name, error`              | Restart limit exceeded; failure escalated upstream |
| `shutdown`                | —                          | Supervisor shutdown                                |
| `exit`                    | `info`                     | Process exited                                     |
| `health:change`           | `processId, IHealthStatus` | Per-process health flipped                         |
| `health:critical`         | `processId, IHealthStatus` | Per-process health became critical                 |

Subscribe with the standard event emitter API:

```typescript
pmService.on('process:crash', (info, error) => {
  logger.error({ pid: info.id, error }, 'child crashed');
});
```

### `titan-notifications`

<span class="module-badge">official · @omnitron-dev/titan-notifications</span>

Lifecycle events are emitted through `titan-events` under the
`NOTIFICATIONS_EVENTS` topic names (only when an `EventsService` is
available). Selected topics:

| Topic constant             | Event name                               | Meaning                     |
| -------------------------- | ---------------------------------------- | --------------------------- |
| `NOTIFICATION_SENT`        | `notifications.notification.sent`        | Send completed              |
| `NOTIFICATION_FAILED`      | `notifications.notification.failed`      | Send failed                 |
| `CHANNEL_DELIVERY_SUCCESS` | `notifications.channel.delivery.success` | Per-channel delivery ok     |
| `CHANNEL_DELIVERY_FAILED`  | `notifications.channel.delivery.failed`  | Per-channel delivery failed |
| `DLQ_MESSAGE_ADDED`        | `notifications.dlq.message.added`        | Message routed to DLQ       |

Dead-lettered messages are also inspectable on the service via
`getDLQStats()` / `getDLQMessages(options?)` and requeued with
`requeueFromDLQ(count?)`.

Health: the module registers `NotificationsHealthIndicator`
(`NOTIFICATIONS_HEALTH`) — automatically picked up by
`titan-health` if both modules are loaded.

### `titan-discovery`

<span class="module-badge">official · @omnitron-dev/titan-discovery</span>

| Event             | Payload                        | Meaning                        |
| ----------------- | ------------------------------ | ------------------------------ |
| `discovery:event` | `{ type, node, service?, ts }` | Node added / removed / updated |

`type` is one of `node.added`, `node.removed`, `node.updated`.

Notable logs (all under the service's own logger namespace):

| Level   | Pattern                                                 |
| ------- | ------------------------------------------------------- |
| `info`  | `Node 'X' registered`, `DiscoveryService started`       |
| `info`  | `Initiating graceful shutdown for node 'X'`             |
| `warn`  | `Heartbeat attempt failed`                              |
| `error` | `All N heartbeat attempts failed`                       |
| `error` | `Failed to publish discovery event`                     |
| `debug` | `Received discovery event`, `Cleaned up inactive nodes` |

### `titan-events`

<span class="module-badge">official · @omnitron-dev/titan-events</span>

`titan-events` *is* the event bus; it doesn't emit framework-level
events of its own. Per-emitter introspection is available via
`EventHistoryService` if `enableHistory: true`.

### `titan-scheduler`

<span class="module-badge">official · @omnitron-dev/titan-scheduler</span>

Doesn't expose a hot event stream — `IJobListener` is the
extension point. Implement one to observe job execution:

```typescript
class MyJobListener implements IJobListener {
  onJobStart?(job, context)         { /* ... */ }
  onJobComplete?(job, result)       { /* ... */ }
  onJobError?(job, error, context)  { /* ... */ }
  onJobRetry?(job, attempt, error)  { /* ... */ }
  onJobCancelled?(job, reason?)     { /* ... */ }
}
```

All hooks are optional. Register listeners via the
`listeners` module option or the `SCHEDULER_LISTENERS_TOKEN`.

### `titan-cache`

<span class="module-badge">official · @omnitron-dev/titan-cache</span>

Cache hit/miss/eviction metrics are exposed through the
`@Cached` decorator metadata and the `CacheService.getStats()` API.

```typescript
const stats = cache.getStats();
// { hits, misses, evictions, size, hitRate, ... }
```

No native event stream — wrap calls if you need per-key
observability.

### `titan-lock`

<span class="module-badge">official · @omnitron-dev/titan-lock</span>

Uses the framework's `FailureTracker` primitive: instead of
logging every failure, it collapses repeated failures of the same
operation into windowed "X failing" warnings. This keeps logs
quiet when Redis briefly hiccups.

| Level   | Pattern                               |
| ------- | ------------------------------------- |
| `warn`  | `[DistributedLock] X started failing` |
| `debug` | `[DistributedLock] X still failing`   |
| `info`  | `[DistributedLock] X recovered`       |

### `titan-database`

<span class="module-badge">official · @omnitron-dev/titan-database</span>

| Level   | Pattern                                    |
| ------- | ------------------------------------------ |
| `warn`  | Slow query (over the configured threshold) |
| `warn`  | Transient error retried (via `withRetry`)  |
| `error` | Pool exhaustion / connection lost          |
| `info`  | Migration applied                          |

The module exposes `DatabaseHealthIndicator` for k8s probes.

### `titan-redis`

<span class="module-badge">official · @omnitron-dev/titan-redis</span>

Inherits ioredis's event model (`connect`, `ready`, `error`,
`close`, `reconnecting`, `end`). The manager logs reconnect
attempts and surfaces a `RedisHealthIndicator` via
`titan-health`.

### `titan-auth`

<span class="module-badge">official · @omnitron-dev/titan-auth</span>

Quiet by design — token-verification details should not be
logged at info level (PII/tokens). Failures log at `warn` with
`error.code` populated; successful verifications are silent.

### `titan-ratelimit`

<span class="module-badge">official · @omnitron-dev/titan-ratelimit</span>

Statistics are pull-based:

```typescript
const stats = rate.getStats();
// { totalChecks, totalAllowed, totalDenied, activeKeys, byTier? }
```

Denied requests are not logged automatically (volume would be
high) — count them via `metrics.recordTyped('counter', ...)` from
the caller if you need to.

### `titan-metrics`

<span class="module-badge">official · @omnitron-dev/titan-metrics</span>

`MetricsService.getStorageStatus()` reports durability, queue samples/bytes,
completed writes, retry failures, known losses and retention/callback errors.
Background errors also reach `onError`; explicit write/flush failures reject.
Omnitron records rate-limited structured failures and exposes the same reading
through daemon health, CLI and Console. Normal successful flushes stay quiet.

The process collector (`collection: { process: true }`) records, for
the daemon's own process: `heap_used_bytes`, `heap_total_bytes`,
`rss_bytes`, `external_bytes`, `cpu_percent`, `uptime_seconds`. The
child/orchestrator collector (`system: true`) additionally records
`memory_bytes`, `rpc_requests_total`, `rpc_errors_total`, and
`app_status` per supervised app. Event-loop lag is not collected by
the built-in collector.

### `titan-telemetry-relay`

<span class="module-badge">official · @omnitron-dev/titan-telemetry-relay</span>

Exposes a pull-based `stats()` snapshot —
`{ totalEmitted, totalSent, totalFailed, totalReceived,
transportConnected, buffer, wal }` — where `buffer`
(`{ size, totalPushed, totalDropped, totalFlushed }`) and `wal`
(`{ segments, totalSize, totalWritten, currentSegment }`) carry the
queue-depth and throughput signals. Surface these through your metrics
module at the leader node; the relay does not push them itself.

### `titan-health`

<span class="module-badge">official · @omnitron-dev/titan-health</span>

| Level   | Pattern                       |
| ------- | ----------------------------- |
| `error` | Indicator threw during check  |
| `warn`  | Indicator returned `degraded` |
| `debug` | Probe cache hit / refresh     |

Health results carry per-check timing in their own result objects;
the module does not push check latency into `titan-metrics`.

## Built-in modules

### `config`

<span class="module-badge">built-in · @omnitron-dev/titan/module/config</span>

Change notifications are delivered through a callback subscription,
not a named event-bus topic:

```typescript
const unsubscribe = config.onChange((event: IConfigChangeEvent) => {
  // event: { path, oldValue, newValue, source, timestamp }
});
```

Notable logs:

- `info` — `Configuration reloaded due to file change` (after a watched source changes)
- `error` — `Failed to reload configuration`

### `logger`

<span class="module-badge">built-in · @omnitron-dev/titan/module/logger</span>

The logger doesn't log about itself except at fatal failures
(e.g., transport open failed at boot). Use `createNullLogger()` in
tests when you want absolute silence.

## Cross-cutting recommendations

### Subscribe instead of poll

The pm/discovery event streams are hot — subscribing is cheaper
and more responsive than polling state. For per-process autoscaling
or alerting, hook the events:

```typescript
pm.on('pool:memory',      handleMemorySpike);
pm.on('worker:unresponsive', handleWorkerHang);
pm.on('circuitbreaker:open', notifyOncall);
```

### Tie metrics to events

For dashboards, route key events through `titan-metrics`:

```typescript
pm.on('process:crash', (info, error) => {
  metrics.recordTyped('counter', 'pm.process.crash.total',
    { class: info.processName }, 1);
});
```

### Alert thresholds — common starting points

| Signal                        | Suggested alert              |
| ----------------------------- | ---------------------------- |
| `rss_bytes` growth rate       | > +50 MB/min sustained       |
| `cpu_percent`                 | sustained near 100 for 2 min |
| `pool:scaled` to/from limit   | repeated within 5 min        |
| `worker:replaced` count       | > 3 in 5 min for same pool   |
| `circuitbreaker:open`         | any open lasting > 30 s      |
| `app_status` flips to 0       | any supervised app offline   |
| Health `degraded`/`unhealthy` | sustained > 1 probe window   |

### Don't log per-call by default

Several modules (`titan-ratelimit`, `titan-cache`, `titan-auth`)
intentionally stay quiet at info level. Per-call logging at request
rate is a recipe for disk/SIEM saturation. Use counters and
sampled debug logging instead.

## See also

- [Best Practices / Observability](/docs/titan/best-practices/observability) — logs, metrics, traces together
- [`titan-metrics`](/docs/titan/modules/metrics) — recording API
- [`titan-health`](/docs/titan/modules/health) — indicator registry
- [`titan-telemetry-relay`](/docs/titan/modules/telemetry-relay) — store-and-forward shipping
- [Lifecycle reference](/docs/titan/modules/lifecycle-reference) — which hook each module uses

---

---
id: "titan/modules/options-patterns"
title: "Options patterns"
description: "forRoot / forRootAsync / forFeature / forWorker across every module."
---

# Options patterns

Every Titan module is configured through one of four static factory
methods. They are intentionally repetitive across modules — the
shape stays the same so muscle memory transfers from one module to
the next. This page gives you the canonical examples plus a
support matrix so you know which factory each module actually
implements.

## Support matrix

<span class="module-badge">official · @omnitron-dev/titan-*</span>

| Module                  | `forRoot` | `forRootAsync` | `forFeature` | `forWorker` |
| ----------------------- | :-------: | :------------: | :----------: | :---------: |
| `titan-auth`            |     ✓     |        ✓       |       —      |      —      |
| `titan-cache`           |     ✓     |        ✓       |       ✓      |      —      |
| `titan-database`        |     ✓     |        ✓       |       ✓      |      —      |
| `titan-discovery`       |     ✓     |        —       |       —      |      —      |
| `titan-events`          |     ✓     |        ✓       |       ✓      |      —      |
| `titan-health`          |     ✓     |        ✓       |       ✓      |      —      |
| `titan-lock`            |     ✓     |        ✓       |       —      |      —      |
| `titan-metrics`         |     ✓     |        ✓       |       —      |      —      |
| `titan-notifications`   |     ✓     |        ✓       |       —      |      ✓      |
| `titan-pm`              |     ✓     |        ✓       |       —      |      —      |
| `titan-ratelimit`       |     ✓     |        ✓       |       —      |      —      |
| `titan-redis`           |     ✓     |        ✓       |       ✓      |      —      |
| `titan-scheduler`       |     ✓     |        ✓       |       ✓      |      —      |
| `titan-telemetry-relay` |     —     |        —       |       —      |      —      |

> `titan-telemetry-relay` does **not** ship a Module class. Use
> `new TelemetryRelayService(options)` directly and wire it via a
> `useValue` provider when DI integration is wanted.

<span class="module-badge">built-in · @omnitron-dev/titan · config + logger</span>

Built-in modules follow the same pattern:

| Module   | `forRoot` | `forRootAsync` | `forFeature` |
| -------- | :-------: | :------------: | :----------: |
| `config` |     ✓     |        ✓       |       —      |
| `logger` |     ✓     |        ✓       |       —      |

## `forRoot(options)` — synchronous configuration

The simplest form. Options are known at boot time.

```typescript
@Module({
  imports: [
    TitanRedisModule.forRoot({
      config: { host: 'localhost', port: 6379, db: 0 },
    }),
    TitanCacheModule.forRoot({
      defaultTtl: 60,
      maxSize:    10_000,
    }),
  ],
})
class AppModule {}
```

Use when:

- Options come from a literal object or `process.env` read.
- No other module's services need to compute the options.

## `forRootAsync({ useFactory, inject?, imports? })` — DI-driven

When the options depend on another module's service — most often
`ConfigService` — use the async variant.

```typescript
import { TitanRedisModule } from '@omnitron-dev/titan-redis';
import { CONFIG_SERVICE_TOKEN, ConfigService }
  from '@omnitron-dev/titan/module/config';

@Module({
  imports: [
    TitanRedisModule.forRootAsync({
      useFactory: (config: ConfigService) => ({
        config: {
          host: config.get('redis.host'),
          port: config.get('redis.port'),
          db:   config.get('redis.db'),
        },
      }),
      inject: [CONFIG_SERVICE_TOKEN],
    }),
  ],
})
class AppModule {}
```

The factory:

- Receives the injected dependencies in the order declared in `inject`.
- Returns the same `options` shape as `forRoot()`.
- May be sync or async (`async (config) => {...}` is fine).

### Importing extra modules into the factory scope

If your factory depends on services that aren't globally exported,
import their host module:

```typescript
TitanCacheModule.forRootAsync({
  imports: [SecretsModule],                    // brings SecretsService into scope
  useFactory: (secrets: SecretsService) => ({
    redis: { url: secrets.get('REDIS_URL') },
  }),
  inject: [SecretsService],
}),
```

### `useClass` / `useExisting` variants

Some modules also accept class-based provider variants:

```typescript
NotificationsModule.forRootAsync({
  useClass: NotificationsOptionsFactory,       // implements `createNotificationsOptions(): Options`
})

// Or reuse an existing factory exported by another module:
NotificationsModule.forRootAsync({
  useExisting: SharedOptionsFactory,
})
```

Refer to per-module pages — `useClass` and `useExisting` are
supported by `titan-notifications` but not uniformly by all
modules.

## `forFeature(...)` — register additional resources

`forFeature` is for *adding* per-feature configuration **after**
the global module has been bound. Shape varies per module.

### `titan-database.forFeature(repositories)`

```typescript
@Module({
  imports: [
    TitanDatabaseModule.forFeature([
      UsersRepository,
      OrdersRepository,
    ]),
  ],
})
class UsersModule {}
```

Each repository becomes injectable by class reference; the
underlying `Database` connection is the one registered by the
global `TitanDatabaseModule.forRoot(...)`.

### `titan-cache.forFeature(caches)`

```typescript
@Module({
  imports: [
    TitanCacheModule.forFeature([
      { name: 'users',  defaultTtl: 300 },
      { name: 'tokens', defaultTtl: 60  },
    ]),
  ],
})
class CacheLayerModule {}
```

Multiple named caches share the same backing store but have
independent TTLs and namespacing.

### `titan-events.forFeature(emitters)`

Adds event metadata / discovery for additional event-emitting
services. See [`titan-events`](/docs/titan/modules/events).

### `titan-health.forFeature(indicators)`

Registers additional `IHealthIndicator` classes after `forRoot`:

```typescript
@Module({
  imports: [
    TitanHealthModule.forFeature([
      StripeHealthIndicator,
      SearchClusterIndicator,
    ]),
  ],
})
class FeatureModule {}
```

### `titan-redis.forFeature(clients)`

Adds named Redis clients:

```typescript
@Module({
  imports: [
    TitanRedisModule.forFeature(['cache', 'queue']),
  ],
})
class MultiClientModule {}
```

Inject via the token factory:

```typescript
import { getRedisClientToken } from '@omnitron-dev/titan-redis';

constructor(
  @Inject(getRedisClientToken('cache')) private cacheRedis: Redis,
  @Inject(getRedisClientToken('queue')) private queueRedis: Redis,
) {}
```

### `titan-scheduler.forFeature(providers)`

Registers additional providers that own `@Cron` / `@Interval` /
`@Timeout` methods, ensuring the scheduler discovers them.

## `forWorker(...)` — separate runtime

`titan-notifications` ships this fourth factory specifically for
the **worker** pod — the one that consumes the messaging
transport and actually delivers messages. The producer pod uses
`forRoot`; the worker pod uses `forWorker`.

```typescript
// Worker pod
@Module({
  imports: [
    NotificationsModule.forWorker({
      targetResolver:   NOTIFICATION_TARGET_RESOLVER,
      persister:        NOTIFICATION_PERSISTER,
      realtimeSignaler: NOTIFICATION_REALTIME_SIGNALER,
      workerOptions:    { concurrency: 8 },
    }),
  ],
  providers: [
    { provide: NOTIFICATION_TARGET_RESOLVER, useClass: MyTargetResolver },
    { provide: NOTIFICATION_PERSISTER,        useClass: MyPersister },
    { provide: NOTIFICATION_REALTIME_SIGNALER,useClass: MyRealtimeSignaler },
  ],
})
class NotificationsWorkerModule {}
```

This separation lets you scale producers (web pods) independently
from workers (delivery pods).

## Global vs scoped modules

Most modules accept `isGlobal: true` (or `global: true` on the
returned `DynamicModule`) — when set, the module's exports are
visible to every other module without explicit re-import.

```typescript
TitanRedisModule.forRoot({ config, isGlobal: true });
```

| When                                             | Recommendation              |
| ------------------------------------------------ | --------------------------- |
| Infrastructure (config, logger, redis, database) | `isGlobal: true`            |
| Feature modules (your business logic)            | leave scoped                |
| Test environments                                | scoped — easier to override |

## Async lifecycle of `forRootAsync`

```mermaid
sequenceDiagram
  participant App as Application.create
  participant Container as DI container
  participant Factory as useFactory
  participant Module as TitanXModule

  App->>Container: resolve dependency graph
  Container->>Container: resolve inject[] tokens
  Container->>Factory: call useFactory(...deps)
  Factory-->>Container: options
  Container->>Module: instantiate with resolved options
  Module->>Container: bind exports
  App->>Module: onInit / onStart
```

The factory is called **once** per app boot, after its declared
`inject:` dependencies are resolved. The resolved options are then
used to instantiate the module's services.

## Common gotchas

- **`inject:` order must match factory arguments.** A misordered
  array silently passes the wrong service to the wrong parameter.
  Prefer named destructuring + tuple typing:
  ```typescript
  useFactory: (config: ConfigService, redis: RedisManager) => ({...}),
  inject: [CONFIG_SERVICE_TOKEN, REDIS_MANAGER],
  ```
- **`forFeature` before `forRoot`** — the global module has to be
  bound first or `forFeature` won't find what to attach to.
  `Application.create({ modules: [GlobalModule, FeatureModule] })`
  resolves order from the dependency graph, but if you're loading
  them in `imports: []` manually, list `forRoot` first.
- **Hardcoding `redisOptions` in two modules** — when you let
  `titan-cache`, `titan-lock`, `titan-discovery`, etc. each spin
  up their own Redis connection, you triple-count connections.
  Configure `TitanRedisModule` once and let the other modules
  pick up the shared client.
- **Calling `forRoot` twice** — the second call typically wins,
  which is rarely what you want. If you need different options for
  different scopes, use `forFeature` (where supported) or
  consciously override via test fixtures.
- **`forRootAsync` factory throwing.** A throw aborts boot — the
  error propagates out of the container's async resolution (the
  Nexus DI layer raises `ResolutionError` / `AsyncResolutionError`
  for resolution failures). Don't log-and-swallow; let it crash so
  the operator sees the configuration mistake.

## Migration tip — sync to async

Start with `forRoot({...})`. The moment you need a value from
`ConfigService` (or any other service), switch the call to
`forRootAsync({useFactory, inject})`. The options object shape
stays identical — only the wrapper changes.

```typescript
// Before
TitanRedisModule.forRoot({ config: { url: 'redis://localhost' } })

// After
TitanRedisModule.forRootAsync({
  useFactory: (cfg: ConfigService) => ({ config: { url: cfg.get('redis.url') } }),
  inject:     [CONFIG_SERVICE_TOKEN],
})
```

## See also

- [Module map](/docs/titan/modules/module-map) — visual dependency graph
- [Tokens & RPC reference](/docs/titan/modules/tokens-reference) — what to put
  in `inject:`
- [Modules system / Dynamic modules](/docs/titan/modules-system/dynamic-modules) —
  the underlying `DynamicModule` mechanism this all sits on
- [Authoring a module](/docs/titan/modules-system/authoring-modules) —
  how to implement these patterns when building your own

---

---
id: "titan/modules/pm"
title: "titan-pm"
---

# titan-pm

<span class="module-badge">official · @omnitron-dev/titan-pm · stable</span>

Process supervision with worker pools, IPC (Unix / TCP / WebSocket),
supervisor patterns, restart policies with exponential backoff,
resource limits, metrics + health collection, a transparent service
proxy that makes every process behave like a local Netron service,
and decorators for in-process workflows and resilience.

```bash
pnpm add @omnitron-dev/titan-pm
```

## When you need it

- **CPU-bound work.** Image processing, ML inference, parsing —
  isolate from the main event loop.
- **Crash isolation.** A worker crash should not take the whole app
  down. Supervisors restart with backoff and policy.
- **Worker pools.** Many parallel workers across CPUs with load
  balancing.
- **Multi-step work.** Dependency-ordered stages with explicit compensation.
  Workflow state is in memory; process or host loss requires an external durable
  operation/recovery contract.

## Quickstart

```typescript
import { ProcessManagerModule } from '@omnitron-dev/titan-pm';

@Module({
  imports: [
    ProcessManagerModule.forRoot({
      isolation: 'worker',                       // 'none' | 'worker' | 'child'
      transport: 'unix',                          // 'unix' | 'tcp' | 'ws'
      restartPolicy: {
        enabled:     true,
        maxRestarts: 3,
        window:      60_000,
        delay:       1_000,
        backoff:     { type: 'exponential', initial: 1_000, max: 30_000, factor: 2 },
      },
      resources: {
        maxMemory: '512MB',
        maxCpu:    1.0,
        timeout:   30_000,
      },
      monitoring: {
        healthCheck: { interval: 5_000, timeout: 2_000 },
        metrics:     true,
        tracing:     false,
      },
    }),
  ],
})
class AppModule {}
```

Async config via `forRootAsync({ useFactory, inject?, useExisting? })`.

## `IProcessManagerConfig`

| Option          | Type                                                                                 |
| --------------- | ------------------------------------------------------------------------------------ |
| `isolation`     | `'none' \| 'worker' \| 'child'` — default `'worker'`                                 |
| `transport`     | `'unix' \| 'tcp' \| 'ws'` — default `'unix'`                                         |
| `restartPolicy` | `{ enabled, maxRestarts, window, delay, backoff?: { type, initial, max, factor? } }` |
| `resources`     | `{ maxMemory, maxCpu, timeout }`                                                     |
| `monitoring`    | `{ healthCheck: { interval, timeout }, metrics, tracing }`                           |
| `testing`       | `{ useMockSpawner }`                                                                 |
| `advanced`      | `{ gracefulShutdownTimeout, livenessSweepIntervalMs, handleSignals }`                |

Defaults are pulled from `DEFAULT_PM_CONFIG`.

## `ProcessManager` — the API

```typescript
import { ProcessManager, PM_MANAGER_TOKEN, Process, Public }
  from '@omnitron-dev/titan-pm';

@Process({ name: 'image-worker' })
class ImageWorker {
  @Public()
  async resize(input: Buffer, width: number): Promise<Buffer> {
    return sharp(input).resize(width).toBuffer();
  }
}

@Service({ name: 'media' })
class MediaService {
  constructor(@Inject(PM_MANAGER_TOKEN) private readonly pm: ProcessManager) {}

  @Public()
  async resize(input: Buffer, width: number) {
    const worker = await this.pm.spawn(ImageWorker);
    return worker.resize(input, width);     // transparent RPC call
  }
}
```

`spawn()` accepts either a process **class** or a **module path string**
(for fork/process isolation where the worker runs in a separate module).

Process options use one contract from the manager through the native spawner to
the worker. `cwd`, environment, logging, startup and shutdown budgets, RPC
`requestTimeout`, and explicit method visibility are preserved. Without
`allMethodsPublic: true`, undecorated methods remain unavailable over RPC.

Use `cwd` with child-process isolation for an application-specific working
directory. Worker threads share the parent's directory and reject a different
`cwd`; they never change the parent's directory. Pools inherit common
`spawnOptions`; `spawnOptionsFactory(index)` supplies per-worker overrides.
Keep mutable application state outside sealed runtime directories.

Use `beforeSpawn` for asynchronous parent-side admission, such as verifying an
installed executable before it starts. The manager awaits it before registering
or spawning a process, including supervisor restarts and pool replacements. A
rejection prevents that spawn; the function is never serialized to the worker.
The pool also rechecks shutdown intent after admission, so a pending check cannot
create a worker after draining begins. This hook does not replace application
readiness or prove that the process subsequently remained healthy.

| Method                                      | Returns                            |
| ------------------------------------------- | ---------------------------------- |
| `spawn<T>(processClassOrPath, options?)`    | `Promise<ServiceProxy<T>>`         |
| `pool<T>(processClassOrPath, options?)`     | `Promise<IProcessPool<T>>`         |
| `createSupervisor(classOrConfig, options?)` | `ProcessSupervisor`                |
| `discover<T>(serviceName)`                  | `Promise<ServiceProxy<T> \| null>` |
| `getProcess(id)`                            | `IProcessInfo \| undefined`        |
| `listProcesses()`                           | `IProcessInfo[]`                   |
| `kill(id, signal?, { shutdownTimeout? })`   | `Promise<boolean>`                 |
| `shutdown({ timeout?, force? })`            | `Promise<void>`                    |
| `getMetrics(processId)`                     | `Promise<IProcessMetrics \| null>` |
| `getHealth(processId)`                      | `Promise<IHealthStatus \| null>`   |

The `ServiceProxy` returned by `spawn()` exposes every `@Public`
method of the process class as a typed remote call — transport
details are hidden.

### Worker pools

```typescript
const pool = await this.pm.pool(ImageWorker, {
  size: 2,
  autoScale: { enabled: true, min: 2, max: 8 },
});
const result = await pool.execute('resize', input, width);
```

The pool routes the call to a free worker, queues if all workers
are busy, and respects the pool's load-balancing policy. The proxy
returned by `pool()` also lets you call `@Public` methods directly
(`pool.resize(input, width)`).

`membershipChanging` reports whether the pool is spawning, replacing, retiring
or draining workers. `pool:membership-settled` notifies observers after a resize
or replacement attempt leaves the lifecycle queue, including a failed attempt.
Read the current flag and worker identities when handling it: a queued successor
may already be running. The event does not mean the requested size was reached
or that every worker is healthy.

Worker retirement excludes the old proxy from both direct and queued requests
before termination begins. A replacement becomes ready first; already admitted
calls finish within `requestTimeout` (30 seconds when disabled). If a planned
drain cannot finish, it rejects and keeps the worker owned for retry. Explicit
pool destruction retains its bounded shutdown policy. A failed termination never
re-enables the old proxy. Requests are not automatically replayed, because an
unknown outcome may already have changed application state.

Only healthy, idle workers accept new calls. Requests wait in the bounded queue
while every worker is busy, unhealthy or retiring, and resume when capacity or
health recovers. Normal recycling can change worker PIDs without restarting the
application or replacing its topology endpoint.

### Supervisors

```typescript
const supervisor = this.pm.createSupervisor(ParentProcess);
```

Create the supervisor, attach observers, then call `await supervisor.start()`:

```typescript
const supervisor = pm.createSupervisor({
  strategy: SupervisionStrategy.ONE_FOR_ONE,
  maxRestarts: 3,
  window: 60_000,
  backoff: { type: 'exponential', initial: 1_000, max: 30_000, factor: 2 },
  children: [{ name: 'worker', process: '/absolute/worker.js', critical: false }],
});
supervisor.on('child:start-failed', (name, error) => logger.error({ name, err: error }, 'Startup failed'));
supervisor.on('child:recovery-stopped', (name, error) => logger.error({ name, err: error }, 'Recovery stopped'));
await supervisor.start();
```

Import `SupervisionStrategy` from `@omnitron-dev/titan-pm`. Both noncritical initial startup failures and failures during a crash restart consume the same configured retry budget and backoff. `maxRestarts: 0` disables automatic retries. A required critical child still fails initial supervisor startup immediately; after the supervisor is running, its recovery follows policy.

Recovery is serialized per supervisor. `ONE_FOR_ONE` preserves unaffected siblings; `ONE_FOR_ALL` and `REST_FOR_ONE` retain declaration order, including children whose previous start failed. `stop()` cancels waiting retries and joins any spawn already in progress before stopping its result. No delayed restart may resurrect a stopped supervisor.

Use `child:start-failed` for the original cause, `child:restart` for attempts, and `child:started` for readiness. `child:recovery-stopped` means policy declined further attempts or failed itself; critical exhaustion emits `escalate`. Explicit `restartChild(name)` can retry a failed declaration. Omnitron preserves the last failure until the whole application is online and reports when the policy stops recovery.

## Decorators

The rich decorator set is what makes processes feel like local
services. Annotated processes can declare lifecycle, observability,
resilience and workflow compensation declaratively.

### Class-level

| Decorator            | Purpose                                                         |
| -------------------- | --------------------------------------------------------------- |
| `@Process(opts?)`    | Mark a class as a runnable process                              |
| `@Workflow()`        | Dependency-ordered in-process workflow                          |
| `@Actor()`           | Compatibility metadata only; no actor runtime or serial mailbox |
| `@Supervisor(opts?)` | Supervisor for child processes                                  |

### Method-level — exposure and lifecycle

| Decorator            | Purpose                                                      |
| -------------------- | ------------------------------------------------------------ |
| `@Public()`          | Expose method over IPC                                       |
| `@Stage()`           | Workflow stage                                               |
| `@Compensate(stage)` | Compensates a completed stage after a known workflow failure |
| `@HealthCheck()`     | Mark method as the health check                              |
| `@OnShutdown()`      | Cleanup hook before the process exits                        |
| `@Child()`           | Supervisor: declare a supervised child                       |

### Method-level — observability

| Decorator            | Purpose                                                          |
| -------------------- | ---------------------------------------------------------------- |
| `@Trace()`           | Distributed tracing span                                         |
| `@Metric(name?)`     | Auto-instrument metric (flags the method for metrics collection) |
| `@Validate(options)` | Validate input (`{ schema?, validator? }`)                       |
| `@Cache(options?)`   | Cache the result (`{ ttl?, key?, condition? }`)                  |

### Method-level — resilience

| Decorator               | Purpose                                                                                               |
| ----------------------- | ----------------------------------------------------------------------------------------------------- |
| `@CircuitBreaker(opts)` | Circuit-break the method (`{ threshold?, timeout?, fallback? }`)                                      |
| `@RateLimit(opts)`      | Rate-limit the method (`{ rps?, burst?, strategy?, key? }`)                                           |
| `@Idempotent(opts)`     | Deduplicate by key (`{ key, ttl? }` — `key` names the arg field, `ttl` a duration string like `'1h'`) |

### Combined example — a worker class

```typescript
import {
  Process, Public, OnShutdown, HealthCheck,
  Trace, Metric, CircuitBreaker, Idempotent,
} from '@omnitron-dev/titan-pm';

@Process({ name: 'image-worker' })
class ImageWorker {
  @Public()
  @Trace()
  @Metric('images.resized')
  @CircuitBreaker({ threshold: 5, timeout: 30_000 })
  @Idempotent({ key: 'requestId', ttl: '1h' })
  async resize(input: Buffer, width: number): Promise<Buffer> {
    return sharp(input).resize(width).toBuffer();
  }

  @HealthCheck()
  async health() {
    return { status: 'healthy', queueDepth: this.queue.size };
  }

  @OnShutdown()
  async cleanup() {
    await this.queue.drain();
  }
}
```

## Architecture

```mermaid
flowchart TB
  Caller[Service.invoke]
  Caller --> Manager[ProcessManager]
  Manager --> Spawner[ProcessSpawner]
  Manager --> Registry[ProcessRegistry]
  Spawner --> P1[Process 1]
  Spawner --> P2[Process 2]
  Spawner --> Pn[Process N]
  Manager --> Sup[ProcessSupervisor]
  Sup -- restart on crash --> Spawner
  Manager --> Metrics[ProcessMetricsCollector]
  Manager --> Health[ProcessHealthChecker]
  Manager --> Sweep[Liveness sweep<br/>defense-in-depth]
  Sweep -- detect dead PID --> Sup
```

The **liveness sweep** is defense-in-depth: even if a process exits
without firing its exit signal, the sweep detects the dead PID
(every `livenessSweepIntervalMs`, default 30 s) and tells the
supervisor to restart per policy.

## Tokens

| Token               |
| ------------------- |
| `PM_CONFIG_TOKEN`   |
| `PM_MANAGER_TOKEN`  |
| `PM_REGISTRY_TOKEN` |
| `PM_SPAWNER_TOKEN`  |
| `PM_METRICS_TOKEN`  |
| `PM_HEALTH_TOKEN`   |

## Lifecycle

`ProcessManager` is registered as a singleton service. On
application shutdown, `shutdown({ timeout?, force? })` runs to
gracefully terminate every spawned process and pool (respects
`gracefulShutdownTimeout`).

Manager shutdown is terminal: it closes new process/pool admission immediately and
owns initialization already in progress. Concurrent and repeated calls share the
same promise and outcome; a later `force` argument does not replace the first
call's policy. Every pool and worker cleanup is attempted within the timeout, with
errors collected rather than hidden. A deadline or failed drain/termination rejects
and remains rejected even if a worker exits later. Inspect retained ownership and
native process evidence before recovery; another `shutdown()` call is not proof
of a clean exit.

Omnitron's application `stopAll` is a separate, reversible operation: it stops
application handles and reports aggregate failures while keeping the manager
available for a later explicit start. Terminal manager shutdown belongs to the
resident execution owner or Titan's application lifecycle.

## Isolation modes

| Isolation | Backed by            | Best for                                                        |
| --------- | -------------------- | --------------------------------------------------------------- |
| `worker`  | `worker_threads`     | CPU-bound work, low IPC latency; shared memory option (default) |
| `child`   | `child_process.fork` | Strong isolation, ability to crash without taking down peers    |
| `none`    | The calling process  | Tests. No Netron management client is created for the worker.   |

`execArgv` forces `child` even when `isolation` is `'worker'`: worker threads
do not accept Node CLI flags.

Sandboxing is a separate option, `security.isolation` on a spawn
(`'none' | 'vm' | 'container'`). It does not select a spawn strategy. Neither
`'vm'` nor `'container'` is implemented — both spawn an ordinary child process
that shares the parent's filesystem and network, and say so in a warning at
spawn time. Do not rely on either as a security boundary.

## Transport choice

| Transport | Best for                                       |
| --------- | ---------------------------------------------- |
| `unix`    | Local IPC; lowest latency on the same host     |
| `tcp`     | Cross-host workers; encrypt with TLS if needed |
| `ws`      | Long-lived workers behind a proxy              |

## Anti-patterns

- **Tiny tasks in `child` isolation.** Process fork has \~50 ms
  startup cost. Reserve for long-lived work or pre-warmed pools.
- **No restart policy.** A crashing worker keeps respawning at
  full speed without `restartPolicy: { backoff }`. Always set
  exponential backoff with a cap.
- **No `@OnShutdown`.** Workers killed mid-task lose work in flight.
  Drain queues in the shutdown hook.
- **Sharing mutable state via IPC.** IPC is for messages, not
  shared mutable state. If you need shared state, put it in Redis
  or a database.

## See also

- [Resilience / Retry](/docs/titan/resilience/retry), [Circuit Breaker](/docs/titan/resilience/circuit-breaker)
- [`titan-metrics`](/docs/titan/modules/metrics) — collects process metrics
  when `monitoring.metrics: true`
- [`titan-events`](/docs/titan/modules/events) — inter-process event delivery

## Workflow failure and recovery

`@Workflow()`, `@Stage({ dependsOn, timeout, retries })` and `@Compensate(stageName)`
are executed by Titan PM. Dependency validation uses Titan's shared
`dependencyBatches` utility, also used by Omnitron application startup. Missing
prerequisites, duplicate stage names and cycles are refused before effects begin.
Compensation declaration order does not matter; subclass metadata does not mutate
its parent workflow.

One instance runs once at a time. Use separate instances for independent concurrent
runs. Each later run receives a new run ID and fresh stage results. A failed
parallel stage waits for its active siblings to settle before compensating completed
stages in reverse dependency order. Compensation outcomes are visible in
`getContext().stages`; failed compensation returns an `AggregateError` containing
both the original failure and recovery failures, and blocks reuse of that instance.

A timeout does not cancel an arbitrary JavaScript handler. Titan reports
`WorkflowStageUncertainError` (exported by `@omnitron-dev/titan-pm`) and stage status
`unknown`, then blocks automatic retry, compensation and instance reuse. The handler
may still complete externally. Reconcile its actual effects before starting a new
workflow. Explicit retries after ordinary failures require idempotent or otherwise
retry-safe handlers. Invalid timeout/retry settings are rejected before execution.

This is not persistent workflow execution or exactly-once external effects. For
infrastructure operations, use the journal, effect receipts and reconciliation
contracts in the [resilient control-plane design](/docs/omnitron/resilient-control-plane).
The `@Actor()` compatibility decorator records metadata only; it does not create an
actor mailbox, serialize messages or provide distributed actor supervision.

## Confirmed process termination

`kill()` resolves `true` after termination and cleanup, and `false` only when the
process is not registered. An unconfirmed exit rejects and retains the worker
handle for a retry. Sending a signal, a closed IPC channel, or an elapsed deadline
is not proof of exit. `SIGKILL` selects an immediate forced shutdown; an optional
`shutdownTimeout` reaches the same manager cleanup path for supervisor children.

Supervisor stop and pool destruction attempt every owned child and report aggregate
failures. Successfully stopped children are removed from ownership; survivors stay
registered, with automatic recovery disabled during teardown. Pool destruction
also joins already admitted spawns, preventing late workers from escaping a stop.
Retry the operation after resolving the reported cause; do not interpret a timeout
as permission to launch a replacement over an unconfirmed process.

---

---
id: "titan/modules/ratelimit"
title: "titan-ratelimit"
---

# titan-ratelimit

<span class="module-badge">official · @omnitron-dev/titan-ratelimit · stable</span>

Unified rate limiting with three algorithms (sliding-window,
fixed-window, token-bucket), pluggable storage (in-memory or Redis),
tiered plans, optional queueing for fair processing of excess
requests, per-method `@RateLimit` / `@Throttle` decorators, and
statistics tracking with per-tier breakdown.

```bash
pnpm add @omnitron-dev/titan-ratelimit
```

## Algorithms

| Strategy           | Behaviour                                            | Best for                            |
| ------------------ | ---------------------------------------------------- | ----------------------------------- |
| `'sliding-window'` | Continuously rolling window; smooth limits           | API per-user / IP throttling        |
| `'fixed-window'`   | Hard-reset at window boundary; simpler / cheaper     | Per-hour quotas, accounting         |
| `'token-bucket'`   | Tokens refill at a rate; can burst up to bucket size | Bursty workloads with smooth refill |

### Fairness comparison

```mermaid
flowchart LR
  subgraph "Fixed window — 100 req / minute"
    F[t=0..60s: 100 ok<br/>t=60..120s: 100 ok<br/>***burst of 200 at t=59..61***]
  end
  subgraph "Sliding window — 100 req / minute"
    S[t=0..60s: 100 ok<br/>any subsequent 60s rolling: ≤ 100]
  end
  subgraph "Token bucket — 100/min refill, burst 150"
    T[Can burst 150 ok<br/>then drained until refill]
  end
```

- **Fixed window** is the cheapest to implement (one counter, one
  TTL) but allows a 2× burst around boundaries — 100 requests in
  the last second of one window and 100 in the first second of
  the next looks like 200 in 2 seconds.
- **Sliding window** is the fairest at the cost of one sorted set
  operation per check.
- **Token bucket** lets clients burst above the steady rate up to
  the bucket size — best for "usually quiet, sometimes spikes"
  workloads.

## Quickstart

### In-memory (single-pod)

```typescript
import { TitanRateLimitModule } from '@omnitron-dev/titan-ratelimit';

@Module({
  imports: [
    TitanRateLimitModule.forRoot({
      enabled:         true,
      strategy:        'sliding-window',
      defaultLimit:    100,
      defaultWindowMs: 60_000,
      storageType:     'memory',
    }),
  ],
})
class AppModule {}
```

### Redis-backed (multi-pod)

```typescript
TitanRateLimitModule.forRoot({
  storageType:     'redis',
  strategy:        'token-bucket',
  defaultLimit:    100,
  defaultWindowMs: 60_000,
  burstLimit:      150,
  tokenRefillRate: 100,
  keyPrefix:       'rl',
  queueEnabled:    true,
  maxQueueSize:    1_000,
  queueTimeoutMs:  5_000,
  tiers: {
    free:       { name: 'free',       limit: 10,    windowMs: 60_000 },
    pro:        { name: 'pro',        limit: 1_000, windowMs: 60_000 },
    enterprise: { name: 'enterprise', limit: 10_000, windowMs: 60_000, burst: 5_000 },
  },
})
```

Async config via `forRootAsync({ useFactory, inject?, imports?, isGlobal? })`.

## `IRateLimitModuleOptions`

| Option            | Type                                                   | Default            |
| ----------------- | ------------------------------------------------------ | ------------------ |
| `enabled`         | `boolean`                                              | `true`             |
| `strategy`        | `'sliding-window' \| 'fixed-window' \| 'token-bucket'` | `'sliding-window'` |
| `keyPrefix`       | `string`                                               | `'ratelimit'`      |
| `defaultLimit`    | `number`                                               | `100`              |
| `defaultWindowMs` | `number` (ms)                                          | `60_000`           |
| `burstLimit`      | `number` (0 disables burst)                            | `0`                |
| `tokenRefillRate` | `number` (tokens / window)                             | `100`              |
| `queueEnabled`    | `boolean`                                              | `false`            |
| `maxQueueSize`    | `number`                                               | `1_000`            |
| `queueTimeoutMs`  | `number` (ms — max wait in queue)                      | `5_000`            |
| `storageType`     | `'memory' \| 'redis'`                                  | `'memory'`         |
| `tiers`           | `Record<string, IRateLimitTier>`                       | —                  |
| `defaultTier`     | `IRateLimitTier`                                       | —                  |
| `isGlobal`        | `boolean`                                              | `false`            |

## `IRateLimitTier`

```typescript
interface IRateLimitTier {
  name:      string;
  limit:     number;
  burst?:    number;       // extra allowance above limit
  priority?: number;       // for queue ordering
  windowMs?: number;
}
```

The effective limit for a check is `tier.limit + (tier.burst ??
options.burstLimit)`.

## `RateLimitService` — the API

```typescript
import { RATE_LIMIT_SERVICE_TOKEN, type IRateLimitService }
  from '@omnitron-dev/titan-ratelimit';

@Service('charges@1.0.0')
class ChargesService {
  constructor(@Inject(RATE_LIMIT_SERVICE_TOKEN) private readonly rate: IRateLimitService) {}

  @Public()
  async charge(userId: string, amount: number) {
    const result = await this.rate.consume(`charge:${userId}`, { tier: 'pro' });
    if (!result.allowed) {
      throw Errors.tooManyRequests('too many charge attempts', { retryAfter: result.retryAfter });
    }
    // proceed
  }
}
```

| Method                     | Returns                                                         |
| -------------------------- | --------------------------------------------------------------- |
| `check(options)`           | `Promise<IRateLimitResult>` — full options                      |
| `consume(key, options?)`   | `Promise<IRateLimitResult>` — convenience: consume = true       |
| `enforce(key, options?)`   | `Promise<void>` — throws `RateLimitExceededError` on reject     |
| `getStatus(key, options?)` | `Promise<IRateLimitResult>` — read-only check (consume = false) |
| `reset(key, tier?)`        | `Promise<void>`                                                 |
| `getStats()`               | `IRateLimitStats` — overall + per-tier breakdown                |

The difference:

- `consume()` returns a result; you inspect `result.allowed`.
- `enforce()` throws `RateLimitExceededError` if rejected — the
  thrown error carries `result` so you can read `retryAfter`,
  `remaining`, `limit`, etc.

Prefer `consume()` when you want to handle the rate-limit case
with a custom response shape; prefer `enforce()` when you want a
single line that "just blocks".

## `IRateLimitResult`

```typescript
interface IRateLimitResult {
  allowed:     boolean;
  remaining:   number;
  limit:       number;       // includes burst (effective limit)
  resetAt:     number;       // epoch ms when window resets
  retryAfter?: number;       // seconds — set only when allowed=false
  tier?:       string;       // set if a tier was used
}
```

Surface to HTTP clients as standard rate-limit headers:

```typescript
const result = await rate.consume(key);
res.setHeader('X-RateLimit-Limit',     result.limit.toString());
res.setHeader('X-RateLimit-Remaining', result.remaining.toString());
res.setHeader('X-RateLimit-Reset',     Math.ceil(result.resetAt / 1000).toString());
if (!result.allowed) {
  res.setHeader('Retry-After', result.retryAfter!.toString());
  return res.status(429).end();
}
```

## Decorators

### `@RateLimit(options?)`

```typescript
import { RateLimit } from '@omnitron-dev/titan-ratelimit';

@Public()
@RateLimit({ limit: 10, windowMs: 60_000 })
async createUser(input: CreateInput) { /* … */ }
```

**What has to be true for this to enforce anything.** The decorator resolves
its service as `this.__rateLimitService__ ?? <the module's own instance>`, so
importing `TitanRateLimitModule` is enough — the module publishes itself for
the decorators at init and withdraws on destroy. Injecting the service under
that exact field name still wins, which is how a per-class override or a test
double is installed.

With neither — the module absent, or destroyed — the decorator **allows the
request**. It writes a warning first, but only if the instance happens to carry
a `logger` or `loggerModule`, so on a class without one the limit is silently
not applied. A declared limit that does nothing is the failure this note exists
to prevent: check that the module is imported before trusting a `@RateLimit`.

Two different decorators are called `RateLimit` in this repository. This one —
from `@omnitron-dev/titan-ratelimit` — wraps the method and enforces, and takes
`windowMs`. The one exported from `@omnitron-dev/titan` records metadata for
Netron's authorization layer to read, and takes `window`. The import path is
the whole difference, and the field names are the tell.

### `@Throttle(rps)` — convenience

```typescript
import { Throttle } from '@omnitron-dev/titan-ratelimit';

@Public()
@Throttle(10)                                  // ≤ 10 requests / sec
async search(q: string) { /* … */ }
```

`@Throttle(rps)` is equivalent to `@RateLimit({ limit: rps,
windowMs: 1_000 })`.

## Storage backends

| Backend    | Class                    | When                                                |
| ---------- | ------------------------ | --------------------------------------------------- |
| `'memory'` | `MemoryRateLimitStorage` | Single-pod; lower latency; counts lost on restart   |
| `'redis'`  | `RedisRateLimitStorage`  | Multi-pod; persisted across restarts; cluster-aware |

### `IRateLimitStorage` interface

Custom backends implement:

```typescript
interface IRateLimitStorage {
  increment(key: string, ttl?: number): Promise<number>;
  get(key: string): Promise<number | null>;
  set(key: string, value: number, ttl?: number): Promise<void>;
  delete(key: string): Promise<void>;

  // Sliding window primitives
  addToSortedSet(key: string, score: number, member: string, ttl?: number): Promise<void>;
  removeFromSortedSetByScore(key: string, min: number, max: number): Promise<void>;
  countSortedSet(key: string): Promise<number>;
}
```

The Redis backend uses Lua scripts for the
sliding/fixed-window counters (atomic increment + TTL set) and
`INCR`/`EXPIRE` for token bucket — single-roundtrip per check.

## Tiered plans

```typescript
TitanRateLimitModule.forRoot({
  defaultTier: { name: 'free', limit: 10, windowMs: 60_000 },
  tiers: {
    free:       { name: 'free',       limit: 10,     windowMs: 60_000 },
    pro:        { name: 'pro',        limit: 1_000,  windowMs: 60_000 },
    enterprise: { name: 'enterprise', limit: 10_000, windowMs: 60_000, burst: 5_000 },
  },
})
```

The tier is selected per call via the `tier` option to `consume()` /
`enforce()`:

```typescript
const tier = user.plan ?? 'free';
await rate.enforce(`api:${user.id}`, { tier });
```

When a tier is specified, the result includes the tier name and
`getStats()` exposes per-tier checks / allowed / denied counts:

```typescript
const stats = rate.getStats();
// stats.byTier?.get('pro') === { checks, allowed, denied }
```

## Queueing

When `queueEnabled: true`, excess requests **wait** in a queue
instead of immediately rejecting — useful when latency variance is
preferred over rejection.

```typescript
TitanRateLimitModule.forRoot({
  queueEnabled:   true,
  maxQueueSize:   1_000,        // drop after this many waiting
  queueTimeoutMs: 5_000,        // give up if not processed in time
})
```

Queueing semantics:

- **`maxQueueSize` exceeded** → new requests are rejected with
  `RateLimitExceededError` immediately.
- **`queueTimeoutMs` elapsed** → waiting request is rejected with
  the same error (`retryAfter` indicates how long to wait).
- **Tier priority** — when tiers specify `priority`, higher-
  priority tiers are processed first inside the queue.

> **Don't enable queueing on unauthenticated endpoints.** It turns
> rejections into work — an attacker can saturate the queue and
> starve legitimate requests. Use queueing only inside trust
> boundaries (after auth).

## Per-algorithm distributed coordination

When `storageType: 'redis'` and multi-pod, each algorithm uses a
slightly different Redis primitive set:

| Strategy           | Redis primitives                              | Atomicity             |
| ------------------ | --------------------------------------------- | --------------------- |
| `'fixed-window'`   | `INCR` + `EXPIRE` (or single Lua)             | Atomic per-key        |
| `'sliding-window'` | `ZADD` + `ZREMRANGEBYSCORE` + `ZCARD`         | Atomic via Lua script |
| `'token-bucket'`   | `INCR` for counter; refill computed in script | Atomic via Lua script |

All Lua scripts touch one key, so cluster-mode hash slots are not
an issue for a single check. Across keys (different users) there's
no cross-key coordination — desired for rate limiting.

## Recipes

### Per-IP throttle (behind trusted proxy)

```typescript
@Public()
@RateLimit({ limit: 60, windowMs: 60_000 })
async search(q: string, @Context() ctx: RequestContext) {
  const ip = ctx.headers['x-forwarded-for']?.split(',')[0]?.trim()
           ?? ctx.peerAddress;
  await this.rate.enforce(`ip:${ip}`);
  return this.searchEngine.run(q);
}
```

> Trust the client IP only behind a known trusted proxy chain.
> Otherwise an attacker spoofs `X-Forwarded-For` and bypasses
> per-IP limits.

### Per-user, tier-aware

```typescript
async getReport(@Auth() user: AuthContext) {
  const tier = user.plan ?? 'free';
  const result = await this.rate.consume(`reports:${user.id}`, { tier });

  if (!result.allowed) {
    throw Errors.tooManyRequests(
      `Report quota exceeded on ${tier} plan`,
      { retryAfter: result.retryAfter, upgrade: `https://app/upgrade?from=${tier}` },
    );
  }
  return this.reports.generate(user.id);
}
```

### Per-API-key, burst-tolerant

```typescript
TitanRateLimitModule.forRoot({
  strategy:        'token-bucket',
  defaultLimit:    100,
  burstLimit:      50,           // tolerate spikes up to 150
  tokenRefillRate: 100,          // refills at 100/min
});

// In your service:
await this.rate.enforce(`api-key:${apiKey}`);
```

Bursts up to 150 succeed; sustained rate above 100/min refills
slower than consumption and starts rejecting.

### Per-endpoint compound key

```typescript
@RateLimit({ limit: 5, windowMs: 60_000 })
async resetPassword(@Auth() user: AuthContext) {
  // Effective key under the hood: `ratelimit:resetPassword:${user.id}`
  // (the decorator builds compound keys from method name + auth context)
}
```

## Performance notes

| Cost dimension              | Memory backend    | Redis backend              |
| --------------------------- | ----------------- | -------------------------- |
| Per-check latency           | sub-μs            | one RTT (\~0.5–2 ms LAN)   |
| Per-check Redis ops         | 0                 | 1 Lua script (atomic)      |
| Memory per active key       | O(window-buckets) | O(window-buckets) on Redis |
| Sliding-window storage cost | O(N) timestamps   | O(N) sorted-set entries    |
| Fixed-window storage cost   | O(1)              | O(1) — single counter      |
| Token-bucket storage cost   | O(1)              | O(1) — counter + timestamp |

For most workloads:

- **High RPS per pod, lax fairness:** in-memory + fixed-window.
- **Multi-pod, smooth fairness:** Redis + sliding-window.
- **Multi-pod, bursty traffic:** Redis + token-bucket.

## Tokens

| Token                      | Type                      |
| -------------------------- | ------------------------- |
| `RATE_LIMIT_SERVICE_TOKEN` | `IRateLimitService`       |
| `RATE_LIMIT_OPTIONS_TOKEN` | `IRateLimitModuleOptions` |
| `RATE_LIMIT_STORAGE_TOKEN` | `IRateLimitStorage`       |

## Lifecycle

The service uses periodic cleanup intervals on the in-memory
storage (sweeping expired keys); no explicit lifecycle hooks
beyond the standard ones. On module shutdown, `destroy()` clears
the cleanup timer and closes the Redis storage client where
applicable.

## Algorithm selection — what to pick

```mermaid
flowchart LR
  Q{Bursty?}
  Q -- yes --> Tok[token-bucket]
  Q -- no --> Smooth{Smooth or hard cutoff?}
  Smooth -- smooth --> Sw[sliding-window]
  Smooth -- hard --> Fw[fixed-window]
```

- **Token bucket** — for bursty traffic where you allow `burstLimit`
  spikes that refill at `tokenRefillRate`.
- **Sliding window** — for smooth, fair rate limiting where you
  don't want users to "save up" allowance.
- **Fixed window** — for accounting-style quotas ("100 requests per
  hour" with hard reset on the hour).

## Integration with `titan-notifications`

`titan-notifications` ships its own per-channel rate limiter
(`NOTIFICATIONS_RATE_LIMITER` / `NOTIFICATIONS_REDIS_RATE_LIMITER`)
built on the same primitives. If you've configured
`titan-ratelimit` already and want notifications to use the same
limiter, override the notifications token:

```typescript
NotificationsModule.forRoot({
  rateLimiter: yourCustomLimiter,    // implements IRateLimiter
})
```

The default is independent per-channel limits with their own
counters — fine for most setups.

## Anti-patterns

- **`@RateLimit` per method without auth context.** Without a per-
  user key, all callers share one bucket. Use a key template that
  includes the user / IP.
- **Tiny window + high limit.** A 1-second window with limit 1_000
  is essentially "no limit". Pick windows that match the workload
  (per-minute or per-hour for human-driven traffic).
- **Redis storage for trivial in-pod limiting.** Each `consume` is
  a Redis call. For per-pod limits, `'memory'` is fine.
- **Trusting `req.ip` without a trusted-proxy chain.** Spoofed
  headers bypass per-IP limits. Configure a trusted proxy list at
  the gateway and pass the verified IP.
- **`queueEnabled: true` on unauthenticated paths.** Queue saturation
  is a DoS amplifier — rejections cost nothing, queue slots cost
  memory and timeouts.
- **Ignoring `retryAfter` on the client.** Clients that retry
  immediately get rejected again immediately. Always respect the
  `retryAfter` value (or the HTTP `Retry-After` header).
- **Storing PII / unbounded values in `keyPrefix`.** The key prefix
  is constant per process. Don't templatise it with per-user data.

## See also

- [Resilience / overview](/docs/titan/resilience/overview) — pair with
  retry/backoff on the caller side
- [`titan-redis`](/docs/titan/modules/redis) — backs the Redis storage
- [`titan-notifications`](/docs/titan/modules/notifications) — has its own
  channel-level rate limiter built on this module's primitives
- [Tokens & RPC reference](/docs/titan/modules/tokens-reference#titan-ratelimit) — token inventory
- [Module map](/docs/titan/modules/module-map#where-redis-sits) — recommended Redis DB split

---

---
id: "titan/modules/redis"
title: "titan-redis"
---

# titan-redis

<span class="module-badge">official · @omnitron-dev/titan-redis · stable</span>

Redis connection management with clustering, Sentinel, TLS, multiple
named instances, pluggable Lua scripts, and the underlying foundation
for `titan-cache` (L2), `titan-lock`, `titan-discovery`,
`titan-ratelimit`, and `titan-notifications`.

```bash
pnpm add @omnitron-dev/titan-redis
```

## Quickstart

### Single client

```typescript
import { RedisModule } from '@omnitron-dev/titan-redis';

@Module({
  imports: [
    RedisModule.forRoot({
      config: { host: 'localhost', port: 6379, db: 0 },
    }),
  ],
})
class AppModule {}
```

### Multiple named clients

```typescript
RedisModule.forRoot({
  clients: [
    { namespace: 'main',   host: 'redis-main',   port: 6379, db: 0 },
    { namespace: 'cache',  host: 'redis-cache',  port: 6379, db: 1 },
    { namespace: 'queue',  host: 'redis-queue',  port: 6379, db: 2 },
  ],
  commonOptions: { maxRetriesPerRequest: 3 },
})
```

### Cluster

```typescript
RedisModule.forRoot({
  config: {
    cluster: {
      nodes: [
        { host: 'redis-1', port: 6379 },
        { host: 'redis-2', port: 6379 },
        { host: 'redis-3', port: 6379 },
      ],
      maxRedirections: 16,
      retryDelayOnClusterDown: 100,
      scaleReads: 'replica',
    },
  },
})
```

### Sentinel

```typescript
RedisModule.forRoot({
  config: {
    sentinels:    [{ host: 'sentinel-1', port: 26379 }, { host: 'sentinel-2', port: 26379 }],
    sentinelName: 'mymaster',
    password:     env.REDIS_PASSWORD,
  },
})
```

### TLS

```typescript
RedisModule.forRoot({
  config: {
    host: 'redis-tls.internal',
    port: 6380,
    tls:  { rejectUnauthorized: true, ca: [fs.readFileSync('./ca.pem', 'utf8')] },
  },
})
```

### Async configuration

```typescript
RedisModule.forRootAsync({
  useFactory: (config: ConfigService) => ({
    config: {
      host: config.get('redis.host'),
      port: config.get('redis.port'),
    },
  }),
  inject: [ConfigService],
})
```

## `RedisModuleOptions`

| Option              | Type                                                                                                  |
| ------------------- | ----------------------------------------------------------------------------------------------------- |
| `config`            | `IRedisClientOptions` — single-client config                                                          |
| `clients`           | `IRedisClientOptions[]` — multiple named clients                                                      |
| `commonOptions`     | `Partial<IRedisClientOptions>` — applied to every client                                              |
| `closeClient`       | `boolean` — close during module shutdown (default: `true`)                                            |
| `shutdownTimeout`   | `number` — QUIT acknowledgement budget in milliseconds (default: `5000`; `0` disconnects immediately) |
| `readyLog`          | `boolean` — log when each client emits 'ready'                                                        |
| `errorLog`          | `boolean` — log on connection errors                                                                  |
| `logger`            | `ILogger` — override default logger                                                                   |
| `onError`           | `(error, client) => void` — global error callback                                                     |
| `onClientCreated`   | `(client) => void`                                                                                    |
| `onClientDestroyed` | `(namespace) => void`                                                                                 |
| `healthCheck`       | `{ enabled?, timeout?, interval? }`                                                                   |
| `scripts`           | `Array<{ name, path?, content? }>` — Lua scripts to preload                                           |
| `pool`              | `{ min?, max?, acquireTimeoutMillis?, idleTimeoutMillis? }`                                           |
| `isGlobal`          | `boolean`                                                                                             |

## `IRedisClientOptions`

| Option                 | Type                                                | Default       |
| ---------------------- | --------------------------------------------------- | ------------- |
| `namespace`            | `string`                                            | `'default'`   |
| `host`                 | `string`                                            | `'localhost'` |
| `port`                 | `number`                                            | `6379`        |
| `db`                   | `number` — DB index                                 | `0`           |
| `path`                 | `string` — Unix socket path (alt. to host/port)     | —             |
| `password`             | `string`                                            | —             |
| `username`             | `string` — Redis 6+ ACL username                    | —             |
| `name`                 | `string` — Connection name visible in `CLIENT LIST` | —             |
| `lazyConnect`          | `boolean` — defer connect to first command          | —             |
| `enableReadyCheck`     | `boolean` — wait for 'ready' before returning       | `true`        |
| `enableOfflineQueue`   | `boolean` — buffer commands during disconnect       | `true`        |
| `connectTimeout`       | `number` (ms)                                       | `10_000`      |
| `commandTimeout`       | `number` (ms)                                       | —             |
| `keepAlive`            | `number` (ms)                                       | —             |
| `maxRetriesPerRequest` | `number`                                            | `20`          |
| `retryStrategy`        | `(times) => number \| null`                         | exponential   |
| `cluster`              | `IRedisClusterOptions`                              | —             |
| `sentinels`            | `Array<{ host, port }>`                             | —             |
| `sentinelName`         | `string`                                            | —             |
| `tls`                  | `IRedisTlsOptions`                                  | —             |

## `RedisService` — the high-level API

The injected default service for one-line ops without managing clients.

```typescript
import { RedisService, REDIS_MANAGER } from '@omnitron-dev/titan-redis';

@Service({ name: 'cart' })
class CartService {
  constructor(private readonly redis: RedisService) {}

  @Public()
  async add(userId: string, item: Item) {
    await this.redis.hset(`cart:${userId}`, item.id, JSON.stringify(item));
  }
}
```

### Connection helpers

| Method                                       | Purpose                             |
| -------------------------------------------- | ----------------------------------- |
| `getClient(namespace?)`                      | Get client by namespace             |
| `getOrThrow(namespace?)`                     | Get client or throw `NotFoundError` |
| `getOrNil(namespace?)`                       | Get client or return `null`         |
| `ping(namespace?)`                           | Health check; returns `boolean`     |
| `isReady(namespace?)`                        | Synchronous ready check             |
| `pipeline(namespace?)` / `multi(namespace?)` | Batch pipeline / transaction        |
| `createSubscriber(namespace?)`               | Dup client for pub/sub              |
| `publish(channel, message, namespace?)`      | Publish to channel                  |

### Strings, counters, expiry

```typescript
await this.redis.set('user:42:tier', 'premium', 3600);   // 3600s TTL
await this.redis.setnx('lock:job', 'taken');             // 1 if set, 0 if exists
await this.redis.incr('orders:today');
await this.redis.expire('cart:42', 600);
const ttl = await this.redis.ttl('cart:42');             // -1 no expiry, -2 not exists
```

### Hashes / sets / lists / sorted sets / streams

`RedisService` proxies every common ioredis method directly: `hget`,
`hset`, `hgetall`, `hdel`, `hexists`, `hincrby`, `hlen`, `hkeys`,
`hvals`, `hmset`, `hmget`; `sadd`, `srem`, `smembers`, `sismember`;
`lpush`, `rpush`, `lpop`, `rpop`, `lrange`, `llen`, `ltrim`; `zadd`,
`zrem`, `zrange`, `zrevrange`, `zcard`, `zscore`, `zincrby`,
`zrangebyscore`; `xadd`, `xgroup`, `xreadgroup`, `xack`, `xlen`,
`xrange`, `xread`, `xtrim`. Each accepts an optional trailing
`namespace?` argument to target a non-default client.

### Lua scripts

```typescript
const sha = await this.redis.loadScript(
  'limit',
  'return redis.call("incr", KEYS[1]) <= tonumber(ARGV[1]) and 1 or 0',
);
const allowed = await this.redis.runScript<number>('limit', ['rate:42'], [100]);
```

Scripts declared in `RedisModule.forRoot({ scripts: [...] })` are
preloaded at startup; `runScript()` re-loads automatically if Redis
evicts the SHA (NOSCRIPT error).

## `RedisManager` — low-level API

Injected via `@Inject(REDIS_MANAGER)` for cases that need the raw
ioredis client (advanced commands not in `RedisService`'s typed
surface, custom connection management, dynamic client creation).

```typescript
@Service({ name: 'analytics' })
class AnalyticsService {
  constructor(@Inject(REDIS_MANAGER) private readonly redis: RedisManager) {}

  @Public()
  async raw() {
    const client = this.redis.getClient('analytics');
    return client.call('OBJECT', 'ENCODING', 'user:42');  // arbitrary command
  }
}
```

| Method                                             | Purpose                               |
| -------------------------------------------------- | ------------------------------------- |
| `getClient(namespace?)`                            | Public `IRedisClient`                 |
| `getInternalClient(namespace?)`                    | Internal — full ioredis surface       |
| `getClients()`                                     | All registered clients                |
| `hasClient(namespace?)`                            | Existence check                       |
| `createClient(options)`                            | Dynamically create a new named client |
| `destroyClient(namespace)`                         | Tear down a client                    |
| `isHealthy(namespace?)`                            | Health check                          |
| `healthCheck()`                                    | All clients' health + latency         |
| `runScript<T>(scriptName, keys, args, namespace?)` | Execute named Lua script              |

## Decorators

### `@InjectRedis(namespace?)` — parameter

```typescript
@Service({ name: 'cart' })
class CartService {
  constructor(@InjectRedis() private readonly redis: IRedisClient) {}
}

@Service({ name: 'sessions' })
class SessionsService {
  constructor(@InjectRedis('sessions') private readonly redis: IRedisClient) {}
}
```

### `@InjectRedisManager()` — parameter

Inject the `RedisManager` directly.

### `@RedisCache(options?)` — method

```typescript
@Public()
@RedisCache({ ttl: 60, namespace: 'cache', key: (id) => `user:${id}` })
async findById(id: string) { /* … */ }
```

Options: `ttl` (seconds), `namespace`, `key` (string or function),
`keyFn` (alias), `condition` (skip if false), `refresh` (always
re-compute, bypass read).

### `@RedisLock(options?)` — method

Lightweight inline lock for simple cases. For full distributed lock
semantics use [`titan-lock`](/docs/titan/modules/lock).

```typescript
@Public()
@RedisLock({ ttl: 10, key: (id) => `invoice:${id}`, retries: 5 })
async charge(id: string) { /* … */ }
```

### `@RedisRateLimit(options)` — method

```typescript
@Public()
@RedisRateLimit({ points: 100, duration: 60_000, keyPrefix: 'rl' })
async search(q: string) { /* … */ }
```

For tier-based / sliding-window strategies use
[`titan-ratelimit`](/docs/titan/modules/ratelimit).

## Tokens

| Token                             | Purpose                                  |
| --------------------------------- | ---------------------------------------- |
| `REDIS_MANAGER`                   | The `RedisManager` instance              |
| `REDIS_MODULE_OPTIONS`            | Resolved options bundle                  |
| `REDIS_DEFAULT_NAMESPACE`         | Default namespace constant (`'default'`) |
| `getRedisClientToken(namespace?)` | Token for a named client                 |

## Lifecycle

The module's provider factory calls `RedisManager.init()` before consumers
receive the manager. It initializes non-lazy clients and preloads declared Lua
scripts. `TitanRedisModule.onStop()` calls `destroy()` after dependent modules
have stopped. Applications using the module do not need a second shutdown hook.
The manager also exposes `onModuleInit()` and `onModuleDestroy()` for hosts that
explicitly invoke those compatibility methods.

Shutdown sends `QUIT` only to a ready client and waits up to `shutdownTimeout`
for its acknowledgement. A client that is lazy, connecting or reconnecting is
disconnected immediately, so stopping an application cannot start a new Redis
connection. The deadline applies independently to clients closed in parallel.
A timeout or failed acknowledgement is logged with its namespace, then the
socket and reconnect loop are closed. This bounds shutdown; it does not prove
that an unacknowledged command reached the server.

`destroyClient(namespace)` uses the same close path and removes the registration
even when closing fails; the caller receives the failure. `destroy()` attempts
to close every owned client and logs individual failures. With
`closeClient: false`, the caller retains responsibility for connections and must
close them explicitly. Set `shutdownTimeout: 0` only when immediate disconnect
is intended; otherwise use a finite non-negative integer up to `2147483647`.

## DB-index isolation pattern

When you co-locate multiple subsystems on the same Redis instance,
isolate them by DB index per named client instead of sharing one
index. A typical layout:

| DB | Example use                  |
| -- | ---------------------------- |
| 0  | Default application data     |
| 1  | Cache (L2 for `titan-cache`) |
| 2  | Queue / messaging            |
| 3  | Sessions                     |
| 4  | Rate-limit buckets           |

Use named clients with an explicit `db` per namespace rather than
swapping `SELECT` at runtime — runtime `SELECT` is per-connection
and hard to reason about under connection-pool churn.

## Anti-patterns

- **Cross-slot ops in cluster `multi()`.** Cluster mode forbids
  multi-slot transactions. Pin keys to one slot via `{hashtag}` or
  use Lua scripts that touch one key.
- **One client for everything.** Pub/sub blocks command pipelines on
  the subscribing client. Use `createSubscriber()` for pub/sub and
  keep the default client for commands.
- **Forgetting `password` in Sentinel/cluster.** Sentinels typically
  require their own password in addition to the data-plane password.
- **Hot-loop without pipeline.** N round-trips for N small writes
  costs N×RTT. Use `pipeline()` or `multi()` for batches.
- **Using `@RedisLock` for critical multi-step work.** It's a
  convenience wrapper. For Lua-safe ownership and TTL extension, use
  [`titan-lock`](/docs/titan/modules/lock).

## See also

- [`titan-cache`](/docs/titan/modules/cache) — L2 backing via this module
- [`titan-lock`](/docs/titan/modules/lock) — robust distributed locks
- [`titan-discovery`](/docs/titan/modules/discovery), [`titan-ratelimit`](/docs/titan/modules/ratelimit),
  [`titan-notifications`](/docs/titan/modules/notifications) — all build on this module

---

---
id: "titan/modules/rls"
title: "RLS context guards"
description: "The shipped context guard decorators, their bypass semantics and the boundary with database policies."
---

# RLS context guards

`@omnitron-dev/titan/module/rls` is a public entry point for guard
decorators. Despite its path, it does not export an `RlsModule` to register.
The decorators read the current `@kysera/rls` context directly.

```typescript
import { RequireRlsContext } from '@omnitron-dev/titan/module/rls';

class TenantReports {
  @RequireRlsContext({ requireUser: true, requireTenant: true })
  async list() {
    // Query a repository whose row policies are configured separately.
    return [];
  }
}
```

Establish a trusted context before invoking the method. A context supplied
from unverified request data is not an authorization boundary.

## Guards and behaviour

| Export                                 | Check                                                                                      |
| -------------------------------------- | ------------------------------------------------------------------------------------------ |
| `RequireRlsContext(options?)`          | Context exists, plus the selected user, tenant, role and permission requirements           |
| `RequireUser()`                        | Context has a user                                                                         |
| `RequireTenant()`                      | Context has a tenant                                                                       |
| `RequireRole(role)`                    | Context has a user and the named role                                                      |
| `RequirePermission(permission)`        | Context has a user and the named permission                                                |
| `RequireAdmin()`                       | Context is explicitly marked `auth.isSystem`; an `admin` role string alone is insufficient |
| `RlsProtected(message?)`               | Any RLS context is present                                                                 |
| `getRlsGuardRequirements(target, key)` | Reads declared guard metadata                                                              |
| `RlsGuardError`                        | Error raised when a guard requirement is not met                                           |
| `RLS_GUARD_METADATA_KEY`               | Metadata key containing the declared requirements                                          |

`RequireRlsContextOptions` accepts `requireUser`, `requireTenant`,
`roles`, `permissions` and `message`. Within `roles` and `permissions`,
the implementation accepts **any** matching entry. Supplying both groups
requires each group to pass. A trusted context with `auth.isSystem`
**bypasses these checks**.

The wrapper is asynchronous. Await guarded methods and test both accepted
and rejected calls.

## Three separate boundaries

1. [RPC authentication](/docs/titan/netron/authentication) establishes identity.
2. A context guard controls whether a method may execute in that context.
3. [Database RLS](/docs/titan/modules/database) constrains the rows read or written.

A decorator alone does not install database row policies, start a
transaction or propagate identity across a remote call. Test tenant
isolation at the repository boundary, including missing context and a
cross-tenant request.

The broader [RLS bridge](/docs/auth/rls-bridge) is a design proposal
for composition. Its permission-version and activity-tracking extensions
are separate from these shipped guards.

---

---
id: "titan/modules/scheduler"
title: "titan-scheduler"
---

# titan-scheduler

<span class="module-badge">official · @omnitron-dev/titan-scheduler · stable</span>

Cron, interval, and timeout job scheduling with auto-discovery of
decorated methods, pluggable persistence (memory / Redis / database),
optional distributed coordination, per-job retry/backoff, concurrency
limits, listener pattern, and lifecycle-aware graceful shutdown.

```bash
pnpm add @omnitron-dev/titan-scheduler
```

## When you need it

- **Periodic background work.** Cron-style cleanup, hourly reports,
  nightly aggregations.
- **Interval polling.** Poll an external system every N seconds.
- **Delayed actions.** Run something exactly once, N ms after
  registration.
- **Distributed jobs.** Multi-pod fleet where exactly-one execution
  is required (pair with [`titan-lock`](/docs/titan/modules/lock) or built-in
  distributed coordination).

## Quickstart

```typescript
import { SchedulerModule } from '@omnitron-dev/titan-scheduler';

@Module({
  imports: [
    SchedulerModule.forRoot({
      enabled:         true,
      timezone:        'UTC',
      persistence:     { enabled: true },           // in-memory default
      metrics:         { enabled: true },
      maxConcurrent:   10,
      shutdownTimeout: 30_000,
    }),
  ],
  providers: [TasksService],
})
class AppModule {}
```

### Decorate methods

```typescript
import {
  Schedulable,
  Cron, CronExpression,
  Interval,
  Timeout,
} from '@omnitron-dev/titan-scheduler';

@Injectable()
@Schedulable()
class TasksService {
  @Cron(CronExpression.EVERY_HOUR, { timezone: 'UTC' })
  async hourlySync() { /* … */ }

  @Interval(30_000, { name: 'queue-poll' })
  async pollQueue() { /* … */ }

  @Timeout(5_000)
  async runOnceAfterBoot() { /* … */ }
}
```

The discovery service finds decorated methods at `onInit` and
registers them with the scheduler.

### Async configuration

```typescript
SchedulerModule.forRootAsync({
  imports: [ConfigModule],
  useFactory: (config: ConfigService) => ({
    enabled:       config.get('scheduler.enabled'),
    persistence:   { enabled: true, provider: new RedisPersistenceProvider(redis) },
    maxConcurrent: config.get('scheduler.concurrency'),
  }),
  inject: [ConfigService],
})
```

For distributed (multi-node) scheduling, also supply a `lockProvider` —
see [Distributed coordination](#distributed-coordination) below.

## `ISchedulerModuleOptions`

| Option                                                 | Type                                                                   | Default  |
| ------------------------------------------------------ | ---------------------------------------------------------------------- | -------- |
| `enabled`                                              | `boolean`                                                              | `true`   |
| `timezone`                                             | `string`                                                               | —        |
| `persistence`                                          | `{ enabled, provider?, options? }`                                     | —        |
| `metrics`                                              | `{ enabled, interval?, includeDetails? }`                              | —        |
| `distributed`                                          | `{ enabled, lockProvider?: 'redis' \| 'database', lockTTL?, nodeId? }` | —        |
| `retry`                                                | `IRetryOptions` — global retry policy                                  | —        |
| `maxConcurrent`                                        | `number`                                                               | —        |
| `queueSize`                                            | `number`                                                               | —        |
| `queueTimeout`                                         | `number` (ms)                                                          | —        |
| `debug`                                                | `boolean`                                                              | —        |
| `shutdownTimeout`                                      | `number` (ms) — grace window                                           | `30_000` |
| `healthCheck`                                          | `{ enabled, path? }`                                                   | —        |
| `listeners`                                            | `IJobListener[]` — global listeners                                    | —        |
| `lockProvider`                                         | `ISchedulerLockProvider` — required when `distributed.enabled` (SC-1)  | —        |
| `executor` / `persistenceProvider` / `metricsProvider` | `InjectionToken` — override the built-in providers                     | —        |

During shutdown the scheduler stops issuing ticks and signals in-flight jobs.
Queued jobs and pending retries settle without starting a new handler.
Handlers that need to unwind receive the remainder of `shutdownTimeout`.
Choose zero for immediate return. A handler that ignores cancellation may
still be executing when that deadline expires. Both `maxConcurrent` and
`queueSize` bound admission; a full queue rejects new work.

## Decorators

### `@Cron(expression, options?)`

```typescript
@Cron('0 3 * * *', { timezone: 'UTC', immediate: false })
async nightly() { /* runs at 03:00 UTC daily */ }

@Cron(CronExpression.EVERY_5_MINUTES)
async refresh() { /* … */ }
```

Options:

| Field                               | Type                          |
| ----------------------------------- | ----------------------------- |
| `timezone?`                         | `string`                      |
| `utcOffset?`                        | `number`                      |
| `startTime?`                        | `Date \| string`              |
| `endTime?`                          | `Date \| string`              |
| `immediate?`                        | `boolean` — run once at start |
| (and `IBaseJobOptions` — see below) |                               |

### `@Interval(ms, options?)`

```typescript
@Interval(60_000, { immediate: true })
async heartbeat() { /* runs immediately then every 60s */ }
```

### `@Timeout(ms, options?)`

```typescript
@Timeout(10_000)
async runOnceAfterBoot() { /* fires 10s after registration */ }
```

### `@Schedulable()`

Class-level marker; optional but recommended for clarity. The
discovery service finds `@Cron`/`@Interval`/`@Timeout` methods on
any `@Injectable()` class.

### `IBaseJobOptions` — shared across all job types

| Field             | Type                                                        |
| ----------------- | ----------------------------------------------------------- |
| `name?`           | `string` — explicit job name                                |
| `priority?`       | `JobPriority` — `CRITICAL \| HIGH \| NORMAL \| LOW \| IDLE` |
| `retry?`          | `{ maxAttempts?, delay?, backoff?, maxDelay?, retryIf? }`   |
| `onError?`        | `(error: Error) => void \| Promise<void>`                   |
| `onSuccess?`      | `(result: any) => void \| Promise<void>`                    |
| `disabled?`       | `boolean` — don't auto-start                                |
| `timeout?`        | `number` (ms) — max execution time                          |
| `persist?`        | `boolean` — save to persistence provider                    |
| `metadata?`       | `Record<string, any>`                                       |
| `preventOverlap?` | `boolean` — reject concurrent executions of the same job    |

### A job that fails to register stops the boot

`disabled: true` is the only way a job is silently absent, and it is silent
because you asked for it.

Every other registration failure now throws at startup rather than dropping the
job. The realistic one is a name collision: two definitions claiming the same
name make `registerJob` throw a conflict — `Job with this name already
exists: <name>` — and the loser used to be
discarded with nothing logged — the application reported a successful start and
the first evidence was a nightly task that had not run. Give jobs distinct
`name`s, or leave `name` off and let it be derived.

This matches what the module already does elsewhere with configuration that
would otherwise be silently wrong: `onStart` refuses distributed mode without a
lock provider rather than let every instance fire every job.

## `CronExpression` enum

```typescript
import { CronExpression } from '@omnitron-dev/titan-scheduler';

CronExpression.EVERY_SECOND          // '* * * * * *'
CronExpression.EVERY_5_SECONDS       // '*/5 * * * * *'
CronExpression.EVERY_30_SECONDS      // '*/30 * * * * *'
CronExpression.EVERY_MINUTE          // '*/1 * * * *'
CronExpression.EVERY_5_MINUTES       // '*/5 * * * *'
CronExpression.EVERY_HOUR            // '0 * * * *'
CronExpression.EVERY_DAY_AT_1AM      // '0 1 * * *'
// …through EVERY_YEAR = '0 0 1 1 *'
```

## `SchedulerService`

Most jobs are registered declaratively with the decorators above. To
register a job **dynamically** at runtime, inject the service and use
`addCronJob` / `addInterval` / `addTimeout` — each takes a `name`, the
pattern, a handler, and optional job options:

```typescript
import { SchedulerService, SCHEDULER_SERVICE_TOKEN } from '@omnitron-dev/titan-scheduler';

@Service({ name: 'jobs' })
class JobsService {
  constructor(@Inject(SCHEDULER_SERVICE_TOKEN) private readonly scheduler: SchedulerService) {}

  @Public()
  async runAfter(ms: number, payload: unknown) {
    this.scheduler.addTimeout(
      `once:${crypto.randomUUID()}`,
      ms,
      async () => { /* … fires once after `ms` … */ },
      { metadata: { payload } },
    );
  }
}
```

| Method                                            | Purpose                                           |
| ------------------------------------------------- | ------------------------------------------------- |
| `addCronJob(name, expression, handler, options?)` | Register + schedule a cron job dynamically        |
| `addInterval(name, ms, handler, options?)`        | Register + schedule an interval job               |
| `addTimeout(name, ms, handler, options?)`         | Register + schedule a one-shot timeout job        |
| `triggerJob(name)`                                | Run a job immediately (bypasses distributed lock) |
| `stopJob(name)` / `startJob(name)`                | Stop (pause) / re-schedule a job by name          |
| `deleteJob(name)`                                 | Stop + remove a job (and its persisted record)    |
| `getJob(name)`                                    | Look up a single job by name                      |
| `getAllJobs()`                                    | All registered jobs                               |
| `findJobs(filter)`                                | Filter jobs (`IJobFilterOptions`)                 |
| `getMetrics()`                                    | Scheduler metrics (`ISchedulerMetrics \| null`)   |
| `isRunning()`                                     | Whether the scheduler has started                 |

> Jobs are keyed by **name**, not an opaque id. Lifecycle (`onInit` /
> `onStart` / `onStop`) is driven by the framework — you rarely call
> these methods yourself for decorator-defined jobs.

## Persistence providers

| Provider class                | Persistence                                            |
| ----------------------------- | ------------------------------------------------------ |
| `InMemoryPersistenceProvider` | Default — keeps last 100 executions per job in memory  |
| `RedisPersistenceProvider`    | Redis-backed job + execution storage; survives restart |
| `DatabasePersistenceProvider` | SQL-backed; durable, transactional history             |

Implement `IPersistenceProvider`:

```typescript
interface IPersistenceProvider {
  saveJob(job: IScheduledJob): Promise<void>;
  loadJob(id: string): Promise<IScheduledJob | null>;
  loadAllJobs(): Promise<IScheduledJob[]>;
  deleteJob(id: string): Promise<void>;
  saveExecutionResult(result: IJobExecutionResult): Promise<void>;
  loadExecutionHistory(jobId: string, limit?: number): Promise<IJobExecutionResult[]>;
  clear(): Promise<void>;
}
```

## Distributed coordination

For multi-pod deployments, two paths:

### A) Module-level distributed config (SC-1)

When `distributed.enabled` is `true`, before each scheduled fire the
scheduler acquires a per-fire-window lock so a given fire runs on
**exactly one** node; the other nodes skip it. This requires a
**lock provider** — an object implementing `ISchedulerLockProvider`
(`acquireLock(key, ttlMs)` / `releaseLock(key, lockId)`). Pass it as
the `lockProvider` module option (or register it at
`SCHEDULER_LOCK_TOKEN`). titan-lock's `DistributedLockService` is
structurally compatible and can be used directly.

> **Fail-fast.** If `distributed.enabled` is true but no lock provider
> is wired, the scheduler throws on start rather than silently running
> every job on every node.

```typescript
import { LOCK_SERVICE_TOKEN } from '@omnitron-dev/titan-lock';

SchedulerModule.forRootAsync({
  imports: [TitanLockModule.forRoot({ /* … */ })],
  useFactory: (locks) => ({
    distributed: {
      enabled: true,
      lockTTL: 60_000,
      nodeId:  process.env.HOSTNAME,
    },
    lockProvider: locks,    // ISchedulerLockProvider (DistributedLockService is compatible)
  }),
  inject: [LOCK_SERVICE_TOKEN],
})
```

The `distributed.lockProvider?: 'redis' | 'database'` field is only an
advisory label describing the kind of backing store; the actual
coordination is done by the `lockProvider` instance you supply. The
scheduler holds each per-fire lock for its full `lockTTL` and does not
release it early (so a clock-skewed-late node can't re-run the same
fire).

### B) Per-job `@WithDistributedLock`

```typescript
import { WithDistributedLock } from '@omnitron-dev/titan-lock';

@Cron(CronExpression.EVERY_HOUR)
@WithDistributedLock('jobs:hourly-report', 10 * 60_000)
async hourlyReport() { /* exactly one pod runs this per hour */ }
```

Per-job locking gives you finer control (only the decorated jobs are
coordinated, and it releases the lock as soon as the method returns);
module-level coordination (A) covers **every** job at once and holds
each fire's lock for the full `lockTTL`.

## Listeners

```typescript
const auditListener: IJobListener = {
  onJobStart:    async (job, ctx) => { /* … */ },
  onJobComplete: async (job, result) => { /* … */ },
  onJobError:    async (job, error, ctx) => { /* … */ },
  onJobRetry:    async (job, attempt, error) => { /* … */ },
  onJobCancelled: async (job, reason) => { /* … */ },
};

SchedulerModule.forRoot({ listeners: [auditListener] });
```

Use for unified observability (metrics, alerting, audit trails)
without coupling each job's body to telemetry concerns.

## Status / priority enums

```typescript
// JobStatus is a const-object string union:
//   'pending' | 'running' | 'completed' | 'failed' | 'cancelled' | 'paused' | 'retrying'
const JobStatus = {
  PENDING: 'pending', RUNNING: 'running', COMPLETED: 'completed',
  FAILED: 'failed', CANCELLED: 'cancelled', PAUSED: 'paused', RETRYING: 'retrying',
} as const;

enum JobPriority {
  CRITICAL = 0,
  HIGH     = 1,
  NORMAL   = 2,
  LOW      = 3,
  IDLE     = 4,
}
```

## Lifecycle

`SchedulerService` implements:

- `async onInit()` — load persisted jobs, run discovery on decorated
  methods, auto-start if `enabled: true`.
- `async onStart()` — activate all registered jobs, emit
  `SCHEDULER_STARTED`.
- `async onStop()` — stop intervals / cron, wait up to
  `shutdownTimeout` for running jobs to finish, persist state.

```mermaid
stateDiagram-v2
  [*] --> PENDING: register / addCronJob()
  PENDING --> RUNNING: trigger fires
  RUNNING --> COMPLETED: success
  RUNNING --> FAILED: error & no retries left
  RUNNING --> RETRYING: error & retries remain
  RETRYING --> RUNNING: backoff elapsed
  RUNNING --> CANCELLED: cancel
  PENDING --> PAUSED: stopJob()
  PAUSED --> PENDING: startJob()
  COMPLETED --> PENDING: next cron tick
  FAILED --> PENDING: startJob()
```

## Tokens

| Token                         |
| ----------------------------- |
| `SCHEDULER_SERVICE_TOKEN`     |
| `SCHEDULER_CONFIG_TOKEN`      |
| `SCHEDULER_REGISTRY_TOKEN`    |
| `SCHEDULER_EXECUTOR_TOKEN`    |
| `SCHEDULER_PERSISTENCE_TOKEN` |
| `SCHEDULER_METRICS_TOKEN`     |
| `SCHEDULER_DISCOVERY_TOKEN`   |
| `SCHEDULER_LISTENERS_TOKEN`   |
| `SCHEDULER_LOCK_TOKEN`        |

## Anti-patterns

- **Long-running jobs with short intervals.** `@Interval(1_000)`
  with a 5s body queues up. Set `preventOverlap: true` or shorten
  the body.
- **Cron-style polling for low-latency triggers.** Cron has ≥1s
  resolution. For sub-second reactions, use `titan-events` or a
  message queue.
- **Forgetting persistence in production.** Without persistence,
  jobs disappear on restart. Use `RedisPersistenceProvider` or
  `DatabasePersistenceProvider`.
- **Distributed coord without lock TTL tuning.** Lock TTL must
  exceed the job's worst-case duration, or another pod picks it up
  mid-execution.

## See also

- [`titan-lock`](/docs/titan/modules/lock) — pair with `@WithDistributedLock` per job
- [`titan-metrics`](/docs/titan/modules/metrics) — picks up scheduler metrics if both modules loaded
- [`titan-health`](/docs/titan/modules/health) — `SchedulerHealthIndicator` exposed

---

---
id: "titan/modules/security-checklist"
title: "Security checklist"
description: "Per-module hardening guidance for production deployments."
---

# Security checklist

A per-module audit checklist focused on what an attacker or
careless operator could exploit. Use it before exposing a Titan
backend publicly, during a pen-test prep, or as compliance
evidence.

The checklist below treats each module as independently hostile —
the box ticks the same whether you're securing a single API pod
or a 50-pod fleet.

## Surface-area overview

| Module                  | External attack surface             | Primary risk                                 |
| ----------------------- | ----------------------------------- | -------------------------------------------- |
| `titan-auth`            | JWT verification endpoint, JWKS URL | Algorithm confusion, key rotation            |
| `titan-cache`           | None directly (L1/L2 reads)         | Sensitive data persistence in L2 (Redis)     |
| `titan-database`        | SQL execution path                  | Injection, missing RLS, leaked DSN           |
| `titan-discovery`       | Registry writes via Redis           | Spoofed registration, registry pollution     |
| `titan-events`          | Event payload deserialization       | PII leakage, untrusted handlers              |
| `titan-health`          | `/healthz/full` JSON (stack info)   | Information disclosure                       |
| `titan-lock`            | Lock keys in Redis                  | Lock pinning, ownership spoofing             |
| `titan-metrics`         | `/metrics` exposition + storage     | Cardinality abuse, label PII                 |
| `titan-notifications`   | Outbound webhooks, channel queue    | Webhook forgery, template injection          |
| `titan-pm`              | Child process spawn parameters      | Argv-leaked secrets, file-descriptor inherit |
| `titan-ratelimit`       | Per-key state                       | IP spoofing without trusted proxy            |
| `titan-redis`           | Direct Redis client                 | Plaintext AUTH, shared keyspaces             |
| `titan-scheduler`       | Stored cron expressions             | Untrusted cron → forced load                 |
| `titan-telemetry-relay` | WAL file, transport messages        | File permissions, unsigned transport         |
| `config` (built-in)     | Config sources                      | Secrets in committed defaults                |
| `logger` (built-in)     | Log sinks                           | PII in structured logs                       |

## Per-module hardening

### `titan-auth`

<span class="module-badge">official · @omnitron-dev/titan-auth</span>

- [ ] **Algorithm pinned explicitly.** Never accept whatever the
  incoming JWT declares; configure `algorithm: 'RS256'` (or
  `'ES256'` / `'HS256'`) and enforce it. JWT spec history is
  littered with algorithm-confusion CVEs.
- [ ] **JWT secret length & source.** HS256 secrets ≥ 32 bytes,
  loaded from secret manager — never from a committed file or
  default value.
- [ ] **JWKS rotation supported.** For RS256/ES256, configure
  `jwksUrl` with a short cache TTL (≤ 1 h) so revoked keys
  propagate.
- [ ] **`verificationKeys` registry for HS256 rotation.** Add new
  keys before issuing tokens with them; remove old keys only after
  all in-flight tokens expire.
- [ ] **Token cache TTL ≤ token lifetime.** Cached verifications
  bypass JWKS fetch; never cache longer than the token's `exp`.
- [ ] **Clock skew bounded.** `clockTolerance` (seconds, or a jose
  string like `'5s'`) defaults to **0** — no skew is allowed at all, so
  a token from a server whose clock runs slightly ahead fails its
  `nbf`/`iat` check and one expiring this instant fails `exp`. Set it
  only as wide as the fleet's clock discipline requires: the window is
  also how long an expired token stays usable.
- [ ] **`audience` and `issuer` enforced.** Both options must be
  populated; otherwise any well-signed token from any service
  authenticates.
- [ ] **`InvalidTokenError` / `TokenExpiredError` not logged with
  the token body.** Log `code` and `kid` only.
- [ ] **No JWT in URLs.** Use the `Authorization: Bearer` header.
  URLs end up in proxy logs, browser history, referer headers.

### `titan-cache`

<span class="module-badge">official · @omnitron-dev/titan-cache</span>

- [ ] **Don't cache sensitive payloads in L2.** L2 is Redis;
  Redis persistence means caches survive process restart and may
  be backed up. Use L1-only for PII / auth state.
- [ ] **Key namespacing per tenant.** `users:{tenantId}:{userId}` —
  never a bare `users:{userId}` if you have multi-tenant data.
- [ ] **TTL always set.** Default to `defaultTTL`; unbounded keys
  drift forever.
- [ ] **Cache stampede protection.** Use `getOrSet` (singleflight)
  not `get` + manual `set` on miss.
- [ ] **No cache-key from user input without hashing/length cap.**
  Multi-MB keys are a DOS vector against the cache itself.

### `titan-database`

<span class="module-badge">official · @omnitron-dev/titan-database</span>

- [ ] **Parameterised queries only.** Kysely query builder is
  parameterised by default. If you reach for `sql.raw()`,
  whitelist inputs.
- [ ] **RLS enabled on multi-tenant tables.** Use
  `defineRLSSchema` + `rlsPlugin` — don't rely on application-layer
  WHERE clauses to enforce isolation.
- [ ] **`validateColumnNames` on dynamic queries.** When column
  names come from a request, use
  [`validateColumnNames`](/docs/titan/modules/database) to block injection.
- [ ] **DSN from secret manager.** `connectionString` should
  resolve via `ConfigService` from a secrets source, not a literal
  in code.
- [ ] **Least-privileged DB user.** App connects as a role with
  only the privileges it needs; migrations run as a separate role.
- [ ] **`enableDatabaseIndicator: true`** on `titan-health` so the
  probe surfaces a dead DB before the load balancer keeps routing.
- [ ] **Backup encryption.** Out of scope here but: rotated
  encryption keys, off-site copies, restore drills.

### `titan-discovery`

<span class="module-badge">official · @omnitron-dev/titan-discovery</span>

- [ ] **Redis ACL on the discovery prefix.** The pod's Redis user
  has `HSET`/`EXPIRE`/`PUBLISH` on `titan:discovery:*` only — no
  full-keyspace access.
- [ ] **Network segmentation.** Discovery's Redis is
  cluster-internal; never reachable from the public network.
- [ ] **Heartbeat TTL ≥ 3 × interval.** Premature ejection from
  the registry causes thundering reconnects when a pod is slow,
  not dead.
- [ ] **`clientMode: true` for read-only consumers.** Any pod that
  doesn't host services shouldn't be able to register fake nodes.
- [ ] **Don't expose `discovery@…` over public RPC.** Internal
  topology is sensitive information.

### `titan-events`

<span class="module-badge">official · @omnitron-dev/titan-events</span>

- [ ] **Schema validation on every event channel.** Untyped
  payloads from one service to another are a wide attack surface.
- [ ] **No PII in events.** Events are often logged or persisted
  in history — assume any payload may end up in a SIEM.
- [ ] **Bounded handler concurrency.** A slow handler shouldn't
  back-pressure the entire bus — use the per-handler concurrency
  option.
- [ ] **History TTL bounded.** `enableHistory: true` with no TTL
  is an unbounded memory growth path.

### `titan-health`

<span class="module-badge">official · @omnitron-dev/titan-health</span>

- [ ] **`/healthz` and `/readyz` exposed; full JSON gated.** The
  detailed `check()` output reveals component names, error
  messages, and the topology — operator-only.
- [ ] **`enableRpcService: false`** if you serve probes via HTTP
  and don't want RPC inspection.
- [ ] **Indicator timeout configured.** A hung indicator blocks
  the whole probe — set `timeout` ≤ probe period (default 5 s).
- [ ] **No secrets in indicator messages.** Don't return the DSN
  in a "database unhealthy" message.

### `titan-lock`

<span class="module-badge">official · @omnitron-dev/titan-lock</span>

- [ ] **TTL always set.** A crashed holder shouldn't pin the lock
  forever. Set TTL to the longest credible work time + margin.
- [ ] **Ownership UUID enforced.** `acquireLock(key, ttlMs)` returns
  a generated `lockId` (UUID); pass it back to `releaseLock(key,
  lockId)` / `extendLock(key, lockId, ttlMs)`. The included Lua
  scripts verify the `lockId` on release. Never release with
  `DEL key` directly. Prefer `withLock(key, fn)` so release is
  automatic.
- [ ] **Extend, don't reset.** Use `extendLock(key, lockId, ttlMs)`
  to push the TTL out periodically while still holding; don't
  release-and-reacquire, which is racy.
- [ ] **Lock granularity.** A single global lock per resource is
  cleaner than a hierarchy; deep lock chains invite deadlocks.

### `titan-metrics`

<span class="module-badge">official · @omnitron-dev/titan-metrics</span>

- [ ] **`/metrics` endpoint authenticated.** Prometheus exposition
  reveals throughput, error rates, internal endpoints — gate
  behind an operator token.
- [ ] **No PII in label values.** `{ userId }` creates a fresh
  series per user — both a cardinality explosion and a PII leak.
  Keep labels to `tier`, `region`, `status`.
- [ ] **`appName` set.** Untagged metrics from multiple services
  collide.
- [ ] **Storage backend access controlled.** Postgres-backed
  metrics give whoever has DB access full historical visibility.

### `titan-notifications`

<span class="module-badge">official · @omnitron-dev/titan-notifications</span>

- [ ] **Webhook signature secret rotated.** Configure
  `webhookConfig.signatureSecret` and verify on the receiver.
- [ ] **Webhook timeout bounded.** Default 5 s; never wait
  indefinitely on a third-party endpoint.
- [ ] **Template engine input escaped.** Templates receive
  attacker-controlled data via `payload.data`; ensure the
  template engine escapes context-appropriately (HTML for email,
  plain for SMS).
- [ ] **Per-recipient rate limit.** Set
  `rateLimiterConfig.defaultLimits` so a compromised account can't
  spam the world via your service.
- [ ] **Preferences honoured before send.** `IPreferenceStore`
  must check unsubscribes / DND windows before delivery, not
  after.
- [ ] **DLQ access controlled.** `subscribeToDLQ()` /
  `requeueFromDLQ()` expose undelivered payloads — operator-only.

### `titan-pm`

<span class="module-badge">official · @omnitron-dev/titan-pm</span>

- [ ] **Secrets via env, not argv.** Process arguments appear in
  `ps`, `/proc/<pid>/cmdline`, and crash dumps. Pass secrets via
  `env: { SECRET: '...' }`.
- [ ] **`stdio` ignored or piped, never `'inherit'` for children
  handling secrets.** Inheritance leaks the child's output into the
  parent's stdout.
- [ ] **Restart limits set.** Without `maxRestarts`, a crash loop
  is a CPU DoS vector against the supervisor.
- [ ] **Health-check command bounded.** There is no
  `healthCheckTimeout`. Per-process monitoring takes
  `health: { interval, timeout, retries }` (timeout default 5000 ms);
  the pool's own check has no timeout field of its own and is bounded
  instead by the service proxy's `requestTimeout` (default 30000 ms),
  which caps every RPC. Set both deliberately — the pool serialises its
  checks, so one slow worker delays the whole sweep by up to
  `requestTimeout`.
- [ ] **No untrusted command paths.** `command` must resolve to
  a known binary — never to a user-supplied path.
- [ ] **Audit `pm@1.0.0` if exposed.** PM control = code
  execution. Admin role minimum, mutual TLS preferred.

### `titan-ratelimit`

<span class="module-badge">official · @omnitron-dev/titan-ratelimit</span>

- [ ] **Per-IP keys behind trusted proxy.** Use the
  proxy-supplied client IP (e.g., `X-Forwarded-For` parsed by a
  trusted-proxies middleware), not the socket peer.
- [ ] **Key includes user / API key when authenticated.** A bare
  IP key punishes shared NATs.
- [ ] **`queueEnabled: false` for unauthenticated endpoints.**
  Queueing turns rejections into work — a DoS amplifier.
- [ ] **Redis-backed for multi-pod.** In-memory limits are
  per-pod; rotating pods lets the attacker reset.
- [ ] **Distinct keyspaces per app.** Configure `keyPrefix` so
  different services don't share counters.

### `titan-redis`

<span class="module-badge">official · @omnitron-dev/titan-redis</span>

- [ ] **TLS to managed Redis.** Configure `tls: {}` in the client
  options; verify the certificate.
- [ ] **`AUTH` password set, rotated.** From secret manager, not
  hardcoded.
- [ ] **DB-per-concern split.** Cache, queues, rate limits, locks,
  discovery — separate `db:` indexes (see the
  [module map](/docs/titan/modules/module-map#where-redis-sits) table).
- [ ] **`enableReadyCheck` and `lazyConnect` aligned with
  reconnect policy.**
- [ ] **Offline queue bounded.** There is no
  `commandQueueMaxLength` — not in this module and not in ioredis.
  What prevents the memory blow-up while Redis is unreachable is
  `enableOfflineQueue: false` (fail commands immediately instead of
  buffering them) and `maxRetriesPerRequest`, both passed through to
  ioredis.
- [ ] **Disable dangerous commands.** Server-side `rename-command`
  for `FLUSHALL`, `FLUSHDB`, `KEYS`, `CONFIG` in shared
  environments.

### `titan-scheduler`

<span class="module-badge">official · @omnitron-dev/titan-scheduler</span>

- [ ] **No user-supplied cron expressions.** `@Cron` strings are
  developer-authored. If you accept schedules from users (multi-
  tenant SaaS), validate against an allow-list of patterns.
- [ ] **Job runtime bounded.** Long-running jobs without a
  timeout pile up missed-fires; configure `timeout` per job.
- [ ] **Persistence backend access controlled.** Same access
  story as the database: persisted job rows reveal what runs and
  when.
- [ ] **`@Cron` handlers idempotent.** Missed-fire replay
  semantics make some jobs run twice; design for it.

### `titan-telemetry-relay`

<span class="module-badge">official · @omnitron-dev/titan-telemetry-relay</span>

- [ ] **WAL file permissions `0600`.** WAL contains buffered
  metric / log payloads — readable WAL = readable observability.
- [ ] **Signed transport between producer and aggregator.** Run
  over a TLS Netron transport or a mutually authenticated
  channel.
- [ ] **Aggregator access controlled.** Whoever can read the
  aggregator's storage can read everyone's telemetry.

### `config` (built-in)

<span class="module-badge">built-in · @omnitron-dev/titan/module/config</span>

- [ ] **No secrets in committed files.** `config/default.yaml` is
  defaults; `config/local.yaml` is git-ignored; secrets come from
  env or a secret-manager source.
- [ ] **Schema covers every required key.** Misconfiguration must
  fail at boot, not at the first dependent call.
- [ ] **`watchForChanges: false` in production for connection
  strings.** Hot-reloading a DB URL won't rebind an open pool —
  reserve hot-reload for safe-to-swap values (log level, flags).
- [ ] **Read through `ConfigService`, never `process.env`
  directly.** Bypasses layered sources and breaks tests.

### `logger` (built-in)

<span class="module-badge">built-in · @omnitron-dev/titan/module/logger</span>

- [ ] **PII redaction processor enabled.** Configure the
  redaction processor for fields that may carry email / phone /
  card data.
- [ ] **Log level production-appropriate.** `info` or `warn` in
  production; `debug` only for time-bounded investigations.
- [ ] **No raw secrets in `error.message`.** Errors are caught
  and logged; ensure secret-laden messages are sanitised before
  throwing.
- [ ] **Sink configured for tamper-evident transport.** Append-only,
  signed transports for compliance scenarios.

## Cross-cutting

### Transport / network

- [ ] **Netron port not publicly exposed** without auth + TLS.
- [ ] **Cluster-internal services pinned to private VPC.**
- [ ] **Egress controlled.** Outbound webhooks / JWKS fetches
  whitelisted.

### Secrets management

- [ ] **All secrets via secret manager** (AWS Secrets Manager,
  Vault, etc.). Not in env vars at rest in container images.
- [ ] **Secret rotation runbook tested.** Rotating without
  downtime requires the multi-key support of `titan-auth`,
  `titan-redis` ACL renaming, and `ConfigService` reload.

### Audit logging

- [ ] **Admin actions logged with actor + outcome.** Lock force-
  release, PM admin RPC, manual notification sends, scheduler
  trigger.
- [ ] **Logs ship to an append-only sink.** Local files alone are
  insufficient for compliance.

### Dependency hygiene

- [ ] **`pnpm audit` clean.** Known CVEs in transitive deps
  patched within SLA.
- [ ] **Modules pinned to known versions.** `^` ranges are fine
  for dev; freeze on lockfile in CI.

## See also

- [Best Practices / Observability](/docs/titan/best-practices/observability) — pairs with this checklist
- [Tokens & RPC reference](/docs/titan/modules/tokens-reference#hardening-checklist) — RPC surface specifics
- [Netron / Authentication](/docs/titan/netron/authentication)
- [Lifecycle reference](/docs/titan/modules/lifecycle-reference) — knowing what runs when helps audit

---

---
id: "titan/modules/shared-session"
title: "Shared-session authentication"
description: "Live session authority, revocation and consistent grants across backends."
---

# Shared-session authentication

`createSharedSessionAuthManager` verifies JWTs through `IJWTService` and binds
human credentials to a live session. The application owns sign-in, persistence,
role vocabulary and session creation. Titan owns claim validation and transport
enforcement. [Access](/docs/auth/access-policy) supplies the pure policy rules.

## Canonical authority

Use this mode when the next request must reflect committed changes to sessions,
accounts or role permissions. Return the session and grants from one consistent
snapshot; do not reconstruct authority from a cached JWT permission array.

```ts
import { createSharedSessionAuthManager, type SessionAuthorityLookup }
  from '@omnitron-dev/titan-auth/preset/shared-session';

// Application adapter: one database snapshot, including account status,
// current role grants and any session-specific conditions.
declare const lookup: SessionAuthorityLookup;

const auth = createSharedSessionAuthManager({
  logger,
  jwtService,
  expectedIssuer: 'my-platform',
  sessionAuthority: lookup,
  roleExpander: applicationRoles.expandRoleSet,
});
app.netron.configureAuth(auth);
```

`sessionAuthority(sessionId, userId)` returns `undefined` for an absent or
revoked principal, or a record with `userId`, `expiresAt`, `isRevoked`, `roles`
and `permissions`. It may include `activeRefreshTokenId`. Account suspension,
MFA proof and role conditions belong in this application-owned lookup. The
preset validates the response and refuses a mismatch between subject and owner.
Current roles and permissions replace the corresponding JWT claims.

A database or RPC failure returns 503; it never becomes an authenticated request
or a false assertion that the user signed out. Missing, expired or revoked
sessions are rejected as 401. Keep tokens and signing material out of logs.

## Cooperating backends

`SessionAuthorityService` from
`@omnitron-dev/titan-auth/preset/session-authority` adapts the lookup to Netron:

```ts
await app.netron.peer.exposeService(new SessionAuthorityService(lookup));
```

`SessionAuthority.resolve({ sessionId, userId })` requires **both**
`service_role` and `sessions.resolve`. It returns a minimal authority snapshot,
without bearer credentials. A consumer supplies a `sessionAuthority` callback
which invokes this service through its authenticated Netron client. Configure
bounded deadlines and dispose that client with the application. Do not fall
back to stale grants when the owner is unavailable.

Deploy the owner endpoint before consumers. For high availability, provide
redundant owner instances and an appropriate database failover policy. The
lookup is read-only; retries must remain bounded. This contract avoids a
second copy of the account schema or permission engine in every backend.

## Registry compatibility mode

Alternatively supply `sessionRedis`, with `get(key)` and an optional
`sessionLookup(sessionId)` for a registry miss or outage. The lookup returns
`userId`, `expiresAt`, `isRevoked` and optionally `activeRefreshTokenId`.
A healthy registry hit does not consult the fallback.

The preset **never writes Redis**. An unconditional cache write after a lookup
can resurrect a concurrently revoked session. Restore entries only through an
adapter serialized with revocation. Registry mode is only as current as that
update protocol; use canonical mode for immediate committed-state enforcement.

## Credentials and expiry

Human tokens require bounded string `sub` and `sid` claims and a finite,
unexpired integer `exp`. A human session claiming `service_role` is rejected.
Service tokens omit `sid` and must be issued through a separately controlled
service-credential flow. Role inheritance cannot promote a human to that tier.

Netron revalidates token-backed persistent peers before calls, property access
and core tasks. The checked context is bound to the invocation, including
property getters and setters. Token expiry bounds authentication cache entries.
An already admitted invocation retains its snapshot; sensitive writes should
also enforce ownership and current authority under their transaction's locks.

| Option             | Meaning                                                                   |
| ------------------ | ------------------------------------------------------------------------- |
| `expectedIssuer`   | Defaults to `omnitron`; `null` explicitly disables issuer checking        |
| `sessionAuthority` | Live session and grants on every validation; bypasses registry mode       |
| `sessionKeyPrefix` | Registry namespace; defaults to `omni:session:`                           |
| `cacheTtl`         | Deprecated; this preset never caches authentication results               |
| `cnfHmacSecret`    | Checks an access token's confirmation against the active refresh-token ID |
| `roleExpander`     | Optional application role hierarchy                                       |

Signature verification may be cached by `IJWTService`; live authority is a
separate decision. When `cnf.fp` is present in canonical mode with a configured
secret, a missing or changed active refresh-token ID rejects the credential.
Legacy registry values without refresh-token IDs cannot provide that guarantee.
Issuer and verifier must agree on the binding secret and rotation protocol.

## Session conditions

`evaluateSessionPolicy` from `@omnitron-dev/access/session-policy` provides a
pure evaluator for `requireMfa`, absolute `maxSessionDuration` in minutes, and
`timeWindows` with IANA timezones. Supply the original session creation time and
persisted proof of completed MFA. Enrollment alone is insufficient. Evaluate on
every protected request and renewal; renewing a token must not reset session age.
Unknown or malformed conditions fail closed.

See [JWT configuration](/docs/titan/modules/auth), [RPC authentication](/docs/titan/netron/authentication)
and [token issuance](/docs/auth/token-issuance) for the surrounding lifecycle.

---

---
id: "titan/modules/storage"
title: "titan-storage"
description: "Local and S3 object storage behind one streaming adapter contract."
---

# titan-storage

`@omnitron-dev/titan-storage` provides `StorageAdapter`, `LocalAdapter`,
`S3Adapter`, `StorageModule` and the `STORAGE_ADAPTER` injection token.
It owns object bytes and adapter lifecycle. Account records, authorization,
tenants, quotas, sharing links and application metadata remain application code.

```ts
import { StorageModule } from '@omnitron-dev/titan-storage';

const storage = StorageModule.forRoot({ backend: 'local' });
// Include `storage` in the application's module imports.
```

The module registers one selected adapter using Titan's config and logger
dependencies. The package currently includes the S3 SDK dependencies; choosing
the local backend avoids initializing an S3 client, but does not remove those
packages from installation. Both adapters implement the same server-side
streaming interface; this is not a browser upload client.

## Configuration and lifecycle

| Backend | Configuration                                                                                                                                                 |
| ------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Local   | `storage.local.basePath`; defaults to `./data/storage`                                                                                                        |
| S3      | `storage.s3`: `bucket`, `accessKeyId`, `secretAccessKey`; optional `endpoint`, `region`, `forcePathStyle`, `maxSockets`, `connectionTimeout`, `socketTimeout` |

If `storage.s3` is absent, S3 reads `S3_ENDPOINT`, `S3_ACCESS_KEY`,
`S3_SECRET_KEY` and `S3_BUCKET` from the environment. Supply credentials through
runtime configuration. Local initialization resolves the configured path against
the application's working directory without creating files. The first storage
operation creates and validates the root; concurrent operations share this work.
An unused local adapter therefore leaves the filesystem untouched when another
provider serves the application. Failed initialization reaches every waiting
caller and can be retried after the storage problem is repaired.

S3 initialization creates its client; teardown destroys it. Lifecycle ownership
belongs to Titan when the adapter is registered as a module provider. A caller
constructing an adapter directly must invoke its lifecycle itself.

`exists()` returns `false` for a missing object. Permission errors, unsafe paths
and corrupt metadata remain errors. Usage calculation fails on unreadable
directories rather than returning an apparently complete partial total.
Keep persistent data outside sealed application and execution-owner runtimes.

## Operations

| Surface                   | Methods                                                                                               |
| ------------------------- | ----------------------------------------------------------------------------------------------------- |
| Object bytes and metadata | `put`, `get`, `head`, `exists`                                                                        |
| Listing and copying       | `list`, `copy`                                                                                        |
| Removal                   | `delete`, `deleteMany` with per-key failures                                                          |
| Temporary delivery        | `getSignedUrl`                                                                                        |
| Multipart uploads         | `createMultipartUpload`, `uploadPart`, `listParts`, `completeMultipartUpload`, `abortMultipartUpload` |
| Usage                     | `getStorageInfo`                                                                                      |

`put` accepts a Node `Readable` or `Buffer` and `PutOptions`; `get` returns a
stream with content metadata and supports byte ranges and conditional reads.
`list` uses continuation tokens and can group prefixes. Copy and upload results
carry an ETag and may carry a backend version ID. Do not infer versioning from
an optional result field.

For local storage, give the service exclusive write access to its root.
Object operations reject paths that escape the root and symbolic links below
it. The `.uploads` directory and `.meta.json` sidecars belong to the adapter.
A listing cursor belongs to the prefix and delimiter that issued it.
A page counts both objects and grouped prefixes towards `maxKeys`.

A multipart upload ID is bound to its original object key. Finalization
validates the supplied part numbers and ETags and assembles the object on
the same filesystem before replacing its contents. A bad part receipt leaves
the previous contents available and the uploaded parts ready for retry.

Use `listParts` to obtain the backend's authoritative part numbers and ETags
before completing a multipart upload. Local multipart bookkeeping is in memory
and is lost on process restart; use a durable backend when that constraint is
unacceptable. The local adapter's signed URL behavior also needs an application
HTTP delivery route. S3 usage reporting is cached; it is not an atomic quota gate.

## Application boundary

Resolve a permitted object key before calling the adapter. Enforce resource
ownership, access policy, upload limits and transactional metadata changes in
application services. Never derive authorization merely from a successful
storage operation. Partial deletion errors must remain visible to cleanup jobs.

See [composition boundaries](/docs/foundations/composition),
[shared access policy](/docs/auth/access-policy) and the
[API index](/docs/reference/api-index) for the full exported types.

---

---
id: "titan/modules/telemetry-relay"
title: "titan-telemetry-relay"
---

# titan-telemetry-relay

<span class="module-badge">official · @omnitron-dev/titan-telemetry-relay · stable</span>

Store-and-forward telemetry pipeline for distributed Titan
applications. Producers buffer telemetry locally, persist to an
on-disk **WAL** (write-ahead log) for crash safety, and ship to a
remote aggregator over an application-supplied transport. Unconfirmed batches in the WAL replay on recovery. Omnitron uses its own SQLite outbox and database receipts for fleet replication. [Follow a measurement from node to controller.](/docs/omnitron/observability)

```bash
pnpm add @omnitron-dev/titan-telemetry-relay
```

> **Not a Titan module.** Unlike most ecosystem packages, this one
> doesn't expose `forRoot` / `forRootAsync`. You instantiate
> `TelemetryRelayService` directly because the relay's role
> (producer / aggregator / both) is a deployment concern, not an
> application-level configuration toggle.

## When you need it

- **Services that emit telemetry but shouldn't ship it directly.**
  Producer pods buffer locally; an aggregator pod takes the hit of
  shipping to long-term storage.
- **Collector outages.** A producer with persistent WAL storage can retry retained batches after restart.
- **Sidecar pattern.** Producer in your app pod; aggregator in a
  sidecar; both forward to a canonical sink.
- **Edge / offline-tolerant deployments.** Branch / IoT nodes that
  may be disconnected for hours buffer to disk and drain when
  connectivity returns.

## Three roles

```mermaid
flowchart LR
  subgraph Producer pod
    A1[App] -- record/emit --> Buf1[Buffer<br/>1000 entries / 10s]
    Buf1 -- onDrain --> Wal1[WAL<br/>~/.omnitron/wal]
    Wal1 -- confirmed write --> Net((Authenticated transport))
  end
  Net --> AggPod[Aggregator pod]
  subgraph Aggregator pod
    AggPod --> Ingest["ingest(nodeId, entries)"]
    Ingest --> Sink[(Backend:<br/>Postgres / OTel /<br/>Loki / S3 / ...)]
  end
```

| Role         | Receives                                                           | Forwards to                                 |
| ------------ | ------------------------------------------------------------------ | ------------------------------------------- |
| `producer`   | Local app via `record` / `emitLog` / `emitMetric` / `emitHealth`   | Aggregator (via transport)                  |
| `aggregator` | Authenticated transport handler calling `receive(nodeId, entries)` | Persistent sink (via `TelemetryAggregator`) |
| `both`       | Local app + remote producers                                       | Persistent sink directly                    |

## Quickstart — producer

```typescript
import { TelemetryRelayService } from '@omnitron-dev/titan-telemetry-relay';

const relay = new TelemetryRelayService({
  role:   'producer',
  nodeId: process.env.HOSTNAME,
  buffer: { maxBufferSize: 1_000, flushIntervalMs: 10_000, maxBatchSize: 500 },
  wal:    { directory: '/var/lib/omnitron/wal', maxSizeBytes: 50 * 1024 * 1024, maxSegments: 10 },
});

relay.setTransport(myAuthenticatedTransport);
await relay.start();

// In your services:
relay.emitLog('users', 'info', 'user created', { userId: 'u_42' });
relay.emitMetric('users.created.total', 1, { source: 'web' });
relay.emitHealth('users', 'ok');
```

## Quickstart — aggregator

```typescript
const relay = new TelemetryRelayService({
  role: 'aggregator',
});

relay.setAggregator(myPersistenceAggregator);   // writes to long-term sink
await relay.start();

// Your transport handler authenticates the peer, then calls:
//   relay.receive(nodeId, entries)
// which delegates to aggregator.ingest(nodeId, entries).
```

## Quickstart — both (single-node / sidecar)

```typescript
const relay = new TelemetryRelayService({
  role: 'both',
});

// Skip transport — the relay calls the local aggregator directly.
relay.setAggregator(myAggregator);
await relay.start();

relay.emitLog('myApp', 'info', 'starting');     // → aggregator.ingest()
```

When `role: 'both'` and no `transport`, `forwardEntries` calls the
local aggregator's `ingest` directly without going through the
network — useful for single-node leader daemons.

## Constructor options — `TelemetryRelayModuleOptions`

| Option      | Type                                   | Default                            |
| ----------- | -------------------------------------- | ---------------------------------- |
| `nodeId`    | `string`                               | `node-{pid}-{base36 timestamp}`    |
| `role`      | `'producer' \| 'aggregator' \| 'both'` | `'both'`                           |
| `buffer`    | `TelemetryBufferConfig` (see below)    | defaults below                     |
| `wal`       | `TelemetryWalConfig \| false`          | defaults below; `false` to disable |
| `transport` | `TelemetryTransport`                   | none (set via `setTransport`)      |

> **`wal: false`** disables WAL persistence entirely. Also auto-
> disabled when `role: 'aggregator'` (the aggregator has no
> outbound transport, so there's nothing to retry).

### `TelemetryBufferConfig`

| Field             | Type          | Default  |
| ----------------- | ------------- | -------- |
| `maxBufferSize`   | `number`      | `1_000`  |
| `flushIntervalMs` | `number` (ms) | `10_000` |
| `maxBatchSize`    | `number`      | `500`    |

### `TelemetryWalConfig`

| Field          | Type     | Default                                |
| -------------- | -------- | -------------------------------------- |
| `directory`    | `string` | `~/.omnitron/wal/`                     |
| `maxSizeBytes` | `number` | `50 * 1024 * 1024` (50 MB per segment) |
| `maxSegments`  | `number` | `10` (max \~500 MB total)              |

Use a separate persistent directory for each producer. Rotation limits the number of segments. A single encoded batch can exceed the target segment size.

## `TelemetryEntry` — the universal unit

```typescript
interface TelemetryEntry {
  id?:        string;          // stable on retry, optional on older WAL entries
  type:       'log' | 'metric' | 'event' | 'health' | 'alert';
  timestamp:  string;          // ISO 8601
  nodeId:     string;          // injected from relay.nodeId
  app?:       string;
  data:       Record<string, unknown>;
  labels?:    Record<string, string>;
}
```

Five entry types route at the aggregator. The relay itself doesn't
care — it just buffers, persists, and ships.

## Convenience emitters

| Method                                   | Resulting `type` / `data`                        |
| ---------------------------------------- | ------------------------------------------------ |
| `emitLog(app, level, message, labels?)`  | `type: 'log'`, `data: { level, message }`        |
| `emitMetric(name, value, labels?, app?)` | `type: 'metric'`, `data: { name, value }`        |
| `emitHealth(app, status, details?)`      | `type: 'health'`, `data: { status, ...details }` |

For `event` / `alert` types or custom payloads, call `record({...})`
directly:

```typescript
relay.record({
  type:   'event',
  app:    'payments',
  data:   { action: 'invoice.paid', invoiceId: 'inv_123', amount: 9900 },
  labels: { region: 'eu-west', tier: 'pro' },
});
```

## Transport contract

```typescript
interface TelemetryTransport {
  send(entries: TelemetryEntry[]): Promise<number>;   // returns ack'd count
  isConnected(): boolean;
  connect():    Promise<void>;
  disconnect(): Promise<void>;
}
```

Implement `connect` and `disconnect` for your transport. Bind the source identity to an authenticated connection before passing a batch to `receive`. Return the committed count from the sink. An empty response, a fraction or a count outside the range of the batch is an invalid acknowledgement. A partial count leaves the entire retained snapshot eligible for replay. If a response is lost after commit, earlier records can arrive again.

Use `entry.id` scoped to the authenticated producer for idempotency. New records receive an ID at admission. Records from a WAL created by older releases can lack one. The receiver decides how to handle their duplicate risk. Where you need durable per-entry evidence or historical reconciliation across Omnitron nodes, use the fleet replication service.

## Aggregator contract

```typescript
interface TelemetryAggregator {
  ingest(nodeId: string, entries: TelemetryEntry[]): Promise<number>;
  query(filter: TelemetryQueryFilter): Promise<TelemetryEntry[]>;
}

interface TelemetryQueryFilter {
  type?:    TelemetryEntry['type'] | TelemetryEntry['type'][];
  nodeId?:  string;
  app?:     string;
  from?:    string;   // ISO 8601
  to?:      string;
  labels?:  Record<string, string>;
  limit?:   number;
  offset?:  number;
}
```

The aggregator owns persistence — write to Postgres, ClickHouse,
S3, OTLP collector, or whatever backend fits. `ingest` returns the
count of entries successfully persisted (used for ack accounting).

## Delivery and recovery

1. `record` places an entry in memory. A process crash before the next flush can lose it. Watch `buffer.totalDropped` for memory pressure.
2. With a WAL, the producer appends the batch and syncs it to disk before sending. Replay and normal sends run in sequence. New arrivals cannot be cleared by a previous send.
3. The sender reads retained entries in batches of at most `maxBatchSize`. It clears the snapshot after the sink confirms every batch in full. Use a transaction and durable IDs at the sink so replay after a lost acknowledgement is safe.
4. Incomplete responses or transport failures leave the snapshot available for retry. Use `stats().lastError` for the reason. `start()` can replay a persisted backlog before any new events arrive. `setTransport` can trigger recovery when the relay is running.
5. WAL retention rotates by segment. Long outages can exceed capacity. Provision persistent space for the expected offline interval and alert on disk usage, `stats().wal?.evictedSegments`, `evictedBytes`, memory drops and delivery failures. Eviction counters cover the current process lifetime.

With `wal: false`, pending entries exist in memory only. A failed sink call can return them to that bounded buffer. An application requiring durable replication should use a persistent WAL directory or the Omnitron outbox.

## WAL on-disk format

NDJSON — one telemetry entry per line:

```jsonl
{"type":"log","timestamp":"2026-05-16T12:34:56.789Z","nodeId":"node-1234-l7x","app":"users","data":{"level":"info","message":"created"},"labels":{"region":"eu"}}
{"type":"metric","timestamp":"2026-05-16T12:34:56.793Z","nodeId":"node-1234-l7x","app":"users","data":{"name":"users.created.total","value":1}}
```

Segment files are named `000001.wal`, `000002.wal`, …, with the
highest-numbered one being the current write target. When it
exceeds `maxSizeBytes`, a new segment opens; segments past
`maxSegments` are deleted (oldest first).

An invalid NDJSON record or unreadable segment is a recovery error. Inspect the journal and the sink's committed IDs before repairing it. Keep a copy of the original segments.

## Statistics

`relay.stats()` returns a snapshot:

```typescript
{
  nodeId:               string;
  role:                 'producer' | 'aggregator' | 'both';
  buffer:               { size, totalPushed, totalDropped, totalFlushed };
  wal:                  { segments, totalSize, totalWritten, currentSegment, evictedSegments, evictedBytes, listError? } | null;
  totalEmitted:         number;
  totalSent:            number;
  totalFailed:          number;
  lastError:            string | null;
  totalReceived:        number;   // aggregator-side
  transportConnected:   boolean;
}
```

Surface these via your existing metrics module — they're the
primary signal for "is the pipeline healthy?".

## Sizing guidance

Each timer tick drains up to `maxBatchSize` entries from the memory buffer. With a ten-second interval and a batch of 500, the upper bound is 50 entries per second before disk and sink latency. At 100 entries per second the backlog will grow with those defaults. Measure under load, then select a batch size and interval that leave room for bursts and slow writes. Size the memory buffer for arrivals between completed flushes.

For a four-hour outage at 100 entries per second and 1,000 encoded bytes per entry, plan for at least 1.44 GB of journal data plus operational headroom. Monitor actual encoded bytes and segment evictions to validate that estimate.

## Lifecycle

`TelemetryRelayService` is a plain `EventEmitter` subclass — no
DI lifecycle hooks. Wire it manually if you want DI integration:

```typescript
import { Injectable } from '@omnitron-dev/titan';
import type { OnStart, OnStop } from '@omnitron-dev/titan/application';
import { TelemetryRelayService } from '@omnitron-dev/titan-telemetry-relay';

@Injectable()
class TelemetryRelayAdapter implements OnStart, OnStop {
  constructor(private readonly relay: TelemetryRelayService) {}
  async onStart() { await this.relay.start(); }
  async onStop()  { await this.relay.stop();  }
}
```

`start()` attempts to deliver a retained WAL through the configured destination; `stop()`
flushes the buffer to WAL, disconnects the transport, and disposes
the WAL (awaits pending writes, closes fds).

## Choosing a destination

The application supplies a destination or changes it with `setTransport`. If several collectors share a store, deduplicate against the same durable identity on every collector. Confirm a record only when the configured durability requirement has been met.

## When to use this vs the built-in logger

| Need                                                                        | Use                                                      |
| --------------------------------------------------------------------------- | -------------------------------------------------------- |
| Local console output during development                                     | `@omnitron-dev/titan/module/logger` (built-in)           |
| Single-pod, ship logs via host log shipper                                  | Built-in logger + stdout                                 |
| Multi-pod, one pipeline for logs/metrics/health with at-least-once delivery | This module                                              |
| Sidecar pattern (one collector per host)                                    | This module with `role: 'both'`                          |
| OTel-compatible sink                                                        | Build a `TelemetryAggregator` wrapping the OTLP exporter |
| Edge node that may be offline for hours                                     | This module with large `maxSegments`                     |

## Operational checks

- If `buffer.totalDropped` grows, the producer has exceeded the memory buffer. Oldest in-memory entries are evicted before they reach disk.
- If WAL eviction counters grow, the outage or input rate has exceeded configured retention. Restore the destination and check for gaps.
- Before accepting remote traffic, configure the receiving aggregator. A receiver with no sink returns zero acknowledgements.
- Place each WAL on persistent storage and recover it under its original producer identity.

## See also

- [`titan-metrics`](/docs/titan/modules/metrics) — emit metrics that this relay
  can ship
- [Best Practices / Observability](/docs/titan/best-practices/observability)
- [Module map](/docs/titan/modules/module-map) — where the relay sits in the
  observability stack

---

---
id: "titan/modules/tokens-reference"
title: "Tokens & RPC reference"
description: "All injection tokens and exposed RPC services across every module."
---

# Tokens & RPC reference

A single-page lookup for every DI token a Titan module exports and
every Netron RPC service a module registers. Useful when you can't
remember which token you need to inject, or when reviewing the
external RPC surface of an app for security/audit purposes.

The token list is checked against the packages by a script rather than
asserted to have been verified — run it yourself:

```bash
node scripts/check-titan-inventories.mjs ../omni
```

It exits non-zero if this page names a token no Titan package exports.

## Built-in module tokens

<span class="module-badge">built-in · @omnitron-dev/titan</span>

### `config` (`@omnitron-dev/titan/module/config`)

| Token                            | Type                     |
| -------------------------------- | ------------------------ |
| `CONFIG_SERVICE_TOKEN`           | `ConfigService`          |
| `CONFIG_OPTIONS_TOKEN`           | `IConfigModuleOptions`   |
| `CONFIG_SCHEMA_TOKEN`            | `unknown` (Zod schema)   |
| `CONFIG_LOADER_SERVICE_TOKEN`    | `ConfigLoaderService`    |
| `CONFIG_VALIDATOR_SERVICE_TOKEN` | `ConfigValidatorService` |
| `CONFIG_WATCHER_SERVICE_TOKEN`   | `ConfigWatcherService`   |

### `logger` (`@omnitron-dev/titan/module/logger`)

| Token                     | Type                             |
| ------------------------- | -------------------------------- |
| `LOGGER_SERVICE_TOKEN`    | `ILoggerModule`                  |
| `LOGGER_TOKEN`            | `ILogger` (root logger instance) |
| `LOGGER_OPTIONS_TOKEN`    | `ILoggerOptions`                 |
| `LOGGER_TRANSPORTS_TOKEN` | `ITransport[]`                   |
| `LOGGER_PROCESSORS_TOKEN` | `ILogProcessor[]`                |

## Official module tokens

<span class="module-badge">official · @omnitron-dev/titan-*</span>

### `titan-auth`

| Token                      | Type                 |
| -------------------------- | -------------------- |
| `AUTH_OPTIONS_TOKEN`       | `IAuthModuleOptions` |
| `AUTH_MIDDLEWARE_TOKEN`    | `IAuthMiddleware`    |
| `JWT_SERVICE_TOKEN`        | `IJWTService`        |
| `SIGNED_URL_SERVICE_TOKEN` | `ISignedUrlService`  |

### `titan-cache`

| Token                         | Type                              |
| ----------------------------- | --------------------------------- |
| `CACHE_SERVICE_TOKEN`         | `ICacheService`                   |
| `CACHE_OPTIONS_TOKEN`         | `ICacheModuleOptions`             |
| `CACHE_DEFAULT_TOKEN`         | `ICache` (default cache instance) |
| `CACHE_ADAPTER_TOKEN`         | `CacheAdapter`                    |
| `CACHE_ADAPTER_OPTIONS_TOKEN` | `ICacheAdapterOptions`            |

### `titan-database`

| Token                       | Type                          |
| --------------------------- | ----------------------------- |
| `DATABASE_MANAGER`          | `DatabaseManager`             |
| `DATABASE_MODULE_OPTIONS`   | `DatabaseModuleOptions`       |
| `DATABASE_CONNECTION`       | `Kysely` (default connection) |
| `DATABASE_HEALTH_INDICATOR` | `DatabaseHealthIndicator`     |

The tokens are `Symbol.for('titan:…')` symbols, but `DatabaseManager`
and `DatabaseHealthIndicator` are also registered against their class
references, so you can inject either way:

```typescript
constructor(private readonly db: DatabaseManager) {}
```

For `forFeature(...)` repository registration, the per-repository
tokens are generated from the repository class name. See the
[`titan-database`](/docs/titan/modules/database) page for `forFeature` patterns.

### `titan-discovery`

| Token                                | Type                         |
| ------------------------------------ | ---------------------------- |
| `DISCOVERY_SERVICE_TOKEN`            | `IDiscoveryService`          |
| `DISCOVERY_OPTIONS_TOKEN`            | `DiscoveryOptions`           |
| `NETRON_DISCOVERY_INTEGRATION_TOKEN` | `NetronDiscoveryIntegration` |
| `REDIS_TOKEN` (module-local)         | `Redis`                      |

> `titan-discovery` exports its **own** `REDIS_TOKEN`
> (`createToken('TitanDiscovery:Redis')`) — it is not the same
> symbol as `titan-redis`'s `REDIS_MANAGER`. Supply a client
> against this token, or let the module create one from its options.

### `titan-events`

| Token                            | Type                     |
| -------------------------------- | ------------------------ |
| `EVENTS_SERVICE_TOKEN`           | `EventsService`          |
| `EVENT_OPTIONS_TOKEN`            | `IEventEmitterOptions`   |
| `EVENT_EMITTER_TOKEN`            | `EnhancedEventEmitter`   |
| `EVENT_BUS_SERVICE_TOKEN`        | `EventBusService`        |
| `EVENT_HISTORY_SERVICE_TOKEN`    | `EventHistoryService`    |
| `EVENT_SCHEDULER_SERVICE_TOKEN`  | `EventSchedulerService`  |
| `EVENT_VALIDATION_SERVICE_TOKEN` | `EventValidationService` |

> `EVENT_METADATA_SERVICE_TOKEN` and `EVENT_DISCOVERY_SERVICE_TOKEN`
> exist internally but are **not** re-exported from the package
> barrel — their backing services are module-private. The seven
> tokens above are the public DI surface.

### `titan-health`

| Token                               | Type                       |
| ----------------------------------- | -------------------------- |
| `HEALTH_SERVICE_TOKEN`              | `IHealthService`           |
| `HEALTH_MODULE_OPTIONS_TOKEN`       | `HealthModuleOptions`      |
| `HEALTH_RPC_SERVICE_TOKEN`          | `HealthRpcService`         |
| `MEMORY_HEALTH_INDICATOR_TOKEN`     | `MemoryHealthIndicator`    |
| `EVENT_LOOP_HEALTH_INDICATOR_TOKEN` | `EventLoopHealthIndicator` |
| `DISK_HEALTH_INDICATOR_TOKEN`       | `DiskHealthIndicator`      |
| `DATABASE_HEALTH_INDICATOR_TOKEN`   | `DatabaseHealthIndicator`  |
| `REDIS_HEALTH_INDICATOR_TOKEN`      | `RedisHealthIndicator`     |

### `titan-lock`

| Token                | Type                      |
| -------------------- | ------------------------- |
| `LOCK_SERVICE_TOKEN` | `IDistributedLockService` |
| `LOCK_OPTIONS_TOKEN` | `ILockModuleOptions`      |

### `titan-metrics`

| Token                   | Type                    |
| ----------------------- | ----------------------- |
| `METRICS_SERVICE_TOKEN` | `IMetricsService`       |
| `METRICS_OPTIONS_TOKEN` | `IMetricsModuleOptions` |
| `METRICS_STORAGE_TOKEN` | `IMetricsStorage`       |

### `titan-notifications`

Note the trailing `_TOKEN` is **omitted** in this package's naming
convention — symbols are bare:

| Token                                  | Type                            |
| -------------------------------------- | ------------------------------- |
| `NOTIFICATIONS_SERVICE`                | `NotificationsService`          |
| `NOTIFICATIONS_MODULE_OPTIONS`         | `NotificationsModuleOptions`    |
| `NOTIFICATIONS_TRANSPORT`              | `MessagingTransport`            |
| `NOTIFICATIONS_CHANNEL_REGISTRY`       | `ChannelRegistry`               |
| `NOTIFICATIONS_CHANNEL_ROUTER`         | `IChannelRouter`                |
| `NOTIFICATIONS_TEMPLATE_ENGINE`        | `TemplateEngine`                |
| `NOTIFICATIONS_PREFERENCE_STORE`       | `IPreferenceStore`              |
| `NOTIFICATIONS_REDIS_PREFERENCE_STORE` | `RedisPreferenceStore`          |
| `NOTIFICATIONS_RATE_LIMITER`           | `IRateLimiter`                  |
| `NOTIFICATIONS_REDIS_RATE_LIMITER`     | `RedisRateLimiter`              |
| `NOTIFICATIONS_EVENT_EMITTER`          | `NotificationsEventEmitter`     |
| `NOTIFICATIONS_HEALTH`                 | `NotificationsHealthIndicator`  |
| `NOTIFICATION_PERSISTER`               | `INotificationPersister`        |
| `NOTIFICATION_REALTIME_SIGNALER`       | `INotificationRealtimeSignaler` |
| `NOTIFICATION_TARGET_RESOLVER`         | `INotificationTargetResolver`   |

### `titan-pm`

| Token               | Type                      |
| ------------------- | ------------------------- |
| `PM_CONFIG_TOKEN`   | `IProcessManagerConfig`   |
| `PM_MANAGER_TOKEN`  | `ProcessManager`          |
| `PM_REGISTRY_TOKEN` | `ProcessRegistry`         |
| `PM_SPAWNER_TOKEN`  | `ProcessSpawnerFactory`   |
| `PM_HEALTH_TOKEN`   | `ProcessHealthChecker`    |
| `PM_METRICS_TOKEN`  | `ProcessMetricsCollector` |

### `titan-ratelimit`

| Token                      | Type                      |
| -------------------------- | ------------------------- |
| `RATE_LIMIT_SERVICE_TOKEN` | `IRateLimitService`       |
| `RATE_LIMIT_OPTIONS_TOKEN` | `IRateLimitModuleOptions` |
| `RATE_LIMIT_STORAGE_TOKEN` | `IRateLimitStorage`       |

### `titan-redis`

| Symbol                                    | Type                            |
| ----------------------------------------- | ------------------------------- |
| `REDIS_MANAGER`                           | `RedisManager`                  |
| `REDIS_MODULE_OPTIONS`                    | `RedisModuleOptions`            |
| `getRedisClientToken(namespace?)` factory | `IRedisClient` (named instance) |

The default client is injected with the `@InjectRedis()` decorator;
`getRedisClientToken(namespace)` is the per-namespace token used for
multi-client setups. (`titan-redis` exports no `REDIS_TOKEN` — its manager token is
`REDIS_MANAGER`. Note that `titan-discovery` *does* export a
`REDIS_TOKEN` of its own, for the client it injects; the two are
unrelated, so check which package an import comes from.)

```typescript
import { getRedisClientToken, type IRedisClient }
  from '@omnitron-dev/titan-redis';

@Inject(getRedisClientToken('cache'))   cacheRedis: IRedisClient;
@Inject(getRedisClientToken('queue'))   queueRedis: IRedisClient;
```

### `titan-scheduler`

| Token                         | Type                        |
| ----------------------------- | --------------------------- |
| `SCHEDULER_SERVICE_TOKEN`     | `SchedulerService`          |
| `SCHEDULER_CONFIG_TOKEN`      | `ISchedulerConfig`          |
| `SCHEDULER_REGISTRY_TOKEN`    | `SchedulerRegistry`         |
| `SCHEDULER_EXECUTOR_TOKEN`    | `SchedulerExecutor`         |
| `SCHEDULER_PERSISTENCE_TOKEN` | `ISchedulerPersistence`     |
| `SCHEDULER_DISCOVERY_TOKEN`   | `SchedulerDiscovery`        |
| `SCHEDULER_METRICS_TOKEN`     | `SchedulerMetricsCollector` |
| `SCHEDULER_LISTENERS_TOKEN`   | `IJobListener[]`            |

### `titan-telemetry-relay`

`titan-telemetry-relay` does **not** define DI tokens. Instantiate
`TelemetryRelayService` directly with config; wire it into the
container as a `useValue` provider if you want DI-style injection.

## Importing tokens — the convention

Every module exports both the token and its type from the package
root so you can pull them in one line:

```typescript
import {
  CACHE_SERVICE_TOKEN,
  type CacheService,
} from '@omnitron-dev/titan-cache';

class UsersService {
  constructor(
    @Inject(CACHE_SERVICE_TOKEN) private readonly cache: CacheService,
  ) {}
}
```

> **Why named tokens, not classes?** Tokens decouple the *contract*
> from the *implementation*. Test code can register a fake
> `CacheService` against the same token without subclassing the
> production class.

## RPC services — what's reachable from the wire

Most modules expose their API only via DI inside the same process.
Two modules ship a Netron `@Service` class so the Omnitron CLI /
web console (or any remote peer) can call them.

| Module                                         | RPC service name  | When                                                                                                     |
| ---------------------------------------------- | ----------------- | -------------------------------------------------------------------------------------------------------- |
| [`titan-health`](/docs/titan/modules/health)   | `Health@1.0.0`    | `HealthRpcService` is **provided by the module** when `enableRpcService: true` (default)                 |
| [`titan-metrics`](/docs/titan/modules/metrics) | `OmnitronMetrics` | `MetricsRpcService` is **exported, not auto-provided** — register it yourself (the Omnitron daemon does) |

Everything else (`titan-auth`, `titan-cache`, `titan-database`,
`titan-discovery`, `titan-events`, `titan-lock`, `titan-notifications`,
`titan-pm`, `titan-ratelimit`, `titan-redis`, `titan-scheduler`,
`titan-telemetry-relay`) **does not contribute an RPC surface**. It's
your own `@Service` classes that own the wire contract; the modules
provide infrastructure consumed via DI.

**Note about `titan-pm`** — it does not expose a `pm@…` service; it
calls `netron.peer.exposeService(...)` on your worker instances when
they boot. The exposed services are the ones you wrote.

**Note about `titan-discovery`** — its `registerService` method is
internal API for adding entries to the Redis registry, not an RPC
endpoint. Discovery lookups happen in-process via DI.

**Disabling the health RPC surface** — pass `enableRpcService: false`:

```typescript
TitanHealthModule.forRoot({
  enableRpcService: false,        // local checks still work
})
```

`OmnitronMetrics` is reachable only if you registered
`MetricsRpcService` yourself — it has no module feature flag.
If you don't want it on the wire, simply don't register it (or
control exposure at the transport layer: don't expose the Netron
port externally, or front it with auth-only middleware).

## Health RPC surface — `Health@1.0.0`

The single canonical example. Source:
[`packages/titan-health/src/health.rpc-service.ts`](https://github.com/omnitron-dev/omni/tree/main/packages/titan-health/src/health.rpc-service.ts).

```typescript
const health = await peer.queryInterface<HealthRpcService>('Health@1.0.0');

await health.check();   // full report
await health.live();    // liveness
await health.ready();   // readiness
```

The exact method set is `HealthRpcService` — refer to its
`@Public()` decorators in source for the current surface.

## Hardening checklist

Before exposing the Netron port outside the cluster:

- [ ] Configure `titan-auth` with a strong JWT secret / JWKS endpoint.
- [ ] Apply `@Auth(...)` on every `@Public()` method that touches state.
- [ ] Front the port with TLS termination (transport options).
- [ ] Run `titan-ratelimit` middleware globally on Netron.
- [ ] Set `enableRpcService: false` on `titan-health` if probes are
  served via plain HTTP routes and you don't want the JSON
  endpoint reachable as RPC.
- [ ] Review every `@Service` *you* wrote — those are the actual
  external surface.

## See also

- [Netron / Authentication](/docs/titan/netron/authentication)
- [Best Practices / Security](/docs/titan/best-practices/observability)
- [Module map](/docs/titan/modules/module-map) — visual dependency graph

---

---
id: "titan/modules-system/authoring-modules"
title: "Authoring a module"
description: "How to build, package, and publish a Titan module to share with the community."
---

# Authoring a module

This is the canonical recipe for building a Titan module. Every
official `titan-*` package is built from this template, and any
community module worth adopting follows the same shape.

The goal is **a module that looks and behaves like an official one**:
typed `forRoot` options, lifecycle-aware services, exported tokens,
documented in the same shape, ready to drop into an existing
application without surprise.

## What "a module" means in Titan

A module is a class with a `@Module` decorator that:

1. **Declares providers** — services it contributes to the container.
2. **Declares exports** — providers visible to importers.
3. **Optionally provides a `forRoot` (and `forRootAsync`)** static
   factory that returns a `DynamicModule`, configuring its providers
   from caller-supplied options.

The pattern lets each Titan application compose its set of modules
with the configuration it needs — same code, different bindings.

## Package skeleton

Start from this layout:

**my-titan-foo/**

- `src/`
  - `foo.module.ts` — the @Module class + forRoot
  - `foo.service.ts` — main service exposed to consumers
  - `foo.types.ts` — public types (options, interfaces)
  - `foo.tokens.ts` — DI tokens for every export
  - `foo.decorators.ts` — optional: decorators if needed
  - `index.ts` — public barrel
- `package.json`
- `tsconfig.json`
- `README.md`

### `package.json` essentials

```json
{
  "name": "@your-scope/titan-foo",
  "version": "0.1.0",
  "type": "module",
  "main": "dist/index.js",
  "types": "dist/index.d.ts",
  "exports": {
    ".": { "types": "./dist/index.d.ts", "default": "./dist/index.js" }
  },
  "peerDependencies": {
    "@omnitron-dev/titan": "^0.1.0"
  },
  "scripts": {
    "build": "tsc -p tsconfig.json"
  }
}
```

**Always** declare `@omnitron-dev/titan` as a `peerDependency` —
never as a regular dependency. Otherwise the consumer ends up with
two copies of the framework and DI breaks silently.

## Step 1 — define the options interface

Every module's `forRoot` accepts a single options bag. Put the
interface in `*.types.ts`:

```typescript
// src/foo.types.ts

export interface IFooModuleOptions {
  /** Required: API key for the upstream service */
  apiKey: string;

  /** Optional: request timeout in milliseconds */
  timeoutMs?: number;

  /** Optional: enable verbose logging */
  debug?: boolean;

  /** Optional: make module global */
  isGlobal?: boolean;
}

export interface IFooModuleAsyncOptions {
  imports?: any[];
  useFactory?: (...args: any[]) => Promise<IFooModuleOptions> | IFooModuleOptions;
  inject?: any[];
  isGlobal?: boolean;
}
```

Conventional naming:

- Interface name = `I` + module name + `ModuleOptions`
- Async variant = same + `Async`
- Every field documented with JSDoc
- Optionals end in `?`; required fields don't

## Step 2 — define the DI tokens

```typescript
// src/foo.tokens.ts

import { createToken } from '@omnitron-dev/titan/nexus';
import type { FooService } from './foo.service.js';
import type { IFooModuleOptions } from './foo.types.js';

export const FOO_SERVICE_TOKEN = createToken<FooService>('FooService');
export const FOO_OPTIONS_TOKEN = createToken<IFooModuleOptions>('FooOptions');
```

Tokens go in their own file so consumers can import them without
pulling in the implementation.

## Step 3 — write the service

```typescript
// src/foo.service.ts

import { Injectable, Inject } from '@omnitron-dev/titan';
import { LOGGER_SERVICE_TOKEN, type ILoggerModule }
  from '@omnitron-dev/titan/module/logger';
import { FOO_OPTIONS_TOKEN } from './foo.tokens.js';
import type { IFooModuleOptions } from './foo.types.js';

@Injectable()
export class FooService {
  private readonly logger;

  constructor(
    @Inject(FOO_OPTIONS_TOKEN)        private readonly options: IFooModuleOptions,
    @Inject(LOGGER_SERVICE_TOKEN)     loggerModule: ILoggerModule,
  ) {
    this.logger = loggerModule.getLogger().child({ context: 'FooService' });
  }

  async fetch(path: string): Promise<unknown> {
    if (this.options.debug) {
      this.logger.debug('fetch', { path });
    }
    // …actual work using this.options.apiKey, this.options.timeoutMs
    return { path };
  }

  // Lifecycle hooks — implement if the service holds external state
  async onInit(): Promise<void> {
    // E.g. validate credentials before serving traffic
  }

  async onDestroy(): Promise<void> {
    // E.g. close connections / drain queues
  }
}
```

**Key conventions:**

- `@Injectable()` so the container can resolve it.
- Inject the options bundle via the dedicated token; **never**
  inject `IFooModuleOptions` directly.
- Inject the logger so consumers can see what your module is doing.
- Implement lifecycle interfaces (`OnInit`, `OnStart`, `OnStop`,
  `OnDestroy`) if your service holds resources.

## Step 4 — write the module class

```typescript
// src/foo.module.ts

import { Module, type DynamicModule } from '@omnitron-dev/titan';
import { FOO_SERVICE_TOKEN, FOO_OPTIONS_TOKEN } from './foo.tokens.js';
import { FooService } from './foo.service.js';
import type { IFooModuleOptions, IFooModuleAsyncOptions } from './foo.types.js';

@Module({})
export class TitanFooModule {
  /**
   * Configure FooModule with static options.
   */
  static forRoot(options: IFooModuleOptions): DynamicModule {
    return {
      module: TitanFooModule,
      global: options.isGlobal,
      providers: [
        { provide: FOO_OPTIONS_TOKEN, useValue: options },
        { provide: FOO_SERVICE_TOKEN, useClass: FooService },
      ],
      exports: [FOO_SERVICE_TOKEN],
    };
  }

  /**
   * Configure FooModule with async options — resolves options via
   * a factory that can inject other providers.
   */
  static forRootAsync(options: IFooModuleAsyncOptions): DynamicModule {
    return {
      module: TitanFooModule,
      global: options.isGlobal,
      imports: options.imports,
      providers: [
        {
          provide:    FOO_OPTIONS_TOKEN,
          useFactory: options.useFactory!,
          inject:     options.inject ?? [],
        },
        { provide: FOO_SERVICE_TOKEN, useClass: FooService },
      ],
      exports: [FOO_SERVICE_TOKEN],
    };
  }
}
```

**`@Module({})` is intentionally empty** on the class — the
`forRoot`/`forRootAsync` methods produce a `DynamicModule` at call
time that contains the actual provider list.

## Step 5 — write the public barrel

```typescript
// src/index.ts

export { TitanFooModule } from './foo.module.js';
export { FooService } from './foo.service.js';
export type {
  IFooModuleOptions,
  IFooModuleAsyncOptions,
} from './foo.types.js';
export {
  FOO_SERVICE_TOKEN,
  FOO_OPTIONS_TOKEN,
} from './foo.tokens.js';

// Decorators, if any:
// export { Foo } from './foo.decorators.js';
```

Only export what's truly public. Internal helpers stay un-exported.

## Step 6 — write the consumer-facing README

The README is the public contract. Use this skeleton (the same
shape every official module uses):

```markdown
# @your-scope/titan-foo

> One-sentence description.

## Install

\`\`\`bash
pnpm add @your-scope/titan-foo @omnitron-dev/titan
\`\`\`

## Quickstart

\`\`\`typescript
import { TitanFooModule } from '@your-scope/titan-foo';

@Module({
  imports: [TitanFooModule.forRoot({ apiKey: env.FOO_API_KEY })],
})
class AppModule {}
\`\`\`

## Options

| Option       | Type     | Default | Description |
| ------------ | -------- | ------- | ----------- |
| `apiKey`     | string   | —       | …           |
| `timeoutMs`  | number   | 5000    | …           |

## Services

| Class         | Token                  | Purpose |
| ------------- | ---------------------- | ------- |
| `FooService`  | `FOO_SERVICE_TOKEN`    | …       |

## Lifecycle

…

## Anti-patterns

…
```

## Optional — async-options patterns

For modules whose options need to be loaded from `ConfigModule` or
another async source, `forRootAsync` supports several factories:

### `useFactory` (most common)

```typescript
TitanFooModule.forRootAsync({
  imports: [ConfigModule],
  useFactory: (config: ConfigService) => ({
    apiKey: config.get('foo.apiKey'),
  }),
  inject: [ConfigService],
})
```

### `useClass` — for factories with their own dependencies

```typescript
@Injectable()
export class FooOptionsFactory implements IFooOptionsFactory {
  constructor(private readonly config: ConfigService) {}

  createOptions(): IFooModuleOptions {
    return { apiKey: this.config.get('foo.apiKey') };
  }
}

TitanFooModule.forRootAsync({
  imports: [ConfigModule],
  useClass: FooOptionsFactory,
})
```

### `useExisting` — when the factory class is already provided

```typescript
TitanFooModule.forRootAsync({
  useExisting: SomeExistingFactory,
})
```

## Optional — decorators

If your module benefits from a decorator (e.g. `@FooCache`,
`@FooLog`), put it in `src/foo.decorators.ts`:

```typescript
// src/foo.decorators.ts

import { Inject } from '@omnitron-dev/titan';
import { FOO_SERVICE_TOKEN } from './foo.tokens.js';

export const InjectFoo = () => Inject(FOO_SERVICE_TOKEN);
```

Or for method-level decorators, follow the `createMethodInterceptor`
pattern from `@omnitron-dev/titan/decorators`.

## Optional — health indicator

If your module talks to an external dependency, ship a health
indicator so consumers can register it with
[`titan-health`](/docs/titan/modules/health):

```typescript
// src/foo.health.ts

import { Injectable } from '@omnitron-dev/titan';
import type { IHealthIndicator, HealthIndicatorResult }
  from '@omnitron-dev/titan-health';

@Injectable()
export class FooHealthIndicator implements IHealthIndicator {
  name = 'foo';

  constructor(private readonly foo: FooService) {}

  async check(): Promise<HealthIndicatorResult> {
    try {
      await this.foo.ping();
      return { status: 'healthy' };
    } catch (e) {
      return { status: 'unhealthy', message: String(e) };
    }
  }
}
```

Export it as `FooHealthIndicator` + `FOO_HEALTH_INDICATOR_TOKEN`.

## Module conventions checklist

When your module is ready to share, verify:

- [ ] `@omnitron-dev/titan` is a `peerDependency` (not `dependency`)
- [ ] Module class is named `TitanFooModule` (matches official naming)
- [ ] `forRoot(options)` returns `DynamicModule` with explicit
  `providers` / `exports`
- [ ] `forRootAsync(options)` exists if options can come from another module
- [ ] Every public export is in `index.ts`; nothing leaks via deep
  imports
- [ ] Every DI token is typed (`createToken<T>(name)`)
- [ ] The main service is `@Injectable()`
- [ ] Lifecycle hooks (`onInit`/`onStart`/`onStop`/`onDestroy`)
  implemented where state requires
- [ ] Options interface is fully documented with JSDoc
- [ ] README follows the official-module shape
- [ ] Health indicator exported if module owns an external dependency
- [ ] Test suite covers happy path + lifecycle + error cases

## Publishing

For npm publish:

```bash
pnpm build
npm publish --access public
```

For inclusion in the Titan documentation:

1. Add an entry under [Community Modules](/docs/titan/modules/community)
   with package URL + one-line purpose.
2. Open a PR against `omnitron-dev/omni` referencing your repo,
   the maintenance commitment, and links to docs / tests.

## Worked example — complete `titan-foo`

Putting it all together:

```typescript
// src/index.ts (final state)

export { TitanFooModule }       from './foo.module.js';
export { FooService }           from './foo.service.js';
export { FooHealthIndicator }   from './foo.health.js';

export {
  FOO_SERVICE_TOKEN,
  FOO_OPTIONS_TOKEN,
  FOO_HEALTH_INDICATOR_TOKEN,
} from './foo.tokens.js';

export type {
  IFooModuleOptions,
  IFooModuleAsyncOptions,
} from './foo.types.js';

export { InjectFoo } from './foo.decorators.js';
```

Consumer:

```typescript
import { TitanFooModule, InjectFoo, type FooService } from '@your-scope/titan-foo';

@Module({
  imports: [TitanFooModule.forRoot({ apiKey: env.FOO_API_KEY })],
  providers: [MyService],
})
class AppModule {}

@Injectable()
class MyService {
  constructor(@InjectFoo() private readonly foo: FooService) {}

  async fetchSomething() {
    return this.foo.fetch('/data');
  }
}
```

A consumer using your module shouldn't be able to tell whether it's
official or community. That's the bar.

## See also

- [Modules system / Defining modules](/docs/titan/modules-system/defining-modules) — module
  fundamentals
- [Modules system / Dynamic modules](/docs/titan/modules-system/dynamic-modules) —
  forRoot / forRootAsync patterns
- [Community modules](/docs/titan/modules/community) — how to register
  your module in the catalogue
- [Testing / Modules](/docs/titan/testing/modules) — write tests for your
  module that consumers can rely on

---

---
id: "titan/modules-system/defining-modules"
title: "Defining Modules"
description: "The unit of composition in Titan — what a module is, how it composes, and how to design well."
---

# Defining Modules

A module is the unit of composition in Titan. It is a class with a
`@Module` decorator that declares three sets:

- **`providers`** — what this module *contributes* to the container.
- **`imports`** — what this module *depends on* from other modules.
- **`exports`** — what this module *exposes* to importers.

```typescript
@Module({
  imports:   [LoggerModule, ConfigModule],
  providers: [UsersService, UsersRepository],
  exports:   [UsersService],
})
export class UsersModule {}
```

That is the entire surface for most modules.

## The visibility rule

A provider is visible to:

1. **Other providers in the same module** — always.
2. **Providers in modules that import this module** — only if the
   provider is listed in `exports`.

A provider in `providers` but not `exports` is **module-private**.
It can be injected by services in this module, but a different
module that imports this one cannot see it.

```typescript
@Module({
  providers: [
    UsersService,         // exported below — public
    UsersRepository,      // not exported — private to this module
    UsersValidator,       // not exported — private
  ],
  exports: [UsersService],
})
export class UsersModule {}
```

This is the same encapsulation rule modules in any modular system give
you. Use it. A module that exports everything is harder to refactor
than a module that exports only its public surface.

## The composition rule

`imports` is transitive in one direction: if `A` imports `B`, then `A`
can use `B`'s exports. It is **not** transitive across — if `A`
imports `B` and `B` imports `C`, then `A` does not automatically see
`C`'s exports unless `B` re-exports them.

```typescript
@Module({
  imports: [LoggerModule, ConfigModule],
  exports: [LoggerModule],   // re-export so importers of MyModule see LoggerModule too
})
export class MyModule {}
```

Re-export when you want to *bundle* a dependency into your public
contract. Otherwise, importers should import what they need directly.

## A complete example

```typescript
import { Module, Service, Injectable, Inject } from '@omnitron-dev/titan';
import { Public } from '@omnitron-dev/titan/decorators';
import { LoggerModule, type LoggerService } from '@omnitron-dev/titan/module/logger';
import { ConfigModule, type ConfigService } from '@omnitron-dev/titan/module/config';

@Injectable()
class UsersRepository {
  constructor(private readonly config: ConfigService) {}
  async findById(id: string) { /* … */ }
}

@Service('users@1.0.0')
class UsersService {
  constructor(
    private readonly repo: UsersRepository,
    private readonly logger: LoggerService,
  ) {}

  @Public()
  async findById(id: string) {
    this.logger.info('findById', { id });
    return this.repo.findById(id);
  }
}

@Module({
  imports:   [LoggerModule, ConfigModule],
  providers: [UsersService, UsersRepository],
  exports:   [UsersService],
})
export class UsersModule {}
```

Read the structure top to bottom:

- `UsersRepository` is module-private. It holds an injected
  `ConfigService` (visible because we imported `ConfigModule`).
- `UsersService` is exported. It depends on `UsersRepository` (same
  module) and `LoggerService` (imported).
- `UsersModule` exports `UsersService` so other modules that import
  `UsersModule` can inject it.

## The root module

One module is special: the one you pass to `Application.create`. It is
the **root**. The root module's transitive `imports` form the entire
container.

A root module typically has no providers of its own — it just composes
the others:

```typescript
@Module({
  imports: [
    AuthModule.forRoot({ jwt: { secret: env.JWT_SECRET } }),
    DatabaseModule.forRoot({ url: env.DATABASE_URL }),
    UsersModule,
    OrdersModule,
    PaymentsModule,
  ],
})
export class AppModule {}

const app = await Application.create(AppModule);
```

This is a strong convention. It means the root module is the
"manifest" of your application — open `AppModule` and you see what
the app is made of.

## Static vs dynamic modules

The example above is a **static module** — it has fixed structure
declared at class-definition time. Most modules in your codebase will
be static.

A **dynamic module** is created by a static `forRoot()` /
`forFeature()` method that returns a `DynamicModule`. Dynamic modules
take configuration:

```typescript
DatabaseModule.forRoot({ url: env.DATABASE_URL, dialect: 'postgres' })
```

See [Dynamic Modules](/docs/titan/modules-system/dynamic-modules) for details.

## Module identity

Modules are identified by **class reference**, not by name. Importing
the same module class twice is a no-op; the framework deduplicates.
Importing two *different* classes that both export the same provider
class is a conflict — the container will throw at registration time
with a clear error.

This means you cannot have two `UsersModule`s in the same
application. If you want to compose two unrelated services that both
manage "users", give the modules different names.

## Designing well

A few rules of thumb that pay off as the codebase grows:

### One module per bounded context

Group services by the domain concept they serve, not by the technical
layer. `UsersModule` contains `UsersService`, `UsersRepository`, and
`UsersValidator`. It does **not** contain `EmailService` even if
users module sends welcome emails — that goes in `NotificationsModule`,
which `UsersModule` imports.

A module that grows past 5–10 providers is usually two modules
masquerading as one. Split.

### Keep `exports` minimal

If a provider does not need to be visible outside the module, do not
export it. Smaller export surfaces are easier to refactor.

### Avoid circular imports

If `A` imports `B` and `B` imports `A`, both modules need to be
restructured. The framework will detect this at boot and throw with a
clear cycle path. Common fixes:

- Extract a third module that both `A` and `B` depend on.
- Use a shared interface in a separate module that both implement.

See [Circular Dependencies](/docs/titan/di/circular-dependencies).

### Bundle related infra modules with `forRoot`

`AuthModule.forRoot({ jwt: { … } })` is better than asking every
caller to wire `JwtSigner` and `JwtVerifier` themselves. Provide a
high-level entry point that composes the low-level providers.

---

---
id: "titan/modules-system/dynamic-modules"
title: "Dynamic Modules"
description: "Modules that take configuration — forRoot, forFeature, async factories."
---

# Dynamic Modules

A dynamic module is a module produced by a function. Use them when
the module's providers depend on configuration the caller supplies.

## The shape

A dynamic module returns a `DynamicModule` descriptor:

```typescript
import { type DynamicModule, Module } from '@omnitron-dev/titan';

@Module({})
export class CacheModule {
  static forRoot(options: ICacheOptions): DynamicModule {
    return {
      module: CacheModule,
      providers: [
        { provide: CACHE_OPTIONS, useValue: options },
        CacheService,
      ],
      exports: [CacheService],
      global: false,
    };
  }
}
```

A `DynamicModule` is a runtime value; the static `@Module({})`
decorator on the class is only there so the class is recognised as a
module.

## `forRoot` vs `forFeature`

A widely-followed convention:

- **`forRoot(options)`** — called once per app, in the root module.
  Configures the module globally.
- **`forFeature(options)`** — called multiple times. Returns a
  feature-specific slice, scoped to the importing module.

```typescript
@Module({
  imports: [
    DatabaseModule.forRoot({ url: env.DATABASE_URL }),     // once, global
    UsersModule,                                            // imports DatabaseModule.forFeature internally
    OrdersModule,                                           // imports DatabaseModule.forFeature internally
  ],
})
export class AppModule {}

// Inside UsersModule:
@Module({
  imports: [
    DatabaseModule.forFeature({ entities: [User, UserSession] }),
  ],
  providers: [UsersService],
})
export class UsersModule {}
```

Not every dynamic module needs both. `forRoot` alone is fine for most
cases.

## Async dynamic modules — `forRootAsync`

Use when the configuration itself comes from async sources (a remote
config service, a secret manager, an environment-specific file):

```typescript
@Module({})
export class CacheModule {
  static forRootAsync(options: IAsyncCacheOptions): DynamicModule {
    return {
      module: CacheModule,
      imports: options.imports ?? [],
      providers: [
        {
          provide: CACHE_OPTIONS,
          useFactory: options.useFactory,
          inject:     options.inject ?? [],
        },
        CacheService,
      ],
      exports: [CacheService],
    };
  }
}
```

Caller side:

```typescript
@Module({
  imports: [
    CacheModule.forRootAsync({
      imports: [ConfigModule],
      inject:  [ConfigService],
      useFactory: (config: ConfigService) => ({
        tier: config.get('cache.tier'),
        max:  config.get('cache.max'),
      }),
    }),
  ],
})
export class AppModule {}
```

The `useFactory` is called once during bootstrap with the resolved
`inject` dependencies. Whatever it returns becomes the value bound
to `CACHE_OPTIONS`.

## Global modules

Setting `global: true` on a `DynamicModule` makes its exports visible
to every module in the application, no `imports` required:

```typescript
return {
  module: ConfigModule,
  providers: [ConfigService],
  exports: [ConfigService],
  global: true,
};
```

Use sparingly — global modules are framework infrastructure
(`ConfigModule`, `LoggerModule`). Domain modules should require
explicit imports.

## Composing options across `forRoot` calls

Some modules need composition (e.g. multiple JWT secrets, multiple
database connections). The pattern is `forRoot` once, `forFeature`
many times:

```typescript
DatabaseModule.forRoot({ default: 'postgres' })

DatabaseModule.forFeature({ name: 'analytics', dialect: 'clickhouse' })
DatabaseModule.forFeature({ name: 'reports',   dialect: 'sqlite'     })
```

The implementation usually:

- `forRoot` registers the *default* connection and the connection
  registry.
- `forFeature` registers an *additional* connection in the registry
  and exposes a feature-scoped service that uses it.

## Authoring a dynamic module

A canonical template:

```typescript
import { Module, type DynamicModule } from '@omnitron-dev/titan';
import { createToken } from '@omnitron-dev/titan/nexus';

export interface IMyModuleOptions {
  apiKey: string;
  timeout?: number;
}

// Hand-write the async-options shape, or use ModuleAsyncOptions from
// '@omnitron-dev/titan/nexus'. There is no IAsyncOptions / createOptionsToken
// export — use createToken<T>(name) for the options token.
export interface IMyModuleAsyncOptions {
  imports?: any[];
  useFactory: (...args: any[]) => IMyModuleOptions | Promise<IMyModuleOptions>;
  inject?: any[];
}

const MY_MODULE_OPTIONS = createToken<IMyModuleOptions>('MyModuleOptions');

@Module({})
export class MyModule {
  static forRoot(options: IMyModuleOptions): DynamicModule {
    return {
      module: MyModule,
      providers: [
        { provide: MY_MODULE_OPTIONS, useValue: options },
        MyService,
      ],
      exports: [MyService],
    };
  }

  static forRootAsync(options: IMyModuleAsyncOptions): DynamicModule {
    return {
      module: MyModule,
      imports: options.imports ?? [],
      providers: [
        {
          provide: MY_MODULE_OPTIONS,
          useFactory: options.useFactory,
          inject:     options.inject ?? [],
        },
        MyService,
      ],
      exports: [MyService],
    };
  }
}

@Injectable()
export class MyService {
  constructor(@Inject(MY_MODULE_OPTIONS) private readonly options: IMyModuleOptions) {}
}
```

This template scales: every ecosystem module in `@omnitron-dev/titan-*`
is built from the same skeleton.

## Anti-patterns

- **Configuration in module-level constants.** If your module reads
  `process.env.X` at file load time, you have lost the configurability
  dynamic modules give you. Read config inside `forRoot` (so the
  caller can override) or through `ConfigService`.
- **Mutating options after `forRoot`.** Options become a `useValue`
  provider; they are frozen at registration time. To change config,
  use the [Configuration](/docs/titan/configuration/overview) module's
  hot-reload.
- **Mixing `forRoot` and `forFeature` semantics.** If your module
  does not need feature-scoped configuration, do not invent a
  `forFeature` — it adds API surface for no benefit.

---

---
id: "titan/modules-system/module-discovery"
title: "Module Discovery"
description: "How modules are found, loaded, and ordered."
---

# Module Discovery

Module discovery is **explicit**. Titan does not scan the filesystem
for modules at boot. Every module the application loads is reachable
from the root module's `imports` graph.

## The discovery walk

When `Application.create(RootModule)` runs:

1. The root module is added to the registry.
2. Each `import` of the root module is walked.
3. For each imported module, its `import`s are walked.
4. Discovery continues breadth-first until no new modules appear.
5. Each unique module is registered exactly once.

The result is a topologically sorted module list. Providers are
registered with the container in this order so dependencies precede
dependents.

## Why this is explicit

The alternative — automatic filesystem scanning — has three serious
problems:

- **Surprise.** Adding a file in the wrong directory loads code
  silently. Renaming a file changes the runtime in ways `git diff`
  does not show.
- **Test fragility.** Tests that load a partial module set diverge
  from production at boot.
- **Slow startup.** Filesystem walks scale poorly; explicit imports
  are O(actually-loaded-modules), not O(repo-size).

The explicit graph is a few extra lines of `imports: [...]` in
exchange for a runtime that is exactly what your code says.

## The registered set

After discovery, the registry holds:

| Property                             | Type         |
| ------------------------------------ | ------------ |
| Module class reference               | `Function`   |
| Module name (defaults to class name) | `string`     |
| Module version (optional)            | `string`     |
| Static or dynamic flag               | `boolean`    |
| Resolved imports (after dedup)       | `IModule[]`  |
| Resolved providers                   | `Provider[]` |
| Resolved exports                     | `Token[]`    |

Inspect via the `module:registered` event or programmatically through
`app.modules`.

## Detecting cycles

The discovery walk detects cycles between modules. A cycle throws at
boot with the full cycle path:

```
Circular module dependency detected: AuthModule -> SessionModule -> UsersModule -> AuthModule
```

(Module-import cycles surface as a `TitanError`. Provider-level
dependency cycles within the DI container are a separate
`CircularDependencyError` from Nexus.)

The fix is structural — extract the shared piece into a third module,
or invert one of the dependencies (often via a callback or interface).
See [Circular Dependencies](/docs/titan/di/circular-dependencies).

## Lazy loading

Some applications want to load modules on demand (admin features that
ship in production but should not run on every boot, optional
integrations enabled by feature flag).

Titan does not have built-in lazy module loading at runtime. The
preferred patterns are:

- **Conditional imports.** Build the root module's `imports` array
  conditionally at boot:

  ```typescript
  const imports = [CoreModule, UsersModule];
  if (env.ENABLE_BILLING) imports.push(BillingModule);

  @Module({ imports })
  export class AppModule {}
  ```

- **Conditional providers.** Inside a module, register providers
  only if a config flag is set, using the `ConditionalProvider`
  type from Nexus:

  ```typescript
  @Module({
    providers: [
      {
        provide:   BillingService,
        useClass:  BillingService,
        condition: (ctx) => ctx.container.resolve(ConfigService).get('billing.enabled'),
        fallback:  { useClass: NoopBillingService },
      },
    ],
  })
  export class BillingModule {}
  ```

  The conditional fields are `condition` (a predicate over the
  `ResolutionContext`) and `fallback` (a provider used when the
  condition is false). The `createConditionalProvider` helper from
  `@omnitron-dev/titan/nexus` produces the same shape.

These are decided at `Application.create` time. Truly dynamic
load/unload at runtime is a multi-app concern; use the Omnitron
orchestrator for that.

See [dependency injection](/docs/titan/di/overview) for provider registration and resolution.

---

---
id: "titan/netron/authentication"
title: "Authentication"
description: "AuthenticationManager, AuthorizationManager, BuiltInPolicies, AuthConfig."
---

# Authentication

> **Info**
>
> This page is the Netron-layer reference for `AuthConfig`,
> PolicyEngine, and the `BuiltInPolicies` library. For the
> end-to-end authorisation surface (permission grammar,
> per-user overrides, RLS bridge, audit), start at
> [Authentication & Authorisation](/docs/auth). For
> the higher-level façade decorators that compile to the
> `@Auth({policies})` form below, see [Permissions →
> Scopes](/docs/auth/permissions#scopes).

Netron separates **authentication** (who is the caller?) from
**authorisation** (is this caller allowed to do this?). Two
managers, one shared policy engine, composable policies.

Import path:

```typescript
import {
  AuthenticationManager,
  AuthorizationManager,
  PolicyEngine,
  BuiltInPolicies,
} from '@omnitron-dev/titan/netron/auth';
```

## The two managers

| Manager                 | Job                                           |
| ----------------------- | --------------------------------------------- |
| `AuthenticationManager` | Validate credentials, produce an auth context |
| `AuthorizationManager`  | Evaluate policies against the auth context    |

```mermaid
flowchart LR
  Req[Incoming RPC call]
  Req --> Authn[AuthenticationManager<br/>validate JWT / token]
  Authn -->|valid| Ctx[AuthContext<br/>userId, roles, scopes]
  Authn -->|invalid| F1[TitanError<br/>UNAUTHORIZED / 401]
  Ctx --> Authz[AuthorizationManager<br/>evaluate policies]
  Authz -->|allowed| Method[Method body]
  Authz -->|denied| F2[TitanError<br/>FORBIDDEN / 403]
```

These are wired by the `titan-auth` ecosystem module (or by your own
integration). You rarely construct them directly.

## One invocation decision across transports

HTTP and the persistent Netron transports use the same authorization decision:
registered service ACLs, then declared method requirements, then optional
`authDefaultDeny`. A missing authentication or authorization manager never disables
an explicitly protected method. Configure the required identity and policy providers;
requests without sufficient authority are refused before the method executes.

With `authDefaultDeny: true`, a published method needs an ACL or an explicit
authorization declaration. A bare `@Public()` is exposure, not permission. Declare
intentional anonymous access explicitly. An unavailable service instance or missing
authorization metadata produces a service-unavailable error rather than bypassing
checks. The HTTP pipeline requires the peer logger during configuration so failed
checks have the normal diagnostic path.

Service selection happens before authorization. The exact published service identity
is used for ACL and policy evaluation, including batch HTTP calls. Selecting a version
never evaluates another version's permissions.

## Forwarding through a gateway

Publishing a remote `Interface` at another Netron endpoint creates a delegation
boundary. Register an `AuthorizationManager` ACL for the forwarded service or
method at that gateway. Netron checks the actual incoming caller against that
ACL before forwarding; the gateway's upstream credentials do not grant access
to its clients. Missing ACLs refuse the request, including anonymous requests
and HTTP authentication exemptions. ACL revocation applies to subsequent calls
and property operations on existing connections.

The upstream service still applies its own guards. Definitions carry transport
restrictions, not executable authorization policies, so the gateway must declare
its own delegation policy. Trusted local calls through an interface retain their
in-process contract. Prefer narrow method grants and explicit service versions.

## Auth context

Every authenticated call carries an auth context with:

```typescript
{
  userId:       string;
  roles:        string[];
  permissions:  string[];
  scopes?:      string[];                // optional
  token?:       { /* … */ };             // optional, integration extras
  metadata?:    Record<string, any>;     // optional
}
```

This is what `AuthConfig` policies match against. (`userId`, `roles`,
and `permissions` are required; `scopes` is optional.)

## `@Auth(config)` — configuring a method

`AuthConfig` (from `@omnitron-dev/titan/decorators`):

```typescript
interface AuthConfig {
  roles?:        string[];     // ANY role grants access
  permissions?:  string[];     // ALL required
  scopes?:       string[];     // ALL OAuth2 scopes required
  policies?:     string[] | { all: string[] } | { any: string[] } | PolicyExpression;
  allowAnonymous?: boolean;
  inherit?:      boolean;      // inherit class-level policies
  override?:     boolean;      // override class-level policies
}
```

Usage:

```typescript
import { Public, Auth } from '@omnitron-dev/titan/decorators';

@Service('orders@1.0.0')
class OrdersService {
  @Public()
  @Auth({ scopes: ['orders:read'] })
  async list() { /* … */ }

  @Public()
  @Auth({ roles: ['admin'] })
  async deleteAll() { /* … */ }

  @Public()
  @Auth({ allowAnonymous: true })          // explicit anonymous
  async ping() { /* … */ }

  @Public()
  @Auth({ policies: { any: ['policy:admin', 'policy:resource-owner'] } })
  async modifyResource(id: string) { /* … */ }
}
```

The same configuration is also accepted inline through `@Public`:

```typescript
@Public({ auth: { scopes: ['orders:read'] } })
async list() { /* … */ }
```

Pick one style per project.

## `BuiltInPolicies` — reusable policy definitions

The `BuiltInPolicies` namespace produces policy definitions you can
register with the policy engine:

```typescript
import { BuiltInPolicies } from '@omnitron-dev/titan/netron/auth';

const policies = [
  BuiltInPolicies.requireRole('admin'),
  BuiltInPolicies.requireAnyRole(['admin', 'support']),
  BuiltInPolicies.requireAllRoles(['user', 'verified']),
  BuiltInPolicies.requirePermission('users:read'),
  // …~18 helpers in the source: requireAnyPermission, requireScope,
  // requireResourceOwner, requireIP, requireRateLimit, …
];

// Register them with the engine:
policyEngine.registerPolicies(policies);   // array form; single: registerPolicy(p)
```

Reference by name in `AuthConfig`:

```typescript
@Auth({ policies: ['role:admin'] })
async deleteAll() { /* … */ }
```

The policy name follows the convention shown in the helper
(`role:admin`, `role:any:admin,support`, etc.) — the registration
sets the name based on the helper's logic.

## Custom policies

A policy is a `PolicyDefinition`:

```typescript
import type { PolicyDefinition } from '@omnitron-dev/titan/netron/auth';

const IsResourceOwner: PolicyDefinition = {
  name:        'resource:owner',
  description: 'Caller must own the resource referenced by the first argument',
  evaluate: async (context) => {
    const resourceId = context.method?.args?.[0];      // method args live here
    const resource   = await ordersRepo.findById(resourceId);  // close over your deps
    const allowed    = resource?.userId === context.auth?.userId;
    return {
      allowed,
      reason: allowed ? 'Owner' : 'Not owner',
    };
  },
};

policyEngine.registerPolicies([IsResourceOwner]);

// Then in your service:
@Public()
@Auth({ policies: ['resource:owner'] })
async getOrder(orderId: string) { /* … */ }
```

The context passed to `evaluate` is an `ExecutionContext`
(`{ auth?, service?, method?: { name, args }, resource?, environment?, request? }`).
Note there is **no** `context.args` (use `context.method?.args`) and
**no** DI hook on the context — close over the dependencies your
policy needs. The exact shapes live in `netron/auth/policy-engine.ts`
and `netron/auth/types.ts`.

## Class-level vs method-level

Both work; method-level overrides class-level for that method:

```typescript
@Service('orders@1.0.0')
@Auth({ scopes: ['orders:*'] })                     // class-level default
class OrdersService {
  @Public()                                          // inherits class-level
  async list() { /* … */ }

  @Public()
  @Auth({ allowAnonymous: true })                    // override
  async listPublic() { /* … */ }

  @Public()
  @Auth({ roles: ['admin'] })                        // override
  async deleteAll() { /* … */ }
}
```

Use `inherit` / `override` flags in `AuthConfig` to control
combination semantics for fine-grained cases.

## Install HTTP authority before listening

Configure native authentication and cookie security before starting the HTTP
transport. Do not reach into a transport's private middleware pipeline from an
`afterStart` hook: the socket can already accept requests at that point.

```typescript
import { CookieTokenTransport, CsrfManager } from '@omnitron-dev/titan/netron/auth';

const csrf = new CsrfManager({
  headerName: 'X-CSRF-Token',
  cookie: { name: 'app_csrf', path: '/', secure: true, sameSite: 'Lax' },
});
netron.configureAuth(authenticationManager, authorizationManager, {
  tokenTransport: new CookieTokenTransport({
    accessCookie: { name: 'app_session', path: '/', httpOnly: true, secure: true, sameSite: 'Lax' },
    csrf,
  }),
  httpSecurity: {
    authenticateEndpoint: 'disabled', // Auth.login owns the application's sign-in flow
    origin: { allowedOrigins: ['https://app.example'], alwaysEnforce: true },
    csrf: { csrf, accessCookieName: 'app_session', exempt: ['Auth.login'] },
  },
});
```

The HTTP server installs these gates during peer binding, before `listen`, for
both single and batch invocations. Configuring HTTP security after transport
startup is rejected. Exempting an initial login from double-submit CSRF does not
exempt it from the Origin gate.

`httpSecurity.authenticateEndpoint` controls the built-in `/netron/authenticate`
endpoint. It defaults to `'enabled'` for existing clients. Applications with their
own native sign-in service should set it to `'disabled'`: the built-in route then
returns 404 without executing the credential provider or returning an auth context.
When enabled, it enforces the configured Origin policy even when no cookie is sent.
The RPC CSRF exemption list does not exempt this endpoint from Origin validation.

The Node HTTP transport rejects declared or streamed request bodies above
`maxRequestSize` with HTTP 413 before invoking a service. Rejection must not escape
the Node request callback or terminate the server. A client disconnect also cancels
a pending response stream, including when the response becomes ready after the
connection has already closed.

The credential verifier can resolve an opaque session against a database; a JWT
is not required. Disable the authentication token cache if role/session revocation
must affect the very next request and there is no explicit cache-invalidation
protocol. Sensitive state transitions should recheck session ownership inside
the same transaction that changes the credential.

`issueTokens` and `clearTokens` publish cookies through the native request scope.
Never accept token-issuance metadata or `authContext` from a client. The HTTP
transport reconstructs authority from the verified credential. A service using
`withAuthRLSContext` can read the current server scope through
`getCurrentRLSAuthContext()`; anonymous invocations must receive their own empty
scope rather than inherit an enclosing actor.

## Verification failure versus service failure

A rejected credential and an unavailable credential verifier have different
meanings. Return the ordinary invalid-credential outcome for a token that the
verifier successfully checked and refused. Throw `AuthenticationUnavailableError`
for a database, identity-provider or equivalent verification infrastructure failure.
The native authentication timeout uses the same unavailable result.

```typescript
import { AuthenticationUnavailableError } from '@omnitron-dev/titan/netron/auth';

async function validateToken(token: string) {
  let session;
  try { session = await sessions.resolve(token); }
  catch (cause) {
    throw new AuthenticationUnavailableError(cause instanceof Error ? cause : undefined);
  }
  if (!session) throw new Error('invalid_session');
  return session.authContext;
}
```

An unavailable verifier returns HTTP 503 with stable error code
`authentication_unavailable`; it does not execute an anonymous handler or clear
cookies. This distinction is preserved in single requests and both sequential
and parallel batches. It lets an editor retain its draft during an outage without
granting authority from a cached browser identity. The public response omits the
underlying database/provider error.

## Failure semantics

The Netron enforcement path throws a plain `TitanError` carrying an
error **code** (not the `AuthError`/`PermissionError` subclasses):

| Failure                         | Thrown       | Code / Status        |
| ------------------------------- | ------------ | -------------------- |
| No credentials                  | `TitanError` | `UNAUTHORIZED` / 401 |
| Invalid token                   | `TitanError` | `UNAUTHORIZED` / 401 |
| Authenticated but policy denied | `TitanError` | `FORBIDDEN` / 403    |

Discriminate on the **code**, not the class — e.g.
`err.code === ErrorCode.UNAUTHORIZED` vs `ErrorCode.FORBIDDEN` (or the
401/403 HTTP status). The `AuthError` / `PermissionError` classes do
exist in the errors module (both extend `HttpError` extends
`TitanError`), but the auth layer does not instantiate them, so
`instanceof AuthError` will not match a thrown auth error.

## Anti-patterns

- **Auth checks in method bodies.** Defeats the policy framework.
  Use `@Auth(...)`. Inline checks duplicate the policy and drift
  from the rest of the codebase.
- **Single super-scope (`'admin'`).** A flat scope model collapses
  under growth. Prefer hierarchical scopes (`orders:*`,
  `orders:read`, `orders:write`) so you can grant least privilege.
- **Auth as the only check.** Auth says *who* can call. It does
  not say *whether the input is valid* or *whether the resource is
  in the right state*. Combine with `@Validate` and domain checks.

---

---
id: "titan/netron/middleware"
title: "RPC Middleware"
description: "Per-call wrapping for cross-cutting concerns."
---

# RPC Middleware

RPC middleware wraps Netron calls over HTTP. It runs on the server
side, around dispatch. Use it for cross-cutting concerns that apply
uniformly — auth checks, rate limiting, tracing, logging, metrics.

Middleware is **an HTTP-transport feature**. The pipeline lives in
`netron/transport/http/middleware/`, and the real surface is the
`MiddlewarePipeline` class (`MiddlewarePipeline` in `netron/transport/http/middleware/pipeline.ts`) — there is no
`INetronMiddleware` interface and no `netron.use()` method. The
persistent transports (WebSocket / TCP / Unix) do **not** instantiate
this pipeline. They call the same invocation authorization function at packet
dispatch that the HTTP authentication middleware uses: registered ACLs, declared
method requirements and optional default-deny. The invocation wrapper receives
the resulting authority context. So the patterns on this page apply when you serve
over HTTP. (The transport-agnostic *context* type used below,
`NetronMiddlewareContext`, is defined once in
`NetronMiddlewareContext` in `netron/transport/http/middleware/types.ts` and reused by the HTTP-specific extension.)

> Do not confuse with **DI middleware**, which wraps container
> resolution. See [DI Middleware](/docs/titan/di/middleware) for the
> distinction.

## The middleware function

A middleware is a plain function, not a class with `handle()`:

```typescript
import {
  type MiddlewareFunction,
  type NetronMiddlewareContext,
  MiddlewareStage,
} from '@omnitron-dev/titan/netron/transport/http/middleware';

const timing: MiddlewareFunction = async (ctx, next) => {
  const t0 = performance.now();
  await next();                       // returns void; result lives on ctx.result
  record(ctx.serviceName, ctx.methodName, performance.now() - t0);
};
```

`ctx` (`NetronMiddlewareContext`) carries:

- `peer` — the `LocalPeer | RemotePeer` handling the call.
- `serviceName` / `methodName` — what's being invoked.
- `input` — the call input; `result` — the return value (set around `next()`).
- `error` — set if dispatch threw.
- `metadata` — a `Map<string, unknown>` for per-call data; the auth
  context and authorization header live here
  (`ctx.metadata.get('authContext')`).
- `timing` — `{ start, middlewareTimes }`.
- `skipRemaining` — set `true` to short-circuit the rest of the stage.

The HTTP-specific `HttpMiddlewareContext` extends this with `request`,
`response`, `route`, `params`, `query`, `body`, `cookies`.

`next()` invokes the next middleware (or dispatch). It returns
`Promise<void>` — there is no result to return; read/write `ctx.result`
instead.

## Registering middleware

Middleware is registered on the HTTP server's `MiddlewarePipeline`,
not on `Netron`. The pipeline exposes:

```typescript
pipeline.use(fn, config?, stage?);                    // global
pipeline.useForService('users@1.0.0', fn, config?);   // one service
pipeline.useForMethod('users@1.0.0', 'findById', fn); // one method
```

Each registration takes an optional `MiddlewareConfig`
(`{ name, priority?, condition?, onError?, services?, methods? }`) and
a `MiddlewareStage`. In practice the HTTP server wires its own
defaults (request-id, token extraction, the `NetronAuthMiddleware`);
app bootstrap code adds custom middleware where it constructs the
transport.

## Stages and order

The pipeline runs in five **stages** (`MiddlewareStage`), and within a
stage middleware is ordered by `priority` (lower runs first, default
`100`):

```
PRE_PROCESS → PRE_INVOKE → [dispatch] → POST_INVOKE → POST_PROCESS
                                                          ↘ ERROR (on throw)
```

- **`PRE_PROCESS`** — earliest; the HTTP server registers request-id
  and token extraction here.
- **`PRE_INVOKE`** — before the method runs; auth/authorization
  (`NetronAuthMiddleware`), rate limiting, validation belong here.
- **`POST_INVOKE` / `POST_PROCESS`** — after dispatch; response
  shaping, compression, logging.
- **`ERROR`** — runs when a stage throws.

Put cheaper checks before expensive ones (rate-limit lookup before a
heavy parse) by giving them a lower `priority`.

## A custom middleware — timing

```typescript
import { type MiddlewareFunction, MiddlewareStage } from '@omnitron-dev/titan/netron/transport/http/middleware';

const timingMiddleware: MiddlewareFunction = async (ctx, next) => {
  const t0 = performance.now();
  try {
    await next();
    metrics.histogram('rpc.duration_ms', {
      service: ctx.serviceName,
      method:  ctx.methodName,
      outcome: 'ok',
    }).observe(performance.now() - t0);
  } catch (e) {
    metrics.histogram('rpc.duration_ms', {
      service: ctx.serviceName,
      method:  ctx.methodName,
      outcome: 'error',
    }).observe(performance.now() - t0);
    throw e;
  }
};

pipeline.use(timingMiddleware, { name: 'timing', priority: 5 }, MiddlewareStage.PRE_INVOKE);
```

Titan also ships ready-made factories on `NetronBuiltinMiddleware`
(metrics, logging, circuit-breaker, retry, caching, …) and
`HttpBuiltinMiddleware` (CORS, compression, body parsing, security
headers).

## Per-method policy via decorators

Method decorators (`@Auth`, `@RateLimit`, `@Cache`) are **not**
middleware classes — they stamp `reflect-metadata` onto the method
(the `Auth` decorator in `decorators/core.ts`, `:805`, `:822`, storing `METHOD_AUTH` /
`METHOD_RATE_LIMIT` / `METHOD_CACHE`). The auth middleware (and, on
WS/TCP, the inline dispatch path) reads that metadata and enforces it:

```typescript
@Public()
@Auth({ roles: ['admin'], scopes: ['write:users'] })
@RateLimit({ limit: 5, window: 60_000 })
async dangerousOp() { /* … */ }
```

The same policy can be expressed inline on `@Public` itself
(`@Public({ auth: { roles: ['admin'] }, rateLimit: { limit: 5, window: 60_000 } })`);
the standalone decorators just keep the concerns visually separate.

Either way, enforcement runs *inside* the auth/rate-limit step of the
pipeline (or inline on WS/TCP) — there is no separate per-method
middleware instance. The global step does the heavy lifting (validate
token, resolve user); the decorator metadata supplies the specific
policy.

## Scoping middleware to specific methods

There is no `@Skip` decorator. To limit (or exclude) middleware, use
the registration filters on `MiddlewareConfig` — `services` /
`methods` (string list or `RegExp`) and `condition`:

```typescript
pipeline.use(authMiddleware, {
  name: 'auth',
  methods: /^(?!ping$).*/,        // everything except `ping`
});
```

The built-in auth middleware (`createAuthMiddleware` /
`NetronAuthMiddleware`) also accepts
`skipServices` / `skipMethods` options (`AuthMiddlewareOptions`), and any
middleware can set `ctx.skipRemaining` to short-circuit the rest of its
stage (honoured by the pipeline at `ctx.skipRemaining` in `MiddlewarePipeline`).

## Modifying the request

Middleware can mutate `ctx.input` before calling `next()` — for
example, normalising input or injecting derived data:

```typescript
const trim: MiddlewareFunction = async (ctx, next) => {
  if (Array.isArray(ctx.input)) {
    ctx.input = ctx.input.map(a => typeof a === 'string' ? a.trim() : a);
  }
  await next();
};
```

Mutate sparingly. Per-method validation is a better fit for
content rules; middleware should handle cross-cutting transforms
(normalisation, trimming, conversion to canonical form).

## Modifying the response

`next()` runs dispatch and populates `ctx.result`; transform it after:

```typescript
const stamp: MiddlewareFunction = async (ctx, next) => {
  await next();
  if (typeof ctx.result === 'object' && ctx.result !== null) {
    ctx.result = { ...ctx.result, _serverTimeMs: Date.now() };
  }
};
```

Useful for response envelopes, version stamping, redaction.

## Error interception

Dispatch errors surface as a throw from `next()` (and are also set on
`ctx.error`). Common pattern:

```typescript
const mapErrors: MiddlewareFunction = async (ctx, next) => {
  try {
    await next();
  } catch (e) {
    if (e instanceof NotFoundError) {
      // Convert framework error to project-specific shape.
      throw new ProjectNotFoundError(e.message, { /* … */ });
    }
    throw e;
  }
};
```

Avoid swallowing. Errors that middleware suppresses become silent
failures; the client sees a successful response that has no real
result.

## Anti-patterns

- **Calling `next()` more than once.** The result is undefined.
  Some middleware in the chain will run twice; client may receive
  two responses or a malformed one.
- **Skipping `next()` based on a condition.** If you bypass the
  method, you must produce a return value. Bypass is the right
  pattern for caching middleware (return a cached value); for
  guards, throw an error instead.
- **Long-running async work.** Every middleware extends the call
  duration. Keep middleware fast or run it asynchronously after
  `next()` returns (e.g. async logging).
- **Sharing state between middlewares via globals.** Use
  `ctx.metadata` — it's per-call and isolated.

---

---
id: "titan/netron/multi-backend"
title: "Multi-Backend"
description: "One client, many servers — pools, failover, method-level routing."
---

# Multi-Backend

A `MultiBackendClient` is a Netron client that fans out to multiple
servers. It exposes the same interface as a single-backend client; the
backend selection is internal. It is exported from
`@omnitron-dev/titan/netron/multi-backend`, and its option shape is the
`MultiBackendClientOptions` interface in
`packages/titan/src/netron/multi-backend/types.ts`.

```mermaid
flowchart LR
  Caller[Caller: users.findById]
  Client[MultiBackendClient]
  Router[ServiceRouter<br/>rule match]
  Pool[BackendPool<br/>health-aware]
  B1[backend-1]
  B2[backend-2]
  B3[backend-3]

  Caller --> Client --> Router
  Router -->|matched rule| Pool
  Pool -->|pick by strategy| B1
  Pool -.->|round-robin| B2
  Pool -.->|sticky / weighted| B3
  Pool -.->|fail → retry| Pool
```

Use when you have:

- **Read replicas.** Round-robin reads across replicas; writes go
  to the primary.
- **Sharded backends.** Route by request key (user ID, tenant ID).
- **Failover targets.** Healthy backends serve traffic; unhealthy
  ones drop out, then come back when probes succeed.

## The minimal example

Every backend needs a unique `id` and a `url`:

```typescript
import { MultiBackendClient } from '@omnitron-dev/titan/netron/multi-backend';

const client = new MultiBackendClient({
  backends: [
    { id: 'api-1', url: 'http://api-1.internal' },
    { id: 'api-2', url: 'http://api-2.internal' },
    { id: 'api-3', url: 'http://api-3.internal' },
  ],
  // strategy is set per-route (see below), not at the top level.
});

// `service(name)`, not `queryInterface` — that is the server-side Netron API.
// It returns a proxy whose every method call goes through `invoke()` and the
// router; there is no round trip to fetch a definition first.
const users = client.service<UsersService>('users@1.0.0');
const user  = await users.findById('u_42');     // routed to one of the three
```

If the call fails and `failover` is on (the default), the client
retries on another backend (up to `maxFailoverAttempts`, default 2).

## Strategies

The load-balancing strategy is a `LoadBalancingStrategy` (`LoadBalancingStrategy` in `types.ts`),
set on a route (`router.routes[].strategy`) or as
`router.defaultStrategy` (default `round-robin`). The router dispatches
on it in `ServiceRouter.selectBackend`:

| Strategy            | Behaviour                                                                                                                                                                                                                        |
| ------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `round-robin`       | Cycle through backends evenly (default)                                                                                                                                                                                          |
| `random`            | Pick a backend at random                                                                                                                                                                                                         |
| `least-connections` | Send to the backend with the fewest active connections                                                                                                                                                                           |
| `weighted`          | Intended to weight by `BackendConfig.weight`, but **currently falls back to random** — `selectWeighted` delegates to `selectRandom` (`selectWeighted`) because `weight` is not carried on the per-backend status it selects from |

Those four are the only values the type and the dispatch allow. There
is no `sticky`, `least-busy`, or `primary` strategy, and no
`stickyKey`/`sessionAffinity` hook. "Primary/fallback" semantics are
expressed through a route's `backends` vs `fallback` lists
(the route's `backends`/`fallback` handling in `ServiceRouter`), not a strategy name.

## Service-level routing rules

Routing matches on the **service name** (with `*` wildcards) — not on
method. Configure it under `router.routes`:

```typescript
new MultiBackendClient({
  backends: [
    { id: 'primary',  url: '…primary'  },
    { id: 'replica1', url: '…replica1' },
    { id: 'replica2', url: '…replica2' },
  ],
  router: {
    routes: [
      { service: 'orders@*', backends: ['primary'], fallback: ['replica1', 'replica2'] },
      { service: 'reports@*', backends: ['replica1', 'replica2'], strategy: 'round-robin' },
    ],
    defaultBackends: ['primary'],
    defaultStrategy: 'round-robin',
  },
});
```

A route targets backend **ids**. `fallback` ids are tried when the
primary `backends` are unavailable. To split reads vs writes by
method, expose them as separate services (e.g. `orders` and
`orders-read`) and route per service.

## Health monitoring

Health-check options are **flat** top-level fields (`MultiBackendClientOptions` in `types.ts`),
not a nested `health` object:

```typescript
new MultiBackendClient({
  backends: [...],
  healthChecks:        true,          // default true
  healthCheckInterval: 30_000,        // default 30s
  unhealthyThreshold:  3,             // consecutive failures → unhealthy (default 3)
  healthyThreshold:    2,             // consecutive successes → healthy (default 2)
});
```

There is no per-check `timeoutMs` field. The router filters out any
backend whose `health` is `'unhealthy'` before selecting
(in `ServiceRouter`, before selection), so unhealthy backends drop out of the
rotation; the pool keeps polling them and they re-enter once the
`healthyThreshold` of consecutive successes is met.

## Failover

`failover` is a **boolean** (default `true`), not an options object,
and `maxFailoverAttempts` (default 2) caps the retries:

```typescript
new MultiBackendClient({
  backends: [...],
  failover:            true,
  maxFailoverAttempts: 2,             // up to 2 extra backends tried
});
```

The failover loop (the catch in `MultiBackendClient`'s failover loop) retries on **any**
thrown error — there is no `retriableErrors` classifier hook, so the
catch block does not inspect the error before retrying. It will,
however, skip a candidate whose circuit breaker is open
(`isCircuitClosed`). A single call can opt out of failover
entirely via a per-request `hints.noFailover` (`hints.noFailover`).

For failure *isolation*, configure the **circuit breaker** (`circuitBreaker` in `types.ts`)
— this is the real mechanism for shedding a bad backend. It tracks
per-backend failures and transitions closed → open → half-open
(`recordFailure` / `isCircuitClosed`):

```typescript
circuitBreaker: {
  enabled:      true,
  threshold:    5,                    // failures inside the window (default 5)
  window:       60_000,               // error-tracking window (default 60s)
  resetTimeout: 30_000,               // wait before half-open (default 30s)
}
```

Only failures inside `window` count towards `threshold`; older ones age
out. A success while half-open closes the circuit and clears the
history. So the breaker measures a failure *rate* — three failures a
minute apart with a 60s window never coincide, and the circuit stays
closed.

## Connection model

There is no sized connection pool. The `BackendPool` holds one
`BackendClient` per backend id (`BackendPool`), and each
`BackendClient` holds at most one transport — a single
`HttpTransportClient` *or* a single `WebSocketConnection`
(`BackendClient`), chosen by the backend's `transport` field.
Per-backend `pool: { min, max, idleTimeoutMs }` is not a real option.

## Observability

Track per-call latency and the chosen backend (the exact metric shape
is up to your metrics layer — illustrative):

```typescript
metrics.histogram('rpc.duration_ms', {
  service: 'users@1.0.0',
  method:  'findById',
  backend: 'api-2',
}).observe(duration);
```

Useful for catching imbalanced traffic, slow backends, or
overlooked failover events.

## When not to use MultiBackend

- **One backend.** Connect a single peer
  (`netron.connect(url)` → `queryInterface`) — simpler.
- **Cross-region routing.** A client in one region routing across
  regions adds round-trip latency that a regional load balancer
  hides better. Put the multi-backend logic at the LB.
- **Stateful sessions.** WebSocket connections carry session state.
  A multi-backend client can re-route mid-session, which breaks
  state. For stateful sessions, pin to a backend.

## Anti-patterns

- **Routing a whole hot service to one backend.** A route that pins a
  high-traffic service to a single backend id forfeits spreading. Give
  busy services multiple backends with `round-robin` /
  `least-connections`.
- **Aggressive `unhealthyThreshold`.** Marking a backend unhealthy
  after one failure is fragile to noise. Three (the default) is a good
  baseline; raise it for very flaky networks.
- **Relying on failover to mask app errors.** Failover currently
  retries on any error, so a deterministic application error will be
  retried against every backend and still fail. Prefer the circuit
  breaker for infrastructure faults, and don't lean on failover to
  paper over 4xx-class errors.

---

---
id: "titan/netron/serialization"
title: "Serialization"
description: "msgpack as the wire format — types, custom codecs, when to override."
---

# Serialization

Netron serialises every payload (arguments, return values, errors)
into a compact binary form via `@omnitron-dev/msgpack` — an in-house,
`SmartBuffer`-based serializer that follows the msgpack format. It is
**not** the npm `@msgpack/msgpack` library and is **not**
wire-compatible with it. The serializer is binary, self-describing, and
shared across every runtime Titan targets. Netron's wire layer keeps a
single shared instance, `serializer`, in
`packages/titan/src/netron/packet/serializer.ts`; the core
encode/decode and the common-type registrations live in
`packages/msgpack/src/`.

## What the serializer handles natively

| Type                    | Wire form                         |
| ----------------------- | --------------------------------- |
| `null` / `undefined`    | nil                               |
| `boolean`               | true / false                      |
| `number`                | int (8/16/32/64) or float (32/64) |
| `string`                | utf-8 with length prefix          |
| `Buffer` / `Uint8Array` | bin (raw bytes)                   |
| Plain objects           | map                               |
| Arrays                  | array                             |

Anything else — `Date`, `BigInt`, `Map`, `Set`, custom classes —
is handled through *registered types* (below).

## Registered types Titan ships

A registered ("extension") type is a single-byte code paired with an
`encode`/`decode` callback. Codes are a single byte, `0..127`, and the
registration map (`registerCommonTypesFor` in `packages/msgpack/src/index.ts`) partitions the
range like this:

- **`119..126`** — standard library / runtime types (the common-type
  registrations).
- **`110..118`** — reserved (`TitanError` is registered here, at
  `110`, by the Titan wire layer).
- **`100..109`** — Netron's own proxy/stream wiring.
- **`1..99`** — free for your own custom types.

The common types registered by
`registerCommonTypesFor` (`packages/msgpack/src/index.ts`):

| Type     | Code  | Wire form                                                       |
| -------- | ----- | --------------------------------------------------------------- |
| `Long`   | `119` | unsigned flag + 64-bit int (the `long` package)                 |
| `BigInt` | `120` | encoded as a decimal string                                     |
| `RegExp` | `121` | `source` + `flags`                                              |
| `Set`    | `123` | size prefix + each value                                        |
| `Map`    | `124` | size prefix + each `key`, `value`                               |
| `Date`   | `125` | epoch millis as `UInt64` (not ISO)                              |
| `Error`  | `126` | std-error id + `name` / `stack` / `message` + own custom fields |

(`122` is reserved and unused.)

The Titan wire layer
(`packages/titan/src/netron/packet/serializer.ts`) registers the rest:

| Type              | Code  | Notes                                                                   |
| ----------------- | ----- | ----------------------------------------------------------------------- |
| `TitanError`      | `110` | Full structured error, preserving class + code                          |
| `Definition`      | `109` | Service definition metadata (internal)                                  |
| `Reference`       | `108` | Service reference (internal proxy wiring)                               |
| `StreamReference` | `107` | Stream handle (internal; see [Streaming](/docs/titan/netron/streaming)) |

`TitanError` (code `110`) is what makes typed errors travel across the
wire intact. It is registered **before** the common types
(`packages/titan/src/netron/packet/serializer.ts`) precisely so its handler is checked ahead of the
generic `Error` handler (`126`) — `TitanError` extends `Error`, so the
more specific codec must win. On decode it reconstructs the matching
error with its `code`, `message`, tracing fields, `cause` chain, and
any subclass-specific fields (`serviceId`, `requiredPermission`,
`retryAfter`, …). Stack traces are **omitted on the wire by default**
(`includeStackTraces` in `packages/titan/src/netron/packet/serializer.ts`, security policy T#38) to avoid leaking
server-internal paths; dev tooling can opt in via
`setSerializerErrorOptions({ includeStackTraces: true })`.

## Custom types

For domain types you want to send as themselves (preserving class
identity on the receiver), register a type on the shared `serializer`.
The signature is **positional**:

```
serializer.register(typeCode, constructor, encode, decode)
```

(`Serializer.register`, `packages/msgpack/src/serializer.ts`). `typeCode` must be in
`0..127` or `register` throws a `RangeError`; pick one in the
**`1..99`** user band, since `100..126` are reserved (see the tables
above). The `encode` callback is `(value, buf) => void` and the
`decode` callback is `(buf) => value` — neither returns bytes. `buf` is
a `SmartBuffer`: write fields with `serializer.encode(field, buf)` and
read them back, in the **same order**, with `serializer.decode(buf)`
(or use the buffer's typed primitives directly, like the built-in
`Date` codec does with `writeUInt64BE`/`readUInt64BE`).

```typescript
import { serializer } from '@omnitron-dev/titan/netron';

class Money {
  constructor(public readonly amount: bigint, public readonly currency: string) {}
}

serializer.register(
  32,                                       // user-defined code in the 1..99 band
  Money,
  (m: Money, buf) => {                      // encode: write fields into buf
    serializer.encode(m.amount.toString(), buf);
    serializer.encode(m.currency, buf);
  },
  (buf) => {                                // decode: read fields back, same order
    const amount   = serializer.decode(buf) as string;
    const currency = serializer.decode(buf) as string;
    return new Money(BigInt(amount), currency);
  },
);
```

Both peers must register the same constructor against the same code,
with codecs that agree on field order. If only one side registers it,
or the codes differ, the receiver cannot reconstruct the value.

## When NOT to use a custom codec

- **For JSON-shaped data.** Plain objects and arrays serialise
  natively. A custom codec for `User` or `Order` adds overhead
  without changing semantics.
- **For one-off types.** Custom codecs are infrastructure. If only
  one method uses a type, just send a plain object and reconstruct
  on the other side.
- **When you can't update both sides.** Codec mismatches are
  silent — the receiver will get raw bytes and not know what to do.

## Payload limits

The default packet ceiling is **16 MiB**
(`DEFAULT_MAX_PACKET_SIZE` in `netron/packet/index.ts`), overridable per
transport via the `maxPacketSize` option (`maxPacketSize` in `netron/transport/types.ts`). It
is checked inside `decodePacket` **before** the msgpack decoder runs
(`decodePacket` in the same file), so an oversized frame can't force a matching
scratch allocation and OOM the host. Over the limit, decoding throws a
`TitanError` with code `PAYLOAD_TOO_LARGE` (HTTP 413). TCP additionally
rejects on the *declared* length before reading the body
(`TcpTransport`).

Each transport also has its own body/frame cap layered on top: the
HTTP transport caps request bodies (default **10 MB**; the public
option is `maxRequestSize` as a human string like `'10mb'` —
`transport/types.ts`), and WebSocket has its own numeric
`maxPayload` (`websocket/types.ts`). There is no `maxPayloadBytes`
option.

For larger transfers (file uploads, bulk data), use:

- **Streaming methods** — yield chunks instead of one big array.
- **Out-of-band transfer** — pass a URL or storage key over Netron;
  fetch/upload bytes via a separate HTTP path.

## Round-trip type fidelity

Most types round-trip cleanly:

- `Date` → encoded as timestamp → decoded as `Date`.
- `BigInt(123n)` → encoded as string → decoded as `123n`.
- `new Map([['a', 1]])` → encoded as `[['a', 1]]` → decoded as `Map`.

A few traps:

- **`undefined` in an object.** Encoded as `null`. The receiver
  cannot distinguish "key was set to undefined" from "key was
  null". Use `?:` optional fields and omit instead.
- **Function-valued properties.** Functions cannot be serialised.
  msgpack throws.
- **Cyclic references.** msgpack throws on cycles. Don't send
  graphs with `parent` back-references.
- **Class methods.** Only data is sent. The receiver gets a plain
  object unless a custom codec reconstructs.

## Performance

For typical service payloads (small objects with primitive fields),
msgpack encode/decode is under 10 µs per call. For large nested
payloads with many strings, expect tens of microseconds.

If serialisation is on your hot path, profile. Common wins:

- **Cache encoded payloads** for repeated identical responses.
- **Trim payloads** — only include fields the client needs.
- **Use streaming** for large collections.

## Anti-patterns

- **Sending entire DB rows.** Repository fetch returns a row with
  every column; the client needs three of them. Project to a smaller
  shape on the server.
- **String IDs as `BigInt`.** msgpack `BigInt` encoding is a
  string conversion both ways. If the value fits in a `Number`
  (≤ 2^53), use `number`; if it's a 64-bit DB ID, use `string`.
- **Encoding errors as plain objects.** Strips class identity. Use
  `TitanError` subclasses; the framework's codec preserves them.

→ Back to [Netron Overview](/docs/titan/netron).

---

---
id: "titan/netron/services"
title: "Services"
description: "@Service, @Public, service descriptors, and peers."
---

# Services

A Netron service is a TypeScript class that the framework binds to a
named, versioned identifier on the wire.

## `@Service('name@version')`

```typescript
@Service('users@1.0.0')
class UsersService {
  // …
}
```

The identifier is `name@semver`. Together they form the service's
*public identity* — what clients pass to `queryInterface`.

The version is part of the identity, not an implementation detail.
`users@1.0.0` and `users@2.0.0` are **distinct services** that can
coexist on the same app, expose different method signatures, and be
called by clients pinning to one or the other.

```typescript
@Service('users@1.0.0')
class UsersServiceV1 { /* legacy */ }

@Service('users@2.0.0')
class UsersServiceV2 { /* new */ }

// Both classes registered; both reachable.
@Module({ providers: [UsersServiceV1, UsersServiceV2] })
class AppModule {}
```

This is the canonical way to do API versioning. Clients pin to the
version they were written for; servers can run multiple versions
side-by-side until clients migrate.

## `@Public()` — opt-in exposure

Methods are private to the process by default. `@Public()` marks one
as RPC-callable.

```typescript
@Service('users@1.0.0')
class UsersService {
  @Public()
  async findById(id: string): Promise<User | null> { /* exposed */ }

  @Public()
  async create(input: CreateInput): Promise<User> { /* exposed */ }

  private async hash(password: string) { /* internal — not callable */ }
}
```

This is a *deliberate* friction point. Many frameworks expose every
public method automatically; Netron does not. The reason: methods
that look fine inside the process can be dangerous when exposed
(internal-only operations, helper methods that bypass validation,
deprecated routes). `@Public()` requires an explicit decision per
method.

## Service descriptors

When you register a `@Service` class, Netron extracts *service
metadata* — a record built from `reflect-metadata` design types:

```typescript
{
  name:    'users',
  version: '1.0.0',
  methods: {
    findById: {
      type:      'Promise',                 // design:returntype name
      arguments: [{ index: 0, type: 'String' }],
    },
    create: { /* … */ },
  },
  properties: { /* @Public properties, if any */ },
}
```

(Reflection records the runtime *type names*, not full schemas — e.g.
`Promise`, `String`, `Object`. Generic type arguments are erased.)

This metadata is what `queryInterface<T>()` resolves against. The
generic `<T>` is your TypeScript interface, supplied client-side;
the wire carries the metadata above.

The descriptor metadata is stamped on the class by the `@Service`
decorator (via `reflect-metadata`) and read back when the service is
exposed. The local peer is reachable through the running `Netron`
instance:

```typescript
const peer = app.netron?.getLocalPeer();   // app.netron may be undefined before start()
const names = peer?.getServiceNames();      // qualified names of exposed services
```

The Omnitron console exposes the same metadata via the operator
service.

## Selecting a service version

HTTP and persistent Netron transports resolve names identically. An exact
unversioned publication such as `orders` takes precedence over `orders@2.0.0`
when the caller requests `orders`. If no unversioned publication exists, an
unqualified request selects the highest valid semantic version. An explicit
request for `orders@1.0.0` requires that exact publication; it never falls back
to another version. Registration order does not change this decision.
Removing one publication leaves the other versions available. Use explicit
versions when an application depends on a specific wire contract.

## Peers

Netron's runtime model has two peer types:

- **`LocalPeer`** — represents the current process. Owns the set of
  registered services. One per `Application`.
- **`RemotePeer`** — a proxy to another process. Calls on the proxy
  marshal as RPC packets.

The application's `LocalPeer` is accessible via
`app.netron?.getLocalPeer()`. You rarely interact with it directly —
the framework wires services on its behalf.

`RemotePeer` is the client-side view. `netron.connect(address)`
returns one; you get the typed proxy via its
`queryInterface<T>('name@version')`.

## Local vs remote calls — the same code

A service in the same process is callable as a normal method —
no RPC involved:

```typescript
@Service('orders@1.0.0')
class OrdersService {
  constructor(private readonly users: UsersService) {}

  @Public()
  async create(userId: string, items: Item[]) {
    const user = await this.users.findById(userId);   // direct method call
    if (!user) throw Errors.notFound('user', userId);
    // …
  }
}
```

The framework detects that `users` is a local provider and routes
calls directly. No serialisation, no transport, no middleware. The
same code from outside the process goes through the wire.

This means **you write services the same way regardless of where they
are called from**. The wire-vs-local optimisation is invisible.

## Multiple services per class? No.

One class = one service = one identifier. If a class has methods that
belong to different services, split it into two classes.

This is intentional. Lumping unrelated methods into one service makes
the wire contract less coherent and complicates versioning.

## Service-level decorators

In addition to method-level decorators, you can apply `@Auth` at the
**class** level — it then covers every `@Public` method on the class:

```typescript
import { Service, Public, Auth } from '@omnitron-dev/titan/decorators';

@Service('orders@1.0.0')
@Auth({ scopes: ['orders:read'] })                    // class-level auth
class OrdersService {
  @Public() async list() { /* … */ }
  @Public() async get(id: string) { /* … */ }
  @Public()
  @Auth({ scopes: ['orders:write'] })                 // override for this method
  async create(input: CreateInput) { /* … */ }
}
```

Method-level `@Auth` **overrides** the class-level config for that
method (use the `inherit` / `override` flags on `AuthConfig` for
finer control). To allow anonymous access on one method, declare
`@Auth({ allowAnonymous: true })`.

> `@Auth` is the only decorator that combines this way at the class
> level. `@RateLimit`, `@Cache`, and `@Public`'s other options are
> method-level only.

## Anti-patterns

- **Mutable service state.** Singleton services are shared across
  calls. State stored in `this` is shared state, with all the
  concurrency hazards that implies. Keep service instance state to
  references (database, cache, configuration); per-call state belongs
  in method-local variables or a `Request`-scoped context.
- **Decorating private methods with `@Public`.** Private methods are
  not meant to be called from outside. If you need a public method,
  rename it and make it part of the contract.
- **Mixing service identities in one class.** "UsersService" and
  "AdminService" should be two classes, even if they share helpers.
  The wire contract is per-class.
- **Skipping the version.** `@Service('users')` leaves the version
  **empty** (the qualified name is just `users`, not `users@1.0.0`).
  That's a valid identity, but clients then resolve `'users'` with no
  version pin. Always write the explicit version.

---

---
id: "titan/netron/streaming"
title: "Streaming"
description: "AsyncIterable methods, server-push, backpressure."
---

# Streaming

Netron supports server-streaming methods. On the server you write an
`async *` generator (or return a `NetronReadableStream` directly). When
a method returns an async generator and the caller is remote, the
service stub wraps the generator in a `NetronWritableStream` and pipes
it (`stream.pipeFrom(generator)`), returning a `StreamReference` over
the wire (the `isAsyncGenerator` branch of `ServiceStub.call`). The receiving peer rebuilds
the other half as a `NetronReadableStream` (`TYPE_STREAM` in `RemotePeer.handlePacket`).

The stream primitives are built on the **`readable-stream` package**
(the userland Node.js streams implementation), not the WHATWG Web
Streams API and not native async iterables: `NetronReadableStream
extends Readable` and `NetronWritableStream extends Writable`, both in
`objectMode` (`netron/streams/readable-stream.ts`,
`netron/streams/writable-stream.ts`). Because the received stream is
a `Readable`, you can consume it either with stream events
(`.on('data')` / `.on('end')`) or with `for await` — a `Readable` is
async-iterable. The examples below use `for await` for brevity.

```mermaid
sequenceDiagram
  participant C as Client
  participant T as Transport (WS/TCP/Unix)
  participant S as Server (async generator)

  C->>T: queryInterface + call watchAll()
  T->>S: open stream
  loop while server yields
    S-->>T: yield event
    T-->>C: chunk
    C->>C: process
  end
  alt client stops reading
    C->>C: destroy received stream (no upstream signal)
    Note over S: server unwinds only when its<br/>next write to the socket fails
    S->>S: finally { release resources }
  else server done
    S-->>T: close
    T-->>C: end of stream
  end
```

Streaming requires WebSocket, TCP, or Unix transport. The HTTP
transport does not open a streaming channel — instead it **collects an
`async *` generator into an array** before sending the response
(`collectAsyncGeneratorValues` in `netron/transport/http/server.ts`).
Collection is bounded by `maxAsyncGeneratorItems` (a server option,
default `10000`, the constant `DEFAULT_MAX_ASYNC_GENERATOR_ITEMS`). When the generator exceeds that
limit the server throws a `TitanError` with code `PAYLOAD_TOO_LARGE`
(from that same method); other (non-`TitanError`) iteration failures return
the partial results collected so far rather than failing the whole
request. So over HTTP you get a bounded batch, not a live stream — use
WS/TCP/Unix for true streaming, or paginate.

## Defining a streaming method

```typescript
@Service('orders@1.0.0')
class OrdersService {
  @Public()
  async *watchAll(): AsyncIterable<Order> {
    for await (const event of this.queue.subscribe('orders.*')) {
      yield event.order;
    }
  }
}
```

The method is an async generator (`async *`). Each `yield` becomes a
value sent to the client.

## Consuming a stream

```typescript
const orders = await client.queryInterface<OrdersService>('orders@1.0.0');

for await (const order of orders.watchAll()) {
  console.log(order);
}
```

The `for await` loop runs as long as the server yields. When the
server's generator returns, the loop ends. If the server throws,
the loop throws.

## Backpressure

Backpressure is real. The producer is wrapped in a `NetronWritableStream`,
which uses the standard `write()` → `await once('drain')` loop; because
`sendPacket` awaits the socket-send callback for stream packets too
(fixed in T#43), the producer feels genuine socket congestion and the
`async *` generator effectively pauses at `yield`.

There *are* bounded buffers, though: the readable side keeps a
reorder buffer (`MAX_BUFFER_SIZE`, 10 000 chunks) and **destroys the
stream** with a backpressure error if it overflows, and there are
per-peer stream-count caps. So overflow is a handled failure mode, not
an impossibility.

This means a slow client can slow down a fast producer. If you need
producer-side draining (for example, the producer is a queue that
will overflow), implement explicit drop-or-disconnect logic:

```typescript
@Public()
async *watchAll(): AsyncIterable<Order> {
  for await (const event of this.queue.subscribe('orders.*')) {
    if (this.queue.depth() > 10_000) {
      // Disconnect slow client; let them reconnect with a checkpoint.
      throw Errors.unavailable('client too slow; reconnect with cursor');
    }
    yield event.order;
  }
}
```

## Cancellation

Cancellation is **one-directional on the wire**, and the direction
matters.

The **producer → consumer** direction is wired. When the server's
`NetronWritableStream` finishes (`end()`) it sends an end-of-stream
final chunk; when it is `destroy()`ed it sends an explicit
`TYPE_STREAM_CLOSE` packet (`NetronWritableStream.finalizing`),
and the consumer's readable reacts by force-closing
(`TYPE_STREAM_CLOSE` in `RemotePeer.handlePacket` → `stream.forceClose(reason)`). The two
are mutually exclusive — a graceful end uses the EOS chunk, a destroy
uses the close packet (the `finalizing` guard in `writable-stream.ts`
prevents emitting both).

The **consumer → producer** direction is *not* wired. The received
stream is a `NetronReadableStream`, and its `destroy()`
(`NetronReadableStream.destroy`) only tears down locally — it
sends **no** packet back to the producer. There is no code path where a
broken/destroyed readable notifies the writable upstream. So a clean
client `break` out of the `for await` loop does **not** signal the
server: the generator keeps being pulled until the next
`sendStreamChunk` write fails against the dead socket (or the
connection drops), at which point `pipeFrom` catches the error and
destroys the writable. Don't rely on prompt server-side teardown from a
client `break` — treat teardown as eventual, driven by the failed
write, not immediate.

Because the generator *does* unwind when the write side fails (or the
connection drops), always release resources in a `finally`:

```typescript
@Public()
async *watchAll(): AsyncIterable<Order> {
  const sub = this.queue.subscribe('orders.*');
  try {
    for await (const event of sub) {
      yield event.order;
    }
  } finally {
    await sub.unsubscribe();          // runs when the generator unwinds
  }
}
```

For deterministic cancellation, give the client an explicit
"unsubscribe"/"stop" method on the service rather than relying on
iterator teardown.

## Reconnection

WebSocket reconnects automatically (configurable via the client). On
reconnect, **streams do not resume** — the client must re-subscribe.
This is intentional: the server cannot know what the client missed.

For resumable streams, your contract should accept a checkpoint:

```typescript
@Public()
async *watchAll(since?: string): AsyncIterable<Order> {
  // Replay from `since`, then live-tail.
  if (since) {
    for await (const o of this.repo.findAfter(since)) yield o;
  }
  for await (const event of this.queue.subscribe('orders.*')) {
    yield event.order;
  }
}
```

The client passes the last-seen ID to resume:

```typescript
let lastSeen: string | undefined;
while (true) {
  try {
    for await (const order of orders.watchAll(lastSeen)) {
      lastSeen = order.id;
      handle(order);
    }
  } catch (e) {
    if (isReconnectable(e)) continue;
    throw e;
  }
}
```

## Bidirectional streaming

There is no single-method *duplex* sugar — a method doesn't return one
two-way channel. But the primitives for full duplex exist: a
`NetronWritableStream` streams **client→server**, and a stream
(readable or writable) can be **passed as a method argument** —
Netron serialises it as a stream reference and rebuilds a live stream
on the far side.

So for full-duplex protocols (collaborative editing, multi-party
games) you compose unidirectional streams: return a server→client
stream and/or pass a client→server stream as an argument.

## Performance

Each stream has:

- A persistent connection (WS / TCP / Unix).
- A per-stream message correlation ID.
- An async iterator on each side.

The cost per yield is the same as a unary call — msgpack encode +
write. Throughput is bounded by the transport.

## Anti-patterns

- **Returning a stream of trivial values.** A stream of strings,
  one per yield, is wasteful — the per-message overhead exceeds
  the payload. Batch into arrays where possible.
- **No `finally` for resource release.** Every stream must clean up
  on cancellation. Forgetting causes leaks that compound over
  reconnects.
- **Unbounded buffers.** Don't buffer values without backpressure.
  Yield directly from the source iterator and let the transport
  apply backpressure.
- **Streaming where polling would do.** A stream that yields
  every 30 seconds is just polling, with more state. Use a unary
  method called on a timer.

---

---
id: "titan/netron/transports"
title: "Transports"
description: "HTTP, WebSocket, TCP, Unix — how to choose and how to configure."
---

# Transports

A transport is the wire protocol Netron speaks. Four are built in;
each is opt-in. The same `@Service` class is reachable over any
transport you bind, with the same calling convention and the same
middleware stack.

## The four transports

| Transport | Best for                                                | Latency | Throughput | Streaming |
| --------- | ------------------------------------------------------- | ------- | ---------- | --------- |
| HTTP      | Browsers, REST integrations, reverse-proxy environments | Med     | Low–med    | No        |
| WebSocket | Browser subscriptions, persistent client sessions       | Low     | High       | Yes       |
| TCP       | Service-to-service inside a cluster                     | Lowest  | Highest    | Yes       |
| Unix      | Sidecars, CLI ↔ daemon, low-overhead local IPC          | Lowest  | Highest    | Yes       |

## Configuring transports

Register transports on the application's `Netron` instance **before
startup**. `registerTransportServer` stores a server configuration;
`app.start()` starts the registered listeners:

```typescript
import { HttpTransport } from '@omnitron-dev/titan/netron/transport/http';
// WebSocket / TCP / Unix transports auto-register in Node; HTTP must be
// registered explicitly.

const app = await Application.create(AppModule);
const netron = app.netron!;

netron.registerTransport('http', () => new HttpTransport());
netron.registerTransportServer('http',      { name: 'api',  options: { port: 3000 } });
netron.registerTransportServer('websocket', { name: 'ws',   options: { port: 3001 } });
netron.registerTransportServer('tcp',       { name: 'tcp',  options: { port: 4001 } });
netron.registerTransportServer('unix',      { name: 'sock', options: { path: '/tmp/myapp.sock' } });
await app.start();
```

`registerTransportServer(name, { name, options })` takes a transport
key, a server label, and an `options` bag (`{ host?, port?, path?, … }`).
Only the transports you start are bound. An unrestricted published service
is reachable over each bound transport. Restrict its remote surface at the
service or member declaration:

```typescript
@Service({ name: 'orders', transports: [new HttpTransport()] })
class OrdersService {
  @Public({ transports: ['http'], auth: { roles: ['operator'] } })
  async list() { /* … */ }
}
```

The effective surface is the intersection of service and member restrictions.
Omitting `transports` adds no restriction; `[]` denies all remote transports.
`ws` normalizes to `websocket`. Custom transports use their implementation's
`name`, not a listener label. Invalid declarations fail during publication.
Restrictions do not start listeners or replace authentication and permissions.

The server determines the ingress transport. A request header or RPC argument
cannot select another transport identity. HTTP calls and batches, persistent
method calls and property operations all enforce the same boundary. Discovery,
OpenAPI and returned nested service interfaces show the corresponding surface;
calling a known service name or definition ID does not bypass it. Trusted
in-process calls remain local operations.

If interface discovery is denied by a transport restriction, the client receives
the same service-not-found response as for an unknown service. The server logs
a warning with the requested service, peer ID and actual ingress transport, so
operators can distinguish a denied route from a missing publication without
revealing a private service to the caller.

Forwarded definitions carry optional transport restrictions. A gateway cannot
widen them; definitions from older publishers without these fields remain
unrestricted by transport. Remote forwarding also requires an explicit
[gateway ACL](/docs/titan/netron/authentication#forwarding-through-a-gateway).

## When to choose what

### HTTP

Choose HTTP when:

- Browsers call your service directly. (Use WebSocket only if you
  need streaming or long-lived sessions.)
- A reverse proxy sits between client and server (NGINX, Cloud Load
  Balancer, K8s Ingress). HTTP/1.1 traverses these without special
  config.
- You want curl-debuggable endpoints.

Cost: per-call HTTP framing overhead (\~50–200 µs). Negligible for
human-driven traffic; significant for service-to-service hot paths.

### WebSocket

Choose WebSocket when:

- You need server-push (subscriptions, long-lived events).
- You need persistent sessions (auth setup once per connection,
  reused across many calls).
- You want low per-call overhead from a browser.

Cost: connection state on the server. WebSocket is stateful; a
per-pod connection limit applies.

### TCP

Choose TCP when:

- Server-to-server inside a cluster, behind a private network.
- You need the lowest possible latency.
- You don't need a reverse proxy (TCP doesn't traverse most HTTP-only
  proxies).

Cost: one persistent TCP connection per peer pair. RPC calls are multiplexed on
that connection; an additional connection pool is not required.

### Unix sockets

Choose Unix when:

- Two processes on the same host need to communicate (sidecar pattern,
  CLI ↔ daemon).
- You want OS-level access control via filesystem permissions.
- You want zero TCP/TLS overhead.

Limitation: same-host only. The Omnitron CLI uses Unix sockets to
talk to local supervisors.

## Transport options — common

All transports extend a shared `TransportOptions` base
(`netron/transport/types.ts`). The most-used fields:

| Option           | Default                                    | Effect                                                                   |
| ---------------- | ------------------------------------------ | ------------------------------------------------------------------------ |
| `port` / `path`  | —                                          | Where to listen (set via the server `options`)                           |
| `host`           | `'0.0.0.0'` (TCP/WS), `'localhost'` (HTTP) | Interface to bind                                                        |
| `maxPacketSize`  | `16 * 1024 * 1024` (16 MiB)                | Hard ceiling on an inbound packet, checked before decode                 |
| `connectTimeout` | `10_000`                                   | Transport establishment and peer greeting deadline, on client and server |
| `requestTimeout` | `5_000`                                    | Per-request timeout                                                      |
| `streamTimeout`  | `30_000`                                   | Idle-stream timeout                                                      |
| `keepAlive`      | `{ enabled?, interval?, timeout? }`        | Keepalive config object                                                  |
| `compression`    | `false`                                    | Enable message compression                                               |

Transport-specific options layer on top:

- **WebSocket** — `maxPayload`, `perMessageDeflate`, `handshakeTimeout`,
  `protocols`, `pathPrefix`.
- **TCP** — `noDelay` (disable Nagle), `keepAliveDelay`, `allowHalfOpen`.
- **HTTP** — `cors`, `maxRequestSize` (e.g. `'10mb'`),
  `keepAliveTimeout`, `customRoutes`, `maxAsyncGeneratorItems`,
  `pathPrefix`.

There is **no** unified `maxPayloadBytes` / `keepAliveMs` /
`connectionTimeoutMs` option — those names do not exist; use the real
ones above.

## Share a request budget across HTTP and WebSocket

Create one limiter per process and pass the same object to the listeners that serve the same clients. A client key has one fixed window across both transports. The limiter also has a fixed window for all clients. Each window starts on its first accepted request and resets after `windowMs`.

```typescript
import { TransportRequestLimiter } from '@omnitron-dev/titan/netron';

const requestLimiter = new TransportRequestLimiter({
  enabled: true, windowMs: 60_000, maxRequests: 300, globalMaxRequests: 2_000,
});
netron.registerTransportServer('http', {
  name: 'api', options: { host: '127.0.0.1', port: 8010, requestLimiter },
});
netron.registerTransportServer('websocket', {
  name: 'events', options: { host: '127.0.0.1', port: 8011, requestLimiter },
});
```

HTTP reserves one unit before reading the body. A batch of N calls requires another N−1 units before the first handler runs. If the batch exceeds the remaining allowance, the ingress request still counts and no handler runs. Opening a WebSocket and initiating a call or task each consume one unit. Responses to requests already in flight can complete when the window is full.

The default key is the remote socket address. Enable `trustProxy` only on a listener reachable exclusively through a trusted proxy that overwrites the forwarding headers on ordinary requests and upgrades. An exhausted HTTP request or upgrade receives `429` and `Retry-After` in seconds. On an established WebSocket the caller receives a Netron rate-limit error. A stream that cannot deliver its next frame, or a peer that keeps sending while its refusal is still pending, closes with WebSocket code 1013. Handle that close with bounded reconnect backoff. A key on the optional `whitelist` bypasses both ceilings. Separate daemon processes maintain independent windows. Method authorization still applies after transport admission.

## Connection limits and lifecycle

`Netron` enforces `maxTotalConnections` (default **100**) and
`maxConnectionsPerPeer` (default **10**) through one connection manager. Pending
incoming greetings occupy capacity. Outbound HTTP peer handles count too;
inbound HTTP requests use their transport middleware limits. Configure limits
from measured peak concurrency before upgrading an older build in which these
settings only emitted a warning.

```typescript
const netron = new Netron(logger, {
  maxTotalConnections: 500,
  maxConnectionsPerPeer: 10,
});
```

`netron.connectionManager.getStats()` reports actual connections, per-peer counts
and transport byte totals. Slots are released on disconnect, rejected handshake
and shutdown, including clients that never start a listening server.

A peer ID is a routing identity, **not authentication**. Incoming greetings cannot
replace an already-connected identity. Use a separate `Netron` instance for each
concurrent authenticated principal; disconnect the old connection before reusing
its identity on the same server. IDs are limited to 256 characters without
whitespace/control characters. A greeting must fit in 4096 bytes and finish within
`connectTimeout`.

Netron owns reconnect together with the peer handshake. Raw transport reconnect
is disabled for sockets it owns, preventing competing recovery loops. Stopping
Netron cancels pending handshakes and reconnects; late results are closed.

Startup succeeds only when every configured server transport is available and has
started. If any listener fails, Netron waits for the other factories to settle,
then closes all created listeners and connections through the same cleanup path
as `stop()`. A late factory cannot leave a socket behind a failed startup.
WebSocket readiness waits for the native listener, including host resolution and
an external HTTP server. Bind errors reject startup instead of returning a ready
adapter. Port `0`, in options or a `ws://host:0` address, asks the OS for a free
port; it never selects the protocol default.
Concurrent `stop()` joins that lifecycle; a new `start()` or `connect()` is refused
while cleanup is running. The configuration remains available for a retry.

A close failure is reported rather than treated as a clean stop. Other resources
are still released; failed listeners remain registered so `stop()` can retry.
`start()` cannot claim readiness over a listener left by an incomplete cleanup.
Inspect the original startup error and any cleanup errors before retrying.

Persistent RPC connections stay leased until disconnect. Pool size, idle cleanup
and pool heartbeat options apply to unleased spare connections managed directly
through `connectionManager`; they neither create additional RPC sockets nor evict
an otherwise idle persistent peer. Transport keepalive owns its liveness.

## TLS

The Netron transports do not currently expose a built-in TLS/`tls`
option. Terminate TLS at the load balancer, ingress, or reverse proxy
in front of the process — HTTP and WebSocket traverse those cleanly.
For TCP between hosts, run it over a private network or a tunnel.

Unix sockets need no transport encryption; their access control is
filesystem permissions.

## Same service, multiple transports — what changes?

Nothing in the service code. Same dispatch logic, same middleware
stack, same error mapping. The transport is invisible to the method
body.

A few client-side differences:

- **Browser** can only use HTTP and WebSocket.
- **Streaming methods** (returning `AsyncIterable<T>`) require WS, TCP,
  or Unix — HTTP cannot stream.
- **WS connection state** persists across calls. The initial credential is
  revalidated before each invocation; revocation and role changes are not frozen
  at connection time. HTTP authenticates per request.

## The transport registry

Transports are pluggable. **Every `Netron` instance owns its own
registry** (`new TransportRegistry()` in the constructor), which
auto-registers `tcp`, `ws`/`websocket` and `unix`. Add your own by
implementing the transport interface and registering it **on the
instance**:

```typescript
import { Netron } from '@omnitron-dev/titan/netron';
// ITransport ships from the http entry point — that is currently its only
// public path, though the interface itself is transport-agnostic.
import type {
  ITransport, ITransportConnection, ITransportServer,
  TransportOptions, TransportAddress,
} from '@omnitron-dev/titan/netron/transport/http';

class QuicTransport implements ITransport {
  readonly name = 'quic';
  readonly capabilities = {
    streaming: true, bidirectional: true, binary: true,
    reconnection: true, multiplexing: true, server: true,
  };

  async connect(address: string, options?: TransportOptions): Promise<ITransportConnection> {
    throw new Error('not implemented');
  }
  async createServer(options?: TransportOptions): Promise<ITransportServer> {
    throw new Error('not implemented');
  }
  isValidAddress(address: string) { return address.startsWith('quic://'); }
  parseAddress(address: string): TransportAddress {
    return { protocol: 'quic', host: 'localhost', port: 4433 };
  }
}

// registerTransport takes a *factory*, not the class itself.
netron.registerTransport('quic', () => new QuicTransport());
netron.registerTransportServer('quic', { name: 'quic', options: { port: 4433 } });
```

`registerTransportServer` throws `NotFound` unless the name was
registered first, so the two calls are ordered.

Register custom transports on each Netron instance that serves them. The instance owns its transport registry.

## Cross-runtime transport availability

| Runtime | HTTP | WebSocket | TCP | Unix |
| ------- | ---- | --------- | --- | ---- |
| Node    | ✓    | ✓         | ✓   | ✓    |
| Bun     | ✓    | ✓         | ✓   | ✓    |
| Deno    | ✓    | ✓         | ✓   | ✓    |
| Browser | ✓    | ✓         | —   | —    |

A fresh `Netron` already has `tcp`, `ws`/`websocket` and `unix`.
**HTTP is never auto-registered** — register it on the instance before
starting an HTTP server:

```typescript
import { HttpTransport } from '@omnitron-dev/titan/netron/transport/http';

netron.registerTransport('http', () => new HttpTransport());
netron.registerTransportServer('http', { name: 'http', options: { port: 8081 } });
```

For browsers, use [the Netron browser client](/docs/frontend/netron/browser).

## HTTP failures and deadlines

Titan and the browser client decode HTTP errors through the same
`netron-protocol` contract. Numeric, decimal-string and shared symbolic codes
retain their category on peer, direct connection and per-request-hint paths.
Unknown application symbols remain in `details.errorCode`, with the HTTP failure
status as the category. Request, correlation and trace headers survive decoding.
A 401/403 refusal must not turn into an internal error or a retryable outage.

A request deadline covers both response headers and body consumption. Each
concurrent request has its own cancellation controller; closing the connection
cancels every pending request. A valid request sent through `connection.send()`
propagates transport failures and remote refusals to its caller. A timeout means
that confirmation was not received, not that a remote mutation was undone.

## Anti-patterns

- **Binding all four transports by default.** Each open transport
  is a port and an attack surface. Bind only what's needed.
- **HTTP for service-to-service hot paths.** Per-call framing cost
  dominates. Use TCP or WS within the cluster.
- **WebSocket for short-lived clients.** A short-lived client that
  connects, makes one call, disconnects pays for the connection
  setup over and over. HTTP is simpler.
- **Leaving the packet ceiling at the 16 MiB default.** Size
  `maxPacketSize` (and HTTP's `maxRequestSize`) down to what your
  contract actually needs — it defends against accidental memory
  amplification on untrusted-peer transports.

---

---
id: "titan/netron"
title: "Netron RPC"
description: "A transport-agnostic RPC plane — same service surface across HTTP, WebSocket, TCP, Unix."
---

# Netron RPC

Netron is the RPC plane Titan ships. It does one thing: take a
TypeScript class, expose its methods over the wire, dispatch incoming
calls back to the methods. The wire format is binary msgpack; the
contract is the TypeScript interface.

This page is the Netron entry point. The mechanics live in:

| Page                                                | Topic                                               |
| --------------------------------------------------- | --------------------------------------------------- |
| [Services](/docs/titan/netron/services)             | Defining `@Service` classes, descriptors, peers     |
| [Transports](/docs/titan/netron/transports)         | HTTP / WebSocket / TCP / Unix — when to choose what |
| [Middleware](/docs/titan/netron/middleware)         | Per-call wrapping (auth, rate limit, tracing)       |
| [Authentication](/docs/titan/netron/authentication) | Identity, claims, policies                          |
| [Streaming](/docs/titan/netron/streaming)           | AsyncIterable methods, backpressure                 |
| [Multi-backend](/docs/titan/netron/multi-backend)   | Client → many servers, failover, load balancing     |
| [Serialization](/docs/titan/netron/serialization)   | Wire format, msgpack, custom codecs                 |

## The short tour

Server — define the service, then bind a transport server on the
running `Netron` instance:

```typescript
import { Service, Public } from '@omnitron-dev/titan/decorators';
import { HttpTransport } from '@omnitron-dev/titan/netron/transport/http';

@Service('users@1.0.0')
class UsersService {
  @Public() async findById(id: string): Promise<User | null> { /* … */ }
}

const app = await Application.create(AppModule);
app.netron!.registerTransport('http', () => new HttpTransport());
app.netron!.registerTransportServer('http', { name: 'api', options: { host: '127.0.0.1', port: 3000 } });
await app.start();
```

Client — connect to get a `RemotePeer`, then resolve a typed proxy:

```typescript
const peer  = await app.netron!.connect('http://127.0.0.1:3000');
const users = await peer.queryInterface<UsersService>('users@1.0.0');
const user  = await users.findById('u_42');
```

That is the core API surface. Versioning, routing, serialisation,
error mapping, and trace propagation all happen behind it.

> The in-browser client is a separate package
> (`@omnitron-dev/netron-browser`) with its own connection API; this
> page covers the server-side `@omnitron-dev/titan/netron` runtime.

## What Netron is

A few specific things, each with a deliberate scope:

- **A service descriptor system.** `@Service` registers a typed
  contract; clients resolve it by name and version.
- **A transport abstraction.** Four transports built in, all sharing
  the same service surface; pluggable for custom ones.
- **A middleware pipeline (HTTP transport).** Per-call wrapping for
  cross-cutting concerns — auth, rate limiting, tracing, logging. The
  persistent transports (WS/TCP/Unix) enforce auth inline at dispatch
  rather than through the pipeline.
- **An auth policy framework.** Authentication and authorisation are
  separate concerns with composable policies via `BuiltInPolicies`
  (`requireRole`, `requireAnyRole`, `requirePermission`, …).
- **A multi-backend client.** One client can route to many servers
  with health-aware failover, circuit breaking, and per-service
  routing rules.
- **A streaming primitive.** `NetronReadableStream` /
  `NetronWritableStream` (Node Web Streams) carry long-lived data over
  WebSocket / TCP / Unix. An `async *` generator method is
  auto-wrapped into a stream for remote callers.

## What Netron is not

- **Not a service mesh.** Discovery, mTLS, traffic shaping at
  cluster scope are out of scope. Use a real mesh (Linkerd, Istio,
  Consul Connect) for that.
- **Not a queue.** Netron calls are request/response; the response
  may be a stream. For decoupled work, use
  [`titan-events`](/docs/titan/modules/events) or
  [`titan-notifications`](/docs/titan/modules/notifications).
- **Not a schema language.** The TypeScript interface is the schema.
  No `.proto` files, no OpenAPI YAML.

## Performance characteristics

Per-call cost on a warmed-up connection (HTTP/1.1, localhost,
no middleware):

| Op                             | Approx cost |
| ------------------------------ | ----------- |
| msgpack encode (small payload) | 5–10 µs     |
| Service descriptor lookup      | <1 µs       |
| Validation (typical schema)    | 1–5 µs      |
| Method invocation (your code)  | varies      |
| msgpack decode                 | 5–10 µs     |
| HTTP framing                   | 50–200 µs   |

For sustained throughput, use WebSocket (no per-call HTTP framing
cost) or TCP. For bursty short-lived clients, HTTP is fine and works
through any reverse proxy.

---

---
id: "titan/overview"
title: "Titan Framework"
description: "A composable TypeScript backend framework — DI, lifecycle, validation, and a transport-agnostic RPC plane."
---

# Titan Framework

Titan is **the backend framework**. It runs standalone — no
daemon, no supervisor, no extra runtime. Build a backend with
Titan + the modules you need, then `node dist/main.js`. That's
the entire production deploy.

> **Titan is self-sufficient.** [Omnitron](/docs/omnitron/overview)
> is a separate opt-in layer on top — useful for multi-app fleets,
> web console, declarative infra provisioning, MCP server for AI
> agents. For a single backend service you don't need it. See
> [Titan vs Omnitron](/docs/foundations/titan-vs-omnitron) for
> the precise boundary.

It is intentionally small at its core and large at its surface —
a focused application kernel plus a deep catalogue of opt-in
modules.

## What Titan is

A backend framework gives you four things you would otherwise build by hand:

1. **A way to declare structure** — what services exist, how they depend on
   each other, when they start and stop.
2. **A way to expose them** — over the network, behind authentication,
   with observability built in.
3. **A way to operate** — configuration, logging, health, metrics,
   traces, scheduled work.
4. **A way to recover** — typed errors, graceful shutdown, retries,
   circuit breakers, locks.

Titan provides all four with a deliberately minimal core (the
`Application`, the `Container`, the `Lifecycle`) and a library of modules
that compose into the runtime you actually want.

## What Titan is not

- **Not a "do everything" runtime.** Modules are independently versioned
  packages. You can run a Titan app with one of them.
- **Not opinion-free.** Titan has strong opinions about explicitness:
  declared providers and explicit lifecycle ownership. Request context
  uses AsyncLocalStorage where the transport and authentication APIs require it.
- **Not a NestJS clone.** Titan shares the decorator-driven aesthetic
  with NestJS, but the underlying container (Nexus) is a different
  design — multi-token providers, contextual injection, middleware
  pipelines, runtime-portable across Node/Bun/Deno/browser.

## The mental model

```mermaid
flowchart TB
  subgraph App["Application (kernel)"]
    Bootstrap[Bootstrap]
    Container[Nexus DI Container]
    Lifecycle[Lifecycle State Machine]
    Events[Application Event Bus]
  end

  subgraph Modules["Your modules + ecosystem modules"]
    UserMod[User Modules]
    Auth[titan-auth]
    DB[titan-database]
    Cache[titan-cache]
    Etc[…]
  end

  subgraph Netron["Netron RPC Plane"]
    Services[Your @Service classes]
    Transports[HTTP / WS / TCP / Unix]
    Auth2[Authn + Authz]
    Middleware[Middleware Stack]
  end

  Bootstrap --> Container
  Container --> Modules
  Container --> Services
  Lifecycle --> Modules
  Modules --> Container
  Services --> Transports
  Auth2 --> Middleware
  Middleware --> Services
```

The Application kernel owns the container, the lifecycle, and the event
bus. Modules register providers in the container. Services are providers
that also register themselves with Netron. Netron exposes them over
transports. Middleware wraps every call.

That is it. Everything else — the optional ecosystem modules, the validation
decorators, the resilience helpers — is built on top of these four
primitives.

## What this section covers

This is the reference section for Titan itself. It is dense, technical,
and assumes you have read the
[introduction](/docs/intro) and worked through the
[quickstart](/docs/getting-started/quickstart).

| Area                                                              | When to read                                        |
| ----------------------------------------------------------------- | --------------------------------------------------- |
| [Concepts](/docs/titan/concepts/design-principles)                | Before designing a new Titan-based system           |
| [Application](/docs/titan/application/bootstrap)                  | When you start, stop, or supervise an app           |
| [Modules](/docs/titan/modules-system/defining-modules)            | When you organise services into reusable units      |
| [Dependency Injection](/docs/titan/di/overview)                   | When you need to control how things are constructed |
| [Decorators](/docs/titan/decorators)                              | Reference for every decorator the framework ships   |
| [Validation](/docs/titan/validation/overview)                     | When inputs cross a trust boundary                  |
| [Errors](/docs/titan/errors/overview)                             | When you need typed, classifiable failures          |
| [Netron RPC](/docs/titan/netron)                                  | When you expose services over the network           |
| [Configuration](/docs/titan/configuration/overview)               | When the runtime needs to read settings             |
| [Logging](/docs/titan/logging/overview)                           | When you need structured, context-aware logs        |
| [Tracing](/docs/titan/tracing)                                    | When you correlate work across services             |
| [Resilience](/docs/titan/resilience/overview)                     | When you defend against partial failure             |
| [Testing](/docs/titan/testing/overview)                           | When you write fast, deterministic tests            |
| [Best Practices](/docs/titan/best-practices/structuring-services) | When you want the team to converge on conventions   |
| [Modules Catalogue](/docs/titan/modules)                          | Public modules and their integrations               |

## Versioning and public boundaries

Titan is pre-1.0. Use declared package entry points, pin versions and verify
upgrades against the contracts your application consumes. A minor release
can change a public contract. See [compatibility](/docs/reference/compatibility).

The [API index](/docs/reference/api-index) derives import paths from package
manifests and symbols from TypeScript exports. Source files that are not
reachable through those entry points are implementation details. A source
annotation alone does not make an arbitrary deep import supported.

Read on: [Design Principles](/docs/titan/concepts/design-principles).

---

---
id: "titan/recipes/api-service"
title: "API service stack"
description: "Public HTTP/WS API — auth, database, cache, rate limiting, health, metrics."
---

# API service stack

The canonical shape for a public API that handles user traffic:
authenticated requests, database reads/writes, cache in front,
rate limiting per user, health probes for Kubernetes, metrics for
dashboards.

## Shape

- **Authenticated.** JWT tokens (or service-key for trusted
  callers) verified on every request.
- **Database-backed.** Postgres + Kysely, soft-delete + timestamps
  plugins on, RLS for tenant isolation.
- **Cache in front.** L1 in-memory per pod, L2 in Redis shared
  across pods.
- **Rate-limited.** Per-user sliding window in Redis, tiered plans.
- **Health-probed.** `/healthz` for liveness, `/readyz` for
  readiness, both Kubernetes-friendly.
- **Metrics.** Counters / histograms for every RPC method,
  exported in Prometheus format.

## Architecture

```mermaid
flowchart LR
  Client[Browser / mobile]
  Client -->|HTTPS + JWT| Edge[Netron HTTP transport]
  Edge --> Auth[titan-auth<br/>JWTService]
  Auth --> Rate[titan-ratelimit<br/>per-user]
  Rate --> Service[Your @Service classes]
  Service --> Cache[titan-cache<br/>L1 → L2]
  Cache -.L2.-> Redis[(Redis)]
  Service --> DB[titan-database<br/>Kysely + RLS]
  DB --> PG[(PostgreSQL)]
  Service --> Metrics[titan-metrics]
  Health[titan-health] -.indicators.-> DB
  Health -.indicators.-> Redis
  Probe[Kubernetes probes] --> Health
  Metrics --> Prom["/metrics endpoint"]
```

## `AppModule`

```typescript
import { Module } from '@omnitron-dev/titan';
import { z } from '@omnitron-dev/titan/validation';
import { ConfigModule } from '@omnitron-dev/titan/module/config';
import { LoggerModule, ConsoleTransport, RedactionProcessor }
  from '@omnitron-dev/titan/module/logger';

import { TitanRedisModule } from '@omnitron-dev/titan-redis';
import { TitanAuthModule } from '@omnitron-dev/titan-auth';
import { TitanDatabaseModule } from '@omnitron-dev/titan-database';
import { TitanCacheModule } from '@omnitron-dev/titan-cache';
import { TitanRateLimitModule } from '@omnitron-dev/titan-ratelimit';
import { TitanHealthModule } from '@omnitron-dev/titan-health';
import { TitanMetricsModule } from '@omnitron-dev/titan-metrics';
import { sql } from 'kysely';

const AppConfigSchema = z.object({
  env: z.enum(['development', 'staging', 'production']),
  redis: z.object({ url: z.string().url() }),
  database: z.object({ url: z.string().url() }),
  auth: z.object({
    jwtSecret: z.string(),
    issuer: z.string(),
    audience: z.string(),
  }),
  rateLimit: z.object({
    defaultLimit: z.number().int(),
    defaultWindowMs: z.number().int(),
  }),
});

@Module({
  imports: [
    // ── Configuration + logging (auto-loaded; shown for completeness) ──
    ConfigModule.forRoot({
      schema:  AppConfigSchema,
      sources: [
        { type: 'file', path: 'config/default.yaml' },
        { type: 'file', path: 'config/${NODE_ENV}.yaml', optional: true },
        { type: 'env',  prefix: 'APP_' },
      ],
      validateOnStartup: true,
      watchForChanges:   process.env.NODE_ENV !== 'production',
    }),

    LoggerModule.forRoot({
      level: process.env.NODE_ENV === 'production' ? 'info' : 'debug',
      transports: [new ConsoleTransport()],
      processors: [
        new RedactionProcessor(['password', 'token', 'headers.authorization', 'creditCard.*']),
      ],
    }),

    // ── Foundation ─────────────────────────────────────────────────────
    TitanRedisModule.forRootAsync({
      useFactory: (config: ConfigService) => ({
        clients: [
          { namespace: 'default', url: config.get('redis.url'), db: 0 },
          { namespace: 'cache',   url: config.get('redis.url'), db: 1 },
          { namespace: 'rl',      url: config.get('redis.url'), db: 4 },
        ],
      }),
      inject: [ConfigService],
    }),

    // ── Database ───────────────────────────────────────────────────────
    TitanDatabaseModule.forRootAsync({
      useFactory: (config: ConfigService) => ({
        connection: {
          dialect:        'postgres',
          connection:     config.get('database.url'),
          pool:           { min: 2, max: 20 },
          migrationsPath: './migrations',
          coerceBigint:   true,
        },
        // Global plugins are a Kysera plugin list, not a boolean map.
        kysera: {
          plugins: ['soft-delete', 'timestamps', 'audit'],
        },
        // RLS is enabled per-repository via @Policy/@Filter decorators
        // (see the Multi-tenant SaaS recipe), not a global plugin toggle.
      }),
      inject: [ConfigService],
    }),

    // ── Cache (L1 in-memory + L2 Redis) ────────────────────────────────
    TitanCacheModule.forRootAsync({
      useFactory: (redis: RedisService) => ({
        multiTier: true,
        l1: { maxSize: 5_000,  ttl: 60 },             // hot — 1 min
        l2: { client: redis.getClient('cache'), ttl: 3_600, prefix: 'cache:' },
        evictionPolicy: 'lru',
      }),
      inject: [RedisService],
    }),

    // ── Auth ───────────────────────────────────────────────────────────
    TitanAuthModule.forRootAsync({
      useFactory: (config: ConfigService) => ({
        algorithm:    'HS256',
        jwtSecret:    config.get('auth.jwtSecret'),
        issuer:       config.get('auth.issuer'),
        audience:     config.get('auth.audience'),
        cacheEnabled: true,
        cacheMaxSize: 5_000,
        cacheTTL:     300_000,                         // 5 min
      }),
      inject: [ConfigService],
    }),

    // ── Rate limiting ──────────────────────────────────────────────────
    TitanRateLimitModule.forRootAsync({
      useFactory: (config: ConfigService) => ({
        enabled:         true,
        strategy:        'sliding-window',
        storageType:     'redis',                      // shared across pods
        defaultLimit:    config.get('rateLimit.defaultLimit'),
        defaultWindowMs: config.get('rateLimit.defaultWindowMs'),
        keyPrefix:       'rl',
        tiers: {
          free:       { name: 'free',       limit: 60,     windowMs: 60_000 },
          pro:        { name: 'pro',        limit: 1_000,  windowMs: 60_000 },
          enterprise: { name: 'enterprise', limit: 10_000, windowMs: 60_000 },
        },
      }),
      inject: [ConfigService],
    }),

    // ── Health probes ──────────────────────────────────────────────────
    TitanHealthModule.forRootAsync({
      // The database indicator wants an object with raw(), execute() or
      // query() — a Kysely instance has none of them, so wrap it. Passing the
      // connection straight through throws on the first check.
      useFactory: async (db: DatabaseManager, redis: RedisService) => ({
        enableMemoryIndicator:    true,
        enableEventLoopIndicator: true,
        enableDatabaseIndicator:  true,
        databaseConnection: {
          execute: async (q: string) => { await sql.raw(q).execute(await db.getConnection()); },
        },
        enableRedisIndicator:     true,
        redisClient:              redis.getClient('default'),
        memoryThresholds:         { heapDegradedThreshold: 0.8, heapUnhealthyThreshold: 0.95 },
        eventLoopThresholds:      { lagDegradedThreshold: 50, lagUnhealthyThreshold: 200 },
        timeout:                  3_000,
        enableCaching:            true,
        cacheTtl:                 1_000,
        enableRpcService:         true,                 // expose via Netron
        version:                  process.env.APP_VERSION,
      }),
      inject: [DATABASE_MANAGER, RedisService],
    }),

    // ── Metrics ────────────────────────────────────────────────────────
    TitanMetricsModule.forRoot({
      appName:    'my-api',
      collection: {
        enabled:  true,
        interval: 10_000,
        process:  true,
        system:   true,
        rpc:      true,                                 // auto-collect Netron call metrics
      },
      storage: { type: 'memory', batchSize: 500, flushInterval: 10_000 },
      retention: { maxAge: '7d', cleanupInterval: 3_600_000 },
    }),

    // ── Your feature modules ───────────────────────────────────────────
    UsersModule,
    OrdersModule,
    BillingModule,
  ],
})
export class AppModule {}
```

## A typical `@Service`

```typescript
import { Service, Inject } from '@omnitron-dev/titan';
import { Public } from '@omnitron-dev/titan/netron';
import { Cacheable, CacheInvalidate } from '@omnitron-dev/titan-cache';
import { InjectRepository } from '@omnitron-dev/titan-database';
import { RequireAuth } from '@omnitron-dev/titan-auth';
import { RateLimit, RATE_LIMIT_SERVICE_TOKEN, type IRateLimitService } from '@omnitron-dev/titan-ratelimit';
import { Metrics }     from '@omnitron-dev/titan-metrics';

@Service('users@1.0.0')
class UsersService {
  constructor(
    @InjectRepository(UsersRepository) private readonly repo: UsersRepository,
    // @RateLimit reads the service off this exact property name.
    @Inject(RATE_LIMIT_SERVICE_TOKEN) private readonly __rateLimitService__: IRateLimitService,
  ) {}

  @Public()
  @RequireAuth({ allowAnonymous: false })
  @RateLimit({ limit: 100, windowMs: 60_000 })   // key = {class}:{method}:{arg0} by default
  @Cacheable({ cacheName: 'users', keyPrefix: 'u', ttl: 60, tags: (id) => [`user:${id}`] })
  @Metrics('users.findById')                      // tracks total + duration + errors
  async findById(id: string) {
    return this.repo.find(id);
  }

  @Public()
  @RequireAuth({ roles: ['admin'] })
  @CacheInvalidate({ cacheName: 'users', tags: (input) => [`user:${input.id}`] })
  async update(input: { id: string; patch: Partial<User> }) {
    return this.repo.update(input.id, input.patch);
  }
}
```

## Cross-module wiring notes

| Concern               | Wiring detail                                                                                                                                                                                                                                                |
| --------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| Redis namespaces      | `default` (general), `cache` (L2), `rl` (rate limits) — isolate per DB index to prevent collisions                                                                                                                                                           |
| Cache L2 client       | `TitanCacheModule.forRootAsync` injects `RedisService` and gets the `cache` namespace explicitly                                                                                                                                                             |
| Health → database     | `await db.getConnection()` — `getConnection()` is async; the awaited `Kysely` instance is what `DatabaseHealthIndicator` accepts                                                                                                                             |
| Health → redis        | Pass `redis.getClient('default')` — uses `.ping()` for liveness                                                                                                                                                                                              |
| Auth cache vs JWT TTL | `cacheTTL: 300_000` < typical JWT expiry; revoked tokens stay valid until cache expires. Lower for high-churn revocation.                                                                                                                                    |
| Rate limit key shape  | `@RateLimit` takes an options object only. The key defaults to `{class}:{method}:{arg0}`; pass `keyGenerator: (...args) => ...` to fold in the auth identity. The service must inject the limiter as `__rateLimitService__` or the decorator silently no-ops |
| Metrics + Netron      | `collection.rpc: true` auto-instruments every `@Public` method — no per-method `@Metrics` needed unless you want a custom metric name (`@Metrics('users.findById')`)                                                                                         |

## Production checklist

- [ ] `validateOnStartup: true` on ConfigModule — misconfig fails at boot
- [ ] `watchForChanges: false` in production — hot-reload is dev-only
- [ ] `LoggerModule.processors` includes `RedactionProcessor` with **every** sensitive path
- [ ] `auth.cacheTTL` ≤ acceptable revocation window
- [ ] Database `pool.max` ≤ Postgres `max_connections` ÷ pod count
- [ ] Rate limit `storageType: 'redis'` (not `'memory'`) for multi-pod
- [ ] Kubernetes probes wired:
  `yaml
      livenessProbe:  { httpGet: { path: /healthz, port: 3000 } }
      readinessProbe: { httpGet: { path: /readyz,  port: 3000 } }
      `
- [ ] `/metrics` endpoint exposed and scraped by Prometheus
- [ ] Logs shipping off-host (sidecar / fluentbit / stdout to log aggregator)
- [ ] Database migrations run as a separate step (not at app boot in production)

## See also

- [Worker fleet](/docs/titan/recipes/worker-fleet) — when the workload is async
  jobs, not request handlers
- [Observability stack](/docs/titan/recipes/observability-stack) — beyond metrics:
  traces + telemetry-relay for cross-pod shipping
- [Multi-tenant SaaS](/docs/titan/recipes/multi-tenant-saas) — extends this stack
  with RLS-driven tenant isolation

---

---
id: "titan/recipes/index"
title: "Recipes"
description: "End-to-end multi-module compositions for common backends."
---

# Recipes

The per-module pages cover what each module does. This section
shows how they **compose** — real stacks for the most common
backend shapes, with full `forRoot` configs and the cross-module
wiring that makes them work together.

Each recipe is read top to bottom: pick the one matching your
workload, copy the `AppModule`, replace the env values, ship.

## Recipes

| Recipe                                                               | Use case                                                                           |
| -------------------------------------------------------------------- | ---------------------------------------------------------------------------------- |
| [API service stack](/docs/titan/recipes/api-service)                 | Public HTTP / WS API: auth, database, cache, rate limiting, health, metrics        |
| [Worker fleet](/docs/titan/recipes/worker-fleet)                     | Background jobs across pods: process manager, distributed lock, scheduler, metrics |
| [Observability stack](/docs/titan/recipes/observability-stack)       | Logs / metrics / traces / health, shipped via the relay                            |
| [Notifications pipeline](/docs/titan/recipes/notifications-pipeline) | Producer → queue → worker → channels, with rate limit + preferences                |
| [Multi-tenant SaaS](/docs/titan/recipes/multi-tenant-saas)           | RLS-isolated database, contextual injection, tenant-scoped rate limits             |

## How to read a recipe

Every recipe has the same shape:

1. **Shape** — what kind of backend this is and when to reach for it.
2. **Architecture diagram** — the modules involved and how they pass data.
3. **`AppModule`** — the canonical root module with every `forRoot`
   you need.
4. **Cross-module wiring notes** — the non-obvious bits: which
   token from module A is consumed by module B, what order
   matters, what config values must agree.
5. **Production checklist** — what to verify before shipping.

## Module dependency map

```mermaid
flowchart LR
  subgraph Foundation
    Redis[titan-redis]
    Logger[LoggerModule built-in]
    Config[ConfigModule built-in]
  end

  subgraph Storage
    DB[titan-database]
    Cache[titan-cache]
  end

  subgraph Coordination
    Lock[titan-lock]
    Discovery[titan-discovery]
    Scheduler[titan-scheduler]
  end

  subgraph Edge
    Auth[titan-auth]
    RateLimit[titan-ratelimit]
  end

  subgraph Workflows
    Events[titan-events]
    Notify[titan-notifications]
    PM[titan-pm]
  end

  subgraph Observability
    Health[titan-health]
    Metrics[titan-metrics]
    Relay[titan-telemetry-relay]
  end

  Cache -.L2.-> Redis
  Lock --> Redis
  Discovery --> Redis
  RateLimit -.optional.-> Redis
  Notify --> Redis
  Notify --> Events
  Scheduler -.distributed.-> Lock
  Health -.indicator.-> DB
  Health -.indicator.-> Redis
  Metrics -.ships via.-> Relay
```

Use this map to plan your stack: start at the workload (API /
worker / notifications) and trace dependencies.

---

---
id: "titan/recipes/multi-tenant-saas"
title: "Multi-tenant SaaS"
description: "Tenant-isolated database (RLS), contextual injection per tenant, tenant-scoped rate limits."
---

# Multi-tenant SaaS

A SaaS backend where one process serves many tenants and **must
not** leak data between them. Three load-bearing primitives make
this safe and ergonomic:

1. **Row-level security on the database.** Every query
   automatically constrained to the current tenant.
2. **Contextual injection in the DI container.** The same
   service token can resolve different per-tenant providers
   without your code knowing.
3. **Tenant-scoped rate limits.** Free / pro / enterprise tiers
   per tenant, not per process.

This recipe extends the [API service stack](/docs/titan/recipes/api-service) —
read that first; the differences here are tenant-isolation
patterns layered on top.

## Shape

- **Identity → tenant.** JWT carries `tenantId`; auth middleware
  sets it on the request context.
- **RLS at every query.** `@Policy`, `@Filter`, `@Allow`, `@Deny`
  on repositories constrain results by `tenantId`.
- **Per-tenant resolution.** A `STORAGE` token resolves to S3
  for enterprise tenants, local disk for the free tier — same
  code, different backend.
- **Tiered limits.** Each tenant's rate limit reflects their plan.
- **Per-tenant cache keys.** Cache keys include the tenant prefix
  so reads never cross tenants.

## Architecture

```mermaid
flowchart LR
  Req[Incoming request<br/>JWT with tenantId]
  Req --> Auth[titan-auth<br/>validate JWT]
  Auth --> Ctx[ContextManager<br/>set TENANT_ID]
  Ctx --> Strat[ResolutionStrategy<br/>per-tenant provider selection]
  Strat --> Service[Your @Service classes]
  Service --> RLS[titan-database<br/>RLS-enforced query]
  Service --> Cache[titan-cache<br/>key prefixed with tenantId]
  Service --> RL[titan-ratelimit<br/>tier from tenant plan]
  RLS --> PG[(PostgreSQL)]
  Cache -.L2.-> Redis[(Redis)]
  RL --> Redis
```

## Setting the tenant context

Custom auth-middleware step that, after JWT verify, stamps the
`tenantId` onto the request context:

```typescript
import { Injectable, Inject } from '@omnitron-dev/titan';
import { ContextManager, createContextKey }
  from '@omnitron-dev/titan/nexus';
import { JWTService, JWT_SERVICE_TOKEN } from '@omnitron-dev/titan-auth';

export const TENANT_ID = createContextKey<string>('tenant.id');
export const USER_TIER = createContextKey<string>('user.tier');

@Injectable()
class TenantContextMiddleware {
  constructor(
    @Inject(JWT_SERVICE_TOKEN) private readonly jwt:     JWTService,
    private readonly context:                            ContextManager,
  ) {}

  async handle(request: { headers: Record<string, string> }, next: () => Promise<unknown>) {
    const token = request.headers['authorization']?.replace(/^Bearer /, '');
    if (token) {
      // IJWTPayload uses snake_case `tenant_id`; `tier` is a custom claim.
      const claims = await this.jwt.verify(token);
      // `set` lives on the ContextProvider, not on the manager: the manager
      // selects and scopes providers, the provider holds the values.
      const ctx = this.context.getCurrentContext();
      ctx.set(TENANT_ID, claims.tenant_id as string);
      ctx.set(USER_TIER, claims.tier as string);
    }
    return next();
  }
}
```

Wire as a Netron middleware (or in your transport adapter), so it
runs before every `@Service` method.

## Per-tenant DI — contextual providers

The intent: a single `STORAGE` token resolves to S3 for enterprise
tenants and local disk for the free tier — same service code, different
backend, no `if` statements leaking into business logic.

`createContextAwareProvider` is an identity helper over the
`ContextAwareProvider` interface — a single object with a
`provide(context)` method and an optional `canProvide(context)` guard
(`createContextAwareProvider` in `packages/titan/src/nexus/context.ts`). There is **no**
`strategies` array and **no** `factory` key; the branching lives inside
`provide`, and the tenant is read from the resolution context's
`metadata`:

```typescript
import { createContextAwareProvider, createToken }
  from '@omnitron-dev/titan/nexus';

const STORAGE = createToken<IStorage>('Storage');

const storageProvider = createContextAwareProvider<IStorage>({
  // `context` is the active ResolutionContext; read tenant from its metadata.
  provide(context) {
    const tenant = context.metadata?.['tenant'] as { id: string; tier: string } | undefined;
    return tenant?.tier === 'enterprise' ? new S3Storage() : new LocalDiskStorage();
  },
});
```

The tenant lands in `context.metadata` because
`ContextManager.createResolutionContext` folds the active
`ContextProvider` into `metadata` (so the `TENANT_ID` your middleware
set, plus the built-in `ContextKeys.Tenant`, are visible there). The
built-in `ResolutionStrategy` classes (`TenantStrategy`, etc.) are a
**separate** mechanism — `ContextManager.selectProvider()` uses them to
pick among *several* registered providers; they are not passed into
`createContextAwareProvider`.

Resolve `storageProvider` in **`Scope.Request`** so the `provide`
callback re-runs per request (a `Singleton` would cache the first
tenant's backend for everyone). See
[DI / Contextual Injection](/docs/titan/di/contextual-injection) for the full
contextual-provider model, `@InjectContext`, and writing a custom
`ResolutionStrategy`.

## RLS on every repository

```typescript
import { BaseRepository, Repository, Policy, Filter, BypassRLS }
  from '@omnitron-dev/titan-database';
import { rlsContext } from '@kysera/rls';

@Repository<Order>({ table: 'orders' })
@Policy({ skipFor: ['admin', 'service_role'] })       // these roles bypass RLS
// BaseRepository is an alias of @kysera/repository's ContextAwareRepository,
// which takes two parameters: the DB schema and the table name. The row type
// comes from Database['orders'] — there is no third parameter.
class OrdersRepository extends BaseRepository<Database, 'orders'> {
  // Filter: auto-adds a WHERE clause to reads. The context is the
  // @kysera/rls auth context (ctx.auth), NOT the DI ContextManager.
  @Filter({ operations: ['select', 'update', 'delete'] })
  tenantFilter(ctx: { auth: { tenantId: string } }) {
    return { tenant_id: ctx.auth.tenantId };
  }

  @BypassRLS()
  async adminListAllAcrossTenants() {
    // Reserved for admin / system flows
    return this.findAll();
  }
}
```

The RLS context is established with `rlsContext.runAsync(...)` (from
`@kysera/rls`) around the query — wire it from your auth middleware so
every request runs inside its tenant's context:

```typescript
import { rlsContext } from '@kysera/rls';

return rlsContext.runAsync(
  { auth: { userId, tenantId, roles, isSystem: false }, timestamp: new Date() },
  async () => this.ordersRepo.findAll(),   // automatically filtered by tenant_id
);
```

Every regular query through `OrdersRepository` is then constrained to
the tenant in the active RLS context. `@BypassRLS` is the explicit
escape hatch for cross-tenant operations — every use should be
audited.

> RLS enforcement requires the Kysera `rlsPlugin` to be active on the
> connection (add it to `kysera.plugins`). The `@Policy`/`@Filter`/
> `@Allow`/`@Deny` decorators declare the rules; the plugin enforces them.

## Tenant-scoped cache keys

`@Cacheable`'s `keyGenerator`/`tags` callbacks receive the **method
arguments** (`(...args)`), not an ambient context. So the `tenantId`
the key is prefixed with must be reachable from the args — the simplest
contract is to pass it explicitly:

```typescript
import { Cacheable, CacheInvalidate } from '@omnitron-dev/titan-cache';

@Service('users@1.0.0')
class UsersService {
  @Public()
  @Cacheable({
    cacheName:    'users',
    keyGenerator: (tenantId: string, id: string) => `${tenantId}:u:${id}`,
    ttl:          60,
    tags:         (tenantId: string, id: string) => [`tenant:${tenantId}:user:${id}`],
  })
  async findById(tenantId: string, id: string) {
    return this.repo.find(id);            // RLS applies inside the repo
  }

  @CacheInvalidate({
    cacheName: 'users',
    tags:      (tenantId: string, input: { id: string }) => [`tenant:${tenantId}:user:${input.id}`],
  })
  async update(tenantId: string, input: { id: string; patch: Partial<User> }) {
    return this.repo.update(input.id, input.patch);
  }
}
```

Cache keys are prefixed with `tenantId`; a cache hit for tenant A
**cannot** be returned to tenant B. (If you prefer to derive the
tenant from the request context instead of a parameter, read it from
the nexus `ContextManager` inside the method body and build the key
with `keyPrefix` + a per-call cache handle rather than `keyGenerator`.)

## Tenant-tier rate limits

```typescript
TitanRateLimitModule.forRoot({
  storageType: 'redis',
  strategy:    'sliding-window',
  defaultTier: { name: 'free', limit: 100, windowMs: 60_000 },
  tiers: {                                          // each tier needs a `name`
    free:       { name: 'free',       limit: 100,     windowMs: 60_000 },
    pro:        { name: 'pro',        limit: 1_000,   windowMs: 60_000 },
    enterprise: { name: 'enterprise', limit: 100_000, windowMs: 60_000 },
  },
})
```

In your service. `@RateLimit` takes an options object only; `keyGenerator`
receives the method args (not a context), and `tier` is a tier-name
string. Pass the tenant (and its resolved tier) as arguments:

```typescript
@Public()
@RateLimit({
  tier:         'pro',                              // tier name from this tenant's plan
  keyGenerator: (tenantId: string) => `tenant:${tenantId}`,
})
async create(tenantId: string, input: CreateInput) { /* … */ }
```

> The decorator also needs the limiter injected as `__rateLimitService__`
> (`@Inject(RATE_LIMIT_SERVICE_TOKEN)`) — see the API service stack — or it
> silently allows every request.

## Cross-module wiring notes

| Concern                        | Wiring detail                                                                                                                                                                                                                                                    |
| ------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Context propagation            | Two distinct contexts: the nexus `ContextManager` (set in middleware) feeds `ResolutionStrategy` + per-tenant DI; the `@kysera/rls` auth context (set via `rlsContext.runAsync`) feeds RLS `@Filter`/`@Allow` rules. Cache/rate-limit keys come from method args |
| RLS bypass                     | `@BypassRLS` is structural — every use case needs a written justification; pair with an audit log                                                                                                                                                                |
| Cache key prefix               | Always fold the tenant into the `keyGenerator` (from a method arg) — never key on a raw id                                                                                                                                                                       |
| Per-tenant database (advanced) | For physical isolation, register multiple named connections (`TitanDatabaseModule.forRoot({ connections: { tenantA, tenantB }})`) + contextual provider picks the right one                                                                                      |
| Rate-limit key                 | Tenant-scoped key prevents one tenant from exhausting another's allowance                                                                                                                                                                                        |
| JWT claims                     | `tenantId` and `tier` must be signed claims on the JWT — they cannot be supplied by the client                                                                                                                                                                   |
| Strategy purity                | A `ResolutionStrategy`'s `applies`/`select` (and a provider's `provide`) must be pure functions of the context — side effects make them untestable and order-dependent                                                                                           |

## Production checklist

- [ ] **JWT carries `tenantId` and `tier` as signed claims** (not header-supplied)
- [ ] **Every repository has `@Filter` for tenant scope** OR `@BypassRLS` justified
- [ ] **Every cache key includes the tenant prefix** — no exceptions
- [ ] **Rate-limit keys include the tenant prefix** — same
- [ ] **`STORAGE` (or other tenant-conditional providers) resolved per request** (per-request scope) — a `Singleton` would cache the first tenant's instance for everyone
- [ ] **Cross-tenant queries (admin flows) audited** — every `@BypassRLS` call logged
- [ ] **`TENANT_ID` and `USER_TIER` context keys defined in one place** and imported everywhere — typos in keys silently break isolation
- [ ] **Integration tests cover cross-tenant attempts** — assert leakage attempts get empty results, not access

## Anti-patterns specific to multi-tenant

- **Caching without tenant prefix.** Cache hit for tenant A returned
  to tenant B. Easy to miss in code review; catch with a lint rule
  that requires `keyGenerator` on `@Cacheable`.
- **`Singleton` contextual providers.** First tenant's instance
  cached forever. Always `Scope.Request`.
- **Per-tenant database connections without pool limits.** A pod
  with 100 tenants × pool max 20 = 2_000 connections; Postgres
  rejects after a few hundred.
- **`@BypassRLS` without an audit trail.** Use a per-method audit
  decorator that logs every bypass call.
- **`tenantId` from a header.** Easily spoofed. Always derive from
  signed JWT claims.

## See also

- [API service stack](/docs/titan/recipes/api-service) — the base this recipe extends
- [DI / Contextual Injection](/docs/titan/di/contextual-injection) — the underlying pattern
- [`titan-database` RLS](/docs/titan/modules/database#row-level-security) — `@Policy`, `@Filter`, `@Allow`, `@Deny`, `@BypassRLS`
- [`titan-cache`](/docs/titan/modules/cache) — `keyGenerator` for per-tenant keys
- [`titan-ratelimit`](/docs/titan/modules/ratelimit) — tiered plans

---

---
id: "titan/recipes/notifications-pipeline"
title: "Notifications pipeline"
description: "Producer → queue → worker → channels, with rate limit and preferences."
---

# Notifications pipeline

The shape for a notifications system that respects user preferences,
rate-limits per channel, isolates flaky delivery providers from the
request path, and runs on a worker fleet — so an SMS provider outage
doesn't slow down API responses.

## Shape

- **Producer.** API pods emit notification events; return
  immediately without waiting for delivery.
- **Queue.** Rotif (Redis-backed) carries events to workers,
  decoupling the request path from delivery.
- **Worker.** A separate pod consumes events, resolves channel
  targets per recipient, persists the notification, renders the
  template, calls the channel.
- **Rate-limited.** Per-user / per-channel limits prevent runaway
  loops from spamming users.
- **Preferences.** Per-user opt-in/out per channel.
- **Templates.** One payload renders to N channels with
  per-channel transformations.

## Architecture

```mermaid
flowchart LR
  subgraph "API pod (producer)"
    Code[Service code]
    Code -- NotificationsService.send --> Pub[Rotif publish]
  end

  Pub --> Q[(Rotif queue<br/>Redis)]

  subgraph "Worker pod (consumer)"
    Sub[Rotif subscribe]
    Q --> Sub
    Sub --> Resolver[TargetResolver<br/>look up channels per recipient]
    Resolver --> Persister[Persister<br/>write to inbox / DB]
    Resolver --> Rate{Rate-limit<br/>per channel}
    Rate -- allowed --> Pref{Preferences<br/>per channel}
    Pref -- opted in --> Tpl[TemplateEngine<br/>render]
    Tpl --> Chan[Channel.deliver<br/>email / sms / push / webhook / inapp]
    Persister --> Real[RealtimeSignaler<br/>WS / SSE push]
  end

  Chan --> Provider[(SMTP / Twilio / FCM / …)]
```

## `AppModule` — producer (API pod)

```typescript
import { Module } from '@omnitron-dev/titan';
import { ConfigModule } from '@omnitron-dev/titan/module/config';
import { LoggerModule } from '@omnitron-dev/titan/module/logger';

import { TitanRedisModule } from '@omnitron-dev/titan-redis';
import { EventsModule } from '@omnitron-dev/titan-events';
import { NotificationsModule } from '@omnitron-dev/titan-notifications';

@Module({
  imports: [
    ConfigModule.forRoot({/* … */}),
    LoggerModule.forRoot({ level: 'info' }),

    TitanRedisModule.forRoot({
      config: { url: process.env.REDIS_URL, db: 0 },
    }),

    EventsModule.forRoot({
      wildcard:     true,
      maxListeners: 100,
      history:      { enabled: false },
    }),

    NotificationsModule.forRoot({
      redis: process.env.REDIS_URL,

      // Optional: route through Rotif for cross-pod delivery
      transport: {
        rotif: { /* rotif transport options */ },
      },

      defaultChannels: ['inApp', 'email'],   // channel ids are camelCase: 'inApp', not 'inapp'
      enableInApp:     true,
      enableWebhook:   true,

      templates: {
        enabled:      true,
        cacheEnabled: true,
        cacheTTL:     60_000,
      },

      rateLimiterConfig: {
        defaultLimits: { perMinute: 10, perHour: 100, perDay: 1_000 },
        channelLimits: { sms: { perMinute: 1, perHour: 10 } },
        enableBurstDetection: true,
      },

      preferenceStoreConfig: {
        // Each channel maps to an object, not a bare boolean.
        // Channel ids are camelCase: 'inApp', not 'inapp'.
        defaultPreferences: {
          channels: {
            email: { enabled: true },
            sms:   { enabled: false },
            push:  { enabled: true },
            inApp: { enabled: true },
          },
        },
      },
    }),

    UsersModule,
  ],
})
export class AppModule {}
```

## Producer side — emitting a notification

```typescript
import { Service, Inject } from '@omnitron-dev/titan';
import { Public } from '@omnitron-dev/titan/netron';
import {
  NOTIFICATIONS_SERVICE,
  type NotificationsService,
} from '@omnitron-dev/titan-notifications';

@Service({ name: 'users' })
class UsersService {
  constructor(
    @Inject(NOTIFICATIONS_SERVICE) private readonly notify: NotificationsService,
  ) {}

  @Public()
  async create(input: CreateInput) {
    const user = await this.repo.create(input);

    // Returns immediately — actual delivery happens on a worker pod.
    // Recipient keys on `id` (not `userId`); payload needs `type`/`title`/`message`.
    await this.notify.send(
      { id: user.id, email: user.email },
      { type: 'announcement', title: 'Welcome', message: `Welcome, ${user.name}!`, data: { name: user.name } },
      { channels: ['email', 'inApp'] },
    );

    return user;
  }
}
```

## Worker pod — consuming and delivering

```typescript
import { Module, createToken } from '@omnitron-dev/titan';
import {
  NotificationsModule,
  type INotificationTargetResolver,
  type INotificationPersister,
  type INotificationRealtimeSignaler,
} from '@omnitron-dev/titan-notifications';

const USER_TARGET_RESOLVER_TOKEN = createToken<INotificationTargetResolver>('UserTargetResolver');
const NOTIFICATION_PERSISTER_TOKEN = createToken<INotificationPersister>('NotificationPersister');
const REALTIME_SIGNALER_TOKEN = createToken<INotificationRealtimeSignaler>('RealtimeSignaler');

@Module({
  imports: [
    ConfigModule.forRoot({/* … */}),
    LoggerModule.forRoot({ level: 'info' }),
    TitanRedisModule.forRoot({ config: { url: process.env.REDIS_URL } }),

    NotificationsModule.forWorker({
      targetResolver:   USER_TARGET_RESOLVER_TOKEN,
      persister:        NOTIFICATION_PERSISTER_TOKEN,
      realtimeSignaler: REALTIME_SIGNALER_TOKEN,
      workerOptions: {
        // There is no `concurrency`. The worker runs one XREADGROUP loop and
        // takes a BATCH per read; `readCount` is the knob that sizes it.
        readCount: 100,
        blockTimeoutMs: 5_000,
      },
    }),
  ],
  providers: [
    { provide: USER_TARGET_RESOLVER_TOKEN, useClass: MyUserResolver },
    { provide: NOTIFICATION_PERSISTER_TOKEN, useClass: MyPersister },
    { provide: REALTIME_SIGNALER_TOKEN, useClass: MyRealtimeSignaler },
  ],
})
export class WorkerModule {}
```

`MyUserResolver`, `MyPersister`, `MyRealtimeSignaler` are the
three integration points you implement once per project. They
wire the abstract worker to your concrete user store, your
inbox storage, and your realtime channel.

## A custom channel — e.g. SMTP email

```typescript
import { AbstractEmailChannel, type EmailContent } from '@omnitron-dev/titan-notifications';
import nodemailer from 'nodemailer';

// AbstractEmailChannel fixes `name = 'email'` / `type = ChannelType.Email`
// and renders the payload; you only implement the transport.
class SmtpEmailChannel extends AbstractEmailChannel {
  private readonly transport = nodemailer.createTransport({/* … */});

  async sendEmail(to: string, content: EmailContent): Promise<{ messageId: string }> {
    const info = await this.transport.sendMail({
      to,
      subject: content.subject,
      html:    content.html,
      text:    content.text,
    });
    return { messageId: info.messageId };
  }
}
```

Register it via `forRoot({ channels: [new SmtpEmailChannel()] })`.

## Cross-module wiring notes

| Concern                 | Wiring detail                                                                                                  |
| ----------------------- | -------------------------------------------------------------------------------------------------------------- |
| Producer vs worker mode | `NotificationsModule.forRoot` for producers, `NotificationsModule.forWorker` for workers — different DI shapes |
| Rotif transport         | Configured under `transport.rotif` on the producer side; worker consumes via the same Redis instance           |
| Rate-limiter location   | `RedisRateLimiter` runs on the worker side, before each channel delivery                                       |
| Preference enforcement  | `RedisPreferenceStore` checked on the worker — producer doesn't know per-user prefs                            |
| Template caching        | `templates.cacheEnabled: true` + `cacheTTL` avoid re-fetching template definitions on every send               |
| Realtime + in-app       | `RealtimeSignaler` pushes to WS/SSE; in-app channel persists the notification to the inbox                     |
| Webhook signature       | `webhookConfig.signatureSecret` + `signatureHeader` for HMAC-signed delivery                                   |
| DLQ                     | Failed deliveries land in a Rotif dead-letter queue (see Rotif config)                                         |
| Redis retention         | Off in every form unless configured — see the checklist item below                                             |

## Production checklist

- [ ] **Producer + worker pods deployed separately** — independent scaling
- [ ] **`rateLimiterConfig.channelLimits.sms` set** — SMS is expensive and rate-limited by providers
- [ ] **`preferenceStoreConfig.defaultPreferences` set** — compliance with user opt-outs
- [ ] **Templates registered** at boot, not at first use
- [ ] **DLQ monitored** — repeated failures should alert
- [ ] **Redis retention configured** — every retention mechanism Rotif has is OFF by default, so a
  pipeline that never sets them grows in Redis memory for as long as it runs:
  \- `maxStreamLength` (or `minStreamId`) — without one of these `publish` never trims the stream
  it writes to. The trim runs inside the publish script, so it is also the producer that bounds
  the stream: a stream stops being trimmed when publishing stops, at whatever size it reached.
  \- `dlqCleanup.enabled` — defaults to `false`, and the sibling limits it gates
  (`dlqCleanup.maxAge`, `dlqCleanup.maxSize`) carry usable values that simply do not apply
  until it is turned on. A monitored DLQ is still an unbounded one.
  Sizing these is a deployment decision, but leaving all three unset is not a neutral default.
- [ ] **Channel-specific timeouts** — e.g. `webhookConfig.timeout: 5_000`
- [ ] **Realtime signaler graceful degradation** — if WS gateway is down, notification is still persisted
- [ ] **Persistence backend backed up** — in-app inbox should survive redis loss
- [ ] **Test channels** in staging — `MockEmailChannel` etc. for end-to-end verification

## See also

- [`titan-notifications`](/docs/titan/modules/notifications) — full reference
- [`titan-events`](/docs/titan/modules/events) — in-process events the producer emits
- [`titan-ratelimit`](/docs/titan/modules/ratelimit) — alternative rate-limit backing
- [`titan-redis`](/docs/titan/modules/redis) — Rotif transport runs on this

---

---
id: "titan/recipes/oauth-callback"
title: "OAuth callback"
description: "PKCE-protected authorisation-code flow with state validation and signed-URL session bootstrap."
---

# OAuth callback

Handling an OAuth 2.0 authorisation-code flow correctly has a long
checklist: state validation, PKCE, token exchange, refresh
storage, and bootstrapping the user's session. This recipe shows
the canonical shape using `titan-auth`, `titan-cache` (for the
short-lived state), and the rest of the standard stack.

## Shape

- **PKCE.** The initiating page generates `code_verifier` +
  `code_challenge` (SHA-256). The callback validates against the
  stored verifier.
- **State parameter.** Random, opaque, short-TTL, tied to one
  session. Replaying a state from another browser fails.
- **Token exchange.** Authorisation code → access + refresh
  tokens — server-side only, never in the browser.
- **Signed session bootstrap.** Issue a signed URL with a JWT
  pointing back to your app's session endpoint.

## Architecture

```mermaid
sequenceDiagram
  participant U as User
  participant App as Your app
  participant P as OAuth provider

  U->>App: GET /auth/start
  App->>App: Generate state + PKCE verifier<br/>cache(state → verifier, TTL 10m)
  App-->>U: Redirect to provider with state + code_challenge
  U->>P: Authenticate
  P-->>U: Redirect to /auth/callback?code=&state=
  U->>App: GET /auth/callback?code=...&state=...
  App->>App: Lookup state<br/>retrieve verifier<br/>pop from cache
  App->>P: POST /token { code, code_verifier, redirect_uri }
  P-->>App: { access_token, refresh_token, id_token }
  App->>App: Verify id_token<br/>create local session
  App->>App: Sign a session-bootstrap JWT
  App-->>U: Redirect to /session/bootstrap?token=...
```

## Setup

```typescript
import { Module } from '@omnitron-dev/titan';
import { ConfigModule } from '@omnitron-dev/titan/module/config';
import { LoggerModule } from '@omnitron-dev/titan/module/logger';

import { TitanAuthModule } from '@omnitron-dev/titan-auth';
import { TitanCacheModule } from '@omnitron-dev/titan-cache';
import { TitanRedisModule } from '@omnitron-dev/titan-redis';
import { TitanRateLimitModule } from '@omnitron-dev/titan-ratelimit';

@Module({
  imports: [
    ConfigModule.forRoot({/* … */}),
    LoggerModule.forRoot({ level: 'info' }),

    TitanRedisModule.forRoot({ config: { url: env.REDIS_URL } }),

    // Short-lived auth state cache
    TitanCacheModule.forRoot({
      maxSize:        10_000,
      defaultTtl:     600,                    // 10 min
      evictionPolicy: 'ttl',
    }),

    TitanAuthModule.forRoot({
      algorithm:     'HS256',
      jwtSecret:     env.JWT_SECRET,
      issuer:        env.APP_ISSUER,
      audience:      env.APP_AUDIENCE,
      urlSigningKey: env.URL_SIGNING_KEY,
    }),

    TitanRateLimitModule.forRoot({
      enabled:         true,
      storageType:     'redis',
      defaultLimit:    10,                   // 10 OAuth starts per minute per IP
      defaultWindowMs: 60_000,
    }),

    AuthFlowModule,
  ],
})
export class AppModule {}
```

## Implementation

```typescript
import { Service, Inject } from '@omnitron-dev/titan';
import { Public } from '@omnitron-dev/titan/netron';
import { Errors } from '@omnitron-dev/titan/errors';
import {
  JWT_SERVICE_TOKEN, SIGNED_URL_SERVICE_TOKEN,
  type IJWTService, type ISignedUrlService,
} from '@omnitron-dev/titan-auth';
import { CACHE_SERVICE_TOKEN, type ICacheService }
  from '@omnitron-dev/titan-cache';
import { createHash, randomBytes } from 'node:crypto';

const PROVIDER_AUTHORIZE = 'https://idp.example.com/oauth/authorize';
const PROVIDER_TOKEN     = 'https://idp.example.com/oauth/token';
const CLIENT_ID          = process.env.OAUTH_CLIENT_ID!;
const CLIENT_SECRET      = process.env.OAUTH_CLIENT_SECRET!;
const REDIRECT_URI       = `${process.env.APP_BASE_URL}/auth/callback`;

@Service({ name: 'auth-flow' })
class AuthFlowService {
  constructor(
    @Inject(CACHE_SERVICE_TOKEN)        private readonly cache:  ICacheService,
    @Inject(JWT_SERVICE_TOKEN)          private readonly jwt:    IJWTService,
    @Inject(SIGNED_URL_SERVICE_TOKEN)   private readonly signer: ISignedUrlService,
  ) {}

  /**
   * Step 1: start the flow. Returns the provider URL to redirect to.
   */
  @Public()
  async start(): Promise<{ redirectTo: string }> {
    const state    = randomBytes(32).toString('base64url');
    const verifier = randomBytes(32).toString('base64url');
    const challenge = createHash('sha256').update(verifier).digest('base64url');

    // Cache state → verifier with TTL = 10 min
    await this.cache.getCache('auth-flow').set(`state:${state}`, verifier, { ttl: 600 });

    const url = new URL(PROVIDER_AUTHORIZE);
    url.searchParams.set('response_type',         'code');
    url.searchParams.set('client_id',             CLIENT_ID);
    url.searchParams.set('redirect_uri',          REDIRECT_URI);
    url.searchParams.set('state',                 state);
    url.searchParams.set('code_challenge',        challenge);
    url.searchParams.set('code_challenge_method', 'S256');
    url.searchParams.set('scope',                 'openid profile email');

    return { redirectTo: url.toString() };
  }

  /**
   * Step 2: handle the provider's callback.
   */
  @Public()
  async callback(code: string, state: string): Promise<{ bootstrapUrl: string }> {
    // Validate state — atomic get-and-delete so it can't be replayed
    const cache = this.cache.getCache<string>('auth-flow');
    const verifier = await cache.get(`state:${state}`);
    if (!verifier) throw Errors.unauthorized('invalid or expired state');
    await cache.delete(`state:${state}`);

    // Exchange code for tokens
    const response = await fetch(PROVIDER_TOKEN, {
      method:  'POST',
      headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
      body:    new URLSearchParams({
        grant_type:    'authorization_code',
        client_id:     CLIENT_ID,
        client_secret: CLIENT_SECRET,
        code,
        code_verifier: verifier,
        redirect_uri:  REDIRECT_URI,
      }),
    });
    if (!response.ok) throw Errors.unauthorized('token exchange failed');
    const tokens = await response.json();
    // tokens: { access_token, refresh_token, id_token, expires_in }

    // Verify the id_token (the OAuth provider's JWT proves the user)
    // For external IDPs, use TitanAuthModule with jwksUrl for this verify.
    // Project-specific: store refresh_token securely (encrypted-at-rest).

    // Bootstrap the local session via signed URL.
    // ISignedTokenPayload is { resourceId, resourcePath, operation, transform? };
    // stash bootstrap claims under `transform`.
    const bootstrapToken = await this.signer.createSignedToken(
      {
        resourceId:   'session',
        resourcePath: 'bootstrap',
        operation:    'read',
        transform: {
          accessTokenHash: createHash('sha256').update(tokens.access_token).digest('hex'),
        },
      },
      120,    // 2 min expiry — single-use
    );

    return {
      bootstrapUrl: `${process.env.APP_BASE_URL}/session/bootstrap?token=${bootstrapToken}`,
    };
  }
}
```

## Cross-module wiring notes

| Concern                        | Wiring detail                                                                                                               |
| ------------------------------ | --------------------------------------------------------------------------------------------------------------------------- |
| State cache name               | Dedicated cache (`'auth-flow'`) — keeps short-lived auth state separate from app data                                       |
| State key vs the value         | Cache key = `state:${state}` (opaque); value = `verifier`. State is what the URL carries; verifier never leaves the server. |
| State TTL                      | 10 minutes is generous — most users finish in seconds. Shorter is fine.                                                     |
| Atomic state consumption       | `get` + `delete` — prevents replaying the same state. The `getOrSet` pattern doesn't apply here.                            |
| PKCE algorithm                 | SHA-256 base64url; matches `code_challenge_method: 'S256'`                                                                  |
| Token storage                  | `refresh_token` is long-lived and sensitive — store encrypted at rest (`pgcrypto` or KMS)                                   |
| Rate-limit per IP              | Prevents brute-force `state` guessing against the cache                                                                     |
| Signed bootstrap URL           | Short TTL (≤ 2 min) and single-use; check + reject reuse on the bootstrap endpoint                                          |
| Tokens never reach the browser | Access + refresh tokens stay server-side; only your session JWT goes to the client                                          |

## Production checklist

- [ ] **PKCE enabled** — `code_challenge_method: 'S256'`, never `'plain'`
- [ ] **State validated atomically** — `get` then `delete`; no race window
- [ ] **State TTL** ≤ 15 minutes
- [ ] **Redirect URI exact-match** on the provider side (no wildcards)
- [ ] **Token exchange uses POST** with `Content-Type: application/x-www-form-urlencoded`
- [ ] **`client_secret` server-side only** — never in the browser
- [ ] **id_token verified** — JWKS-backed verify against the IDP's keys
- [ ] **`refresh_token` encrypted at rest** in the database
- [ ] **Bootstrap JWT** with very short TTL (≤ 2 min) and single-use semantics
- [ ] **Rate-limit `/auth/start` and `/auth/callback`** per IP
- [ ] **Detect and reject reused authorisation codes** — the provider should also do this; defense in depth
- [ ] **Detect impossible-time-window flows** (state created seconds ago coming back from a different IP)

## See also

- [`titan-auth`](/docs/titan/modules/auth) — JWT + signed URLs
- [`titan-cache`](/docs/titan/modules/cache) — state storage
- [`titan-ratelimit`](/docs/titan/modules/ratelimit) — protect both endpoints
- [API service stack](/docs/titan/recipes/api-service) — what runs after the user is authenticated

---

---
id: "titan/recipes/observability-stack"
title: "Observability stack"
description: "Logs, metrics, traces, health — collected and shipped via the telemetry relay."
---

# Observability stack

End-to-end observability without external SDKs: structured logs
with trace correlation, native counters / gauges / histograms,
W3C trace propagation across RPC calls, and a store-and-forward
relay that ships everything to your sink even through network
outages.

## Shape

- **Logs.** Structured JSON via `LoggerModule`, with `traceId` /
  `spanId` attached automatically.
- **Metrics.** Counters / histograms via `titan-metrics`, with
  Prometheus exposition + persistent storage.
- **Traces.** W3C `traceparent` carried through async scopes via
  `@omnitron-dev/titan/tracing`.
- **Health.** Standard probes via `titan-health`.
- **Shipping.** All three pillars buffered locally and shipped via
  `titan-telemetry-relay` — survives collector outages without
  data loss.

## Architecture

```mermaid
flowchart LR
  subgraph "Producer pod (your service)"
    Code[Service code]
    Code -- log --> Log[LoggerModule]
    Code -- metric --> Met[titan-metrics]
    Code -- span --> Trace[tracing]
    Health[titan-health]

    Log -- processor adds traceId --> Buf[Telemetry buffer]
    Met -- recordTyped --> Buf
    Trace -- end --> Buf
    Buf -- WAL on backpressure --> WAL[(local WAL)]
  end

  subgraph "Aggregator pod"
    AggBuf[Telemetry buffer]
    AggBuf --> Sink[Sink writer]
  end

  Buf -- Netron transport --> AggBuf
  Sink --> Backend[(OTLP collector / Loki / Tempo)]
  Sink --> Prom[(Prometheus)]
  Health --> Probe[Kubernetes probes]
```

## `AppModule` — producer side

```typescript
import { Module } from '@omnitron-dev/titan';
import { ConfigModule } from '@omnitron-dev/titan/module/config';
import {
  LoggerModule,
  ConsoleTransport,
  RedactionProcessor,
  type ILogProcessor,
} from '@omnitron-dev/titan/module/logger';
import { currentTrace } from '@omnitron-dev/titan/tracing';

import { TitanMetricsModule } from '@omnitron-dev/titan-metrics';
import { TitanHealthModule }  from '@omnitron-dev/titan-health';

// Custom processor: stamp traceId/spanId on every log line
const TraceContextProcessor: ILogProcessor = {
  process(record) {
    const trace = currentTrace();
    if (trace) {
      record.traceId = trace.traceId;
      record.spanId  = trace.spanId;
    }
    return record;
  },
};

@Module({
  imports: [
    ConfigModule.forRoot({/* ... */}),

    // ── Logging with redaction + trace correlation ─────────────────────
    LoggerModule.forRoot({
      level: process.env.NODE_ENV === 'production' ? 'info' : 'debug',
      transports: [new ConsoleTransport()],
      processors: [
        new RedactionProcessor(['password', 'token', 'headers.authorization']),
        TraceContextProcessor,
      ],
    }),

    // ── Metrics (with auto RPC instrumentation) ────────────────────────
    TitanMetricsModule.forRoot({
      appName: 'my-service',
      collection: {
        enabled:  true,
        interval: 10_000,
        process:  true,      // CPU / RSS / heap
        system:   true,      // load, free mem
        rpc:      true,      // every Netron call auto-counted
        custom:   true,
      },
      storage:   { type: 'memory', batchSize: 500, flushInterval: 10_000 },
      retention: { maxAge: '24h', cleanupInterval: 600_000 },
    }),

    // ── Health probes ──────────────────────────────────────────────────
    TitanHealthModule.forRoot({
      enableMemoryIndicator:    true,
      enableEventLoopIndicator: true,
      memoryThresholds:         { heapDegradedThreshold: 0.8, heapUnhealthyThreshold: 0.95 },
      eventLoopThresholds:      { lagDegradedThreshold: 50, lagUnhealthyThreshold: 200 },
      timeout:                  3_000,
      enableCaching:            true,
      cacheTtl:                 1_000,
      enableRpcService:         true,
      version:                  process.env.APP_VERSION,
    }),

    // ── Your feature modules ───────────────────────────────────────────
    UsersModule,
  ],
})
export class AppModule {}
```

## Wiring the telemetry relay

The relay is **not** a Titan module — instantiate it in your
bootstrap and wire it as a clientModule or via your DI container:

```typescript
import { TelemetryRelayService } from '@omnitron-dev/titan-telemetry-relay';
import { Application } from '@omnitron-dev/titan';

const relay = new TelemetryRelayService({
  role:   'producer',
  nodeId: process.env.HOSTNAME ?? 'unknown',
  buffer: { maxBufferSize: 10_000, flushIntervalMs: 5_000, maxBatchSize: 500 },
  // `wal: false` disables it; otherwise pass a config (retention is by segment count).
  wal:    { directory: './.wal', maxSizeBytes: 10 * 1024 * 1024, maxSegments: 7 },
});

relay.setTransport(buildNetronTransportToAggregator());
await relay.start();

const app = await Application.create({ modules: [AppModule] });
await app.start();

// Bridge: pipe LoggerService outputs into the relay
// Bridge: pipe MetricsService periodic snapshots into the relay
// (project-specific glue — implement once per app)

process.on('SIGTERM', async () => {
  await app.stop();
  await relay.stop();
});
```

## Aggregator pod

```typescript
const relay = new TelemetryRelayService({
  role: 'aggregator',
});

relay.setAggregator({
  // The aggregator interface is { ingest(nodeId, entries), query(filter) }.
  // Entry types are 'log' | 'metric' | 'event' | 'health' | 'alert'.
  async ingest(nodeId, entries) {
    // Ship to your backend(s): OTLP, Loki, Tempo, Prometheus remote-write, etc.
    await Promise.all([
      shipLogs(entries.filter(e => e.type === 'log')),
      shipMetrics(entries.filter(e => e.type === 'metric')),
      shipEvents(entries.filter(e => e.type === 'event')),
    ]);
    return entries.length;        // count acknowledged
  },
  async query(filter) { /* … */ return []; },
});

await relay.start();

// Expose a Netron RPC endpoint so producer pods can call relay.receive(nodeId, entries)
const app = await Application.create({ modules: [AggregatorModule] });
await app.start();
```

## Trace propagation in your code

```typescript
import { startSpan, withTrace } from '@omnitron-dev/titan';

@Public()
async findById(id: string) {
  // startSpan(parent?) returns a new child TraceContext; run work inside
  // withTrace so currentTrace() — and the log processor — see this span.
  const span = startSpan();
  return withTrace(span, () => this.repo.find(id));
}
```

Netron propagates the `traceparent` across RPC calls automatically;
log lines inside `withTrace` automatically carry `traceId` / `spanId`
through the `TraceContextProcessor` above.

## A typical instrumented service

```typescript
import { Service, startSpan, withTrace } from '@omnitron-dev/titan';
import { Public } from '@omnitron-dev/titan/netron';
import { Metrics } from '@omnitron-dev/titan-metrics';

@Service('users@1.0.0')
class UsersService {
  constructor(
    private readonly logger: ILogger,
    private readonly metrics: MetricsService,
  ) {}

  @Public()
  @Metrics('users.findById')                                    // name only; tracks total + duration + errors
  async findById(id: string) {
    this.logger.info('findById', { id });                       // gets traceId/spanId via processor
    return withTrace(startSpan(), () => this.repo.find(id));
  }
}
```

## Cross-module wiring notes

| Concern                      | Wiring detail                                                                                                             |
| ---------------------------- | ------------------------------------------------------------------------------------------------------------------------- |
| Trace → log correlation      | `TraceContextProcessor` reads `currentTrace()` and stamps `traceId` / `spanId` on every log record                        |
| Auto-RPC metrics             | `collection.rpc: true` auto-counts Netron calls — no per-method `@Metrics` needed                                         |
| Health → readiness           | `/readyz` returns `200` only when every indicator is `healthy` or `degraded` — not `unhealthy`                            |
| Relay producer vs aggregator | Producer pods buffer + ship via transport; aggregator receives + writes to sink. Either or both.                          |
| WAL crash safety             | Provide a `wal` config object (omit or `wal: false` to disable); buffer overflow spills to disk and replays on next start |
| Metrics retention            | `retention.maxAge` is the in-storage window; ship metrics to long-term backend before that elapses                        |

## Production checklist

- [ ] **Logs**: `RedactionProcessor` includes every sensitive header / field
- [ ] **Logs**: `TraceContextProcessor` (or equivalent) attached — without it, trace correlation breaks
- [ ] **Metrics**: `appName` set; otherwise cross-pod metrics pile together
- [ ] **Metrics**: storage backend chosen — `'memory'` is lost on restart
- [ ] **Metrics**: scraped by Prometheus (`/metrics`) OR shipped via relay (pick one)
- [ ] **Traces**: trace context propagated across worker thread boundaries via `withTrace`
- [ ] **Health**: `enableRpcService: true` so the orchestrator can poll via Netron
- [ ] **Relay**: a `wal` config provided (not `wal: false`) in production
- [ ] **Relay**: producer / aggregator topology mirrors your deployment shape
- [ ] **Relay**: aggregator's `write()` is idempotent (handles retries from buffer re-enqueue)
- [ ] **K8s**: `terminationGracePeriodSeconds` ≥ relay's flush interval + buffer drain time

## See also

- [Logging / Overview](/docs/titan/logging/overview) and per-page references
- [Tracing](/docs/titan/tracing) — full tracing primitive reference
- [`titan-metrics`](/docs/titan/modules/metrics) — full metrics reference
- [`titan-health`](/docs/titan/modules/health) — full health reference
- [`titan-telemetry-relay`](/docs/titan/modules/telemetry-relay) — relay reference
- [Best Practices / Observability](/docs/titan/best-practices/observability)

---

---
id: "titan/recipes/webhook-receiver"
title: "Webhook receiver"
description: "Validate signatures, enforce idempotency, route to handlers, isolate flaky downstream work."
---

# Webhook receiver

Receiving webhooks from a third party (Stripe, GitHub, payment
processor, etc.) seems trivial — but it has subtle requirements
that go wrong in production: signatures, replay protection,
idempotency, and isolating slow downstream work from the
acknowledgement window.

## Shape

- **Signature verification.** Reject anything not signed by the
  expected sender — usually HMAC with a shared secret.
- **Idempotency.** Webhook senders retry on timeout. Process each
  event ID at most once.
- **Fast 2xx response.** Senders treat slow ack as failure and
  retry. Acknowledge quickly; process asynchronously.
- **Dead-letter on failure.** Failures persist somewhere for
  inspection / manual retry — not lost.

## Architecture

```mermaid
flowchart LR
  Sender[Webhook sender]
  Sender -->|POST /webhooks/...| Edge[HTTP handler]
  Edge --> Verify{Signature valid?}
  Verify -- no --> R401[401]
  Verify -- yes --> Dedup{Event ID seen?}
  Dedup -- yes --> R200a[200 — already processed]
  Dedup -- no --> Mark[Mark seen<br/>Redis SETEX]
  Mark --> Enqueue[Emit event for worker]
  Enqueue --> R200b[200]
  Enqueue --> Q[(Queue / event bus)]
  Q --> Worker[Worker]
  Worker --> Handler[Domain handler]
  Worker -- failure --> DLQ[(DLQ)]
```

## Setup

```typescript
import { Module } from '@omnitron-dev/titan';
import { ConfigModule } from '@omnitron-dev/titan/module/config';
import { LoggerModule } from '@omnitron-dev/titan/module/logger';

import { TitanRedisModule } from '@omnitron-dev/titan-redis';
import { TitanRateLimitModule } from '@omnitron-dev/titan-ratelimit';
import { EventsModule } from '@omnitron-dev/titan-events';
import { TitanMetricsModule } from '@omnitron-dev/titan-metrics';

@Module({
  imports: [
    ConfigModule.forRoot({/* … */}),
    LoggerModule.forRoot({ level: 'info' }),
    TitanRedisModule.forRoot({ config: { url: env.REDIS_URL } }),
    TitanRateLimitModule.forRoot({
      enabled:         true,
      storageType:     'redis',
      strategy:        'sliding-window',
      defaultLimit:    500,
      defaultWindowMs: 60_000,
    }),
    EventsModule.forRoot({ wildcard: true, maxListeners: 50 }),
    TitanMetricsModule.forRoot({ appName: 'webhooks' }),
    WebhooksModule,                          // your handlers
  ],
})
export class AppModule {}
```

## Handler — verify, deduplicate, enqueue

```typescript
import { Service, Inject } from '@omnitron-dev/titan';
import { Public } from '@omnitron-dev/titan/netron';
import { Errors } from '@omnitron-dev/titan/errors';
import { InjectRedis, type IRedisClient } from '@omnitron-dev/titan-redis';
import { RateLimit, RATE_LIMIT_SERVICE_TOKEN, type IRateLimitService } from '@omnitron-dev/titan-ratelimit';
import { EVENTS_SERVICE_TOKEN, type EventsService }
  from '@omnitron-dev/titan-events';
import { LOGGER_SERVICE_TOKEN, type ILoggerModule } from '@omnitron-dev/titan/module/logger';
import { createHmac, timingSafeEqual } from 'node:crypto';

const SIGNING_SECRET = process.env.WEBHOOK_SIGNING_SECRET!;
const DEDUP_TTL_S    = 7 * 24 * 60 * 60;     // 7 days

@Service({ name: 'webhooks' })
class WebhooksService {
  // ILoggerModule exposes create(); the child ILogger is what has
  // .info/.warn/.error.
  //
  // A GETTER, not a field initializer. `private readonly logger =
  // this.loggerModule.create(…)` looks equivalent and crashes: field
  // initializers run BEFORE constructor parameter properties are assigned, so
  // `this.loggerModule` is still undefined and construction throws
  // "Cannot read properties of undefined". A getter defers the read to first
  // use, by which time the constructor has run.
  private get logger() {
    return this.loggerModule.create('webhooks');
  }

  constructor(
    @InjectRedis() private readonly redis: IRedisClient,
    @Inject(EVENTS_SERVICE_TOKEN) private readonly events: EventsService,
    @Inject(LOGGER_SERVICE_TOKEN) private readonly loggerModule: ILoggerModule,
    // @RateLimit reads the limiter off this exact property name.
    @Inject(RATE_LIMIT_SERVICE_TOKEN) private readonly __rateLimitService__: IRateLimitService,
  ) {}

  @Public()
  @RateLimit({ limit: 1_000, windowMs: 60_000 })   // options object only; no string key
  async receive(
    body:      string,                       // raw body for signature verification
    signature: string,
    eventId:   string,
    eventType: string,
  ) {
    // 1. Verify signature — constant-time comparison
    const expected = createHmac('sha256', SIGNING_SECRET).update(body).digest('hex');
    const ok = signature.length === expected.length &&
               timingSafeEqual(Buffer.from(signature), Buffer.from(expected));
    if (!ok) throw Errors.unauthorized('invalid webhook signature');

    // 2. Deduplicate — SETEX returns OK on first set, nil on existing key
    const key = `webhook:seen:${eventId}`;
    const wasSet = await this.redis.set(key, '1', 'EX', DEDUP_TTL_S, 'NX');
    if (wasSet === null) {
      this.logger.info('webhook duplicate', { eventId, eventType });
      return { status: 'duplicate' };          // 200 — already processed
    }

    // 3. Enqueue for async processing — acknowledge fast
    await this.events.emitAsync(`webhook.${eventType}`, { eventId, body: JSON.parse(body) });

    return { status: 'accepted', eventId };
  }
}
```

> **Always verify on the RAW body**, before any JSON parsing. JSON
> serialisation isn't byte-stable; even round-tripping
> `JSON.parse → JSON.stringify` can change a single byte and break
> the HMAC.

## Handler — async processing

```typescript
import { Injectable } from '@omnitron-dev/titan';
import { OnEvent } from '@omnitron-dev/titan-events';
import { Retry } from '@omnitron-dev/titan/decorators';

@Injectable()
class StripePaymentHandler {
  @OnEvent({ event: 'webhook.charge.succeeded', async: true })
  @Retry({ attempts: 3, delay: 500 })
  async onChargeSucceeded(payload: { eventId: string; body: any }) {
    try {
      await this.applyPayment(payload.body);
    } catch (e) {
      // Failure persisted to DLQ by the metrics + logger combo
      this.logger.error('payment processing failed', { eventId: payload.eventId, error: e });
      throw e;
    }
  }
}
```

## Cross-module wiring notes

| Concern               | Wiring detail                                                                                                                                    |
| --------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------ |
| Signature on raw body | Read the raw request body *before* JSON parsing — `@Validate(Schema)` runs after deserialise                                                     |
| Deduplication key TTL | Match or exceed the sender's max retry window (e.g. Stripe = 3 days, GitHub = 8 hours)                                                           |
| Rate limit per sender | Use a key that includes the sender ID — otherwise one sender saturates the bucket for all of them                                                |
| Idempotency on writes | Downstream handlers should also be idempotent — webhooks at-least-once even with dedup                                                           |
| Event bus vs queue    | In-process `EventsService` is sufficient for single-pod; for multi-pod use [`titan-notifications`](/docs/titan/modules/notifications) over Rotif |
| DLQ                   | If using `titan-notifications` worker mode, failed deliveries auto-route to the Rotif DLQ                                                        |
| Metric per event-type | `metrics.recordTyped('counter', 'webhook.received.total', { type: eventType, status }, 1)`                                                       |

## Production checklist

- [ ] **Raw body** captured before JSON parse — required for HMAC
- [ ] **Constant-time signature comparison** (`timingSafeEqual`) — never `===` on secrets
- [ ] **Dedup TTL** ≥ sender's max retry window
- [ ] **Dedup uses `SET NX EX`** — atomic check-and-set
- [ ] **Fast ack** — return 2xx within sender's timeout (typically 5-10 s)
- [ ] **Async processing** via event bus or queue — never block the ack
- [ ] **Handler `@Retry`** for transient failures
- [ ] **DLQ for non-retryable failures** — inspectable, replayable
- [ ] **Rate limit per-sender** if you receive from many sources
- [ ] **Metrics on every step**: received, deduplicated, accepted, processed, failed
- [ ] **Replay test in staging** — re-send the same event, assert it's deduplicated

## See also

- [`titan-events`](/docs/titan/modules/events) — async dispatch
- [`titan-notifications`](/docs/titan/modules/notifications) — for cross-pod queueing + DLQ
- [`titan-ratelimit`](/docs/titan/modules/ratelimit) — per-sender throttling
- [Resilience / Retry](/docs/titan/resilience/retry) — transient failure handling

---

---
id: "titan/recipes/worker-fleet"
title: "Worker fleet"
description: "Background-job processing across pods — process manager, distributed lock, scheduler, metrics."
---

# Worker fleet

The canonical shape for **background work** at scale: scheduled
jobs that should run exactly-once across a fleet, CPU-bound work
isolated from the main event loop, and per-job retry / circuit-
breaking / metrics — all coordinated through Redis.

## Shape

- **Scheduled jobs.** Cron and intervals via `titan-scheduler`,
  auto-discovered by decorator.
- **Exactly-once across pods.** Distributed locks via
  `titan-lock`, so the same scheduled job doesn't run on every
  pod.
- **CPU isolation.** Heavy work in worker processes via
  `titan-pm`, with pools / restart policies.
- **Persistence.** Job state survives restarts (Redis-backed
  scheduler).
- **Metrics + health.** Per-job execution counts, durations,
  failures.

## Architecture

```mermaid
flowchart TB
  subgraph "Pod A"
    SchedA[titan-scheduler<br/>cron tick]
    SchedA --> WL_A{distributed lock?}
    WL_A -- acquired --> RunA[run job]
    WL_A -- held --> SkipA[skip]
    RunA --> PoolA[titan-pm worker pool]
    PoolA --> WorkerA1[Worker]
    PoolA --> WorkerA2[Worker]
  end

  subgraph "Pod B"
    SchedB[titan-scheduler<br/>cron tick]
    SchedB --> WL_B{distributed lock?}
    WL_B -- held --> SkipB[skip]
  end

  WL_A -.atomic.-> Redis[(Redis)]
  WL_B -.atomic.-> Redis
  RunA --> Metrics[titan-metrics]
  Health[titan-health] -.indicators.-> Redis
```

## `AppModule`

```typescript
import { Module } from '@omnitron-dev/titan';
import { z } from '@omnitron-dev/titan/validation';
import { ConfigModule } from '@omnitron-dev/titan/module/config';
import { LoggerModule, ConsoleTransport } from '@omnitron-dev/titan/module/logger';

import { TitanRedisModule } from '@omnitron-dev/titan-redis';
import { TitanLockModule } from '@omnitron-dev/titan-lock';
import { SchedulerModule, RedisPersistenceProvider } from '@omnitron-dev/titan-scheduler';
import { ProcessManagerModule } from '@omnitron-dev/titan-pm';
import { TitanMetricsModule } from '@omnitron-dev/titan-metrics';
import { TitanHealthModule } from '@omnitron-dev/titan-health';

const WorkerConfigSchema = z.object({
  env:   z.enum(['development', 'staging', 'production']),
  redis: z.object({ url: z.string().url() }),
  pm:    z.object({
    maxRestarts:    z.number().int(),
    maxMemory:      z.string(),
    poolMinWorkers: z.number().int(),
    poolMaxWorkers: z.number().int(),
  }),
});

@Module({
  imports: [
    ConfigModule.forRoot({
      schema:  WorkerConfigSchema,
      sources: [
        { type: 'file', path: 'config/default.yaml' },
        { type: 'env',  prefix: 'WORKER_' },
      ],
      validateOnStartup: true,
    }),

    LoggerModule.forRoot({
      level:      'info',
      transports: [new ConsoleTransport()],
    }),

    // ── Foundation ─────────────────────────────────────────────────────
    TitanRedisModule.forRootAsync({
      useFactory: (config: ConfigService) => ({
        clients: [
          { namespace: 'default', url: config.get('redis.url'), db: 0 },
          { namespace: 'lock',    url: config.get('redis.url'), db: 5 },
          { namespace: 'sched',   url: config.get('redis.url'), db: 6 },
        ],
      }),
      inject: [ConfigService],
    }),

    // ── Distributed locks (used by scheduler + decorators) ─────────────
    TitanLockModule.forRoot({
      defaultTtl:        60_000,                     // 1 min default
      keyPrefix:         'lock',
      defaultRetries:    3,
      defaultRetryDelay: 200,
      redisClientName:   'lock',
    }),

    // ── Scheduler with Redis persistence ───────────────────────────────
    SchedulerModule.forRootAsync({
      useFactory: (redis: RedisService) => ({
        enabled:         true,
        timezone:        'UTC',
        persistence:     {
          enabled:  true,
          provider: new RedisPersistenceProvider(redis.getClient('sched')),
        },
        metrics:         { enabled: true, interval: 5_000 },
        maxConcurrent:   10,
        shutdownTimeout: 30_000,
      }),
      inject: [RedisService],
    }),

    // ── Process manager for CPU work ───────────────────────────────────
    ProcessManagerModule.forRootAsync({
      useFactory: (config: ConfigService) => ({
        isolation: 'worker',
        transport: 'unix',
        restartPolicy: {
          enabled:     true,
          maxRestarts: config.get('pm.maxRestarts'),
          window:      60_000,
          delay:       1_000,
          backoff: { type: 'exponential', initial: 1_000, max: 30_000, factor: 2 },
        },
        resources: {
          maxMemory: config.get('pm.maxMemory'),
          maxCpu:    1.0,
          timeout:   60_000,
        },
        monitoring: {
          healthCheck: { interval: 5_000, timeout: 2_000 },
          metrics:     true,
          tracing:     false,
        },
        advanced: { gracefulShutdownTimeout: 30_000 },
      }),
      inject: [ConfigService],
    }),

    // ── Metrics ────────────────────────────────────────────────────────
    TitanMetricsModule.forRoot({
      appName:    'worker-fleet',
      collection: { enabled: true, interval: 10_000, process: true, system: true, custom: true },
      storage:    { type: 'sqlite', batchSize: 200, flushInterval: 10_000 },
      retention:  { maxAge: '14d', cleanupInterval: 3_600_000 },
    }),

    // ── Health ─────────────────────────────────────────────────────────
    TitanHealthModule.forRootAsync({
      useFactory: (redis: RedisService) => ({
        enableMemoryIndicator:    true,
        enableEventLoopIndicator: true,
        enableRedisIndicator:     true,
        redisClient:              redis.getClient('default'),
        timeout:                  2_000,
        enableCaching:            true,
        version:                  process.env.APP_VERSION,
      }),
      inject: [RedisService],
    }),

    // ── Your task / worker modules ─────────────────────────────────────
    TasksModule,
    ReportsModule,
  ],
})
export class AppModule {}
```

## A scheduled task (exactly-once)

```typescript
import { Injectable, Inject } from '@omnitron-dev/titan';
import { Schedulable, Cron, CronExpression } from '@omnitron-dev/titan-scheduler';
import {
  WithDistributedLock,
  LOCK_SERVICE_TOKEN,
  type IDistributedLockService,
} from '@omnitron-dev/titan-lock';
import { LOGGER_SERVICE_TOKEN, type ILoggerModule } from '@omnitron-dev/titan/module/logger';

@Injectable()
@Schedulable()
class CleanupTasks {
  constructor(
    @Inject(LOCK_SERVICE_TOKEN)   private readonly __lockService__: IDistributedLockService,
    @Inject(LOGGER_SERVICE_TOKEN) private readonly loggerModule:    ILoggerModule,
  ) {}

  @Cron(CronExpression.EVERY_HOUR, { timezone: 'UTC' })
  @WithDistributedLock('cleanup:hourly', 30 * 60_000)     // up to 30 min
  async hourlyCleanup() {
    // Runs on exactly one pod per hour.
    await this.deleteExpired();
  }

  @Cron(CronExpression.EVERY_DAY_AT_3AM)                 // typed; raw strings must match a CronExpression value
  @WithDistributedLock('cleanup:nightly', 60 * 60_000)   // up to 1 hour
  async nightlyCleanup() {
    await this.archiveOldRecords();
  }
}
```

## A CPU-bound worker pool

```typescript
import {
  Process, Public, OnShutdown, Trace, Metric, CircuitBreaker, Idempotent,
  PM_MANAGER_TOKEN, ProcessManager,
} from '@omnitron-dev/titan-pm';
import { Service, Inject } from '@omnitron-dev/titan';

@Process({ name: 'image-worker' })
class ImageWorker {
  @Public()
  @Trace()
  @Metric('images.resize')                            // metric name (or omit for default)
  @CircuitBreaker({ threshold: 5, timeout: 30_000 })  // `threshold`, not `failureThreshold`
  @Idempotent({ key: 'jobId', ttl: '1h' })            // dedup on input.jobId for 1h
  async resize(input: { jobId: string; data: Buffer; width: number }): Promise<Buffer> {
    return sharp(input.data).resize(input.width).toBuffer();
  }

  @OnShutdown()
  async cleanup() {
    // Drain pending tasks before exit
  }
}

@Service({ name: 'media' })
class MediaService {
  constructor(@Inject(PM_MANAGER_TOKEN) private readonly pm: ProcessManager) {}

  @Public()
  async resize(input: { jobId: string; data: Buffer; width: number }) {
    // `size`, not `min`/`max` — the pool is a fixed size (or `'auto'`,
    // which sizes it to the CPU count). Use `pool.scale(n)` to change it.
    const pool = await this.pm.pool(ImageWorker, { size: 8 });
    // The pool IS a service proxy: call the worker's own methods on it.
    // There is no `.execute(method, …args)`.
    return pool.resize(input);
  }
}
```

## Cross-module wiring notes

| Concern                       | Wiring detail                                                                                                         |
| ----------------------------- | --------------------------------------------------------------------------------------------------------------------- |
| Redis namespaces              | `default`, `lock` (db 5), `sched` (db 6) — separate DBs prevent key collisions and ease debugging                     |
| `@WithDistributedLock` wiring | Decorator looks up the lock service on the instance under `__lockService__`; inject it there explicitly               |
| Lock TTL > job duration       | `WithDistributedLock('key', ttl)` — `ttl` must exceed the worst-case duration or another pod takes over mid-execution |
| Scheduler persistence         | `RedisPersistenceProvider` lets jobs resume after pod restart; without it, scheduled state is lost                    |
| PM restart backoff            | `restartPolicy.backoff` with `type: 'exponential'` prevents crash-loop flooding; cap with `max`                       |
| Worker resource limits        | `resources.maxMemory` + `resources.maxCpu` enforce limits; without them, runaway workers can starve peers             |
| Liveness sweep                | PM's defense-in-depth — even if a worker dies without exit signal, the sweep notices and restarts                     |

## Production checklist

- [ ] `SchedulerModule.persistence.enabled: true` — without it, jobs lost on restart
- [ ] Every scheduled job that mutates shared state uses `@WithDistributedLock`
- [ ] Lock `ttl` ≥ worst-case job duration × 2
- [ ] `PM.restartPolicy.backoff` configured to prevent crash-loops
- [ ] Worker `resources.maxMemory` set — otherwise leaks bring down the pod
- [ ] `monitoring.metrics: true` on PM — without metrics you can't see worker health
- [ ] `MetricsModule.storage: 'sqlite'` or `'postgres'` — `'memory'` is lost on restart
- [ ] Logs shipped off-host — workers crash, logs need to survive
- [ ] Kubernetes worker pods use `terminationGracePeriodSeconds` ≥ `shutdownTimeout`
- [ ] Scheduler `shutdownTimeout` ≥ longest job's expected duration

## See also

- [`titan-scheduler`](/docs/titan/modules/scheduler) — full scheduler reference
- [`titan-lock`](/docs/titan/modules/lock) — distributed lock semantics
- [`titan-pm`](/docs/titan/modules/pm) — process manager full reference
- [Resilience / Retry](/docs/titan/resilience/retry) — pair with retry for transient failures

---

---
id: "titan/resilience/circuit-breaker"
title: "Circuit Breaker"
description: "Stop calling a failing dependency. Let it cool down. Resume gradually."
---

# Circuit Breaker

`CircuitBreaker` is a class from `@omnitron-dev/titan/utils`. There is
no `@CircuitBreaker` decorator — instantiate the class and use
`.execute()` around the work you want to protect.

## States

```typescript
enum CircuitState {
  Closed   = 'closed',
  Open     = 'open',
  HalfOpen = 'half-open',
}
```

| State        | Behaviour                                                      |
| ------------ | -------------------------------------------------------------- |
| **Closed**   | Calls pass through. Failures count toward the threshold.       |
| **Open**     | Calls fail immediately. No backend hit.                        |
| **HalfOpen** | One probe call goes through. Success → Closed. Failure → Open. |

Transitions:

```mermaid
stateDiagram-v2
  [*] --> Closed
  Closed --> Open: N consecutive failures
  Open --> HalfOpen: timeout elapsed
  HalfOpen --> Closed: probe success
  HalfOpen --> Open: probe failure
```

## Basic usage

```typescript
import { CircuitBreaker, CircuitState } from '@omnitron-dev/titan/utils';

const breaker = new CircuitBreaker({
  failureThreshold: 5,             // 5 failures opens the circuit
  resetTimeout:     60_000,        // ms in Open before allowing a probe
  // …additional knobs in CircuitBreakerConfig
});

// Use it to wrap calls.
try {
  const result = await breaker.execute(() => callStripe(req));
} catch (e) {
  // If the circuit is open, .execute throws immediately.
}
```

The full `CircuitBreakerConfig` shape lives in
`utils/resilience.ts`. Common knobs (consult the source for the
canonical list):

- `failureThreshold` — failures before opening (default `5`).
- `resetTimeout` — ms in `Open` before transitioning to `HalfOpen`
  (default `30000`).
- `successThreshold` — successful probes in `HalfOpen` before
  returning to `Closed` (default `1`).
- `volumeThreshold` — minimum number of calls before the failure
  threshold takes effect (default `10`; avoids tripping on a single
  bad sample).
- `failureRateThreshold` — open when the failure *rate* (0–100)
  over the sliding window exceeds this, instead of an absolute count.
- `requestTimeout` — optional per-call timeout (ms) applied inside
  `execute()`.

## Per-instance state

Each `CircuitBreaker` instance has its own state. Use one instance
per dependency you want to protect:

```typescript
@Service({ name: 'Payments' })
class PaymentsService {
  private readonly stripeBreaker  = new CircuitBreaker({ failureThreshold: 5, resetTimeout: 60_000 });
  private readonly paypalBreaker  = new CircuitBreaker({ failureThreshold: 3, resetTimeout: 30_000 });

  @Public()
  async chargeStripe(req: ChargeRequest) {
    return this.stripeBreaker.execute(() => this.stripe.charge(req));
  }

  @Public()
  async chargePaypal(req: ChargeRequest) {
    return this.paypalBreaker.execute(() => this.paypal.charge(req));
  }
}
```

A failure in `chargeStripe` opens *only* the Stripe circuit;
PayPal stays available.

## Observability

`CircuitBreaker` extends `EventEmitter`. The emitted events are
`stateChange`, `success`, `failure`, `rejected`, and `reset` — there
are no separate `open` / `close` / `halfOpen` events. Watch
`stateChange` and branch on the new state:

```typescript
breaker.on('stateChange', ({ previousState, newState }) => {
  if (newState === CircuitState.Open)     metrics.counter('breaker.open').inc();
  if (newState === CircuitState.Closed)   metrics.counter('breaker.close').inc();
  if (newState === CircuitState.HalfOpen) log.info('probing recovery');
});

breaker.on('rejected', () => metrics.counter('breaker.rejected').inc());
```

## Reading state

```typescript
breaker.getState()             // CircuitState (state is private — use the getter)
breaker.getMetrics()           // CircuitBreakerMetrics — total calls, failures, etc.
```

For imperative gating without `execute()`, the breaker also exposes
`isOpen()`, `recordSuccess()`, and `recordFailure()`.

## Composing with retry

```typescript
import { CircuitBreaker, retry, isOperationalError } from '@omnitron-dev/titan/utils';

const breaker = new CircuitBreaker({ failureThreshold: 5, resetTimeout: 60_000 });

await breaker.execute(() =>
  retry(() => callBackend(), { maxRetries: 3, shouldRetry: isOperationalError }),
);
```

Breaker outside, retry inside. When the circuit is open, retries
don't run — the breaker rejects before `retry()` is called.

## Anti-patterns

- **Threshold too low.** A breaker that opens on the first failure
  flaps constantly. Pair `failureThreshold` with `volumeThreshold`
  (or per-window counts).
- **`timeout` too short.** The dependency hasn't recovered; the
  probe fails; the circuit re-opens. Real recovery takes seconds
  to minutes.
- **Same breaker for unrelated dependencies.** A failure in one
  service opens the circuit for another. Use one breaker per
  dependency.
- **No fallback.** Every breaker should have a "what do we do when
  the circuit is open" plan. Cached value? Default? Error to the
  user? Decide explicitly.

---

---
id: "titan/resilience/overview"
title: "Resilience"
description: "Defending against partial failure — retry, circuit breaker, timeout, backoff."
---

# Resilience

Titan ships a small set of resilience primitives in
`@omnitron-dev/titan/utils`. They are **functions and classes**, not
decorators — designed to be composed inside your service methods at
the call boundary.

| Primitive              | What it does                                                                           | Page                                                      |
| ---------------------- | -------------------------------------------------------------------------------------- | --------------------------------------------------------- |
| `computeBackoff`       | Compute delay between attempts                                                         | shared in [Retry](/docs/titan/resilience/retry)           |
| `retry()`              | Re-attempt a failed operation                                                          | [Retry](/docs/titan/resilience/retry)                     |
| `CircuitBreaker` class | Stop calling a backend after failures                                                  | [Circuit Breaker](/docs/titan/resilience/circuit-breaker) |
| `withTimeout()`        | Abandon a call that takes too long                                                     | [Timeout](/docs/titan/resilience/timeout)                 |
| `FailureTracker`       | Sliding-window error counting                                                          | source: `utils/failure-tracker.ts`                        |
| `ResilientHandle`      | Self-healing cached handle to a flaky external resource (auto-recreate on fatal error) | source: `utils/resilience.ts`                             |

There is also a `@Retry` decorator in
`@omnitron-dev/titan/decorators` for simple fixed-delay retry on a
method — but it is intentionally minimal (no exponential backoff, no
classifier). For sophisticated resilience, use the primitives from
`utils/`.

## When you need each

- **Retry** — for transient failures: network errors, 503s,
  timeouts. Use `isOperationalError(e)` to classify.
- **Circuit breaker** — when retrying a failing dependency makes
  the problem worse. Open the circuit; let it cool down.
- **Timeout** — for any external call. A call without a timeout can
  hang forever, taking a request slot with it.
- **FailureTracker** — when you need a sliding-window error count
  for custom logic (e.g. dynamic throttling based on recent error
  rate).

These compose. A typical "outbound call" pattern:

```mermaid
flowchart LR
  Call[Outbound call]
  Call --> T{withTimeout}
  T --> CB{CircuitBreaker}
  CB -- closed --> R{retry}
  R --> Op[the actual operation]
  CB -- open --> Fail1[fail fast]
  T -. deadline elapsed .-> Fail2[TimeoutError]
  Op -- success --> Result[result]
  Op -- transient err --> R
  R -- exhausted --> Fail3[last error]
```

- Timeout outermost — even retries shouldn't extend the deadline.
- Circuit breaker next — if open, fail fast (no retries).
- Retry — re-attempt on transient failures.

Compose the primitives directly (as in the example below). Note
that `ResilientHandle` is **not** a retry+breaker+timeout bundle —
it is a self-healing cache for a single external resource (a Redis
client, the esbuild sidecar, a gRPC channel) that recreates the
instance via a `factory` when a call throws a fatal error. See
`utils/resilience.ts` for its API.

## The minimal example

```typescript
import { retry, withTimeout, CircuitBreaker } from '@omnitron-dev/titan/utils';

@Service({ name: 'Payments' })
class PaymentsService {
  private readonly stripeBreaker = new CircuitBreaker({
    failureThreshold: 5,
    resetTimeout:     60_000,   // ms in Open before a probe
  });

  @Public()
  async charge(req: ChargeRequest) {
    // withTimeout takes a Promise (not a thunk) and a number of ms
    // (or { timeout, errorMessage }).
    return withTimeout(
      this.stripeBreaker.execute(() =>
        retry(() => this.stripe.charge(req), { maxRetries: 3 })
      ),
      5_000,
    );
  }
}
```

## Read on

- [Retry](/docs/titan/resilience/retry) — backoff strategies, the `retry()` function,
  the `@Retry` decorator.
- [Circuit Breaker](/docs/titan/resilience/circuit-breaker) — `CircuitBreaker` class
  semantics.
- [Timeout](/docs/titan/resilience/timeout) — `withTimeout()`, abort signals.

---

---
id: "titan/resilience/retry"
title: "Retry"
description: "The retry() function, the @Retry decorator, and the backoff helpers."
---

# Retry

Three retry surfaces, from simplest to most flexible.

## The `@Retry` decorator (simplest)

Fixed-delay retry on a method. Three attempts by default, 1000 ms
between them.

```typescript
import { Retry } from '@omnitron-dev/titan/decorators';

@Retry({ attempts: 3, delay: 200 })
async fetchUpstream() {
  return await fetch(this.url);
}
```

| Option     | Default | Effect                                   |
| ---------- | ------- | ---------------------------------------- |
| `attempts` | `3`     | Total attempts (including the first try) |
| `delay`    | `1000`  | ms between attempts                      |

`@Retry` retries on **any** thrown error — there is no classifier
in the decorator. If you need to retry only on specific errors
(transient), wrap the work yourself with the `retry()` function
described below.

## The `retry()` function (flexible)

From `@omnitron-dev/titan/utils`:

```typescript
import { retry, BackoffStrategy } from '@omnitron-dev/titan/utils';
import { isOperationalError } from '@omnitron-dev/titan/utils';

await retry(
  () => fetch(url),
  {
    maxRetries:       3,
    initialDelay:     100,
    maxDelay:         5_000,
    backoffMultiplier: 2,
    jitterFactor:     0.1,
    backoffStrategy:  BackoffStrategy.Exponential,
    shouldRetry:      (error) => isOperationalError(error),
    onRetry:          (error, attempt, delay) => log.warn('retry', { attempt, delay }),
  },
);
```

Real shape of `RetryOptions` lives in `utils/resilience.ts`. Common
knobs (note the exact field names):

- `maxRetries` — maximum retry attempts (in addition to the first try). Default `3`.
- `initialDelay` / `maxDelay` — bounds (ms). Defaults `100` / `30_000`.
- `backoffMultiplier` — multiplier for exponential backoff. Default `2`.
- `jitterFactor` — `0..1` random variance applied to the delay. Default `0.1`.
- `backoffStrategy` — `BackoffStrategy.Fixed | .Linear | .Exponential |
  .ExponentialWithJitter`. Default `.Exponential`.
- `shouldRetry(error, attempt)` — classifier predicate; default retries everything.
- `onRetry(error, attempt, delay)` — callback per attempt.

## The `computeBackoff` helper

For custom retry loops where `retry()` doesn't fit:

```typescript
import { computeBackoff } from '@omnitron-dev/titan/utils';

let attempt = 0;
while (true) {
  try {
    return await fetch(url);
  } catch (e) {
    if (++attempt >= 3) throw e;
    const delayMs = computeBackoff({
      attempt,            // 0-indexed
      baseMs: 100,
      maxMs:  5_000,
      factor: 2,          // exponential growth factor (not `multiplier`)
      jitter: 0.25,       // 0..1 fraction, additive by default
    });
    await sleep(delayMs);
  }
}
```

`IBackoffOptions` covers exponential backoff with jitter — the same
math `retry()` uses internally.

## Retry strategies registry

`utils/retry.ts` exposes a richer family of named strategies:

```typescript
import {
  RetryStrategies,
  calculateStrategyDelay,
  createRetryDelayFn,
  addJitter,
  getFibonacciDelay,
} from '@omnitron-dev/titan/utils';
```

`RetryStrategies` is a registry of named delay functions
(exponential, linear, fixed, fibonacci, custom). Use when you want to
plug a strategy by name from configuration.

## When NOT to retry

- **Validation failures.** They will fail again on retry.
- **Auth failures (401, 403).** They won't change with retry.
- **Idempotency-unsafe operations.** A POST that creates a resource
  should not be retried unless you have an idempotency key.

The default `shouldRetry: (e) => isOperationalError(e)` handles the
first two. The third is your responsibility — wrap idempotency-
sensitive calls with an idempotency key your backend honours.

## Compose with a circuit breaker

A retry without a circuit breaker can hammer a failing backend:

```typescript
import { CircuitBreaker, retry, isOperationalError } from '@omnitron-dev/titan/utils';

const breaker = new CircuitBreaker({ failureThreshold: 5, resetTimeout: 60_000 });

await breaker.execute(() =>
  retry(() => callBackend(), { maxRetries: 3, shouldRetry: isOperationalError }),
);
```

When the circuit is open, retries fail fast — they never call the
backend, so no exponential pile-up.

## Anti-patterns

- **Unlimited retries.** Always bound with `maxRetries`.
- **Retry on every error.** Use a `shouldRetry` predicate
  (`isOperationalError` is the safe default).
- **No backoff.** Immediate retry is a thundering herd. Use one of
  the named strategies.
- **No jitter.** Synchronised retries from many clients create
  bursts. Set a non-zero `jitterFactor` (e.g. `0.3`), or use
  `BackoffStrategy.ExponentialWithJitter`.

---

---
id: "titan/resilience/timeout"
title: "Timeout"
description: "Bounded calls — every external dependency needs a deadline."
---

# Timeout

Every call to an external dependency should have a timeout. A call
without one can hang forever, holding a request slot, a connection,
a worker thread.

## `withTimeout()` — the function

```typescript
import { withTimeout } from '@omnitron-dev/titan/utils';

// withTimeout takes a Promise (already-invoked), and either a number
// of milliseconds or a { timeout, errorMessage } object.
const result = await withTimeout(fetch(url), 5_000);

// With a custom message:
const result2 = await withTimeout(
  fetch(url),
  { timeout: 5_000, errorMessage: 'upstream fetch timed out' },
);
```

`TimeoutOptions` is `{ timeout: number; errorMessage?: string }`
(see `utils/resilience.ts`). On timeout, `withTimeout` rejects with
an `Error` whose `name` is `'TimeoutError'` (use `isTimeoutError(e)`
to check).

The underlying operation is **not** automatically cancelled — see
"Cancellation" below.

## `@Timeout` — the decorator

`@omnitron-dev/titan/decorators` exports a `Timeout` method
interceptor:

```typescript
import { Timeout } from '@omnitron-dev/titan/decorators';

@Timeout({ ms: 5_000 })
async fetchUpstream() {
  return await fetch(this.url);
}
```

Wraps the method body in a timeout. Like `withTimeout`, does not
cancel the underlying operation.

## Cancellation

A timeout that doesn't cancel the underlying operation merely
*ignores* the result — the operation keeps running, holding
resources. For cooperative cancellation, pass an `AbortSignal`:

```typescript
async function fetchWithDeadline(url: string, signal: AbortSignal) {
  return fetch(url, { signal });
}

const controller = new AbortController();
setTimeout(() => controller.abort(), 5_000);

try {
  await fetchWithDeadline(url, controller.signal);
} catch (e) {
  if (e.name === 'AbortError') {
    // cleanly aborted
  }
}
```

For libraries without abort support, the operation runs to
completion in the background. Your method returns the timeout error
anyway.

## Choosing a timeout

A timeout should be longer than the worst legitimate latency you
expect, plus margin:

| Operation             | Reasonable range |
| --------------------- | ---------------- |
| Cache lookup          | 100 ms           |
| In-cluster RPC        | 1–5 s            |
| External HTTP API     | 5–30 s           |
| Long-running upstream | 60 s+            |

Too short → spurious timeouts on legitimate slow paths.
Too long → resource exhaustion when the dependency hangs.

Measure first; pick a timeout at p99.9 × 2.

## Cascading timeouts

A timeout on a chain (your service → A → B → C) is the *minimum*
along the chain. If the inner C has a 30 s timeout and your service
has a 10 s timeout, the user sees 10 s and you see C still running
in the background.

Pattern: pass deadlines down. Make the inner timeouts shorter than
the outer, by some margin.

## Detecting timeouts

```typescript
import { isTimeoutError } from '@omnitron-dev/titan/utils';

catch (e) {
  if (isTimeoutError(e)) {
    // handle timeout specifically
  }
}
```

## Anti-patterns

- **No timeout on outbound calls.** The most common production bug
  in distributed systems.
- **Same timeout everywhere.** A 30 s timeout on a cache lookup is
  pointless; a 5 s timeout on a long upload is broken. Match to
  the operation.
- **Timeout but no cancellation.** The framework returns; the
  operation keeps running. Resources accumulate. Use abort
  signals.
- **Catching `TimeoutError` and continuing.** A timed-out operation
  may still complete in the background. Catching can produce
  duplicate side effects when both paths run.

→ Back to [Resilience Overview](/docs/titan/resilience/overview).

---

---
id: "titan/testing/di-overrides"
title: "DI Overrides"
description: "Replacing providers for tests — fakes, stubs, in-memory implementations."
---

# DI Overrides

The container's central role makes testing easy: replace any
provider with a test double, and every consumer resolves the double
without code changes. This page is the reference for *how* to swap a
provider in a Titan test — at the container level, at the
`Application.create` level, and with the `@omnitron-dev/testing/titan`
helpers.

## How overriding works

A Nexus container keys registrations by token, and **the last
registration for a token wins**. Overriding is therefore just
re-registering a token with a different provider definition after the
original is in place.

A provider definition is the standard Nexus shape (one of `useClass`,
`useValue`, `useFactory`, `useToken`, or a bare constructor — see
`Provider` in `Provider` in `packages/titan/src/nexus/types.ts`):

```typescript
import { Container } from '@omnitron-dev/titan/nexus';

const container = new Container();

container.register(DATABASE, { useClass: FakeDatabase });
container.register(USERS_SERVICE, { useClass: UsersService });
// UsersService resolves with FakeDatabase as its `database` dep.
```

To force a re-registration over an existing one, pass
`{ override: true }`:

```typescript
container.register(DATABASE, { useClass: RealDatabase });
container.register(DATABASE, { useClass: FakeDatabase }, { override: true });
// resolve(DATABASE) → FakeDatabase
```

`Application` exposes the same surface as `app.register(token,
provider, { override: true })`.

## Overriding through `Application.create`

When you boot a full app for an integration test, pass the
`providers` option — an array of `[token, providerDefinition]`
**tuples**:

```typescript
import { Application } from '@omnitron-dev/titan';

const app = await Application.create({
  modules:   [AppModule],
  providers: [
    [Database, { useClass: FakeDatabase }],
    [REDIS,    { useValue: fakeRedis     }],
  ],
  disableGracefulShutdown: true,
});
```

The shape of `providers` is defined by `CreateOptions` in
`IModuleOptions.imports` in `packages/titan/src/application/application.ts`:

```typescript
providers?: Array<[InjectionToken<unknown>, Provider<unknown>]>;
```

`create()` registers these into the container **after** the modules
are wired (`Application.create`–288 — modules at lines 269–274,
`providers` immediately after), so each tuple's definition wins over
whatever the module declared for the same token. There is no
`overrides` option — `providers` is the override mechanism.

Each tuple's provider definition uses the standard Nexus shape:

```typescript
providers: [
  [Database, { useClass: FakeDatabase }],          // construct a fake
  [Clock,    { useValue: { now: () => 0 } }],       // pin a value
  [Cache,    { useFactory: () => new MapCache() }], // build per resolution
]
```

After all modules and providers are registered, `create()` calls
`container.eagerlyInitialize()` (`Application.create`), so singleton
providers — including your fakes — are constructed before `create()`
resolves.

## Fake vs mock vs stub

Three styles, all valid:

- **Fake** — a working implementation backed by a simpler store
  (in-memory database, in-memory cache). Best for integration tests
  of wiring.
- **Mock** — a programmable double that records calls and returns
  configured responses (`vi.fn()`). Best for unit tests of business
  logic.
- **Stub** — a hard-coded response, no programming.

## A canonical fake

```typescript
import type { Database, User, NewUser } from './types.js';

export class FakeDatabase implements Database {
  private readonly users = new Map<string, User>();

  async findUser(id: string): Promise<User | null> {
    return this.users.get(id) ?? null;
  }

  async createUser(input: NewUser): Promise<User> {
    const user = { id: crypto.randomUUID(), ...input };
    this.users.set(user.id, user);
    return user;
  }

  async deleteUser(id: string): Promise<void> {
    this.users.delete(id);
  }

  /** Test affordance: wipe between tests. */
  reset(): void {
    this.users.clear();
  }
}
```

Wire it through `providers`, then reset it in `beforeEach`:

```typescript
const app = await Application.create({
  modules:   [AppModule],
  providers: [[Database, { useClass: FakeDatabase }]],
  disableGracefulShutdown: true,
});
await app.start();

beforeEach(() => {
  (app.resolve(Database) as FakeDatabase).reset();
});
```

The same fake works at every test level — unit, integration, e2e.

## The `@omnitron-dev/testing/titan` helpers

The testing package ships container-level test helpers under the
**`@omnitron-dev/testing/titan`** subpath (the `"./titan"` entry in the
package's `exports` map). They are **not** exported from
the package root — always import from the subpath:

```typescript
import {
  createTestModule,
  MockProvider,
  createMockProvider,
} from '@omnitron-dev/testing/titan';
```

These wrap a `TestContainer` (a `Container` subclass), not a booted
`Application`. Use them when you want fine-grained mocking, spying,
and interaction recording without standing up the full lifecycle.

### `createTestModule` / `TestModule`

`createTestModule(options)` returns a `TestModule`
(`createTestModule` in `packages/testing/src/titan/test-module.ts`). Its options
(`TestModuleOptions`, same file lines 11–21):

```typescript
interface TestModuleOptions {
  modules?:   IModule[];
  providers?: Array<[InjectionToken<any>, ProviderDefinition<any>]>;
  mocks?:     Array<{ token: InjectionToken<any>; mock: any; spy?: boolean }>;
  config?:    Partial<IApplicationConfig>;
  autoMock?:  boolean;
}
```

`mocks` is the override channel here — each entry registers
`{ useValue: mock }` for its token (with optional spy wrapping).
`TestModule` also exposes a fluent API:

```typescript
import { createTestModule } from '@omnitron-dev/testing/titan';

const harness = createTestModule({
  providers: [[USERS_SERVICE, { useClass: UsersService }]],
  mocks:     [{ token: DATABASE, mock: new FakeDatabase() }],
});

// Fluent overrides (each returns `this`):
harness
  .override(CLOCK).useValue({ now: () => 0 })
  .mock(MAILER, { send: vi.fn() }, /* spy */ true)
  .stub(FEATURE_FLAGS, { isEnabled: true });

const users = harness.get(USERS_SERVICE);  // resolves from the test container
```

Method reference (`TestModuleBuilder.mock`–159):

| Method                                        | Effect                                                                        |
| --------------------------------------------- | ----------------------------------------------------------------------------- |
| `mock(token, mock, spy?)`                     | Register `mock` as the token's value; optionally wrap its methods in spies.   |
| `stub(token, partial)`                        | Register a partial implementation.                                            |
| `override(token)`                             | Returns `{ useValue, useClass, useFactory }` — pick one.                      |
| `spy(token, method)`                          | Spy on a method of the already-registered instance.                           |
| `get(token)`                                  | Resolve from the test container.                                              |
| `resetMocks()` / `clearMocks()` / `restore()` | Mock lifecycle control.                                                       |
| `cleanup()`                                   | Stop the app (if `createApplication()` was called) and restore the container. |

> `TestModule.createApplication()` constructs an `Application` and
> attaches the test container, but **does not call `start()`** — it
> is for container-resolution tests, not lifecycle tests. To exercise
> the real lifecycle (`onStart`, module ordering, hooks), boot with
> `Application.create({ ..., providers: [...] })` and call
> `app.start()` yourself (see [Integration](/docs/titan/testing/integration)).

### `MockProvider` and `createMockProvider`

Two distinct helpers (`packages/testing/src/titan/nexus/mock-provider.ts`):

**`MockProvider`** (lines 112–126) — wrap a plain object so every
function becomes a `vi.fn()`. Hand the instance straight to a
`useValue` provider:

```typescript
import { MockProvider } from '@omnitron-dev/testing/titan';

const mailer = new MockProvider({
  send:  async () => ({ id: 'msg_1' }),
  flush: () => {},
});

const app = await Application.create({
  modules:   [AppModule],
  providers: [[MAILER, { useValue: mailer }]],
  disableGracefulShutdown: true,
});

// `mailer.send` is a vi.fn — assert on it directly:
expect(mailer.send).toHaveBeenCalledWith(expect.objectContaining({ to: 'ada@x.com' }));
```

**`createMockProvider(config)`** (lines 95–97) — builds a
`MockProviderDI` from a config object
(`MockProviderConfig`, lines 9–15: `{ token, value?, factory?,
spy?, autoMock? }`). Call `.getProvider()` to get the Nexus provider
definition:

```typescript
import { createMockProvider } from '@omnitron-dev/testing/titan';

const dbMock = createMockProvider({
  token: DATABASE,
  value: new FakeDatabase(),
});

const app = await Application.create({
  modules:   [AppModule],
  providers: [[DATABASE, dbMock.getProvider()]],  // → { useValue: ... }
  disableGracefulShutdown: true,
});
```

With `autoMock: true` (and no `value`/`factory`), the provider is a
proxy that lazily returns a `vi.fn()` for every accessed property —
handy for a dependency you must satisfy but never call.

## Module-level overrides

Sometimes you want to override a whole *module*, not a single
provider. Define a test variant and import it instead of the real one:

```typescript
@Module({
  providers: [{ provide: Database, useClass: FakeDatabase }],
  exports:   [Database],
})
class FakeDatabaseModule {}

@Module({
  imports: [
    FakeDatabaseModule,   // instead of DatabaseModule
    UsersModule,
    OrdersModule,
  ],
})
class TestAppModule {}

const app = await Application.create(TestAppModule, {
  disableGracefulShutdown: true,
});
```

Useful for swapping infrastructure (database, redis, file storage) to
test-friendly implementations across many tests. Note this uses the
module-decorator `{ provide, useClass }` shape, distinct from the
`create({ providers: [[token, def]] })` tuple shape.

## Spying without replacing

Sometimes you want to observe calls without changing behaviour:

```typescript
import { vi } from 'vitest';

const real = app.resolve(LoggerService);
const spy  = vi.spyOn(real, 'info');

// run code that should log
expect(spy).toHaveBeenCalledWith('ready', expect.objectContaining({ port: 3000 }));
```

The spy wraps the implementation; it doesn't replace it. The
`TestModule.spy(token, method)` helper does the same against the test
container.

## The "two real, one fake" pattern

A common integration setup — real logger, real config, fake database,
real services:

```typescript
const app = await Application.create({
  modules:   [LoggerModule, ConfigModule.forRoot({ /* ... */ }), UsersModule],
  providers: [[Database, { useClass: FakeDatabase }]],
  disableGracefulShutdown: true,
});
```

Catches wiring bugs without paying for a real database.

## Anti-patterns

- **Overriding too much.** A test where every provider is mocked is
  testing the test, not the code. Override only what you need to
  control.
- **Mutable shared fakes across tests.** Two tests that share one
  `FakeDatabase` see each other's data. Use a fresh fake per test, or
  `reset()` in `beforeEach`.
- **Testing the framework.** "Does the container resolve providers?"
  is the framework's job. Test your code; trust the framework.

---

---
id: "titan/testing/integration"
title: "Integration Tests"
description: "Boot a real Application with fake infrastructure."
---

# Integration Tests

An integration test boots a real Titan `Application` with selectively
faked infrastructure. It catches wiring, lifecycle, and module-
composition bugs that pure unit tests cannot.

Fakes are injected through the `providers` option — an array of
`[token, providerDefinition]` tuples that `Application.create`
registers after your modules, so they win over the modules' own
providers. See [DI Overrides](/docs/titan/testing/di-overrides) for the full
mechanism.

## The pattern

```typescript
import { Application } from '@omnitron-dev/titan';

describe('UsersModule (integration)', () => {
  let app: Application;
  let users: UsersService;

  beforeAll(async () => {
    app = await Application.create({
      modules:   [AppModule],
      providers: [
        [Database, { useClass: FakeDatabase }],
        [REDIS,    { useValue: fakeRedis     }],
      ],
      disableGracefulShutdown: true,
    });
    await app.start();
    users = await app.resolve(UsersService);
  });

  afterAll(async () => {
    await app.stop();
  });

  beforeEach(async () => {
    await (await app.resolve(Database)).reset();   // FakeDatabase impl
  });

  it('creates a user and finds it', async () => {
    const created = await users.create({ email: 'ada@x.com' });
    const found   = await users.findById(created.id);
    expect(found).toEqual(created);
  });
});
```

## What you get from a real Application

- **Lifecycle** runs. `onInit` and `onStart` for every provider.
  Catches "I forgot to await `connect()`" bugs.
- **Wiring** is real. The container resolves providers from your
  modules, not your test setup. Catches missing or mis-typed
  exports.
- **Decorators** apply. `@Cache`, `@RateLimit`, `@Auth`, `@Validate`
  all run. Catches order-of-decorators bugs.
- **Events** fire. Subscribers receive `module:initialized`,
  lifecycle events. Catches handlers that crash on subscription.

What you skip:

- The transport. Calls go directly through the container, not over
  the wire.
- Real backends (database, redis, third-party APIs). Fake those
  through the `providers` tuple array.

## Fixture lifetime

Three patterns:

### Per-suite — fastest

```typescript
beforeAll(...);          // boot once
afterAll(...);           // shut down once
beforeEach(...);         // reset state per test
```

Good when most tests share the same setup. Risk: state leaks
between tests if you forget a `beforeEach` reset.

### Per-test — safest

```typescript
beforeEach(async () => {
  app = await Application.create(...);
  await app.start();
});

afterEach(async () => {
  await app.stop();
});
```

Slower (boot per test), but no state leakage. Use when tests need
distinct configurations.

### Hybrid

```typescript
beforeAll(...);          // boot the heavy stuff (modules) once
beforeEach(...);         // reset light stuff (database, cache) per test
```

The right balance for most suites.

## Helpers

Common test setup deserves a helper. The `providers` option is typed
as `Array<[InjectionToken<unknown>, Provider<unknown>]>`
(`CreateOptions`, `IModuleOptions.imports` in `packages/titan/src/application/application.ts`),
so a per-suite booter just spreads in any extra tuples:

```typescript
// test/setup.ts
import { Application } from '@omnitron-dev/titan';
import type { InjectionToken, Provider } from '@omnitron-dev/titan/nexus';

export async function bootTestApp(
  extraProviders: Array<[InjectionToken<unknown>, Provider<unknown>]> = [],
) {
  const app = await Application.create({
    modules:   [AppModule],
    providers: [
      [Database, { useClass: FakeDatabase }],
      ...extraProviders,
    ],
    disableGracefulShutdown: true,
  });
  await app.start();
  return app;
}
```

Reduces boilerplate per test file:

```typescript
const app = await bootTestApp([[REDIS, { useValue: fakeRedis }]]);
```

For container-level mocking, spying, and interaction recording
without booting the full lifecycle, reach for the
`@omnitron-dev/testing/titan` helpers (`createTestModule`,
`MockProvider`, `createMockProvider`) documented in
[DI Overrides](/docs/titan/testing/di-overrides).

## Testing lifecycle hooks

```typescript
it('calls onInit in dependency order', async () => {
  const order: string[] = [];

  @Injectable()
  class A implements OnInit {
    async onInit() { order.push('A'); }
  }
  @Injectable()
  class B implements OnInit {
    constructor(private a: A) {}
    async onInit() { order.push('B'); }
  }

  @Module({ providers: [A, B] })
  class TestModule {}

  const app = await Application.create(TestModule, { disableGracefulShutdown: true });
  await app.start();

  expect(order).toEqual(['A', 'B']);

  await app.stop();
});
```

## Testing async startup failures

```typescript
it('aborts start when onInit throws', async () => {
  @Injectable()
  class Broken implements OnInit {
    async onInit() { throw new Error('boom'); }
  }

  @Module({ providers: [Broken] })
  class TestModule {}

  const app = await Application.create(TestModule, { disableGracefulShutdown: true });
  await expect(app.start()).rejects.toThrow('boom');

  // Failed start rolls back started modules and disposes the
  // container; the app settles in ApplicationState.Failed ('failed').
  expect(app.state).toBe('failed');
});
```

## Anti-patterns

- **Sharing one app across all suites.** Test isolation breaks.
  Each suite gets its own; or use one and reset rigorously.
- **Forgetting `disableGracefulShutdown`.** Without it, every
  test app installs SIGTERM handlers; multiple test apps stomp
  on each other.
- **Using real backends.** Tests become slow and flaky. Use fakes;
  reserve real-backend tests for a separate, less frequent suite.
- **Testing one method's wiring.** That's a unit test's job.
  Integration tests should cover module-level interactions.

→ Back to [Testing Overview](/docs/titan/testing/overview).

---

---
id: "titan/testing/modules"
title: "Testing modules"
description: "How to mock each ecosystem module in unit and integration tests."
---

# Testing modules

The framework-level pages cover the test pyramid ([Overview](/docs/titan/testing/overview))
and DI-driven mocking ([DI Overrides](/docs/titan/testing/di-overrides)). This page
goes per-module: the smallest mock that lets your code under test
run without the real backend.

The rule of thumb stays the same: unit tests instantiate the class
with mock dependencies; integration tests boot a real `Application`
with the heavy infrastructure overridden via the `providers:` tuple
array on `Application.create` (see
[DI Overrides](/docs/titan/testing/di-overrides) — there is no `overrides:` option).

## Built-in modules

### ConfigModule

Most tests just want config values. Skip the real loader; provide
the values directly:

```typescript
import { ConfigService } from '@omnitron-dev/titan/module/config';

const mockConfig = {
  get: <T>(path: string, defaultValue?: T): T => {
    const map: Record<string, unknown> = {
      'database.url':  'postgres://test',
      'cache.ttlMs':   60_000,
    };
    return (map[path] ?? defaultValue) as T;
  },
  has:      () => true,
  onChange: () => () => {},   // returns an unsubscribe fn
} as unknown as ConfigService;
```

For integration tests with a real `ConfigModule`, use the
`'object'` source:

```typescript
ConfigModule.forRoot({
  sources: [{ type: 'object', data: { database: { url: 'postgres://test' } } }],
})
```

### LoggerModule

The framework ships `createNullLogger()` for this exact case:

```typescript
import { createNullLogger } from '@omnitron-dev/titan/module/logger';

const logger = createNullLogger();   // discards everything
```

For tests that assert on log calls, use a spying logger:

```typescript
import { vi } from 'vitest';

const logger = {
  debug: vi.fn(),
  info:  vi.fn(),
  warn:  vi.fn(),
  error: vi.fn(),
  child: vi.fn(() => logger),
} as any;

// Later:
expect(logger.info).toHaveBeenCalledWith('findById', expect.objectContaining({ id: 'u_42' }));
```

## Official modules

### titan-auth

Mock `JWTService` to bypass real signing:

```typescript
import type { IJWTService, IAuthContext, IJWTPayload } from '@omnitron-dev/titan-auth';

const claims: IJWTPayload = { sub: 'u_42', role: 'user' };

const fakeJwt: IJWTService = {
  verify:        vi.fn(async () => claims),
  createContext: vi.fn(
    async (): Promise<IAuthContext> => ({
      userId: 'u_42',
      role: 'user',            // singular, and a string — not `roles: string[]`
      tenantId: 't_1',
      isServiceRole: false,
      claims,
    }),
  ),
  clearCache:    vi.fn(),
  getCacheStats: () => ({ size: 0, maxSize: 1000, hits: 0, misses: 0, hitRate: 0 }),
};
```

> **Three different auth contexts share these names**
>
> The fake above is typed against `IAuthContext` from
> `@omnitron-dev/titan-auth`: `{ userId, role, tenantId, isServiceRole,
> claims }`, all required. Two others exist and are not interchangeable:
>
> | Type           | Where                                | Shape                                                                     |
> | -------------- | ------------------------------------ | ------------------------------------------------------------------------- |
> | `IAuthContext` | `@omnitron-dev/titan-auth`           | `role: string`, plus `tenantId`, `isServiceRole`, `claims` — all required |
> | `IAuthContext` | titan `netron/interfaces/core-types` | `roles?: string[]`, `permissions?`, `claims?` — all optional              |
> | `AuthContext`  | titan `netron/auth/types`            | `roles: string[]`, `permissions: string[]`, `scopes?`, `token?`           |
>
> An earlier version of this example wrote `{ userId, roles: ['user'] }`
> — correct for the netron shapes, wrong for the one it imported. If a
> context literal will not typecheck, check which of the three you have
> before adding a cast; the cast is how the mismatch reaches runtime.

For `@RequireAuth` decorator tests, supply a mock middleware:

```typescript
const fakeAuthMiddleware = {
  authenticate:         vi.fn(async () => fakeAuthContext),
  authenticateRequired: vi.fn(async () => fakeAuthContext),
  extractToken:         vi.fn(() => 'fake-token'),
  validateApiKey:       vi.fn(() => ({ valid: true, type: 'service' })),
};

const service = new MyService();
(service as any).__authMiddleware__ = fakeAuthMiddleware;
```

### titan-cache

For services that depend on a cache, a `Map`-backed fake covers
most cases:

```typescript
class FakeCache<T> {
  private readonly store = new Map<string, T>();

  async get(key: string) { return this.store.get(key); }
  async set(key: string, value: T) { this.store.set(key, value); }
  async delete(key: string) { return this.store.delete(key); }
  async clear() { this.store.clear(); }
  async has(key: string) { return this.store.has(key); }
  async getOrSet(key: string, factory: () => Promise<T>) {
    if (!this.store.has(key)) this.store.set(key, await factory());
    return this.store.get(key)!;
  }
  async invalidateByTags() { /* ignore */ return 0; }
  // …minimal surface for your tests
}
```

For testing decorator-driven caching (`@Cacheable`), an integration
test with an in-memory cache is cleaner than mocking:

```typescript
const app = await Application.create({
  modules: [TitanCacheModule.forRoot({ maxSize: 100, defaultTtl: 60 })],
  providers: [/* your service */],
});
```

### titan-redis

For unit tests, stub the small surface your code uses:

```typescript
const fakeRedis = {
  get:     vi.fn(async () => null),
  set:     vi.fn(async () => 'OK'),
  del:     vi.fn(async () => 1),
  exists:  vi.fn(async () => 0),
  expire:  vi.fn(async () => 1),
  ttl:     vi.fn(async () => -2),
  ping:    vi.fn(async () => 'PONG'),
  // hash / set / list / sorted-set methods as your code needs
};
```

For integration tests that exercise Redis semantics (Lua,
clustering), use `ioredis-mock` or spin up Redis in Docker via
`pnpm test:up`.

### titan-lock

The most common pattern: stub `withLock` to either run the callback
or skip it.

```typescript
import type { IDistributedLockService } from '@omnitron-dev/titan-lock';

// "Lock always available" — runs the callback unconditionally
const fakeLockUnlocked: Partial<IDistributedLockService> = {
  withLock: async (key, fn) => fn(),
  acquireLock: async () => 'fake-lock-id',
  releaseLock: async () => true,
  isLocked:    async () => false,
};

// "Lock always held" — skips the callback
const fakeLockHeld: Partial<IDistributedLockService> = {
  withLock: async (_key, _fn, opts) => {
    if (opts?.skipOnLockFailure) return undefined;
    throw new Error('LockUnavailable');
  },
  acquireLock: async () => null,
  isLocked:    async () => true,
};
```

For `@WithDistributedLock` decorator tests, inject the mock under
`__lockService__`:

```typescript
const service = new MyScheduledTasks();
(service as any).__lockService__ = fakeLockUnlocked;
(service as any).loggerModule   = { getLogger: () => fakeLogger };
await service.myJob();   // runs the body
```

### titan-database

Tests should hit a **real** database — even a transient SQLite —
because most bugs live in the SQL, not the surrounding code.

```typescript
TitanDatabaseModule.forRoot({
  connection: {
    dialect:    'sqlite',
    connection: ':memory:',
    migrationsPath: './migrations',
  },
})
```

Run migrations in `beforeAll`; reset between tests via a fresh DB
file or `TRUNCATE`-style cleanup.

For unit-testing a service that depends on a repository, mock the
repository directly:

```typescript
const fakeRepo = {
  find:    vi.fn(async () => ({ id: 'u_42', email: 'ada@example.com' })),
  create:  vi.fn(async (data) => ({ id: 'new-id', ...data })),
  update:  vi.fn(async (id, patch) => ({ id, ...patch })),
  delete:  vi.fn(async () => true),
} as unknown as UsersRepository;

const service = new UsersService(fakeRepo);
```

### titan-events

`EventsService` works fine in tests with a fresh in-process
instance — no mock needed:

```typescript
import { EventsModule } from '@omnitron-dev/titan-events';

const app = await Application.create({
  modules: [EventsModule.forRoot({ wildcard: true, history: { enabled: false } })],
  providers: [MyEmitter, MyHandler],
});

await app.start();

// Trigger emission
await app.resolve(MyEmitter).doSomething();

// Assert handler ran
await new Promise(r => setImmediate(r));   // let async handlers flush
expect(spy).toHaveBeenCalled();
```

Use `events.waitFor(name, timeout)` for deterministic assertions:

```typescript
const promise = events.waitFor('user.created', 1000);
await service.create({ /* … */ });
const payload = await promise;
expect(payload.id).toBeDefined();
```

### titan-health

Register a mock indicator that returns a controllable status:

```typescript
class FakeHealthIndicator implements IHealthIndicator {
  name = 'fake';
  constructor(public status: 'healthy' | 'degraded' | 'unhealthy' = 'healthy') {}
  async check() { return { status: this.status }; }
}

const fake = new FakeHealthIndicator();
TitanHealthModule.forRoot({
  enableMemoryIndicator: false,
  indicators: [class extends FakeHealthIndicator {}],   // or register at runtime
});

// In test:
fake.status = 'unhealthy';
expect(await healthService.isReady()).toBe(false);
```

### titan-metrics

For tests, a no-op metrics service is usually sufficient:

```typescript
const fakeMetrics = {
  record:      vi.fn(),
  recordBatch: vi.fn(),
  recordTyped: vi.fn(),
  getSnapshot: async () => ({ metrics: [] }),
  start:       vi.fn(),
  stop:        vi.fn(),
};
```

For tests that assert metric calls:

```typescript
expect(fakeMetrics.recordTyped).toHaveBeenCalledWith(
  'counter', 'users.created.total', { source: 'web' }, 1,
);
```

### titan-notifications

Use the mock channels the module already exports:

```typescript
import { MockEmailChannel, MockSMSChannel, MockPushChannel }
  from '@omnitron-dev/titan-notifications';

NotificationsModule.forRoot({
  channels: [new MockEmailChannel(), new MockSMSChannel(), new MockPushChannel()],
  enableInApp: false,
})
```

Mock channels record every delivery so you can assert in tests:

```typescript
const emailMock = new MockEmailChannel();
await notify.send({ userId, email }, { template: 'welcome', data: { name } });
expect(emailMock.deliveries).toHaveLength(1);
expect(emailMock.deliveries[0].to).toBe('ada@example.com');
```

### titan-pm

Set `testing.useMockSpawner: true` to swap the real spawner for a
mock that runs worker classes in the same process:

```typescript
ProcessManagerModule.forRoot({
  isolation: 'worker',
  testing:   { useMockSpawner: true },
})
```

The mock spawner runs `@Process`-decorated classes in-process so
tests don't pay the worker-thread cost.

### titan-ratelimit

Stub `consume` / `enforce` for unit tests:

```typescript
const fakeRate = {
  consume: vi.fn(async () => ({ allowed: true, remaining: 99, resetAt: Date.now() + 60_000 })),
  enforce: vi.fn(),
  getStatus: vi.fn(async () => ({ allowed: true, remaining: 99, resetAt: 0 })),
  reset:   vi.fn(),
};
```

For integration tests with real limiting, use `storageType: 'memory'`
and the small-window option to avoid waiting:

```typescript
TitanRateLimitModule.forRoot({
  storageType:     'memory',
  defaultLimit:    2,
  defaultWindowMs: 100,
})

// Test:
await rate.consume('key');
await rate.consume('key');
const r = await rate.consume('key');
expect(r.allowed).toBe(false);
```

### titan-discovery

Stub the service for unit tests:

```typescript
const fakeDiscovery = {
  registerNode:        vi.fn(async () => {}),
  deregisterNode:      vi.fn(async () => {}),
  getActiveNodes:      vi.fn(async () => [{ nodeId: 'test-node', address: 'http://localhost' }]),
  findNodesByService:  vi.fn(async () => []),
  isNodeActive:        vi.fn(async () => true),
};
```

For integration tests, point at a transient Redis (Docker compose
or `ioredis-mock`).

### titan-scheduler

The scheduler accepts a `disabled: true` per-job option to register
without auto-starting:

```typescript
@Cron(CronExpression.EVERY_HOUR, { disabled: true })
async myJob() { /* … */ }
```

In tests, manually invoke the registered job:

```typescript
const job = scheduler.getJob('TaskService.myJob');
await job!.handler();   // invoke without waiting for cron
```

Or invoke the underlying method directly — the decorator doesn't
prevent direct calls.

### titan-telemetry-relay

Use the `'aggregator'` mode with an in-memory `TelemetryAggregator`
for assertions:

```typescript
class InMemoryAggregator implements TelemetryAggregator {
  entries: TelemetryEntry[] = [];
  async write(entries: TelemetryEntry[]) { this.entries.push(...entries); }
  async query() { return this.entries; }
}

const agg = new InMemoryAggregator();
const relay = new TelemetryRelayService({ role: 'aggregator' });
relay.setAggregator(agg);
await relay.start();

// Producer side:
await relay.receive('test-node', [
  { id: '1', timestamp: Date.now(), type: 'log', app: 'test', data: { msg: 'hello' } },
]);

expect(agg.entries).toHaveLength(1);
```

## Generic patterns

### One mock per test, not shared

A `FakeCache` shared across tests will see each other's data. Use
`beforeEach` to give each test a fresh fake.

### `disableGracefulShutdown: true`

Integration tests boot many `Application` instances. Without this
flag, every test installs SIGTERM handlers — they stomp on each
other.

```typescript
const app = await Application.create({
  modules: [/* … */],
  disableGracefulShutdown: true,
});
```

### Cross-runtime testing

`@omnitron-dev/testing` covers cross-runtime concerns (Node / Bun /
Deno). If your module needs to ship to all three, run the same test
file against each runtime via the package's harness.

## Anti-patterns

- **Mocking everything.** A test where every dep is mocked tests
  the test, not the code. Mock the slow / external / unreliable;
  use real instances for the rest.
- **Shared fake state across tests.** Use `beforeEach` to reset.
- **Real Redis / Postgres per unit test.** Spin once per test
  suite (`beforeAll`); reset between tests.
- **Testing the framework.** "Does `@Inject` work?" is the
  framework's job. Test your code's behaviour through it.

## See also

- [Testing / Overview](/docs/titan/testing/overview) — the test pyramid
- [Testing / DI Overrides](/docs/titan/testing/di-overrides) — replacing providers
- [Testing / Integration](/docs/titan/testing/integration) — real `Application`
  with selective fakes

---

---
id: "titan/testing/overview"
title: "Testing"
description: "Unit, integration, end-to-end — patterns that scale."
---

# Testing

Titan is designed to be tested at three layers, each with a distinct
purpose:

| Layer       | What it tests                                     | Speed         |
| ----------- | ------------------------------------------------- | ------------- |
| Unit        | One class with mocked dependencies                | <1 ms / test  |
| Integration | Several classes through the container, real fakes | 10 ms / test  |
| End-to-end  | The whole app over a real transport, real backend | 100 ms / test |

The pyramid: many unit tests, fewer integration, fewer still e2e.

This page is the entry point. Detail in:

- [DI Overrides](/docs/titan/testing/di-overrides) — patterns for mocking through
  the container.
- [Integration](/docs/titan/testing/integration) — partial app boots, real
  ConfigModule and LoggerModule, fake everything else.

## The unit test pattern

A `@Service` is just a class. Test it like one:

```typescript
import { describe, it, expect, vi } from 'vitest';

describe('UsersService', () => {
  it('returns the user when found', async () => {
    const repo   = { findById: vi.fn().mockResolvedValue({ id: 'u_1', email: 'x' }) };
    const logger = { info: vi.fn() };

    const svc = new UsersService(repo as any, logger as any);

    const user = await svc.findById('u_1');
    expect(user).toEqual({ id: 'u_1', email: 'x' });
    expect(repo.findById).toHaveBeenCalledWith('u_1');
  });

  it('throws NotFoundError when missing', async () => {
    const repo   = { findById: vi.fn().mockResolvedValue(null) };
    const svc    = new UsersService(repo as any, { info: vi.fn() } as any);

    await expect(svc.findById('missing')).rejects.toBeInstanceOf(NotFoundError);
  });
});
```

No container, no application, no Netron. Just the class and its
dependencies.

This is the right level for **business logic** tests. They run in
microseconds. You can have thousands.

## When to use the container

Use the container (and a real `Application`) when:

- You're testing **wiring** — does this module actually export the
  right providers?
- You're testing **lifecycle** — does my `onInit` run after the
  database connects?
- You're testing **decorators** — does `@Cache` actually short-
  circuit on a hit?

For all of these, see [DI Overrides](/docs/titan/testing/di-overrides) and
[Integration](/docs/titan/testing/integration).

## When to go end-to-end

End-to-end tests boot a real `Application`, bind a real transport, and
drive it with a second `Netron` acting as the client. Use them
sparingly — they catch wire-format and transport bugs that unit tests
miss, but they're slow.

(`NetronClient` from `@omnitron-dev/netron-browser` is the *browser*
client. It works against the same server, but a Node test does not
need it and would have to take the extra dependency.)

Pattern:

```typescript
import { Application } from '@omnitron-dev/titan';
import { Netron } from '@omnitron-dev/titan/netron';
import { HttpTransport } from '@omnitron-dev/titan/netron/transport/http';

let app: Application;
let client: Netron;

beforeAll(async () => {
  app = await Application.create(AppModule, { disableGracefulShutdown: true });

  // Register the transport BEFORE app.start(): starting the application
  // starts Netron, and a second netron.start() throws "Netron already
  // started". There is no `netron` option on Application.create that binds
  // a server for you — the options bag has an index signature, so anything
  // you invent there is accepted and ignored.
  const netron = app.netron!;
  netron.registerTransport('http', () => new HttpTransport());
  netron.registerTransportServer('http', {
    name: 'http',
    options: { host: '127.0.0.1', port: 0 },   // 0 = pick a free port
  });
  await app.start();

  const port = netron.transportServers.get('http')!.port;

  client = new Netron(logger, {});
  client.registerTransport('http', () => new HttpTransport());
  await client.start();
});

afterAll(async () => {
  await client.stop();
  await app.stop();
});

it('end-to-end: create + find', async () => {
  const port  = app.netron!.transportServers.get('http')!.port;
  const peer  = await client.connect(`http://127.0.0.1:${port}`);
  const users = await peer.queryInterface<UsersService>('users@1.0.0');

  const user  = await users.create({ email: 'ada@x.com' });
  const found = await users.findById(user.id);
  expect(found).toEqual(user);
});
```

Two details worth knowing before you debug them:

- **`transportServers.get('http')!.port`** is the bound port — with
  `port: 0` it is the one the OS picked. The server also has an
  `address` property, but it is a **string holding the host**, not a
  `net.Server`-style `address()` method; calling it throws.
- **The registration order is load-bearing.** `app.start()` starts
  Netron, so a transport registered afterwards is never bound and
  `netron.start()` reports `Netron already started`.

## The right ratio

A typical Titan service ends up with:

- **80% unit tests** — fast, focused, business-logic-only.
- **15% integration tests** — wiring, lifecycle, module
  composition.
- **5% e2e tests** — happy paths over the wire.

If you find yourself writing many integration tests for one service,
the service is doing too much. Split it.

[Integration tests](/docs/titan/testing/integration) exercise a running application over a real transport.

---

---
id: "titan/tracing"
title: "Tracing"
description: "W3C-compatible trace context propagation, zero dependencies."
---

# Tracing

Titan ships a minimal tracing layer that carries a W3C `traceparent`-
compatible context across async scopes via `AsyncLocalStorage`.
No OpenTelemetry SDK is required.

This is the foundation. For full observability (spans exported to a
collector, exemplars, sampling), layer the
[`titan-telemetry-relay`](/docs/titan/modules/telemetry-relay) or your own
OTel integration on top.

## Public surface

```typescript
import {
  // From '@omnitron-dev/titan' main entry (re-exports a subset):
  createSpanId,
  createTraceId,
  currentTrace,
  extractTraceparent,
  formatTraceparent,
  parseTraceparent,
  startSpan,
  withTrace,
  type TraceContext,
} from '@omnitron-dev/titan';

// From '@omnitron-dev/titan/tracing' (the full surface):
import {
  INVALID_SPAN_ID,
  INVALID_TRACE_ID,
  TRACE_FLAGS,
  makeTraceContext,
} from '@omnitron-dev/titan/tracing';
```

## `TraceContext`

```typescript
interface TraceContext {
  readonly traceId:       string;   // 32 hex chars (16 bytes)
  readonly spanId:        string;   // 16 hex chars (8 bytes)
  readonly parentSpanId?: string;   // undefined for a root span
  readonly flags:         number;   // W3C trace-flags byte; bit 0 = sampled
}
```

The field is `flags` (not `traceFlags`), and there is **no**
`traceState` field — the type ships only the identifier portion of
W3C trace context. It is immutable; produce children with
`startSpan()` rather than mutating it.

Constants:

```typescript
INVALID_TRACE_ID     // '00000000000000000000000000000000'
INVALID_SPAN_ID      // '0000000000000000'
TRACE_FLAGS.SAMPLED  // 0x01
TRACE_FLAGS.NONE     // 0x00   (there is no TRACE_FLAGS.NOT_SAMPLED)
```

## Reading the active context

```typescript
const trace = currentTrace();   // TraceContext | undefined
```

Returns `undefined` if no trace is active in the current async
scope.

## Establishing a context

```typescript
import { makeTraceContext, withTrace, TRACE_FLAGS } from '@omnitron-dev/titan/tracing';

await withTrace(
  makeTraceContext({ traceId, spanId, flags: TRACE_FLAGS.SAMPLED }),
  async () => {
    await this.doWork();    // currentTrace() returns the bound context inside this scope
  }
);
```

`withTrace(ctx, fn)` installs `ctx` for the duration of `fn` (sync or
async) via `AsyncLocalStorage`, then restores the previous context.
`makeTraceContext` is a convenience for building a context literal
with `flags` defaulted to `SAMPLED`.

## Starting a span

```typescript
import { startSpan, withTrace } from '@omnitron-dev/titan/tracing';

// startSpan(parent?) returns a NEW TraceContext (a child of the
// current trace, or of `parent` if given). It does not take a name
// or attributes, and does not return an `end()` callback.
const span = startSpan();

await withTrace(span, async () => {
  await this.db.query(...);
});
```

`startSpan(parent?)` returns a fresh `TraceContext`: same `traceId`,
a new `spanId`, and `parentSpanId` set to the inherited span. If no
parent (and no current trace) exists, it mints a new root trace.
There is **no** span lifecycle, attributes, or `end()` here — this
module is propagation-only. Span timing/attributes/export belong to
a telemetry layer (the relay module or an OTel sink) on top.

## W3C `traceparent` header

```typescript
// Parse incoming header
const trace = extractTraceparent('00-T-S-01');     // returns TraceContext
// or:
const trace = parseTraceparent('00-T-S-01');

// Serialise for outgoing
const header = formatTraceparent(trace);            // '00-T-S-01'
```

Netron uses these helpers internally to propagate trace context
across RPC calls.

## Manual propagation

`AsyncLocalStorage` does not propagate across worker threads,
queue boundaries, or some Promise pool implementations. Capture
and restore manually:

```typescript
const captured = currentTrace();
worker.postMessage({ work, trace: captured });

// On the worker side:
worker.on('message', async ({ work, trace }) => {
  await withTrace(trace, async () => {
    await processWork(work);
  });
});
```

## Anti-patterns

- **Reading the trace context manually inside business logic.**
  Defeats the point. Trust the framework's propagation; let the
  trace context attach to logs and downstream calls automatically.
- **Trace IDs in error payloads.** Errors carry their own
  structure. Trace correlation happens in the log aggregator by
  `traceId`, not by passing the ID in the error.
- **Skipping `withTrace` across async boundaries.** A trace
  context that doesn't propagate across a worker thread leaves
  the downstream work uncorrelated. Capture and restore.

→ Back to [Titan Overview](/docs/titan/overview).

---

---
id: "titan/validation/contracts"
title: "Contracts"
description: "Method contracts — bind input + output schemas as a single declaration."
---

# Contracts

A `Contract` bundles per-method input/output schemas (plus optional
metadata) for a whole service. `@Contract` applies it to the service
**class**, mapping each method name to its `{ input, output, … }`.

## Basic shape

```typescript
import { contract } from '@omnitron-dev/titan/validation';
import { Contract } from '@omnitron-dev/titan/decorators';

// A contract is a map of method-name → MethodContract, plus metadata.
const UsersContract = contract(
  {
    findById: {
      input:  z.object({ id: z.string().uuid() }),
      output: z.union([UserSchema, z.null()]),
    },
  },
  { name: 'users', version: '1.0.0', description: 'User service' },
);

@Service('users@1.0.0')
@Contract(UsersContract)          // class-level, not per-method
class UsersService {
  @Public()
  async findById(input: { id: string }) {
    return this.repo.findById(input.id);
  }
}
```

Each `MethodContract` may declare `input`, `output`,
`errors` (a `Record<httpStatus, ZodSchema>`), `stream`, `options`
(`ValidationOptions`), and an `http` extension
(`{ status?, contentType?, streaming?, responseHeaders?, openapi? }`).
What happens when the contract is applied — the `ValidationMiddleware`
wraps each method whose contract declares an `input` or `output`
(`src/validation/validation-middleware.ts`):

1. **Input validation.** The first argument is parsed against the
   method's `input` before the method body runs. Failures throw
   `ValidationError` (status 422).
2. **Output validation.** The return value is parsed against
   `output` after the method body runs. Failures throw the same
   `ValidationError` — output validation runs through the identical
   `validateAsync` path as input. For a `stream` method, each yielded
   item is validated against `output`.
3. **Metadata exposure.** The decorator stores the contract in
   reflection metadata (`validation:contract`) and merges it into the
   service's `netron:service` metadata, so clients (and the Omnitron
   console) can introspect it.

## Why contracts, not just schemas

A method-level `@Validate(mc)` validates one method in place. A
class-level `@Contract` adds:

- **One contract object for the whole service.** Define every method's
  `input`/`output` in a single map, import it, reuse it on the client.
- **Output validation at the boundary.** Catches "I returned
  `undefined` when I promised a `User`" bugs before the value leaves
  the service. (Runs in dev and prod alike — see below.)
- **Wire-format introspection.** Because the contract is stored in the
  service's metadata, the Omnitron console can render it for any
  registered service.
- **Versioning.** A contract carries metadata including `version`,
  alongside the service version.
- **Per-status error schemas.** A `MethodContract`'s `errors` map
  binds a Zod schema to each HTTP status the method can return.

## Versioning

A contract's `version` lives in its metadata, independent of the
service version:

```typescript
const UsersContractV2 = contract(
  {
    findById: {
      input:  z.object({ id: z.string().uuid(), include: z.array(z.string()).optional() }),
      output: UserSchemaV2,
    },
  },
  { name: 'users', version: '2.0.0' },   // contract metadata
);
```

## Predefined contract templates

The `Contracts` namespace ships ready-made shapes:

```typescript
import { Contracts } from '@omnitron-dev/titan/validation';

// create/read/update/delete/list; second arg overrides the id schema
// (default z.string().uuid()). Ships 409 on create, 404 on update/delete,
// and a paginated { items, total, offset, limit } list output.
const UserContract = Contracts.crud(UserSchema);
const KeyedContract = Contracts.crud(UserSchema, z.number());

// subscribe (stream: true) + unsubscribe; second arg is the filter schema
const FeedContract = Contracts.streaming(PostSchema);

// single execute(input) -> output method
const CalcContract = Contracts.rpc(InputSchema, OutputSchema);
```

For a fully custom contract there is also a fluent `contractBuilder()`:
`contractBuilder().method(name, mc).withMetadata({ … }).build()`.

## Per-status error schemas

A `MethodContract` can declare the error payload for each status (the
`errors` field is `Record<number, ZodSchema>` — HTTP status → payload
schema):

```typescript
const UsersContract = contract({
  create: {
    input:  CreateUserSchema,
    output: UserSchema,
    errors: {
      409: z.object({ code: z.literal('ALREADY_EXISTS'), message: z.string() }),
    },
  },
});
```

These schemas are *typed contracts for the errors you emit*, not an
automatic interceptor. Throw a contract-validated error with
`ContractError.create(contract, method, status, payload)` — it looks
up the schema for that method+status and parses `payload` against it,
so a malformed error payload fails fast at the source:

```typescript
import { ContractError } from '@omnitron-dev/titan/errors';

async create(input: CreateUser) {
  if (await this.repo.exists(input.email)) {
    // payload is type-checked against the 409 schema above
    throw ContractError.create(UsersContract, 'create', 409, {
      code: 'ALREADY_EXISTS',
      message: 'A user with that email already exists',
    });
  }
  return this.repo.create(input);
}
```

The static type of `payload` is inferred from the contract via
`ContractTypes.Errors`, so an unlisted status or a payload that
doesn't match the schema is a compile-time error.

## When to use `@Validate` vs `@Contract`

Both decorators carry the **same `MethodContract` shape** — `@Validate`
takes one `MethodContract` and `@Contract` takes a map of them. So the
choice is not about *which validation features* you get (both support
`input`, `output`, `errors`, `stream`, `options`); it is about *where
the schema lives* and whether you want a single named, versioned,
introspectable contract object.

| Use `@Validate(mc)` (method) when …       | Use `@Contract(c)` (class) when …               |
| ----------------------------------------- | ----------------------------------------------- |
| You want to annotate one method in place  | You want one contract covering many methods     |
| The schema is local to that method        | You want a single reusable contract object      |
| You don't need a named/versioned contract | The contract is stable and worth versioning     |
| The contract surface is small / ad hoc    | Clients/console should introspect the whole API |

`@Validate` is the per-method method decorator; `@Contract` is the
service-wide class decorator. When both are present, the method-level
`@Validate` wins for that method (the middleware checks
`validation:method` before the class contract). Convenience aliases
`@ValidateInput(schema)` and `@ValidateOutput(schema)` wrap `@Validate`
for the input-only / output-only cases.

## Output validation

When a `MethodContract` declares `output`, the return value is
validated (in dev and prod alike). The cost is small but non-zero
(\~1µs per object). There is no built-in per-environment toggle field
on the contract; gate it yourself (e.g. omit `output` outside dev, or
via `options`) if profiling shows it matters. Output validation
catches bugs that would otherwise ship as wire-incompatible responses.

## Anti-patterns

- **Schema inline at every method.** Define schemas once (in a
  `schemas.ts` file) and import. Inline schemas lead to drift
  between server and client.
- **Validating outputs of every method.** Reserve output schemas for
  `@Public` methods. Internal methods are typed by TypeScript;
  runtime validation there is overhead with no benefit.

---

---
id: "titan/validation/error-handling"
title: "Validation Error Handling"
description: "What clients see when validation fails — and how to shape it."
---

# Validation Error Handling

When validation fails, the framework throws a `ValidationError`. The
client receives the same class with the same payload over the wire.
This page covers what's in the error and how to customise the
client-facing shape.

## Default error shape

The canonical `ValidationError` (from `@omnitron-dev/titan/errors`)
extends `TitanError`, so it carries `code` / `httpStatus` / `details`
like any other. The validation issues live in `details.errors`, and
each issue's location is a **dotted-string `path`**:

```typescript
{
  name:       'ValidationError',
  code:       422,                       // ErrorCode.VALIDATION_ERROR
  httpStatus: 422,
  message:    'Validation failed',
  details: {
    errors: [
      { path: 'email', message: 'Invalid email address', code: 'invalid_format' },
      { path: 'age',   message: 'Too small: expected number to be >=13', code: 'too_small' },
    ],
  },
  timestamp:  1747339200000,
}
```

> The lightweight `ValidationError` from
> `@omnitron-dev/titan/validation` (used by `ValidationEngine`)
> differs: there, `code` is the string `'VALIDATION_ERROR'` and
> `statusCode` is `422`, and its `toJSON()` emits `{ code, message,
> errors }`. Both expose the issues under an `errors` array, not
> `fields`.

Status code `422` follows REST convention for "syntactically valid but
semantically wrong". For requests that fail to parse at all (malformed
msgpack, unknown service), the framework uses `400`.

## On the client side

```typescript
import { ValidationError } from '@omnitron-dev/titan/errors';

try {
  await users.create(input);
} catch (e) {
  if (e instanceof ValidationError) {
    for (const err of e.validationErrors) {       // also at e.details.errors
      console.log(`field=${err.path}: ${err.message}`);
    }
  }
}
```

The error is preserved across the wire — Netron serialises the
`TitanError` and rehydrates it as the same class (the hierarchy is
shared via `@omnitron-dev/netron-protocol`), so `code`, `httpStatus`,
and `details` survive intact.

## Structured field errors

Each entry in `validationErrors` (mirrored at `details.errors`):

| Field      | Type       | Meaning                                        |
| ---------- | ---------- | ---------------------------------------------- |
| `path`     | `string`   | Dotted path to the bad field (`'address.zip'`) |
| `message`  | `string`   | Human-readable message (from Zod or override)  |
| `code`     | `string`   | Zod issue code (`invalid_format`, `too_small`) |
| `expected` | `unknown?` | What was expected (when applicable)            |
| `received` | `unknown?` | What was received (when applicable)            |

`path` is built by joining the Zod issue path with `.` — so nested
fields read like `'address.zip'`. Frontend forms can use it directly:

```typescript
catch (e) {
  if (e instanceof ValidationError) {
    e.validationErrors.forEach((err) => {
      form.setFieldError(err.path, err.message);
    });
  }
}
```

## Customising messages

Per-field messages via Zod:

```typescript
const Schema = z.object({
  email: z.string().email({ message: 'Please enter a valid email address.' }),
  age:   z.number().int().min(13, { message: 'You must be 13 or older.' }),
});
```

Or globally through a custom error map:

```typescript
import { z } from '@omnitron-dev/titan/validation';

// zod 4: the map takes ONE argument. A second parameter makes the function
// unassignable to `$ZodErrorMap`, and there is no `ctx.defaultError` — return
// `undefined` to fall through to zod's own message.
z.setErrorMap((issue) => {
  if (issue.code === 'invalid_format' && issue.format === 'email') {
    return { message: 'Bad email.' };
  }
  return undefined;
});
```

## Translating to client formats

The canonical `ValidationError` has helper methods for common
formats and per-field queries:

```typescript
const e = ...; // ValidationError instance

e.getSimpleFormat();
// { code: 'VALIDATION_ERROR', message, errors: ['Invalid email format', …] }

e.getDetailedFormat();
// { code: 'VALIDATION_ERROR', message, errors: [{ path, message, code, expected?, received? }, …] }

e.hasFieldError('email');   // boolean
e.getFieldErrors('email');  // [{ message, code }, …]
```

(`getSimpleFormat` / `getDetailedFormat` are also the right things to
put in an API response body.)

## Validation vs domain errors

Two error categories are easy to confuse:

- **`ValidationError`** — the input does not match the schema. Status
  422\. Client should fix the input.
- **A conflict / domain error** — the input matches the schema but
  cannot be applied (email already taken, order already shipped).
  Status 409, thrown via `Errors.conflict(...)` (a `TitanError` with
  `ErrorCode.CONFLICT`) or a `DomainError`. Client should retry with
  different input or show a domain message.

Use the right one. Throwing `ValidationError` for "email already
taken" tells the client "your email is malformed", which is wrong.

```typescript
@Public()
@Validate({ input: CreateUserSchema })
async create(input: z.infer<typeof CreateUserSchema>) {
  const existing = await this.repo.findByEmail(input.email);
  if (existing) throw Errors.conflict('email already registered', { email: input.email });
  // … create user
}
```

## Anti-patterns

- **Catching `ValidationError` and re-throwing as `Error`.** Strips
  the structured detail; the client sees an opaque 500.
- **Validating in the method body manually.** Defeats the
  pre-compiled validator's performance gain and duplicates the
  schema. Use `@Validate` or `@Contract`.
- **Returning validation errors as success values.** A 200 response
  with an `errors` array makes the client check both the HTTP status
  and the body. Throw the error; let the framework signal failure.

→ Back to [Validation Overview](/docs/titan/validation/overview).

## Rejected HTTP uploads

The native Node HTTP transport returns a correlated `413` before invoking a
service when either the declared length or the streamed body exceeds
`maxRequestSize`. It discards the remaining input without retaining its chunks.
Closing the socket immediately after sending the refusal can reset an upload
still in flight, causing a client to see `EPIPE` or `ECONNRESET` instead of the
HTTP result. The transport therefore allows the response and request directions
to finish before a requested close.

Rejected uploads remain bounded by `requestTimeout`; if ordinary request
execution has explicitly disabled that timeout, the normal 30-second default
still bounds disposal of a refused body. A client that never completes the
upload cannot keep that rejected request alive indefinitely. Admission limits
and service validation remain independent.

---

---
id: "titan/validation/overview"
title: "Validation"
description: "Schema-driven input validation at the service boundary."
---

# Validation

A service's job is to trust its inputs. To do that, the framework
needs to know the *shape* the inputs are supposed to have. Titan's
validation system uses Zod schemas at the service boundary —
declarative, fast, and type-safe.

## What Titan validates

By default, **only what you ask it to**. Titan does not auto-validate
every parameter — that would force every method to have a schema even
when the type system already gives you the guarantee.

Where validation pays:

| Boundary                                                  | Validate |
| --------------------------------------------------------- | -------- |
| `@Public` method called from a Netron client              | Yes      |
| Internal method called from another `@Service` in-process | Optional |
| Private helper inside a class                             | No       |
| Constructor parameters                                    | No       |

The general rule: validate at *trust boundaries*. A `@Public` method
is the canonical trust boundary; what crosses the wire is untrusted
until it has been parsed against a schema.

## The minimal example

```typescript
import { z } from '@omnitron-dev/titan/validation';
import { Validate } from '@omnitron-dev/titan/decorators';

const CreateUserSchema = z.object({
  email: z.string().email(),
  name:  z.string().min(1).max(120),
  age:   z.number().int().min(13).optional(),
});

@Service('users@1.0.0')
class UsersService {
  // @Validate is a METHOD decorator taking a MethodContract
  // ({ input, output?, options? }) — not a parameter decorator.
  @Public()
  @Validate({ input: CreateUserSchema })
  async create(input: z.infer<typeof CreateUserSchema>) {
    // The first argument is parsed and trusted here.
    // The TypeScript type matches the schema exactly.
    return this.repo.create(input);
  }
}
```

`@Validate` validates the method's **first argument** against
`input`. Shorthands `@ValidateInput(schema, options?)` and
`@ValidateOutput(schema, options?)` exist for the common cases.

The body of `create` runs only if the input passes validation. If
not, the caller receives a typed `ValidationError` with the schema's
field-level error messages.

## How validation runs

```mermaid
flowchart LR
  Wire[Bytes arrive] --> Decode[msgpack decode]
  Decode --> Lookup[Service descriptor lookup]
  Lookup --> Validate{"@Validate / @Contract"}
  Validate -- valid --> Body[Method body runs]
  Validate -- invalid --> Err[ValidationError 422]
  Body --> Output[Result]
  Output --> Encode[msgpack encode]
  Err --> Encode
  Encode --> WireOut[Bytes sent]
```

Validation happens **before** the method body, as part of the
Netron call dispatch:

1. Transport receives bytes.
2. msgpack decodes into the call's argument object.
3. `@Validate({ input })` (or a class-level `@Contract`) runs the
   schema against the method's first argument.
4. On success: the parsed (and possibly transformed) value replaces
   the raw input. The method body runs with the trusted value.
5. On failure: a `ValidationError` is sent back to the client. The
   method body does not run.

The schema is **pre-compiled** at decorator time, not on every call.
Validation overhead per call is microseconds for typical schemas.

## What you can express

Zod gives you the full toolkit:

- Primitive types (`string`, `number`, `boolean`, `date`).
- Composite types (`object`, `array`, `tuple`, `union`, `intersection`).
- Refinements (`min`, `max`, `regex`, `email`, custom predicates).
- Transformations (`trim`, `toLowerCase`, custom maps).
- Discriminated unions (`z.discriminatedUnion`).
- Recursive schemas (`z.lazy`).
- Branded types (`.brand<'UserId'>`).

```typescript
const OrderSchema = z.object({
  id:        z.string().uuid(),
  items:     z.array(z.object({
    productId: z.string(),
    quantity:  z.number().int().positive(),
  })).min(1),
  payment:   z.discriminatedUnion('method', [
    z.object({ method: z.literal('card'),    last4: z.string().length(4) }),
    z.object({ method: z.literal('bank'),    iban:  z.string() }),
  ]),
  metadata:  z.record(z.string(), z.unknown()).optional(),
});
```

The TypeScript type `z.infer<typeof OrderSchema>` is the type your
method body sees.

## Validation result options

Behaviour is controlled by the contract's `options`
(`ValidationOptions`). The `mode` field decides how unknown keys are
treated:

```typescript
@Validate({ input: Schema, options: { mode: 'strip' } })        // unknown keys dropped
@Validate({ input: Schema, options: { mode: 'passthrough' } })  // unknown keys preserved
@Validate({ input: Schema, options: { mode: 'strict' } })       // unknown keys rejected
```

Other `ValidationOptions` include `abortEarly`, `coerce`,
`errorFormat` (`'simple' | 'detailed'`), `errorMap`, and
`cacheValidators`. The exact default depends on the schema (a plain
`z.object` strips unknown keys unless `.strict()`/`.passthrough()` is
applied); for production APIs, be explicit.

## Read on

| Page                                                    | When to read                                   |
| ------------------------------------------------------- | ---------------------------------------------- |
| [Contracts](/docs/titan/validation/contracts)           | Per-method input + output schemas as contracts |
| [Error Handling](/docs/titan/validation/error-handling) | What clients see when validation fails         |

---

---
id: "troubleshooting"
title: "FAQ + Troubleshooting"
description: "Common questions and stumbling blocks across the stack."
---

# FAQ + Troubleshooting

The questions and errors that come up most often. Per-module
pages each have an "Anti-patterns" section — this page collects
the cross-cutting ones.

## FAQ

### Why decorators?

Decorators describe **intent** declaratively. The container
reads the metadata and wires the rest. Alternatives (factory
functions, builder APIs, configuration files) move the same
information to a less discoverable place. Decorators put the
contract next to the code it controls.

Trade-off: decorator-based DI requires
`experimentalDecorators` + `emitDecoratorMetadata` in
`tsconfig.json`. Everyone shipping production TypeScript has
this enabled anyway; new projects can use TC39 stage-3
decorators (also supported).

### Why a custom DI container (Nexus)?

InversifyJS, TypeDI, tsyringe — each has its own ergonomics
constraint. Nexus needed:

- Class tokens **and** symbol tokens **and** factory tokens.
- Scopes (singleton, transient, request, scoped).
- **Contextual injection** — one token, different providers per
  request context (multi-tenant, multi-environment).
- Cycle detection at startup.
- Lifecycle hooks tied to dependency order.

No existing container does all five. Nexus does.

### Why MessagePack over JSON for the wire?

RPC must round-trip types exactly. JSON loses Date, Map, Set,
BigInt, Error, Buffer; MessagePack with the package's extensions
preserves all of them — and the wire is \~25% smaller. For
service-to-service calls this is non-negotiable.

For human-readable APIs (debugger, curl), JSON over HTTP still
works via a separate REST gateway.

### Why one daemon, not per-app sidecars?

Sidecar-per-app multiplies operator overhead by N. Omnitron's
single daemon:

- Owns one Unix socket; CLI calls cost nothing.
- Aggregates metrics / logs across all apps in one ring buffer.
- Survives app crashes — the daemon stays up, apps restart.
- Hosts the web console + MCP server at the same address.

For multi-tenant strict isolation, run one daemon per
`~/.omnitron/` home directory.

### Why monorepo?

The stack's value comes from cross-layer typing. A monorepo
with TypeScript references lets the frontend's `useService<T>`
import the backend's `interface T` directly — no codegen, no
publish step.

Polyrepo works (publish the shared interfaces as a package), but
costs an extra round-trip on every contract change.

### Is Titan production-ready?

Yes. The stack runs production workloads at meaningful scale.
The modules ship with anti-patterns docs because production
shaped them.

If "production-ready" means "wide adoption + many StackOverflow
answers" — not yet at the NestJS level. The reference is the
docs you're reading.

### Why isn't there X module?

Because the work-set was deliberate. We shipped the modules
that show up in **every** backend (cache, auth, db, queue,
scheduler, …). For everything else, the [authoring guide](/docs/titan/modules-system/authoring-modules)
describes how to publish a `@your-scope/titan-*` package that
plugs into the same DI grammar.

Domain-specific modules (e-commerce, ML, telecom, …) belong
outside the core to keep the ecosystem focused.

### Can I use Bun or Deno?

- **Apps**: yes, where the runtime matches your dependencies.
  The utility packages (`common`, `cuid`, `eventemitter`,
  `msgpack`) are explicitly cross-runtime tested.
- **Omnitron daemon**: Node only — uses Node's `child_process`
  in a way Bun and Deno don't fully match.
- **Tests**: see [Cross-runtime testing](/docs/testing/cross-runtime).

### How does pricing / licensing work?

MIT across every package. No commercial fork, no enterprise
tier, no support contracts. Use at any scale.

### When should I NOT use this stack?

- **Static sites / marketing pages** — overkill. Use Astro / Next.
- **Single-script automation** — overkill. Use plain TypeScript.
- **Languages other than TS** — the value is end-to-end TS. If
  the team writes Go on the backend, this gives nothing over a
  protobuf pipeline.
- **Edge-only workloads** — Workers / Pages don't run the Node
  process model Titan assumes.
- **You hate decorators** — the DI grammar is decorator-first.
  Alternatives (Hono, Fastify, raw Node) work fine.

## Troubleshooting

### "Cannot resolve token X"

The container couldn't find a provider for token `X`. Causes:

| Cause                                                                       | Fix                                                                 |
| --------------------------------------------------------------------------- | ------------------------------------------------------------------- |
| Module not imported                                                         | Add it to `imports: [...]` of the parent module                     |
| Provider missing                                                            | Add `{provide: TOKEN, useClass: ...}` to `providers: [...]`         |
| Circular dependency                                                         | Restructure or use `@Optional()` + lazy resolution                  |
| Two `@omnitron-dev/titan` versions installed                                | `pnpm why @omnitron-dev/titan` — should report one resolved version |
| Wrong module reference (e.g., importing app `Foo` when service is in `Bar`) | Use `app.resolve(TOKEN)` to introspect, then trace declarations     |

### "Container class identity mismatch"

You see this when launching an app under Omnitron's
module-worker mode:

```
The 'Container' class imported by the app's Titan modules is
not the same physical class as the daemon's 'Container'.
```

Cause: the app pinned a different `@omnitron-dev/titan` version
than the daemon, **or** the workspace has parallel
`node_modules` resolving differently.

Fix: `pnpm dedupe`, then `pnpm why @omnitron-dev/titan` — must
return one resolved version. If not, align peer-dependency
ranges across packages.

### App boots, but first request hangs

Likely a heavy service hasn't initialised yet. Eager-load in
`afterCreate`:

```typescript
hooks: {
  afterCreate: async (app) => {
    await app.container.resolveAsync(HEAVY_SERVICE_TOKEN);
    // ... auth wiring ...
  },
}
```

See [Best practices / Eager-loading heavy services](/docs/omnitron/best-practices#eager-loading-heavy-services).

### Watcher fires constantly in dev

First identify the watcher. Bootstrap applications use esbuild's import graph;
command-based applications use the filesystem watcher. Test files, dependencies
and generated output directories are ignored by the filesystem watcher by
default, including `node_modules`, `dist`, `.turbo`, `coverage` and `.next`.

Additional ignores are **path segments**, not glob expressions:

```typescript
watch: {
  directory: './apps/api',
  ignore: ['generated-assets'],
}
```

Do not use `**/dist/**`: it is not a matching segment. Inspect the changed path in
the log and remove a redundant watcher rather than watching generated output
from a bootstrap application's external packages. See
[shared workspace development](/docs/omnitron/recipes#12-dev-hot-reload-with-shared-workspace-deps).

### Source workspace package fails while reading a bootstrap definition

If a source-exporting package imports a sibling `.js` module that exists only as
TypeScript, plain Node loading is insufficient. The definition reader must
compile that workspace graph with the same policy as the application bundler.
Decorators and parameter properties also require compilation; merely changing
an import extension does not resolve that contract.

Check the active owner as well as the backend with `omnitron backend status`.
A backend handoff preserves the owner's loaded code. Verify the development
source path against the exact owner runtime before starting workloads; keep any
required owner replacement behind a concrete pause and rollback plan. Do not
copy JavaScript files into source directories or substitute a fixed release while
claiming that backend source watching is enabled.

### "Daemon already running"

The PID file at `~/.omnitron/daemon.pid` exists and points to a
live PID. Either:

- Daemon is up: `omnitron status` to confirm; use it.
- Stale lock: previous daemon crashed without cleanup. `omnitron
  kill` removes the lock; then `omnitron up`.

### Heartbeat / discovery flapping

Nodes briefly disappear from `omnitron node check`.

Causes:

- `heartbeatInterval` too short (default 5 s is usually fine).
- Network briefly slow (TLS handshake to managed Redis).
- Redis cluster failover.

Fix: bump `heartbeatTTL` to `5 × heartbeatInterval` (default is
3×). See [titan-discovery / TTL tuning](/docs/titan/modules/discovery#ttl-tuning-guide).

### Build fails with "broken anchor"

Lumen Markdown links use heading anchor IDs derived
from headings. If you link to `#someAnchor` but the heading is
`## Some long heading`, the anchor is `#some-long-heading`.

Fix: match the auto-generated slug. Run the build; the warning
shows the resolved path.

### React component re-renders too often

Most common cause: `useQuery` running every render because args
change identity.

```tsx
// BAD — new array every render → cache miss every time
const { data } = users.getUser.useQuery([id, { include: ['profile'] }]);

// GOOD — memoise the args
const args = useMemo(() => [id, { include: ['profile'] }] as const, [id]);
const { data } = users.getUser.useQuery(args);
```

Or extract the options:

```tsx
const opts = useMemo(() => ({ include: ['profile'] }), []);
const { data } = users.getUser.useQuery([id, opts]);
```

### TypeScript "Type X is not assignable to type Y" after server change

Symptom of the win. The contract is the type — when the server
signature changes, every caller must update.

Fix: update the caller. Don't `as any` to silence — that's
exactly the bug end-to-end types are designed to catch.

### "Maximum call stack exceeded" in middleware

Recursive middleware — usually a middleware calling `next()`
that re-enters the same middleware (forgotten guard, infinite
retry).

Fix: cap retries, ensure `next()` is called exactly once per
invocation.

### Webapp shows ChunkLoadError after deploy

Old HTML cached in user's browser tries to fetch deleted JS chunks.

Fix: catch `ChunkLoadError` in the app shell (e.g. a global
`error`/`unhandledrejection` listener or a React error boundary)
and reload the page once per session so the browser fetches the
new HTML and chunk manifest.

### Redis "ECONNREFUSED" on dev boot

The infrastructure container isn't up. Either:

- `omnitron up` (auto-provisions infra).
- `omnitron infra up` (just infra).
- Check `omnitron infra status` (alias `ps`).

### Database migration locked

Symptom: `MigrationLockError`. Cause: a previous deploy left the
advisory lock held — usually because the migration runner
crashed.

Fix:

```sql
SELECT pg_advisory_unlock_all();
```

Then re-run the migration. Investigate why it crashed before
running again in prod.

### "Health probe times out"

The health endpoint takes longer than `health.timeout`
(default 5 s).

Causes:

- Probe does a real DB query — should use the cached health
  summary.
- One indicator hangs — run `omnitron health-check <app>` for the
  detailed per-probe report and find the slow indicator.

Fix: pre-cache the health result; set per-indicator timeouts.

### "JWT expired" but I just signed in

Clock skew between server and client > `clockTolerance`
(default 5 s).

Fix:

- Sync NTP on both.
- For pathological cases, bump `clockTolerance` to 30 s. Wider
  windows enable replay-after-expiry; don't go above 60 s in
  prod.

### CLI command returns nothing

Use `--json` to see the raw output. Most commands suppress
output if there's nothing to display.

```bash
omnitron --json status
```

### Pull-request fails CI on unrelated test

Check for shared state across tests. Common culprits:

- `vi.useFakeTimers()` not restored — leaks into next test.
- Singleton mock state — `vi.clearAllMocks()` in `beforeEach`.
- DB row left over from a non-rollback test.
- File-system writes outside `tmp/`.

Run failing test in isolation: `vitest run path/to/file.test.ts -t 'test name'`.

## Where to look next

If the question above didn't help:

| Symptom                         | Page                                                                                           |
| ------------------------------- | ---------------------------------------------------------------------------------------------- |
| Module-specific issue           | The module's page → "Anti-patterns" section                                                    |
| RPC / wire issue                | [Netron transports](/docs/frontend/netron/transports) + [errors](/docs/frontend/netron/errors) |
| Auth issue                      | [Auth & RBAC](/docs/omnitron/auth-rbac)                                                        |
| Daemon won't start              | [Daemon](/docs/omnitron/daemon)                                                                |
| Test issue                      | [Testing](/docs/testing)                                                                       |
| Slow / high CPU                 | [Observability](/docs/omnitron/observability)                                                  |
| Infrastructure (Postgres/Redis) | [Infrastructure](/docs/omnitron/infrastructure)                                                |
| Component renders wrong         | [Prism components](/docs/frontend/prism/components)                                            |
| Want to contribute a fix        | [Contributing](/docs/community/contributing)                                                   |

## See also

- [Comparison](/docs/comparison) — context for "why this stack"
- [Best practices](/docs/omnitron/best-practices) — patterns that
  prevent most of the issues above
- [Architecture](/docs/foundations/architecture) — how the layers compose

---

---
id: "tutorial/scaffold"
title: "1. Scaffold"
description: "Empty folder → working monorepo → \"hello world\" Titan app."
---

# Step 1 — Scaffold

By the end of this step: an empty folder turns into a pnpm
workspace with one Titan app saying hello.

## Create the monorepo

```bash
mkdir my-platform && cd my-platform
git init
echo "node_modules/" > .gitignore
echo "dist/" >> .gitignore
echo ".env" >> .gitignore
```

Initialise pnpm workspace:

```bash
pnpm init
```

Edit the generated `package.json`:

```json
{
  "name":    "my-platform",
  "private": true,
  "type":    "module",
  "scripts": {
    "dev":      "pnpm -F api dev",
    "build":    "pnpm -r build",
    "test":     "pnpm -r test"
  },
  "engines": { "node": ">=22" }
}
```

Create `pnpm-workspace.yaml`:

```yaml
packages:
  - 'apps/*'
  - 'packages/*'
```

Create a root `tsconfig.json`:

```json
{
  "compilerOptions": {
    "target":                       "ES2022",
    "module":                       "ESNext",
    "moduleResolution":             "bundler",
    "strict":                       true,
    "esModuleInterop":              true,
    "skipLibCheck":                 true,
    "isolatedModules":              true,
    "experimentalDecorators":       true,
    "emitDecoratorMetadata":        true,
    "useDefineForClassFields":      false,
    "lib":                          ["ES2022"],
    "resolveJsonModule":            true,
    "allowImportingTsExtensions":   false
  }
}
```

The decorator flags are mandatory for Titan.

## Create the first app

```bash
mkdir -p apps/api/src
cd apps/api
```

`apps/api/package.json`:

```json
{
  "name":    "api",
  "private": true,
  "type":    "module",
  "main":    "dist/main.js",
  "scripts": {
    "dev":   "pnpm build && node --watch dist/main.js",
    "build": "tsc -p tsconfig.json",
    "start": "node dist/main.js"
  },
  "dependencies": {
    "@omnitron-dev/titan": "latest"
  },
  "devDependencies": {
    "@types/node": "latest",
    "typescript":  "latest"
  }
}
```

`apps/api/tsconfig.json`:

```json
{
  "extends": "../../tsconfig.json",
  "compilerOptions": {
    "outDir":  "./dist",
    "rootDir": "./src"
  },
  "include": ["src/**/*.ts"]
}
```

## The first Titan app

`apps/api/src/main.ts`:

```typescript
import { Application, Module, Service } from '@omnitron-dev/titan';
import { HttpTransport } from '@omnitron-dev/titan/netron/transport/http';
import { Public } from '@omnitron-dev/titan/decorators';

// 1. A trivial service.
@Service('greetings@1.0.0')
class GreetingsService {
  @Public()
  async hello(name: string): Promise<string> {
    return `Hello, ${name}!`;
  }
}

// 2. Wire it in a module.
@Module({ providers: [GreetingsService] })
class AppModule {}

// 3. Boot.
const app = await Application.create(AppModule);
if (!app.netron) throw new Error('Netron is unavailable');
app.netron.registerTransport('http', () => new HttpTransport());
app.netron.registerTransportServer('http', {
  name: 'api',
  options: { port: 3001, host: '127.0.0.1' },
});
await app.start();
console.log('api ready on http://localhost:3001');

// Application installs graceful shutdown handlers by default.
```

Install + run:

```bash
cd ../..
pnpm install
pnpm dev
```

You should see:

```
api ready on http://localhost:3001
```

The development command compiles decorators with their metadata before
starting Node. In a second terminal, run `pnpm -F api exec tsc --watch`
to recompile edits; Node restarts when the emitted JavaScript changes.

## Verify

In another terminal, from the workspace root:

```bash
pnpm add -wD @omnitron-dev/netron-browser
node --input-type=module <<'JS'
import { createClient } from '@omnitron-dev/netron-browser';
const client = createClient({ url: 'http://127.0.0.1:3001' });
try {
  await client.connect();
  console.log(await client.invoke('greetings@1.0.0', 'hello', ['World']));
} finally {
  await client.disconnect();
}
JS
```

Output:

```text
Hello, World!
```

The service signature flowed through: the server `hello(name:
string): Promise<string>` is callable from anywhere that can
reach `:3001`.

## What just happened

The startup sequence has four explicit responsibilities:

| Your line                                                  | What it triggered                                                                       |
| ---------------------------------------------------------- | --------------------------------------------------------------------------------------- |
| `@Service('greetings@1.0.0')`                              | Registered the class as a Netron RPC service identified by `greetings@1.0.0`            |
| `@Public()`                                                | Exposed the method on the wire (default: anonymous-allowed)                             |
| `@Module({providers: [...]})`                              | Registered the class with the DI container                                              |
| `Application.create` → register transports → `app.start()` | Creates the module graph, then starts lifecycle hooks and opens the registered listener |

The setup provides:

- RPC with an explicit service identity
- ✓ MessagePack wire format
- ✓ DI container
- ✓ Lifecycle management
- ✓ Graceful shutdown wiring

## Commit

```bash
git add .
git commit -m "step 1: scaffold pnpm workspace + first Titan app"
```

## Troubleshooting

| Symptom                                        | Fix                                                                         |
| ---------------------------------------------- | --------------------------------------------------------------------------- |
| `Cannot use import statement outside a module` | Ensure `"type": "module"` in `apps/api/package.json`                        |
| `Decorators are not valid here`                | Check `experimentalDecorators` + `emitDecoratorMetadata` in `tsconfig.json` |
| `EADDRINUSE :::3001`                           | Another process on 3001; choose a free port in the server and client        |
| TypeScript errors on `@Service`                | Run `pnpm install` again; ensure `@omnitron-dev/titan` resolved             |

---

---
id: "tutorial/service"
title: "2. Service with a database"
description: "Replace the hello service with a real one backed by Postgres."
---

# Step 2 — A real service

By the end: a `UsersService` with `findById` / `create` /
`list`, persisting to Postgres via `titan-database`.

## Add the database module

```bash
cd apps/api
pnpm add @omnitron-dev/titan-database @omnitron-dev/titan-redis @omnitron-dev/cuid kysely pg zod
cd ../..
```

`ConfigModule` and `LoggerModule` need no install — they ship
inside `@omnitron-dev/titan`, which you already have, and are
imported from its `./module/config` and `./module/logger`
subpaths. A subpath is not a package: `pnpm add
@omnitron-dev/titan/module/config` asks the registry for a
package by that name and fails.

Spin up Postgres locally:

```bash
docker run -d --name pg -e POSTGRES_PASSWORD=dev -e POSTGRES_DB=platform -p 127.0.0.1:5432:5432 postgres:16-alpine
```

## Schema

`apps/api/src/db/schema.ts`:

```typescript
import type { Generated } from 'kysely';

export interface UsersTable {
  id:        Generated<string>;
  email:     string;
  name:      string;
  createdAt: Generated<Date>;
}

export interface Database {
  users: UsersTable;
}
```

## Migration

`apps/api/migrations/001_users.sql`:

```sql
CREATE TABLE users (
  id         TEXT PRIMARY KEY DEFAULT gen_random_uuid()::text,
  email      TEXT NOT NULL UNIQUE,
  name       TEXT NOT NULL,
  created_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
```

Run it:

```bash
PGPASSWORD=dev psql -U postgres -h localhost -d platform < apps/api/migrations/001_users.sql
```

## Repository

`apps/api/src/users/user.repo.ts`:

```typescript
import { Injectable, Inject } from '@omnitron-dev/titan';
import type { Kysely } from 'kysely';
import { DATABASE_CONNECTION } from '@omnitron-dev/titan-database';
import { cuid } from '@omnitron-dev/cuid';
import type { Database } from '../db/schema.js';

export interface User {
  id:        string;
  email:     string;
  name:      string;
  createdAt: Date;
}

@Injectable()
export class UserRepo {
  constructor(@Inject(DATABASE_CONNECTION) private db: Kysely<Database>) {}

  async findById(id: string): Promise<User | null> {
    const row = await this.db.selectFrom('users')
      .selectAll()
      .where('id', '=', id)
      .executeTakeFirst();
    return row ?? null;
  }

  async create(input: { email: string; name: string }): Promise<User> {
    return await this.db.insertInto('users')
      .values({ id: cuid(), ...input })
      .returningAll()
      .executeTakeFirstOrThrow();
  }

  async list(): Promise<User[]> {
    return await this.db.selectFrom('users')
      .selectAll()
      .orderBy('createdAt', 'desc')
      .limit(100)
      .execute();
  }
}
```

## Service

`apps/api/src/users/users.service.ts`:

```typescript
import { Service } from '@omnitron-dev/titan';
import { Public, Validate } from '@omnitron-dev/titan/decorators';
import { Errors } from '@omnitron-dev/titan/errors';
import { z } from 'zod';
import { UserRepo, type User } from './user.repo.js';

const CreateUserSchema = z.object({
  email: z.string().email(),
  name:  z.string().min(1).max(120),
});
type CreateUser = z.infer<typeof CreateUserSchema>;

@Service('users@1.0.0')
export class UsersService {
  constructor(private readonly repo: UserRepo) {}

  @Public()
  async findById(id: string): Promise<User> {
    const user = await this.repo.findById(id);
    if (!user) throw Errors.notFound('user', id);
    return user;
  }

  @Public()
  @Validate(CreateUserSchema)
  async create(input: CreateUser): Promise<User> {
    return this.repo.create(input);
  }

  @Public()
  async list(): Promise<User[]> {
    return this.repo.list();
  }
}
```

## Wire the module

`apps/api/src/app.module.ts`:

```typescript
import { Module } from '@omnitron-dev/titan';
import { ConfigModule } from '@omnitron-dev/titan/module/config';
import { TitanDatabaseModule } from '@omnitron-dev/titan-database';
import { UserRepo }     from './users/user.repo.js';
import { UsersService } from './users/users.service.js';

@Module({
  imports: [
    ConfigModule.forRoot({
      sources: [{ type: 'env', prefix: 'API_' }],
    }),
    TitanDatabaseModule.forRoot({
      camelCase: true, // created_at in SQL ↔ createdAt in TypeScript
      // `connection` is the whole connection, not just its URL — dialect and
      // DSN travel together, which is what lets `connections: { … }` name
      // several of them.
      connection: {
        dialect: 'postgres',
        connection: process.env['DATABASE_URL'] ?? 'postgres://postgres:dev@localhost:5432/platform',
      },
    }),
  ],
  providers: [UserRepo, UsersService],
})
export class AppModule {}
```

Update `apps/api/src/main.ts`:

```typescript
import { Application } from '@omnitron-dev/titan';
import { HttpTransport } from '@omnitron-dev/titan/netron/transport/http';
import { AppModule }   from './app.module.js';

const app = await Application.create(AppModule);
if (!app.netron) throw new Error('Netron is unavailable');
app.netron.registerTransport('http', () => new HttpTransport());
app.netron.registerTransportServer('http', {
  name: 'api',
  options: { host: '127.0.0.1', port: 3001 },
});
await app.start();
console.log('api ready');

// Application owns signal handling and shutdown.
```

Run + test:

```bash
pnpm dev
```

```bash
node --input-type=module <<'JS'
import { createClient } from '@omnitron-dev/netron-browser';
const c = createClient({ url: 'http://127.0.0.1:3001' });
try {
  await c.connect();

  // Create:
  const u = await c.invoke('users@1.0.0', 'create', [{ email: 'alice@example.com', name: 'Alice' }]);
  console.log('created:', u);

  // Fetch:
  const fetched = await c.invoke('users@1.0.0', 'findById', [u.id]);
  console.log('found:', fetched);

  // List:
  const all = await c.invoke('users@1.0.0', 'list', []);
  console.log('all:', all);
} finally {
  await c.disconnect();
}
JS
```

The round trip exercises the database through the HTTP transport. Use a typed service proxy, as in the [quickstart](/docs/getting-started/quickstart), when calling from TypeScript.

## What changed

| Piece                                               | What it does                                                                                        |
| --------------------------------------------------- | --------------------------------------------------------------------------------------------------- |
| `TitanDatabaseModule.forRoot({...})`                | Bootstraps a Kysely instance against your DB                                                        |
| `@Inject(DATABASE_CONNECTION) db: Kysely<Database>` | DI the typed query builder                                                                          |
| `UserRepo`                                          | Database access layer — repos handle SQL                                                            |
| `UsersService`                                      | Business layer — services compose repos + cross-cutting concerns                                    |
| `@Validate(schema)`                                 | Zod schema runs as input validation; bad input throws `Errors.validation`                           |
| `Errors.notFound(...)`                              | Returns a typed `NOT_FOUND` over the wire — the client sees a `TitanError` with `code: 'NOT_FOUND'` |

Repo + service split is the canonical pattern — easier to test,
clearer responsibilities.

## Commit

```bash
git add .
git commit -m "step 2: real users service with Postgres"
```

## Troubleshooting

| Symptom                             | Fix                                                                 |
| ----------------------------------- | ------------------------------------------------------------------- |
| `ECONNREFUSED 5432`                 | Postgres not running; `docker start pg`                             |
| `password authentication failed`    | Wrong `DATABASE_URL`; check env                                     |
| `relation "users" does not exist`   | Migration didn't run; re-run the psql command                       |
| `Errors.notFound is not a function` | Import path: `import { Errors } from '@omnitron-dev/titan/errors';` |

---

---
id: "tutorial/auth"
title: "3. Auth"
description: "JWT sign-in, sessions, role-gated methods."
---

# Step 3 — Auth

By the end: users can sign in, get a JWT, and the `UsersService.list`
method is admin-only.

## Add the auth module + Redis

```bash
cd apps/api
pnpm add @omnitron-dev/titan-auth
docker run -d --name redis -p 6379:6379 redis:7-alpine
```

## Schema additions

`apps/api/migrations/002_auth.sql`:

```sql
ALTER TABLE users ADD COLUMN password_hash TEXT NOT NULL DEFAULT '';
ALTER TABLE users ADD COLUMN roles TEXT[] NOT NULL DEFAULT ARRAY['user'];

CREATE TABLE sessions (
  id           TEXT PRIMARY KEY,
  user_id      TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE,
  expires_at   TIMESTAMPTZ NOT NULL,
  is_revoked   BOOLEAN NOT NULL DEFAULT FALSE,
  created_at   TIMESTAMPTZ NOT NULL DEFAULT now()
);

CREATE INDEX sessions_user_id_idx ON sessions(user_id);
```

```bash
psql -U postgres -h localhost -d platform < apps/api/migrations/002_auth.sql
```

Update `apps/api/src/db/schema.ts`:

```typescript
import type { Generated } from 'kysely';

export interface UsersTable {
  id:            Generated<string>;
  email:         string;
  name:          string;
  passwordHash:  string;
  roles:         string[];
  createdAt:     Generated<Date>;
}

export interface SessionsTable {
  id:         string;
  userId:     string;
  expiresAt:  Date;
  isRevoked:  Generated<boolean>;
  createdAt:  Generated<Date>;
}

export interface Database {
  users:    UsersTable;
  sessions: SessionsTable;
}
```

## Auth service

`apps/api/src/auth/auth.service.ts`:

```typescript
import { Injectable, Inject, Service } from '@omnitron-dev/titan';
import { Public, Validate } from '@omnitron-dev/titan/decorators';
import { Errors } from '@omnitron-dev/titan/errors';
import { JWT_SERVICE_TOKEN, type IJWTService } from '@omnitron-dev/titan-auth';
import { DATABASE_CONNECTION } from '@omnitron-dev/titan-database';
import type { Kysely } from 'kysely';
import { cuid } from '@omnitron-dev/cuid';
import * as argon2 from 'argon2';
import { z } from 'zod';
import type { Database } from '../db/schema.js';

const SignInSchema = z.object({
  email:    z.string().email(),
  password: z.string().min(8),
});
type SignIn = z.infer<typeof SignInSchema>;

@Service('auth@1.0.0')
export class AuthService {
  constructor(
    @Inject(JWT_SERVICE_TOKEN)    private jwt: IJWTService,
    @Inject(DATABASE_CONNECTION)  private db:  Kysely<Database>,
  ) {}

  @Public({ auth: { allowAnonymous: true } })
  @Validate(SignInSchema)
  async signIn({ email, password }: SignIn) {
    const user = await this.db.selectFrom('users')
      .selectAll()
      .where('email', '=', email)
      .executeTakeFirst();

    if (!user || !(await argon2.verify(user.passwordHash, password))) {
      throw Errors.unauthorized('invalid credentials');
    }

    const sessionId = cuid();
    const expiresAt = new Date(Date.now() + 30 * 24 * 60 * 60 * 1000); // 30 days

    await this.db.insertInto('sessions')
      .values({ id: sessionId, userId: user.id, expiresAt })
      .execute();

    const token = await this.jwt.sign({
      sub:   user.id,
      sid:   sessionId,
      roles: user.roles,
    });

    return { token, user: { id: user.id, email: user.email, roles: user.roles } };
  }

  @Public()
  async signOut({ sessionId }: { sessionId: string }) {
    await this.db.updateTable('sessions')
      .set({ isRevoked: true })
      .where('id', '=', sessionId)
      .execute();
    return { success: true };
  }
}
```

## Wire the module

Update `apps/api/src/app.module.ts`:

```typescript
import { Module }                from '@omnitron-dev/titan';
import { ConfigModule }          from '@omnitron-dev/titan/module/config';
import { TitanDatabaseModule }   from '@omnitron-dev/titan-database';
import { TitanRedisModule }      from '@omnitron-dev/titan-redis';
import { TitanAuthModule }       from '@omnitron-dev/titan-auth';

import { UserRepo }     from './users/user.repo.js';
import { UsersService } from './users/users.service.js';
import { AuthService }  from './auth/auth.service.js';

@Module({
  imports: [
    ConfigModule.forRoot({ sources: [{ type: 'env', prefix: 'API_' }] }),
    TitanRedisModule.forRoot({ config: { url: process.env.REDIS_URL ?? 'redis://localhost:6379' } }),
    TitanDatabaseModule.forRoot({
      // `connection` is the whole connection, not just its URL — dialect and
      // DSN travel together, which is what lets `connections: { … }` name
      // several of them.
      connection: {
        dialect: 'postgres',
        connection: process.env['DATABASE_URL'] ?? 'postgres://postgres:dev@localhost:5432/platform',
      },
    }),
    TitanAuthModule.forRoot({
      algorithm:  'HS256',
      jwtSecret:  process.env.JWT_SECRET ?? 'dev-secret-do-not-use-in-prod',
      issuer:     'my-platform',
      audience:   'my-platform',
    }),
  ],
  providers: [UserRepo, UsersService, AuthService],
})
export class AppModule {}
```

## Gate UsersService.list to admins

Update `apps/api/src/users/users.service.ts`:

```typescript
import { Service } from '@omnitron-dev/titan';
import { Public } from '@omnitron-dev/titan/decorators';
import { RequireAuth, RequireRole } from '@omnitron-dev/titan-auth';
// ...

@Service('users@1.0.0')
export class UsersService {
  // ...

  @Public()
  @RequireAuth()                                        // any logged-in user
  async findById(id: string): Promise<User> {
    // ...
  }

  @Public()
  @RequireRole(['admin'])                                // admin only
  async list(): Promise<User[]> {
    return this.repo.list();
  }
}
```

## Seed an admin user

```bash
node -e "
const argon2 = require('argon2');
const { Pool } = require('pg');
const { cuid } = require('@omnitron-dev/cuid');

(async () => {
  const pool = new Pool({ connectionString: 'postgres://postgres:dev@localhost__LUMEN_COLON__5432/platform' });
  const hash = await argon2.hash('correct-horse-battery-staple');
  await pool.query(
    'INSERT INTO users (id, email, name, password_hash, roles) VALUES (\$1, \$2, \$3, \$4, \$5)',
    [cuid(), 'admin@example.com', 'Admin', hash, ['admin']]
  );
  console.log('admin@example.com / correct-horse-battery-staple');
  await pool.end();
})();
"
```

## Verify

```bash
JWT_SECRET=dev-secret-do-not-use-in-prod pnpm dev
```

```bash
node -e "
const { createClient } = require('@omnitron-dev/netron-browser');
(async () => {
  const c = createClient({ url: 'http://localhost:3001' });
  await c.connect();

  // Sign in:
  const { token, user } = await c.invoke('auth', 'signIn',
    [{ email: 'admin@example.com', password: 'correct-horse-battery-staple' }]);
  console.log('signed in:', user);

  // List (admin-only) with the token:
  const all = await c.invoke('users', 'list', [],
    { headers: { authorization: 'Bearer ' + token } });
  console.log('users:', all);
})();
"
```

Try without the token:

```bash
node -e "
const { createClient } = require('@omnitron-dev/netron-browser');
(async () => {
  const c = createClient({ url: 'http://localhost:3001' });
  await c.connect();
  try {
    await c.invoke('users', 'list', []);
  } catch (e) {
    console.log('rejected:', e.code, e.message);  // → UNAUTHORIZED
  }
})();
"
```

## What's new

| Piece                                         | What it does                            |
| --------------------------------------------- | --------------------------------------- |
| `TitanAuthModule.forRoot({...})`              | Wires JWT signing + verification        |
| `IJWTService`                                 | Inject to sign / verify tokens manually |
| `@RequireAuth()`                              | Method-level: any logged-in user        |
| `@RequireRole(['admin'])`                     | Method-level: specific roles            |
| `@Public({ auth: { allowAnonymous: true } })` | Method-level: explicit anonymous        |

For multi-app fan-out (one identity app + N specialists sharing
the same JWT + Redis session registry), see
[Best practices / Shared auth](/docs/omnitron/best-practices#shared-authentication-across-apps).

## Commit

```bash
git add .
git commit -m "step 3: JWT auth + role-gated methods"
```

## Troubleshooting

| Symptom                         | Fix                                                          |
| ------------------------------- | ------------------------------------------------------------ |
| `Cannot find module 'argon2'`   | `pnpm add argon2` in apps/api                                |
| `JWT_SECRET is required`        | Export it: `export JWT_SECRET=dev-secret-do-not-use-in-prod` |
| `UNAUTHORIZED` with valid token | Check JWT signing key matches verification key               |
| `FORBIDDEN` (403)               | Token is valid but role insufficient                         |

---

---
id: "tutorial/frontend"
title: "4. Frontend"
description: "Vite + React + Prism + netron-react with end-to-end types."
---

# Step 4 — Frontend

By the end: a React app that signs in and lists users using the
backend service contract — no codegen.

## Scaffold the frontend

```bash
cd apps
pnpm create vite web --template react-ts
cd web
pnpm install
```

`apps/web/package.json` — add deps:

```bash
pnpm add @omnitron-dev/netron-browser @omnitron-dev/netron-react @omnitron-dev/prism react-router-dom@7
```

## Share the service types

The frontend imports the backend's service interfaces directly.
Easiest way: a workspace package.

```bash
mkdir -p packages/api-contracts/src
```

`packages/api-contracts/package.json`:

```json
{
  "name":    "@my-platform/api-contracts",
  "type":    "module",
  "main":    "./src/index.ts",
  "types":   "./src/index.ts",
  "private": true
}
```

`packages/api-contracts/src/index.ts`:

```typescript
export interface User {
  id:    string;
  email: string;
  name:  string;
  roles: string[];
}

export interface AuthService {
  signIn(input: { email: string; password: string }): Promise<{
    token: string;
    user:  Pick<User, 'id' | 'email' | 'roles'>;
  }>;
  signOut(input: { sessionId: string }): Promise<{ success: boolean }>;
}

export interface UsersService {
  findById(id: string): Promise<User>;
  list(): Promise<User[]>;
}
```

In `apps/api/`, import these and use as the implementation type:

```typescript
// apps/api/src/users/users.service.ts
import type { UsersService as UsersServiceContract, User } from '@my-platform/api-contracts';

@Service('users@1.0.0')
export class UsersService implements UsersServiceContract {
  // ...
}
```

And in `apps/web/`:

```bash
pnpm add @my-platform/api-contracts@workspace:*
```

## The client

`apps/web/src/client.ts`:

```typescript
import { NetronReactClient } from '@omnitron-dev/netron-react';

export const client = new NetronReactClient({
  url:       import.meta.env.VITE_API_URL ?? 'http://localhost:3001',
  transport: 'http',
  auth: {
    // 'local' | 'session' | 'memory' — not the Web Storage API's names.
    storage: 'local',
  },
});
```

## Providers

`apps/web/src/main.tsx`:

```tsx
import React           from 'react';
import ReactDOM        from 'react-dom/client';
import { BrowserRouter } from 'react-router-dom';
import { PrismProvider } from '@omnitron-dev/prism/core';
import { NetronProvider } from '@omnitron-dev/netron-react';
import { AuthProvider }   from '@omnitron-dev/netron-react/auth';

import { client } from './client.js';
import App        from './App.js';

ReactDOM.createRoot(document.getElementById('root')!).render(
  <React.StrictMode>
    <NetronProvider client={client}>
      <AuthProvider onLogin={(credentials) => client.invoke('auth', 'signIn', [credentials])}>
        <PrismProvider defaultSettings={{ mode: 'dark', primaryColor: '#7c4dff' }}>
          <BrowserRouter>
            <App />
          </BrowserRouter>
        </PrismProvider>
      </AuthProvider>
    </NetronProvider>
  </React.StrictMode>,
);
```

## Sign-in page

`apps/web/src/SignInPage.tsx`:

```tsx
import { useState } from 'react';
import { useAuth }  from '@omnitron-dev/netron-react/auth';
import { useNavigate } from 'react-router-dom';

export function SignInPage() {
  const [email,    setEmail]    = useState('');
  const [password, setPassword] = useState('');
  const [error,    setError]    = useState<string>();
  const navigate = useNavigate();

  const [pending, setPending] = useState(false);
  const { login } = useAuth();

  // `login` runs the AuthProvider's `onLogin` handler (the auth.signIn RPC) and,
  // on success, stores the session via the client internally — no manual token
  // handling needed.
  const onSubmit = async (e: React.FormEvent) => {
    e.preventDefault();
    setError(undefined);
    setPending(true);
    try {
      await login({ email, password });
      navigate('/');
    } catch (err) {
      setError(err instanceof Error ? err.message : String(err));
    } finally {
      setPending(false);
    }
  };

  return (
    <form onSubmit={onSubmit}>
      <h1>Sign in</h1>
      {error && <p style={{ color: 'red' }}>{error}</p>}
      <input type="email"    placeholder="Email"    value={email}    onChange={(e) => setEmail(e.target.value)} />
      <input type="password" placeholder="Password" value={password} onChange={(e) => setPassword(e.target.value)} />
      <button type="submit" disabled={pending}>
        {pending ? 'Signing in…' : 'Sign in'}
      </button>
    </form>
  );
}
```

## Users list page

`apps/web/src/UsersPage.tsx`:

```tsx
import { useService } from '@omnitron-dev/netron-react';
import type { UsersService } from '@my-platform/api-contracts';

export function UsersPage() {
  const users = useService<UsersService>('users');
  const { data, isLoading, error } = users.list.useQuery([]);

  if (isLoading) return <p>Loading…</p>;
  if (error)     return <p style={{ color: 'red' }}>{error.message}</p>;

  return (
    <ul>
      {data?.map(u => (
        <li key={u.id}>
          {u.name} — {u.email} — [{u.roles.join(', ')}]
        </li>
      ))}
    </ul>
  );
}
```

## Routes

`apps/web/src/App.tsx`:

```tsx
import { Routes, Route, Navigate } from 'react-router-dom';
import { AuthGuard, GuestGuard }   from '@omnitron-dev/netron-react/auth';
import { SignInPage } from './SignInPage.js';
import { UsersPage }  from './UsersPage.js';

export default function App() {
  return (
    <Routes>
      <Route path="/sign-in" element={
        <GuestGuard><SignInPage /></GuestGuard>
      } />
      <Route path="/" element={
        <AuthGuard><UsersPage /></AuthGuard>
      } />
      <Route path="*" element={<Navigate to="/" replace />} />
    </Routes>
  );
}
```

## Run

In two terminals:

```bash
# api:
cd apps/api && JWT_SECRET=dev-secret-do-not-use-in-prod pnpm dev

# web:
cd apps/web && pnpm dev
```

Open `http://localhost:5173/`:

1. Redirected to `/sign-in`.
2. Enter `admin@example.com` / `correct-horse-battery-staple`.
3. Redirected to `/` — users list renders.

## Note the type flow

When you call:

```tsx
const users = useService<UsersService>('users');
const { data } = users.list.useQuery([]);
//      ^? User[] | undefined
```

`data` is `User[] | undefined` — typed from the shared contract.
Change `UsersService.list` to return `User[]` instead of
`Pick<User, ...>` on the server, the frontend's `data.email`
access fails the build.

No codegen. No schema sync.

## Verify auth gating

Replace `admin@example.com` with a non-admin user; you'll see
`FORBIDDEN` from the backend, surfaced as `error` in the
`useQuery` hook. The `<AuthGuard>` redirected non-auth requests
back to `/sign-in`.

## Commit

```bash
git add .
git commit -m "step 4: React UI with end-to-end types"
```

## Troubleshooting

| Symptom                        | Fix                                                                       |
| ------------------------------ | ------------------------------------------------------------------------- |
| CORS error                     | Add `cors: true` to api's `netron.http` config                            |
| `data` typed as `unknown`      | Verify `@my-platform/api-contracts` is installed in `apps/web`            |
| `useService` undefined         | Check `<NetronProvider client={client}>` wraps `<App>`                    |
| Sign-in succeeds but list 401s | Auth manager not attaching token; check `AuthProvider` order in providers |

---

---
id: "tutorial/tests"
title: "5. Tests"
description: "Unit / module / integration / E2E coverage for the stack you built."
---

# Step 5 — Tests

By the end: the app has tests at every level of the
[testing pyramid](/docs/testing).

## Add Vitest

```bash
# In each app:
cd apps/api
pnpm add -D vitest @omnitron-dev/testing

cd ../web
pnpm add -D vitest @testing-library/react @testing-library/user-event @testing-library/jest-dom jsdom
```

## Compile backend decorators

Titan's constructor injection needs emitted decorator metadata. Run the
backend tests against TypeScript's compiled output rather than relying on
Vitest's default TypeScript transform.

`apps/api/tsconfig.test.json`:

```json
{
  "extends": "./tsconfig.json",
  "compilerOptions": { "rootDir": ".", "outDir": ".test-build" },
  "include": ["src/**/*.ts", "test/**/*.ts"]
}
```

`apps/api/vitest.config.ts`:

```typescript
import { defineConfig } from 'vitest/config';
export default defineConfig({
  test: { include: ['.test-build/**/*.test.js'] },
});
```

Set the API's test script to
`"test": "tsc -p tsconfig.test.json && vitest run"` and add `.test-build/`
to `.gitignore`. Set `DATABASE_URL` to a disposable database before running
repository or integration tests; the tests below do not roll back their writes.

## Repository integration test (api)

`apps/api/src/users/user.repo.test.ts`:

```typescript
import { describe, it, expect, beforeEach, afterEach } from 'vitest';
import { TestApplication } from '@omnitron-dev/testing/titan';
import { AppModule }       from '../app.module.js';
import { UserRepo }        from './user.repo.js';

describe('UserRepo', () => {
  let app: TestApplication;
  let repo: UserRepo;

  beforeEach(async () => {
    // `create` bootstraps the module graph with logging off and graceful
    // shutdown disabled — test defaults you would otherwise set by hand.
    app = await TestApplication.create(AppModule);
    repo = app.get(UserRepo);
  });

  afterEach(() => app.close());

  it('creates and finds', async () => {
    const email = `test-${crypto.randomUUID()}@example.com`;
    const u = await repo.create({ email, name: 'Alice' });
    expect(u.email).toBe(email);

    const fetched = await repo.findById(u.id);
    expect(fetched).toEqual(u);
  });

  it('returns null on miss', async () => {
    const u = await repo.findById('missing');
    expect(u).toBeNull();
  });
});
```

`TestApplication` has no transaction mode of its own: isolation
is the repository's business. `@omnitron-dev/titan-database`
gives you `withTransaction`, so wrap the body and let it roll
back, or point the test at a schema you drop afterwards —
`app.addCleanup(fn)` runs teardown in `close()`.

## Service test with mocks

`apps/api/src/users/users.service.test.ts`:

```typescript
import { describe, it, expect, vi } from 'vitest';
import { Container } from '@omnitron-dev/titan/nexus';
import { UsersService } from './users.service.js';
import { UserRepo, type User } from './user.repo.js';

describe('UsersService', () => {
  it('throws NOT_FOUND on miss', async () => {
    const container = new Container();
    // `register(token, provider)` — the token is the first argument, not a
    // `provide` key inside the provider. Nexus has no NestJS-style
    // `{ provide, useValue }` single-argument form.
    container.register(UserRepo, {
      useValue: { findById: vi.fn().mockResolvedValue(null) } as any,
    });
    container.register(UsersService, { useClass: UsersService });

    const service = await container.resolveAsync(UsersService);

    await expect(service.findById('missing')).rejects.toMatchObject({
      code: 'NOT_FOUND',
    });
  });
});
```

Pure DI test — no real DB, no real Application.

## Integration test (real api over the wire)

This example exercises the public service from step 2. After adding
authentication in step 3, sign in with a seeded test account and attach
its token before making protected calls. Use a disposable test database.

`apps/api/test/users.integration.test.ts`:

```typescript
import { describe, it, expect, beforeAll, afterAll } from 'vitest';
import { Application } from '@omnitron-dev/titan';
import { HttpTransport } from '@omnitron-dev/titan/netron/transport/http';
import { createClient }    from '@omnitron-dev/netron-browser';
import { AppModule }       from '../src/app.module.js';

describe('users service end-to-end', () => {
  let app:    Application;
  let client: ReturnType<typeof createClient>;

  beforeAll(async () => {
    app = await Application.create(AppModule, { disableGracefulShutdown: true });
    if (!app.netron) throw new Error('Netron is unavailable');
    app.netron.registerTransport('http', () => new HttpTransport());
    app.netron.registerTransportServer('http', {
      name: 'api',
      options: { host: '127.0.0.1', port: 0 },
    });
    await app.start();
    const server = app.netron.transportServers.get('http');
    if (!server) throw new Error('HTTP server did not start');
    const { port } = server as typeof server & { port: number };
    client = createClient({ url: `http://127.0.0.1:${port}` });
    await client.connect();
  });

  afterAll(async () => {
    await client.disconnect();
    await app.stop();
  });

  it('CRUD round-trip', async () => {
    const email = `wire-${crypto.randomUUID()}@example.com`;
    const u = await client.invoke('users@1.0.0', 'create', [{ email, name: 'I' }]);
    expect(u.email).toBe(email);

    const fetched = await client.invoke('users@1.0.0', 'findById', [u.id]);
    expect(fetched).toEqual(u);
  });

  it('rejects missing user with NOT_FOUND', async () => {
    await expect(client.invoke('users@1.0.0', 'findById', ['missing']))
      .rejects.toMatchObject({ code: 'NOT_FOUND' });
  });
});
```

Real wire format, real serialisation, real error round-trip.
Catches a class of bugs unit tests miss.

## Frontend component test

`apps/web/src/UsersPage.test.tsx`:

```tsx
import { describe, it, expect } from 'vitest';
import '@testing-library/jest-dom/vitest';
import { render, screen } from '@testing-library/react';
import { TestNetronProvider } from '@omnitron-dev/netron-react/test';
import { UsersPage } from './UsersPage.js';

describe('<UsersPage>', () => {
  it('renders user list', async () => {
    render(
      <TestNetronProvider
        testConfig={{
          mocks: [
            {
              service:  'users',
              method:   'list',
              response: [
                { id: '1', name: 'Alice', email: 'a@b.c', roles: ['user'] },
                { id: '2', name: 'Bob',   email: 'b@c.d', roles: ['admin'] },
              ],
            },
          ],
        }}
      >
        <UsersPage />
      </TestNetronProvider>
    );

    await screen.findByText('Alice — a@b.c — [user]');
    expect(screen.getByText('Bob — b@c.d — [admin]')).toBeInTheDocument();
  });
});
```

Vitest config for the web app:

`apps/web/vitest.config.ts`:

```typescript
import { defineConfig } from 'vitest/config';
import react from '@vitejs/plugin-react';

export default defineConfig({
  plugins: [react()],
  test:    {
    environment: 'jsdom',
    globals:     false,
  },
});
```

## E2E test (Playwright)

```bash
cd apps/web
pnpm add -D @playwright/test
npx playwright install --with-deps chromium
```

`apps/web/playwright.config.ts`:

```typescript
import { defineConfig } from '@playwright/test';

export default defineConfig({
  testDir: './e2e',
  use: { baseURL: 'http://localhost:5173' },
  webServer: [
    {
      command: 'cd ../api && JWT_SECRET=test-secret pnpm dev',
      url:     'http://localhost:3001/healthz',
      reuseExistingServer: !process.env.CI,
    },
    {
      command: 'pnpm dev',
      url:     'http://localhost:5173',
      reuseExistingServer: !process.env.CI,
    },
  ],
});
```

`apps/web/e2e/sign-in.spec.ts`:

```typescript
import { test, expect } from '@playwright/test';

test('signs in and sees users', async ({ page }) => {
  await page.goto('/');

  // Redirected to sign-in
  await expect(page).toHaveURL(/\/sign-in$/);

  await page.getByPlaceholder('Email').fill('admin@example.com');
  await page.getByPlaceholder('Password').fill('correct-horse-battery-staple');
  await page.getByRole('button', { name: /sign in/i }).click();

  // Lands on home
  await expect(page).toHaveURL(/\/$/);
  await expect(page.getByText('admin@example.com')).toBeVisible();
});
```

Run:

```bash
pnpm exec playwright test
```

## Cross-runtime test (a utility)

If you publish a utility consumed by Bun / Deno users:

`packages/api-contracts/test/cross.test.ts`:

```typescript
import { loadRuntimeAdapter } from '@omnitron-dev/testing';

const t = await loadRuntimeAdapter();

t.test('User type is structural', () => {
  const u = { id: '1', email: 'a@b.c', name: 'A', roles: ['user'] };
  t.expect(u.email).toBe('a@b.c');
});
```

Runs on Node + Bun + Deno via the adapter.

## CI workflow

`.github/workflows/test.yml`:

```yaml
name: test

on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    services:
      postgres:
        image: postgres:16-alpine
        env: { POSTGRES_PASSWORD: dev, POSTGRES_DB: platform_test }
        ports: ['5432:5432']
        options: >-
          --health-cmd "pg_isready -U postgres" --health-interval 10s
          --health-timeout 5s --health-retries 5
      redis:
        image: redis:7-alpine
        ports: ['6379:6379']
    steps:
      - uses: actions/checkout@v4
      - uses: pnpm/action-setup@v3
        with: { version: 9 }
      - uses: actions/setup-node@v4
        with: { node-version: 22, cache: pnpm }
      - run: pnpm install --frozen-lockfile
      - run: pnpm build
      - run: psql -h localhost -U postgres -d platform_test < apps/api/migrations/001_users.sql
        env: { PGPASSWORD: dev }
      - run: psql -h localhost -U postgres -d platform_test < apps/api/migrations/002_auth.sql
        env: { PGPASSWORD: dev }
      - run: pnpm test
        env:
          DATABASE_URL: postgres://postgres:dev@localhost__LUMEN_COLON__5432/platform_test
          REDIS_URL:    redis://localhost:6379
          JWT_SECRET:   test-secret
```

## What we covered

| Layer       | What                                     | Speed          |
| ----------- | ---------------------------------------- | -------------- |
| Unit        | `UsersService.findById` with mocked repo | ms             |
| Repository  | `UserRepo` with a disposable database    | tens of ms     |
| Integration | Full app + Netron client                 | hundreds of ms |
| Component   | `<UsersPage>` with mocked services       | tens of ms     |
| E2E         | Real browser → real api                  | seconds        |

Run the suite:

```bash
pnpm test                  # all packages
pnpm -F api test           # just api
pnpm -F web exec playwright test    # E2E
```

## Commit

```bash
git add .
git commit -m "step 5: tests across the pyramid"
```

## Troubleshooting

| Symptom                                                    | Fix                                                                                                 |
| ---------------------------------------------------------- | --------------------------------------------------------------------------------------------------- |
| `cannot find module '@my-platform/api-contracts'` in tests | Check `pnpm install` ran after adding the workspace dep                                             |
| Vitest hangs                                               | Await client disconnect and application close; release any database connections created by the test |
| Playwright fails to start servers                          | Check `webServer.url` matches reality                                                               |
| `act()` warnings in React tests                            | Use `findBy*` (async) instead of `getBy*` (sync)                                                    |

---

---
id: "tutorial/deploy"
title: "6. Deploy"
description: "Package everything into Docker; ship to a server."
---

# Step 6 — Deploy

By the end: a `docker-compose.yml` running the api + webapp +
Postgres + Redis on a single server, with TLS via nginx.

## Build the api

`apps/api/Dockerfile`:

```dockerfile
FROM node:22-alpine AS build
WORKDIR /app
RUN corepack enable && corepack prepare pnpm@latest --activate
COPY pnpm-lock.yaml pnpm-workspace.yaml package.json ./
COPY apps     apps
COPY packages packages
RUN pnpm install --frozen-lockfile
RUN pnpm -F api build

FROM node:22-alpine
WORKDIR /app
RUN apk add --no-cache tini && corepack enable && corepack prepare pnpm@latest --activate
COPY --from=build /app/node_modules         ./node_modules
COPY --from=build /app/apps/api/dist        ./apps/api/dist
COPY --from=build /app/apps/api/package.json ./apps/api/
COPY --from=build /app/packages              ./packages
COPY --from=build /app/apps/api/migrations  ./apps/api/migrations

RUN addgroup -S app && adduser -S app -G app
USER app

ENV NODE_ENV=production
EXPOSE 3001
ENTRYPOINT ['/sbin/tini', '--']
CMD ['node', '/app/apps/api/dist/main.js']
```

## Build the webapp

`apps/web/Dockerfile`:

```dockerfile
FROM node:22-alpine AS build
WORKDIR /app
RUN corepack enable && corepack prepare pnpm@latest --activate
COPY pnpm-lock.yaml pnpm-workspace.yaml package.json ./
COPY apps     apps
COPY packages packages
RUN pnpm install --frozen-lockfile

ARG VITE_API_URL=https://api.example.com
ENV VITE_API_URL=$VITE_API_URL
RUN pnpm -F web build

FROM nginx:alpine
COPY --from=build /app/apps/web/dist /usr/share/nginx/html
COPY apps/web/nginx.conf /etc/nginx/conf.d/default.conf
EXPOSE 80
```

`apps/web/nginx.conf`:

```nginx
server {
  listen 80 default_server;
  root /usr/share/nginx/html;
  index index.html;

  # SPA fallback
  location / {
    try_files $uri $uri/ /index.html;
  }

  # Static-asset caching
  location ~* \.(?:js|css|woff2?|svg|png|jpg|jpeg|webp)$ {
    expires 1y;
    add_header Cache-Control "public, immutable";
  }
}
```

## docker-compose.yml

```yaml
version: '3.9'

services:
  api:
    build:
      context:    .
      dockerfile: apps/api/Dockerfile
    restart: unless-stopped
    environment:
      NODE_ENV:     production
      DATABASE_URL: postgres://platform:${DB_PASS}@postgres__LUMEN_COLON__5432/platform
      REDIS_URL:    redis://redis:6379
      JWT_SECRET:   ${JWT_SECRET}
      API_LOG_LEVEL: info
    depends_on:
      postgres: { condition: service_healthy }
      redis:    { condition: service_healthy }
      migrate:  { condition: service_completed_successfully }
    healthcheck:
      test:     ['CMD', 'wget', '-qO-', 'http://localhost:3001/healthz']
      interval: 30s
      timeout:  10s
      retries:  3

  migrate:
    build:
      context:    .
      dockerfile: apps/api/Dockerfile
    command: ['sh', '-c', 'cat /app/apps/api/migrations/*.sql | psql ${DATABASE_URL}']
    environment:
      DATABASE_URL: postgres://platform:${DB_PASS}@postgres__LUMEN_COLON__5432/platform
    depends_on:
      postgres: { condition: service_healthy }
    restart: 'no'

  web:
    build:
      context:    .
      dockerfile: apps/web/Dockerfile
      args:
        VITE_API_URL: https://api.example.com
    restart: unless-stopped

  postgres:
    image: postgres:16-alpine
    restart: unless-stopped
    environment:
      POSTGRES_USER:     platform
      POSTGRES_PASSWORD: ${DB_PASS}
      POSTGRES_DB:       platform
    volumes:
      - pg-data:/var/lib/postgresql/data
    healthcheck:
      test:     ['CMD', 'pg_isready', '-U', 'platform']
      interval: 10s
      timeout:  5s
      retries:  5

  redis:
    image: redis:7-alpine
    restart: unless-stopped
    healthcheck:
      test:     ['CMD', 'redis-cli', 'ping']
      interval: 10s
      timeout:  5s
      retries:  5

  caddy:
    image: caddy:alpine
    restart: unless-stopped
    ports:
      - '80:80'
      - '443:443'
    volumes:
      - ./Caddyfile:/etc/caddy/Caddyfile
      - caddy-data:/data
      - caddy-config:/config
    depends_on:
      - api
      - web

volumes:
  pg-data:
  caddy-data:
  caddy-config:
```

## Caddyfile (TLS auto-provisioned)

```
api.example.com {
  reverse_proxy api:3001
  encode gzip
}

app.example.com {
  reverse_proxy web:80
  encode gzip
}
```

Caddy auto-provisions Let's Encrypt certs on first start —
no certbot setup needed.

## .env (gitignored)

```bash
DB_PASS=$(openssl rand -hex 32)
JWT_SECRET=$(openssl rand -hex 48)
```

## Deploy script

```bash
#!/bin/bash
# deploy.sh
set -euo pipefail

SERVER="${1:?usage: deploy.sh user@host}"

echo "→ Building images locally..."
docker compose build

echo "→ Saving images..."
docker save \
  $(docker compose config --images) \
  | gzip > /tmp/images.tar.gz

echo "→ Shipping to $SERVER..."
ssh "$SERVER" 'mkdir -p /srv/platform'
scp /tmp/images.tar.gz docker-compose.yml Caddyfile "$SERVER:/srv/platform/"
scp .env "$SERVER:/srv/platform/.env"

echo "→ Loading + deploying..."
ssh "$SERVER" '
  cd /srv/platform
  docker load < images.tar.gz
  docker compose up -d --remove-orphans
'

echo "✓ Deployed."
```

Use:

```bash
chmod +x deploy.sh
./deploy.sh admin@your-server.example.com
```

## DNS

Point `api.example.com` and `app.example.com` to your server's
IP. Caddy will auto-provision Let's Encrypt certificates on
first request.

## Verify

After \~1 minute (TLS cert acquisition):

```bash
curl https://api.example.com/healthz
# {"status":"healthy"}

open https://app.example.com
# Webapp loads; sign in works.
```

## Updates

```bash
./deploy.sh admin@your-server.example.com
```

Re-runs the deploy. Docker Compose detects new images and
recreates the api/web containers; Postgres + Redis stay up.

For zero-downtime API updates use:

```bash
ssh admin@your-server.example.com 'cd /srv/platform && docker compose exec api kill -USR2 1'
```

If your api had multiple workers (via `instances > 1`), they'd
cycle one-by-one without dropping traffic.

## Backups

Daily Postgres dump:

```bash
# /etc/cron.d/platform-backup
30 2 * * *  admin  docker exec platform-postgres-1 pg_dump -U platform platform | gzip > /backups/platform-$(date +\%Y\%m\%d).sql.gz
```

For object storage of backups, push to S3 (or any S3-compatible)
in a follow-up step.

## Monitoring

- **Logs** — `docker compose logs -f --tail 100`. For real
  aggregation, ship to Loki via the docker logging driver.
- **Metrics** — scrape `api:3001/metrics` from a Prometheus
  container.
- **Uptime** — external ping of `/healthz`.

## What you've shipped

| Layer                   | Implementation                                    |
| ----------------------- | ------------------------------------------------- |
| Backend                 | Titan app with auth + Postgres + Redis            |
| Frontend                | React + Prism + netron-react                      |
| Wire                    | Netron HTTP with MessagePack + JWT                |
| Persistence             | Postgres + Redis                                  |
| Reverse proxy           | Caddy with auto-TLS                               |
| Container orchestration | Docker Compose                                    |
| Tests                   | Unit + integration + E2E with Vitest + Playwright |
| Deploy                  | One-command shell script                          |

## Where to go from here

| Want                  | Where                                                                               |
| --------------------- | ----------------------------------------------------------------------------------- |
| Multi-app supervision | Add `@omnitron-dev/omnitron` daemon — [Omnitron overview](/docs/omnitron/overview)  |
| Web console UI        | [Console](/docs/omnitron/console)                                                   |
| Multi-node fleet      | [Cluster + Fleet](/docs/omnitron/cluster)                                           |
| K8s deployment        | [Kubernetes guide](/docs/deployment/kubernetes)                                     |
| More modules          | [Module catalogue](/docs/titan/modules)                                             |
| Production patterns   | [Best practices](/docs/omnitron/best-practices) + [Recipes](/docs/omnitron/recipes) |

## Commit

```bash
git add .
git commit -m "step 6: docker deploy + caddy TLS"
git tag v0.1.0
```

## Congratulations

Six steps, \~2.5 hours, full-stack production-shaped TypeScript
app with zero codegen. Now go build something real with it.

→ [Reference docs](/docs/intro)

---

---
id: "tutorial/index"
title: "Overview"
description: "A 6-step tutorial — from empty folder to deployed app."
---

# Tutorial — from zero to deployed

A pragmatic 6-step walkthrough. Each step adds one layer of the
stack and ends with a working artefact you can commit.

By the end you'll have:

- A Titan backend with auth + database.
- A React + Prism + netron-react frontend.
- Tests across unit / integration / E2E.
- A production-ready Docker deploy.

## The steps

| Step                                   | What you build                                       | Time   |
| -------------------------------------- | ---------------------------------------------------- | ------ |
| [1. Scaffold](/docs/tutorial/scaffold) | Empty monorepo → first Titan app saying hello        | 10 min |
| [2. Service](/docs/tutorial/service)   | A typed RPC service backed by a real database        | 20 min |
| [3. Auth](/docs/tutorial/auth)         | JWT sign-in, sessions, role-gated methods            | 25 min |
| [4. Frontend](/docs/tutorial/frontend) | React + Prism + netron-react UI calling your service | 30 min |
| [5. Tests](/docs/tutorial/tests)       | Unit / module / integration / E2E coverage           | 25 min |
| [6. Deploy](/docs/tutorial/deploy)     | Docker Compose stack to a server                     | 30 min |

Total: \~2.5 hours from cold start.

## Conventions

- Code blocks are runnable as-is. Copy and replace placeholders
  marked `<like this>`.
- Each step starts from where the previous one ended.
- The final code lives at `apps/api/` + `apps/web/` in your
  monorepo.

## Prerequisites

- Node.js 22+
- pnpm 9+
- Docker (for the deploy step)
- A code editor (VS Code recommended for TypeScript)

## What this tutorial assumes you know

- TypeScript at the level of "I've shipped a real app".
- React at the level of "useState + useEffect".
- A bit of SQL.
- Git basics.

What you **don't** need:

- DI experience (we'll introduce as needed).
- RPC frameworks (Netron will explain itself).
- Decorators (TypeScript-level, picked up by example).

## When you're done

You'll know how to:

- Define a Titan service and expose it over Netron.
- Use the DI container effectively.
- Wire authentication + RLS.
- Build a React UI with end-to-end types.
- Test at every layer.
- Ship to production.

Then deepen with the [reference docs](/docs/intro) — every
module, every API, every recipe.

---

---
id: "utilities/common"
title: "common"
description: "Type predicates, promise helpers, object tools, data structures."
---

# @omnitron-dev/common

```bash
pnpm add @omnitron-dev/common
```

A focused TypeScript utility library. Type predicates with proper
narrowing, promise lifecycle helpers, ergonomic object tools,
small high-performance data structures. Works in Node.js and Bun.

Verified against `packages/common/src/`.

## Modules

| Module        | Surface                                                                                   |
| ------------- | ----------------------------------------------------------------------------------------- |
| `predicates`  | 50+ type predicates (`isString`, `isPlainObject`, `isPromise`, …) + platform flags        |
| `promise`     | `defer`, `delay`, `timeout`, `retry`, `props`, `runExclusive`, `promisify`, `callbackify` |
| `omit`        | Deep / path-based / predicate-based property omission                                     |
| `entries`     | Typed `entries` / `keys` / `values`                                                       |
| `primitives`  | `noop`, `identity`, `truly`, `falsely`, `arrify`                                          |
| `list-buffer` | O(1) unbounded FIFO queue                                                                 |
| `timed-map`   | TTL Map (auto-expiring entries)                                                           |
| `p-limit`     | Promise concurrency limiter                                                               |
| `decimal`     | Lossless decimal arithmetic                                                               |

## Type predicates

50+ predicates with proper `is X` return types that narrow:

```typescript
import {
  isString, isNumber, isBoolean, isArray, isFunction,
  isPlainObject, isObject, isPromise, isError, isDate,
  isBuffer, isNull, isUndefined, isNil, isClass, isAsyncFunction,
} from '@omnitron-dev/common';

function process(value: unknown) {
  if (isString(value))     return value.toUpperCase();   // ← narrowed to string
  if (isPlainObject(value)) return Object.keys(value);   // ← narrowed to object
  if (isPromise(value))    return value.then(handle);    // ← narrowed to Promise
  return value;
}
```

Plus platform flags (set once at import):

```typescript
import { isWindows, linux, darwin, isNodejs } from '@omnitron-dev/common';

if (darwin && isNodejs) {
  // macOS + Node.js — both true / false constants
}
```

## Promise helpers

### `delay(ms, value?, opts?)`

```typescript
await delay(1_000);                                  // sleep 1s
const result = await delay(500, 'hello');           // sleep 500ms then resolve 'hello'
await delay(60_000, undefined, { unref: true });    // doesn't block event loop
```

### `defer()` — externally-controlled promise

```typescript
import { defer } from '@omnitron-dev/common';

const d = defer<User>();

// somewhere else, once the value is known:
d.resolve(currentUser);
// or:
d.reject(new Error('cancelled'));

// wherever it is needed:
const user = await d.promise;
```

Useful when you need to expose `.resolve` / `.reject` to code
that isn't inside the promise's executor.

### `timeout(promise, ms, opts?)`

```typescript
import { timeout } from '@omnitron-dev/common';

const result = await timeout(
  fetchData(),
  5_000,
  { signal: abortController.signal },
);
```

Wraps a promise; rejects with `Error('Timeout of 5000ms exceeded')`
if it doesn't settle in time. Options are `{ unref?, signal? }` —
`signal` lets external code abort early (rejects with
`Error('Timeout aborted')`).

### `retry(callback, opts | max)`

```typescript
import { retry } from '@omnitron-dev/common';

const data = await retry(
  ({ current }) => fetch(url).then(r => r.json()),   // callback gets { current } attempt no.
  {
    max:             5,           // max attempts
    timeout:         8_000,       // per-attempt timeout (ms), optional
    backoffBase:     500,         // base delay (ms), default 100
    backoffExponent: 2,           // multiplier, default 1.1
    match:           [NetworkError, /ECONN/],  // only retry matching errors
    name:            'fetchJson',
    report:          (msg, opts, err) => log(msg),
  },
);

// Shorthand — just a max attempt count:
await retry(() => doThing(), 3);
```

Exponential backoff (`backoffBase * backoffExponent^(attempt-1)`),
optional per-attempt `timeout`, and `match` to restrict which
errors trigger a retry (strings, RegExps, or error classes). Pass a
bare number to set `max` with defaults for the rest.

### `props(obj)`

```typescript
const result = await props({
  user:    fetchUser(id),
  posts:   fetchPosts(id),
  friends: fetchFriends(id),
});
// { user: User, posts: Post[], friends: User[] } — all awaited
```

`Promise.all` for objects — preserves keys.

### `runExclusive(resource, operation)`

```typescript
import { runExclusive } from '@omnitron-dev/common';

const output = await runExclusive(renderer, async () => {
  renderer.configure(options);
  return renderer.render(source);
});
```

Calls sharing the same object run in arrival order; different objects remain
independent. The next operation runs even when its predecessor rejects. This
coordinates mutable library instances inside one JavaScript runtime, as used by
Lumen and Prism for a shared Mermaid engine. Do not await another exclusive call
for that same object from inside its operation: the queue is not reentrant. It
provides neither inter-process locking nor transaction rollback.

### `promisify(fn)` / `callbackify(fn)`

Bridge Node-style callbacks ↔ promises.

```typescript
import { promisify } from '@omnitron-dev/common';
import { exec } from 'node:child_process';

const execAsync = promisify(exec);
const { stdout } = await execAsync('git log');
```

## Object tools

### `omit(obj, keys | predicate | regex | string, opts?)`

```typescript
import { omit } from '@omnitron-dev/common';

omit(user, ['password', 'salt']);                       // remove specific keys
omit(payload, (key, value) => value == null);           // predicate (key, value, object)
omit(state, /^_/);                                       // regex on key
omit(deep, 'user.credentials', { path: true });         // dotted path (requires { path: true })
```

The predicate receives `(key, value, object)`. Dotted-path removal
requires the third-arg flag `{ path: true }`; without it a string is
treated as an exact key name.

### `entries(obj, opts?)` / `keys(obj, opts?)` / `values(obj, opts?)`

```typescript
import { entries, keys } from '@omnitron-dev/common';

entries({ a: 1, b: 2 });                           // [['a', 1], ['b', 2]]
keys(obj, { all: true });                           // include non-enumerable + prototype keys
keys(obj, { followProto: true });                   // walk the prototype chain
```

Options are `{ enumOnly = true, followProto = false, all = false }`.
These operate on plain objects (own/prototype string keys) — they do
**not** iterate `Map`/`Set` entries.

## Data structures

### `ListBuffer<T>` — O(1) FIFO queue

```typescript
import { ListBuffer } from '@omnitron-dev/common';

const buffer = new ListBuffer<LogEntry>();
buffer.push(entry);                  // O(1) enqueue
const oldest = buffer.shift();       // O(1) dequeue (undefined when empty)
buffer.length;                        // current count (getter)
buffer.clear();                       // empty it
```

Backed by a singly-linked list — no array reallocation. It is an
**unbounded** queue: there is no capacity argument and no automatic
eviction; size it by `shift()`-ing as you consume.

### `TimedMap<K, V>` — auto-expiring map

```typescript
import { TimedMap } from '@omnitron-dev/common';

const cache = new TimedMap<string, User>(60_000);      // default 60s TTL per key
cache.set('u_42', user);                               // expires after 60s
cache.set('u_43', user, undefined, 300_000);           // custom 5min TTL (callback, ttl)
cache.get('u_42');                                     // returns User or undefined
```

Constructor is `new TimedMap(timeoutMs?, callback?)`; on expiry the
key is deleted (or `callback(key)` runs if provided). Each entry has
its own `setTimeout` — there is no shared GC sweep and no `.dispose()`.
Pass a per-key `(callback, timeout)` to `set()` to override the default.

### `pLimit(n)` — concurrency limiter

```typescript
import { pLimit } from '@omnitron-dev/common';

const limit = pLimit(5);
const results = await Promise.all(
  urls.map(url => limit(() => fetch(url))),
);
// At most 5 fetches in flight at any moment.
```

### `Decimal` — lossless decimal arithmetic

```typescript
import { Decimal } from '@omnitron-dev/common';

const total = Decimal.from('0.1').add('0.2');    // exact 0.3 (use Decimal.from, not new)
const tax   = Decimal.from(amount).multiplyBy('0.08');
const cents = total.toString();                   // string
const num   = total.toNumber();                   // number (lossy — use sparingly)
```

Construct via the static `Decimal.from(value, precision?)` /
`Decimal.zero()` (the constructor is private). Methods: `add`,
`subtract`, `multiply(n)` / `multiplyBy(other)`, `divide(n)` /
`divideBy(other)`, `abs`, `negate`, comparisons (`equals`, `gt`,
`gte`, `lt`, `lte`, `compare`), `round`/`floor`/`ceil(decimalPlaces)`,
and `toString` / `toNumber` / `toBigInt`.

For money / currency. Never trust `Number` for those.

#### The standalone helpers

The module also exports 31 string-in / string-out functions for fixed-precision
maths. They are the whole surface, listed rather than sampled, because an
incomplete list of money helpers is an invitation to write a thirty-second one
by hand. Every one takes a trailing `precision` argument defaulting to
`DEFAULT_PRECISION` (12).

| Arithmetic                                                           |                                                |
| -------------------------------------------------------------------- | ---------------------------------------------- |
| `addDecimals(a, b)` · `subtractDecimals(a, b)`                       | both operands are strings                      |
| `multiplyDecimal(a, multiplier: number)`                             | scale by a plain number                        |
| `multiplyDecimals(a, b: string)`                                     | multiply two decimal strings                   |
| `divideDecimal(a, divisor: number)`                                  | divide by a plain number; throws on 0          |
| `divideDecimals(a, b: string)`                                       | divide two decimal strings                     |
| `sumDecimals(values: string[])`                                      | sum an array                                   |
| `percentOf(amount, percent)`                                         | `percentOf('100', '2.5')` → `'2.500000000000'` |
| `absDecimal(v)` · `minDecimal(a, b)` · `maxDecimal(a, b)` · `zero()` |                                                |

**Watch the singular/plural pair.** `multiplyDecimal` takes a `number`;
`multiplyDecimals` takes a decimal `string`. Same for `divideDecimal` /
`divideDecimals`. One letter apart and a different second argument — the
compiler catches a swap, an `any` does not.

| Comparison                                                             |                |
| ---------------------------------------------------------------------- | -------------- |
| `compareDecimals(a, b)`                                                | `-1 \| 0 \| 1` |
| `isGreater` · `isGreaterOrEqual` · `isLess` · `isLessOrEqual` `(a, b)` |                |
| `isZero(v)` · `isPositive(v)` · `isNegative(v)`                        |                |

| Rounding and formatting                                           |                                  |
| ----------------------------------------------------------------- | -------------------------------- |
| `roundDecimal(v, decimalPlaces)` · `floorDecimal` · `ceilDecimal` |                                  |
| `formatDecimal(v)`                                                | normalise to the precision       |
| `parseDecimal(v: string \| number)`                               | accepts a number at the boundary |

| Crypto units                                               |   |
| ---------------------------------------------------------- | - |
| `satoshisToBtc(sat: bigint \| number \| string)` → string  |   |
| `btcToSatoshis(btc: string)` → `bigint`                    |   |
| `atomicToXmr(atomic: bigint \| number \| string)` → string |   |
| `xmrToAtomic(xmr: string)` → `bigint`                      |   |

| Validation                                                                                   |                                                                               |
| -------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------- |
| `isValidDecimal(v)`                                                                          | format only                                                                   |
| `validateAmount(amount, { minAmount?, maxAmount?, precision?, allowZero?, allowNegative? })` | returns `{ valid, error? }`; `allowZero` defaults true, `allowNegative` false |

**Results are padded to the precision**, not trimmed: `percentOf('100', '2.5')`
is `'2.500000000000'`, not `'2.5'`. Compare with `compareDecimals` or
`isZero`, never with `===` against a literal you typed by hand.

The unit converters are the exception — they answer in their coin's own
precision, so `satoshisToBtc(100000000n)` is `'1.00000000'` (8 places), not 12.

They return `bigint` in the atomic direction on purpose: satoshis and piconeros
exceed `Number.MAX_SAFE_INTEGER` for large amounts, so a `number` there is the
same lossy trap `Decimal` exists to avoid.

## Native process identity

The Node-only `@omnitron-dev/common/process-identity` entry combines a host boot
identity with the native process birth. Omnitron ownership checks and Lumen
publication recovery use the same implementation. The browser-safe root does not
load native process probes.

```typescript
import {
  readProcessLifetime,
  compareProcessLifetimes,
} from '@omnitron-dev/common/process-identity';

const before = readProcessLifetime(process.pid);
const after = readProcessLifetime(process.pid);
if (before.state === 'alive' && after.state === 'alive') {
  console.log(compareProcessLifetimes(before.identity, after.identity));
}
```

`readProcessLifetime(pid)` returns `alive` with an opaque identity, `dead` when
native observation proves the PID absent, or `unknown` when it cannot establish
either. Comparisons return `same`, `different` or `unknown`. Never interpret an
unknown observation as permission to take another process's resources.

Linux uses the kernel boot UUID and process start ticks. macOS uses the boot
session UUID and UTC native process birth. Legacy local-time births cannot prove
a process replacement from a changed timestamp alone. These probes do not yet
support Windows. Keep policy, cleanup and lifecycle decisions in the caller.

## Primitives

```typescript
import { noop, identity, truly, falsely, arrify } from '@omnitron-dev/common';

noop();                                  // returns undefined
identity(x);                              // returns x
truly();                                  // returns true
falsely();                                // returns false
arrify(value);                            // [value] if not array, else value
arrify(null);                             // []
arrify([1, 2]);                          // [1, 2]
```

`arrify` is the one you'll reach for most — coalescing
"single-or-many" args into a consistent array.

## Use across the stack

- Titan's lifecycle hooks use `delay` for graceful drain.
- Netron's middleware uses `retry` for transient errors.
- Omnitron's daemon uses `TimedMap` for the token cache.
- `titan-cache` ships its own LRU but uses `TimedMap` for
  auxiliary structures.
- `titan-events` uses `defer` for the request/response pattern.

## See also

- [eventemitter](/docs/utilities/eventemitter) — sibling utility, async emission patterns
- [cuid](/docs/utilities/cuid) — ID generation
- [msgpack](/docs/utilities/msgpack) — wire format

---

---
id: "utilities/cuid"
title: "cuid"
description: "Collision-resistant URL-safe unique IDs for distributed systems."
---

# @omnitron-dev/cuid

```bash
pnpm add @omnitron-dev/cuid
```

Fast, collision-resistant unique identifier generator. URL-safe,
monotonically sortable, suitable for distributed systems where
coordination-free ID generation is required.

Verified against `packages/cuid/src/`.

## API

```typescript
import { cuid, isCuid, createOptimizedCuid } from '@omnitron-dev/cuid';

const id = cuid();             // e.g. 'k3j7v0a8z4m1qp9x'
isCuid(id);                    // true

// Custom-length / pre-seeded factory:
const makeId = createOptimizedCuid({ length: 24 });
const longId = makeId();
```

- `cuid()` — generate an ID (default 16 chars).
- `isCuid(id)` — cheap validity check (first char `a-z`, rest `[a-z0-9]`, length 2–32).
- `createOptimizedCuid({ length?, fingerprint?, initialCount? })` — returns a closure
  generating IDs of a custom length / fingerprint / counter seed.

## Properties

| Property            | Value                                                                                                                        |
| ------------------- | ---------------------------------------------------------------------------------------------------------------------------- |
| Length              | 16 characters (default; configurable via `createOptimizedCuid`)                                                              |
| Alphabet            | `[a-z0-9]` (URL-safe, case-insensitive)                                                                                      |
| Time-seeded         | Yes — `Date.now()` + a per-process counter feed the hash, but the output is a SHA3-512 digest, **not** a literal time prefix |
| Collision-resistant | Hash of time + per-call salt + monotonic counter + per-process fingerprint                                                   |
| Predictable         | No — salted hash output                                                                                                      |
| Speed               | \~3M IDs/sec on modern CPUs                                                                                                  |

## Format

```text
k    3j7v0a8z4m1qp9x
↑    ↑
↑    SHA3-512 digest of (time + salt + counter + fingerprint), base36, sliced
random first letter (a–z)
```

The first character is a random `a–z` letter; the remainder is a base36
SHA3-512 digest of the timestamp, a per-call salt, a monotonic counter, and a
per-process fingerprint. There is **no** literal time/counter substring you can
parse back out — sortability is approximate, not lexical.

## When to use cuid

- **Distributed primary keys** — generate on the client, ship to
  server, no coordination round-trip.
- **Public-facing IDs** — short enough for URLs, opaque enough
  not to leak ordering or scale info.
- **Opaque, coordination-free identifiers** — generate anywhere
  without a central allocator and without leaking row counts or
  insertion order.

## When NOT to use cuid

- **Cryptographic randomness** — cuid is collision-resistant but
  not cryptographically random. For session tokens / API keys
  use `crypto.randomBytes`.
- **Lexically time-sortable keys** — the output is a salted hash,
  not a sortable time prefix. Sorting by cuid does **not** give
  chronological order. Use a snowflake/ULID scheme if you need that.
- **Numeric IDs** — cuid is a string. If you need integer
  primary keys, use a snowflake-style scheme.

## Examples

### Generate IDs for entities

```typescript
import { cuid } from '@omnitron-dev/cuid';

interface User {
  id:    string;
  email: string;
  // ...
}

const user: User = {
  id:    cuid(),
  email: 'a@b.c',
};
```

### Request IDs

```typescript
const requestId = cuid();
logger.info({ requestId }, 'incoming request');
```

### Idempotency keys

```typescript
async function chargeOnce(amount: number) {
  const idempotencyKey = cuid();
  return paymentProvider.charge({ amount, idempotencyKey });
}
```

The client generates a key, retries on network error using the
same key — server deduplicates.

### Database default

```typescript
// titan-database / Kysera repository
class UsersTable {
  id:        Generated<string>;   // defaults via cuid()
  email:     string;
  createdAt: Generated<Date>;
}
```

Most repositories default ID columns to cuid; you almost never
generate them manually for inserts.

## Performance

| Scenario                                   | Throughput            |
| ------------------------------------------ | --------------------- |
| Single-thread modern CPU                   | \~3M IDs/sec          |
| Within hot loop (no I/O)                   | \~5M+ IDs/sec         |
| Cross-process (per-process counter resets) | linear, no contention |

The generator is stateless from the caller's view — no
lock contention across calls.

## Where it's used in the stack

- **`titan-database`** — default for repository primary keys.
- **Omnitron** — internal request IDs, process IDs, run IDs.
- **`titan-pm`** — worker pool IDs.
- **`titan-events`** — message IDs.

## See also

- [common](/docs/utilities/common) — sibling utility (predicates, promises, …)
- [titan-database](/docs/titan/modules/database) — primary use case for cuid
- [`crypto.randomUUID`](https://nodejs.org/api/crypto.html#cryptorandomuuidoptions) —
  cryptographic alternative when collision-resistance isn't enough

---

---
id: "utilities/eventemitter"
title: "eventemitter"
description: "Async event emitter with parallel/serial/reduce patterns."
---

# @omnitron-dev/eventemitter

```bash
pnpm add @omnitron-dev/eventemitter
```

An async-first event emitter (custom implementation — no
`eventemitter3` dependency) that works in Node, Bun, and the
browser. Adds parallel/serial/reduce emission patterns on top of a
standard sync emitter, plus an `EnhancedEventEmitter` subclass with
wildcards, history, metrics, and scheduling.

Verified against `packages/eventemitter/src/`.

## Why not just a plain sync emitter?

The Node built-in `EventEmitter` calls handlers **synchronously** in
registration order. That's wrong for two common cases:

1. **Async handlers that need to run in parallel** — synchronous
   emit doesn't await; promises slip through `.emit()`.
2. **Async handlers that produce a derived result** — there's no
   built-in way to reduce.

This package solves both. The standard `on` / `once` / `emit` /
`off` surface is preserved; the async `emit*` methods are additive.

## Quick start

```typescript
import { EventEmitter } from '@omnitron-dev/eventemitter';

const bus = new EventEmitter();

bus.on('user.created', async (user) => {
  await sendWelcomeEmail(user);
});

await bus.emitParallel('user.created', newUser);
```

> The base `EventEmitter` is **not** generic over an event map and
> uses loose `(event, ...args)` typing. For type-safe mapped events
> (`emitTyped`), wildcards, history, and metrics, use
> `EnhancedEventEmitter<TEventMap>` (below).

## Four emission patterns

| Method                            | Behaviour                                                                         | Use case                                           |
| --------------------------------- | --------------------------------------------------------------------------------- | -------------------------------------------------- |
| `emit(event, ...args)`            | Synchronous, returns `boolean`                                                    | Fire-and-forget, sync handlers                     |
| `emitParallel(event, ...args)`    | Awaits all handlers concurrently; resolves to an array of results                 | Independent side-effects (email + log + analytics) |
| `emitSerial(event, ...args)`      | Awaits handlers one-by-one in registration order; resolves to an array of results | Order-sensitive pipelines                          |
| `emitReduce(event, ...args)`      | Chains handlers; each receives the prior return                                   | Building up a result through middleware            |
| `emitReduceRight(event, ...args)` | Like `emitReduce` but reverse order                                               | Right-associative pipelines                        |

### `emitParallel`

```typescript
bus.on('order.placed', async (order) => sendConfirmation(order));
bus.on('order.placed', async (order) => updateInventory(order));
bus.on('order.placed', async (order) => trackAnalytics(order));

await bus.emitParallel('order.placed', order);
// All three handlers run concurrently; resolves when all complete.
// If any throws, the others still run; the error propagates.
```

### `emitSerial`

```typescript
bus.on('migration.run', async () => createTables());
bus.on('migration.run', async () => seedDefaults());
bus.on('migration.run', async () => buildIndexes());

await bus.emitSerial('migration.run');
// Runs in order; awaits each before next.
// If one throws, subsequent handlers don't run.
```

### `emitReduce`

```typescript
bus.on('request.transform', async (req) => ({ ...req, signed: true }));
bus.on('request.transform', async (req) => ({ ...req, compressed: true }));
bus.on('request.transform', async (req) => ({ ...req, retries: 3 }));

const final = await bus.emitReduce('request.transform', initialReq);
// final = { ...initialReq, signed: true, compressed: true, retries: 3 }
```

The seed is the argument(s) you pass after the event name; each
handler receives the **return value** of the previous one as its
argument. Middleware-style.

## Concurrency control

```typescript
const bus = new EventEmitter(3);     // or: bus.setConcurrency(3)

bus.on('image.process', heavyTransform);

// Even if you emit 100 events in a tight loop, at most 3
// instances of heavyTransform run concurrently.
for (const img of images) {
  bus.emitParallel('image.process', img);
}
```

Useful to avoid stampeding handlers that hit limited resources
(database connections, third-party APIs, CPU-bound work).

## Subscription management

```typescript
const unsubscribe = bus.subscribe('user.created', handler);
// later:
unsubscribe();

// Or use once:
bus.once('user.created', handleFirstOnly);
```

`subscribe()` returns an `() => void` — cleaner than `.off()` in
React `useEffect` cleanup.

## Wildcard subscriptions

```typescript
import { EnhancedEventEmitter } from '@omnitron-dev/eventemitter';

const bus = new EnhancedEventEmitter();   // wildcards on by default; delimiter '.'

bus.on('user.*', (data, metadata) => {
  // wildcard handlers receive (data, metadata); the matched event
  // name is in metadata.event
  console.log(`user event: ${metadata.event}`, data);
});

bus.emit('user.created', user);    // matches
bus.emit('user.deleted', user);    // matches
bus.emit('order.placed', order);   // doesn't match
```

`EnhancedEventEmitter` (a subclass of `EventEmitter`) adds wildcard

- namespace handling, plus history, metrics, and scheduling. Its
  constructor takes `{ wildcard?, delimiter?, concurrency? }` —
  wildcards are enabled unless `wildcard: false`. Use `**` to match
  across delimiter levels.

## History

History is built into `EnhancedEventEmitter` — enable it, then
query or replay:

```typescript
import { EnhancedEventEmitter } from '@omnitron-dev/eventemitter';

const bus = new EnhancedEventEmitter();
bus.enableHistory({ maxSize: 1_000, ttl: 60_000 });   // options: { maxSize?, ttl?, filter?, storage? }

bus.emit('user.created', user);

await bus.getHistory({ event: 'user.*' });   // EventFilter: { event?, from?, to?, tags?, correlationId? }
await bus.replay({ event: 'user.created' });
// Re-fires matching events through the bus (useful for debugging / disaster recovery)
```

Backed by `MemoryEventStorage` (a bounded buffer) by default; pass
a custom `storage` implementing the `EventStorage` interface. The
`EventHistory` / `MemoryEventStorage` classes are also exported if
you want to wire history up by hand.

## Metrics

Metrics are likewise built into `EnhancedEventEmitter`:

```typescript
const bus = new EnhancedEventEmitter();
bus.enableMetrics({ slowThreshold: 100, sampleRate: 1 });

// Some time later:
const m = bus.getMetrics();
// {
//   eventsEmitted, eventsFailed,
//   eventCounts:       Map<event, count>,
//   errorCounts:       Map<event, count>,
//   avgProcessingTime: Map<event, ms>,
//   slowestEvents:     Array<{ event, duration }>,
//   listenerCount:     Map<event, count>,
//   memoryUsage,
// }

bus.getMetricsSummary();   // human-readable string
```

Per-event counts, error counts, latency. The underlying collector
class is exported as `MetricsCollector`.

## Scheduled emissions

Scheduling is built into `EnhancedEventEmitter` via `schedule()`:

```typescript
const bus = new EnhancedEventEmitter();

// Fire once in 5 seconds:
bus.schedule('reminder.fire', { userId }, { delay: 5_000 });

// Fire at a specific time:
const id = bus.schedule('report.run', {}, { at: new Date('2026-01-01T00:00:00Z') });

// Cancel:
bus.cancelSchedule(id);
```

`ScheduleOptions` is `{ delay?, at?, cron?, retry?, persistent? }`.
The standalone `EventScheduler` class — `schedule(event, data,
options, emitFn)`, `cancel(id)`, `cancelAll()` — is also exported.

## Where it's used in the stack

| Module                  | What for                                                      |
| ----------------------- | ------------------------------------------------------------- |
| `titan-events`          | Foundation of `EventsService` / `EventBusService`             |
| `titan-telemetry-relay` | Internal bus for telemetry entries (subclass of EventEmitter) |
| Omnitron daemon         | Inter-subsystem signalling                                    |
| `titan-pm`              | Process lifecycle events (`process:crash`, `pool:scaled`, …)  |
| `titan-discovery`       | Node-registry events                                          |

## When to use this vs `titan-events`

| If you…                                                                | Use                                                        |
| ---------------------------------------------------------------------- | ---------------------------------------------------------- |
| Need an in-process emitter in a single class / module                  | This package directly                                      |
| Need a Titan-integrated module with DI, schema validation, persistence | [`titan-events`](/docs/titan/modules/events)               |
| Need cross-process events with at-least-once delivery                  | [`titan-notifications`](/docs/titan/modules/notifications) |

This package is the **primitive**; `titan-events` is the
framework integration around it.

## Performance

| Op                                   | Time                                   |
| ------------------------------------ | -------------------------------------- |
| Single sync `emit()`                 | < 1 μs (Map-backed listener lookup)    |
| `emitParallel` with N async handlers | dominated by the slowest handler       |
| `emitSerial` with N async handlers   | sum of all handlers + Promise overhead |
| Wildcard match                       | O(handlers) per emission (linear scan) |
| Subscribe / unsubscribe              | O(1)                                   |

## See also

- [titan-events](/docs/titan/modules/events) — Titan module built on this
- [common](/docs/utilities/common) — sibling utility (`pLimit` powers the concurrency limiter)

---

---
id: "utilities/index"
title: "Overview"
description: "Five utility packages the rest of the stack is built on."
---

# Utilities

Five small, focused packages that every other package in the
ecosystem depends on. Each is independently versioned, has zero
peer-dependencies on the rest of the stack, and is usable in
non-Omnitron projects.

| Package                                                      | Purpose                                                              | Footprint   |
| ------------------------------------------------------------ | -------------------------------------------------------------------- | ----------- |
| [`@omnitron-dev/common`](/docs/utilities/common)             | Type predicates, promise helpers, object tools, data structures      | \~12 kB gz  |
| [`@omnitron-dev/cuid`](/docs/utilities/cuid)                 | Collision-resistant URL-safe IDs                                     | <1 kB gz    |
| [`@omnitron-dev/eventemitter`](/docs/utilities/eventemitter) | Async event emitter with parallel/serial/reduce patterns             | \~3 kB gz   |
| [`@omnitron-dev/msgpack`](/docs/utilities/msgpack)           | Extensible MessagePack with native JS types + a custom type registry | \~8 kB gz   |
| [`@omnitron-dev/kb`](/docs/utilities/kb)                     | Knowledge-base framework for code intelligence                       | server-side |

## Where they're used

- **`common`** — anywhere primitives are needed. `isPlainObject`,
  `delay`, `omit` show up in every module.
- **`cuid`** — Omnitron's IDs (process IDs, run IDs, request IDs)
  - `titan-database` repository defaults.
- **`eventemitter`** — `titan-events`'s emitter base,
  `titan-telemetry-relay` internal bus, Omnitron daemon's
  scheduler.
- **`msgpack`** — Netron's default wire format. Why your typed
  errors survive the wire.
- **`kb`** — backs `omnitron kb mcp` MCP server + KB index used
  by AI agents.

## Standalone use

Each works without the rest of the stack:

```typescript
import { delay, retry, isPlainObject } from '@omnitron-dev/common';
import { cuid }                          from '@omnitron-dev/cuid';
import { EventEmitter }                  from '@omnitron-dev/eventemitter';
import { encode, decode }                from '@omnitron-dev/msgpack';
```

Pick the one you need; ignore the rest. No transitive Omnitron
runtime gets pulled in.

---

---
id: "utilities/kb"
title: "kb"
description: "Knowledge-base framework — code intelligence + semantic search."
---

# @omnitron-dev/kb

```bash
pnpm add @omnitron-dev/kb
```

A knowledge-base framework for extracting, indexing, and
querying structured intelligence from TypeScript codebases.
Powers Omnitron's MCP server (`omnitron kb mcp`) so agents can
reason about the codebase using the same primitives the rest of
the stack uses.

Verified against `packages/kb/src/`.

## What it does

- **Extracts** structured knowledge (symbols, modules, patterns,
  gotchas, docs) from TypeScript packages.
- **Indexes** the extracts into a hybrid full-text + semantic
  store.
- **Queries** with natural-language questions and precise
  symbol/pattern lookups.

End-to-end: from `.ts` files on disk to "find the canonical way
to do X" answers in milliseconds.

## Architecture

```mermaid
flowchart LR
  TS[TypeScript source files]
  Manifest[kb/kb.config.ts<br/>per-package]
  Extr[Extractors<br/>symbols, specs, gotchas, patterns]
  Embed[Embeddings<br/>optional]
  Store[(SurrealKV store<br/>~/.omnitron/kb.db)]
  Q[Query API]
  CLI[omnitron kb ...]
  MCP[MCP server]

  TS --> Extr
  Manifest --> Extr
  Extr --> Embed
  Extr --> Store
  Embed --> Store
  Store --> Q
  Q --> CLI
  Q --> MCP
```

## Entry types

| Type        | What                                   | Example                                       |
| ----------- | -------------------------------------- | --------------------------------------------- |
| **Symbol**  | Class, interface, type, function, enum | `TitanError`, `IUserService`, `cuid()`        |
| **Module**  | Package or feature module              | `titan-auth`, `omnitron/orchestrator`         |
| **Pattern** | Canonical development recipe           | "service-rpc-module", "preset-infrastructure" |
| **Gotcha**  | Known pitfall + critical warning       | "Don't await inside @PostConstruct"           |
| **Spec**    | Specification document                 | "Payments engine spec v0.2"                   |
| **Doc**     | Long-form markdown                     | README, CONTRIBUTING, design docs             |

Each entry has: id, type, name, summary, body, related (cross-
references), source-location (file + line).

## Per-package configuration — `kb/kb.config.ts`

Each package contributes its own extraction rules from a
`kb/kb.config.ts` file:

```typescript
// packages/titan/kb/kb.config.ts
import { defineKnowledge } from '@omnitron-dev/kb';

export default defineKnowledge({
  module: 'titan',
  name: 'Titan Framework',
  tags: ['di', 'modules', 'lifecycle'],
  extract: {
    symbols: true,                                   // or an array of entry points
    decorators: ['Injectable', 'Module', 'Inject'],
    entryPoints: ['src/index.ts'],
  },
  specs: './specs',
  relationships: {
    extends: ['titan/nexus'],
    integrates: ['titan/netron'],
  },
});
```

`defineKnowledge` fills in sensible defaults (`symbols: true`, a
standard decorator set, `entryPoints: ['src/index.ts']`,
`specs: './specs'`). Discovery walks the workspace for every
`kb/kb.config.ts` and the indexer merges the results into the
central store.

## CLI

```bash
# Build / rebuild the index:
omnitron kb index --full        # ignore cached manifests; reindex everything
omnitron kb index --watch       # incremental + watch for file changes

# Inspect the index:
omnitron kb status

# Test a query (hybrid full-text + semantic):
omnitron kb query "how do I expose a service over Netron with auth"

# Start the MCP server:
omnitron kb mcp                 # stdio MCP for AI agents
```

→ [Omnitron MCP page](/docs/omnitron/mcp) covers the agent
integration in full.

## Programmatic API

```typescript
import { KnowledgeBase } from '@omnitron-dev/kb';
import { SurrealKbStore } from '@omnitron-dev/kb/surreal';   // store lives on a subpath

const store = new SurrealKbStore({ url: 'surrealkv://./.omnitron/kb.db' });
const kb    = new KnowledgeBase({ store, root: process.cwd() });
await kb.initialize();

// Build:
await kb.reindex({ full: false });   // { indexed, skipped }

// Query (question is a positional string; options are optional):
const results = await kb.query(
  'how do I make a service stream events to the browser',
  { maxResults: 10, scope: 'titan' },
);

// Direct lookups:
const api        = await kb.getApi('TitanAuthModule');     // ISymbolDoc | null
const moduleInfo = await kb.getModule('packages/titan-auth');
const gotchas    = await kb.getGotchas('titan-events');    // module is a positional arg
const pattern    = await kb.getPattern('service-rpc-module');
const patterns   = await kb.listPatterns();
const repoMap    = await kb.getRepoMap({ scope: 'titan', detail: 'signatures' });
const stats      = await kb.status();
await kb.close();
```

The return shapes mirror the MCP tool responses — see
[Omnitron MCP](/docs/omnitron/mcp#kb-tools).

## Store backends

| Backend                 | When              | Notes                                 |
| ----------------------- | ----------------- | ------------------------------------- |
| **SurrealKV** (default) | Local single-user | File-based; survives restarts         |
| **SurrealDB** (remote)  | Shared team index | Requires a running SurrealDB instance |
| **In-memory**           | CI / tests        | Volatile                              |

The store interface (`IKbStore`) is pluggable — type-specific
upsert methods plus a hybrid `query`:

```typescript
interface IKbStore {
  initialize(): Promise<void>;
  upsertModule(m: IModuleInfo): Promise<void>;
  upsertSymbols(s: ISymbolDoc[]): Promise<void>;
  upsertSpecs(s: ISpecDoc[]): Promise<void>;
  upsertChunks(c: ICodeChunk[]): Promise<void>;
  upsertGotchas(g: IGotchaDoc[]): Promise<void>;
  upsertPatterns(p: IPatternDoc[]): Promise<void>;
  upsertDependencies(d: IDependency[]): Promise<void>;
  query(question: string, embedding: number[] | null, options: IQueryOptions): Promise<IQueryResult>;
  getSymbol(name: string): Promise<ISymbolDoc | null>;
  getModule(path: string): Promise<IModuleInfo | null>;
  getGotchas(modulePath?: string): Promise<IGotchaDoc[]>;
  getPattern(name: string): Promise<IPatternDoc | null>;
  searchSymbols(query: string, kind?: SymbolKind | SymbolKind[]): Promise<ISymbolDoc[]>;
  getStats(): Promise<IKbStats>;
  close(): Promise<void>;
  // …plus getModuleSpecs / getDependencies / getDependents / manifest helpers
}
```

`SurrealKbStore` (from `@omnitron-dev/kb/surreal`) is the only
shipped implementation; implement `IKbStore` to point kb at any
backing store you prefer.

## Hybrid search

Two indexes per entry:

| Index                   | Driver                                 | Use case                                      |
| ----------------------- | -------------------------------------- | --------------------------------------------- |
| **BM25 full-text**      | SurrealKV native                       | Exact / partial name matches; keyword queries |
| **Semantic embeddings** | Optional (depends on configured model) | Open-ended questions, paraphrases             |

The default query runs both and ranks-combined. Pure-keyword
queries work without embeddings; semantic-aware queries need an
embedding model wired up.

### Embedding model setup

`KnowledgeBase` defaults to `NullEmbeddingProvider` (no semantic
vectors). To enable semantic search, pass a real provider — the
embedding providers live on the `@omnitron-dev/kb/embeddings`
subpath:

```typescript
import { KnowledgeBase } from '@omnitron-dev/kb';
import { OllamaEmbeddingProvider } from '@omnitron-dev/kb/embeddings';

const embeddings = new OllamaEmbeddingProvider({
  model: 'nomic-embed-text',          // local, free; 768-dim by default
  url:   'http://localhost:11434',
});

const kb = new KnowledgeBase({ store, root, embeddings });
```

Shipped providers: `NullEmbeddingProvider` (default, also exported
from the package root), `OllamaEmbeddingProvider`,
`OpenAIEmbeddingProvider`, and `VoyageEmbeddingProvider`. Without a
real provider, semantic search is a no-op — full-text still works.

## Discovery

```typescript
import { KnowledgeDiscovery } from '@omnitron-dev/kb';

const discovery = new KnowledgeDiscovery('/path/to/builtin/specs');
const sources   = await discovery.discover(process.cwd());
// → IKbSource[] — every package with a kb/kb.config.ts, plus built-in specs
```

`KnowledgeBase` runs this internally during `reindex()`. The
constructor takes the path to the built-in cross-cutting specs; the
`discover(workspaceRoot)` method finds every `kb/kb.config.ts` in
the workspace without you listing them manually.

## When you'd use this directly

- **Building a custom MCP server** with project-specific tools
  on top of the standard kb base.
- **Generating documentation** programmatically from KB entries.
- **Internal search UI** — pipe kb queries into a search box.
- **Architectural reports** — query "what depends on titan-auth"
  to generate dependency graphs.

For most users, the `omnitron kb mcp` CLI is enough.

## Performance

| Op                                    | Time                             |
| ------------------------------------- | -------------------------------- |
| Index 1 000 source files (full)       | \~10–30 s                        |
| Incremental reindex of 1 changed file | \~50–200 ms                      |
| Full-text query                       | sub-10 ms                        |
| Semantic query (with embeddings)      | \~50–200 ms cold; sub-50 ms warm |
| MCP server boot (cold)                | \~1–3 s                          |
| MCP tool call (warm)                  | \~10–50 ms                       |

Embedding model is the dominant cost; the rest is near-free.

## Where it sits in the stack

```mermaid
flowchart LR
  Source[packages/**/*.ts]
  Source --> Kb[kb framework<br/>extract + index]
  Kb --> Store[(~/.omnitron/kb.db)]
  Store --> Mcp[omnitron kb mcp<br/>MCP server]
  Store --> CLI[omnitron kb query]
  Mcp --> Agents[Claude, IDE assistants, CI agents]
  CLI --> Operator[Operator at the terminal]
```

The store is the same artefact serving both human and agent
queries — no separate "agent API".

## See also

- [Omnitron / MCP](/docs/omnitron/mcp) — agent integration
  built on top of this package
- [Omnitron / CLI knowledge base](/docs/omnitron/cli#knowledge-base-mcp) —
  `omnitron kb ...` commands
- [common](/docs/utilities/common) — utility primitives kb uses internally

---

---
id: "utilities/msgpack"
title: "msgpack"
description: "Extensible MessagePack with native JS types + custom extensions + streaming."
---

# @omnitron-dev/msgpack

```bash
pnpm add @omnitron-dev/msgpack
```

High-performance MessagePack implementation. Full spec
compliance plus built-in encoders for Date, Map, Set, RegExp,
BigInt, Error, and a custom type registry. Used as the default
wire format for Netron RPC.

Verified against `packages/msgpack/src/`.

## Why MessagePack (not JSON)?

| Property                         |         JSON        |                       MessagePack                      |
| -------------------------------- | :-----------------: | :----------------------------------------------------: |
| Compact binary                   |          ✗          |                            ✓                           |
| Schema-less                      |          ✓          |                            ✓                           |
| Binary data (Buffer/ArrayBuffer) |    base64 string    |                         native                         |
| Date                             |     string (ISO)    |               native (with this package)               |
| Map / Set                        |    not supported    |               native (with this package)               |
| Error with stack                 |   partial (manual)  |               native (with this package)               |
| BigInt                           |        string       |               native (with this package)               |
| Round-trip fidelity              | lossy on edge cases |                          exact                         |
| Throughput                       |   \~500 MB/s parse  |                    \~700 MB/s parse                    |
| Wire size                        |       baseline      | \~25% smaller typical, \~50% for binary-heavy payloads |

For Netron RPC — where a service call's return value should
arrive on the client as the same object — MessagePack with
native types is non-negotiable.

## Quick start

```typescript
import { encode, decode } from '@omnitron-dev/msgpack';

const data = {
  name:    'Alice',
  joined:  new Date(),
  tags:    new Set(['admin', 'pro']),
  roles:   new Map([['users', 'owner'], ['projects', 'editor']]),
  amount:  10n ** 18n,                     // BigInt
};

const buf = encode(data);                  // Uint8Array
const result = decode(buf);

result.joined instanceof Date;             // true
result.tags  instanceof Set;               // true
result.roles instanceof Map;               // true
typeof result.amount === 'bigint';         // true
```

Every native type survives the round-trip unchanged.

## Built-in native types

| Type                       | Ext id     | Notes                                      |
| -------------------------- | ---------- | ------------------------------------------ |
| `Error` (incl. subclasses) | 126        | name + stack + message + custom own-fields |
| `Date`                     | 125        | uint64 ms epoch                            |
| `Map<K, V>`                | 124        | Keys can be any encodable type             |
| `Set<T>`                   | 123        |                                            |
| `RegExp`                   | 121        | Source + flags                             |
| `BigInt`                   | 120        | Arbitrary precision (encoded as string)    |
| `Long`                     | 119        | 64-bit ints via the `long` library         |
| `Buffer` / `Uint8Array`    | bin format | Native — no wrapping                       |

Registered by `registerCommonTypesFor()` on the default `serializer`
at import time, so `encode`/`decode` handle them out of the box.

## Custom extensions

Register your own type on a `Serializer` with a unique id. User
types use **1–99** (100–109 are reserved for Netron, 110–118 for
internal types, 119–126 for the built-ins above; the valid range is
0–127). The `encode` callback **writes into the provided buffer**;
`decode` reads from it:

```typescript
import { Serializer, registerCommonTypesFor } from '@omnitron-dev/msgpack';

class GeoPoint {
  constructor(public lat: number, public lng: number) {}
}

const serializer = new Serializer();
registerCommonTypesFor(serializer);    // keep Date/Map/Set/Error/… support

serializer.register(
  20,                                          // user ext id (1–99)
  GeoPoint,                                    // class constructor
  (point, buf) => {                            // encode: write into buf
    serializer.encode(point.lat, buf);
    serializer.encode(point.lng, buf);
  },
  (buf) => {                                   // decode: read from buf
    const lat = serializer.decode(buf);
    const lng = serializer.decode(buf);
    return new GeoPoint(lat, lng);
  },
);

const data = serializer.encode(new GeoPoint(51.5, -0.12));   // Uint8Array
const point = serializer.decode(data);   // GeoPoint { lat: 51.5, lng: -0.12 }
```

A bare `new Serializer()` starts empty — call `registerCommonTypesFor`
if you also need the native types. The receiver needs the same
registration to decode the type correctly.

## Safe decoding — `tryDecode`

A non-throwing decode wrapper:

```typescript
import { tryDecode } from '@omnitron-dev/msgpack';

const result = tryDecode(buf);
if (result) {
  const { value, bytesConsumed } = result;
  handleMessage(value);
}
// result is null if the buffer could not be decoded
```

`tryDecode`:

- Returns `{ value, bytesConsumed }` on success (`bytesConsumed` is
  the input buffer length).
- Returns `null` on any decode failure (it swallows the error rather
  than throwing).

Note: this is a whole-buffer decode that won't throw — it is **not**
an incremental parser that resumes across partial network chunks.
Netron's transports do their own length-prefixed framing and hand a
complete frame to `decode`.

## Performance

| Benchmark                        |  This package | reference `msgpackr` | JSON.stringify |
| -------------------------------- | :-----------: | :------------------: | :------------: |
| Encode 1 KB object               |    \~12 μs    |        \~10 μs       |     \~30 μs    |
| Decode 1 KB object               |    \~14 μs    |        \~11 μs       |     \~25 μs    |
| Encode 1 MB array                |     \~3 ms    |       \~2.5 ms       |     \~12 ms    |
| Decode 1 MB array                |     \~4 ms    |        \~3 ms        |     \~18 ms    |
| Wire size (typical API response) | \~70% of JSON |     \~70% of JSON    |    baseline    |

Slightly slower than `msgpackr` (which uses extension shortcuts
not in the spec); strictly spec-compliant in exchange.

## Error round-trips

`Error` (and its standard subclasses) are encoded by the built-in
ext-type 126 handler — `name`, `message`, `stack`, and any custom
**own** fields all survive:

```typescript
const err = new TypeError('must be a valid email');
(err as any).field = 'email';

const wire    = encode(err);
const decoded = decode(wire) as any;

decoded instanceof TypeError;   // true — standard error classes reconstruct
decoded.message;                // 'must be a valid email'
decoded.field;                  // 'email' — custom field preserved
decoded.stack;                  // original server-side stack string
```

The server-side stack string is preserved — useful for distributed
debugging. Note the constructor is matched against a small table of
**built-in** error classes (`Error`, `TypeError`, `RangeError`,
`SyntaxError`, `ReferenceError`, `EvalError`, `URIError`); a custom
`class FooError extends Error` round-trips its data and `.name` but
decodes back as a plain `Error`, not as `FooError`. There is no
`registerError` method on `Serializer`.

## SmartBuffer

Internal auto-growing buffer-builder, exported from the
`./smart-buffer` subpath:

```typescript
import { SmartBuffer } from '@omnitron-dev/msgpack/smart-buffer';
import Long from 'long';

const buf = new SmartBuffer();        // optional initialCapacity arg
buf.writeUInt32BE(0xdeadbeef);
buf.writeUInt8(0x2a);
// 64-bit takes `number | Long`, NOT a bigint literal — `123n` does not
// typecheck. Pass a number when the value fits in a double, or a `Long`.
buf.writeInt64BE(123);
buf.writeInt64BE(Long.fromString('9007199254740993'));
const out = buf.toBuffer();

// Reading back:
const reader = SmartBuffer.wrap(out);
reader.readUInt32BE();                // 0xdeadbeef
```

Provides typed `writeXxx` / `readXxx` methods (UInt8/Int8,
16/32-bit BE, 64-bit BE via `long`, float/double) plus `toBuffer()`,
`getRemainingBuffer()`, and the static `SmartBuffer.wrap()` /
`SmartBuffer.fromBuffer()` constructors. Auto-grows; no upfront
sizing needed.

## API surface

```typescript
// Top-level, from '@omnitron-dev/msgpack' (uses the default `serializer`):
function encode(obj: any): Uint8Array;            // enhanced Buffer (has .toBuffer())
function decode(buf: Buffer | SmartBuffer): any;
function tryDecode(buf: any): { value: any; bytesConsumed: number } | null;

const serializer: Serializer;                     // the pre-configured default instance
function registerCommonTypesFor(s: Serializer): void;   // register native types on a Serializer

// Stateful (custom types) — `Serializer` is the default export of the package:
class Serializer {
  constructor(initialCapacity?: number);          // starts EMPTY; call registerCommonTypesFor()
  register(type: number, ctor: any, encode: (obj, buf) => void, decode: (buf) => any): Serializer;
  registerEncoder(type: number, check: (obj) => boolean, encode: (obj) => Buffer): Serializer;
  registerDecoder(type: number, decode: (buf) => any): Serializer;
  encode(x: any): any;                            // → buffer
  encode(x: any, buf: SmartBuffer): void;         // → writes into buf
  decode(buf: Buffer | SmartBuffer): any;
}

// Buffer builder, from '@omnitron-dev/msgpack/smart-buffer':
class SmartBuffer { /* writeXxx / readXxx / toBuffer / static wrap, fromBuffer */ }
```

`Serializer` has no `registerError` or `unregister` method, and there is
no top-level `Encoder` / `Decoder` export.

`tryDecode` is not a method either — but it **does** exist, as a free
function from `@omnitron-dev/msgpack`. It returns
`{ value, bytesConsumed }`, or `null` when the buffer does not yet hold a
complete value, which is what you want when decoding off a stream:

```typescript
import { tryDecode } from '@omnitron-dev/msgpack';

const result = tryDecode(buffer);
if (result) {
  handle(result.value);
  buffer = buffer.subarray(result.bytesConsumed);
}
```

## Where it's used in the stack

- **Netron RPC** — default wire format on all four transports.
- **`titan-events`** — payload serialisation for history persistence.
- **`titan-telemetry-relay`** — WAL on-disk format.
- **`titan-cache`** L2 (Redis) — encoded payload format.

## Spec compliance

Full compliance with [MessagePack spec v5](https://github.com/msgpack/msgpack/blob/master/spec.md).
The native-type registry uses ext type ids in the user-defined
range (0–127); ids ≥ 128 are reserved.

## When to use vs JSON

| Use MessagePack                     | Use JSON                      |
| ----------------------------------- | ----------------------------- |
| RPC where types must round-trip     | Browser-debuggable HTTP APIs  |
| Storage where size matters          | Configuration files           |
| Binary-heavy payloads               | Human-readable logs           |
| Cross-language preserving structure | Web standards (REST, GraphQL) |

For Omnitron itself: MessagePack for the wire, JSON for the
config / human inspection layer.

## See also

- [Netron transports](/docs/frontend/netron/transports) — uses this as default wire format
- [common](/docs/utilities/common) — sibling utility
- [eventemitter](/docs/utilities/eventemitter) — used together in titan-events

---

---
id: "page:/404"
title: "Page not found"
---

# Page not found

The requested route is unavailable. Open the [documentation](/docs/intro), [search](/search), or return [home](/).

---

---
id: "page:/blog/archive"
title: "Archive"
---

# Archive

All published engineering notes, ordered by publication date.

## 2026

- [Welcome to Omnitron](/blog/welcome) — 15 May 2026. The first public documentation preview.

## Current project

- [Release notes](/docs/community/changelog)
- [Compatibility](/docs/reference/compatibility)
- [Contributing](/docs/community/contributing)

---

---
id: "page:/blog/authors/omnitron-team"
title: "Omnitron team"
---

# Omnitron team

Maintainers of the Omnitron stack. [GitHub profile](https://github.com/omnitron-dev).

The contributors building and documenting the Omnitron stack.

## 2026

- [Welcome to Omnitron](/blog/welcome) — 15 May 2026. The first public documentation preview.

## Current project

- [Release notes](/docs/community/changelog)
- [Compatibility](/docs/reference/compatibility)
- [Contributing](/docs/community/contributing)

---

---
id: "page:/blog/authors"
title: "Authors"
---

# Authors

Omnitron documentation is maintained by the project contributors.

## 2026

- [Welcome to Omnitron](/blog/welcome) — 15 May 2026. The first public documentation preview.

## Current project

- [Release notes](/docs/community/changelog)
- [Compatibility](/docs/reference/compatibility)
- [Contributing](/docs/community/contributing)

---

---
id: "page:/blog/tags/intro"
title: "Introduction"
---

# Introduction

Start with an overview of the stack and its documentation.

## 2026

- [Welcome to Omnitron](/blog/welcome) — 15 May 2026. The first public documentation preview.

## Current project

- [Release notes](/docs/community/changelog)
- [Compatibility](/docs/reference/compatibility)
- [Contributing](/docs/community/contributing)

---

---
id: "page:/blog/tags/release"
title: "Releases"
---

# Releases

Announcements describe their publication-time scope. Use compatibility and the current changelog before installing.

## 2026

- [Welcome to Omnitron](/blog/welcome) — 15 May 2026. The first public documentation preview.

## Current project

- [Release notes](/docs/community/changelog)
- [Compatibility](/docs/reference/compatibility)
- [Contributing](/docs/community/contributing)

---

---
id: "page:/blog/tags"
title: "Topics"
---

# Topics

Browse the project introduction and release announcements.

## 2026

- [Welcome to Omnitron](/blog/welcome) — 15 May 2026. The first public documentation preview.

## Current project

- [Release notes](/docs/community/changelog)
- [Compatibility](/docs/reference/compatibility)
- [Contributing](/docs/community/contributing)

## Introduction

[Project introduction](/blog/tags/intro)

## Releases

[Release announcements](/blog/tags/release)

---

---
id: "page:/blog/welcome"
title: "Welcome to Omnitron"
---

# Welcome to Omnitron

Published 15 May 2026 · Omnitron team

The Omnitron documentation site goes live today at
[omnitron.dev](https://omnitron.dev). This is the v0.1 public preview of
the unified TypeScript stack — Titan, Netron, Prism, and the Omnitron
supervisor — collected in one place.

## What's in v0.1

- **Titan core** — DI container, modules, lifecycle, validation.
- **Netron** — RPC plane over HTTP, WebSocket, TCP, Unix.
- **14 Titan modules** — auth, cache, database, discovery, events,
  health, lock, metrics, notifications, pm, ratelimit, redis, scheduler,
  telemetry-relay.
- **Frontend trio** — netron-browser, netron-react, prism.
- **Omnitron app** — supervisor, CLI, web console.

## What's next

- More cookbook examples per module.
- Tutorials for common end-to-end patterns.
- Migration guides from existing Node frameworks.

Read the [introduction](/docs/intro) or jump straight to the
[quickstart](/docs/getting-started/quickstart).

---

---
id: "page:/blog"
title: "Engineering notes"
---

# Engineering notes

Release announcements and implementation notes from the Omnitron project.

## 2026

- [Welcome to Omnitron](/blog/welcome) — 15 May 2026. The first public documentation preview.

## Current project

- [Release notes](/docs/community/changelog)
- [Compatibility](/docs/reference/compatibility)
- [Contributing](/docs/community/contributing)

---

---
id: "page:/docs/category/community"
title: "Community"
---

# Community

Work from reproducible source, report security issues responsibly and follow changes across releases.

- [Contributing](/docs/community/contributing)
- [Security](/docs/community/security)
- [Changelog](/docs/community/changelog)

---

---
id: "page:/docs/category/foundations"
title: "Foundations"
---

# Foundations

Understand the boundaries between runtime, transport, interface and operations before composing them.

- [Principles](/docs/foundations/principles)
- [Architecture](/docs/foundations/architecture)
- [Composition](/docs/foundations/composition)
- [Titan vs omnitron](/docs/foundations/titan-vs-omnitron)
- [Monorepo](/docs/foundations/monorepo)

---

---
id: "page:/docs/category/getting-started"
title: "Getting Started"
---

# Getting Started

Choose a path from installation to your first running service. The developer map connects each task to its implementation guide.

- [Installation](/docs/getting-started/installation)
- [Quickstart](/docs/getting-started/quickstart)
- [Developer map](/docs/getting-started/developer-map)
- [Project structure](/docs/getting-started/project-structure)

---

---
id: "page:/docs/titan/best-practices"
title: "Best Practices"
---

# Best Practices

Follow these guides in order, or open the reference for the part of your application you are changing.

- [Structuring services](/docs/titan/best-practices/structuring-services)
- [Error handling](/docs/titan/best-practices/error-handling)
- [Observability](/docs/titan/best-practices/observability)
- [Performance](/docs/titan/best-practices/performance)

---

---
id: "page:/docs/titan/concepts"
title: "Concepts"
---

# Concepts

Follow these guides in order, or open the reference for the part of your application you are changing.

- [Design principles](/docs/titan/concepts/design-principles)
- [Architecture](/docs/titan/concepts/architecture)
- [Mental model](/docs/titan/concepts/mental-model)

---

---
id: "page:/docs/titan/configuration"
title: "Configuration"
---

# Configuration

Follow these guides in order, or open the reference for the part of your application you are changing.

- [Overview](/docs/titan/configuration/overview)
- [Sources](/docs/titan/configuration/sources)
- [Validation](/docs/titan/configuration/validation)
- [Hot reload](/docs/titan/configuration/hot-reload)

---

---
id: "page:/docs/titan/di"
title: "Dependency Injection"
---

# Dependency Injection

Follow these guides in order, or open the reference for the part of your application you are changing.

- [Overview](/docs/titan/di/overview)
- [Providers](/docs/titan/di/providers)
- [Scopes](/docs/titan/di/scopes)
- [Tokens](/docs/titan/di/tokens)
- [Multi injection](/docs/titan/di/multi-injection)
- [Contextual injection](/docs/titan/di/contextual-injection)
- [Middleware](/docs/titan/di/middleware)
- [Circular dependencies](/docs/titan/di/circular-dependencies)
- [Devtools](/docs/titan/di/devtools)

---

---
id: "page:/docs/titan/errors"
title: "Errors"
---

# Errors

Follow these guides in order, or open the reference for the part of your application you are changing.

- [Overview](/docs/titan/errors/overview)
- [Hierarchy](/docs/titan/errors/hierarchy)
- [Factories](/docs/titan/errors/factories)
- [Classification](/docs/titan/errors/classification)

---

---
id: "page:/docs/titan/logging"
title: "Logging"
---

# Logging

Follow these guides in order, or open the reference for the part of your application you are changing.

- [Overview](/docs/titan/logging/overview)
- [Transports](/docs/titan/logging/transports)
- [Processors](/docs/titan/logging/processors)
- [Child loggers](/docs/titan/logging/child-loggers)

---

---
id: "page:/docs/titan/modules-system"
title: "Module System"
---

# Module System

Follow these guides in order, or open the reference for the part of your application you are changing.

- [Defining modules](/docs/titan/modules-system/defining-modules)
- [Dynamic modules](/docs/titan/modules-system/dynamic-modules)
- [Module discovery](/docs/titan/modules-system/module-discovery)
- [Authoring modules](/docs/titan/modules-system/authoring-modules)

---

---
id: "page:/docs/titan/resilience"
title: "Resilience"
---

# Resilience

Follow these guides in order, or open the reference for the part of your application you are changing.

- [Overview](/docs/titan/resilience/overview)
- [Retry](/docs/titan/resilience/retry)
- [Circuit breaker](/docs/titan/resilience/circuit-breaker)
- [Timeout](/docs/titan/resilience/timeout)

---

---
id: "page:/docs/titan/testing"
title: "Testing"
---

# Testing

Follow these guides in order, or open the reference for the part of your application you are changing.

- [Overview](/docs/titan/testing/overview)
- [Di overrides](/docs/titan/testing/di-overrides)
- [Integration](/docs/titan/testing/integration)
- [Modules](/docs/titan/testing/modules)

---

---
id: "page:/docs/titan/validation"
title: "Validation"
---

# Validation

Follow these guides in order, or open the reference for the part of your application you are changing.

- [Overview](/docs/titan/validation/overview)
- [Contracts](/docs/titan/validation/contracts)
- [Error handling](/docs/titan/validation/error-handling)

---

---
id: "page:/"
title: "Own the whole system"
---

---

---
id: "page:/search"
title: "Search the documentation"
---

# Search the documentation

Search runs locally against this build. To look up an exported symbol, open the [API index](/docs/reference/api-index).