@omnitron-dev/msgpack

pnpm add @omnitron-dev/msgpack

High-performance MessagePack implementation. Full spec compliance plus built-in encoders for Date, Map, Set, RegExp, BigInt, Error, and a custom type registry. Used as the default wire format for Netron RPC.

Verified against packages/msgpack/src/.

Why MessagePack (not JSON)?

For Netron RPC — where a service call's return value should arrive on the client as the same object — MessagePack with native types is non-negotiable.

Quick start

import { encode, decode } from '@omnitron-dev/msgpack';

const data = {
  name:    'Alice',
  joined:  new Date(),
  tags:    new Set(['admin', 'pro']),
  roles:   new Map([['users', 'owner'], ['projects', 'editor']]),
  amount:  10n ** 18n,                     // BigInt
};

const buf = encode(data);                  // Uint8Array
const result = decode(buf);

result.joined instanceof Date;             // true
result.tags  instanceof Set;               // true
result.roles instanceof Map;               // true
typeof result.amount === 'bigint';         // true

Every native type survives the round-trip unchanged.

Built-in native types

Registered by registerCommonTypesFor() on the default serializer at import time, so encode/decode handle them out of the box.

Custom extensions

Register your own type on a Serializer with a unique id. User types use 1–99 (100–109 are reserved for Netron, 110–118 for internal types, 119–126 for the built-ins above; the valid range is 0–127). The encode callback writes into the provided buffer; decode reads from it:

import { Serializer, registerCommonTypesFor } from '@omnitron-dev/msgpack';

class GeoPoint {
  constructor(public lat: number, public lng: number) {}
}

const serializer = new Serializer();
registerCommonTypesFor(serializer);    // keep Date/Map/Set/Error/… support

serializer.register(
  20,                                          // user ext id (1–99)
  GeoPoint,                                    // class constructor
  (point, buf) => {                            // encode: write into buf
    serializer.encode(point.lat, buf);
    serializer.encode(point.lng, buf);
  },
  (buf) => {                                   // decode: read from buf
    const lat = serializer.decode(buf);
    const lng = serializer.decode(buf);
    return new GeoPoint(lat, lng);
  },
);

const data = serializer.encode(new GeoPoint(51.5, -0.12));   // Uint8Array
const point = serializer.decode(data);   // GeoPoint { lat: 51.5, lng: -0.12 }

A bare new Serializer() starts empty — call registerCommonTypesFor if you also need the native types. The receiver needs the same registration to decode the type correctly.

Safe decoding — tryDecode

A non-throwing decode wrapper:

import { tryDecode } from '@omnitron-dev/msgpack';

const result = tryDecode(buf);
if (result) {
  const { value, bytesConsumed } = result;
  handleMessage(value);
}
// result is null if the buffer could not be decoded

tryDecode:

• Returns { value, bytesConsumed } on success (bytesConsumed is the input buffer length).

• Returns null on any decode failure (it swallows the error rather than throwing).

Note: this is a whole-buffer decode that won't throw — it is not an incremental parser that resumes across partial network chunks. Netron's transports do their own length-prefixed framing and hand a complete frame to decode.

Performance

Slightly slower than msgpackr (which uses extension shortcuts not in the spec); strictly spec-compliant in exchange.

Error round-trips

Error (and its standard subclasses) are encoded by the built-in ext-type 126 handler — name, message, stack, and any custom own fields all survive:

const err = new TypeError('must be a valid email');
(err as any).field = 'email';

const wire    = encode(err);
const decoded = decode(wire) as any;

decoded instanceof TypeError;   // true — standard error classes reconstruct
decoded.message;                // 'must be a valid email'
decoded.field;                  // 'email' — custom field preserved
decoded.stack;                  // original server-side stack string

The server-side stack string is preserved — useful for distributed debugging. Note the constructor is matched against a small table of built-in error classes (Error, TypeError, RangeError, SyntaxError, ReferenceError, EvalError, URIError); a custom class FooError extends Error round-trips its data and .name but decodes back as a plain Error, not as FooError. There is no registerError method on Serializer.

SmartBuffer

Internal auto-growing buffer-builder, exported from the ./smart-buffer subpath:

import { SmartBuffer } from '@omnitron-dev/msgpack/smart-buffer';
import Long from 'long';

const buf = new SmartBuffer();        // optional initialCapacity arg
buf.writeUInt32BE(0xdeadbeef);
buf.writeUInt8(0x2a);
// 64-bit takes `number | Long`, NOT a bigint literal — `123n` does not
// typecheck. Pass a number when the value fits in a double, or a `Long`.
buf.writeInt64BE(123);
buf.writeInt64BE(Long.fromString('9007199254740993'));
const out = buf.toBuffer();

// Reading back:
const reader = SmartBuffer.wrap(out);
reader.readUInt32BE();                // 0xdeadbeef

Provides typed writeXxx / readXxx methods (UInt8/Int8, 16/32-bit BE, 64-bit BE via long, float/double) plus toBuffer(), getRemainingBuffer(), and the static SmartBuffer.wrap() / SmartBuffer.fromBuffer() constructors. Auto-grows; no upfront sizing needed.

API surface

// Top-level, from '@omnitron-dev/msgpack' (uses the default `serializer`):
function encode(obj: any): Uint8Array;            // enhanced Buffer (has .toBuffer())
function decode(buf: Buffer | SmartBuffer): any;
function tryDecode(buf: any): { value: any; bytesConsumed: number } | null;

const serializer: Serializer;                     // the pre-configured default instance
function registerCommonTypesFor(s: Serializer): void;   // register native types on a Serializer

// Stateful (custom types) — `Serializer` is the default export of the package:
class Serializer {
  constructor(initialCapacity?: number);          // starts EMPTY; call registerCommonTypesFor()
  register(type: number, ctor: any, encode: (obj, buf) => void, decode: (buf) => any): Serializer;
  registerEncoder(type: number, check: (obj) => boolean, encode: (obj) => Buffer): Serializer;
  registerDecoder(type: number, decode: (buf) => any): Serializer;
  encode(x: any): any;                            // → buffer
  encode(x: any, buf: SmartBuffer): void;         // → writes into buf
  decode(buf: Buffer | SmartBuffer): any;
}

// Buffer builder, from '@omnitron-dev/msgpack/smart-buffer':
class SmartBuffer { /* writeXxx / readXxx / toBuffer / static wrap, fromBuffer */ }

Serializer has no registerError or unregister method, and there is no top-level Encoder / Decoder export.

tryDecode is not a method either — but it does exist, as a free function from @omnitron-dev/msgpack. It returns { value, bytesConsumed }, or null when the buffer does not yet hold a complete value, which is what you want when decoding off a stream:

import { tryDecode } from '@omnitron-dev/msgpack';

const result = tryDecode(buffer);
if (result) {
  handle(result.value);
  buffer = buffer.subarray(result.bytesConsumed);
}

Where it's used in the stack

• Netron RPC — default wire format on all four transports.

• titan-events — payload serialisation for history persistence.

• titan-telemetry-relay — WAL on-disk format.

• titan-cache L2 (Redis) — encoded payload format.

Spec compliance

Full compliance with MessagePack spec v5. The native-type registry uses ext type ids in the user-defined range (0–127); ids ≥ 128 are reserved.

When to use vs JSON

For Omnitron itself: MessagePack for the wire, JSON for the config / human inspection layer.

See also

• Netron transports — uses this as default wire format

• common — sibling utility

• eventemitter — used together in titan-events