Concepts

Live subscriptions

A dabadata live subscription is a query that keeps itself up to date. One WebSocket carries all of a client's subscriptions; the server sends the matching rows once, then every change that the subscriber is allowed to see.

Connecting

Open a WebSocket to /v1/live, with the token as a query parameter. On a hosted project, add the environment's publishable key as ?key=:

wss://dabadata.com/v1/live?key=pk_live_EXAMPLEexample0000000000&token=<user token>
ws://localhost:4600/v1/live?token=<user token>   # a self-hosted server's default project: no key

• A user token subscribes under that user's read rules.
• No token is an anonymous socket: it may subscribe to tables whose read rule lets anonymous callers in, such as "anyone".
• A view token (or the project key) reads with a bypass and also receives every new trace row and every WordPress mirror event.
• A token that matches nothing is refused with a 401 before the upgrade.
• A secret key in the URL (?token=sk_... or ?key=sk_...) is refused with 401 { "error": "unauthorized", "message": "secret keys do not open sockets; use a user token" }. Keys never go in a URL: proxies and logs keep URLs. A backend that needs to hear about changes uses HTTP instead.
• On a hosted project, the key is what places an anonymous socket, or one opened with a provider JWT, in your environment; a user or view token dabadata minted finds its environment without it. A key and a token of different environments are a 401.

A hosted project can also refuse the upgrade with 403 origin_not_allowed (the page's origin is not in its allowed origins), 403 plan_limit (the workspace has all its connections open), 410 (the project is being deleted) or 429 (the key's request rate is used up). A browser sees none of these bodies: the socket just closes before it opens, and the client tries again later.

The server greets every socket with { "type": "hello", "project": "default", "projectId": null, "env": "live" }; on a hosted project project is the environment's name and projectId the project id, such as "prj_example12345". Send { "type": "ping" } at any time to get { "type": "pong" } back.

Messages

Every message is one JSON object with a type. Subscription messages carry the id you chose when subscribing, so one socket can hold many.

Subscribe

{ "type": "subscribe", "id": "s1", "table": "tasks", "where": { "done": false } }

The server answers with every row the subscriber may read, filtered by the read rule and by where (newest first, up to 1000):

{ "type": "rows", "id": "s1", "rows": [ { "id": "5f1c...", "title": "Order chairs", "done": false } ] }

Then, for every mutation in that table, a change if the subscriber may see it. A change that lands while the first query is still running is held back and sent right after the rows message, so nothing falls in the gap.

where is a flat equality map on declared columns, the same as for list reads. It applies to changes too, which matters for updates (below). The subscription is recorded in the trace with op: "subscribe" and source: "ws".

What a change looks like

A row "passes" for a subscription when the subscriber's read rule allows it and it matches the subscription's where. For each mutation the server compares the row before and after, each judged at the instant of the change:

Two consequences worth handling in your code:

• An update can arrive for a row you have never seen - for example a task that was just marked done: false on a where: { done: false } subscription. Treat update as "insert or replace".
• A delete may carry only the id. Remove by id.

{ "type": "change", "id": "s1", "op": "update", "row": { "id": "5f1c...", "title": "Order chairs", "done": true } }
{ "type": "change", "id": "s1", "op": "delete", "row": { "id": "5f1c..." } }

What each subscriber sees

Every subscriber gets rows shaped for them, from the first rows to every change and every reset:

• Their rows. The read rule is evaluated for that subscriber - their role, their id, row conditions at the instant of the change.
• Their columns. A column whose read rule fails for the subscriber on a row is left out of that row (the key is absent).
• Only what they can observe. An update whose before and after rows are both visible to a subscriber and project to the same row (ignoring _seq and updated_at, which every write moves) is not sent to that subscriber: a change of columns they cannot see is not observable, not even through updated_at.
• Replace, do not merge. An update carries the whole visible row. A key missing from the new row is gone - a column that has just become hidden must disappear on the client too.

The secret key, view tokens and the console see every row and column, and get every change, as before.

Changes without a write

With a condition on $now or $today - in a read rule, a column read rule, or a view - a row can enter or leave a subscriber's view, or a column can appear, when nobody writes anything. The server keeps one timer for all of these instants and sends:

They arrive within about a second of the instant (each table is checked at most every 250 ms). These pushes write no trace row; each pass that reached someone writes one realtime-log row, fanout with detail time. A pass that finds more than 5000 rows on its instants sends the readers their rows again instead (reason: "overflow"). Subscribers whose read does not depend on time - a role-level rule, the secret key, view tokens, the console - get no time messages.

For the same reason, GET /v1/db/<table>?since= for a caller whose read depends on time answers { "reset": true, "cursor" } with no rows: load the rows again. See catching up.

Rows again: resets

The server may send rows again for an open subscription at any time. Replace what you hold, exactly as after a reconnect:

{ "type": "rows", "id": "s1", "rows": [ ... ], "reason": "definition" }

The first rows of a subscription has no reason. Changes that arrive while a reset loads wait and follow its rows. When a definition change leaves a subscription's table gone or no longer readable - including a table that just became strict - the subscriber gets { "type": "error", "id", ... } instead, and the subscription ends.

Subscribing to a view

A subscription may name a declared view instead of a table:

{ "type": "subscribe", "id": "s2", "view": "open_questions", "params": { "round": "r1" } }

The first rows holds the view's rows, in the subscriber's shape. Then every committed change of the view's base table goes through the view: an insert or update that passes the view's rows (and the subscription's where) arrives as the shaped row, computed fields evaluated at the change's instant; one that stops passing arrives as a delete. Computed fields and conditions with $now follow the clock as above. Exactly one of table and view may be given. In the client: db.subscribe({ view: "open_questions", params: { round } }, {}, handlers).

Unsubscribe

{ "type": "unsubscribe", "id": "s1" }

No reply is sent. Closing the socket ends all of its subscriptions.

Errors

When the read rule leaves nothing the subscriber could ever see, the subscription is refused:

{ "type": "error", "id": "s1", "error": "forbidden", "explanation": "read tasks: denied, anonymous is not authenticated" }

A strict table is refused for end users the same way, with "explanation": "read questions: denied, questions is read through views only (access: views)". Other problems - an unknown table, a where on a column that does not exist or that the subscriber may not read (where: column "answer" is not readable by this caller), a value of the wrong type, both table and view (subscribe takes a table or a view, not both) - come back as { "type": "error", "id": "s1", "error": "<message>" }. A subscribe without an id gets subscribe needs an id (a string). Either way the subscription is not created.

How the client reconnects

The JavaScript client wraps all of this. What it does about the connection:

• The socket opens lazily, with the first subscribe. Until then the status is "closed"; while opening it is "connecting", then "open".
• All subscriptions share one socket. Each gets an id s1, s2, ...
• When the socket drops, the status becomes "reconnecting" and the client tries again after 1, 2, 4, 8 and then every 10 seconds (configurable with reconnectDelays; the last delay repeats). A successful connection resets the sequence.
• On reconnect it sends every subscription again and the server answers each with a fresh rows message. Your onRows handler is called again: treat it as a reset, not an append. Changes made while you were offline are in those rows.
• It sends a ping every 25 seconds (pingInterval). If nothing at all has arrived for two intervals, it treats the link as dead, closes it and reconnects.
• A subscription refused with forbidden is not sent again on reconnect.
• A client made with a secret key as its token never opens the socket: each subscription gets one onError with status 401, code unauthorized and the message secret keys do not open sockets; use a user token, the status stays "closed" and nothing retries. Its HTTP calls work as usual.
• When a hosted project is paused for being idle, or deleted, the server closes its sockets with close code 4001 and the reason project paused or project deleted. The client reconnects as after any drop: a paused project wakes up on the reconnect, a deleted one keeps refusing it.
• client.close() closes the socket and forgets every subscription; nothing reconnects after that until you subscribe again.

How fan-out works

Every write goes through the server's /v1/mutate. After the write commits, the server checks the change against each open subscription on that table - using that subscriber's own read rule, column rules and where - and sends it to those that pass, shaped for each. Changes that come from the clock rather than a write are sent by a separate timer, above. The number of subscriptions reached is written to the trace row's fanout. On a server with several projects, a change only ever reaches sockets of the environment it was made in.