Qbix Streams as a Graph Database

[Qbix]

Qbix Streams: A Searchable, Distributed Graph Database Hiding in Plain Sight

When most people think of graph databases, they imagine Neo4j with its Cypher queries, or Firebase/Firestore for document search. These tools give you nodes, edges, and the ability to traverse relationships.

But what if you could get all of that — plus history, access control, federation, and SQL-speed search — without leaving the comfort of relational databases?

That’s exactly what the Qbix Streams architecture provides. And it turns out, it’s not just “another storage layer” — it’s a searchable, distributed graph database, deeply integrated with how communities, apps, and services actually evolve.

---

1. Streams as Nodes

Every unit of information in the system is a stream, identified globally by (publisherId, streamName). Examples:

• Users/user/123
• Assets/product/456
• Places/user/location/95.90.209.105

Unlike a raw document or row in SQL, each stream:

• Has type, attributes, messages, and history
• Supports forking and merging, for moderation or customization
• Is governed by access control lists (ACLs)
• Can be hosted anywhere, but remains globally addressable

In graph terms: every stream is a node.

---

2. Relations as Edges

Streams don’t live in isolation. Their attributes automatically generate relations, which are directed edges of the graph, stored in normalized SQL tables:

• streams_related_from (outgoing edges)
• streams_related_to (incoming edges, pre-indexed for search, with weights)

Whenever a stream changes — say a user updates their location or a product gets tagged — the system runs syncRelations(), which recomputes edges. That means:

• Edges are always consistent with node state
• You never worry about stale indexes
• Every relation is automatically queryable

Example:

• Attribute: Places/geohash = u33dc
• Relation: attribute/Places/geohash=u33dc
• Target stream: Streams/search/all

That edge says: “this stream belongs to the search bucket, under this geohash label.”

In graph theory terms: relations are labeled directed edges.

---

3. SQL as the Query Engine

Most graph databases invent a new query language (Cypher, Gremlin). Streams doesn’t have to. Relations are stored in plain SQL tables with strong indexes. That means you can query them directly with SELECT, GROUP BY, and HAVING.

For example, here’s a search for all streams that are both in a specific geohash and in a credits category:

SELECT fromPublisherId, fromStreamName
FROM streams_related_to
WHERE toPublisherId = 'Streams'
  AND toStreamName = 'Streams/search/all'
  AND type IN ('attribute/Places/geohash=u33dc',
               'attribute/Assets/category=credits')
GROUP BY fromPublisherId, fromStreamName
HAVING COUNT(*) = 2;

That’s it. You just found all streams that match both conditions.

---

Faceted Search (Amazon-Style)

The same pattern powers e-commerce product filters. Every attribute (brand, color, price range, etc.) is stored as a relation. To get facet counts for all attributes at once:

SELECT type, COUNT(*) AS matches
FROM streams_related_to
WHERE toPublisherId = 'Streams'
  AND toStreamName = 'Streams/search/all'
  AND type LIKE 'attribute/Assets/%'
GROUP BY type;

Results might look like:

• attribute/Assets/brand=Sony → 152 products
• attribute/Assets/color=Black → 879 products
• attribute/Assets/priceRange=200-500 → 65 products

When a user selects a filter (“Sony”), you simply add it to the IN (…) clause:

... AND type IN ('attribute/Assets/brand=Sony')
HAVING COUNT(*) = 1;

Selecting multiple filters (“Sony + Black”):

... AND type IN ('attribute/Assets/brand=Sony',
                 'attribute/Assets/color=Black')
HAVING COUNT(*) = 2;

The HAVING COUNT(*) = 2 ensures that only products matching both criteria are returned.

---

Sorting by Relevance

Sometimes you don’t just want matches — you want the best matches ranked by how many filters they satisfy. That’s one SQL tweak away:

SELECT fromPublisherId, fromStreamName, COUNT(*) as relevance
FROM streams_related_to
WHERE toPublisherId = 'Streams'
  AND toStreamName = 'Streams/search/all'
  AND type IN ('attribute/brand=Sony', 'attribute/color=Black', 'attribute/price<500')
GROUP BY fromPublisherId, fromStreamName
ORDER BY relevance DESC;

---

Multi-Valued Attributes

Some attributes are arrays — for example, a product can have multiple colors (["Black","Red"]) or a person can speak multiple languages (["English","Spanish"]). Each value is stored as a separate relation row:

• attribute/Assets/color=Black
• attribute/Assets/color=Red

This means queries work the same way: if you filter by color=Black, all products with Black in their array of colors will be included.

---

People Search (Dating-App Style)

In a dating app, user profiles publish attributes like:

• attribute/Profile/height=5'10"
• attribute/Profile/ageRange=25-30
• attribute/Profile/religion=Jewish
• attribute/Profile/languages=Spanish

A search for “Spanish-speaking, age 25-30” is the same as before:

... AND type IN ('attribute/Profile/languages=Spanish',
                 'attribute/Profile/ageRange=25-30')
HAVING COUNT(*) = 2;

Every filter is just another relation type. Adding or removing criteria is as simple as adjusting the IN (…) list, while HAVING COUNT(*) = n guarantees the match covers all selected filters.

---

Because relations are flattened into rows and indexed, you get:

• Fast lookups (comparable to document stores)
• Aggregations with counts (COUNT, SUM, HAVING) — something Firebase/Firestore can’t do natively
• Support for scalar and array attributes without extra schema work
• No custom runtime — just MySQL or MariaDB

In short: graph search at SQL speed, whether you’re finding people by height and age, or products by brand and color.

---

4. Distribution by Design

Unlike Neo4j or Firebase, which assume a single global DB, Streams was built for federation.

• Every stream belongs to a publisherId (community, app, or user).
• Publishers can host their streams anywhere.
• Relations between publishers work seamlessly because the schema itself is global.

That makes Streams a multi-tenant graph database, where each community can govern its own data — but search still works across the network.

---

5. Beyond Graph: Streams Bring History & ACLs

Here’s where Streams leap beyond graph databases:

• History: every change to a stream is logged as a message, so you can replay or audit evolution.
• Forks: communities can fork streams (e.g., moderate user content) while preserving lineage.
• ACLs: streams enforce read/write/admin levels out of the box.

Try doing that in Neo4j or Firebase — you’d have to bolt it on manually.

---

6. Example Queries That Just Work

• Find people near me:
  Attribute relations: attribute/Places/geohash=u33d* → search by prefix

• Find products and services in the same area:
  Both Assets/product/* and Assets/service/* streams relate to Streams/search/all by geohash

• Find users with >100 credits who wrote an article:
  Two relations: attribute/Assets/credits>100, attribute/Assets/articleAuthor=true

• Find dogs and people weighing 100–200 lbs:
  attribute/Animals/weight and attribute/Users/weight are just different edge labels in the same search bucket.

No joins across 20 different tables — just SQL on one relation index.

---

7. Comparison to Other Systems

Feature	Streams	Neo4j	Firebase/Firestore
Nodes/edges	Streams/relations	Yes	No (documents only)
Query language	SQL (portable)	Cypher (custom)	Proprietary API
Federation	Built-in (publisherId)	No (single DB)	No
History & forks	Yes	No	No
ACLs	Yes	No	Limited
Aggregations	Native (COUNT, SUM, HAVING)	Complex	Weak
Hosting model	Federated, self-hostable	Centralized	Cloud-only

Streams is not just another graph DB — it’s a graph + event log + ACL system, built on relational DBs, and distributed by design.

---

Conclusion

What started as a way to organize user data has evolved into something bigger: a graph database you didn’t know you had.

By treating streams as nodes and relations as edges, with automatic updates whenever streams change, the system creates a searchable, distributed graph on top of SQL. And because every stream also has history, forking, and ACLs, this isn’t just graph storage — it’s the backbone for decentralized, trustworthy apps.

If Neo4j is a graph DB, and Firebase is a document store, then Streams is a graph database for communities — one that was hiding in plain sight all along.

GitHub - Qbix/Streams: Streams plugin

[Greg]

So, a year later we are adding vector similarity search, in all 3 of our adapters: Sqlite, Postgres and MariaDB (though not MySQL). They can handle it. These days a lot of the work is done in collaboration with agentic AI, and we had it write this update post:

Adding “similar to” to the graph

The relation model in this post handles exact predicates beautifully. Every facet is an edge, HAVING COUNT(*) = n guarantees the AND, and it’s all one indexed table. Spanish-speaking, 25-30, within a geohash — three edges, one query.

What it couldn’t express is resemblance. “Profiles like this one.” “Products that match this description.” “Other posts about roughly this.” Those aren’t a set of equalities, so there’s no IN (...) list that captures them.

The usual answer is to bolt on a second system — Pinecone, Weaviate, a Qdrant container — and then reconcile two sources of truth, two access-control stories, and two failure modes. For a federated system where each publisher hosts their own data, that’s a bad trade.

So we added vectors to the Db layer instead.

The shape of it

Db_Vector is a value type alongside Db_Range, and vectorNearestTo() is a query method alongside where() and orderBy():

Streams_Embedding::select()
    ->where(array('publisherId' => $communityId))
    ->vectorNearestTo('embedding', Db::vector($queryEmbedding), array('limit' => 20))
    ->fetchDbRows();

Same call in Node:

Streams.Embedding.SELECT('*')
    .where({publisherId: communityId})
    .vectorNearestTo('embedding', Db.vector(queryEmbedding), {limit: 20})
    .execute(callback);

Three engines render it in their own dialect — MariaDB 11.7+ viaVEC_DISTANCE_COSINE and an HNSW VECTOR INDEX, Postgres via pgvector’s <=> operator, SQLite via sqlite-vec. Cosine and euclidean distances agree to four decimals across all three, so an app developed against SQLite behaves the same in production on MariaDB.

Generating embeddings stays out of Db entirely — that’s the AI plugin’s job. Db stores and searches them, the way MariaDB itself draws the line.

Why this matters here specifically

Because it composes with the relation graph rather than replacing it. Filter exactly, then rank by similarity, in one statement:

SELECT m.fromPublisherId, m.fromStreamName,
       VEC_DISTANCE_COSINE(e.embedding, VEC_FromText(:q)) AS dist
FROM (
  SELECT fromPublisherId, fromStreamName
  FROM streams_related_to
  WHERE toPublisherId = 'Streams' AND toStreamName = 'Streams/search/all'
    AND type IN ('attribute/Profile/languages=Spanish',
                 'attribute/Profile/ageRange=25-30')
  GROUP BY fromPublisherId, fromStreamName
  HAVING COUNT(*) = 2
) m
JOIN streams_embedding e
  ON e.publisherId = m.fromPublisherId AND e.streamName = m.fromStreamName
ORDER BY dist
LIMIT 20;

“Spanish-speaking, 25-30, sorted by how close their profile reads to this one.” The hard requirements stay hard — the graph enforces them exactly — and similarity only decides the ordering within that set. That’s usually what you actually want, and it’s the opposite of what a bolted-on vector store gives you, where you get the 50 nearest neighbours and then discover that only 6 speak Spanish.

It also inherits everything this post argues for. The embedding lives in a stream, so it’s subject to the same ACLs. It’s published by a publisherId, so federation works unchanged. It’s in the same transaction, so there’s no window where the graph and the index disagree.

Honest limits

Pre-filter vs. index. In the query above, the relation filter runs first and the distance is computed over the survivors. That’s exact, but it doesn’t use the HNSW index. Reverse the order — nearest-20 first, then filter — and you use the index but may return fewer than 20. Neither is wrong; it depends on how selective your facets are. This is the standard ANN tradeoff, not something specific to Streams.

Metric is chosen at index time. MariaDB silently falls back to a full scan if you query with a different metric than you built with; SQLite’s vec0 will quietly answer in the metric it was built with. vectorNearestTo() refuses a mismatch on SQLite rather than returning numbers in the wrong units, but on MariaDB you just lose the index. Build for cosine, query with cosine.

Sharding and KNN don’t compose. Top-k across N shards means asking each shard for its top-k and re-merging, which the current pipe doesn’t do. Keep embedding tables unsharded and this never comes up.

Record which model produced each vector. Comparing embeddings from two different models produces plausible-looking nonsense — no error, no warning, just quietly wrong rankings. An embeddingModel column next to the vector makes mismatches detectable and lets you re-embed incrementally.

Dimension is part of the schema. VECTOR(768) bakes it in. Switchingembedding models later is an ALTER TABLE plus a full re-embed, not a configchange.

Community MySQL can’t do this. MySQL 9 has a VECTOR column type, but DISTANCE() ships only with HeatWave and MySQL AI. The adapter checks the server version and reports no vector support rather than emitting SQL thatreferences a function which isn’t there. MariaDB 11.7+, Postgres with pgvector, or SQLite with sqlite-vec.

What this unlocks

The interesting queries aren’t “find similar” on its own — every vector store does that. They’re the ones that need both halves:

• Products in this geohash, in stock, ranked by similarity to a photo caption
• Events this weekend I have access to, ranked against my interest profile
• Posts in communities I belong to, ranked against what I’ve been reading
• Duplicate detection scoped to one publisher, so moderation stays local

Each of those is a relation filter the graph already does well, plus an ordering the graph couldn’t express. Now it’s one query, one engine, one ACL check.