🤝 Federated Clustering (v1.2)

HyperspaceDB v1.2 introduces a Federated Leader-Follower architecture. This goes beyond simple read-replication, introducing Node Identity, Logical Clocks, and Topology Awareness to support future Edge-Cloud synchronization scenarios.

Concepts

Node Identity

Every node in the cluster is assigned a persistent, unique UUID (node_id) upon first startup. This ID is used to track the origin of write operations in the replication log.

Roles

Leader (Coordinator):
- Accepts Writes (Insert, Delete, CreateCollection).
- Manages the Cluster Topology.
- Streams WAL events to connected Followers.
Follower (Replica):
- Read-Only.
- Replicates state from the Leader in real-time.
- Can be promoted to Leader if needed.
Edge Node (Planned v1.4):
- Offline-first node that accumulates writes and syncs via Merkle Trees when online.

Configuration

Leader

Simply start the server. By default, it assumes the Leader role.

./hyperspace-server --port 50051

Follower

Start with --role follower and point to the leader's URL.

./hyperspace-server --port 50052 --role follower --leader http://127.0.0.1:50051

Monitoring Topology

You can inspect the cluster state via the HTTP API on the Dashboard port (default 50050).

Request:

curl http://localhost:50050/api/cluster/status

Response:

{
  "node_id": "e8b37fde-6c60-427f-8a09-47103c2da80e",
  "role": "Leader",
  "upstream_peer": null,
  "downstream_peers": [],
  "logical_clock": 1234
}

This JSON response tells you:

The node's unique ID.
Its current role.
Who it is following (if Follower).
Who is following it (if Leader).
The current logical timestamp of its database state.

Edge-to-Edge Gossip Swarm (v3.0)

Beyond centralized replication, v3.0 introduces a decentralized Peer-to-Peer UDP Swarm network. This feature is crucial for robotics and offline-first autonomous agents.

Features

Zero-Configuration Topology: Nodes broadcast heartbeat logs via UDP (tokio::net::UdpSocket).
Self-Healing: Unresponsive nodes (TTL > 30s) are automatically dropped from the registry.
Auto-Discovery: Swarm nodes discover each other and exchange Logical Clocks and Collection Digests for the Merkle Delta Sync.

Swarm Configuration

Add these variables to your environment or .env file to start joining the global Swarm:

# Enable the Gossip listener on the specified local port
HS_GOSSIP_PORT=7946

# Bootstrapping nodes to connect to
HS_GOSSIP_PEERS=192.168.1.10:7946,192.168.1.11:7946

Swarm State Monitoring

You can monitor the active mesh structure from the dashboard UI or standard HTTP:

Request:

curl http://localhost:50050/api/swarm/peers

Response:

{
  "gossip_enabled": true,
  "peer_count": 1,
  "peers": [
    {
      "node_id": "a92jfe...",
      "addr": "192.168.1.10:50050",
      "http_port": 50050,
      "role": "Leader",
      "logical_clock": 4200,
      "collections": [
        {
          "name": "vision_system",
          "state_hash": 6712399120,
          "vector_count": 500
        }
      ],
      "last_seen_secs": 1729384910,
      "healthy": true
    }
  ]
}

HyperspaceDB Documentation