RPC Summary

• Schema Language is Cap'n Proto

  Cap’n Proto is designed for deserialization speed and schema evolution. Flexible and well supported in Rust.

• RPC is fully in-schema and documented

  Both ‘Question/Answer’ and ‘Statement’ RPC modes are supported. All schema fields are documented.

• RPC fully supports Private Routing

  All private routing structures are expressed in the RPC schema itself, no magic encrypted blobs.

• Schema Evolution is built-in

  Fields can be added and removed with full backward and forward compatibility. New features won’t break older Veilid nodes.

• RPC Schema is cryptography-independent

  As cryptosystems change, the language spoken by Veilid nodes remains the same.

Distributed Hash Table

Distributed Hash Tables are a way of storing data in records that have keys that are close to nodes in the network.

DHT Is Just ‘Search’

It may look complicated, but all the DHT algorithms out there are just ‘search’ algorithms. Finding data that is stored on some node somewhere out there.

Improving Search

We built a better DHT by making both search and data locality more relevant. Veilid synchronizes popular data when nodes come and go from the network.

Locating a node by its ID. Here the node with the prefix 0011 finds the node with the prefix 1110 by successively learning of and querying closer and closer nodes. The line segment on top represents the space of 160-bit IDs, and shows how the lookups coverge to the target node. Below we illustrate RPC messages made by 1110. The first RPC is to node 101, already known to 1110. Subsequent RPCs are to nodes returned by the previous RPC.

DHT Schema

Veilid DHT is built using GetValue and SetValue RPC operations. Nodes can opt out of DHT storage if they do not want to participate.

Veilid DHT records have schemas that define subkeys that are individually addressable and can have multiple writers.

DHT record subkeys have sequence numbers and are eventually consistent across multiple writes and background synchronizations.