Veilid provides applications guarantees about how data is handled on the wire and at rest.

Cryptosystems were chosen that work well together and provide a balance of speed and cryptographic hardness.

Current Cryptography Systems

• Authentication is Ed25519

  Elliptic curve25519 was chosen to provide public/private key authentication and signing capabilities

• Key Exchange is x25519

  Curve25519 has a DH function that allows nodes to generate a symmetric key to communicate privately.

• Encryption is XChaCha20-Poly1305

  ChaCha20 with a 192-bit extended nonce is a fast authenticated stream cipher with associated data (AEAD).

• Message Digest is BLAKE3

  BLAKE3 is a extremely fast cryptographic hash that is highly parallelizable and as strong as SHA3-256 and over 17 times faster.

• Key Derivation is Argon2

  Password hash generation should be slow and resistant to GPU attacks Argon2 was the winner of the 2015 Password Hashing Competition.

Upgrading Cryptography Systems

Nothing lasts forever and cryptography is no exception. As computing power improves and cryptographic attacks evolve, weaknesses in cryptosystems are inevitable.

Veilid has ensured that upgrading to newer cryptosystems is streamlined and minimally invasive to app developers, and handled transparently at the node level.

• Multiple Routing Tables

  Because changing cryptosystems changes node ids, there will be different distance measurements between nodes, necessitating a separate routing table per cryptosystem. We support this today.

• Typed Keys

  Cryptographic keys, signatures, and hashes are all tagged with their cryptosystem to ensure that we know exactly how they were generated and how they should be used and persisted.

• Migration Support

  Reading persisted data will automatically use the correct cryptosystem and will default to always writing it back using the newest/best cryptosystem. This allows for data to be easily migrated just by reading it and writing it back to storage.

• Simultaneous Cryptosystems

  While transitioning cryptosystems, nodes can respond to other nodes using either the old system or the new one, or both.

Secure Storage

• Device-level secret storage APIs are available for all platforms
• Encrypted table store APIs are exposed to applications to make safe data storage easy
• Device data keys can also be password protected
• Apps never need to write anything to disk unencrypted

On The Wire

• All Protocols Same Encryption

  Each low-level protocol uses the same message and receipt encapsulation. No protocol is special and all protocols offer the same safety guarantees.

• Encrypted And Signed

  Messages between nodes are signed by the sender and encrypted for only the receiver. Messages can be relayed without decryption and authentication covers the entire contents including headers.

• Everything Is Timestamped

  Envelopes include timestamps and unique nonces and reject old or replayed messages.

• Node Information Is Signed

  When a node publishes routing table entries they are signed. No node can lie about another node's dial info, capabilities, availability, or replay old node info when newer info is available.

All devices are welcome and treated equally

You can use the public Veilid Network or build your own

Nodes help each other like mutual aid for connectivity

All Veilid applications running veilid-core are 'nodes', and they are all equal in the eyes of the network. No nodes are 'special'.

Nodes are only limited by the resources they bring and the configuration of the network they are on.

DNS is only used one time during ‘bootstrap'; it is not required though.

SSL is optional and only for HTTPS Websockets for Veilid Webapps.

Protocols

Veilid uses UDP, TCP, and Websockets.

Low level protocols supported by Veilid are kept simple, to minimize complications. Everything uses framed RPC operations up to 64KB in size. Protocol support is extensible and may add WebRTC and other specialized protocols in the future.

Network Topology

To zoom in on the details, view the full size image directly.

Bootstrapping

Bootstrap nodes not 'special' nodes. Any node can bootstrap a Veilid network. Networks can be 'keyed' to keep nodes off that don't have the key. You can join the ‘big Veilid network' or make your own isolated network.

• Ask Bootstraps To ‘Find Self'

  A single initial DNS TXT record request returns some bootstrap nodes that are known to exist. Those are asked to return nodes that are ‘close' to your own node.

• Public Address Detection

  Nodes are often behind various forms of NAT. Validating one's own public ‘Dial Info' is essential for publishing one's Node Info and answering Find Node requests.

• Relay Configuration

  Low-capability network classes may require the use of Inbound or Outbound relays in order to achieve reachability Nodes help each other out to the best of their ability and incur no penalty for not being able to assist other nodes.

• Peer Minimum Refresh

  Nodes in your routing table are asked to return nodes that are near you as well. Finding nodes close to your own is always harder than finding nodes far away, so we focus on that with our requests.

• Network Class Detection

  Determining NAT type and what mechanisms can be used to achieve connectivity. Direct connection techniques like reverse connections and UDP hole punching may be inappropriate for some network classes.

• Ping Validation

  Nodes come and go, change address, and are unreliable. Checking routing table nodes for proof-of-life is done with exponential backoff. Nodes are removed from the routing table on a LIFO basis.

Compiled Routes

Secure Envelopes

Toward The Future

• Per-Hop Payload Keying

  Ensuring that there is nothing common between packets at each hop will reduce the risk of mass data collection being able to deanonymize routes.

• Simplify Directionality

  Routes are currently bidirectional, but are allocated directionally. We may be able to simplify our allocation mechanism by enforcing bidirectionality. Bidirectional routes are faster, but directional routes could provide more anonymity.

• Elimination of Hop Counting

  Currently the protocol keeps an internal hop count that is not necessary. Efforts should be made to ensure that individual nodes don’t know how far along in a route they are.

• Hop Caching

  Route hop NodeInfo could be cached to save on-the-wire size as well as speed things up.

• Increasing Hop Count

  Currently the default is one hop chosen by the Safety Route, and one hop chosen by the Private Route, which leads to three hops total once compiled.

  It may be important to increase hop count to 2 for users with critical safety needs and to protect from nation-state-level deanonymization where appropriate.

  Existing research (on Tor) suggests that our existing hop count should be sufficient and provide comparable anonymity, but this should be revisited.

• Schema Language is Cap'n Proto

  Cap’n Proto is designed for deserialization speed and schema evolution. Flexible and well supported in Rust.

• RPC is fully in-schema and documented

  Both ‘Question/Answer’ and ‘Statement’ RPC modes are supported. All schema fields are documented.

• RPC fully supports Private Routing

  All private routing structures are expressed in the RPC schema itself, no magic encrypted blobs.

• Schema Evolution is built-in

  Fields can be added and removed with full backward and forward compatibility. New features won’t break older Veilid nodes.

• RPC Schema is cryptography-independent

  As cryptosystems change, the language spoken by Veilid nodes remains the same.

Distributed Hash Table

Distributed Hash Tables are a way of storing data in records that have keys that are close to nodes in the network.

DHT Is Just ‘Search’

It may look complicated, but all the DHT algorithms out there are just ‘search’ algorithms. Finding data that is stored on some node somewhere out there.

Improving Search

We built a better DHT by making both search and data locality more relevant. Veilid synchronizes popular data when nodes come and go from the network.

Locating a node by its ID. Here the node with the prefix 0011 finds the node with the prefix 1110 by successively learning of and querying closer and closer nodes. The line segment on top represents the space of 160-bit IDs, and shows how the lookups coverge to the target node. Below we illustrate RPC messages made by 1110. The first RPC is to node 101, already known to 1110. Subsequent RPCs are to nodes returned by the previous RPC.

DHT Schema

Veilid DHT is built using GetValue and SetValue RPC operations. Nodes can opt out of DHT storage if they do not want to participate.

Veilid DHT records have schemas that define subkeys that are individually addressable and can have multiple writers.

DHT record subkeys have sequence numbers and are eventually consistent across multiple writes and background synchronizations.