The Tari RFCs

Tari is a community-driven project. The documents presented in this RFC collection have typically gone through several iterations before reaching this point:

Ideas and questions are posted in #tari-dev on #FreeNode IRC. This is typically short-form content with rapid feedback. Often, these conversations will lead to someone posting a proposal as a pull request into proposals.
Proposals get debated and refined using Github's Review and pull request systems. After the content has stabilised, and the community accepts the proposal, it can be written up as a formal RFC.
RFCs are "Requests for Comment", so although the proposals in these documents are usually well-thought out, they are not cast in stone. RFCs can, and should, undergo further evaluation and discussion by the community. RFC comments are best made using Github issues.

New RFC's should follow the format given in the RFC template.

Lifecycle

RFCs go through the following lifecycle, which roughly corresponds to the COSS:

Status		Description
Draft		Changes, additions and revisions can be expected.
Stable		Typographical and cosmetic changes aside, no further changes should be made. Changes to the Tari code base w.r.t. a stable RFC will lead to the RFC becoming out of date, deprecated, or retired.
Out of date		This RFC has become stale due to changes in the code base. Contributions will be accepted to make it stable again if the changes are relatively minor, otherwise it should eventually become deprecated or retired.
Deprecated		This RFC has been replaced by a newer RFC document, but is still is use in some places and/or versions of Tari.
Retired		The RFC is no longer in use on the Tari network.

RFC-0001/Overview

Overview of Tari Network

status: draft

Maintainer(s): Cayle Sharrock

Licence

The 3-Clause BSD Licence.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

Redistributions of this document must retain the above copyright notice, this list of conditions and the following disclaimer.
Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.

THIS DOCUMENT IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS", AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

Language

The keywords "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY" and "OPTIONAL" in this document are to be interpreted as described in BCP 14 (covering RFC2119 and RFC8174) when, and only when, they appear in all capitals, as shown here.

Disclaimer

This document and its content are intended for information purposes only and may be subject to change or update without notice.

This document may include preliminary concepts that may or may not be in the process of being developed by the Tari community. The release of this document is intended solely for review and discussion by the community of the technological merits of the potential system outlined herein.

Goals

The aim of this proposal is to provide a very high-level perspective of the moving parts of the Tari protocol.

Related Requests for Comment

Description

Abstract

The Tari network comprises three layers:

A base layer that deals with Tari coin transactions. It governed by a proof-of-work (PoW) blockchain that is merged-mined with Monero. The base layer is highly secure, decentralized and relatively slow.
A multiparty payments channel that allows rapid, secure, low-cost, off-chain payments that are periodically settled on the base layer.
A digital assets network (DAN) that manages all things to do with native digital assets. It is built for liveness, speed and scalability at the expense of decentralization.

Tari Network Overview

Currency Tokens and Digital Assets

There are two major digital entities on the Tari network: the coins that are the unit of transfer for the Tari cryptocurrency, and the digital assets that could represent anything from tickets to in-game items.

Tari coins are the fuel that drives the entire Tari ecosystem. They share many of the properties of money, so security is a non-negotiable requirement. In a cryptocurrency context, this is usually achieved by employing a decentralized network running a censorship-resistant protocol such as Nakamoto consensus over a proof-of-work blockchain. As we know, PoW blockchains are not scalable or very fast.

On the other hand, the Tari network will be used to create and manage digital assets.

In Tari parlance, a digital asset is defined as a finite set of digital stateful tokens that are governed by predefined rules. A single digital asset may define anything from one to thousands of tokens within its scope.

For example, in a ticketing context, an event will be an asset. The asset definition will allocate tokens representing the tickets for that event. The ticket tokens will have state, such as its current owner and whether or not it has been redeemed. Users might be interacting with digital assets hundreds of times a second, and state updates need to be propagated and agreed upon by the network very quickly. A blockchain-enabled ticketing system is practically useless if a user has to wait for "three block confirmations" before the bouncer will let her into a venue. Users expect near-instant state updates because centralized solutions offer them that today.

Therefore the Tari DAN must offer speed and scalability.

Multiple Layers

The distributed system trilemma tells us that these requirements are mutually exclusive.

We can't have fast, cheap digital assets and also highly secure and decentralized currency tokens on a single system.

Tari overcomes this constraint by building three layers:

A base layer that provides a public ledger of Tari coin transactions, secured by PoW to maximize security.
A multiparty payment channel that allows funds to be sent to parties in the channel instantly, securely and with very low fees.
A DAN that manages the state of digital assets. It is very fast and cheap, at the expense of decentralization.

If required, the digital assets layer can refer back to the base layer to temporarily give up speed in exchange for increased security. This fallback is used to resolve consensus issues on the digital assets layer that may crop up from time to time as a result of the lower degree of decentralization.

Base Layer

Refer to RFC-0100/BaseLayer for more detail.

The Tari base layer has the following primary features:

PoW-based blockchain using Nakamoto consensus
Transactions and blocks based on the Mimblewimble protocol

Mimblewimble is an exciting new blockchain protocol that offers some key advantages over other UTXO-based cryptocurrencies such as Bitcoin:

Transactions are private. This means that casual observers cannot ascertain the amounts being transferred or the identities of the parties involved.
Mimblewimble employs a novel blockchain "compression" method called cut-through, which dramatically reduces the storage requirements for blockchain nodes.
Multi-signature transactions can be easily aggregated, making such transactions very compact, and completely hiding the parties involved, or the fact that there were multiple parties involved at all.

"Mimblewimble is the most sound, scalable 'base layer' protocol we know" -- @fluffypony

Proof of Work

There are a few options for the PoW mechanism for Tari:

Implement an existing PoW mechanism. This is a bad idea, because a nascent cryptocurrency that uses a non-unique mining algorithm is incredibly vulnerable to a 51% attack from miners from other currencies using the same algorithm. Bitcoin Gold and Verge have already experienced this, and it's a matter of time before it happens to others.
Implement a unique PoW algorithm. This is a risky approach and comes close to breaking the number one rule of cryptocurrency design: never roll your own crypto.
Merged mining. This approach is not without its own risks, but offers the best trade-offs in terms of bootstrapping the network. It typically provides high levels of hash rate from day one, along with 51% attack resistance, assuming mining pools are well-distributed.
A hybrid approach, utilizing two or more of the above mechanisms.

Given Tari's relationship with Monero, a merged-mining strategy with Monero makes the most sense. However, the PoW mechanism SHOULD be written in a way that makes it relatively easy to code, implement and switch to a different strategy in the future.

Multiparty Payment Channel

Further details about the Tari multiparty payment channel technology are given in RFC-500/PaymentChannels.

Digital Assets Network

A more detailed proposal for the DAN is presented in RFC-0300/DAN. Digital assets are discussed in more detail in RFC-0310/Assets.

The Tari DAN consists of a peer-to-peer network of Validator nodes. These nodes ensure the safe and efficient operation of all native digital assets on the Tari network.

Validator nodes are responsible for:

Registering themselves on the base layer.
Validating and executing the contracts that create and issue new digital assets on the network.
Validating and executing instructions for changes in state of digital assets, e.g. allowing the transfer of ownership of a token from one person to another.
Maintaining consensus with other validator nodes managing the same asset.
Submitting periodic checkpoints to the base layer for the state of assets under their management.

The DAN is focused on achieving high speed and scalability, without compromising on security. To achieve this, we make the explicit trade-off of sacrificing decentralization.

In many ways this is desirable, since the vast majority of assets (and their issuers) don't need or want the entire network to validate every state change in their asset contracts.

Digital assets necessarily have state. Therefore the digital assets layer must have a means of synchronizing and agreeing on state that is managed simultaneously by multiple servers, i.e. reaching consensus.

Please refer to Tari Labs University (TLU) for detailed discussions on layer 2 scaling solutions and consensus mechanisms.

Interaction between Base Layer and Digital Assets Network

The base layer provides supporting services to the DAN. In general, the base layer only knows about Tari coin transactions. It knows nothing about the details of any digital assets and their state.

This is by design: the network cannot scale if details of digital asset contracts have to be tracked on the base layer. We envisage that there could be tens of thousands of contracts deployed on Tari. Some of those contracts may be enormous; imagine controlling every piece of inventory and their live statistics for a massively multiplayer online role-playing game (MMORPG). The base layer is also too slow. If any state relies on base layer transactions being confirmed, there is an immediate lag before that state change can be considered final, which kills the liveness properties we seek for the DAN.

It is better to keep the two networks almost totally decoupled from the outset, and allow each network to play to its strength.

That said, there are key interactions between the two layers. The base layer is a ledger and can be used as a source of truth for the DAN to use as a type of registrar as well as final court of appeal in the case of consensus disputes. This is what gives the DAN a secure fallback should bad actors try to manipulate asset state by taking advantage of its non-decentralization.

These interactions require making provision for additional transaction types, in addition to payment and coinbase transactions, which mark validator node registrations, contract collateral, etc.

The interplay between base layer and DAN is what incentivizes every actor in the system to maintain an efficient and well-functioning network, even while acting in their own self-interest.

Summary

The following table summarizes the defining characteristics of the Tari network layers:

	Base Layer	Payment Channels	Digital Assets Network
Speed	Slow	Fast	Fast
Scalability	Moderate	High	Very high
Security	High	High	Moderate (High with fallback)
Decentralization	High	Low - Medium	Low - Med
Processes Tari coin transactions	Yes	Yes	No
Processes digital asset instructions	Only checkpoints	No	Yes

RFC-0010/CodeStructure

Tari Code Structure and Organization

status: draft

Maintainer(s): Cayle Sharrock

Licence

The 3-Clause BSD Licence.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

Redistributions of this document must retain the above copyright notice, this list of conditions and the following disclaimer.
Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.

Language

Disclaimer

This document and its content are intended for information purposes only and may be subject to change or update without notice.

Goals

The aim of this Request for Comment (RFC) is to describe and explain the Tari codebase layout.

Related Requests for Comment

None.

Description

The code follows a Domain-driven Design (DDD) layout, with top-level directories falling into infrastructure, domain and application layers.

Infrastructure Layer

The infrastructure layer provides a set of crates that have general infrastructural utility. The rest of the Tari codebase can make use of these crates to obtain persistence, communication and cryptographic services. The infrastructure layer doesn't know anything about blockchains, transactions or digital assets.

We recommend that code in this layer generalizes infrastructure services behind abstraction layers as much as is reasonable, so that specific implementations can be swapped out with relative ease.

Domain Layer

The domain layer houses the Tari "business logic". All protocol-related concepts and procedures are defined and implemented here.

This means that any and all terms defined in the Glossary will have a software implementation here, and only here. They can be used in the application layer, but must be implemented in the domain layer.

The domain layer can make use of crates in the infrastructure layer to achieve its goals.

Application Layer

In the application layer, applications build on top of the domain layer to produce the executable software that is deployed as part of the Tari network.

As an example, the following base layer applications may be developed as part of the Tari protocol release:

A standalone miner (tari_miner)
A pool miner (tari_pool_miner)
A Command Line Interface (CLI) wallet for the Tari cryptocurrency (cli_wallet)
A base node executable (tari_basenode)
A REST Application Programming Interface (API) server for the base node

Code Layout

The top-level source code directories in this repository reflect the respective DDD layers; except that there are two domain layer directories, corresponding to the two network layers that make up the Tari network.

The infrastructure directory contains application-layer code and is not Tari-specific. It holds the following crates:
- comms - the networking and messaging subsystem;
- crypto - all cryptographic services, including a Curve25519 implementation;
- storage - data persistence services, including a Lightning Memory-mapped Database (LMDB) persistence implementation;
- merklemountainrange - an independent implementation of a Merkle Mountain Range;
- derive - a crate to contain derive(...) macros.
base_layer is a domain-layer directory and contains:
- blockchain - the Tari consensus code;
- core - common classes and traits, such as Transactions and Blocks;
- mempool - the unconfirmed transaction pool implementation;
- mining - the merged-mining modules;
- p2p - the block and transaction propagation module;
- api - interfaces for clients and wallets to interact with the base layer components.
digital_assets_layer is a domain-layer directory. It contains code related to the management of native Tari digital assets. Its substructure is to be determined.
applications contains crates for all the application-layer executables that form part of the Tari codebase.

RFC-0100/BaseLayer

The Tari Base Layer

status: draft

Maintainer(s): Cayle Sharrock

Licence

The 3-Clause BSD Licence.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

Redistributions of this document must retain the above copyright notice, this list of conditions and the following disclaimer.
Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.

Language

Disclaimer

This document and its content are intended for information purposes only and may be subject to change or update without notice.

Goals

The aim of this Request for Comment (RFC) is to describe the major software components of the Tari Base Layer network.

Related Requests for Comment

Description

The Tari Base Layer network comprises the following major pieces of software:

Base Layer full node implementation. The base layer full nodes are the consensus-critical pieces of software for the Tari base layer and cryptocurrency. The base nodes validate and transmit transactions and blocks, and maintain consensus about the longest valid proof-of-work blockchain.
Mining software. Mining nodes perform proof-of-work to secure the base layer and compete to submit the next valid block into the Tari blockchain. Tari is merge-mined with Monero. The Tari source provides two alternatives for Tari miners:
- A standalone miner
- A stratum-compatible pool miner.
Wallet software. Client software and Application Programming Interfaces (APIs) offering means to construct transactions, query nodes for information and maintain personal private keys.

These three major pieces of software make use of common functionality provided by the following libraries within the Tari project source code:

Local data storage
Cryptography services
Peer-to-peer networking and messaging services

RFC-0010 provides more detail on how the source code is structured within the Tari codebase.

RFC-0110/BaseNodes

Base Layer Full Nodes (Base Nodes)

status: draft

Maintainer(s): Cayle Sharrock and S W van heerden

Licence

The 3-Clause BSD Licence.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

Redistributions of this document must retain the above copyright notice, this list of conditions and the following disclaimer.
Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.

Language

Disclaimer

This document and its content are intended for information purposes only and may be subject to change or update without notice.

Goals

The aim of this Request for Comment (RFC) is to describe the roles that base nodes play in the Tari network as well as their general approach for doing so.

Related Requests for Comment

Description

Broad Requirements

Tari Base Nodes form a peer-to-peer network for a proof-of-work based blockchain running the Mimblewimble protocol. The proof-of-work is performed via merge mining with Monero. Arguments for this design are presented in the overview.

Tari Base Nodes MUST carry out the following tasks:

validate all Tari coin transactions;
propagate valid transactions to peer nodes;
validate all new blocks received;
propagate validated new blocks to peer nodes;
connect to peer nodes to catch up (sync) with their blockchain state;
provide historical block information to peers that are syncing.

Once the Digital Assets Network (DAN) goes live, Base Nodes will also need to support the tasks described in RFC-0300/Assets. These requirements are omitted for the moment.

To carry out these tasks effectively, Base Nodes SHOULD:

save the blockchain into an indexed local database;
maintain an index of all Unspent Transaction Outputs (UTXOs);
maintain a list of all pending, valid transactions that have not yet been mined (the mempool);
manage a list of Base Node peers present on the network.

Tari Base Nodes MAY implement chain pruning strategies that are features of Mimblewimble, including transaction cut-through and block compaction techniques.

Tari Base Nodes MAY also implement the following services via an Application Programming Interface (API) to clients:

Block queries
Kernel data queries
Transaction queries
Submission of new transactions

Such clients may include "light" clients, block explorers, wallets and Tari applications.

Transaction Validation and Propagation

Base nodes can be notified of new transactions by:

connected peers;
clients via APIs.

When a new transaction has been received, it has the unvalidated ValidationState. The transaction is then passed to the transaction validation service, where its state will become rejected, timelocked or validated.

The transaction validation service checks that:

All inputs to the transaction are valid UTXOs.
No inputs are duplicated.
All inputs are able to be spent (they are not time-locked).
All inputs are signed by their owners.
All outputs have valid range proofs.
No outputs currently exist in the UTXO set.
The transaction does not have timelocks applied, limiting it from being mined and added to the blockchain before a specified block height or timestamp has been reached.
The transaction excess has a valid signature.
The transaction excess is a valid public key. This proves that: $$ \Sigma \left( \mathrm{inputs} - \mathrm{outputs} - \mathrm{fees} \right) = 0 $$.

Rejected transactions are dropped silently.

Timelocked transactions are:

marked with a timelocked status and get added to the mempool;
will be evaluated again at a later state to determine if the timelock has passed and if it can be upgraded to "Validated" status.

Note: More detailed information is available in the timelocks RFC document.

Validated transactions are:

added to the mempool;
forwarded to peers using the transaction BroadcastStrategy.

Block Validation and Propagation

The block validation and propagation process is analogous to that of transactions. New blocks are received from the peer-to-peer network, or from an API call if the Base Node is connected to a Miner.

When a new block is received, it is assigned the unvalidated ValidationState. The block is then passed to the block validation service. The validation service checks that:

The block has not been processed before.
Every transaction in the block is valid.
The proof-of-work is valid.
The block header is well-formed.
The block is being added to the chain with the highest accumulated proof-of-work.
- It is possible for the chain to temporarily fork; Base Nodes SHOULD account for forks up to some configured depth.
- It is possible that blocks may be received out of order, particularly while syncing. Base Nodes SHOULD keep blocks that have block heights greater than the current chain tip in memory for some preconfigured period.
The sum of all excesses is a valid public key. This proves that: $$ \Sigma \left( \mathrm{inputs} - \mathrm{outputs} - \mathrm{fees} \right) = 0$$.
Check if cut-through was applied and, if a block contains already spent outputs, reject that block.

Because Mimblewimble blocks can simply be seen as large transactions with multiple inputs and outputs, the block validation service checks all transaction verification on the block as well.

Rejected blocks are dropped silently.

Base Nodes are not obliged to accept connections from any peer node on the network. In particular:

Base Nodes MAY refuse connections from peers that have been added to a blacklist.
Base Nodes MAY be configured to exclusively connect to a given set of peer nodes.

Validated blocks are

added to the blockchain;
forwarded to peers using the block BroadcastStrategy.

In addition, when a block has been validated and added to the blockchain:

The mempool MUST also remove all transactions that are present in the newly validated block.
The UTXO set MUST be updated by removing all inputs in the block, and adding all the new outputs into it.

Synchronizing and Pruning of the Chain

Syncing, pruning and cut-through are discussed in detail in RFC-0140.

Archival Nodes

Archival nodes are used to keep a complete history of the blockchain since genesis block. They do not employ pruning at all. These nodes will allow full syncing of the blockchain, because normal nodes will not keep the full history to enable this.

RFC-0111/BaseNodesArchitecture

Base Node Architecture

status: draft

Maintainer(s): Cayle Sharrock

Licence

The 3-Clause BSD Licence.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

Redistributions of this document must retain the above copyright notice, this list of conditions and the following disclaimer.
Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.

Language

Disclaimer

This document and its content are intended for information purposes only and may be subject to change or update without notice.

Goals

The aim of this Request for Comment (RFC) is to describe the high-level Base Node architecture.

Architectural Layout

The Base Node architecture is designed to be modular, robust and performant.

Base Layer architecture

The major components are separated into separate modules. Each module exposes a public Application Programming Interface (API), which communicates with other modules using asynchronous messages via futures.

Base Node Service

The Base Node Service is an instantiation of a Tari Comms Service, which subscribes to and handles specific messages coming from the P2P Tari network via the Comms Module of a live Tari communications node. The Base Node Service's job is to delegate the jobs required by those messages to its sub-modules, consisting primarily of the Transaction Validation Service, the Block Validation Service and the Block synchronisation service, using an asynchronous Request-Response pattern.

The Base Node Service will pass messages back to the P2P network via the Comms Module, based on the results of its actions.

The primary messages that a Base Node will subscribe to are:

NewTransaction. A New Transaction is being propagated over the network. If it has not seen the transaction before, the Base Node will validate the transaction and, if it is valid:
- add it to its mempool;
- pass the transaction on to peers.
Otherwise, the transaction is dropped.
NewBlock. A newly mined block is being propagated over the network. If the node has not seen the block before, the node will validate it. Its action depends on the validation outcome:
- Invalid block - drop the block.
- Valid block appending to the longest chain - add the block to the local state; propagate the block to peers.
- Valid block forking off main chain - add the block to the local state; propagate the block to peers.
- Valid block building off unknown block - add the orphan block to the local state.
Sync Request. A peer is synchronizing state and is asking for block data. The node can decide to:
- Ignore or ban the peer (based on previous behaviour heuristics).
- Try and provide the data, returning an appropriate response. Note that most nodes can only offer block data up until their pruning horizon. Only full archival nodes can return the full block history. Refer to RFC-0140 for more details.

The validation procedures are complex and are thus encapsulated in their own sub-services. These services hold references to the blockchain state API, the mempool API, a range proof service and whatever other modules they need to complete their work. Each validation module has a single primary method, validate_xxx(), which takes in the transaction or block to be validated and returns a future that resolves once the validation task is complete.

Distributed Hash Table (DHT) Service

Peer discovery is a key service that blockchain nodes provide so that the peer mesh network can be navigated by the full nodes making up the network.

In Tari, the peer-to-peer network is not only used by full nodes (Base Nodes), but also by Validator Nodes, and

Tari and Digital Assets Network (DAN) clients.

For this reason, peer management is handled internally by the Comms layer. If a Base Node wants to propagate a message, new block or transaction, for example, it simply selects a BROADCAST strategy for the message and the Comms layer will do the rest.

When a node wishes to query a peer for its peer list, this request will be handled by the DHTService. It will communicate with its Comms module's Peer Manager, and provide that information to the peer.

Blockchain State Module

The blockchain state module is responsible for providing a persistent storage solution for blockchain state data. For Tari, this is delivered using the Lightning Memory-mapped Database (LMDB). LMDB is highly performant, intelligent and straightforward to use. An LMDB is essentially treated as a hash map data structure that transparently handles memory caching, disk Input/Output (I/O) and multi-threaded access.

The blockchain module is able to run as a standalone service, but must be thread-safe. Block and transaction validation requests are futures-based. These are asynchronous requests, which means that multiple validation requests can and should be handled in parallel, in separate threads. Initially, all the logic for a single block or transaction validation can be executed in sequence, wrapped inside a single future. However, there is scope to optimise this in future; for example: Validating a block entails checking the proof-of-work (very slow), checking signatures (fast, but many of them), and checking the accounting (slow). Each of these sub-tasks could also be spun off as a future, with a master future co-ordinating the sub-futures and assembling the final results.

Tokio is becoming the de facto standard for asynchronous programming in Rust.

Tokio's default task executor provides multi-threaded work-stealing work queues and CPU-bound worker threads out of the box. This is a good fit for the type of work that base nodes must perform. In addition, the Tower project provides a set of traits and middleware that will be very useful in Tari services, and so it is recommended to follow the Services pattern as used by that project.

This RFC proposes that the 0.1 version of tokio is used in the Tari project until the standard futures library has stabilised before making a switch.

Mempool Module

The mempool module tracks (valid) transactions that the node knows about, but that have not yet been included in a block. The mempool is ephemeral and non-consensus critical, and as such may be a memory-only data structure. Maintaining a large mempool is far more important for Base Nodes serving miners than those serving wallets. A mempool will slowly rebuild after a node reboots.

That said, the mempool module must be thread safe. The Tari mempool module handles requests in the same way as the Blockchain state module: via futures. The mempool structure itself is a set of hash maps as described in RFC-0190. For performance reasons, it may be worthwhile using a concurrent hash map implementation.

gRPC Interface

Base Nodes need to provide a local communication interface in addition to the P2P communication interface. This is best achieved using gRPC. The Base Node gRPC interface provides access to the public API methods of the Base Node Service, the mempool module and the blockchain state module, as discussed above.

gRPC access is useful for tools such as local User Interfaces (UIs) to a running Base Node; client wallets running on the same machine as the Base Node that want a more direct communication interface to the node than the P2P network provides; third-party applications such as block explorers; and, of course, miners.

A non-exhaustive list of methods the base node module API will expose includes:

Blockchain state calls, including:
- checking whether a given Unspent Transaction Output (UTXO) is in the current UTXO set;
- requesting the latest block height;
- requesting the total accumulated work on the longest chain;
- requesting a specific block at a given height;
- requesting the Merklish root commitment of the current UTXO set;
- requesting a block header for a given height;
- requesting the block header for the chain tip;
- validating signatures for a given transaction kernel;
- validating a new block without adding it to the state tree;
- validating and adding a (validated) new block to the state, and informing of the result (orphaned, fork, re-org, etc.).
Mempool calls
- The number of unconfirmed transactions
- The number of orphaned transactions
- Returning a list of transaction ranked by some criterion (of interest to miners)
- The current size of the mempool (in transaction weight)
Block and transaction validation calls
Block synchronisation calls

RFC-0120/Consensus

Base Layer Full Nodes (Base Nodes)

status: draft

Maintainer(s): Cayle Sharrock and S W van heerden

Licence

The 3-Clause BSD Licence.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

Redistributions of this document must retain the above copyright notice, this list of conditions and the following disclaimer.
Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.

Language

Disclaimer

This document and its content are intended for information purposes only and may be subject to change or update without notice.

Goals

The aim of this Request for Comment (RFC) is to describe the fields that a block should contain as well as all consensus rules that will determine the validity of a block.

Related Requests for Comment

Description

Blocks

Every block MUST conform to the following:

Have a single valid coinbase UTXO and kernel
have a single valid blockheader;
cut-through MUST have been applied where possible;
every UTXO has a valid range proof.

If a block does not confirm to the above, the block should be rejected as invalid and the peer from which it was received marked as a malicious peer.

Blockheaders

Every block header MUST contain the following fields:

version;
height;
prev_hash;
timestamp;
output_mr;
range_proof_mr;
kernel_mr;
total_kernel_offset;
nonce;
pow.

The block header MUST confirm to the following:

The nonce and PoW must be valid for the block header.
All the merkle roots must be valid for the states after the block was applied to the local state.

If the block header does not confirm to any of the above, the block MUST be rejected. If a peer rejects multiple blocks from a given peer, it MAY blacklist the peer and ignore further communication from it.

Version

This is the version currently running on the chain.

The version MUST confirm to the following:

It is represented as an unsigned 16-bit integer.
Version numbers MUST be incremented whenever there is a change in the blockchain schema starting from 1.

Height

A counter indicating how many blocks have passed since the genesis block (inclusive).

The height MUST confirm to the following:

Represented as an unsigned 64-bit integer.
The height MUST be exactly 1 more than the block referenced in the prev_hash block header field.
The genesis block MUST have a height of 0.

Prev_hash

This is the hash of the previous block's header.

The prev_hash MUST confirm to the following:

represented as an array of unsigned 8-bit integers (bytes) in little-endian format.
MUST be a hash of the entire contents of the previous block's header.

Timestamp

This is the timestamp at which the block was mined.

The timestamp MUST confirm to the following:

Must be transmitted as UNIX timestamp.
MUST be less than FTL.
MUST be higher than the MTP.

Output_mr

This is the merkle root of the outputs. This is calculated in the following way: Hash (txo MMR root || roaring bitmap hash of UTXO indices).

The output_mr MUST confirm to the following:

Represented as an array of unsigned 8-bit integers (bytes) in little-endian format.
The hashing function used MUST be blake2b with a 256 bit digest.

Range_proof_mr

This is the merkle root of the range proofs.

The range_proof_mr MUST confirm to the following:

Represented as an array of unsigned 8-bit integers (bytes) in little-endian format.
The hashing function used must be blake2b with a 256 bit digest.

Kernel_mr

This is the merkle root of the outputs.

The kernel_mr MUST confirm to the following:.

Must be transmitted as an array of unsigned 8-bit integers (bytes) in little-endian format.
The hashing function used must be blake2b with a 256 bit digest.

Total_kernel_offset

This is total summed offset of all the transactions contained in this block.

The total_kernel_offset MUST confirm to the following:

Must be transmitted as an array of unsigned 8-bit integers (bytes) in little-endian format

Total_difficulty

This is the total accumulated difficulty of the mined chained.

The total_difficulty MUST confirm to the following:

Must be transmitted as unsigned 64-bit integer.
MUST be larger than the previous block's total_difficulty.
meet the difficulty target for the block as determined by the consensus difficulty algorithm.

Nonce

This is the nonce used in solving the Proof of Work.

The nonce MUST confirm to the following:

Must be transmitted as unsigned 64-bit integer;

PoW

This is Proof of Work algorithm that was used to solve the Proof of Work. This is used in conjunction with the Nonce

The [PoW] MUST contain the following:

accumulated_monero_difficulty as unsigned 64-bit integer.
accumulated_blake_difficulty as unsigned 64-bit integer.
pow_algo as an enum (0 for monero, 1 for blake).
pow_data as array of unsigned 8-bit integers (bytes) in little-endian format.

FTL

The Future Time Limit. This is how far into the future a time is accepted as a valid time. Any time that is more than the FTL is rejected until such a time that it is not less than the FTL. The FTL is calculated as (T*N)/20 with T and N defined as: T: Target time - This is the ideal time that should pass between blocks which have been mined. N: Block window - This is the amount of blocks used when calculating difficulty adjustments.

MTP

The Median Time Past. This is the lower limit of a time. Any time that is less than the MTP is rejected. THe MTP is calculated as the median timestamp of the previous 11 blocks.

Total accumulated proof of work

This is defined as the total accumulated proof of work done on a single block chain. Because we use two proof of work algorithms which are rated at different difficulties and we would like to weight them as equal we need to compare them. To compare them we use a Geometric mean. Because we only have two algorithms this is calculated as the ceil of SQRT(accumulated_monero_difficulty*accumulated_blake_difficulty). This number is never transmitted and is simply calculated from the tip of the chain.

RFC-0130/Mining

Full-node Mining on Tari Base Layer

status: draft

Maintainer(s): [Yuko Roodt] (https://github.com/neonknight64)

Licence

The 3-Clause BSD Licence.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

Redistributions of this document must retain the above copyright notice, this list of conditions and the following disclaimer.
Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.

Language

Disclaimer

This document and its content are intended for information purposes only and may be subject to change or update without notice.

Goals

The aim of this Request for Comment (RFC) is to provide a brief overview of the Tari merged mining process and introduce the primary functionality required of the Mining Server and Mining Worker.

Related Requests for Comment

Description

Assumptions

The Tari blockchain will be merged mined with Monero.
The Tari Base Layer has a network of Base Nodes that verify and propagate valid transactions and blocks.

Abstract

The process of merged mining Tari with Monero on the Tari Base Layer is performed by Mining Servers and Mining Workers. Mining Servers are responsible for constructing new blocks by bundling transactions from the mempool of a connected Base Node. They then distribute Proof-of-Work (PoW) tasks to Mining Workers in an attempt to solve newly created blocks. Solved solutions and shares are sent by the Mining Workers to the Mining Server, which in turn verifies the solution and distributes the newly created blocks to the Base Node and Monero Node for inclusion in their respective blockchains.

Merged Mining on Tari Base Layer

This document is divided into three parts:

A brief overview of the merged mining process and the interactions between the Base Node, Mining Server and Mining Worker.
The primary functionality required of the Mining Server.
The primary functionality required of the Mining Worker.

Overview of Tari Merged Mining Process using Mining Servers and Mining Workers

Mining on the Tari Base Layer consists of three primary entities: the Base Nodes, Mining Servers and Mining Workers. A description of the Base Node is provided in RFC-0110/Base Nodes. A Mining Server is connected locally or remotely to a Tari Base Node and a Monero Node, and is responsible for constructing Tari and Monero Blocks from their respective mempools. The Mining Server should retrieve transactions from the mempool of the connected Base Node and assemble a new Tari block by bundling transactions together.

Mining servers may re-verify transactions before including them in a new Tari block, but this enforcement of verification and transaction rules such as signatures and timelocks is the responsibility of the connected Base Node. Mining Servers are responsible for cut-through, as this is required for scalability and privacy.

To enable merged mining of Tari with Monero, both a Tari and a Monero block need to be created and linked. First, a new Tari block is created and then the block header hash of the new Tari block is included in the coinbase transaction of the new Monero block. Once a new merged mined Monero block has been constructed, PoW tasks can be sent to the connected Mining Workers, which will attempt to solve the block by performing the latest released version of the PoW algorithm selected by Monero.

Assuming the Tari difficulty is less than the Monero difficulty, miners get rewarded for solving the PoW at any difficulty above the Tari difficulty. If the block is solved above the Tari difficulty, a new Tari block is mined. If the difficulty is also greater than the Monero difficulty, a new Monero block is mined as well. In either event, the header for the candidate Monero block is included in the Tari block header.

If the PoW solution was sufficient to meet the difficult level of both the Tari and Monero blockchains, then the individual blocks for each blockchain can be sent from the Mining Server to the Base Node and Monero Node to be added to the respective blockchains.

Every Tari block must include the solved Monero block's information (block header hash, Merkle tree branch and hash of the coinbase transaction) in the PoW summary section of the Tari block header. If the PoW solution found by the Mining Workers only solved the problem at the Tari difficulty, the Monero block can be discarded.

This process will ensure that the Tari difficulty remains independent. Adjusting the difficulty will ensure that the Tari block times are preserved. Also, the Tari block time can be less than, equal to or greater than the Monero block times. A more detailed description of the merged mining process between a Primary and Auxiliary blockchain is provided in the Merged Mining TLU report.

Functionality Required by Tari Mining Server

The Tari blockchain MUST have the ability to be merged mined with Monero.
The Tari Mining Server:
- MUST maintain a local or remote connection with a Base Node and a Monero Node.
- MUST have a mechanism to construct a new Tari and Monero block by selecting transactions from the different Tari and Monero mempools that need to be included in the different blocks.
- MUST apply cut-through when mining Tari transactions from the mempool and only add the excess to the list of new Tari block transactions.
- MAY have a configurable transaction selection mechanism for the block construction process.
- MAY have the ability to re-verify transactions before including them in a new Tari block.
- MUST have the ability to include the block header hash of the new Tari block in the coinbase section of a newly created Monero block to enable merged mining.
- MUST be able to include the Monero block header hash, Merkle tree branch and hash of the coinbase transaction of the Monero block into the PoW summary field of the new Tari block header.
- MUST have the ability to transmit and distribute PoW tasks for the newly created Monero block, which contains the Tari block information, to connected Mining Workers.
- MUST verify PoW solutions received from Mining Workers and MUST reject and discard invalid solutions or solutions that do not meet the minimum required difficulty.
- MAY keep track of mining share contributions of the connected Mining Workers.
- MUST submit completed Tari blocks to the Tari Base Node.
- MUST submit completed Monero blocks to the Monero Network.

Functionality Required by Tari Mining Worker

The Tari Mining Worker:

MUST maintain a local or remote connection to a Mining Server.
MUST have the ability to receive PoW tasks from the connected Mining Server.
MUST have the ability to perform the latest released version of Monero's PoW algorithm on the received PoW tasks.
MUST attempt to solve the PoW task at the difficulty specified by the Mining Server.
MUST submit completed shares to the connected Mining Server.

RFC-0140/SyncAndSeeding

Syncing Strategies and Objectives

status: draft

Maintainer(s): S W van Heerden

Licence

The 3-Clause BSD Licence.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

Redistributions of this document must retain the above copyright notice, this list of conditions and the following disclaimer.
Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.

Language

Disclaimer

This document and its content are intended for information purposes only and may be subject to change or update without notice.

Goals

The aim of this Request for Comment (RFC) is to describe the syncing, seeding, pruning and cut-through process.

Related Requests for Comment

RFC-0110: Base Nodes

Descriptions

Syncing

When a new node comes online, loses connection or encounters a chain reorganization that is longer than it can tolerate, it must enter syncing mode. This will allow it to recover its state to the newest up-to-date state. Syncing can be divided into two SynchronizationStrategys: complete sync and sync. Complete sync means that the node communicates with an archive node to get the complete history of every single block from genesis block. Sync involves the node getting every block from its pruning horizon to current head, as well as every block header from genesis block.

Complete Sync

Complete sync is only available from archive nodes, as these will be the only nodes that will be able to supply the complete history required to sync every block with every transaction from genesis block up onto current head.

Syncing Process

The syncing process MUST be done in the following steps:

Set SynchronizationState to Synchronizing.
Ask peers for their latest block, so it can get the total Proof-of-Work (PoW).
Choose the longest chain based on total PoW done on that chain.
Select a connected peer with the longest chain to sync from. This is based on the following criteria:
- Does the peer have a high enough pruning horizon?
- Does the peer allow syncing?
- Does the peer have a low latency?
Download all headers from genesis block up onto current head, and validate the headers as you receive them.
Download Unspent Transaction Output (UTXO) set at pruning horizon.
Download all blocks from pruning horizon up to current head. If the node is doing a complete sync, the pruning horizon will be infinite, which means you will download all blocks ever created.
Validate blocks as if they were just mined and then received, in chronological order.

After this process, the node will be in sync, and will be able to process blocks and transactions normally.

Keeping in Sync

The node SHOULD periodically test its peers with ping messages to ensure that they are alive. When a node sends a ping message, it MUST include the current total PoW, hash of the current head and genesis block hash of its own current longest chain in the ping message. The receiving node MUST reply with a pong message, also including the total PoW, current head and genesis block hash of its longest chain. If the two chains do not match up, the node with the lowest PoW is responsible for asking the peer for syncing information and set SynchronizationState to Synchronizing.

If the genesis block hashes do not match, the node is removed from its peer list, as this node is running a different blockchain.

This will be handled by the node asking for each block header from the current head, going backward for older blocks, until a known block is found. If a known block is found, and if it has missing blocks, it MUST set SynchronizationState to Synchronizing while it is busy catching up those blocks.

If no block is found, the node will enter sync mode and resync. It cannot recover from its state, as the fork is older than its pruning horizon.

Chain Forks

Chain forks can be a problem, since in Mimblewimble not all nodes keep the complete transaction history. The design philosophy is more along the lines of only keeping the current [Blockchain state]. However, if such a node only maintains the current [Blockchain state], it is not possible for the node to "rewind" its state to handle forks in the chain history. In this case, a mode must resync its chain to recover the necessary transaction history up onto its pruning horizon.

To counter this problem, we use pruning horizon. This allows every node (Base Node) to be a "light" archival node. This in effect means that the node will keep a full history for a short while. If the node encounters a fork, it can easily rewind its state to apply the fork. If the fork is longer than the pruning horizon, the node will enter a sync state, where it will resync.

Pruning and Cut-through

In Mimblewimble, the state can be completely verified using the current UTXO set (which contains the output commitments and range proofs), the set of excess signatures (contained in the transaction kernels) and the PoW. The full block and transaction history is not required. This allows base layer nodes to remove old used inputs from the blockchain and or the mempool. Cut-through happens in the mempool while pruning happens in the blockchain with already confirmed transactions. This will remove the spent inputs and outputs, but will retain the excesses of each transaction.

Pruning is only for the benefit of the local Base Node, as it reduces the local blockchain size. Pruning only happens after the block is older than the pruning horizon height. A Base Node will either run in archive mode or prune mode. If the Base Node is running in archive mode, it MUST NOT prune.

When running in pruning mode, Base Nodes MUST remove all spent outputs that are older than the pruning horizonin their current stored UTXO when a new block is received from another Base Node.

RFC-0150/Wallets

Base Layer Wallet Module

status: draft

Maintainer(s): Yuko Roodt, Cayle Sharrock

Licence

The 3-Clause BSD Licence.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

Redistributions of this document must retain the above copyright notice, this list of conditions and the following disclaimer.
Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.

Language

Disclaimer

This document and its content are intended for information purposes only and may be subject to change or update without notice.

Goals

The aim of this Request for Comment (RFC) is to propose the functionality and techniques required by the Base Layer Tari wallet module. The module exposes the core wallet functionality on which user-facing wallet applications may be built.

Related Requests for Comment

RFC-0100: Base Layer

This RFC is derived from a proposal first made in this issue.

Description

Key Responsibilities

The wallet software is responsible for constructing and negotiating transactions for transferring and receiving Tari coins on the Base Layer. It should also provide functionality to generate, store and recover a master seed key and derived cryptographic key pairs that can be used for Base Layer addresses and signing of transactions.

Details of Functionality

A detailed description of the required functionality of the Tari software wallet is provided in three parts:

basic transaction functionality;
key management features; and
the different methods for recovering the wallet state of the Tari software wallet.

Basic Transaction Functionality

It MUST be able to send and receive Tari coins using Mimblewimble transactions.
It SHOULD be able to establish a connection between different user wallets to negotiate:
- the construction of a transaction; and
- the signing of multi-signature transactions.
The Tari software wallet SHOULD be implemented as a library or Application Programming Interface (API) so that Graphic User Interface (GUI) or Command Line Interface (CLI) applications can be developed on top of it.
It MUST be able to establish a connection to a Base Node to submit transactions and monitor the Tari blockchain.
It SHOULD maintain an internal ledger to keep track of the Tari coin balance of the wallet.
It MAY offer transaction fee estimation, taking into account:
- transaction byte size;
- network congestion; and
- desired transaction priority.
It SHOULD be able to monitor and return the states (Spent, Unspent or Unconfirmed) of previously submitted transactions by querying information from the connected Base Node.
It SHOULD present the total Spent, Unspent or Unconfirmed transactions in summarized form.
It SHOULD be able to update its software to patch potential security vulnerabilities. Automatic updating SHOULD be selected by default, but users can decide to opt out.
Wallet features requiring querying a base node for information SHOULD have caching capabilities to reduce bandwidth consumption.

Key Management Features

It MUST be able to generate a master seed key for the wallet by using:
- input from a user (e.g. when restoring a wallet, or in testing); or
- a user-defined set of mnemonic word sequences using known word lists; or
- a cryptographically secure random number generator.
It SHOULD be able to generate derived transactional cryptographic key pairs from the master seed key using deterministic key pair generation.
It SHOULD store the wallet state using a password or passphrase encrypted persistent key-value database.
It SHOULD provide the ability to back up the wallet state to a single encrypted file to simplify wallet recovery and reconstruction at a later stage.
It MAY provide the ability to export the master seed key or wallet state as a printable paper wallet, using coded markers.

Different Methods for Recovering Wallet State of Tari Software Wallet

It MUST be able to reconstruct the wallet state from a manually entered master seed key.
It MUST have a mechanism to systematically search through the Tari blockchain and mempool for unspent and unconfirmed transactions, using the keys derived from the master key.
The master seed key SHOULD be derivable from a specific set of mnemonic word sequences using known word lists.
It MAY enable the reconstruction of the master seed key by scanning a coded marker of a paper wallet.

TransactionProtocol

Transaction Protocol

status: draft

Maintainer(s): Cayle Sharrock

Licence

The 3-Clause BSD Licence.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

Redistributions of this document must retain the above copyright notice, this list of conditions and the following disclaimer.
Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.

Language

Disclaimer

This document and its content are intended for information purposes only and may be subject to change or update without notice.

Goals

This Request for Comment (RFC) describes the transaction protocol for peer-to-peer Tari payments using the Mimblewimble protocol. It also considers some attacks that may be launched against the protocol and offers some discussion around those attacks and potential alternatives to the protocol.

The goal is to describe a transaction protocol that:

permits multiple recipients;
preserves privacy regarding how many parties are involved in the transaction; and
is secure against all reasonable attacks.

Related Requests for Comment

RFC-0100: The Base Layer

Description

The Tari base layer is built using Mimblewimble, which requires that all parties involved in a Tari transfer must interact to construct a valid Mimblewimble transaction.

A valid transaction involves:

a set of one or more inputs being spent by the Sender;
a set of zero or more outputs being sent to the Sender;
a set of recipients, each of whom MUST construct exactly one output; and
a set of partial Schnorr signatures which, when aggregated, validates the transaction construction and indicates every party's satisfaction with the terms.

The Issue with Multiple Recipients

Each party involved in a Tari transaction must produce a partial signature, signing the same challenge. This challenge is defined as

$$ e = H(\Sigma R_i \Vert \Sigma P_i \Vert m) $$

where $R_i$ are public nonces generated by each party for this signature; $P_i$ are the public Spending Keys; and m is the additional metadata for the transaction. $\Sigma P_i$ is the value of the (pre-offset) excess that is stored in the transaction kernel.

Notice that every signing party needs to know the sum of all the nonces and public spending keys. This suggests that every party knows how many parties are involved in the transaction, which is not an ideal privacy scenario. It would be preferable if a secure scheme could be found where each recipient interacts only with the sender and does not need to calculate these sums themselves.

Unfortunately, as discussed below, it seems that using any known scheme, it's not possible to satisfy this privacy goal while achieving the desired security level.

The Issue with Multiple Senders

To increase privacy, the public excess values are offset by a constant random value. The choice of this value, as well as fee selection, can only be set once per transaction. The privilege of selecting these values is generally bestowed on the sender, since the sender pays the fee. Allowing multiple sending parties (or equivalently, allowing recipients to provide inputs) would require a negotiation round to set the fee and offset before the transaction could be constructed. This is a complication we don't want to deal with, and so all schemes presented here allow exactly one sender.

Two-party Transactions

Two-party transactions are fairly straightforward and are described in detail by Tari Labs University (TLU). (Refer to Mimblewimble Transaction.)

It is proposed that Tari implement this single-round two-party transaction scheme as a special case to support both online two-party transactions as well as "offline" transactions such as via email, text message and carrier pigeon.

Multiple-recipient Transaction Scheme

sequenceDiagram participant Sender participant Receivers # Sender Initializes the transaction activate Sender Sender-->>Sender: initialize deactivate Sender # Sender transmits initial data to all receivers activate Sender Sender-->>+Receivers: [tx_id, amt_i,H(Rs||Xs),m] note left of Sender: CollectingCommitments note right of Receivers: SendingCommitment Receivers-->>-Sender: [tx_id, H(Ri||Pi)] deactivate Sender # Receiving Outputs activate Sender Sender-->>+Receivers: [tx_id, [Hashed commitments]] note left of Sender: CollectingPubKeys note right of Receivers: SendingOutput Receivers-->>-Sender: [tx_id, Pi, Ri, C_i, RP_i] deactivate Sender alt inflation_error()? Sender--XReceivers: [tx-id, failed] end # Signature collection activate Sender Sender-->>+Receivers: [tx_id, [Rs, Ri] [Xs, Pi]] note left of Sender: CollectingSignatures note right of Receivers: Signing Receivers-->>Receivers: validate nonces, pubkeys and pubkey sum alt invalid Receivers--XSender: failed end Receivers-->>Receivers: Sign Receivers-->>-Sender: [tx_id, s_i] deactivate Sender # Sender sends final notification of result note left of Sender: Finalizing alt is_valid() Sender-->>Receivers: [tx_id, OK] else invalid Sender--XReceivers: [tx_id, Failed] end

** Legend **

Symbol	Meaning
tx_id	Transaction identifier
amt_i	Amount sent to i-th recipient
Rs, Ri	Public nonce
Xs, Pi	Public excess/key
m	Message metadata
C_i	Commitment
RP_i	Range proof
[..]	Vector of data

Transaction ID

The scheme above makes use of a tx_id field in every peer-to-peer message. Since all messages are stateless and asynchronous, peers need some way of figuring out which message refers to which transaction. The transaction ID fulfils this role.

The ID does not appear on the blockchain in any manner; is purely used to disambiguate Tari transaction messages and can be discarded after the transaction is broadcast to the network.

The tx_id is unique for every receiver so that any observers of the communication will not be able to group receivers together (however, the communication should be over secure channels in general).

The format of the transaction ID is a four-byte, little-endian integer (u64) and is calculated as

H(Rs||i)[0..4]

where i is the i-th recipient in the transaction. The sender can use the tx_id as a hash map key to identify and differentiate recipients.

Replay Attacks

If any party can be convinced to sign a different message with the same nonce, its private keys will be lost. One way of achieving this would be if a virtual machine could be "snapshotted" or otherwise cloned at any point between sharing the public nonce and signing the message. Both copies of the victim's machine will now continue, unaware that there's a copy participating in a signature round. What then happens is:

$$ \begin{align} &\text{Clone A} & &\text{Clone B} \\ e_1 &= H(r_1 \Vert r_s \Vert \dots) & e_2 &= H(r_1 \Vert r_s^* \Vert \dots) \\ s_1 &= r_1 + e_1 \cdot k_1 & s_2 &= r_1 + e_2 \cdot k_1 \\ \end{align} $$

The attacker receives both signatures and trivially calculates the secret key:

$$ \begin{align} \Delta s &= s_1 - s_2 \\ &= k_1(e_1 - e_2) = k_1\Delta e \\ \Rightarrow k_1 &= \frac{\Delta s}{\Delta e} \end{align} $$

We've demonstrated this with the attacker changing their nonce, but literally any alteration to the challenge will provide a new challenge $e_2$, enabling the attack.

What can we do about this? In fact, it's not possible to eliminate this attack at all! The reason sits with the proof that the Schnorr scheme works as a zero-knowledge protocol; the demonstration of this proof is precisely the attack we're trying to avoid [GOL19]. If we could eliminate this attack, we'd need to come up with a completely different way of proving the zero-knowledge property.

So we can't stop it, but we can make it as tricky as possible for the attacker to trick the receiver into replaying the signature. MuSig does this by requiring parties to share the hash of their nonces beforehand. At its extreme: in the two-party, single-round scheme, for example, the attacker would need to be able to control the victim's machine code execution (like running a debugger), at which point one might think the attacker could read the private key directly from memory anyway.

Rogue Key Attacks

Rogue Key attacks are another type of attack that can occur in multi-signature schemes.

In this case, the attacker has the freedom to choose a key or nonce after the victim has already disclosed theirs. This may allow the attacker to forge a valid signature on behalf of the victim. A recent paper, [DRI19], suggests that any Schnorr-based, two-round multi-signature scheme is vulnerable to a rogue key attack.

How this might apply in an insecure two-round Tari multi-signature scheme is as follows: A receiver sends their public nonce; output commitment and range proof; and public spending key to the sender, but then decides to cancel the transaction by refusing to provide a signature and sending an "Abort" message to the sender instead. The sender could, if they wanted, forge the 2-of-2 signature using this rogue key attack and broadcast the transaction anyway.

Note: This attack is not applicable in the one-round, two-party scheme, since the receiver returns their information in an all-or-nothing manner. However, the receiver could attempt to forge a signature, since they have the Sender's public nonce, but there's nothing they can really do with this signature; they certainly cannot broadcast a transaction with it because they don't have any of the transaction data at this stage.

We avoid rogue-key attacks in the Tari multi-recipient scheme by employing three rounds. In the first round, parties share a hash of their public nonces, which each party can later use to verify that no nonces were changed after the actual public nonces were shared.

RFC-0152/EmojiId

Emoji Id specification

Status: Draft

Maintainer(s):Cayle Sharrock

Licence

The 3-Clause BSD Licence.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

Redistributions of this document must retain the above copyright notice, this list of conditions and the following disclaimer.
Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.

Language

Disclaimer

This document and its content are intended for information purposes only and may be subject to change or update without notice.

This document may include preliminary concepts that may or may not be in the process of being developed by the Tari community. The release of this document is intended solely for review and discussion by the community regarding the technological merits of the potential system outlined herein.

Goals

This document describes the specification for Emoji Ids. Emoji Ids are encoded node ids used for humans to easily verify peer node addresses.

Related Requests for Comment

None

Description

Tari Communication Nodes are identified on the network via their Node ID; which in turn are derived from the node's public key. Both the node id and public key are simple large integer numbers.

The most common practice for human beings to copy large numbers in cryptocurrency software is to scan a QR code or copy and paste a value from one application to another. These numbers are typically encoded using hexadecimal or Base58 encoding. The user will then typically scan (parts) of the string by eye to ensure that the value was transferred correctly.

For Tari, we propose encoding values, the node ID in particular, using emoji. The advantages of this approach are:

Emoji are more easily identifiable; and if selected carefully, less prone to identification errors (e.g. mistaking an O for a 0).
The alphabet can be considerably larger than hexadecimal (16) or Base58 (58), resulting in shorter character sequences in the encoding.

The specification

The emoji character map

An emoji alphabet of 1,024 characters is selected. Each emoji is assigned a unique index from 0 to 1023 inclusive. This list is the emoji map. For example,

😀 => 0
😘 => 1
...
🦊 => 1023

The emoji SHOULD be selected such that

Similar looking emoji are excluded from the map. e.g. Neither 😁 or 😄 should be included. Similarly the Irish and Côte d'Ivoirean flags look very similar, and both should be excluded.
Modified emoji (skin tones, gender modifiers) are excluded. Only the "base" emoji is considered.

Encoding

The essential strategy in the encoding process is to map a sequence of 8-bit values onto a 10-bit alphabet. The general encoding procedure is as follows:

Given a large integer value, represented as a byte array, S, in little-endian format (most significant digit last). Assume the string is addressable, i.e. S[i] is the ith byte in the array.

Set CURSOR to 0, Set L to a multiple of 10 that is <= len(S).
Set IDX to [] (an empty array)
While CURSOR < L:
- Set L <= S[CURSOR/8 + 1], the current low byte; if the index would overflow, set L to zero.
- Set H <= S[CURSOR/8], the current high byte
- Set n <= CURSOR % 8, the position of the cursor in the current high byte
- Set i <= ((H as u8) << n) << 2 + (L >> (6 - n)), where the first shift left (H as u8 <<n) is on a one-byte width (effectively losing the first n bits) and the second shift left is on a 8-byte width (u64).
- Push i onto IDX
- CURSOR <= CURSOR + 10
Return IDX

The emoji string is created by mapping the IDX array to the emoji map.

Emoji ID definition

The emoji ID is an emoji string of 12 characters. Each character encodes 10 bits according to the bitmap:

+---------------------+------------------+-------------------+
|  Node Id (104 bits) | Version (6 bits) | Checksum (10 bits)|
+---------------------+------------------+-------------------+

The emoji ID is calculated from a 104-bit node id represented as 13 bytes (B) as follows:

Take the current emoji ID version number, v and add v << 2 as an additional byte to B. This "right-pads" the version in the last byte. This is necessary since we have a 14 byte (112 bit) sequence, which is not divisible by 10. This padding sets the last 2 bits, which will be discarded, to zero.
Encode B into an emoji string with L = 11.
Calculate a 12th emoji using the Luhn mod 1024 checksum algorithm.

Decoding

One can extract the node id from an emoji ID as follows:

Calculate the checksum of the first 11 emoji using the Luhn mod 1024 algorithm. If it does not match the 12th emoji, return with an error. if any emoji character is not in the emoji map, return an error.
Extract the version number:
1. Do a reverse lookup of emoji[10] to find its index. Store this u64 value in I.
2. The Version number is (I && 0x3F) >> 2. This can be used to set the Emoji map accordingly (and may have to be done iteratively, since the version is encoded into the emoji string).
Set CURSOR = 0.
Set B = [], and empty byte array
While CURSOR <= 11:
1. Set k <= CURSOR * 2
2. Do a reverse lookup of the emoji[CURSOR] to find its index. Store this u64 value in L.
3. If k > 0, set H to the reverse lookup index of emoji[CURSOR-1] as u8 (first 2 bits are discarded), else H=0.
4. Set v = ((H as u8) << (8-k)) + (L >> (2+k)). Push v onto tho B.
5. Set CURSOR <= CURSOR + 1

If the algorithm completes, B holds the node ID.

Versioning

The current emoji ID version number is 1. If the emoji alphabet changes, the version number MUST be incremented. This will usually cause incompatible versions of the emoji ID to be detected. However, this is not fail-safe.

The last 6 bits of the 11th emoji encodes the version; this means that the first 4 bits are part of the node ID. On a reverse mapping, there is a chance that the reverse mapping would offer a valid, but incorrect version number if the new mapping are not chosen carefully.

Example.

In version 1, 😘 => 0b0000_000001 = 1 in the map. Seeing 😘 as the 11th emoji in a string would result in a version code of 1, which is consistent and expected.

However, in unlucky version 13, if 😘 moves in the map to number 13 (0b0000_001101), the version decoding would also be valid and thus we wouldn't be able to unambiguously identify the version.

RFC-0170/NetworkCommunicationProtocol

The Tari Communication Network and Network Communication Protocol

status: draft

Maintainer(s): Yuko Roodt

License

The 3-Clause BSD License.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

Redistributions of this document must retain the above copyright notice, this list of conditions and the following disclaimer.
Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.

THIS DOCUMENT IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

Language

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 (covering RFC2119 and RFC8174) when, and only when, they appear in all capitals, as shown here.

Disclaimer

The purpose of this document and its content is for information purposes only and may be subject to change or update without notice.

This document may include preliminary concepts that may or may not be in the process of being developed by the Tari community. The release of this document is intended solely for review and discussion by the community regarding the technological merits of the potential system outlined herein.

Goals

This document will introduce the Tari communication network and the communication protocol used to select, establish and maintain connections between peers on the network. Communication Nodes and Communication Clients will be introduced and their required functionality will be proposed.

Related RFCs

Description

Assumptions

A communication channel can be established between two peers once their online communication addresses are known to each other.
A Validator Node is able to derive a node ID from the Validator Node registration transaction on the Base Layer.

Abstract

The backbone of the Tari communication network consists of a large number of nodes that maintain peer connections between each other. These nodes forward and propagate encrypted and unencrypted data messages through the network such as joining requests, discovery requests, transactions and completed blocks. Network clients, not responsible for maintaining the network, are able to create ad hoc connections with nodes on the network to perform joining and discovery requests. The majority of communication between clients and nodes will be performed using direct Peer-to-peer (P2P) communication once the discovery process was used to obtain the online communication addresses of peers. Where possible the efficient Kademlia based directed forwarding of encrypted data messages can be used to perform quick node discovery and joining of clients and nodes on the Tari communication network. Where messages are of importance to a wide variety of entities on the network, Gossip protocol based message propagation can be performed to distribute the message to the entire network.

Overview

The Tari communication network is a variant of a Kademlia network that allows for fast discovery of nodes, with an added ability to perform Gossip protocol based broadcasting of data messages to the entire network. The majority of the communication required on the Base Layer and Digital Asset Network (DAN) will be performed via direct P2P communication between known clients and nodes. Alternatively, the Tari communication network can be used for broadcasting joining requests, discovery requests and propagating data messages such as completed blocks, transactions and data messages that are of interest to a large part of the Tari communication network.

The Tari communication network consists of a number of different entities that need to communicate in a distributed and ad-hoc manner. The primary entities that need to communicate are Validator Nodes (VN), Base Nodes (BN), Wallets (W) and Token Wallets (TW). Here are some examples of different communication tasks that need to be performed by these entities on the Tari Communication network:

Base Nodes on the Base Layer need to propagate completed blocks and transactions to other Base Nodes using Gossip protocol based broadcasting.
Validator Nodes need to efficiently discover other Validator Nodes in the process of forming Validator Node committees.
Wallets need to communicate and negotiate with other Wallets to create transactions. They also need the ability to submit transactions to the mempool of Base Nodes.
Token Wallets need to communicate with Validator Node committees and other Token Wallets to construct and send DAN instructions.

Here is an overview communication matrix that show which source entities SHOULD initiate communication with destination entities on the Tari Communication network:

\Destination Source	Validator Node	Base Node	Wallet	Token Wallet
Validator Node	Yes	Yes	Yes	Yes
Base Node	No	Yes	No	No
Wallet	No	Yes	Yes	No
Token Wallet	Yes	No	Yes	Yes

Communication Nodes and Communication Clients

To simplify the description of the Tari communication network, the different entities with similar behaviour were grouped into two groups: Communication Nodes and Communication Clients. Validator Nodes and Base Nodes are Communication Nodes (CN). Wallets and Token Wallets are Communication Clients (CC). CNs form the core communication infrastructure of the Tari communication network and are responsible for maintaining the Tari communication network by receiving, forwarding and distributing joining requests, discovery requests, data messages and routing information. CCs are different from CNs in that they do not maintain the network and they are not responsible for propagating any joining requests, discovery requests, data messages and routing information. They do make use of the network to submit their own joining requests and perform discovery request of other specific CNs and CCs when they need to communicate with them. Once a CC has discovered the CC or CN they want to communicate with, they will establish a direct P2P channel with them. The Tari communication network is unaware of this direct P2P communication once discovery is completed.

The different entity types MUST be grouped into the different communication node types as follows:

Entity Type	Communication Node Type
Validator Node	Communication Node
Base Node	Communication Node
Wallet	Communication Client
Token Wallet	Communication Client

Unique identification of Communication Nodes and Communication Clients

In the Tari communication network, each CN or CC makes use of a node ID to determine their position in the network. This node ID is either assigned based on registration on the Base Layer or can be derived from the CNs or CCs identification public key. The method used to obtain a node ID will either enhance or limit the trustworthiness of that entity when propagating messages through them on the Tari communication network. When performing the broadcasting of data messages or propagating discovery requests, nodes with registration assigned node IDs are considered more trustworthy compared to nodes with derived node IDs.

Obtaining a node ID from registration on the Base Layer is important as it will limit the potential of some parties performing Eclipse attacks on the network. Registration makes it more difficult for Bad Actors to position themselves in ideal patterns on the network to perform disruptive operations and actions. In sensitive situations or situations where the Kademlia-style directed propagation of messages are vulnerable, gossip protocol-based broadcasting of messages can be performed as a less efficient, but safer alternative to ensure that the message will successfully reach the rest of the network.

The similarity or distance between different node IDs can be calculated by performing the Hamming distance between the bits of the two node ID numbers. The Hamming distance can be implemented as an Exclusive OR (XOR) between the bits of the numbers and the summation of the resulting true bits. CCs and/or CNs that have similar node IDs, that produce a small Hamming distance, are located in similar regions of the Tari communication network. This does not mean that their geographic locations are near each other, but rather that their location in the network is similar. A thresholding scheme can be applied to the Hamming distance to ensure that only neighboring CNs with similar node IDs are allowed to share and propagate specific information. As an example, only routing table information that contains similar node IDs to the requesting CCs or CNs node ID should be shared with them. Limiting the sharing of routing table information makes it more difficult to map the entire Tari communication network.

The recommended method of node ID assignment for each Tari communication network entity type MUST be implemented as follows:

Entity Type	Communication Node Type	Node ID Assignment
Validator Node	Communication Node	Registration
Base Node	Communication Node	Derived
Wallet	Communication Client	Derived
Token Wallet	Communication Client	Derived

Note that Mining Servers and Mining Workers are excluded from the Tari communication network. A Mining Worker will have a local or remote P2P connection with a Mining Server. A Mining Server will have a local or remote P2P connection with a Base Node. They do not need to make use of the communication network and they are not responsible for propagating any messages on the network. The parent Base Node will perform any communication tasks on the Tari communication network on their behalf.

Online Communication Address, Peer Address and Routing Table

Each CC and CN on the Tari communication network will have identification cryptographic keys, a node ID and an online communication address. The online communication address SHOULD be either an IPv4, IPv6, URL, Tor (Base32) or I2P (Base32) address and can be stored using the network address type as follows:

Description	Data type	Comments
address type	uint4	Specify if IPv4/IPv6/Url/Tor/I2P
address	char array	IPv4, IPv6, URL, Tor (Base32), I2P (Base32) address
port	uint16	port number

The address type is used to determine how to interpret the address characters. An I2P address can be interpreted as "{52 address characters}.b32.i2p". The Tor address should be interpreted as "http://{16_or_52_address_chars}.onion/". The IPv4 and IPv6 address can be stored in the address field without modification. URL addresses can be used for nodes with dynamic IP addresses.

A Tor or I2P address can be used when anonymity is important for a CC or CN. The IPv4, IPv6 and URL address types do not provide any privacy features but do provide increased bandwidth.

Each CC or CN has a local routing table that contains the online communication addresses of all CCs and CNs on the Tari communication network known to that CC or CN. When a CC or CN wants to join the Tari communication network, the online communication address of at least one other CN that is part of the network needs to be known. The online communication address of the initial CN can either be manually provided or a bootstrapped list of "reliable" and persistent CNs can be provided with the Validator Node, Base Node, Wallet or Token Wallet software. The new CC or CN can then request additional peer contact information of other CNs from the initial peers to extend their own routing table.

The routing table consists of a list of peer addresses that link node IDs, public identification keys and online communication addresses of each known CC and CN.

The Peer Address stored in the routing table MAY be implemented as follows:

Description	Data type	Comments
network address	network_address	The online communication address of the CC or CN
node_ID	node_ID	Registration Assigned for VN, Self selected for BN, W and TW
public_key	public_key	The public key of the identification cryptographic key of the CC or CN
node_type	node_type	VN, BN, W or TW
linked asset IDs	list of asset IDs	Asset IDs can be used as an address on Tari network similar to a node ID
last_connection	timestamp	Time of last successful connection with peer
update_timestamp	timestamp	A timestamp for the last peer address update

When a new CC or CN wants to join the Tari communication network they need to submit a joining request to the rest of the network. The joining request contains the peer address of the new CC or CN. Each CN that receives the joining request can decide if they want to add the new CCs or CNs contact information to their local routing table. When a CN, that received the joining request, has a similar node ID to the new CC or CN then that node must add the peer address to their routing table. All CNs with similar node IDs to the new CC or CN should have a copy of the new peer address in their routing tables.

To limit potential attacks, only one registration for a specific node type with the same online communication address can be stored in the routing table of a CN. This restriction will limit Bad Actors from spinning up multiple CNs on a single computer.

Joining the Network using a Joining Request

A new CC or CN needs to register their peer address on the Tari communication network. This is achieved by submitting a network joining request to a subset of CNs selected from the routing table of the new CC or CN. These peers will forward the joining request, with the peer address, to the rest of the network until CNs with similar node IDs have been reached. CNs with similar node IDs will then add the new peer address of the new node to their routing table, allowing for fast discovery of the new CC or CN.

Other CCs and CNs will then be able to retrieve the new CCs or CNs peer address by submitting discovery requests. Once the peer address of the desired CC or CN has been discovered then a direct P2P communication channel can be established between the two parties for any future communication. After discovery, the rest of the Tari communication network will be unaware of any further communication between the two parties.

Sending Data Messages and Discovery Requests

The majority of all communication on the Tari communication network will be performed using direct P2P channels established between different CCs and CNs once they are aware of the peer addresses of each other that contain their online communication addresses. Message propagation on the network will typically consist only of joining and discovery requests where a CC or CN wants to join the network or retrieve the peer address of another CC or CN so that a direct P2P channel can be established.

Messages can be transmitted in this network in either an unencrypted or encrypted form. Typically messages that have been sent in unencrypted form are of interest to a number of CNs on the network and should be propagated so that every CN that is interested in that data message obtains a copy. Block and Transaction propagation are examples of data messages where multiple entities on the Tari communication network are interested in that data message; this requires propagation through the entire Tari communication network in unencrypted form.

Encrypted data messages make use of the source and destinations identification cryptographic keys to construct a shared secret with which the message can be encoded and decoded. This ensures that only the two parties are able to decode the data message as it is propagated through the communication network. This mechanism can be used to perform private discovery requests, where the online communication address of the source node is encrypted and propagated through the network until it reached the destination node. Private discovery requests can only be performed if both parties are online at the same time. Encryption of the data message ensures that only the destination node is able to view the online address of the source node as the data message moves through the network. Once the destination node receives and decrypts the data message, that node is then able to establish a P2P communication channel with the source node for any further communication.

Propagation of completely private discovery request, hidden as an encrypted data message, can be performed as a broadcast through the entire network using the Gossip protocol. Propagation of public discovery requests can be performed using more efficient directed propagation using the Kademlia protocol. As encrypted message with visible destinations tend to not be of interest to the rest of the network, directed propagation using the Kademlia protocol to forward these messages to the correct parties are preferred. Privacy of a CCs online address, whom is sending a transaction to a Base Node, may be enhanced if the transaction is encrypted and sent to a Base Node with a node ID that is not the closest to the CCs node ID. This should prevent linking a transaction to the originating online address.

This same encryption technique can be used to send encrypted messages to a single node or a group of nodes, where the group of nodes have shared identification keys. A Validation Committee is an example of a group of CNs that have shared identification keys for the committee. The shared identification keys ensure that all members of that committee are able to receive and decrypt data messages that were sent to the committee.

Maintaining connections with peers

CCs and CNs establish and maintain connections with peers differently. CCs only create a few short-lived ad hoc channels and CNs create and maintain long-lived channels with a number of peers.

If a CC is unaware of a destination CNs or CCs online communication address then the address first needs to be obtained using a discovery request. When a CC already knows the communication address of the CC or CN that he wants to communicate with, then a direct P2P channel can be established between the two peers for the duration of the communication task. The communication channel can then be closed as soon as the communication task has been completed.

CNs consisting of VNs and BNs typically attempt to maintain communication channels with a large number of peers. The distribution of peers (VNs vs BNs) that a single CN keeps communication channels open with can change depending on the type of node. A CN that is also a BN should maintain more peer connections with other BNs, but should also have some connections with other VNs.

Having some connections with VNs are important as BNs have derived node IDs and not registered node IDs such as VNs, making it possible for the CN to be separated from the main network and become victim to an Eclipse attack. Having some connections with VNs will make it more difficult to separate the CN from the network and will ensure successful propagation of transactions and completed blocks from that CN.

A CN that is also a VN should maintain more peer connections with other VNs, but also have some connections with BNs. CNs that are part of Validator Node committees should attempt to maintain permanent connections with the other members of the committee to ensure that quick consensus can be achieved.

To maintain connections with peers, the following process can be performed. Discover peers using discovery requests, and add their details to the local routing table. The CN can decide how the peer connections should be selected from the routing table by either:

manually selecting a subset,
automatically selecting a random subset or
selecting a subset of neighbouring nodes with similar node IDs.

Maintaining communication channels is important and the following process can be followed in an attempt to keep peer connections alive: For an existing peer connection. When more than 30 minutes have passed since the last communication with that peer, then a heartbeat message should be sent to the peer in an attempt to keep the connection with the peer alive. If the peer connection was not established for a specific purpose, such as with the connections between committee members, then a new replacement peer can be selected from the local routing table. A new connection can then be established and maintained between the current node and the newly selected node. If that specific connection is important, such as with the connections between committee members, then the current CN or CC must wait and attempt to create a new connection with that same peer. If more than 90 minutes have passed since the last successful communication with the peer node, a new discovery request can be sent on the Tari communication network in an attempt to locate that peer again. Losing of a peer might happen in cases where the CN or CC went temporarily offline and their dynamic communication address changed, requiring the discovery process to be performed again before a direct P2P communication channel can be established.

Functionality Required of Communication Nodes

It MUST select a cryptographic key pair used for identification on the Tari Communication network.
A CN MAY request the peer addresses of CNs with similar node IDs from other CNs to extend their local routing table.
If a CN is a VN, then a node ID MUST be obtained by registering on the Base layer.
If a CN is a BN, then a node ID MUST be derived from the nodes identification public key.
A new CN MUST submit a joining request to the Tari communication network so that the nodes peer address can be added to the routing table of neighbouring peers in the network.
If a CN receives a new joining request with a similar node ID (within a network selected threshold), then the peer address specified in the joining request MUST be added to its local routing table.
When a CN receives an encrypted message, the node MUST attempt to open the message.
When a CN receives an encrypted message that the node is unable to open, and the destination node ID is known then the CN MUST forward it to all connected peers that have node IDs that are closer to the destination.
When a CN receives an encrypted message that the node is unable to open and the destination node is unknown then the CN MUST forward the message to all connected peers.
A CN MUST have the ability to verify the content of unencrypted messages to limit the propagation of spam messages.
If an unencrypted message is received by the CN with a unspecified destination node ID, then the node MUST verify the content of the message and forward the message to all connected peers.
If an unencrypted message is received by the CN with an specified destination node ID, then the node MUST verify the content of the message and forward the message to all connected peers that have closer node IDs.
A CN MUST have the ability to select a set of peer connections from its routing table.
Connections with the selected set of peers MUST be maintained by the CN.
A CN MUST have a mechanism to construct encrypted and unencrypted joining requests, discovery requests or data messages.
A CN MUST construct and provide a list of peer addresses from its routing table that is similar to a requested node ID so that other CCs and CNs can extend their routing tables.
A CN MUST keep its routing table up to date by removing unreachable peer addresses and adding newly received addresses.
It MUST have a mechanism to determine if a node ID was obtained through registration or was derived from an identification public key.
A CN MUST calculate the similarity between different node IDs by calculating the Hamming distance between the bits of the two node ID numbers.

Functionality Required of Communication Clients

It MUST select a cryptographic key pair used for identification on the Tari Communication network.
It MUST have a mechanism to derive a node ID from the self-selected identification public key.
A CC must have the ability to construct a peer address that links its identification public key, node ID and an online communication address.
A new CC MUST broadcast a joining request with its peer address to the Tari communication network so that CNs with similar node IDs can add the peer address of the new CC to their routing tables.
A CC MAY request the peer addresses of CNs with similar node IDs from other CNs to extend their local routing table.
A CC MUST have a mechanism to construct encrypted and unencrypted joining and discovery requests.
A CC MUST maintain a small persistent routing table of Tari Communication network peers with which ad hoc connections can be established.
As the CC becomes aware of other CNs and CCs on the communication network, the CC SHOULD extend its local routing table by including the newly discovered CCs or CNs contact information.
Peers from the CCs routing table that have been unreachable for a number of attempts SHOULD be removed from the its routing table.
A CC MUST calculate the similarity between different node IDs by calculating the Hamming distance between the bits of the two node ID numbers.

RFC-0171/MessageSerialization

Message Serialization

status: draft

Maintainer(s): Cayle Sharrock

Licence

The 3-Clause BSD Licence.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

Redistributions of this document must retain the above copyright notice, this list of conditions and the following disclaimer.
Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.

Language

Disclaimer

This document and its content are intended for information purposes only and may be subject to change or update without notice.

Goals

The aim of this Request for Comment (RFC) is to describe the message serialization formats for message payloads used in the Tari network.

Related Requests for Comment

RFC-0710: Tari Communication Network and Network Communication Protocol

Description

One way of interpreting the Tari network is that it is a large peer-to-peer messaging application. The entities chatting on the network include:

Users
Wallets
Base nodes
Validator nodes

The types of messages that these entities send might include:

Text messages
Transaction messages
Block propagation messages
Asset creation instructions
Asset state change instructions
State Checkpoint messages

For successful communication to occur, the following needs to happen:

The message is translated from its memory storage format into a standard payload format that will be transported over the wire.
The communication module wraps the payload into a message format, which may entail any/all of
- adding a message header to describe the type of payload;
- encrypting the message;
- signing the message;
- adding destination/recipient metadata.
The communication module then sends the message over the wire.
The recipient receives the message and unwraps it, possibly performing any/all of the following:
- decryption;
- verifying signatures;
- extracting the payload;
- passing the serialized payload to modules that are interesting in that particular message type.
The message is deserialized into the correct data structure for use by the receiving software

This document only covers the first and last steps, i.e. serializing data from in-memory objects to a format that can be transmitted over the wire. The other steps are handled by the Tari communication protocol.

In addition to machine-to-machine communication, we also standardize on human-to-machine communication. Use cases for this include:

Handcrafting instructions or transactions. The ideal format here is a very human-readable format.
Copying transactions or instructions from cold wallets. The ideal format here is a compact but easy-to-copy format.
Peer-to-peer text messaging. This is just a special case of what has already been described, with the message structure containing a unicode message_text field.

When sending a message from a human to the network, the following happens:

The message is deserialized into the native structure.
The deserialization acts as an automatic validation step.
Additional validation can be performed.
The usual machine-to-machine process is followed, as described above.

Binary Serialization Formats

The ideal properties for binary serialization formats are:

widely used across multiple platforms and languages, but with excellent Rust support;
compact binary representation; and
serialization "Just Works"(TM) with little or no additional coding overhead.

Several candidates fulfill these properties to some degree.

ASN.1

Pros:
- Very mature (was developed in the 1980s)
- Large number of implementations
- Dovetails nicely into ZMQ
Cons:
- Limited Rust/Serde support
- Requires schema (additional coding overhead if no automated tools for this exist)

Message Pack

Pros:
- Very compact
- Fast
- Multiple language support
- Good Rust/Serde support
- Dovetails nicely into ZMQ
Cons:
- No metadata support

Protobuf

Similar to Message Pack, but also requires schemas to be written and compiled. Serialization performance and size are similar to Message Pack. It Can work with ZMQ, but is better designed to be used with gRPC.

Cap'n Proto

Similar to Protobuf, but claims to be much faster. Rust is supported.

Hand-rolled Serialization

Hintjens recommends using hand-rolled serialization for bulk messaging. While Pieter usually offers sage advice, I'm going to argue against using custom serializers at this stage for the following reasons:

We're unlikely to improve hugely over MessagePack.
Since Serde does 95% of our work for us with MessagePack, there's a significant development overhead (and new bugs) involved with a hand-rolled solution.
We'd have to write de/serializers for every language that wants Tari bindings; whereas every major language has a MessagePack implementation.

Serialization in Tari

Deciding between these protocols is largely a matter of preference, since there isn't that much difference between them. Given that ZMQ is used in other places in the Tari network, MessagePack looks to be a good fit while offering a compact data structure and highly performant de/serialization. In Rust, in particular, there's first-class support for MessagePack in Serde.

For human-readable formats, it makes little sense to deviate from JSON. For copy-paste semantics, the extra compression that Base64 offers over raw hex or Base58 makes it attractive.

Many Tari data types' binary representation will be the straightforward MessagePack version of each field in the related struct. In these cases, a straightforward #[derive(Deserialize, Serialize)] is all that is required to enable the data structure to be sent over the wire.

However, other structures might need fine-tuning, or hand-written serialization procedures. To capture both use cases, it is proposed that a MessageFormat trait be defined:


#![allow(unused_variables)]
fn main() {
pub trait MessageFormat: Sized {
    fn to_binary(&self) -> Result<Vec<u8>, MessageFormatError>;
    fn to_json(&self) -> Result<String, MessageFormatError>;
    fn to_base64(&self) -> Result<String, MessageFormatError>;

    fn from_binary(msg: &[u8]) -> Result<Self, MessageFormatError>;
    fn from_json(msg: &str) -> Result<Self, MessageFormatError>;
    fn from_base64(msg: &str) -> Result<Self, MessageFormatError>;
}
}

This trait will have default implementations to cover most use cases (e.g. a simple call through to serde_json). Serde also offers significant ability to tweak how a given struct will be serialized through the use of attributes.

RFC-0172/PeerToPeerMessaging

Peer to Peer Messaging Protocol

status: draft

Maintainer(s): Stanley Bondi, Cayle Sharrock and Yuko Roodt

Licence

The 3-Clause BSD Licence.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

Redistributions of this document must retain the above copyright notice, this list of conditions and the following disclaimer.
Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.

Language

Disclaimer

This document and its content are intended for information purposes only and may be subject to change or update without notice.

Goals

The aim of this Request for Comment (RFC) is to describe the peer-to-peer messaging protocol for communication nodes and communication clients on the Tari network.

Related Requests for Comment

Description

Assumptions

Either every communication node or communication client has access to a Tor/I2P proxy, or a native Tor/I2P implementation exists, which allows communication across the Tor network.
All messages are de/serialized as per RFC-0171: Message Serialisation.

Broad Requirements

Tari network peer communication must facilitate secure, private and efficient communication between peers. Broadly, a communication node or communication client MUST be capable of:

bidirectional communication between multiple connected peers;
private and secure over-the-wire communication;
understanding and constructing Tari messages;
encrypting and decrypting message payloads;
gracefully reestablishing dropped connections; and (optionally)
communicating to a SOCKS5 proxy (for connections over Tor and I2P).

Additionally, communication nodes MUST be capable of performing the following tasks:

opening a control port for establishing secure peer channels;
maintaining a list of known peers in the form of a routing table;
forwarding directed messages to neighbouring peers; and
broadcasting messages to neighbouring peers.

Overall Architectural Design

The Tari communication layer has a modular design to allow for the various communicating nodes and clients to use the same infrastructure code.

The design is influenced by an open-source library called ZeroMQ and the ZeroMQ C bindings are a dependency of the project. ZeroMQ's over-the-wire protocol is relatively simple, and replicating ZeroMQ framing in a custom implementation should not be prohibitively difficult. However, ZeroMQ offers many valuable features, which would be a significantly larger undertaking to reproduce. Fortunately, bindings or native ports are available in numerous languages.

To learn more about ZeroMQ, read the guide. It's an enjoyable and worthwhile read.

A quick overview of what ZeroMQ provides:

A simple socket Application Programming Interface (API).
Some well-defined patterns to connect sockets together.
Sockets that are tiny asynchronous message queues, which:
- abstract away complexity around the underlying socket;
- are transport agnostic, meaning you can choose between Transmission Control Protocol (TCP), Pragmatic General Multicast (PGM), Inter-process Communication (IPC) and in-process (inproc) transports with little or no changes to code; and
- transparently reconnect when connections are dropped.
The inproc transport for message passing between threads without mutex locks.
Built-in protocol for asymmetric encryption over the wire using Curve25519.
Ability to send and receive multipart messages using a simple framing scheme. More info here.
Support for Secure Socket (SOCKS) proxies.

This document will refer to several ZeroMQ sockets. These are referred to by prepending ZMQ_ and the name of the socket in capitals. For example, ZMQ_ROUTER.

Note about ZeroMQ frames and multipart messages:

ZeroMQ frames are length-specified blocks of binary data and can be strung together to make multipart messages.

|5|H|E|L|L|O|*|0|*|3|F|O|O|+|
* = more flag
+ = no more flag

A multipart message consisting of three frames "HELLO", a zero-length frame and "FOO"

When this RFC mentions 'multipart messages', this is what it's referring to.

Establishing a Connection

Every participating communication node SHOULD open a control socket (refer to ControlService) to allow peers to negotiate and establish a peer connection. The NetAddress of the control socket is what is stored in peers' routing tables and will be used to establish new ephemeral PeerConnections. Any peer that wants to connect MUST establish a connection to the control socket of the destination peer to negotiate a new encrypted PeerConnection.

Once a connection is established, messages can be sent and received directly to or from the Peer.

Incoming messages are validated, deserialized and handled as appropriate.

Encryption

Two forms of encryption are used:

Over-the-wire encryption, in which traffic between nodes is encrypted using ZMQ's CURVE implementation.
Payload encryption, in which the MessageEnvelopeBody is encrypted in such a way that it can only be decrypted by the destination recipient.

Components

The following components are proposed:

graph TD CSRV[ControlService] IMS[InboundMessageService] PM[PeerManager] CM[ConnectionManager] BCS[BroadcastStrategy] OMS[OutboundMessageService] PC[PeerConnection] NA[NetAddress] STR[Storage] PEER[Peer] RT[RoutingTable] IMS --> PM IMS --> CM CSRV --> CM CSRV --> PM OMS -->|execute| BCS OMS --> PM OMS --> CM PM --> RT PM --> STR CM --> PC PC -->|has one| PEER PEER -->|has many| NA

NetAddress

Represents:

IP address;
Onion address; or
I2P address.


#![allow(unused_variables)]
fn main() {
#[derive(Clone, PartialEq, Eq, Debug)]
/// Represents an address which can be used to reach a node on the network
pub enum NetAddress {
    /// IPv4 and IPv6
    IP(SocketAddress),
    Tor(OnionAddress),
    I2P(I2PAddress),
}
}

Messaging Structure

The following illustrates the structure of a Tari message:

+----------------------------------------+
|             MessageEnvelope            |
|  +----------------------------------+  |
|  |        MessageEnvelopeHeader     |  |
|  +----------------------------------+  |
|  +----------------------------------+  |
|  |      MessageEnvelopeBody         |  |
|  |     (optionally encrypted)       |  |
|  | +------------------------------+ |  |
|  | |            Message           | |  |
|  | |   +-----------------------+  | |  |
|  | |   |     MessageHeader     |  | |  |
|  | |   +-----------------------+  | |  |
|  | |                              | |  |
|  | |   +-----------------------+  | |  |
|  | |   |      MessageBody      |  | |  |
|  | |   +-----------------------+  | |  |
|  | +------------------------------+ |  |
|  +----------------------------------+  |
+----------------------------------------+

MessageEnvelope Wire format

Every Tari message MUST use the MessageEnvelope format. This format consists of four frames of a multipart message.

A MessageEnvelope represents a message that has either just come off or is about to go on to the wire. and consists of the following:

Name	Frame	Length (Octets)	Type	Description
identity	0	8	`[u8;8]`	The identifier that a `ZMQ_ROUTER` socket expects so that it knows the intended destination of the message. This can be thought of as a session token.
version	1	1	`u8`	The wire protocol version.
header	2	Varies	`Vec<u8>`	Serialized bytes of data containing an unencrypted MessageEnvelopeHeader.
body	3	Varies	`Vec<u8>`	Serialized bytes of data containing an unencrypted or encrypted MessageEnvelopeBody.

The header and decrypted body MUST be deserializable as per RFC-0171: MessageSerialization.

MessageEnvelopeHeader

Every MessageEnvelope MUST have an unencrypted header containing the following fields:

Name	Type	Description
version	`u8`	Message protocol version.
source	`PublicKey`	Source public key.
dest	`Option<NodeDestination>`	Destination node ID or public key. A destination is optional.
signature	`[u8]`	Signature of the message header and body, signed with the private key of the source.
flags	`u8`	bit 0: 1 indicates that the message body is encrypted bits 1-7: reserved .

A communication node and communication client:

MUST validate the signature of the message using the source's public key.
MUST reject the message if the signature verification fails.
If the encryption bit flag is set:
- MUST attempt to decrypt the MessageEnvelopeBody; or failing that
- MUST forward the message to a subset of peers using the Closest BroadcastStrategy.
- MUST discard the message if the body is not encrypted.

MessageEnvelopeBody

A MessageEnvelopeBody is the payload of the [MessageEnvelope]. A MessageEnvelopeBody may be encrypted as required.

It consists of a MessageHeader and Message of a particular predefined MessageType.

MessageType

An enumeration of the messages that are part of the Tari network. MessageTypes are represented as an unsigned eight-bit integer and each value must be mapped to a corresponding Message struct.

All MessageTypes fall within a particular numerical range according to the message's concern:

Category	Range	# Message Types	Description
`reserved`	0	1	Reserved for control messages such as `Ack`.
`net`	1-32	32	Network-related messages such as `join` and `discover`.
`peer`	33-64	32	Peer connection messages, such as `establish connection`.
`blockchain`	65-96	32	Messages related to the blockchain, such as `add block`.
`vn`	97-224	128	Messages related to the validator nodes, such as `execute instruction`.
`extended`	225-255	30	Reserved for future use.

In documentation, MessageTypes can be referred to by the category and name. For example, peer::EstablishConnection and net::Discover.

MessageHeader

Every Tari message MUST have a header containing the following fields:

Name	Type	Description
version	`u8`	The message version.
message_type	`u8`	An enumeration of the message type of the body. Refer to MessageType.

As this is part of the MessageEnvelopeBody, it can be encrypted along with the rest of the message, which keeps the type of message private.

MessageBody

Messages are an intention to perform a task. MessageType names should thus be a verb such as net::Join or blockchain::AddBlock.

All messages can be categorized as follows; each categorization has rules for how they should be handled:

A propagation message
- SHOULD NOT have a destination in the MessageHeader;
- MUST be forwarded;
- SHOULD use the Random BroadcastStrategy;
- SHOULD discard a message that it has seen within the DuplicateMessageWindow.
A direct message
- MUST have a destination in the MessageHeader;
- SHOULD be discarded if it does not have a destination;
- SHOULD discard a message that it has seen before;
- MUST use the Direct BroadcastStrategy if a destination peer is known;
- SHOULD use the Closest BroadcastStrategy if a destination peer is not known.
An encrypted message
- MUST undergo an attempt to be decrypted by all recipients;
- MUST be forwarded by recipients if it cannot be decrypted;
- SHOULD discard a message that it has seen before.

The MessageType in the header MUST be used to determine the type of the message deserialized. If the deserialization fails, the message SHOULD be discarded.

DuplicateMessageWindow

A configurable length of time for which message signatures should be tracked in order to eliminate duplicate messages. This should be long enough to make it highly unlikely that a particular message will be processed again and short enough to not be a burden on the node.

InboundConnection

A thin wrapper around a ZMQ_ROUTER socket, which binds to a NetAddress and accepts incoming multipart messages. This connection blocks until there is data to read, or a timeout is reached. In both cases, the receive method can be called again (i.e. in a loop) to continue listening for messages. Client code should run this loop in its own thread. send is only called (if at all) in response to an incoming message.

Fields may include

a NetAddress;
a timeout;
underlying ZeroMQ socket.

Methods may include:

receive()
send(data)
set_encryption(secret_key)
set_socks_proxy(address)
set_hwm(v)

An InboundConnection:

MUST perform the "server-side" CurveZMQ encryption protocol if encryption is set.
- Using ZeroMQ. this means setting the socketopts ZMQ_CURVE_SERVER to 1 and ZMQ_CURVE_SECRETKEY to the secret key before binding.
MUST listen for and accept TCP connections.
- For an IP NetAddress, bind on the given host IP and port.
- For an Onion NetAddress, bind on 127.0.0.1 and the given port.
- For an I2P NetAddress, as yet undetermined.
MUST read multipart messages and return them to the caller.
- If the timeout is reached, return an error to be handled by the calling code.

OutboundConnection

A thin wrapper around a ZMQ_DEALER socket, which connects to a NetAddress and sends outbound multipart messages. This connection blocks until data can be written, or a timeout is reached. The timeout should never be reached, as ZeroMQ internally queues messages to be sent.

Fields may include:

a NetAddress;
underlying ZeroMQ socket.

Methods may include:

send(msg)
receive()
disconnect()
set_encryption(server_pk, client_pk, client_sk)
set_socks_proxy(address)
set_hwm(v)

An OutboundConnection:

MUST perform the "client-side" CurveZMQ encryption protocol if encryption is set.
- Using ZeroMQ, this means setting the socketopts ZMQ_CURVE_SERVERKEY, ZMQ_CURVE_SECRETKEY and ZMQ_CURVE_PUBLICKEY.
MUST connect to a TCP endpoint.
- For an IP NetAddress, connect to the given host IP and port.
- For an Onion NetAddress, connect to the onion address using the TCP, e.g. tcp://xyz...123.onion:1234.
- For an I2P NetAddress, as yet undetermined.
MUST write the parts of the given MessageEnvelope to the socket as a multipart message consisting of, in order:
- identity;
- version;
- header;
- body.
If specified, MUST set a High Water Mark (HWM) on the underlying ZeroMQ socket.
If the HWM is reached, a call to send MUST return an error and any messages received SHOULD be discarded.

Peer

A single peer that can communicate on the Tari network.

Fields may include:

addresses - a list of NetAddresses associated with the peer, perhaps accompanied by some bookkeeping metadata, such as preferred address;
node_type - the type of node or client, i.e. BaseNode, ValidatorNode, Wallet or TokenWallet);
last_seen - a timestamp of the last time a message has been sent/received from this peer;
flags - 8-bit flag;
- bit 0: is_banned,
- bit 1-7: reserved.

A peer may also contain reputation metrics (e.g. rejected_message_count, avg_latency) to be used to decide if a peer should be banned. This mechanism is yet to be decided.

PeerConnection

Represents direct bidirectional connection to another node or client. As connections are bidirectional, the PeerConnection need only hold a single InboundConnection or OutboundConnection, depending on if the node requested a peer connect to it or if it is connecting to a peer.

PeerConnection will send messages to the peer in a non-blocking, asynchronous manner as long as the connection is maintained.

It has a few important functions:

managing the underlying network connections, with automatic reconnection if necessary;
forwarding incoming messages onto the given handler socket; and
sending outgoing messages.

Unlike InboundConnection and OutboundConnection, which are essentially stateless, PeerConnection maintains a particular ConnectionState.

Idle - the connection has not been established.
Connecting - the connection is in progress.
Connected - the connection has been established.
Suspended - the connection has been suspended. Incoming messages will be discarded, calls to send() will error.
Dead - the connection is no longer active because the connection was dropped.
Shutdown - the connection is no longer active because it was shut down.

Fields may include:

a connection state;
a control socket;
a peer connection NetAddress;
a direction (either Inbound or Outbound);
a public key obtained from the connection negotiation;
(optional) SOCKS proxy.

Methods may include:

establish()
shutdown()
suspend()
resume()
send(msg)

A PeerConnection:

MUST listen for data on the given NetAddress using an InboundConnection;
MUST sequentially try to connect to one of the peer's NetAddresses until one succeeds or all fail using an OutboundConnection;
MUST immediately reject and dispose of a multipart message not consisting of four parts, as detailed in MessageEnvelope;
MUST construct a MessageEnvelope from the multiple parts;
MUST pass the constructed MessageEnvelope to the message handler;
MUST transition to Connecting state and retry the connection, should a connection drop;
MUST send a net::Disconnect message and drop the connection when a shutdown signal is received.

ConnectionManager

The ConnectionManager manages a set of live PeerConnections. It provides an abstraction for other components to initiate and use PeerConnections without having to worry about attaching the new PeerConnection to message handlers.

It consists of a list of active peer connections and an inproc message handler socket. This socket is 'written to' whenever a message is received from any active PeerConnection for other components to act on.

Methods may include:

establish_connection(Peer) - create and return a new PeerConnection;
disconnect(peer) - disconnect a particular peer;
suspend() - temporarily suspend connections;
resume() - temporarily suspend connections;
shutdown - cleanly shut down all PeerConnections.

The ConnectionManager:

MUST call suspend on every PeerConnection if its suspend method is called;
MUST call resume on every PeerConnection if its resume method is called;
MUST call shutdown on every PeerConnection if its shutdown method is called
MUST create a new PeerConnection with the given Peer and NetAddress, when establish_connection is called;
MUST call shutdown on the PeerConnection and remove the connection for the given peer, when disconnect(peer) is called;
MAY disconnect peers if the connection has not been used for an extended period;
SHOULD disconnect the least recently used peer if the connection pool is greater than max connections

ControlService

The purpose of this service is to negotiate a new secure PeerConnection.

The control service accepts a single message:

peer::EstablishConnection(pk, curve_pk, net_address).

A ControlService:

MUST listen for connections on a predefined CONTROL PORT;
SHOULD deny connections from banned peers.

The steps to establish a peer connection are as follows:

Alice wants to connect to Bob

Alice creates a PeerConnection to which Bob can connect.
- A new CURVE key pair is generated.
Alice connects to Bob's control server and Bob accepts the connection.
Alice sends a peer::establish_connection message, with:
- the CURVE public key for the socket connection;
- the node's public key corresponding to its Node ID; and
- the NetAddress of the new PeerConnection.
Bob accepts this request and opens a new PeerConnnection socket using Alice's CURVE public key.
Bob connects to the given NetAddress and sends a peer::establish_connection message.
If Alice accepts the connection, they can begin sending messages. If not, both sides terminate the connection.

PeerManager

The PeerManager is responsible for managing the list of peers with which the node has previously interacted. This list is called a routing table and is made up of Peers.

The PeerManager can

add a peer to the routing table;
search for a peer given a node ID, public key or NetAddress;
delete a peer from the list;
persist the peer list using a storage backend;
restore the peer list from the storage backend;
maintain lightweight views of peers, using a filter criterion, e.g. a list of peers that have been banned, i.e. a blacklist; and
prune the routing table based on a filter criterion, e.g. last date seen.

MessageDispatcher

The MessageContext contains:

the requesting PeerConnection;
the MessageHeader;
the deserialized message;
the OutboundMessageService.

Basically, all the tools the handler needs to interact with the network.

A MessageDispatcher is responsible for:

constructing the MessageContext;
finding the message handler that is associated with the MessageType;
passing the MessageContext to the handler; and
ignoring the message if the handler cannot be found.

An example API may be:


#![allow(unused_variables)]
fn main() {
let dispatcher = MessageDispatcher::<MessageType>::new()
    .middleware(logger)
    .route(BlockchainMessageType::NewBlock, BlockHandlers::store_and_broadcast)
    ...
    .route(NetMessageType::Ping, send_pong);

inbound_msg_service.set_handler(dispatcher.handler);
}

InboundMessageService

InboundMessageService is a service that receives messages over a non-blocking asynchronous socket and determines what to do with it. There are three options: handle, forward and discard.

A pool of worker threads (with a configurable size) is started and each one listens for messages on its $1:n$ inproc message socket. A ZMQ_DEALER socket is suggested for fair-queueing work amongst workers, who listen for work with a ZMQ_REP. The workers read off this socket and process the messages.

An InboundMessageService:

MUST receive messages from all PeerConnections; and
MUST write the message to the worker socket.

A worker:

MUST deserialize the MessageHeader.
- If unable to deserialize, MUST discard the message.
MUST check the message signature.
- MUST discard the message if the signature is invalid.
- MUST discard the message if the signature has been processed within the DuplicateMessageWindow.
If the encryption flag is set:
- MUST attempt to decrypt the message.
  - If successful, process and handle the message.
  - Otherwise, MUST forward the message using the Random BroadcastStrategy.
  - If the message is not encrypted, MUST discard it.
If the destination node ID is set:
- If the destination matches this node's ID - process and handle the message.
- If the destination does not match this node's ID - MUST forward the message using the Closest BroadcastStrategy.
If the destination is not set
- If the MessageType is a kind of propagation message:
  - MUST handle the message;
  - MUST forward the message using the Random BroadcastStrategy.
- If the MessageType is a kind of encrypted message:
  - MUST attempt to decrypt and handle the message;
  - if successful, MUST handle the message;
  - if unsuccessful, MUST forward the message using the Random or Flood BroadcastStrategy,

OutboundMessageService

OutboundMessageService is responsible for using the connection and peer infrastructure to send messages to the rest of the network.

In particular, it is responsible for:

serializing the message body;
constructing the MessageEnvelope;
executing the required BroadcastStrategy; and
sending messages using the [ConnectionManager].

The actual sending of messages can be requested via the public send_message method, which takes a MessageHeader, MessageBody and BroadcastStrategy as parameters.

send_message then selects an appropriate peer(s) from the ConnectionManager according to the BroadcastStrategy and sends the message to each of the selected peers.

BroadcastStrategy determines how a set of peer nodes will be selected and can be:

Direct - send to a particular peer matching the given node ID;
Flood - send to all known peers who are not [communication clients];
Closest - send to $n$ closest peers who are not [communication clients]; or
Random - send to a random set of peers of size $n$ who are not [communication clients].

Privacy Features

The following privacy features are proposed:

A communication node or communication client MAY communicate solely over the Tor/I2P networks.
All traffic (with the exception of the control service) MUST be encrypted.
Messages MAY encrypt the body of a MessageEnvelope, which only a particular recipient can decrypt.
The destination header field can be omitted when used in conjunction with body encryption; the destination is completely unknown to the rest of the network.

Store and Forward Strategy

Sometimes it may be desirable for messages to be sent without a destination node/client being online. This is especially important for a modern chat/messaging application.

The mechanism for this is proposed as follows:

Each communication node MUST allocate some disk space for storage of messages for offline recipients. Only some whitelisted MessageTypes are permitted to be stored. A sender sends a message destined for a particular node ID to its closest peers, which forward the message to their closest peers, and so on.

Eventually, the message will reach nodes that either know the destination or are very close to the destination. These nodes MUST store the message in some pending message bucket for the destination. The maximum number of buckets and the size of each bucket SHOULD be sufficiently large as to be unlikely to overflow, but not so large as to approach disk space problems. Individual messages should be small and responsibilities for storage spread over the entire network.

A communication node

MUST store messages for later retransmission, if all of the following conditions are true:
- the MessageType is permitted to be stored;
- there are fewer than $n$ closer online peers to the destination.
MUST retransmit pending messages when a closer peer comes online or is added to the routing table.
MAY remove a bucket, in any of the following conditions:
- The bucket is empty;
- A configured maximum number of buckets has been reached. Discard the bucket with the earliest creation timestamp.
- The number of closer online peers to the destination is equal to or has exceeded $n$.
MAY expire individual messages after a sufficiently long Time to Live (TTL)

This approach has the following benefits:

When a destination comes online, it will receive pending messages without having to query them.
The "closer within a threshold" metric is simple.
Messages are stored on multiple peers, which makes it less likely for messages to disappear as nodes come and go (depending on threshold $n$).

Queue Overflow Strategy

Inbound/OutboundConnections (and therefore PeerConnection) have an HWM set.

If the HWM is hit:

any call to send() should return an error; and
incoming messages should be silently discarded.

Outstanding Items

A PeerConnection will probably need to implement a heartbeat to detect if a peer has gone offline.
InboundConnection(Service) may want to send small replies (such as OK, ERR) when the message has been accepted or rejected.
OutboundConnection(Service) may want to receive and handle small replies.
Encrypted communication for the ControlService would be better privacy, but since ZMQ requires a CURVE public key before the connection is bound, a dedicated "secure connection negotiation socket" would be needed.
Details of distributed message storage.
Which NetAddress to use if a peer has many.

Mempool

The Mempool for Unconfirmed Transactions on the Tari Base Layer

status: draft

Maintainer(s): Yuko Roodt

License

The 3-Clause BSD License.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

Redistributions of this document must retain the above copyright notice, this list of conditions and the following disclaimer.
Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.

Language

Disclaimer

The purpose of this document and its content is for information purposes only and may be subject to change or update without notice.

This document may include preliminary concepts that may or may not be in the process of being developed by the Tari community. The release of this document is intended solely for review and discussion by the community regarding the technological merits of the potential system outlined herein.

Goals

This document will introduce the Tari base layer Mempool that consists of a Transaction Pool, Pending Pool, Orphan Pool and Reorg Pool. The Mempool is used for storing and managing unconfirmed and time-lock restricted transactions.

Related RFCs

RFC-0100: The Tari Base Layer

Description

Assumptions

Each base node is connected to a number of peers that maintain their own copies of the Mempool.

Abstract

The Mempool is responsible for managing, verifying and maintaining all unconfirmed transactions that have not yet been included in a block and added to the Tari blockchain. It consists of a Transaction Pool, Pending Pool, Orphan Pool and Reorg Pool to achieve these tasks. It is also responsible for propagating valid transactions and sharing the Mempool state with connected peers. An overview of the required functionality for the Mempool and each of its component pools will be provided.

Overview

Every base node maintains a Mempool that consists of four separate pools: the Transaction Pool, Pending Pool, Orphan Pool and Reorg Pool. These four pools have different tasks and work together to form the Mempool used for maintaining unconfirmed transactions.

This is the role descriptions for each component pool:

Transaction Pool: contains all unconfirmed transactions that have been verified, have passed all checks, that only spend valid UTXOs and don't have any time-lock restrictions.
Pending Pool: contains unconfirmed transactions that have time-lock restrictions. The transactions in this pool either attempt to spend UTXOs with time-locks or the transactions themselves have time-locks limiting them from being included in new blocks until a specified future time or block height has been reached.
Orphan Pool: out of order unconfirmed transactions are managed by this pool, these transactions attempt to spend non-existent UTXOs.
Reorg Pool: stores a backup of all transactions that have recently been included into blocks, in case a blockchain reorganization occurs and these transactions have to be restored to the Transaction Pool so that they can be included in future blocks.

Prioritizing Unconfirmed Transactions

The maximum storage capacity used for storing unconfirmed transactions by the Mempool and each of its component pools can be configured. If a new transaction is received and the storage capacity limits have been reached, then transactions are prioritized according to the transaction priority metric. The transaction priority metric consist of the fees and the maturity of the UTXOs being spent by the transaction and should be applied to determine the priority of each transaction in the Mempool. As the allocated storage space of the Mempool becomes limited, the transactions with the highest priority are kept in the pool. The lowest priority transactions are discarded to make room for higher priority incoming transactions.

The transaction priority metric has the following behavior:

Transactions spending UTXOs with higher block height maturity SHOULD be prioritized over transactions spending UTXOs with lower block height maturity.
Transactions with higher fees per transaction message size SHOULD be prioritized over lower fee transactions.

Syncing and Updating of the Memory Pool State

On the initial startup of the Mempool, the complete state of the Mempool that consists of the Transaction Pool, Pending Pool, Orphan Pool and Reorg Pool can be requested and downloaded from the connected peers. A memory pool typically doesn't make use of persistent storage but could be configured to keep a backup of the last known state. If an existing Mempool state is locally available then a more efficient update process can be performed by requesting only the unconfirmed transactions missing from the current Mempool state. When no state is available then the entire Mempool state must be downloaded from the connected peers. During downloading or updating of the Mempool state, the validity of all transaction in the pool must be verified and the priority of each transaction must be calculated.

Functional behavior required for sharing and updating of the Mempool state, and propagation of transactions between peers:

All verified transaction MUST be propagated to neighboring peers.
Duplicate transactions MUST NOT be propagated to peers.
Unvalidated or invalid transactions MUST NOT be propagated to peers.
Verified transactions that were discarded due to low priority levels MUST be propagated to peers.
The Mempool MUST have an interface that can be used by neighboring peers to query the content and state of the memory pool.
It MUST have a mechanism that enables peers to download the memory pool state in full or in part.
It MUST accept all transactions received from peers but MAY decide to discard low priority transactions.
It MUST allow wallets to track payments by monitoring that a particular transaction has been added to the Mempool.
A Mempool MAY choose:
- to discard the Mempool state on restarts and then download the full state from its peers or
- to store the state of the Mempool using persistent storage to reduce communication bandwidth required when reinitializing the Mempool after a restart.

Transaction Pool

The Transaction Pool consists of all unconfirmed transactions that have been received, verified and have passed all checks. These unconfirmed transactions in the Transaction Pool are ready to be included and can be used to construct new blocks for the Tari blockchain.

Functional behavior required of the Transaction Pool:

It MUST verify that incoming transactions only spend existing UTXOs.
It MUST ensure that incoming transactions don't have a processing time-lock or has a time-lock that has expired.
It MUST ensure that all time-locks of the UTXOs that will be spent by the transaction have expired.
Transactions that have been used to construct new blocks MUST be removed from the Transaction Pool and added to the Reorg Pool.

Pending Pool

The Pending Pool contains all transactions that are restricted by time-locks. A transaction could have a time-lock limiting it from being processed or it can attempt to spend UTXOs with time-locks. These transactions require their time-locks or the time-locks of the input UTXOs to expire before they can be processed and included into new blocks. All transactions in the Pending Pool have been verified and passed all checks except their own time-lock has not yet expired or some of the UTXOs that will be spent have time-lock restrictions that are not yet valid. Once the transactions time-lock or the time-locks on the UTXOs have expired then the Pending Pool transactions can be moved to the Transaction Pool for inclusion in future blocks.

Functional behavior required of the Pending Pool:

Once the transaction time-lock or UTXO time-lock restricting the processing of a transaction has expired then the pending transaction MUST be moved to the Transaction Pool.

Orphan Pool

The Orphan Pool contains all the received transactions that have passed all the verification steps and checks, except they attempt to spend UTXOs that don't exist. A possible reason these UTXOs do not yet exist is that they may not yet have been created and might exist in the future. Typically, these orphaned transactions are from a series or bundle of transactions that need to be processed in a specific order but

have been either received out of order,
or the order of processing the bundled transactions might not have been known or specified.

As transactions are processed and the missing UTXOs have been created, then the orphaned transactions can be moved to the Transaction Pool for possible inclusion in future blocks. Another possibility why the input UTXOs might not be available is that the UTXOs were double spent by other transactions. In time, the double spent transactions will be discarded from the Orphan Pool once they have reached the appropriate maturity threshold.

Functional behavior required of the Orphan Pool:

Each newly received transaction MUST be verified and pass all checks except the UTXO validity check before it is placed in the Orphan Pool.
Orphaned transactions must be upgraded and moved to the Transaction Pool once the previously unavailable UTXOs become available.
Orphaned transactions that have surpassed the expiration time threshold MUST be removed from the Orphan Pool.

Reorg Pool

The Reorg Pool consists of all unconfirmed transaction that have recently been added to blocks, resulting in their removal from the Transaction Pool. When a potential blockchain reorganization occurs that invalidates previously assembled blocks, the transactions used to construct these discarded blocks can be recovered from the Reorg Pool and can be added back into the Transaction Pool. This will ensure that high priority transactions are not lost during blockchain reorganization but can be added into future blocks without retransmission of these transactions.

Functional behavior required of the Reorg Pool:

Copies of the verified transactions removed from the Transaction pool that were placed in blocks MUST be stored in the Reorg Pool.
Transactions in the Reorg pool MAY be removed after the threshold expiration time has been reached.
When a blockchain reorganization is detected, all affected transactions from the Reorg Pool MUST be moved to the Transaction Pool.

Mempool

The Mempool manages the four component pools and interacts with peers to share and retrieve transactions and the Mempool state. During the operation of the Mempool it will distribute incoming transactions to the appropriate component pools. When a new incoming transaction is received a number of checks and verification steps need to be performed to determine if the transaction can be added to the Mempool and determine which of the component pools should be responsible for that particular transaction. Only when the new transaction has passed these checks can it be added to the Mempool and should it be propagated to the connected peers.

Functional behavior required of the Mempool:

If a duplicate transaction is received, that already exist in the Mempool, then the duplicate copy MUST be discarded.
When considering transactions that attempt to double spend UTXOs, the highest priority transaction MUST be kept and any other transactions that spend the same UTXO MUST be discarded.
When the storage limit of the Mempool has been reached, new incoming transactions SHOULD be prioritized according to the Priority metric.
Lower priority Mempool transactions MUST be discarded to make room for higher priority incoming transactions.
Incoming transactions with lower priorities than the minimum transaction priority in the Mempool MUST be discarded.
The Mempool MUST verify that incoming transactions do not have duplicate outputs.
It MUST check that all coinbase outputs that will be spent have matured sufficiently.
The distribution of storage space allocated to each component pool in the memory pool MAY be configured and adjusted.
The memory pool SHOULD have a mechanism to estimate fee categories from the current Mempool state. As an example, a priority fee can be estimated that will ensure that a new transactions will have the appropriate priority to be added into a new block in a timely manner.

Functional behavior required for distributing incoming transactions to the component pools:

Verified transactions that have passed all checks such as spending of valid UTXOs and expired time-locks MUST be placed in the Transaction Pool
All transactions that attempt to spend UTXOs with valid time-locks MUST be added to the Pending Pool.
Incoming transactions with time-locks prohibiting them from being included into new blocks should be added to the Pending Pool.
Newly received verified transaction attempting to spend a UTXO that does not yet exist MUST be added to the Orphan pool.
Transactions that have been added to blocks and were removed from the Transaction Pool should be added to the Reorg Pool.

Tari-specific Base Layer Extensions

This section covers RFCs that describe extensions to the Mimblewimble protocol that enable key functionality for the Tari Base layer token and Digital Asset network.

RFC-0230/Time-related Transactions

Time-related Transactions

status: draft

Maintainer(s): S W van Heerden

Licence

The 3-Clause BSD Licence.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

Redistributions of this document must retain the above copyright notice, this list of conditions and the following disclaimer.
Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.

Language

Disclaimer

This document and its content are intended for information purposes only and may be subject to change or update without notice.

Goals

The aim of this Request for Comment (RFC) is to describe a few extensions to Mimblewimble to allow time-related transactions.

Related Requests for Comment

RFC-0200: Base Layer Extensions

Description

Time-locked Contracts

In Mimblewimble, time-locked contracts can be accomplished by modifying the kernel of each transaction to include a block height. This limits how early in the blockchain lifetime the specific transaction can be included in a block.

This means that users constructing a transaction:

MUST include a lock height in the kernel of their transaction; and
MUST include the lock height in the transaction signature to prevent lock height malleability.

Tari base nodes MUST NOT add any transaction to the mined block that has not already exceeded its lock height.

This also adds the following requirement to a base node:

It MUST reject any block that contains a kernel with a lock height greater than the current head.

Time-locked UTXOs

Time-locked Unspent Transaction Outputs (UTXOs) can be accomplished by adding a feature flag to a UTXO and a lock height. This allows a limit on when in the blockchain lifetime the specific UTXO can be spent.

This requires that users constructing a transaction:

MUST include a feature flag of their UTXO; and
MUST include a lock height in their UTXO.

This adds the following requirement for a base node:

A base node MUST NOT allow a UTXO to be spent if the current head has not already exceeded the UTXO's lock height.

This also adds the following requirement for a base node:

A base node MUST reject any block that contains a UTXO with a lock height not already past the current head.

Hashed Time-locked Contract

Hashed time-locked contracts are a way of reserving funds for a certain payment, but they only pay out to the receiver if certain conditions are met. If these conditions are not met within a time limit, the funds are paid back to the sender.

Unlike Bitcoin, where this can be accomplished with a single transaction, in Mimblewimble, HTLCs involve a multi-step process to construct a time-locked contract.

The steps are as follows:

The sender MUST pay all the funds into an n-of-n multisig UTXO.
All parties involved MUST construct a refund transaction to pay back all funds to the sender who has spent this n-of-n multisig UTXO. However, this transaction has a transaction lock height set in the future and cannot be mined immediately. It therefore lives in the mempool. This means that if anything goes wrong from here on, the sender will get their money back after the time lock expires.
The sender MUST publish both above transactions at the same time to ensure the receiver cannot hold the sender hostage.
The parties MUST construct a third transaction that pays the receiver the funds. This transaction typically makes use of a preimage to allow spending of the transaction if the user reveals some knowledge, allowing the user to unlock the UTXO.

HTLCs in Mimblewimble make use of double-spending of the n-of-n multisig UTXO. The first valid published transaction can then be mined and claim the n-of-n multisig UTXO.

An example of an HTLC in practice can be viewed at Tari University:

RFC-0322/VNRegistration

Validator Node Registration

status: draft

Maintainer(s): Cayle Sharrock

Licence

The 3-Clause BSD Licence.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

Redistributions of this document must retain the above copyright notice, this list of conditions and the following disclaimer.
Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.

Language

Disclaimer

This document and its content are intended for information purposes only and may be subject to change or update without notice.

Goals

The aim of this Request for Comment (RFC) is to describe Validator Node (VN) registration. Registration accomplishes two goals:

Provides a register of Validator Nodes with an authority (the Tari base layer).
Offers Sybil resistance against gaining probabilistic majority of any given publicly nominated VN committee.

Related Requests for Comment

Description

VNs register themselves on the Base layer using a special transaction type. The registration transaction type requires the spending of a certain minimum amount of Tari coin, the (Registration Deposit), which has a time lock on the output for a minimum amount of time (Registration Term), as well as some metadata, such as the VN's public key.

The Node ID is calculated after registration to prevent mining of VN public keys that can be used to manipulate routing on the Distributed Hash Table (DHT).

Once a VN's Registration Term has expired, so will this specific VN registration. The Unspent Transaction Output (UTXO) time lock will have elapsed so the Registration Deposit can be reclaimed and a new VN registration needs to be performed. This automatic registration expiry will ensure that the VN registry stays up to date with active VN registrations, and inactive registrations will naturally be removed.

Requiring nodes to register themselves serves two purposes:

makes VN Sybil attacks expensive; and
provides an authoritative "central-but-not-centralized" registry of VNs from the base layer.

Node ID

The VN ID can be calculated deterministically after the VN registration transaction is mined. This ensures that VNs are randomly distributed over the DHT network.

VN IDs MUST be calculated as follows:

 NodeId = Hash( pk || h || kh )

Where

Field	Description
pk	The VN's DHT public key
h	The block height of the block in which the registration transaction was mined
kh	The hash of the registration transaction's kernel

Base Nodes SHOULD maintain a cached list of VNs and MUST return the Node ID in response to a get_validator_node_id request.

Validator Node Registration

A VN MUST register on the base layer before it can join any Distributed Area Network (DAN) committees. Registration happens by virtue of a VN registration transaction.

VN registrations are valid for the Registration Term.

The registration term is set at SIX months.

A VN registration transaction is a special transaction.

The transaction MUST have EXACTLY ONE UTXO with the VN_Deposit flag set.
This UTXO MUST also:
- set a time lock for AT LEAST the Registration Term (or equivalent block periods);
- provide the value of the UTXO in the signature metadata; and
- provide the public key for the spending key for the output in the signature metadata.
The value of this output MUST be equal to or greater than the Registration Deposit.
The UTXO MUST store:
- the value of the VN deposit UTXO as a u64; and
- the value of the public key for the spending key for the output as 32 bytes in little-endian order.
The KernelFeatures bit flag MUST have the VN_Registration flag set.
The kernel MUST also store the VN's DHT public key as 32 bytes in little-endian order.

Validator Node Registration Renewal

If a VN owner does not renew the registration before the Registration Term has expired, the registration will lapse and the VN will no longer be allowed to participate in any committees.

The number of consecutive renewals MAY increase the VN's reputation score.

A VN may only renew a registration in the TWO-WEEK period prior to the current term expiring.

A VN renewal transaction is a special transaction:

The transaction MUST have EXACTLY ONE UTXO with the VN_Deposit flag set.
The transaction MUST spend the previous VN deposit UTXO for this VN.
This UTXO MUST also:
- set a time lock for AT LEAST six months (or equivalent block periods);
- provide the value of the transaction in the signature metadata; and
- provide the public key for the spending key for the output in the signature metadata.
This UTXO MUST also store:
- The value of the VN deposit UTXO as a u64.
- The value of the public key for the spending key for the output as 32 bytes in little-endian order;
- The VN's Node ID. This can be validated by following the Renewal transaction kernel chain.
- The kernel hash of this transaction's kernel.
- A counter indicating that this is the n-th consecutive renewal. This counter will be confirmed by nodes and miners. The first renewal will have a counter value of one.
The previous VN deposit UTXO MUST NOT be spendable in a standard transaction (i.e. its time lock has not expired).
The previous VN deposit UTXO MUST expire within the next TWO WEEKS.
The transaction MAY provide additional inputs to cover transaction fees and increases in the Registration Deposit.
The transaction kernel MUST have the VN_Renewal bit flag set.
The transaction kernel MUST also store the hash of the previous renewal transaction kernel, or the registration kernel, if this is the first renewal.

One will notice that a VN's Node ID does not change as a result of a renewal transaction. Rather, every renewal adds to a chain linking back to the original registration transaction. It may be desirable to establish a long chain of renewals, in order to offer evidence of longevity and improve a VN's reputation.

RFC-0341: Asset Registration

Asset Registration Process

status: draft

Maintainer(s): Philip Robinson

Licence

The 3-Clause BSD Licence.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

Redistributions of this document must retain the above copyright notice, this list of conditions and the following disclaimer.
Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.

Language

Disclaimer

This document and its content are intended for information purposes only and may be subject to change or update without notice.

Goals

The aim of this Request for Comment (RFC) is to describe the process in which an Asset Issuer (AI) will need to engage to register a Digital Asset (DA) and commence its operation on the Digital Asset Network (DAN).

Related Requests for Comment

Description

Abstract

This document will describe the process through which an AI will go in order to:

register a DA on the base layer;
assemble a committee of Validator Nodes (VNs); and
commence operation of the DA on the DAN.

Asset Creation Instruction

The first step in registering and commencing the operation of an asset is that the AI MUST issue an asset creation transaction to the base layer.

This transaction will be time-locked for the length of the desired nomination period. This ensures that this transaction cannot be spent until the nomination period has elapsed so that it is present during the entire nomination process. The value of the transaction will be the asset_creation_fee described in RFC-0311. The AI will spend the transaction back to themselves, but locking this fee up at this stage achieves two goals:

Firstly, it makes it expensive to spam the network with asset creation transactions that a malicious AI does not intend to complete.
Secondly, it proves to the VNs that participate in the nomination process that the AI does indeed have the funds required to commence operation of the asset once the committee has been selected.

If the asset registration process fails, e.g. if there are not enough available VNs for the committee, then the AI can refund the fee to themselves after the time lock expires.

The transaction will contain the following extra metadata to facilitate the registration process:

The value of the transaction in clear text and the public spending key of the commitment so that it can be verified by third parties. A third party can verify the value of the commitment by using the information in (1) and (2) below, to calculate (3):
1. The output commitment is $ C = k \cdot G + v \cdot H $.
2. $ v $ and $ k \cdot G $ are provided in the metadata.
3. A verifier can calculate $ C - k \cdot G = v \cdot H $ and verify this value by multiplying the clear text $ v $ by $ H $ themselves.
A commitment (hash) to the asset parameters as defined by a DigitalAssetTemplate described in RFC-0311. This template will define all the parameters of the asset that the AI intends to register, including information the VNs need to know, such as what AssetCollateral is required to be part of the committee.

Once this transaction has been confirmed to the required depth on the blockchain, the nomination phase can begin.

Nomination Phase

The next step in registering an asset is for the AI to select a committee of VNs to manage the asset. The process to do this is described in RFC-0304. This process lasts as long as the time lock on the asset creation transaction described above. The VNs have until that time lock elapses to nominate themselves (in the case of an asset being registered using the committee_mode::PUBLIC_NOMINATION parameter in the DigitalAssetTemplate).

Asset Commencement

Once the nomination phase is complete and the AI has selected a committee as described in RFC-0304, the chosen committee and AI are ready to commit their asset_creation_fee and AssetCollaterals to commence the operation of the asset. This is done by the AI and the committee members collaborating to build the initial Checkpoint of the asset. When this Checkpoint transaction is published to the base layer, the digital asset will be live on the DAN. The Checkpoint transaction is described in RFC_0220.

RFC-0300/DAN

Digital Assets Network

status: draft

Maintainer(s): Cayle Sharrock and Philip Robinson

Licence

The 3-Clause BSD Licence.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

Redistributions of this document must retain the above copyright notice, this list of conditions and the following disclaimer.
Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.

Language

Disclaimer

This document and its content are intended for information purposes only and may be subject to change or update without notice.

Goals

The aim of this Request for Comment (RFC) is to describe the key features of the Tari second layer, also known as the Digital Assets Network (DAN)

Related Requests for Comment

Description

Abstract

Digital Assets (DAs) are managed by committees of special nodes called Validator Nodes (VNs):

VNs manage digital asset state change and ensure that the rules of the asset contracts are enforced.
VNs form a peer-to-peer communication network that together defines the Tari DAN.
VNs register themselves on the base layer and commit collateral to prevent Sybil attacks.
Scalability is achieved by sacrificing decentralization. Not all VNs manage every asset. Assets are managed by subsets of the DAN, called VN committees. These committees reach consensus on DA state amongst themselves.
VNs earn fees for their efforts.
DA contracts are not Turing complete, but are instantiated by Asset Issuers (AIs) using DigitalAssetTemplates that are defined in the DAN protocol code.

Digital Assets

DA contracts are not Turing complete, but are selected from a set of DigitalAssetTemplates that govern the behaviour of each contract type. For example, there could be a Single-use Token template for simple ticketing systems, a Coupon template for loyalty programmes, and so on.
The template system is intended to be highly flexible and additional templates can be added to the protocol periodically.
Asset issuers can link a Registered Asset Issuer Domain (RAID) ID in an OpenAlias TXT public Domain Name System (DNS) record to a Fully Qualified Domain Name (FQDN) that they own. This is to help disambiguate similar contracts and improve the signal-to-noise ratio from scam or copycat contracts.

An AI will issue a DA by constructing a contract from one of the supported set of DigitalAssetTemplates. The AI will choose how large the committee of VNs will be for this DA, and have the option to nominate Trusted Nodes to be part of the VN committee for the DA. Any remaining spots on the committee will be filled by permissionless VNs that are selected according to a CommitteeSelectionStrategy. This is a strategy that an AI will use to select from the set of potential candidate VNs that nominated themselves for a position on the committee when the AI broadcast a public call for VNs during the asset creation process. For the VNs to accept the appointment to the committee, they will need to put up the specified collateral.

RFC-0301/NamespaceRegistration

Namespace Registration on Base Layer

status: raw

Maintainer(s): Hansie Odendaal

Licence

The 3-Clause BSD Licence.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

Redistributions of this document must retain the above copyright notice, this list of conditions and the following disclaimer.
Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.

Language

Disclaimer

This document and its content are intended for information purposes only and may be subject to change or update without notice.

Goals

The aim of this Request for Comment (RFC) is to describe and specify the process for creating and linking an asset issuer specified domain name with a digital asset on the Digital Assets Network (DAN).

Related Requests for Comment

Description

Background

Alternative Approaches

In order to easily differentiate digital assets in the DAN, apart from some unique unpronounceable character string, a human-readable identifier (domain name) is required. It is perceived that shorter names will have higher value due to branding and marketability. The question is, how this can be managed elegantly? It is also undesirable if, for example, the real Disney is forced to use the long versioned "disney.com-goofy-is-my-asset-yes" because some fake Disneys claimed "goofy" and "disney.com-goofy" and everything in-between.

One method to curb name space squatting is to register names on the base layer layer with a domain name registration transaction. Let us call such a name a Registered Asset Issuer Name (RAIN). To make registering RAINs difficult enough to prevent spamming the network, a certain amount of Tari coins must be committed in a burn (permanently destroy) or a time-locked pay-to-self type transaction. Lots of management overhead will be associated with such a scheme, even if domainless assets are allowed. However, it would be impossible to stop someone from registering, say, a "disney.com" RAIN if they did not own the real "disney.com" Fully Qualified Domain Name (FQDN).

Another approach would be to make use of the public Domain Name System (DNS) and to link the FQDNs that are already registered, to the digital assets in the DAN, making use of OpenAlias text (TXT) DNS records on an FQDN. Let us call this a Registered Asset Issuer Domain (RAID) TXT record. If we hash a public key and FQDN pair, it will provide us with a unique RAID_ID (RAID Identification). The RAID_ID will serve a similar purpose in the DAN as a Top Level Domain (TLD) in a DNS, as all digital assets belonging to a specific asset issuer could then be grouped under the FQDN by inference. To make this scheme more elaborate, but potentially unnecessary, all such RAID_IDs could also be registered on the base layer, similar to the RAIN scheme.

If the standard Mimblewimble protocol is followed, a new output feature can be defined to cater for a RAID tuple (RAID_ID, PubKey) that is linked to a specific Unspent Transaction Output (UTXO). The RAID_ID could be based on a Base58Check variant applied to Hash256(PubKey || FQDN). If the amount of Tari coins associated with the RAID tuple transaction is burned, (RAID_ID, PubKey) will forever be present on the blockchain and can increase blockchain bloat. On the other hand, if those Tari coins are spent back to their owner with a specific time lock, it will be possible to spend and prune that UTXO later on. While that UTXO remains unspent, the RAID tuple will be valid, but when all or part of it is spent, the RAID tuple will disappear from the blockchain. Such a UTXO will thus be "coloured" while unspent, as it will have different properties to a normal UTXO. It will also be possible to recreate the original RAID tuple by registering it using the original Hash256(PubKey || FQDN).

Thinking about the make-up of the RAID_ID, it is evident that it can easily be calculated on the fly using the public key and FQDN, the values of which will always be known. The biggest advantage of having the RAID tuple on the base layer is that of embedded consensus, where it will be validated (as no duplicates can be allowed) and mined before it can be used. However, this comes at the cost of more complex code, a more elaborate asset registration process and higher asset registration fees.

This Request for Comment

This RFC explores the creation and use of RAID TXT records to link asset issuer-specified domain names with digital assets on the DAN, without RAID_IDs being registered on the base layer.

Requirements

OpenAlias TXT DNS Records

An OpenAlias TXT DNS record [1] on an FQDN is a single string and starts with "oa1:<name>" field followed by a number of key-value pairs. Standard (optional) key values are: "recipient_address"; "recipient_name"; "tx_description"; "tx_amount"; "tx_payment_id"; "address_signature"; and "checksum". Additional key values may also be defined. Only entities with write access to a specific DNS record will be able to create the required TXT DNS record entries.

TXT DNS records are limited to multiple strings of size 255, and as the User Datagram Protocol (UDP) size is 512 bytes, a TXT DNS record that exceeds that limit is less optimal [2, Sections 3.3 and 3.4]. Some hosting sites also place limitations on TXT DNS record string lengths and concatenation of multiple strings as per [2]. The basic idea of this specification is to make the implementation robust and flexible at the same time.

Req - Integration with public DNS records MUST be used to ensure valid ownership of an FQDN that needs to be linked to a digital asset on the DAN.

Req - The total size of the OpenAlias TXT DNS record SHOULD NOT exceed 255 characters.

Req - The total size of the OpenAlias TXT DNS record MUST NOT exceed 512 characters.

Req - The OpenAlias TXT DNS record implementation MUST make provision to interpret entries that are made up of more than one string as defined in [2].

Req - The OpenAlias TXT DNS record SHOULD adhere to the formatting requirements as specified in [1].

Req - The OpenAlias TXT DNS record MUST be constructed as follows:

OpenAlias TXT DNS Record Field	OpenAlias TXT DNS Record Data
oa1:<name>	"oa1:tari"
pk	<256 bit public key, in hexadecimal format (64 characters), that is converted into a `Base58` encoded string (44 characters)>
raid_id	<`RAID_ID` (refer to RAID_ID) (15 characters)>
nonce	<256 bit public nonce, in hexadecimal format (64 characters), that is converted into a `Base58` encoded string (44 characters)>
sig	<Asset issuer's 256 bit Schnorr signature for the `RAID_ID` (refer to RAID_ID), in hexadecimal format (64 characters), that is converted into a `Base58` encoded string (44 characters)>
desc	<Optional RAID description>; ASCII String; Up to 48 characters for the condensed version (using only one string) and up to 235 characters (when spanning two strings).
crc	<CRC-32 checksum of the entire record, up to but excluding the checksum key-value pair (starting at "oa1:tari" and ending at the last ";" before the checksum key-value pair) in hexadecimal format (eight characters)>

Examples: Two example OpenAlias TXT DNS records are shown; the first is a condensed version and the second spans two strings:

RAID_ID:
public key    = ca469346d7643336c19155fdf5c6500a5232525ce4eba7e4db757639159e9861
FQDN          = disney.com
 -> id        = RYqMMuSmBZFQkgp
   
base58 encodings:
public key    = ca469346d7643336c19155fdf5c6500a5232525ce4eba7e4db757639159e9861
 -> base58    = EcbmnM6PLosBzpyCcBz1TikpNXRKcucpm73ez6xYfLtg
public nonce  = fc2c5fce596338f43f70dc0ce14659fdfea1ba3e588a7c6fa79957fc70aa1b4b
 -> base58    = 5ctFNnCfBrP99rT1AFmj1WPyMD8uAdNUTESHhLoV3KBZ
signature     = 7dc54ec98da2350b0c8ed0561537517ac6f93a37f08a34482824e7df3514ce0d
 -> base58    = 9TxTorviyTJAaVJ4eY4AQPixwLb6SDL4dieHff6MFUha
   
OpenAlias TXT DNS record (condensed: 212 characters):
IN   TXT = "oa1:tari pk=EcbmnM6PLosBzpyCcBz1TikpNXRKcucpm73ez6xYfLtg;id=RYqMMuSmBZFQkgp;
nonce=5ctFNnCfBrP99rT1AFmj1WPyMD8uAdNUTESHhLoV3KBZ;sig=9TxTorviyTJAaVJ4eY4AQPixwLb6SDL4dieHff6MFUha;
desc=Cartoon characters;crc=176BE80C"

OpenAlias TXT DNS record (spanning two strings: string 1 = 179 characters and string 2 = 250 characters):
IN   TXT = "oa1:tari pk=EcbmnM6PLosBzpyCcBz1TikpNXRKcucpm73ez6xYfLtg; id=RYqMMuSmBZFQkgp; 
nonce=5ctFNnCfBrP99rT1AFmj1WPyMD8uAdNUTESHhLoV3KBZ; sig=9TxTorviyTJAaVJ4eY4AQPixwLb6SDL4dieHff6MFUha; "
"desc=Cartoon characters: Mickey Mouse\; Minnie Mouse\; Goofy\; Donald Duck\; Pluto\; Daisy Duck\; 
Scrooge McDuck\; Launchpad McQuack\; Huey, Dewey and Louie\; Bambi\; Thumper\; Flower\; Faline\; 
Tinker Bell\; Peter Pan and Captain Hook.; crc=54465902"

RAID_ID

Because the RAID_ID does not exist as an entity on the base layer or in the DAN, it cannot be owned or transferred, but can only be verified as part of the OpenAlias TXT DNS record [1] verification. If an asset creator chooses not to link a RAID_ID and FQDN, a default network-assigned RAID_ID will be used in the digital asset registration process.

Req - A default RAID_ID MUST be used where it will not be linked to an FQDN, e.g. it MAY be derived from a default input string "No FQDN".

Req - An FQDN-linked (non-default) RAID_ID MUST be derived from the concatenation PubKey || <FQDN>.

Req - All concatenations of inputs for any hash algorithm MUST be done without adding any spaces.

Req - The hash algorithm MUST be Blake2b using a 32 byte digest size, unless otherwise specified.

Req - Deriving a RAID_ID MUST be calculated as follows:

Inputs for all hashing algorithms used to calculate the RAID_ID MUST be lower-case characters.
Stage 1 - MUST select the input string to use (either "No FQDN" or PubKey || <FQDN>).
- Example: Mimblewimble public key ca469346d7643336c19155fdf5c6500a5232525ce4eba7e4db757639159e9861 and FQDN disney.com are used here, resulting in ca469346d7643336c19155fdf5c6500a5232525ce4eba7e4db757639159e9861disney.com.
Stage 2 - MUST perform Blake2b hashing on the result of stage 1 using a 10 byte digest size.
- Example: In hexadecimal representation ff517a1387153cc38009 or binary representation\xffQz\x13\x87\x15<\xc3\x80\t.
Stage 3 - MUST concatenate the RAID_ID identifier byte, 0x62, with the result of stage 2.
- Example: 62ff517a1387153cc38009 `.
Stage 4 - MUST convert the result of stage 3 from a byte string into a Base58 encoded string. This will result in a 15 character string starting with R.
- Example: The resulting RAID_ID will be RYqMMuSmBZFQkgp.

Req - A valid RAID_ID signature MUST be a 256 bit Schnorr signature defined as s = PvtNonce + e·PvtKey with the challenge e being e = Blake2b(PubNonce || PubKey || RAID_ID).

Sequence of Events

The sequence of events leading up to digital asset registration is perceived as follows:

The asset issuer will decide if the default RAID_ID or a RAID_ID that is linked to an FQDN must be used for asset registration. (Note: A single linked (RAID_ID, FQDN) tuple may be associated with multiple digital assets from the same asset issuer.)
Req - If a default RAID_ID is required:
1. The asset issuer MUST use the default RAID_ID (refer to RAID_ID).
2. The asset issuer MUST NOT sign the RAID_ID.
Req - If a linked (RAID_ID, FQDN) tuple is required:
1. The asset issuer MUST create a RAID_ID.
2. The asset issuer MUST sign the RAID_ID as specified (refer to RAID_ID).
3. The asset issuer MUST create a valid TXT DNS record (refer to OpenAlias TXT DNS Records).
Req - Validator Nodes (VNs) MUST only allow a valid RAID_ID to be used in the digital asset registration process.
Req - VNs MUST verify the OpenAlias TXT DNS record if a linked (RAID_ID, FQDN) tuple is used:
1. Verify that all fields have been completed as per the specification (refer to OpenAlias TXT DNS Records).
2. Verify that the RAID_ID can be calculated from information provided in the TXT DNS record and the FQDN of the public DNS record it is in.
3. Verify that the asset issuer's RAID_ID signature is valid.
4. Verify the checksum.

Confidentiality and Security

To prevent client lookups from leaking, OpenAlias recommends making use of DNSCrypt, resolution via DNSCrypt-compatible resolvers that support Domain Name System Security Extensions (DNSSEC) and without DNS requests being logged.

Req - Token Wallets (TWs) and VNs SHOULD implement the following confidentiality and security measures when dealing with OpenAlias TXT DNS records:

All queries SHOULD make use of the DNSCrypt protocol.
Resolution SHOULD be forced via DNSCrypt-compatible resolvers that:
- support DNSSEC;
- do not log DNS requests.
The DNSSEC trust chain validity:
- SHOULD be verified;
- SHOULD be reported back to the asset issuer.

References

[1] Crate Openalias [online]. Available: https://docs.rs/openalias/0.2.0/openalias/index.html. Date accessed: 2019-03-05.

[2] RFC 7208: Sender Policy Framework (SPF) for Authorizing Use of Domains in Email, Version 1 [online]. Available: https://tools.ietf.org/html/rfc7208. Date accessed: 2019-03-06.

RFC-0302/ValidatorNode

Validator Nodes

status: draft

Maintainer(s): Cayle Sharrock and Philip Robinson

Licence

The 3-Clause BSD Licence.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

Redistributions of this document must retain the above copyright notice, this list of conditions and the following disclaimer.
Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.

Language

Disclaimer

This document and its content are intended for information purposes only and may be subject to change or update without notice.

Goals

The aim of this Request for Comment (RFC) is to describe the responsibilities of Validator Nodes (VNs) on the Digital Asset Network (DAN).

Related Requests for Comment

Description

Abstract

Validator Nodes form the basis of the second-layer DAN. All actions on this network take place by interacting with VNs. Some examples of actions that VNs will facilitate are:

issuing a Digital Asset (DA);
querying the state of a DA and its constituent tokens; and
issuing an instruction to change the state of a DA or tokens.

VNs will also perform archival functions for the assets they manage. The lifetime of these archives and the fee structure for this function are still being discussed.

Registration

VNs register themselves on the Base Layer using a special transaction type.

Validator node registration is described in RFC-0322.

Execution of Instructions

VNs are expected to manage the state of DAs on behalf of DA issuers. They receive fees as reward for doing this.

DAs consist of an initial state plus a set of state transition rules. These rules are set by the Tari protocol, but will usually provide parameters that must be specified by the Asset Issuer.
The set of VNs that participate in managing state of a specific DA is called a Committee. A committee is selected during the asset issuance process and membership of the committee can be updated at Checkpoints.
The VN is responsible for ensuring that every state change in a DA conforms to the contract's rules.
VNs accept DA Instructions from clients and peers. Instructions allow for creating, updating, expiring and archiving DAs on the DAN.
VNs provide additional collateral, called AssetCollateral, when accepting an offer to manage an asset, which is stored in a multi-signature (multi-sig) Unspent Transaction Output (UTXO) on the base layer. This collateral can be taken from the VN if it is proven that the VN engaged in malicious behaviour.
VNs participate in fraud-proof validations in the event of consensus disputes (which could result in the malicious VN's collateral being slashed).
DA metadata (e.g. large images) is managed by VNs. The large data itself will not be stored on the VNs, but in an external location, and a hash of the data can be stored. Whether the data is considered part of the state (and thus checkpointed) or out of state depends on the type of DA contract employed.

Fees

Fees will be paid to VNs based on the amount of work they did during a checkpoint period. The fees will be paid from a fee pool, which will be collected and reside in a UTXO that is accessible by the committee. The exact mechanism for the payment of the fees by the committee and who pays the various types of fees is still under discussion.

Checkpoints

VNs periodically post checkpoint summaries to the base layer for each asset that they are managing. The checkpoints will form an immutable record of the DA state on the base layer. There will be two types of checkpoints:

An Opening Checkpoint (OC) will:
- specify the members of the VN committee;
- lock up the collateral for the committee members for this checkpoint period; and
- collect the fee pool for this checkpoint period from the Asset Issuer into a multi-sig UTXO under the control of the committee. This can be a top-up of the fees or a whole new fee pool.
A Closing Checkpoint (CC) will:
- summarize the DA state on the base layer;
- release the fee payouts;
- release the collateral for the committee members for this checkpoint period; and
- allow for committee members to resign from the committee.

After a DA is issued, there will immediately be an OC. After a checkpoint period there will then be a CC, followed immediately by an OC for the next period. We will call this set of checkpoints an Intermediate checkpoint, which could be a compressed combination of an OC and CC. This will continue until the end of the asset's lifetime, when there will be a final CC that will be followed by the retirement of the asset.

graph LR; subgraph Asset Issuance IssueAsset-->OC1; end OC1-->CC1; subgraph Intermediate Checkpoint CC1-->OC2; end OC2-->CC2; subgraph Intermediate Checkpoint CC2-->OC3; end OC3-->CC3; subgraph Asset Retirement CC3-->RetireAsset; end

Consensus

Committees of VNs will use a ConsensusStrategy to manage the process of:

propagating signed instructions between members of the committee; and
determining when the threshold has been reached for an instruction to be considered valid.

Part of the Consensus Strategy will be mechanisms for detecting actions by Bad Actors. The nature of the enforcement actions that can be taken against bad actors is still to be decided.

Network Communication

The VNs will communicate using a Peer-to-Peer (P2P) network. To facilitate this, the VNs must perform the following functions:

VNs MUST maintain a list of peers, including which assets each peer is managing.
VNs MUST relay instructions to members of the committee that are managing the relevant asset.
VNs MUST respond to requests for information about DAs that they manage on the DAN.
VNs and clients can advertise public keys to facilitate P2P communication encryption.

RFC-0304/VNCommittees

Validator Node Committee Selection

status: draft

Maintainer(s): Philip Robinson

Licence

The 3-Clause BSD Licence.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

Redistributions of this document must retain the above copyright notice, this list of conditions and the following disclaimer.
Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.

Language

Disclaimer

This document and its content are intended for information purposes only and may be subject to change or update without notice.

Goals

The aim of this Request for Comment (RFC) is to describe the process an Asset Issuer (AI) will go through in order to select the committee of Validator Nodes (VNs) that will serve a given Digital Asset (DA).

Related Requests for Comment

Description

Abstract

Digital Assets (DAs) are managed by committees of nodes called Validator Nodes (VNs), as described in RFC-0300 and RFC-0302. During the asset creation process, described in RFC-0341, the Asset Issuer (AI) MUST select a committee of VNs to manage their asset. This process consists of the following steps:

Candidate VNs MUST be nominated to be considered for selection by the AI.
The AI MUST employ a CommitteeSelectionStrategy to select VNs from the set of nominated candidates.
The AI MUST make an offer of committee membership to the selected VNs.
Selected VNs MAY accept the offer to become part of the committee by posting the required AssetCollateral.

Nomination

The first step in assembling a committee is to nominate candidate VNs. As described in RFC-0311, an asset can be created with two possible committee_modes - CREATOR_NOMINATION or PUBLIC_NOMINATION:

In CREATOR_NOMINATION mode, the AI nominates candidate committee members directly. The AI will have a list of permissioned Trusted Nodes that they want to act as the committee. The AI will contact the candidate VNs directly to inform them of their nomination.
In PUBLIC_NOMINATION mode, the AI does not have a list of Trusted Nodes and wants to source unknown VNs from the network. In this case, the AI broadcasts a public call for nomination to the Tari network using the peer-to-peer messaging protocol described in RFC-0172. This call for nomination contains all the details of the asset. VNs that want to participate will then nominate themselves by contacting the AI.

Selection

Once the AI has received a list of nominated VNs, it must make a selection, assuming enough VNs were nominated to populate the committee. The AI will employ some CommitteeSelectionStrategy in order to select the committee from the candidate VNs that have been nominated. This strategy might aim for a perfectly random selection, or perhaps it will consider some metrics about the candidate VNs, such as the length of their VN registrations. These metrics might indicate that they are reliable and have not been blacklisted for poor or malicious performance.

A consideration when selecting a committee in PUBLIC_NOMINATION mode will be the size of the pool of nominated VNs. The size of this pool relative to the size of the committee to be selected will be linked to a risk profile. If the pool contains very few candidates, then it will be much easier for an attacker to have nominated their own nodes in order to obtain a majority membership of the committee. For example, if the AI is selecting a committee of 10 members using a uniformly random selection strategy and only 12 public nominations are received, an attacker only requires control of six VNs to achieve a majority position in the committee. In contrast, if 100 nominations are received and the AI performs a uniformly random selection, an attacker would need to control more than 50 of the nominated nodes in order to achieve a majority position in the committee.

Offer Acceptance

Once the selection has been made by the AI, the selected VNs will be informed and they will be made an offer of membership. If the VNs are still inclined to join the committee, they will accept the offer by posting the AssetCollateral required by the asset to the base layer during the initial Checkpoint transaction built to commence the operation of the asset.

Asset Management

This section contains RFCs that describe various aspects of asset management in the Tari network.

RFC-0311/AssetTemplates

Digital Asset Templates

status: draft

Maintainer(s): Cayle Sharrock

Licence

The 3-Clause BSD Licence.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

Redistributions of this document must retain the above copyright notice, this list of conditions and the following disclaimer.
Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.

Language

Disclaimer

This document and its content are intended for information purposes only and may be subject to change or update without notice.

Goals

The aim of this Request for Comment (RFC) is to describe the Tari Digital Asset templating system for smart contract definition.

The term “smart contracts” in this document is used to refer to a set of rules enforced by computers. These smart contracts are not Turing complete, such as those executed by the Ethereum Virtual Machine (VM).

Related Requests for Comment

Description

Motivation

The reasons for issuing assets on Tari under a templating system, rather than a scripting language (whether Turing complete or not), are manifold:

A scripting language, irrespective of how simple it is, limits the target market for asset issuers to developers, or people who pay developers.
The market doesn’t want general smart contracts. This is evidenced by the fact that the vast majority of Ethereum transactions go through ERC-20 or ERC-721 contracts, which are literally contract templates.
The attack surface for smart contracts is reduced considerably, to the node software itself.
Bugs can be fixed for all contracts simultaneously by using a template versioning system. Existing assets can opt in to fixes by migrating assets to a new version of the contract.
Contracts will have better Quality Assurance (QA), since more eyes are looking at fewer contract code sets.
Transmission, storage and processing of contracts will be more efficient, as one only has to deal with the parameters, and not the logic of the contract. Furthermore, the cost for users is usually lower, since there's no need to add friction or extra costs to contract execution (e.g. Ethereum gas) to work around the halting problem.

Implementation

Assets are created on the Tari network by issuing a create_asset instruction from a wallet or client, and broadcasting it to the Tari Digital Assets Network (DAN).

The instruction is in JSON format and MUST contain the following fields:

Name	Type	Description
Asset Description
issuer	PubKey	The public key of the creator of the asset. Refer to issuer.
name	`string[64]`	The name or identifier for the asset. Refer to Name and Description.
description	`string[216]`	A short description of the asset - with name, fits in a tweet. Refer to Name and Description.
raid_id	`string[15]`	The Registered Asset Issuer Domain (RAID_ID) for the asset.
fqdn	`string[*]`	The Fully Qualified Domain Name (FQDN) corresponding to the `raid_id`. Up to 255 characters in length; or "No_FQDN" to use the default.
public_nonce	PubKey	Public nonce part of the creator signature.
template_id	`u64`	The template descriptor. Refer to Template ID.
asset_expiry	`u64`	A timestamp or block height after which the asset will automatically expire. Zero for arbitrarily long-lived assets.
Validation Committee Selection
committee_mode	`u8`	The validation committee nomination mode, either `CREATOR_NOMINATION` (0) or `PUBLIC_NOMINATION` (1).
committee_parameters	Object	Refer to Committee Parameters.
asset_creation_fee	`u64`	The fee the issuer is paying, in microTari, for the asset creation process.
commitment	`u256`	A time-locked commitment for the asset creation fee.
initial_state_hash	`u256`	The hash of the canonical serialization of the initial template state (of the template-specific data).
initial_state_length	`u64`	Size in bytes of initial state.
Template-specific Data	Object	Template-specific metadata can be defined in this section.
Signatures
metadata_hash	`u256`	A hash of the previous three sections' data, in canonical format (`m`).
creator_sig	`u256`	A digital signature of the message `H(R ‖ P ‖ RAID_ID ‖ m)`, using the asset creator’s private key corresponding to the `issuer` Public Key Hash (PKH).
commitment_sig	`u256`	A signature proving the issuer is able to spend the commitment to the asset fee.

Committee Parameters

If committee_mode is CREATOR_NOMINATION, the committee_parameters object is:

Name	Type	Description
trusted_node_set	Array of PKH	See below.

Only the nodes in the trusted node set will be allowed to execute instructions for this asset.

If committee_mode is PUBLIC_NOMINATION, the committee_parameters object is:

Name	Type	Description
node_threshold	`u32`	The required number of Validator Nodes (VNs) that must register to execute instructions for this asset.
minimum_collateral	`u64`	The minimum amount of Tari a VN must put up in collateral in order to execute instructions for this asset.
node_selection_strategy	`u32`	The selection strategy to employ allowing nodes to register to manage this asset.

Issuer

Anyone can create new assets on the Tari network from their Tari Collections client. The client will provide the Public Key Hash (PKH) and sign the instruction. The client needn’t use the same private key each time.

Name and Description

These fields are purely for information purposes. They do not need to be unique and do not act as an asset ID.

RAID ID

The RAID_ID is a 15-character string that associates the asset issuer with a registered Internet domain name on the Domain Name System (DNS).

If it is likely that a digital asset issuer will be issuing many assets on the Tari Network (hundreds or thousands), the issuer should strongly consider using a registered domain (e.g. acme.com). This is done via OpenAlias on the domain owner's DNS record, as described in RFC-0301. A RAID prevents spoofing of assets from copycats or other malicious actors. It also simplifies asset discovery.

Assets from issuers that do not have a RAID are all grouped under the default RAID.

RAID owners must provide a valid signature proving that they own the given domain when creating assets.

Fully Qualified Domain Name

The Fully Qualified Domain Name (FQDN) that corresponds to the raid_id or the string "NO FQDN" to use the default RAID ID. Validator Nodes (VNs) will calculate and check that the RAID ID is valid when validating the instruction signature.

Public Nonce

A single-use public nonce to be used in the asset signature.

Asset Identification

Assets are identified by a 64-character string that uniquely identifies an asset on the network:

Bytes	Description
8	Template type (hex)
4	Template version (hex)
4	Feature flags (hex)
15	RAID identifier (Base58)
1	A period character, `.`
32	Hex representation of the `metadata_hash` field

This allows assets to be deterministically identified from their initial state. Two different creation instructions leading to the same hash refer to the same single asset, by definition. VNs maintain an index of assets and their committees, and so can determine whether a given asset already exists; and MUST reject any create_asset instruction for an existing asset.

Template ID

Tari uses templates to define the behaviour for its smart contracts. The template ID refers to the type of digital asset being created.

Note: Integer values are given in little-endian format, i.e. the least significant bit is first.

The template number is a 64-bit unsigned integer and has the following format, with 0 representing the least significant bit:

Bit Range	Description
0 - 31	Template type (0 - 4,294,967,295)
32 - 47	Template version (0 - 65,535)
48	Beta Mode flag
49	Confidentiality flag
50 - 63	Reserved (must be 0)

The lowest 32 bits refer to the canonical smart contract type, i.e. the qualitative types of contracts the network supports. Many assets can be issued from a single template.

Template types below 65,536 (2¹⁶) are public, community-developed templates. All VNs MUST implement and be able to interpret instructions related to these templates.

Template types 65,536 and above are opt-in or proprietary templates. There is no guarantee that any given VN will be able to manage assets on these templates. Part of the committee selection and confirmation process for new assets will be an attestation by VNs that they are willing and able to manage the asset under the designated template rules.

A global registry of opt-in template types will be necessary to prevent collisions (public templates existence will be evident from the Validator Node source code), possibly implemented as a special transaction type on the base layer, which is perfectly suited for hosting such a registry. The details of this will be covered in a separate proposal.

Examples of template types may be:

Template Type	Asset
1	Simple single-use tokens
2	Simple coupons
20	ERC-20-compatible
...	...
120	Collectible cards
144	In-game items
721	ERC-721-compatible
...	...
65,537	Acme In game items
723,342	CryptoKitties v8

The template ID may also set one or more feature flags to indicate that the contract is:

Experimental, or in testing phase (bit 48).
Confidential. The definition of confidential assets and their implementation had not been finalized at the time of writing.

Wallets/client apps SHOULD have settings to allow, or otherwise completely ignore, asset types on the network that have certain feature flags enabled. For instance, most consumer wallets should never interact with templates that have the “Beta mode” bit set. Only developers' wallets should ever even see that such assets exist.

Asset Expiry

Asset issuers can set a future expiry date or block height, after which the asset will expire and nodes will be free to expunge any/all state relating to the asset from memory after a fixed grace period. The grace period is to allow interested parties (e.g. the issuer) to take a snapshot of the final state of the contract if they wish (e.g. proving that you had a ticket for that epic World Cup final game, even after the asset no longer exists on the DAN).

Nodes will publish a final checkpoint on the base layer soon after expiry and before purging an asset.

The expiry_date is a Unix epoch, representing the number of seconds since 1 January 1970 00:00:00 UTC if the value is greater than 1,500,000,000; or a block height if it is less than that value (with 1 min blocks this scheme is valid until the year 4870).

Expiry times should not be considered exact, since nodes don’t share the same clocks and block heights, and time proxies become more inaccurate the further out you go (since height in the future is dependent on hash rate).

Signature Validation

Validator nodes will verify the creator_sig for every create_asset instruction before propagating the instruction to the network. The process is as follows:

The VN MUST calculate the metadata hash by hashing the canonical representation of all the data in the first three sections of the create_asset instruction.
The VN MUST compare this calculated value to the value given in the metadata_hash field. If they do not match, the VN MUST drop the instruction and STOP.
The VN MUST calculate the RAID ID from the fqdn and issuer fields as specified in RFC-0301.
The VN MUST compare the calculated RAID ID with the value given in the raid_id field. If they do not match, the VN MUST drop the instruction and STOP.
If the fqdn is `"No FQDN", then skip to step 9.
The VN MUST Look up the OpenAlias TXT record at the domain given in fqdn. If the record does not exist, then the VN MUST drop the instruction and STOP.
The VN MUST check that each of the public key and RAID ID in the TXT record match the values in the create_asset instruction. If any values do not match, the VN MUST then drop the instruction and STOP.
The VN MUST validate the registration signature in the TXT record, using the TXT record's nonce, the issuer's public key and the RAID ID. If the signature does not verify, the VN MUST drop the instruction and STOP.
The VN MUST validate the signature in the creator_sig field against the challenge built up from the issuer's public key, the nonce given in public_nonce field, the raid_id field and the metadata_hash field.

If step 9 passes, then the VN has proven that the create_asset contains a valid RAID ID, and that if a non-default FQDN was provided, the owner of that domain provided the create_asset instruction. In this case, the VN SHOULD propagate the instruction to the network.

RFC-0345/AssetLifeCycle

Asset Life Cycle

status: draft

Maintainer(s): Cayle Sharrock

Licence

The 3-Clause BSD Licence.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

Redistributions of this document must retain the above copyright notice, this list of conditions and the following disclaimer.
Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.

Language

Disclaimer

This document and its content are intended for information purposes only and may be subject to change or update without notice.

This document may include preliminary concepts that may or may not be in the process of being developed by the Tari community. The release of this document is intended solely for review and discussion by the community regarding the technological merits of the potential system outlined herein.

Goals

Related Requests for Comment

Description

Introduction

Tari digital assets are created on the Digital Assets Network, managed by a Validator Node (VN) committee, see RFC-0340, and are operated under the rules of the template that governs the asset.

A given version of a template provides an immutable definition of the type of information (the state) that is managed during the lifetime of the asset. All the rules that govern reading and updating of the asset state, and rules regarding any transfers of tokens that may be defined as part of the asset are governed by the template code.

This immutability is what allows VN committees to reach rapid consensus about what an asset’s state looks
like after instructions are executed.

However, immutability is a major problem when faced with typical software and business development challenges such as software bugs, or changes in legal and operational requirements.

The Tari network is designed to accommodate these requirements by offering a migration path for assets from one version of a template to another version of a template.

Asset migration

To carry out a successful migration, the following requirements must be met:

The validator node committee for the asset must support the migration path. This entails that every VN in the committee has a MigrateAsset class from the template_type.version combination of the existing asset to the template_type.version of the new asset.
The original asset issuer provides a valid migrate_asset instruction to the DAN.
- The asset issuer MUST provide any additional state that exists in the new template version and not the original.
- The original asset SHOULD be marked as retired. If so, the superseded_by field in the old asset will carry the new asset id once the new asset has been confirmed. We recommend retiring the old asset because all the keys that indicate ownership of tokens will be copied over; effectively re-using them; which can damage privacy.
- A policy is provided to determine the course of action to follow if any state from the old asset is illegal under the new template rules (e.g. If a new rule requires token.age to be > 21; what happens to any tokens where this condition fails?)

As part of the migration,

An entirely new asset is created with the full state of the old asset copied over into the new asset; supplemented with any additional state required in the new template.
A state validation run is performed; and any invalid state from the old asset is modified according to the migration policy.
Step 2 is repeated until a valid initial state is produced.
If Step 2 has run STATE_VALIDATION_LIMIT times and the initial state is still not valid, the migration instruction is rejected; the migrate_asset instruction will advise what should be done with the original asset in this case: either allow the original asset to continue as before, or retire it.
Once a valid initial state is produced, a new create_asset instruction is generated from the initial state and the migrate_asset instruction. Typically the same VN committee will be used for the new asset, but this needn’t be the case.

Once a successful migration has completed, any instructions to the old asset can return a simple asset_migrated response with the new asset ID, which will allow clients and wallets to seamlessly update their records.

Retiring Assets

Retiring an asset follows the same procedure as when as asset reaches its natural end-of-life: A final checkpoint is posted to the base layer and a grace period is given to allow DAN nodes and clients to take a snapshot of the final state if desired.

Resurrecting assets

It’s unreasonable to expect VNs to hold onto large chunks of state for assets that are effectively dead (e.g. ticket stubs long after the event is over). For this reason, assets are allowed to expire after which VNs can forget about the state and use that storage for something else.

However, it may be that interest in an asset resurfaces long after the asset expires (nostalgia being the multi-billion dollar industry it is today). The resurrect_asset instruction provides a mechanism to bring an asset back to life.

To resurrect an asset, the following conditions must be met:

The asset must have expired.
It must not be currently active (i.e. it hasn’t already been resurrected).
An asset issuer (not necessarily the original asset issuer) must provide funding for the new lifetime of the asset.
The asset issuer needs to have a copy of the state corresponding to the final asset checkpoint of the original asset.
The new asset issuer transmits a resurrect_asset instruction to the network. This instruction is identical to the original create_asset instruction with the following exceptions:
- The “initial state” merkle root must be equal to the final state checkpoint merkle root.
- The asset owner public key will be provided by the new asset issuer.
Third parties can interrogate the new committee asking them to provide a Merkle Proof for pieces of state that the third party (perhaps former asset owners) knows about. This can mitigate fraud attempts where parties can attempt to resurrect assets without actually having a copy of the smart contract state. If enough random state proofs are requested, or a single proof of enough random pieces of state, we can be confident that the asset resurrection is legitimate.

The VN committee for the resurrected asset need not bear any relation to the original VN committee. Once confirmed, the resurrected asset operates exactly like any other asset. An asset can expire and be resurrected multiple times (sequentially).

RFC-0340/VNConsensusOverview

Validator node consensus algorithm

status: draft

Maintainer(s): Cayle Sharrock

License

The 3-Clause BSD License.

Validator node consensus algorithm

status: draft

Maintainer(s): Cayle Sharrock

License

The 3-Clause BSD License.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

Redistributions of this document must retain the above copyright notice, this list of conditions and the following disclaimer.
Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

Redistributions of this document must retain the above copyright notice, this list of conditions and the following disclaimer.
Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.

Language

Disclaimer

The purpose of this document and its content is for information purposes only and may be subject to change or update without notice.

This document may include preliminary concepts that may or may not be in the process of being developed by the Tari community. The release of this document is intended solely for review and discussion by the community regarding the technological merits of the potential system outlined herein.

Goals

This document describes at a high level how smart contract state is managed on the Tari Digital Assets Network.

Related RFCs

Description

Overview

The primary problem under consideration here is for multiple machines running the same program (in the form of a Tari smart contract) to maintain agreement on what the state of the program is, often under adverse conditions, including unreliable network communication, malicious third parties, or even malicious peers running the smart contract.

In computer science terms, the problem is referred to as State Machine Replication, or SMR. If we want our honest machines (referred to as replicas in SMR parlance) to reach agreement in the face of arbitrary failures, then we talk about our system being Byzantine Fault Tolerant.

Tari Asset committees are chosen by the asset issuer according to RFC-0304. The committees form a fixed set of replicas, at the very least from checkpoint to checkpoint, and will typically be limited in size, usually less than ten, and almost always under 100. Note: These numbers are highly speculative based on an intuitive guess about the main use cases for Tari DAs, where we have

many 1-3-sized committees where the asset issuer and the VN committee are the same entity,
semi-decentralised assets of ±4-10 where speed trumps censorship-resistance,
a small number of 50-100 VNs where censorship-resistance trumps speed.

Because nodes cannot join and leave the committees at will, robust yet slow and expensive consensus approaches such as Nakamoto consensus can be dropped in favour of something more performant.

There is a good survey of consensus mechanisms on Tari Labs University.

From the point of view of a DAN committee, the ideal consensus algorithm is one that

Allows a high number of transactions per second, and doesn't have unnecessary pauses (i.e. a partially synchronous or asynchronous model).
Is Byzantine Fault tolerant.
Is relatively efficient from a network communication point of view (number of messages passed per state agreement).
Is relatively simple to implement (to reduce the bug and vulnerability surface in implementations).

A summary of some of the most well-known BFT algorithms is presented in this table.

A close reading of the algorithms presented suggest that LinBFT, which is based on HotStuff BFT provide the best trade-offs for the goals that a DAN committee is trying to achieve:

The algorithm is optimistic, i.e. as soon as quorum is reached on a particular state, the committee can move onto the next one. There is no need to wait for the "timeout" period as we do in e.g. Tendermint. This allows instructions to be executed almost as quickly as they are received.
The algorithm is efficient in communication, requiring O(n) messages per state agreement in most practical cases. This is compared to e.g. PBFT which requires O(n⁴) messages.
The algorithm is modular and relatively simple to implement.

Potential drawbacks to using HotStuff include:

Each round required the election of a leader. Having a leader dramatically simplifies the consensus algorithm; it allows a linear number of messages to be sent between the leader and the other replicas in order to agree on the current state; and it allows a strict ordering to be established on instructions without having to resort to e.g. proof of work. However, if the choice of leader is deterministic, attackers can identify and potentially DDOS the leader for a given round, causing the algorithm to time out. There are ways to mitigate this attack for a specific round, as suggested in the LinBFT paper, such as using Verifiable Random Functions, but DDOSing a single replica means that, on average, the algorithm will time out every 1/n rounds.
The attack described above only pauses progress in Hotstuff for the timeout period. In similar protocols, e.g. Tendermint it can be shown to delay progress indefinitely.

Given these trade-offs, there is strong evidence to suggest that HotStuff BFT, when implemented on the Tari DAN will provide BFT security guarantees with liveness performance in the sub-second scale and throughput on the order of thousands of instructions per second, if the benchmarks presented in the HotStuff paper are representative.

Implementation

The HotStuff BFT algorithm provides a detailed description of the consensus algorithm. Only a summary of it is presented here. To reduce confusion, we adopt the HotStuff nomenclature to describe state changes, rounds and function names where appropriate.

Every proposed state change, as a result of replicas receiving instructions from clients is called a view. There is a function that every node can call that will tell it which replica will be the leader for a given view. Every view goes through three phases (Prepare, PreCommit, Commit) before final consensus is reached. Once a view reaches the Commit phase, it is finalised and will never be reversed.

As part of their normal operation, every replica broadcasts instructions it receives for its contract to its peers. These instructions are stored in a replica's instruction mempool.

When the leader selection function designates a replica as leader for the next view, it will try and execute all the instructions it currently has in its mempool to update the state for the next view. Following this it compiles a tuple of <valid-instructions, rejected-instructions, new-state>. This tuple represents the CMD structure described in HotStuff.

In parallel with this, the leader expects a set of NewView messages from the other replicas, indicating that the other replicas know that this replica is the leader for the next view.

Once a super-majority of these messages have been received, the leader composes a proposal for the next state by adding a new node to the state history graph (I'm calling it a state history graph to avoid naming confusion, but it's really a blockchain). It composes a message containing the new proposal, and broadcasts it to the other replicas.

Replicas, on receipt of the proposal, decide whether the proposal is valid, both from a protocol point of view (i.e. did the leader provide a well-formed proposal) as well as whether they agree on the new state (e.g. by executing the instructions as given and comparing the resulting state with that of the proposal). If there is agreement, they vote on the proposal by signing it, and sending their partial signature back to the leader.

When the leader has received a super-majority of votes, it sends a message back to the replicas with the (aggregated) set of signatures.

Replicas can validate this signature and provide another partial signature indicating that they've received the first aggregated signature for the proposal.

At this point, all replicas know that enough other replicas have received the proposal and are in agreement that it is valid.

In Tendermint, replicas would now wait for the timeout period to make sure that the proposal wasn't going to be superseded before finalising the proposal. But there is an attack described in the HotStuff paper that could stall progress at this point.

The HotStuff protocol prevents this by having a final round of confirmations with the leader. This prevents the stalling attack and also lets replicas finalise the proposal immediately on receipt of the final confirmation from the leader. This lets HotStuff proceed at "network" speed, rather than with a heartbeat dictated by the BFT synchronicity parameter.

Although there are 4 round trips of communication between replicas and the leader, the number of messages sent are O(n). It's also possible to stagger and layer these rounds on top of each other, so that there are always four voting rounds happening simultaneously, rather than waiting for one to complete in its entirety before moving onto the next one. Further details are given in the HotStuff paper.

Forks and byzantine failures

The summary of the HotStuff protocol given above describes the "Happy Path", when there are no breakdowns in communication, or when the leader is an honest node. In cases where the leader is unavailable, the protocol will time out, the current view will be abandoned, and all replicas will move onto the next view.

If a leader is not honest, replicas will reject its proposal, and move onto the next view.

If there is a temporary network partition, the chain may fork (up to a depth of three), but the protocol guarantees safety via the voting mechanism, and the chain will reconcile once the partition resolves.

Leader selection

HotStuff leaves the leader selection algorithm to the application. Usually, a round-robin approach is suggested for its simplicity. However, this requires the replicas to reliably self-order themselves before starting with SMR, which is a non-trivial exercise in byzantine conditions.

For Tari DAN committees, the following algorithm is proposed:

Every replica knows the Node ID of every other replica in the committee.
For a given view number, the Node ID with the closest XOR distance to the hash of the view number will be the leader for that view, where the hash function provides a uniformly random value of the same length as the Node ID.

Quorum Certificate

A Quorum certificate, or QC is proof that a super-majority of replicas have agreed on a given state. In particular, a QC consists of

The type of QC (depending on the phase in which the HotStuff pipeline the QC was signed),
The view number for the QC
A reference to the node in the state tree being ratified,
A signature from a super-majority of replicas.

Tari-specific considerations

As soon as a state is finalised, replicas can inform clients as to the result of instructions they have submitted (in the affirmative or negative). Given that HotStuff proceeds optimistically, and finalisation happens after 4 rounds of communication, it's anticipated that clients can receive a final response from the validator committee in under 500 ms for reasonably-sized committees (this value is speculation at present and will be updated once exploratory experiments have been carried out).

The Tari communication platform was designed to handle peer-to-peer messaging of the type described in HotStuff, and therefore the protocol implementation should be relatively straightforward.

The "state" agreed upon by the VN committee will not only include the smart-contract state, but instruction fee allocations and periodic checkpoints onto the base layer.

Checkpoints onto the base layer achieve several goals:

Offers a proof-of-work backstop against "evil committees". Without proof of work, there's nothing stopping an evil committee (one that controls a super-majority of replicas) from rewriting history. Proof-of-work is the only reliable and practical method that currently exists to make it expensive to change the history of a chain of records. Tari gives us a "best of both worlds" scenario wherein an evil committee would have to rewrite the base layer history (which does use proof-of-work) before they could rewrite the digital asset history (which does not).
They allow the asset issuer to authorise changes in the VN committee replica set.
It allows asset owners to have an immutable proof of asset ownership long after the VN committee has dissolved after the useful end-of-life of a smart contract.
Provides a means for an asset issuer to resurrect a smart contract long after the original contract has terminated.

When Validator Nodes run smart contracts, they should be run in a separate thread so that if a smart contract crashes, it does not bring the consensus algorithm down with it.

Furthermore, VNs should be able to quickly revert state to at least four views back in order to handle temporary forks. Nodes should also be able to initialise/resume a smart contract (e.g. from a crash) given a state, view number, and view history.

This implies that VNs, in addition to passing around HotStuff BFT messages, will expose additional APIs in order to

allow lagging replicas to catch up in the execution state.
Provide information to (authorised) clients regarding the most recent finalised state of the smart contract via a read-only API.
Accept smart-contract instructions from clients and forward these onto the other replicas in the VN committee.

RFC-0500/PaymentChannels

Payment channels

status: draft

Maintainer(s): Cayle Sharrock

Licence

The 3-Clause BSD Licence.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

Redistributions of this document must retain the above copyright notice, this list of conditions and the following disclaimer.
Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.

Language

Disclaimer

This document and its content are intended for information purposes only and may be subject to change or update without notice.

This document may include preliminary concepts that may or may not be in the process of being developed by the Tari community. The release of this document is intended solely for review and discussion by the community regarding the technological merits of the potential system outlined herein.

Goals

Related Requests for Comment

Description

Introduction

The base layer is slow. The DAN is fast. However, every DAN instruction that results in a state change has a fee (i.e. base layer transaction) associated with it.
To bridge the speed gap between the two layers, a payment channel solution is required.
This document provides the high-level overview of how this is done.

The Clacks

The Clacks is a multi-party side-channel off-chain scalability proposal for Tari. The essential idea behind the Clacks is:

Users give control of some Tari to a Clacks Committee.
The committee creates an off-chain UTXO for the user(s). This is called the peg-in transaction.
Users can transact amongst each other without those transactions touching the base layer. This allows a very high throughput of transactions and instant finality. However, the locus of trust move significantly towards the Clacks committee. In other words, base layer transactions are slow, but do not require users to trust anyone. Clacks transactions are fast, but requires some level of trust in the entity or entities controlling the user's funds.
Users can send off-chain Tari to any other users, including users that have not made deposits into the Clacks committee.
Users can request a withdrawal for their Tari at any time. At predefined intervals, the Clacks committee will process those withdrawal requests and give the Tari UTXO control back to the user.

Users also have a pre-signed, time-locked transaction that returns the user's fund back to them. This refund transaction can be used in case the Clacks committee stops processing withdrawals and provides insurance for users against having their funds locked up forever.

The peg-in, peg-out cycle are represented by standard Tari transactions. This means that full nodes verify that no funds have been created or destroyed over the course of the cycle. They do not check that the "balances" associated with users are what those users would expect in an honestly run side chain.

However, if the side-chain is run as a standard mimblewimble process, any third party running a full node and that access to the opening and closing channel balances, can in fact verify that the committee has operated honestly.

RFC-1000/TariUseCases

Digital Asset Framework

status: draft

Maintainer(s): Leland Lee

Licence

The 3-Clause BSD Licence.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

Redistributions of this document must retain the above copyright notice, this list of conditions and the following disclaimer.
Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.

Language

Disclaimer

This document and its content are intended for information purposes only and may be subject to change or update without notice.

Goals

There will be many types of digital assets that can be issued on Tari. The aim of this Request for Comment (RFC) is to help potential asset issuers identify use cases for Tari that lead to the design of new types of digital assets that may or may not exist in their ecosystems today.

Related Requests for Comment

Description

Tari digital assets may exist on the Tari Digital Assets Network (DAN), are perceived to have value and can be owned, with the associated digital data being classified as intangible, personal property.

Types

Digital assets may have the following high-level classification scheme:

Symbolic
- Insignia
- Mascots
- Event-driven or historical, e.g. a rivalry or a highly temporal event
Artefacts or objects
- Legendary
- Rare
- High demand
- Artistic
- Historical
Utility
- Tickets
- In-game items
- Points
- Currency
- Full or fractional representation
- Bearer instruments/access token
- Chat stickers
Persona
- Personality trait(s)
- Emotion(s)
- Statistics
- Superpower(s)
- Storyline(s)
- Relationship(s)
- Avatar

Behaviours

Digital asset tokens may influence the following behavioural types:

Advance purchase
Experience enhancement
Sharing
Loyalty
Reward for performance
Tipping
Donating to a charity
Collecting
Trading
Building/combining

Attributes

Digital assets have many different properties, which may be one or more of the following:

Digital assets can be interactive:
- Easter eggs;
  - media
  - dynamic, e.g. imagine if assets were similar to sounds, visualizations or other kinds of "demos" or "gifs")
- Game mechanics;
- Evolutionary.
Digital assets can be combined to create super assets.
Digital assets can be attribute(s) of another digital asset, e.g. wheels of a vehicle or a VIP ticket has two drink cards.
Digital assets can have contingencies, e.g. ownership of a digital asset is contingent on ownership of a different digital asset. Using this digital asset is contingent on holding it for a particular duration, etc.
Digital assets can have utility, e.g. be useful.
Digital assets can be used across platforms, e.g. a digital asset for a game could be used as avatars in a social network.
Digital assets can have history.
Digital assets can have user-generated tags and/or metadata.

Interactions

Digital asset owners may have the following interactions with the DAN and/or other people:

Digital asset owners can attest that they have ownership over their assets at time t.
Digital asset owners may attest ownership to an individual, to a group of friends or to the entire world.

Rules

Rules are the governance of how digital assets may be used or transferred, as defined by the asset issuer:

Royalty fees. Digital asset issuers can set a royalty that charges a fee every time the digital asset is transferred between parties. The fee as defined by the issuer can be fixed or dynamic, or follow a complex formula, and value is granted to the issuer(s) and/or other entities.
Contingency. Digital asset ownership/interaction may be contingent/dependent on another asset.
Timing controls. Digital assets can only be transferred or used at particular times.
Sharing. Digital assets can be shared with others or even co-owned.
Privacy. Ownership of a digital asset can be changed from private to public.
Upgradability/versioning. Digital assets can be upgraded and/or versioned.
Redeemability. Digital assets can be used once or multiple times.

Examples

Some examples of how different types of digital assets with different attributes, rules and interactions may be manifested are provided here:

Crystal Skull of Akador

Is rare, it is 1 of 5.
Is legendary:
- https://indianajones.fandom.com/wiki/Crystal_Skull_of_Akator;
- press reports that this artefact could be worth $X.
Drives collectability.
Drives advance purchase, e.g. if you are one of the first 100,000 people to buy tickets to Indiana Jones World, you have a chance of winning this 1 of 5 artefact.
Has superpowers and utility, e.g. if you have this item while visiting Indiana Jones World, you get to skip the line three times.
Is a contingency for another asset, e.g. if you collect this item, two Sankara stones and the Cross of Coronado, you can buy the ark of the covenant:
- Ark of the covenant is rare. It is 1 of 1.
- Ark of the covenant is legendary.
- Ark of the covenant gives you lifetime access to Indiana Jones World.
- Ark of the covenant has rules; 20% of the resale price goes to Indiana Jones World.

AB de Villiers' bat

Is not rare, it is 1 of 100,000.
Is legendary - https://www.youtube.com/watch?v=HK6B2da3DPA.
Drives collectability, it is part of a series of bats from famous batsmen.
Can be combined with other assets, e.g. be one of the first 10 people to combine six bats to turn this asset into a One Day International (ODI) century bat:
- ODI century bats are rare, they are 1 of 10;
- ODI century bats are legendary.
Has no superpowers.
Has no utility.
Has no rules.

OVO Owl x Supreme

Is rare, it is 1 of 200.
Is legendary:
- https://www.supremenewyork.com;
- https://us.octobersveryown.com.
Has a game mechanic:
- Every time it’s transferred, it may become a golden ticket that grants you access to any Drake show.
- If it becomes a golden ticket and is transferred, it loses its golden ticket superpower.
Has utility:
- Unlocks exclusive media content feat. Drake hosted by OVO SOUND.
- May become a golden ticket that grants you access to any Drake show.
Has rules - every time its transferred Supreme and OVO SOUND receive 25% of the transaction value.

Tari Network Terminology

Below are a list of terms and their definitions that are used throughout the Tari code and documentation. Let's use this glossary to disambiguate ideas, and work towards a ubiquitous language for this project.

Archive node

This is a full history base node. It will keep a complete history of every transaction ever received and it will not implement pruning.

AssetCollateral

The amount of tari coin that a Validator Node must put up on the base layer in order to become part of an asset committee.

Asset Issuer

An entity that creates digital assets on the Tari DAN. The Asset Issuer will specify the parameters of the contract template that defines the rules that govern the asset and the number and nature of its constituent tokens on issuance. The Asset Issuer will, generally, be the initial owner of the tokens.

Bad Actor

A participant that acts maliciously or negligently to the detriment of the network or another participant.

Base layer

The Tari Base layer is a merge-mined blockchain secured by proof-of-work. The base layer is primarily responsible for the emission of new Tari, for securing and managing Tari coin transfers.

Base Node

A full Tari node running on the base layer. It's primary role is validating and propagating Tari coin transactions and blocks to the rest of the network.

Block

A collection transactions and associated metadata recorded as a single entity in the Tari blockchain. The ordering of Tari transactions is set purely by the block height of the block they are recorded in.

Block Header

A data structure that validates the information contained in a block.

Block reward

The amount of Tari created by the coinbase transaction in every block. The block reward is set by the emission schedule.

Blockchain

A sequence of tari blocks. Each block contains a hash of the previous valid block. Thus the blocks form a chain with the property that changing anything in a block other than the head block requires rewriting the entire blockchain from that point on.

Blockchain state

The complete state of the blockchain at a specific block height. This means a pruned utxo set, a complete set of kernels and headers up to that block height from the genesis block.

BroadcastStrategy

A strategy for propagating messages amongst nodes in a peer-to-peer network. Example implementations of BroadcastStrategy include the Gossip protocol, Dandelion and flood fill.

Checkpoint

A hash of the state of a Digital Asset that is recorded on the base layer.

Coinbase transaction

The first transaction in every Tari block yields a Block Reward according to the Tari emission Schedule and is awarded to the miner that performed the Proof of Work for the block.

Committee

A group of Validator Nodes that are responsible for managing the state of a specific Digital Asset. A committee is selected during asset issuance and can be updated at Checkpoints.

CommitteeSelectionStrategy

A strategy for an Asset Issuer to select candidates for the committee from the available registered Validator Nodes who responded to the nomination call for that asset.

ConsensusStrategy

The approach that will be taken for a committee to reach consensus on the validity of instructions that are performed on a given Digital Asset.

Commitment

A commitment is a cryptographic primitive that allows one to commit to a chosen value while keeping it hidden from others, with the ability to reveal the committed value later. Commitments are designed so that one cannot change the value or statement after they have committed to it.

Communication Node

A Communication Node is either a Validator Node or Base Node that is part of the Tari communication network. It maintains the network and is responsible for forwarding and propagating joining requests, discovery requests and data messages on the communication network.

Communication Client

A Communication Client is a Wallet or Asset Manager that makes use of the Tari communication network to send joining and discovery requests. A Communication Client does not maintain the communication network and is not responsible for forwarding or propagating any requests or data messages.

Creator Nomination Mode

An asset runs in creator nomination mode when every validator node in a validator committee is a Trusted Node that was directly nominated by the Asset Issuer.

Current head

The last block of the base layer that represents the latest valid block. This block must be from the longest proof-of-work chain to be the current head.

Digital asset

Digital assets (DAs) are the sets or collections of native digital tokens (both fungible and non-fungible) that are created by asset issuers on the Tari 2nd layer. For example, a promoter might create a DA for a music concert event. The event is the digital asset, and the tickets for the event are digital asset tokens.

Digital Asset Network

The Tari second layer. All digital asset interactions are managed on the Tari Digital Assets Network (DAN). These interactions (defined in instructions) are processed and validated by Validator Nodes.

DigitalAssetTemplate

A DigitalAssetTemplate is one of a set of contract types supported by the DAN. These contracts are non-turing complete and consist of rigid rule-sets with parameters that can be set by Asset Issuers.

Digital asset tokens

Digital asset tokens (or often, just "tokens") are the finite set of digital entities associated with a given digital asset. Depending on the DA created, tokens can represent tickets, in-game items, collectibles or loyalty points. They are bound to the digital asset that created them.

Hashed Time Locked Contract

A time locked contract that only pays out after a certain criteria has been met or refunds the originator if a certain period has expired.

Emission schedule

An explicit formula as a function of the block height, h, that determines the block reward for the h^th block.

Instructions

Instructions are the digital asset network equivalent of transactions. Instructions are issued by asset issuers and client applications and are relayed by the DAN to the validator nodes that are managing the associated digital asset.

Mempool

The mempool consists of the transaction pool, pending pool, orphan pool and reorg pool, and is responsible for managing unconfirmed transactions that have not yet been included in the longest proof-of-work chain. Miners usually draw verified transactions from the mempool to build up transaction blocks.

Mimblewimble

Mimblewimble is a privacy-centric cryptocurrency protocol. It was dropped in the Bitcoin Developers chatroom by an anonymous author and has since been refined by several authors, including Andrew Poelstra.

Mining Server

A Mining Server is responsible for constructing new blocks by bundling transactions from the mempool of a connected Base Node. It also distributes Proof-of-Work tasks to Mining Workers and verifies PoW solutions.

Mining Worker

A Mining Worker is responsible for performing Proof-of-Work tasks received from its parent Mining Server.

Multisig

Multi-signatures (Multisigs) are also known as N-of-M signatures, this means that a minimum of N number of the M peers need to agree before a transaction can be spent. N and M can be equal; which is a special case and is often referred to as an N-of-N Multisig.

TLU musig

Node ID

A node ID is a unique identifier that specifies the location of a communication node or communication client in the Tari communication network. The node ID can either be obtained from registration on the Base Layer or can be derived from the public identification key of a communication node or communication client.

Orphan Pool

The orphan pool is part of the mempool and manages all transactions that have been verified but attempt to spend UTXOs that do not exist or haven't been created yet.

Pending Pool

The pending pool is part of the mempool and manages all transactions that have a time-lock restriction on when it can be processed or attempts to spend UTXOs with time-locks.

Pruning horizon

This is a local setting for each node to help reduce syncing time and bandwidth. This is the number of blocks from the chain tip beyond which a chain will be pruned.

Public Nomination Mode

An asset runs in public nomination mode when the Asset Issuer broadcasts a call for nominations to the network and VNs from the network nominate themselves as candidates to become members of the committee for the asset. The Asset Issuer will then employ the CommitteeSelectionStrategy to select the committee from the list of available candidates.

Range proof

A mathematical demonstration that a value inside a commitment (i.e. it is hidden) lies within a certain range. For Mimblewimble, range proofs are used to prove that outputs are positive values.

Registration Deposit

An amount of tari coin that is locked up on the base layer when a Validator Node is registered. In order to make Sybil attacks expensive and to provide an authorative base layer registry of validator nodes they will need to lock up a amount of Tari Coin on the Base Layer using a registration transaction to begin acting as a VN on the DAN.

Registration Term

The minimum amount of time that a VN registration lasts, the Registration Deposit can only be released after this minimum period has elapsed.

Reorg Pool

The reorg pool is part of the mempool and stores all transactions that have recently been included in blocks in case a blockchain reorganization occurs and the transactions need to be restored to the transaction pool.

Spending Key

A private spending key is a private key that permits spending of a UTXO. It is also sometimes referred to as a Blinding Factor, since is Tari (and Mimblewimble) outputs, the value of a UTXO is blinded by the spending key:

$$ C = v.H + k.G $$

The public key, $P = k.G$ is known as the public spending key.

SynchronisationState

The current synchronisation state of a Base Node. This can either be

new - The blockchain state is empty and is waiting for synchronisation to begin.
synchronising - The blockchain state is in the process of synchronising with the rest of the network.
synchronised - The blockchain state has synchronised with the rest of the network and is in a position to validate transactions.

SynchronisationStrategy

The generalised approach for a Base Node to obtain the current state of the blockchain from the peer-to-peer network. Specific implementations may differ based on different trade-offs and constraints with respect to bandwidth, local network conditions etc.

Tari Coin

The base layer token. Tari coins are released according to the emission schedule on the Tari base layer blockchain in coinbase transactions.

Transaction

Transactions are activities recorded on the Tari blockchain running on the base layer. Transactions always involve a transfer of Tari coins.

Transaction Pool

The transaction pool is part of the mempool and manages all transactions that have been verified, that spend valid UTXOs and don't have any time-lock restrictions.

Trusted Node

A permissioned Validator Node nominated by an Asset Issuer that will form part of the committee for that Digital Asset.

Token Wallet

A Tari Token Wallet is responsible for managing Digital assets and Tokens, and for constructing and negotiating instructions for transferring and receiving Assets and Tokens on the Digital Asset Network.

Unspent transaction outputs

An unspent transaction output (UTXO) is a discrete number of Tari that are available to be spent. The sum of all UTXOs represents all the Tari currently in circulation. In addition, the sum of all UTXO values equals the sum of the block rewards for all blocks up to the current block height.

UTXO values are hidden by their commitments. Only the owner of the UTXO and (presumably) the creator of the UTXO (either a Coinbase transaction or previous spender) know the value of the UTXO.

Validator Node

Validator nodes (VNs) make up the Tari second layer, or Digital Asset Network. VNs are responsible for creating and updating digital assets living on the Tari network.

ValidationState

Transactions or blocks are unvalidated when first received by a Base Node. After validation, they are either rejected or validated.

Validated transactions can be added to the mempool and propagated to peers.

Validated blocks are added to the blockchain and propagated to peers.

Wallet

A Tari Wallet is responsible for managing key pairs, and for constructing and negotiating transactions for transferring and receiving tari coins on the Base Layer.

Disclaimer

This document is subject to the disclaimer.