RFC-0160/BlockSerialization

Tari Block Binary Serialization

Maintainer(s): Byron Hambly

Licence

Copyright 2021 The Tari Development Community

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

1. Redistributions of this document must retain the above copyright notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.

THIS DOCUMENT IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS", AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

Language

The keywords "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY" and "OPTIONAL" in this document are to be interpreted as described in BCP 14 (covering RFC2119 and RFC8174) when, and only when, they appear in all capitals, as shown here.

Disclaimer

This document and its content are intended for information purposes only and may be subject to change or update without notice.

This document may include preliminary concepts that may or may not be in the process of being developed by the Tari community. The release of this document is intended solely for review and discussion by the community of the technological merits of the potential system outlined herein.

Goals

The aim of this Request for Comment (RFC) is to specify the binary serialization of:

1. a mined Tari block
2. a Tari block mining template

This is to facilitate interoperability of mining software and hardware.

Specification

By reviewing the block and mining template fields below, we have the following underlying data types for serialization:

1. bool
2. u8
3. u16
4. u32
5. Vec<u8>

For 1. to 4. and all numbers, Base 128 Varint encoding MUST be used.

From the Protocol Buffers documentation:

Varints are a method of serializing integers using one or more bytes. Smaller numbers take a smaller number of bytes. Each byte in a varint, except the last byte, has the most significant bit (msb) set – this indicates that there are further bytes to come. The lower 7 bits of each byte are used to store the two's complement representation of the number in groups of 7 bits, least significant group first.

For 5., the dynamically sized Vec array type, the encoded array MUST be preceded by a number indicating the length of the array. This length MUST also be encoded as a varint. By prepending the length of the array, the decoder knows how many elements to decode as part of the sequence.

Block field ordering

Using this varint encoding, all fields of the complete block MUST be encoded in the following order:

1. Version
2. Height
3. Previous block hash
4. Timestamp
5. Output Merkle root
6. Witness Merkle root
7. Output Merkle mountain range size
8. Kernel Merkle root
9. Kernel Merkle mountain range size
10. Input Merkle root
11. Total kernel offset
12. Total script offset
13. Nonce
14. Proof of work algorithm
15. Proof of work supplemental data
16. Transaction inputs - for each input:
• Flags
• Maturity
• Commitment
• Script
• Input data
• Script signature
• Sender Offset
17. Transaction outputs - for each output:
• Flags
• Maturity
• Commitment
• Range proof
• Script
• Sender Offset
• Signature
18. Transaction kernels - for each kernel:
• Features
• Fee
• Lock height
• Excess
• Excess signature public nonce
• Excess signature

Mining template field ordering

The new block template is provided to miners to complete. Its fields MUST also be encoded using varints, in the following order:

1. Version
2. Height
3. Previous block hash
4. Total kernel offset
5. Total script offset
6. Proof of work algorithm
7. Proof of work supplemental data
8. Target difficulty
9. Reward
10. Total fees
11. Transaction inputs - for each input:
• Flags
• Maturity
• Commitment
• Script
• Input data
• Script signature
• Sender Offset
12. Transaction outputs - for each output:
• Flags
• Maturity
• Commitment
• Range proof
• Script
• Sender Offset
• Signature
13. Transaction kernels - for each kernel:
• Features
• Fee
• Lock height
• Excess
• Excess signature public nonce
• Excess signature

Tari Block and Mining Template - Data Types

A Tari block is comprised of the block header and aggregate body.

Here we describe the respective Rust types of these fields in the tari codebase, and their underlying data types:

Block Header

FieldAbstract TypeData TypeDescription
Versionu16u16The Tari protocol version number, used for soft/hard forks
Heightu64u64Height of this block since the genesis block
Previous Block HashBlockHash[u8;32]Hash of the previous block in the chain
TimestampEpochTimeu64Timestamp at which the block was built (number of seconds since Unix epoch)
Output Merkle RootBlockHash[u8;32]Merkle Root of the unspent transaction ouputs
Witness Merkle RootBlockHash[u8;32]MMR root of the witness proofs
Output MMR Sizeu64u64The size (number of leaves) of the output and range proof MMRs at the time of this header
Kernel Merkle RootBlockHash[u8;32]MMR root of the transaction kernels
Kernel MMR Sizeu64u64Number of leaves in the kernel MMR
Input Merkle RootBlockHash[u8;32]Merkle Root of the transaction inputs in this block
Total Kernel OffsetBlindingFactor[u8;32]Sum of kernel offsets for all transaction kernels in this block
Total Script OffsetBlindingFactor[u8;32]Sum of script offsets for all transaction kernels in this block
Nonceu64u64Nonce increment used to mine this block
Proof of Work AlgorithmPowAlgorithmu8Proof of Work Algorithm used to mine this block (Monero or SHA3 )
Proof of Work DataVec<u8>Vec<u8>Supplemental proof of work data. For Sha3 this would be empty, but for a Monero block we need the Monero header and RandomX seed hash.

[u8;32] indicates an array of 32 unsigned 8-bit integers

Block Body

FieldAbstract TypeData TypeDescription
Transaction InputsVec<TransactionInput>TransactionInputList of inputs spent
Transaction OutputsVec<TransactionOutput>TransactionOutputList of outputs produced
Transaction KernelsVec<TransactionKernel>TransactionKernelKernels contain the excesses and their signatures for the transactions

A further breakdown of the body fields is described below:

TransactionInput

FieldAbstract TypeData TypeDescription
FeaturesOutputFeaturesSee OutputFeaturesThe features of the output being spent. We will check maturity for all outputs.
CommitmentPedersenCommitment[u8;32]The commitment referencing the output being spent.
ScriptTariScriptVec<u8>The serialised script, maximum size is 512
Input DataExecutionStackVec<u8>The script input data, maximum size is 512
Script SignatureComSignatureSee ComSignatureA signature with $k_s$, signing the script, input data, and mined height
Sender OffsetPublicKey[u8;32]The offset public key, $K_O$
OutputFeatures
FieldAbstract TypeData TypeDescription
FlagsOutputFlagsu8Feature flags that differentiate the output, for example to specify a coinbase output
Maturityu64u64The block height at which the output can be spent
ComSignature
FieldAbstract TypeData TypeDescription
Public NoncePedersenCommitment[u8;32]public (Pedersen) commitment nonce created with the two random nonces
uSecretKey[u8;32]the first publicly known private key of the signature signing with the value
vSecretKey[u8;32]the second publicly known private key of the signature signing with the blinding factor

Find out more about Commitment signatures:

TransactionOutput

FieldAbstract TypeData TypeDescription
FeaturesOutputFeaturesSee OutputFeaturesOptions for the output's structure or use
CommitmentPedersenCommitment[u8;32]The homomorphic commitment representing the output amount
Range ProofRangeProofVec<u8>A proof that the commitment is in the right range
ScriptTariScriptVec<u8>The script that will be executed when spending this output
Sender OffsetPublicKey[u8;32]Tari script offset pubkey, K_O
SignatureComSignatureSee ComSignatureUTXO signature with the script offset private key, k_O

TransactionKernel

FieldAbstract TypeData TypeDescription
FeaturesKernelFeaturesu8Options for a kernel's structure or use
FeeMicroTariu64Fee originally included in the transaction this proof is for.
Lock Heightu64u64This kernel is not valid earlier than this height. The max maturity of all inputs to this transaction
ExcessPedersenCommitment[u8;32]Remainder of the sum of all transaction commitments (minus an offset). If the transaction is well-formed, amounts plus fee will sum to zero, and the excess is a valid public key.
Excess SignatureRistrettoSchnorrSee RistrettoSchnorrAn aggregated signature of the metadata in this kernel, signed by the individual excess values and the offset excess of the sender.

RistrettoSchnorr

FieldAbstract TypeData TypeDescription
Public noncePublicKey[u8;32]The public nonce of the Schnorr signature
SignatureSecretKey[u8;32]The signature of the Schnorr signature

Mining Template Header

FieldAbstract TypeData TypeDescription
Versionu16u16The Tari protocol version number, used for soft/hard forks
Heightu64u64Height of this block since the genesis block
Previous HashBlockHash[u8;32]Hash of the previous block in the chain
Total Kernel OffsetBlindingFactor[u8;32]Sum of kernel offsets for all transaction kernels in this block
Total Script OffsetBlindingFactor[u8;32]Sum of script offsets for all transaction kernels in this block
Proof of Work AlgorithmPowAlgorithmu8Proof of Work Algorithm used to mine this block (Monero or SHA3 )
Proof of Work DataVec<u8>Vec<u8>Supplemental proof of work data. For Sha3 this would be empty, but for a Monero block we need the Monero header and RandomX seed hash.
Target DifficultyDifficultyu64The minimum difficulty required to satisfy the Proof of Work for the block
RewardMicroTariu64The value of the emission for the coinbase output for the block
Total FeesMicroTariu64The sum of all transaction fees in this block

Mining Template Body

FieldAbstract TypeData TypeDescription
Transaction InputsVec<TransactionInput>TransactionInputList of inputs spent
Transaction OutputsVec<TransactionOutput>TransactionOutputList of outputs produced
Transaction KernelsVec<TransactionKernel>TransactionKernelKernels contain the excesses and their signatures for the transactions