Initialize your Rust Project

In this step, we will initialize a basic rust project, where we can start building our simple Rust state machine.

cargo init

1. Create a directory where you want your project to live, and navigate to that folder. We will be using a folder named rust-state-machine.

   mkdir rust-state-machine
   cd rust-state-machine
   

2. In that folder, initialize your rust project using cargo init:

   cargo init
   

   This will scaffold a basic Rust executable which we can use to start building.

3. You can verify that your new project is working as expected by running:

   cargo run
   

   You should see “Hello, World!” appear at the end of the compilation:

   ➜  rust-state-machine git:(master) ✗ cargo run
   Compiling rust-state-machine v0.1.0 (/Users/shawntabrizi/Documents/GitHub/rust-state-machine)
   	Finished dev [unoptimized + debuginfo] target(s) in 2.19s
   	Running `target/debug/rust-state-machine`
   Hello, world!
   

If we look at what has been generated, in that folder, you will see the following:

• src/main.rs - This is the entry point to your program. We will be building everything for this project in the src folder.
• Cargo.toml - This is a configuration file for your Rust project. Quite similar to a package.json that you would see in a Node.js project. We will modify this in the future when we import crates to use in our project, but We can leave this alone for now.
• Cargo.lock - This is an autogenerated lock file based on your cargo.toml and the compilation. This usually defines the very specific versions of each crate being imported, and should not be manually edited.
• target/* - You might also see a target folder if you did cargo run. This is a folder where all the build artifacts are placed during compilation. We do not commit this folder into our git history.

All of this should be pretty familiar to you if you have already had some minimal experience with Rust. If any of this is new, I would suggest you first walk through the Rust Book and Rust by Example, as this is already an indication that this guide might not be targeted at your level of knowledge.

Rust Tooling

In this step, we will initialize a basic rust project, where we can start building our simple Rust state machine.

rustfmt

To keep your code clean and easy to read, we use a tool called rustfmt. To access all the latest features of rustfmt we specifically use the nightly toolchain.

To install rustfmt for nightly:

rustup component add rustfmt --toolchain nightly

To configure the behavior of rustfmt, we will create a rustfmt.toml file:

1. Create a new file in your project’s root directory called rustfmt.toml.

   touch rustfmt.toml
   

2. Use the provided rustfmt.toml file to configure your formatting preferences.

3. Run the code formatter using the following command:

   cargo +nightly fmt
   

You shouldn’t see any changes this time around, but as you write more code, you will be able to see cargo +nightly fmt make everything look pretty, consistent, and easy to read.

We recommend you run cargo +nightly fmt after every step!

Rust Analyzer

Another popular tool in the Rust community is Rust Analyzer.

It provides many features like code completion and goto definition for code editors like VS Code.

However, to provide the full functionality that it does, Rust Analyzer needs to compile your code. For a small project like this one, this is not a problem, however working with a large project like Substrate / Polkadot-SDK, it is.

It is my personal recommendation that Rust Analyzer is not needed in this workshop, and generally you should not use it for Substrate development. However, this section might be updated in the future to include special configurations of Rust Analyzer which will work well with Polkadot SDK in the future.

However, if you would like to use it anyway, now is the right time to set it up.

Creating a Balances Pallet

As mentioned earlier, at the heart of a blockchain is a state machine.

We can create a very naive state machine using simple Rust abstractions, and through this help learn about Rust in the context of blockchains.

We want keep our code organized, so we will not really start building in the main.rs file, but actually in separate Rust modules. We can think of the main.rs file as glue which brings everything together, and we will see that over the course of this workshop.

“Pallet” is a term specific to the Polkadot SDK, which refers to Rust modules which contain logic specific for your blockchain runtime. We are going to start using this term here because what we build here will closely mirror what you will see with the Polkadot SDK.

Balances

Pretty much every blockchain has logic that handles the balances of users on that blockchain.

This Pallet will tell you: how much balance each user has, provide functions which allow users to transfer those balances, and even some low level functions to allow your blockchain system to manipulate those balances if needed. Think for example if you want to mint new tokens which don’t already exist.

This is a great starting point, and the very first Pallet we will build.

Creating a Struct

1. Create a new file in your src folder named balances.rs

   touch src/balances.rs
   

2. In this file, create a struct, which will act as the state and entry point for this module:

   #![allow(unused)]
   fn main() {
   pub struct Pallet {}
   }

3. Now go back to src/main.rs, and import this new module, which will include all the logic inside of it:

   #![allow(unused)]
   fn main() {
   mod balances;
   }

4. If we run your program now, you will see it still compiles and runs, but might show you some warnings like:

   warning: struct `Pallet` is never constructed
   --> src/balances.rs:1:12
   |
   1 | pub struct Pallet {    }
   |              ^^^^^^
   |
   = note: `#[warn(dead_code)]` on by default
   
   warning: `pr` (bin "pr") generated 1 warning
   

   That’s fine! We haven’t started using our Pallet yet, but you can see that the Rust compiler is detecting our new code, and bringing that logic into our main program. This is the start of building our first state machine module.

Adding State to Our Pallet

So let’s add some simple state to our balances.rs module.

We can do this by adding fields into our Pallet struct.

For a balance system, we really only need to keep track of one thing: how much balance each user has in our system.

For this we will use a BTreeMap, which we can import from the Rust std library.

Maps are simple key -> value objects, allowing us to define an arbitrary sized storage where we can map some user identifier (key) to their account balance (value).

1. Import the BTreeMap object.

   #![allow(unused)]
   fn main() {
   use std::collections::BTreeMap;
   }

2. Create a balances field in Pallet using the BTreeMap.

   For the key, we are using a string for now. This way we can access users like "alice", "bob", etc… This will be changed in the future.

   For the value, we will use a u128, which is the largest natively supported type in Rust. This will allow our users to have very, very large balances if we want.

   In the end, that looks like:

   #![allow(unused)]
   fn main() {
   pub struct Pallet {
   	balances: BTreeMap<String, u128>,
   }
   }

3. Finally, we need a way to initialize this object and its state. For this, we will implement a function on the Pallet called fn new():

   #![allow(unused)]
   fn main() {
   impl Pallet {
   	pub fn new() -> Self {
   		Self { balances: BTreeMap::new() }
   	}
   }
   }

You can confirm at this point that everything should still be compiling, and that you haven’t made any small errors. Warnings are okay.

Next we will actually start to use this module.

Notes

It is important to note that this is NOT how Pallet storage works with the Polkadot SDK, but just a simple emulation of the behaviors.

In the Polkadot SDK, there is a separate storage layer which manages a proper key-value database which holds all the information (past and present) of our blockchain system. There are abstractions which look and behave just like a BTreeMap in the Polkadot SDK, but the underlying logic which maintains that data is much more complex.

Using simple fields in a struct keeps this project simple, and illustrates that each Pallet really is meant to manage it’s own storage. However, this simplification also leads to issues if you design more complex systems where multiple pallets interact with one another.

We won’t have any cross pallet interactions in this workshop, however, this is definitely doable with the Polkadot SDK and a proper database.

Interacting with Balances

Now that we have established the basics of our balances module, let’s add ways to interact with it.

To do this, we will continue to create more functions implemented on Pallet which grants access to read, write, and update the balances: BTreeMap we created.

Finally, we will see what it looks like to actually start interacting with our balances pallet from the main.rs file.

Rust Prerequisite Knowledge

Before we continue, let’s take a moment to go over some Rust which we will be using in this next section.

Option and Option Handling

One of the key principles of Rust is to remove undefined behavior from your code.

One way undefined behavior can happen is by allowing states like null to exist. Rust prevents this by having the user explicitly handle all cases, and this is where the creation of the Option type comes in. Spend a moment to re-review the section on Option from the Rust book if needed.

The BTreeMap api uses an Option when reading values from the map, since it could be that you ask to read the value of some key that you did not set. For example:

#![allow(unused)]
fn main() {
use std::collections::BTreeMap;

let mut map = BTreeMap::new();
map.insert("alice", 100);
assert_eq!(map.get(&"alice"), Some(&100));
assert_eq!(map.get(&"bob"), None);
}

Once we have an Option type, there are lots of different ways we can interact with it using Rust.

The most verbose way is using a match statement:

#![allow(unused)]
fn main() {
let maybe_value = map.get(&"alice");
match maybe_value {
	Some(value) => {
		// do something with the `value`
	},
	None => {
		// perhaps return an error since there was no value there
	}
}
}

IMPORTANT NOTE!

What you SHOULD NOT do is blindly unwrap() options. This will result in a panic in your code, which is exactly the kind of thing Rust was designed to prevent! Instead, you should always explicitly handle all of your different logical cases, and if you let Rust do it’s job, your code will be super safe.

In the context of what we are designing for with the balances module, we have a map which has an arbitrary number of user keys, and their balance values.

What should we do when we read the balance of a user which does not exist in our map?

Well, the trick here is that in the context of blockchains, a user having None balance, and a user having 0 balance is the same. Of course, there is some finer details to be expressed between a user who exists in our state with value 0 and a user which does not exist at all, but for the purposes of our APIs, we can treat them the same.

What does this look like?

Well, we can use unwrap_or(...) to safely handle this condition, and make our future APIs more ergonomic to use. For example:

#![allow(unused)]
fn main() {
use std::collections::BTreeMap;

let mut map = BTreeMap::new();
map.insert("alice", 100);
assert_eq!(*map.get(&"alice").unwrap_or(&0), 100);
assert_eq!(*map.get(&"bob").unwrap_or(&0), 0);
}

As you can see, by using unwrap_or(&0) after reading from our map, we are able to turn our Option into a basic integer, where users with some value have their value exposed, and users with None get turned into 0.

Let’s see how that can be used next.

Setting and Reading User Balances

As you can see, our initial state machine starts that everyone has no balance.

To make our module useful, we need to at least have some functions which will allow us to mint new balances for users, and to read those balances.

1. Create a new function inside impl Pallet called fn set_balance:

   #![allow(unused)]
   fn main() {
   impl Pallet {
   	pub fn set_balance(&mut self, who: &String, amount: u128) {
   		self.balances.insert(who.clone(), amount);
   	}
   
   	// -- snip --
   }
   }

   As you can see, this function simply takes input about which user we want to set the balance of, and what balance we want to set. This then pushes that information into our BTreeMap, and that is all.

2. Create a new function inside impl Pallet called fn balance:

   #![allow(unused)]
   fn main() {
   pub fn balance(&self, who: &String) -> u128 {
   	*self.balances.get(who).unwrap_or(&0)
   }
   }

   As you can see, this function allows us to read the balance of users in our map. The function allows you to input some user, and we will return their balance.

   Important Detail!

   Note that we do our little trick here! Rather than exposing an API which forces the user downstream to handle an Option, we instead are able to have our API always return a u128 by converting any user with None value into 0.

As always, confirm everything is still compiling. Warnings are okay.

Next we will write our first test and actually interact with our balances module.

Basic Balance Test

Now that we have the basics of our Pallet set up, let’s actually interact with it.

For that, we will go back to the main.rs file, and create our first #[test] which will play with the code we have written so far.

1. In your src/balances.rs file, add a new #[test] named fn init_balances():

   #![allow(unused)]
   fn main() {
   #[test]
   fn init_balances() { }
   }

2. To begin our test, we need to initialize a new instance of our Pallet:

   #![allow(unused)]
   fn main() {
   #[test]
   fn init_balances() {
   	let mut balances = super::Pallet::new();
   }
   }

   Note that we make this variable mut since we plan to mutate our state using our newly created API.

3. Finally, let’s check that our read and write APIs are working as expected:

   #![allow(unused)]
   fn main() {
   #[test]
   fn init_balances() {
   	let mut balances = super::Pallet::new();
   
   	assert_eq!(balances.balance(&"alice".to_string()), 0);
   	balances.set_balance(&"alice".to_string(), 100);
   	assert_eq!(balances.balance(&"alice".to_string()), 100);
   	assert_eq!(balances.balance(&"bob".to_string()), 0);
   }
   }

4. We can run our tests using cargo test, where hopefully you should see that it passes. There should be no compiler warnings now!

I hope at this point you can start to see the beginnings of your simple blockchain state machine.

Enable Balance Transfers

Now that we have initialized and started to use our balances module, let’s add probably the most important API: transfer.

Learn

Before we write our function, it is important that we review some of the principles of blockchain and Rust.

Bad Actors

In a blockchain system, security is paramount. Bad actors may attempt to exploit vulnerabilities, such as insufficient balances during fund transfers, or overflow / underflow issues. Rust’s safe math and error handling mechanisms help mitigate these risks.

Safe Math

Rust’s safe math operations prevent overflow and underflow. The checked_add and checked_sub methods return an Option that allows handling potential arithmetic errors safely.

In Rust, the Option type is a fundamental part of the standard library, designed to handle scenarios where a value may or may not be present. It’s commonly used in situations where the result of an operation might be undefined or absent.

Methods like checked_add and checked_sub return Option to indicate success or failure due to overflow or underflow.

#![allow(unused)]
fn main() {
let result = a.checked_add(b);
match result {
    Some(sum) => println!("Sum: {sum}"),
    None => println!("Overflow occurred."),
}
}

Error Handling

In Rust, error handling is an integral part of writing robust and safe code. The Result type is commonly used for functions that may encounter errors during their execution.

The Result type is an enum defined in the standard library. It has two variants: Ok(value) for a successful result and Err(error) for an error:

#![allow(unused)]
fn main() {
enum Result<T, E> {
    Ok(T),
    Err(E),
}
}

T and E are generic parameters that allow you to customize the result type for your needs. For the purposes of this tutorial, we will always return Ok(()) when everything completes okay, and a Err(&'static str) to describe any errors with a basic string.

You can then define the Result type like:

#![allow(unused)]
fn main() {
Result<(), &'static str>
}

Options and Results

You can use the Option type to trigger an Err, which is helpful when you only want your function to execute when everything goes as expected.

In this context, we want a function that will return an error whenever some safe math operation returns None.

For this, we can chain ok_or along with ? directly after the safe math operation like so:

#![allow(unused)]
fn main() {
let new_from_balance = from_balance
    .checked_sub(amount)
    .ok_or("Not enough funds.")?;
}

If checked_sub returns None, we will then return an Err with the message "Not enough funds." that can be displayed to the user. Otherwise, if checked_sub returns Some(value), we will assign new_from_balance directly to that value.

In this case, we are writing code which completely handles the Option type in a safe and ergonomic way.

Create Transfer

Follow the instructions in the template to create a safe and simple transfer function in your Balances Pallet.

Create a test showing that everything is working as expected, including error handling. There should be no compiler warnings!

Introduce the System Pallet

We have basically completed the creation of a basic Balances Pallet. This is the pallet that most users will interact with.

However, your blockchain usually needs to keep track of many other pieces of data to function properly.

For this, we will create a new pallet called the System Pallet.

What is the System Pallet?

The System Pallet is a “meta”-pallet which stores all the metadata needed for your blockchain to function. For example, the current blocknumber or the nonce of users on your blockchain.

This pallet does not need to expose any functions to end users, but can still play an important role in our overall state transition function.

We will see the importance of the System Pallet evolve as you walk through the steps of building it.

Create the System Pallet

1. Create a new file src/system.rs in your project.
2. Copy the starting template provided, then complete the steps outlined by the template code.
3. Import the system module into your main.rs file.

You will notice that the instructions here are quite brief. You have already done all of these steps before, so you should already be familiar with everything you need to complete this step.

Confirm everything is compiling. You should expect some “never used/constructed” warnings. That is okay.

Making Your System Functional

We have again established the basis of a new Pallet.

Let’s add functions which make it useful.

Block Number

Your blockchain’s blocknumber is stored in the System Pallet, and the System Pallet needs to expose functions which allow us to access and modify the block number.

For this we need two simple functions:

• fn block_number - a function that returns the currently stored blocknumber.
• fn inc_block_number - a function that increments the current block number by one.

This should be everything that a basic blockchain needs to function.

Nonce

The nonce represents “a number used once”.

In this context, each user on your blockchain has a nonce which gives a unique value to each transaction the user submits to the blockchain.

Remember that blockchains are decentralized and distributed systems, and transactions do not inherently have a deterministic order. For a user, we can assign an order to different transactions by using this nonce to keep track of how many transactions the user has executed on the blockchain.

For this, we again use a BTreeMap to give each user their own nonce counter.

Our simple blockchain won’t use this value, but for the sake of example, we will keep track of it by creating an inc_nonce function. If you were creating a more complex blockchain, the user nonce would become an important part of your system.

Safe Math?

We just explained the importance of using safe math when writing the Balances Pallet.

In that context, it is easy to see how a user could provide malicious inputs, and cause simple underflows or overflows if our system did not check the math.

However, you will see in the templates provided, that these new functions in the System Pallet do not return a result, and thus do not provide error handling.

Is this okay?

As you will notice, the blocknumber and nonce storage items only provide APIs to increment by one. In our System, both of these numbers are represented by u32, which means that over 4.2 billion calls to those functions need to occur before an overflow would happen.

Assuming a user does one transaction every block, and a new block is generated every 6 seconds, it would take over 800 years for an overflow to occur. So in this situation, we are preferring an API which requires no error handling rather than one which does.

End of the day, this is a design decision and a preference which is left to the developer. This tutorial chooses this API because this is exactly the API exposed by Substrate and the Polkadot SDK. There is nothing wrong with making these functions handle errors, so feel free to do this if you choose.

Build Your System Pallet

Follow the instructions in the template to complete:

1. fn block_number
2. fn inc_block_number
3. fn inc_nonce

Then write tests which verify that these functions work as expected, and that your state is correctly updated. There should be no compiler warnings after this step!

Creating Our Runtime

We have now established two different Pallets for our blockchain: the System and Balances Pallet.

How do these pallets work together to create a unified blockchain system?

For that, we will need to create a Runtime.

What is the Runtime?

Remember that there is a separation between the blockchain client and the state transition function of our blockchain.

You can think of the runtime as the accumulation of all logic which composes your state transition function. It will combine all of your pallets into a single object, and then expose that single object as the entry point for your users to interact with.

Certainly this sounds pretty abstract, but it will make more sense as we complete this tutorial.

Create the Runtime

Just like our Pallets, our Runtime will be represented with a simple struct, however in this case, the fields of our struct will be our Pallets!

Complete the instructions for creating a new runtime which includes our System and Balances pallets. For this, you will need to take advantage of the new() functions we exposed for each of the Pallets.

Make sure your code is formatted and everything is still compiling. Compiler warnings about “never read/used” are okay.

Using Our Runtime

Until now, we have just been scaffolding parts of our blockchain. Tests have ensured that the code we have written so far make sense, but we haven’t actually USED any of the logic we have written for our main program.

Let’s change that by using our Runtime and actually executing logic on our blockchain.

Simulating a Block

The input to any blockchain state transition function is a block of transactions.

Later in the tutorial we will actually spend more time to build proper blocks and execute them, but for now, we can “simulate” all the basics of what a block would do by individually calling the functions our Pallets expose.

Genesis State

The state of your blockchain will propagate from block to block. This means if Alice received 100 tokens on block 4, that she can transfer at least 100 tokens on block 5, and so on.

But how do users get any balance to begin with?

The answer to this question can be different for different blockchains, but in general most modern blockchains start with a Genesis State. This is the starting state of your blockchain on “block 0”.

This means anything set in the genesis state can be used on block 1, and can bootstrap your blockchain to being functional.

In our situation, you can simply call low level functions like set_balance before we simulate our first block to establish our genesis state.

Steps of a Basic Block

Let’s quickly break down the steps of executing a basic block:

1. First we increment the blocknumber, since each new block will have a new blocknumber.
2. Then we go through and execute each transaction in that block:
   1. Each transaction for our blockchain will come from a user, thus we will increment the users nonce as we process their transaction.
   2. Then we will attempt to execute the function they want to call, for example transfer.
   3. Repeat this process for every transaction.

Handling Errors

The main() function in Rust cannot propagate or handle errors itself. Either everything inside of it is handled, or you will have to trigger a panic.

As you have already learned, triggering a panic is generally not good, but may be the only thing you can do if something is seriously wrong. For our blockchain, the only thing which can really cause a panic is importing a block which does not match the expected blocknumber. There is nothing in this case we can do to “handle” this error. If someone is telling us to execute the wrong block, then we have some larger problem with our overall system that needs to be fixed.

However, users can also submit transactions which result in an error. For example, Alice trying to send more funds than she has in her account.

Should we panic?

Absolutely not! This is the kind of error that our runtime should be able to handle since it is expected that such errors would occur. A block can be valid even if transactions in the block are invalid!

When a transaction returns an error we should show that error to the user, and then “swallow” the result. For example:

#![allow(unused)]
fn main() {
let _res = i_can_return_error().map_err(|e| eprintln!("{e}"));
}

In this case, you can see that any error that i_can_return_error would return gets printed to the console, but otherwise, the Result of that function gets placed in an unused variable _res.

You should be VERY CAREFUL when you do this. Swallowing an error is exactly the opposite of proper error handling that Rust provides to developers. However, we really do not have a choice here in our main function, and we fully understand what we are doing here.

On real blockchain systems, users are still charged a transaction fee, even when their transaction results in an Err. This ensures that users are still paying a cost for triggering logic on the blockchain, even when the function fails. This is an important part of keeping our blockchain resilient to DDOS and sybil attacks.

Simulate Your First Block

Do you think you understand everything it takes to simulate your first block?

Follow the instructions provided by the template to turn your main function from “Hello, World!” to actually executing your blockchain’s runtime.

At the end of this step, everything should compile and run without warnings!

Derive Debug

In Rust, derive macros provide a convenient way to automatically implement trait functionality for custom data structures.

Macros

In the most simple terms, Macros are rust code that write more rust code.

Macros can make your code easier to read, help avoid repetition, and even let you create your own special rules for coding in Rust.

We will be using (but not writing) macros heavily near the end of this tutorial, and you will see how powerful they can be.

For now, treat them as “magic”.

Traits

Think of traits in Rust as shared rules for different types. They allow you to define a set of things that types must be able to do. This way, you can make sure different parts of your code follow the same rules.

Take a look at this example or re-read the Rust Book if you need a refresher on Traits.

We will make and use custom traits later in this tutorial, but know for this step that #[derive(Debug)] is a macro which implements the Debug trait for your custom types.

Debug Trait

The Debug trait in Rust is part of the standard library and is used to print and format values for debugging purposes. It provides a default implementation through the `#[derive(Debug)] annotation.

For example:

#![allow(unused)]
fn main() {
#[derive(Debug)]
pub struct MyStruct {
    field1: i32,
    field2: String,
}
}

With the Debug trait derived, you can now print the struct to console:

#![allow(unused)]
fn main() {
let my_instance = MyStruct { field1: 42, field2: "Hello".to_string() };
println!("{my_instance:#?}");
}

The characters :#? help format the output to make it more readable.

Derive the Debug Trait for Your Runtime

This is a very simple, but helpful step!

We want to be able to print out the current state of our Runtime at the end of our main to allow us to easily inspect what it looks like and that everything is functioning as we expect.

To do this, we need to add #[derive(Debug)] to the struct Runtime.

However… struct Runtime is composed of system::Pallet and balances::Pallet, so these structs ALSO need to implement the Debug trait.

Complete the TODOs across the different files in your project and print out your final runtime at the end of the main function.

You can use cargo run to see the output of your println. Everything should compile and run without warnings.

Using Named Types

Up till now, we have just been hardcoding raw types into our structs and function definitions.

There are already examples where this can be confusing, for example if you see a function accept a u32 parameter, is it a blocknumber or a nonce?

To make our code more clear, let’s extract all of our raw types and define custom named types for our structs and functions.

Across the Balances and System Pallet, we need to define the following types:

1. type AccountId = String;
2. type Balance = u128;
3. type Nonce = u32;
4. type BlockNumber = u32;

Note that extracting these types into common type definitions also allows us to update the types more easily if we choose to.

As we go further into this tutorial, we will show you how we can make these type definitions even more flexible and customizable in the context of building a blockchain SDK for developers like yourself.

Create Custom Types

Follow the TODOs in the template to add these type definitions to each of your Pallets, and update all of your structs and functions to use these types.

Import the Num Crate

Rust is designed to be very lightweight and provides very little right out of the box.

Within the ecosystem, many functions and features which you might expect to be included into Rust std or core are actually delegated to small, well-known, and widely used crates.

For our next step, we want to access traits for basic numerical operations like:

• CheckedAdd - A type which supports checked_add
• CheckedSub - A type which supports checked_sub
• Zero - A type which can return the value zero when calling zero()
• One - A type which can return the value one when calling one()

To access these traits, we will need to import a new crate into our project.

Cargo.toml

When we first initialized our project, a Cargo.toml file was generated for us.

As mentioned before, it is very similar to a package.json file you would expect to find in a Node.js project.

Already in your Cargo.toml is metadata like the name of your project, the version of the crate you are building, and the edition of Rust you are using.

What you can see is that you can also add dependencies to your crate which will allow you to use other external crates and libraries in your project.

You can add the dependency by hand by editing your Cargo.toml file or you can run cargo add num.

Crates.io

Where is this crate coming from?

The Rust community has a large registry of available crates on crates.io. When you import a crate, it will use crates.io by default.

You can also import crates directly from github by specifying the repo where the source code can be found.

That would look something like:

[dependencies]
pallet-balances = { version = "4.0.0-dev", default-features = false, git = "https://github.com/paritytech/substrate.git", branch = "polkadot-v1.0.0" }

Add the Num Crate to Your Project

This step is short and simple.

Run cargo add num in your project directory and make sure your project compiles afterward.

You should see something like:

➜  rust-state-machine git:(master) ✗ cargo add num
    Updating crates.io index
      Adding num v0.4.1 to dependencies.
             Features:
             + std
             - alloc
             - libm
             - num-bigint
             - rand
             - serde
    Updating crates.io index
➜  rust-state-machine git:(master) ✗ cargo run
    Finished dev [unoptimized + debuginfo] target(s) in 0.01s
     Running `target/debug/rust-state-machine`

Make Balances Pallet Generic

Our goal over the next few steps will be to continually make our runtime more generic and configurable over the types we use in our Pallets.

Why Generic?

The flexibility of generic runtime means that we can write code which works for multiple different configurations and types.

For example, up until now, we have been using String to represent the accounts of users. This is obviously not the right thing to do, but is easy to implement for a basic blockchain tutorial like this.

What would you need to change in order to use more traditional cryptographic public keys?

Well, currently there are definitions of the account type in both the Balances Pallet and the System Pallet. Imagine if you had many more Pallets too! Such refactoring could be very difficult, but also totally avoided if we used generic types to begin with.

Truthfully, the advantage of generic types will not be super obvious in this tutorial, but when building a blockchain SDK like the Substrate, this kind of flexibility will allow ecosystem developers to reach their full potential.

For example, teams have used Substrate to build fully compatible Ethereum blockchains, while other teams have experimented with cutting edge cryptographic primitives. This generic framework allows both teams to be successful.

Generic Types

You have already been lightly exposed to generic types with the Result type. Remember that this type is flexible to allow for you to configure what type is returned when there is Ok or Err.

If we wanted to make our Pallet generic, it would look something like:

#![allow(unused)]
fn main() {
pub struct Pallet<AccountId, Balance> {
 	balances: BTreeMap<AccountId, Balance>,
}
}

And implementing functions on Pallet would look like:

#![allow(unused)]
fn main() {
impl<AccountId, Balance> Pallet<AccountId, Balance> {
	// functions which use these types
}
}

In this case, we have not defined what the AccountId and Balance type are concretely, just that we will be storing a BTreeMap where the AccountId type is a key and Balance type is a value.

Trait Constraints

The Result generic type is extremely flexible because there are no constraints on what the Ok or Err type has to be. Every type will work for this situation.

However, our Pallets are not that flexible. The Balance type cannot literally be any type. Because we have functions like fn transfer, we must require that the Balance type at least has access to the function checked_sub, checked_add, and has some representation of zero.

This is where the num crate will come in hand. From the num crate, you can import traits which define types which expose these functions:

#![allow(unused)]
fn main() {
use num::traits::{CheckedAdd, CheckedSub, Zero};
}

Then, where applicable, you need to constrain your generic types to have these traits.

That will look like:

#![allow(unused)]
fn main() {
impl<AccountId, Balance> Pallet<AccountId, Balance>
where
	AccountId: Ord,
	Balance: Zero + CheckedSub + CheckedAdd + Copy,
{
	// functions which use these types and have access to the traits specified
}
}

You will notice other types like Copy and Ord that have been added. These constrains come from using structures like the BTreeMap, which requires that the key type is “orderable”.

You can actually try compiling your code without these type constraints, and the compiler will tell you which traits you are missing, and what you need to include.

Instantiating a Generic Type

The final piece of the puzzle is instantiating our generic types.

Previously we could simply write:

#![allow(unused)]
fn main() {
let mut balances = super::Pallet::new();
}

But now that Pallet is generic, we need to concretely define those types when we instantiate it.

That syntax looks like:

#![allow(unused)]
fn main() {
let mut balances = super::Pallet::<String, u128>::new();
}

You will notice that now the types are defined wherever the generic struct Pallet is being instantiated. This means that you can extract the types out of your Pallets, and move them into the Runtime.

Get Generic!

Its time to turn your balances pallet generic.

1. Follow the TODOs in the balances.rs file to make Pallet generic.
2. Move the type definitions for AccountId and Balance to your main.rs.
3. Update your struct Runtime to use these types when defining the balances::Pallet.

To be honest, this is one of the places that developers most frequently have problems when learning Rust, which is why there is such an emphasis on teaching you and having you learn by doing these steps yourself.

Don’t be afraid in this step to peek at the solution if you get stuck, but do try and learn the patterns of using generic types, and what all the syntax means in terms of what the compiler is trying to guarantee about type safety.

Make System Pallet Generic

Now that you have some practice with the Balances Pallet, let’s do the same task for the System Pallet.

1. In this case we need to make System generic over AccountId, BlockNumber, and Nonce.

2. You will also need to figure out the trait constraints needed for these types to be compatible with the logic you have previously written. The compiler is your friend here to help you navigate everything.

3. Update your tests.

4. Finally move your type definitions to your main.rs file and update your Runtime.

Make sure everything compiles and all tests pass after this step.

If you need help, I recommend to look at your Balances Pallet rather than the solution for this step. All of the patterns are the same as before, so it is better that you start to connect the dots yourself rather than relying on the solution.

If you struggled here, it is a good opportunity to take a pause and re-review generic types from other examples across the Rust ecosystem.

Make System Configurable

We have one more step to take to make our Runtime as generic and configurable as possible.

To do it, we will need to take advantage of traits.

Custom Traits

We have already used traits provided to us in order to make our types generic.

Let’s take a quick look at how you can define a custom trait:

#![allow(unused)]
fn main() {
pub trait Config {}
}

Traits can contain within it two things:

1. Functions which must be implemented by the type.
2. Associated types.

Custom Functions

The more obvious use of traits is to define custom functions.

Let’s say we want to expose a function which returns the name of something.

You could create a trait GetName:

#![allow(unused)]
fn main() {
pub trait GetName {
	fn name() -> String;
}
}

Then you could implement this trait for any object.

#![allow(unused)]
fn main() {
struct Shawn;
impl GetName for Shawn {
	fn name() -> String {
		"shawn".to_string()
	}
}
}

And then call that function on the object which implements it.

fn main() {
	let name = Shawn::name();
	println!("{name}");
}

We won’t actually use this feature of traits in our simple blockchain, but there are plenty of use cases for this when developing more complex blockchain systems.

Associated Types

The other thing you can do with traits is define Associated Types.

This is covered in chapter 19 of the Rust Book under “Advanced Traits”.

Let’s learn this concept by first looking at the problem we are trying to solve.

So far our simple blockchain code looks perfectly fine with generic types. However, let’s imagine that our blockchain becomes more and more complex, requiring more and more generic types.

For example:

#![allow(unused)]
fn main() {
pub struct Pallet<AccountId, BlockNumber, BlockLength, BlockWeight, Hash, Nonce, Runtime, Version, ...> {
	// a bunch of stuff
}
}

Imagine every time you wanted to instantiate this struct, you would need to fill out each and every one of those types. Well systems do get this complex, and more, and the ability to abstract these types one level further can really simplify your code and make it much more readable.

For this we will use a trait with a bunch of associated types:

#![allow(unused)]
fn main() {
pub trait Config {
	type AccountId: Ord + Clone;
	type BlockNumber: Zero + One + AddAssign + Copy;
	type Nonce: Zero + One + Copy;
	// and more if needed
}
}

Then we can define our generic type using a single generic parameter!

#![allow(unused)]
fn main() {
pub struct Pallet<T: Config> {
	block_number: T::BlockNumber,
	nonce: BTreeMap<T::AccountId, T::Nonce>,
}
}

and implement functions using:

#![allow(unused)]
fn main() {
impl<T: Config> Pallet<T> {
	// functions using types from T here
}
}

Let’s try to understand this syntax real quick.

1. There is a generic type T. T has no meaningful name because it represents a bunch of stuff, and this is the convention most commonly used in Rust.
2. T is required to implement the trait Config, which we previously defined.
3. Because T implements Config, and Config has the associated types AccountId, BlockNumber, and Nonce, we can access those types like so:
   • T::AccountId
   • T::BlockNumber
   • T::Nonce

While this may seem like a purely stylistic change, it enforces a powerful constraint: for any given type implementing Config (like our Runtime), there can only be one corresponding AccountId, BlockNumber, and Nonce. This guarantees type consistency across the pallet.

In this context, we call the trait Config because it is used to configure all the types for our Pallet.

Implementing the Config Trait

Let’s round this out by showing how you can actually implement and use the Config trait.

Just like before, we need some object which will implement this trait. In our case, we can use the Runtime struct itself.

#![allow(unused)]
fn main() {
impl system::Config for Runtime {
	type AccountId = String;
	type BlockNumber = u32;
	type Nonce = u32;
}
}

Then, when defining the system::Pallet within the Runtime, we can use the following syntax:

#![allow(unused)]
fn main() {
pub struct Runtime {
	system: system::Pallet<Self>,
}
}

Here we are basically saying that Pallet will use Runtime as its generic type, but this is defined within the Runtime, so we refer to it as Self.

The Power of Associated Types

Using traits with associated types does more than just make the code less verbose; it fundamentally changes how we can configure our blockchain system.

1. It creates a single place where all types that our blockchain system requires are defined: the implementation of the Config trait. As long as you use this single implementation, you ensure type consistency across your entire runtime.

2. It allows us to avoid hardcoding specific types, like String or u32, and instead use generic data types, like T::AccountId and T::BlockNumber. This allows us to swap out or change the final configured types in the runtime without changing any of the pallet code.

3. It allows us to create a single spot for us to configure those final types. You can see that we use the mod types {} section in main.rs to define all the concrete types in one place, so they can be consistently referenced. Without this, you might accidentally use different concrete types in different pallets (like u32 in one place and u64 in another), causing compilation errors or, worse, subtle bugs.

4. Finally, it allows us to configure different runtimes with completely different concrete types. This is an extremely powerful and often used feature in Substrate / the Polkadot-SDK.

   For example, we can create two runtime configurations, one for production and one for testing, and configure them completely differently:

   #![allow(unused)]
   fn main() {
   // In a production runtime
   struct Runtime;
   impl my_pallet::Config for Runtime {
   	type AccountId = AccountId32;
   	type BlockNumber = u32;
   	type Nonce = u64;
   	// etc...
   }
   
   // In a test runtime
   struct TestConfig;
   impl my_pallet::Config for TestConfig {
   	type AccountId = String;
   	type BlockNumber = u16;
   	type Nonce = u8;
   	// etc...
   }
   }

The Runtime is a Configuration Hub

You should be able to see a bigger idea forming based on how we have designed our blockchain system.

You don’t program the runtime, you configure it.

• We have designed our blockchain system with reusable pieces of application logic: the pallets.
• Each of these pallets are written generically, and are entirely configurable.
• Our runtime includes all of the pallets and pieces of logic it wants to use.
• Our runtime aggregates and propagates all the expected types for our blockchain system.

This pattern allows pallets to be completely independent and reusable. A pallet doesn’t need to know which other pallets exist in the runtime, it only needs its Config trait implemented. This means you can mix and match different pallets in different runtimes (production, test, development) without modifying the pallet code itself.

This is the difference between building a single purpose blockchain and building a blockchain SDK, allowing for reusable, modular components, and ultimately bootstrapping a powerful ecosystem of blockchain developers.

Make Your System Configurable

Phew. That was a lot.

Let’s practice all you have learned to create a Config trait for your System Pallet, and then configure the pallet for the Runtime in main.rs.

1. Define the Config trait which will have your 3 associated types AccountId, BlockNumber, and Nonce.
2. Make sure these types have their trait constraints defined in Config.
3. Update your struct Pallet to use T: Config and reference your types using the T:: syntax.
4. Update all of your functions to use the T:: syntax.
5. Update your test, creating a struct TestConfig, and implementing Config for it, and using it to instantiate your Pallet struct.
6. Go to your main.rs file, and implement system::Config for the Runtime struct.
7. Update your Runtime definition to instantiate system::Pallet with Self.

Again, this is a big step for new Rust developers, and a common place that people can get very confused.

You will have the opportunity to do this whole process again for the Balances Pallet, so don’t be afraid to peek at the solution this time around if you cannot get your code working.

Really take time to understand this step, what is happening, and what all of this syntax means to Rust.

Remember that Rust is a language which is completely type safe, so at the end of the day, all of these generic types and configurations need to make sense to the Rust compiler.

Make Balances Configurable

There is nothing new to learn in this step, just repeating the same process we did for our System Pallet for the Balances pallet.

In this case, our Config trait will only have two associated types: AccountId and Balance.

For this step, try to avoid looking at the solution, and instead refer to the changes you made to get the System Pallet configurable.

1. Define the Config trait which will have your associated types.
2. Make sure these types have their trait constraints defined in Config.
3. Update your struct Pallet to use T: Config and reference your types using the T:: syntax.
4. Update all of your functions to use the T:: syntax.
5. Update your test, creating a struct TestConfig, and implementing Config for it, and using it to instantiate your Pallet struct.
6. Go to your main.rs file, and implement balances::Config for the Runtime struct.
7. Update your Runtime definition to instantiate balances::Pallet with Self.

If you have made it this far, I think it is fair to say you have made it over the hardest part of this tutorial, and the hardest part of using Rust in Substrate.

It is important to take a step back and remember that while these abstractions make your code a bit more complicated to fully understand, it also makes your code extremely flexible, at zero cost to performance and safety thanks to the Rust compiler.

Tight Coupling

You might have noticed some redundancy when making our pallets generic and configurable. Both pallets defined an AccountId type, and technically we could define their concrete type differently!

We wouldn’t want this on a real production blockchain. Instead, we would want to define common types in a single spot, and use that everywhere.

Trait Inheritance

Rust has the ability for traits to inherit from one another. That is, that for you to implement some trait, you also need to implement all traits that it inherits.

Let’s look at some examples.

Trait Functions

We can extend our previous example to show what trait inheritance does with functions:

#![allow(unused)]
fn main() {
pub trait GetName {
	// returns a string representing the object's name
	fn name() -> String;
}

pub trait SayName: GetName {
	// will print the name from `name()` to console
	fn say_name() {
		println!("{}", Self::name());
	}
}
}

Note how in the definition of trait SayName, we reference GetName after a colon. This SayName, your object, must also implement GetName. Note that we could even program a “default” implementation of get_name by using the Self::name() function.

So when we implement these traits, it looks like:

#![allow(unused)]
fn main() {
struct Shawn;
impl GetName for Shawn {
	fn name() -> String {
		return "shawn".to_string();
	}
}

impl SayName for Shawn {}
}

We could choose to implement our own version of the SayName function, for example like:

#![allow(unused)]
fn main() {
impl SayName for Shawn {
	fn say_name() {
		println!("My name is {}!", Self::name());
	}
}
}

But we don’t have to do this. What we do have to do is make sure that GetName is implemented for Shawn or you wont be able to use the SayName trait. Again, we won’t be using this in our tutorial, but it is nice to see examples of how this can be used.

Associated Types

Rather than redefining type AccountId in each Pallet that needs it, what if we just defined it in system::Config, and inherit that type in other Pallet configs?

Let’s see what that would look like:

#![allow(unused)]
fn main() {
pub trait Config: crate::system::Config {
	type Balance: Zero + CheckedSub + CheckedAdd + Copy;
}
}

Here you can see our balances::Config trait is inheriting from our crate::system::Config trait. This means that all types defined by system::Config, including the AccountId, is accessible through the balances::Config trait. Because of this, we do not need to redefine the AccountId type in balances::Config.

In the Polkadot SDK ecosystem, we call this “tight coupling” because a runtime which contains the Balances Pallet must also contain the System Pallet. In a sense these two pallets are tightly coupled to one another. In fact, with Substrate, all pallets are tightly coupled to the System Pallet, because the System Pallet provides all the meta-types for your blockchain system.

Tightly Couple Balances To System

Let’s remove the redundant AccountId definition from the Balances Pallet Config.

1. Inherit the crate::system::Config trait in the balances::Config trait.
2. Remove the AccountId type from your balances::Config definition.
3. Implement crate::system::Config for TestConfig.
4. In main.rs, simply remove type AccountId from balances::Config.

Add Our Support Module

In this step, we will introduce a support module to help bring in various types and traits that we will use to enhance our simple state machine.

This support module parallels something similar to the frame_support crate that you would find in the Polkadot SDK.

The reason the frame_support crate exists, is to allow multiple other crates use common types and trait, while avoiding cyclic dependencies, which is not allowed in Rust.

Our simple state machine will not experience this problem explicitly, since we are building everything in a single crate, but the structure of the project will still follow these best practices.

Constructing a Block

The first set of primitives provided by the support module are a set of structs that we need to construct a simple Block.

The Block

A block is basically broken up into two parts: the header and a vector of extrinsics.

You can see that we keep the Block completely generic over the Header and Extrinsic type. The exact contents and definitions of these sub-types may change, but the generic Block struct can always be used.

The Header

The block header contains metadata about the block which is used to verify that the block is valid. In our simple state machine, we only store the blocknumber in the header, but real blockchains like Polkadot have:

• Parent Hash
• Block Number
• State Root
• Extrinsics Root
• Consensus Digests / Logs

The Extrinsic

In our simple state machine, extrinsics are synonymous with user transactions.

Thus our extrinsic type is composed of a Call (the function we will execute) and a Caller (the account that wants to execute that function).

The Polkadot SDK supports other kinds of extrinsics beyond a user transactions, which is why it is called an Extrinsic, but that is beyond the scope of this tutorial.

Dispatching Calls

The next key change we are going to make to our simple state machine is to handle function dispatching. Basically, you can imagine that there could be multiple different pallets in your system, each with different calls they want to expose.

Your runtime, acting as a single entrypoint for your whole state transition function needs to be able to route incoming calls to the appropriate functions. For this, we need the Dispatchable trait.

You will see how this is used near the end of this tutorial.

Dispatch Result

One last thing we added to the support module was a simple definition of the Result type that we want all dispatchable calls to return. This is exactly the type we already used for the fn transfer function, and allows us to return Ok(()) if everything went well, or Err("some error message") if something went wrong.

Create the Support Module

Now that you understand what is in the support module, add it to your project.

1. Create the support.rs file:

   touch src/support.rs
   

2. Copy and paste the content provided into your file.

3. Import the support module at the top of your main.rs file.

4. Finally, replace your Result<(), &'static str> with crate::support::DispatchResult in the fn transfer function in your Balances Pallet.

Introducing this new module will cause your compiler to emit lots of “never constructed” warnings. Everything should still compile, so that is okay. We will use these new types soon.

Create Your Block Type

The support module provided for us a bunch of generic types which can be customized for our simple state machine. To actually start using them, we need to define concrete versions of these types using our other concrete types.

Runtime Call

You will see the template provides an empty enum RuntimeCall which we will expand later. This is an object which is supposed to represent all the various calls exposed by your blockchain to users and the outside world. We need to mock this enum at this step so that it can be used to build a concrete Extrinsic type.

For now, there is just the transfer function exposed by the Balances Pallet, but we will add more before this tutorial is complete, and figure out ways to automate the creation of our RuntimeCall.

You can access this type within mod types with crate::RuntimeCall.

Building the Block Type

It’s time to define the concrete Block type that we will use to enhance our simple state machine.

1. Using the RuntimeCall enum and the AccountId type, you can define a concrete Extrinsic type.
2. Using the BlockNumber type, you can define a concrete Header type.
3. Using the concrete Header and Extrinsic types, you can define a concrete Block type.

As you can see, the Block is composed of layers of generic types, allowing the whole structure to be flexible and customizable to our needs.

Pay attention to the generic type definitions to ensure that you use all the correct generic parameters in all the right places.

Your code should still compile with some “never constructed/used” warnings.

Executing Blocks

We will now start the process to replace the simple block simulation in our main function with a proper block execution pipeline.

Execute Block

We have introduced a new function to our Runtime called fn execute_block.

The steps of this function is exactly the same as our current main function, but using the concrete Block type we defined to extract details like the expected block number and the extrinsics that we want to execute.

Iterating Over a Vector

In order to build our execute_block function, we will need to iterate over all the extrinsics in our block, and dispatch those calls. In rust, the common way to access the elements of a vector is to turn it into an iterator.

There are two functions used for turning a vector into an interator, iter and into_iter, and their difference lies in ownership:

• iter: This method creates an iterator that borrows each element from the vector, allowing you to read the values without taking ownership. It’s useful when you want to iterate over the vector while keeping it intact.

• into_iter: This method consumes the vector, transferring ownership of each element to the iterator. It’s handy when you want to move or transfer ownership of the vector’s elements to another part of your code. After using into_iter, the original vector can’t be used anymore, as ownership has been transferred.

In our context, we want to use into_iter(), so you will get something that looks like:

#![allow(unused)]
fn main() {
for support::Extrinsic { caller, call } in block.extrinsics.into_iter() {
	// do stuff with `caller` and `call`
}
}

Here you can see we also do a trick to separate out the fields of the Extrinsic in a single line, since ultimately we want to work with caller and call. You can of course break this process up into multiple lines if you want.

Dispatching a Call

Once we have the call and caller, what should we do with them?

This is where the Dispatch trait starts to come into play. You will see in our template, we included the shell of an unimplemented() fn dispatch. We will write this logic in the next step, but we need to already use the dispatch function in our execute_block logic.

Once we have the call and caller, we want to pass them to the dispatch logic, which you see is implemented on the Runtime.

That will look something like:

#![allow(unused)]
fn main() {
let _res = self.dispatch(caller, call).map_err(|e| eprintln!("{e}"));
}

Note that in Rust, if you want to access a function within a trait, like we do here with dispatch, you need to explicitly import that trait into your project.

We left a TODO at the top of main.rs where we ask you to import crate::support::Dispatch, which will allow you access to calling dispatch on Runtime.

Better Error Messages

Since this is a more permanent function of our project, it also makes sense to expand the message being printed when there are extrinsic errors. For example:

#![allow(unused)]
fn main() {
eprintln!(
	"Extrinsic Error\n\tBlock Number: {}\n\tExtrinsic Number: {}\n\tError: {}",
	block.header.block_number, i, e
)
}

This allows you to see the block number, extrinsic number, and the error message whenever there is an extrinsic error. This can be very helpful when you have many blocks being imported each with potentially many extrinsics.

To get the extrinsic number i, chain the enumerate() function after the into_iter().

Build Your Execute Block Function

You should now have all the tools and information needed to successfully write your execute_block function.

Follow the TODOs provided by the template, and make sure to include the impl crate::support::Dispatch for Runtime that we provided for you, and that we will implement in the next steps.

Your code should still compile with some “never constructed/used” warnings.

Dispatching Calls

We have built our execute_block logic depending on the dispatch logic we have not implemented yet.

Let’s do that.

Adding Our Calls

Dispatch logic is all about routing a user’s extrinsic to the proper Pallet function. So far, the only user callable function we have created is the transfer function in the Balances Pallet.

So let’s add that call to our RuntimeCall enum.

Our transfer function expects 3 inputs:

• caller: The account calling the transfer function, and whose balance will be reduced.
• to: The account where the funds will be sent.
• amount: The amount of funds to transfer.

However, remember that our dispatch logic already has information about the caller which is coming from the Extrinsic in the Block. So we do not need this data again in the RuntimeCall.

In fact, every Call in our runtime should omit the caller, and know that it is being provided by our dispatch logic.

So when adding a new variant to RuntimeCall, it should look something like:

#![allow(unused)]
fn main() {
pub enum RuntimeCall {
	BalancesTransfer { to: types::AccountId, amount: types::Balance },
}
}

A user submitting an extrinsic to our state machine can use this enum variant to specify which function they want to call (transfer), and the parameters needed for that call.

Dispatch Logic

The core logic in the dispatch function is a simple match statement.

Basically, given some RuntimeCall, we need to match on the variant being provided to us, and then pass the appropriate parameters to the correct Pallet function. As mentioned before, dispatch already has access to the caller information, so the final logic is as simple as:

#![allow(unused)]
fn main() {
match runtime_call {
	RuntimeCall::BalancesTransfer { to, amount } => {
		self.balances.transfer(caller, to, amount)?;
	}
}
}

Dispatch logic really is that simple!

Note that we propagate up any errors returned by our function call with the ? operator. This is important if you want to see the error messages that we set up in the execute_block logic.

Write Your Dispatch Logic

Follow the TODOs provided in the template to build your RuntimeCall and complete your dispatch logic.

Your code should still compile with some “never constructed/used” warnings. Just one more step and we will get rid of all those warnings!

Using Execute Block

We have now successfully implemented the execute_block and dispatch logic needed to build and execute real Blocks.

Let’s bring that logic into our main function.

Creating a Block

You can create a new Block by filling out all the fields of the struct and assigning it to a variable.

For example:

#![allow(unused)]
fn main() {
let block_1 = types::Block {
	header: support::Header { block_number: 1 },
	extrinsics: vec![
		support::Extrinsic {
			caller: &"alice",
			call: RuntimeCall::BalancesTransfer { to: &"bob", amount: 69 },
		},
	],
};
}

It is important that you set the block number correctly since we verify this in our execute_block function. The first block in our state machine will have the number 1.

Also remember that you can add multiple extrinsics in a single block by extending the vector.

Executing a Block

Once you have constructed your Block, you can pass it to the execute_block function implemented on your runtime.

#![allow(unused)]
fn main() {
runtime.execute_block(block_1).expect("invalid block");
}

Note how we panic with the message "invalid block" if the execute_block function returns an error. This should only happen when something is seriously wrong with your block, for example the block number is incorrect for what we expect.

This panic will NOT be triggered if there is an error in an extrinsic, as we “swallow” those errors in the execute_block function. This is the behavior we want.

Update Your Main Function

Go ahead and use the Block type and execute_block function to update the logic of your main function.

Follow the TODOs provided in the template to complete this step. Note that execute_block is now updating the caller’s nonce for us.

By the end of this step, your code should compile, test, and run successfully, all without compiler warnings!

Pallet Level Dispatch

We want to make our code more modular and extensible.

Currently, all dispatch happens through the RuntimeCall, which is hardcoding dispatch logic for each of the Pallets in our system.

What we would prefer is for Pallet level dispatch logic to live in the Pallet itself, and our Runtime taking advantage of that. We have already seen end to end what it takes to set up call dispatch, so let’s do it again at the Pallet level.

Pallet Call

To make our system more extensible, we want to keep all the calls for a pallet defined at the pallet level.

For this, we define an enum Call in our Balances pallet, and just like before, we introduce a new enum variant representing the function that we want to call.

Note that this enum needs to be generic over T: Config because we need access to the types defined by our configuration trait!

Pallet Dispatch

You will also notice in the template, we have included the shell for you to implement Pallet level dispatch.

Everything should look the same as the Runtime level dispatch, except the type Call is the Pallet level call we just created.

Just like before, you simply need to match the Call variant with the appropriate function, and pass the parameters needed by the function.

Create Your Pallet Level Dispatch

Follow the TODOs in the template to complete the logic for Pallet level dispatch.

The “never constructed” warning for the Transfer variant is okay.

In the next step, we will use this logic to improve our dispatch logic in our Runtime.

Nested Dispatch

Now that we have defined Pallet level dispatch logic in the Pallet, we should update our Runtime to take advantage of that logic.

After this, whenever the Pallet logic is updated, the Runtime dispatch logic will also automatically get updated and route calls directly. This makes our code easier to manage, and prevent potential errors or maintenance in the future.

Nested Calls

The Balances Pallet now exposes its own list of calls in balances::Call. Rather than list them all again in the Runtime, we can use a nested enum to route our calls correctly.

Imagine the following construction:

#![allow(unused)]
fn main() {
pub enum RuntimeCall {
	Balances(balances::Call<Runtime>),
}
}

In this case, we have a variant RuntimeCall::Balances, which itself contains a type balances::Call. This means we can access all the calls exposed by balances:Call under this variant. As we create more pallets or extend our calls, this nested structure will scale very well.

We call the RuntimeCall an “outer enum”, and the balances::Call an “inner enum”. This construction of using outer and inner enums is very common in the Polkadot SDK.

Re-Dispatching to Pallet

Our current dispatch logic directly calls the functions in the Pallet. As we mentioned, having this logic live outside of the Pallet can increase the burden of maintenance or errors.

But now that we have defined Pallet level dispatch logic in the Pallet itself, we can use this to make the Runtime dispatch more extensible.

To do this, rather than calling the Pallet function directly, we can extract the inner call from the RuntimeCall, and then use the balances::Pallet to dispatch that call to the appropriate logic.

That would look something like:

#![allow(unused)]
fn main() {
match runtime_call {
	RuntimeCall::Balances(call) => {
		self.balances.dispatch(caller, call)?;
	},
}
}

Here you can see that the first thing we do is check that the call is a Balances variant, then we extract from it the call which is a balances::Call type, and then we use self.balances which is a balances::Pallet to dispatch the balances::Call.

Updating Your Block

Since we have updated the construction of the RuntimeCall enum, we will also need to update our Block construction in fn main. Nothing magical here, just needing to construct a nested enum using both RuntimeCall::Balances and balances::Call::Transfer.

Enable Nested Dispatch

Now is the time to complete this step and glue together Pallet level dispatch with the Runtime level dispatch logic.

Follow the TODOs provided in the template to get your full end to end dispatch logic running. By the end of this step there should be no compiler warnings.

Proof of Existence Pallet

We have gone a long way since we built our very first Balances Pallet.

The structure of our Runtime and Pallets have evolved quite a bit since.

• Generic Types
• Config Trait
• Nested Dispatch
• and more…

This will be the last pallet we build for this tutorial, but we will build it knowing all of the tips and tricks we have learned so far.

The goal here is for you to ensure that all of the intricacies of Pallet development is well understood and that you are able to navigate all of the Rust code.

What is Proof of Existence?

The Proof of Existence Pallet uses the blockchain to provide a secure and immutable ledger that can be used to verify the existence of a particular document, file, or piece of data at a specific point in time.

Because the blockchain acts as an immutable ledger whose history cannot be changed, when some data is placed on the blockchain, it can be referenced at a future time to show that some data already existed in the past.

For example, imagine you discovered a cure to cancer, but before you reveal it, you want to make sure that you can prove when you had made the discovery. To do this, you could put some sort of data on the blockchain which represents the cure. At a later date, when you get your research published and reviewed, you would be able to use the blockchain as verifiable evidence of when you first made the discovery.

Normally, you would not put the raw contents of your claim on the blockchain but a hash of the data, which is both smaller and obfuscates the data in your claim before you are ready to reveal it.

However, for the purposes of this tutorial, we won’t introduce hash functions yet.

Pallet Structure

The BTreeMap is again the best tool to use for storing data in this Pallet. However, you will notice that the construction of the storage is a bit different than before. Rather than having a map from accounts to some data, we will actually map the content we want to claim to the user who owns it.

This construction of content -> account allows an account to be the owner of multiple different claims, but having each claim only be owned by one user.

Create Your Pallet

Let’s start to create this pallet:

1. Create a new file for your Proof of Existence Pallet.

   touch src/proof_of_existence.rs
   

2. Copy the contents from the template into your new file.

3. Complete the TODOs to add a storage to your new pallet and allow it to be initialized.

4. In your main.rs file, import the proof_of_existence module.

Make sure that everything compiles after you complete these steps.

Compiler warnings about “never read/used” are okay.

Proof Of Existence Functions

The Proof of Existence Pallet is quite simple, so let’s build out the logic needed.

Get Claim

Our Pallet has a simple storage map from some claimed content to the owner of that claim.

The get_claim function should act as a simple read function returning the T::AccountId of the owner, if there is any. In the case we query a claim which has no owner, we should return None.

This is not a function that a user would call from an extrinsic, but is useful for other parts of your state machine to access the data in this Pallet.

Create Claim

Any user can add a new claim to the Proof of Existence Pallet.

The only thing that is important is that we check that the claim has not already been made by another user.

Each claim should only have one owner, and whoever makes the claim first gets priority.

You can check if some claim is already in the claims storage using the contains_key api:

#![allow(unused)]
fn main() {
if self.claims.contains_key(&claim) {
	return Err(&"this content is already claimed");
}
}

Revoke Claim

Data on the blockchain is not free, and in fact is very expensive to maintain. Giving users the ability to clean up their data is not only good, but encouraged. If a user no longer has a need to store their claim on chain, they should clean it up.

Furthermore, the history of the blockchain is immutable. Even if the data about a claim does not exist in the “current state”, it can be shown to have existed in the past.

Keeping things in the current state just makes querying for information easier.

To revoke a claim, we need to check two things:

1. The claim exists.
2. The person who wants to revoke the claim is the owner of that claim.

You should be able to handle all of this logic by calling the get_claim function and using ok_or to return an error when the claim does not exist. If the claim does exist, you should be able to directly extract the owner from the state query.

Build Your Functions

Complete the TODOs outlined in the template.

Afterward, create a basic_proof_of_existence test to check that all your functions are working as expected.

This includes both the success and possible error conditions of your Pallet.

Add Proof of Existence Dispatch

We have already established the nested dispatch pipeline for Pallets in the Runtime.

Let’s build Pallet level dispatch logic for the Proof of Existence to take advantage of that.

Create Pallet Level Dispatch

There is nothing new here, but we have left more for you to fill out than before.

1. Create the variants for CreateClaim and RevokeClaim for your Call enum.
2. Implement the Dispatch trait for your Pallet.

If you get stuck, try not to look at the solution provided here, but instead look at what you did in the Balances Pallet. Everything we have done here, we have already done in the past. This is an opportunity to catch where you may have outstanding questions or misunderstandings.

Don’t worry about compiler warnings like “never used/constructed”.

Integrate PoE Into Your Runtime

The Proof of Existence pallet is done, but we still need to integrate it into your Runtime.

Let’s take a look at that process.

Integration Steps

1. The first place to start is adding the proof_of_existence field to your struct Runtime.

2. Next you need to update your fn new() to also initialize proof_of_existence.

3. After, create a new concrete type Content which is a String. As mentioned, normally this would be a hash, but for simplicity we are once again using a simple string.

   If you want to use a hash now or in the future, it would be as simple as updating this one line to change all the types in your Runtime and Pallet. That is the kind of flexibility we have been working toward!

4. Then, implement proof_of_existence::Config for Runtime, using your types::Content.

5. At this point, things should already compile successfully, so use this as a checkpoint.

6. Introduce a new variant ProofOfExistence for the RuntimeCall.

7. Finally, update your fn dispatch logic to handle re-dispatching ProofOfExistence calls to the proof_of_existence::Pallet.

Hopefully from this process, you can see how all of the abstractions we have introduced has made integrating new Pallets into your runtime quite easy.

We will make this process even easier in the near future using macros!

By the end of this step, everything should compile without warnings.

Add PoE Extrinsics to Blocks

The Proof Of Existence Pallet is fully integrated into your runtime at this point, but we aren’t really using it.

Create some new Blocks in your fn main() to test out the functionality of the Proof of Existence Pallet.

Be creative, and even feel free to introduce some extrinsics which will trigger errors based on the logic of your pallets.

Don’t forget to increment your block number and actually call execute_block for each of those blocks.

Take a look at the final output and check that the state of your machine makes sense!

Introducing Macros

If you have made it this far, then you have finished designing your simple state machine.

At this point, our goal is to see if we can use the power of Rust macros to make future development even easier.

All of this is in preparation for you to work with the Polkadot SDK, which heavily relies on macros like the ones you will see here.

Auto Generated Code

As mentioned earlier, Rust macros are basically code which can generate more code.

As you can see from our simple state machine, there is a lot of boiler plate code that we could generate, following the simple patterns and structures we have designed.

For example:

• We expect that each Pallet will expose some callable functions with Call.
• We know that each Call will have all the same parameters of the underlying Pallet function, except the caller.
• We know that each Pallet will implement Dispatch logic on the Pallet struct.
• We know that the Runtime will accumulate all the pallet::Calls into the RuntimeCall outer enum.
• We know that the Runtime will have logic to re-dispatch runtime level calls to the pallet level.
• and so on…

The more we abstract our Pallet and Runtime into consistent and and extensible pieces, the more we can automate, and ultimately this can provide a better developer experience.

Navigating the Macros

This tutorial is not attempting to teach you how to write these macros. That information would take a whole tutorial itself.

Instead, we are providing you with macros which should work directly with your existing code, and replace a lot of code that you have already written.

Macros in general are “magical”. If you have not written the macro yourself, there can be very little insight into what is happening underneath. In this context, the macros we are providing to you will directly replace code you have already written, so you should completely understand what is being generated, and how they work.

The macros folder contains a lib.rs, which exposes the two attribute macros built for this tutorial:

1. #[macros::call]
2. #[macros::runtime]

You can find the code for these two macros in their respective call and runtime folders.

In each of these folders there are 3 files:

1. mod.rs - The entry point for the macro, where code is parsed, and then generated.
2. parse.rs - The parsing logic for the macro, extracting the information we need to generate code.
3. expand.rs - The expansion / generation code, which will write new code for us with the data provided.

We will go through each of these more deeply as we include the macros into our code.

Adding the Macros to Our Project

All of the macros are contained within their own crate which will be a folder in your project.

Download the folder contents for the macros here: download

If that link does not work, you can extract the macros folder however is best for you from the source repository for this tutorial: https://github.com/shawntabrizi/rust-state-machine/tree/gitorial/

Once you have the contents of the macros folder:

1. Copy the contents into a macros folder in the root of your project.

2. Update your cargo.toml file to include this crate into your project:

   [dependencies]
   num = "0.4.1"
   macros = { path = "./macros/" }
   

Recompile your project, and you should see this new create and its sub-dependencies being compiled.

In the next step we will actually start integrating these macros into your simple state machine.

Adding Call Macro to Balances

Let’s start by adding the #[macros::call] macro to our Balances Pallet.

The Call Macro

The purpose of the #[macros::call] macro is to automatically generate the enum Call from the functions of the pallet and the pallet level Dispatch logic found in each Pallet.

We can place the #[macros::call] attribute over our impl<T: Config> Pallet<T> where the callable functions are implemented. From there, the macro can parse the whole object, and extract the data it needs. Not all of your functions are intended to be callable, so you can isolate the functions which should be in their own impl<T: Config> Pallet<T> as the template does.

Parse

In order to generate the code that we want, we need to keep track of:

1. Each callable function that the developer wants to expose through the Runtime.
   1. The name of that function.
   2. The argument names and types of that function.
2. The name of the struct where those functions are implemented. Normally this is Pallet, but we can allow the developer flexibility in their naming.

These things are tracked with CallDef and CallVariantDef.

Also, during the parsing process, we might want to check for certain consistencies in the code being parsed. In this case, we require that every callable function must have caller as their first parameter with type T::AccountId. This should make sense to you since you have designed a number of different callable functions, and they all follow this pattern.

This checking logic is handled by fn check_caller_arg.

Expand

Once we have parsed all the data we need, generating the code is pretty straight forward.

If you jump down to let dispatch_impl = quote! you will see a bunch of code that looks like the templates we used earlier in the tutorial. We just left markers where the macro generation logic should place all the information to write the code we need.

Macro Quirks

Macros are often very “quirky” when you use them. Since all of the input going into the macro is other code, sometimes the format of that code might not match what you expect.

For example, the original Call enum we have constructed looks like:

#![allow(unused)]
fn main() {
pub enum Call<T: Config> {
	Transfer { to: T::AccountId, amount: T::Balance },
}
}

The variant is called Transfer because the function it represents is named fn transfer.

However, if we want to generate the Call enum, and we only have fn transfer, where will we get the specific string Transfer with a capital T?

It is possible to do string manipulation and adjust everything to make it consistent to what Rust expects, but in this case it is better for our macros to make minimal modifications to user written code.

What does this mean?

When the #[macros::call] macro generates our enum Call, it will actually look like this:

#![allow(unused)]
fn main() {
#[allow(non_camel_case_types)]
pub enum Call<T: Config> {
	transfer { to: T::AccountId, amount: T::Balance },
}
}

Here you see that transfer is exactly the string which comes from the name of the function. Normally all enum variants should be CamelCase, but since rust functions are snake_case, our enum will have variants which are also snake_case. We won’t see any warnings about this because we enabled #[allow(non_camel_case_types)].

Ultimately, this has no significant impact on your underlying code. It is just ergonomics and expectations.

Indeed, macros can be quirky, but the amount of time they save you makes them worth it.

Time to Add Your Call Macro

1. If you haven’t, move your transfer function into its own impl<T: Config> Pallet<T>. We only want to apply the macro to this one function, so we need to isolate it from the other functions which are not meant to be callable.
2. Add the #[macros::call] attribute over this new impl<T: Config> Pallet<T>.
3. Delete your existing enum Call.
4. Delete your existing implementation of Dispatch for Pallet.
5. Then, in your main.rs file, change instances of balances::Call::Transfer to balances::Call::transfer with a lowercase t.

At this point, everything should compile just like before! We are witnessing the power of macros to generate code for us auto-magically!

Adding Call Macro to PoE

We have already seen the #[macros::call] macro help clean up the Balances Pallet.

Let’s also add it to the Proof of Existence Pallet, where there is even more code that can be eliminated.

Add Your Call Macro

We basically need to repeat the steps that we did for the Balances Pallet here:

1. Move your create_claim and revoke_claim functions into its own impl<T: Config> Pallet<T>.
2. Add the #[macros::call] attribute over this new impl<T: Config> Pallet<T>.
3. Delete your existing enum Call.
4. Delete your existing implementation of Dispatch for Pallet.
5. Then, in your main.rs file, change instances of:
   • proof_of_existence::Call::CreateClaim to proof_of_existence::Call::create_claim using snake_case.
   • proof_of_existence::Call::RevokeClaim to proof_of_existence::Call::revoke_claim using snake_case.

Check that everything is compiling and running just as before.

Expand your Rust Code

Let’s take the opportunity to show you how you can peek deeper into what the macros are doing.

Rust provides the command cargo expand which allows you to output the generated rust code after all macros have been applied to your project.

To install cargo expand:

cargo install cargo-expand

Then, run the following command:

cargo expand > out.rs

This will output your project’s generated code into a file out.rs.

Then take a look at that file.

Here are some things you should notice:

• All of your different mod files have been combined together into a single file with your main.rs.

• You will see that our final Pallet code has all of the Call and Dispatch logic generated!

• You might notice that the very first #[derive(Debug)] macro has generated code

  #![allow(unused)]
  fn main() {
  #[automatically_derived]
  impl<T: ::core::fmt::Debug + Config> ::core::fmt::Debug for Pallet<T>
  where
      T::Content: ::core::fmt::Debug,
      T::AccountId: ::core::fmt::Debug,
  {
      fn fmt(&self, f: &mut ::core::fmt::Formatter) -> ::core::fmt::Result {
          ::core::fmt::Formatter::debug_struct_field1_finish(f, "Pallet", "claims", &&self.claims)
      }
  }
  }

• You might even notice that other smaller macros like vec![] have changed:

  #![allow(unused)]
  fn main() {
  extrinsics: <[_]>::into_vec(
  	#[rustc_box]
  	::alloc::boxed::Box::new([
  		// stuff
  	])
  )
  }

• And println!() :

  #![allow(unused)]
  fn main() {
  {
  	::std::io::_print(format_args!("{0:#?}\n", runtime));
  };
  }

• etc…

There are two main takeaways for you:

1. Macros ultimately follow all the same rules as regular Rust code, because it does generate regular Rust code. They feel magical, but there is really nothing magic about them.
2. Macros are an important part of the Rust ecosystem, and heavily used to improve developer experience and code quality.

If you ever use externally developed macros, and you want to look closer at what is going on, cargo expand can be a useful tool for you to better understand some of the hidden architectural details of a project. As you jump into the Polkadot SDK, I recommend you continue to use this tool to enhance your learning and understanding.

Use the Runtime Macro

Finally, let’s add the #[macros::runtime] macro to our main.rs file, and really clean up a ton of boilerplate code.

Runtime Macro

The purpose of the #[macros::runtime] macro is to get rid of all of the boilerplate function we implemented for the Runtime, including fn new() and fn execute_block(). Similar to the Call macro, it also generates the enum RuntimeCall and all the dispatch logic for re-dispatching to pallets.

We apply the #[macros::runtime] attribute on top of the main struct Runtime object.

Parse

In order to generate the code we want, we need to keep track of:

1. The name of the struct representing our Runtime. Usually this is Runtime, but we provide flexibility to the developer.
2. The list of Pallets included in our Runtime
   1. Their name, as specified by the user.
   2. The specific type for their Pallet, for example balances::Pallet vs proof_of_existence::Pallet.

All of this information is tracked in the RuntimeDef struct.

We are also checking that our Runtime definition always contains the System Pallet, and does so as the first pallet in our Runtime definition. We will explain more about the assumption of the macros below.

Expand

Once we have parsed all the data we need, we just need to generate the code that we expect.

Starting with let runtime_impl = quote!, you will see the entire impl Runtime code block has been swallowed into the macro. Since we know all the pallets in your Runtime, we can automatically implement functions like new(). The execute_block function does not take advantage of any of the parsed data, but the code is completely boilerplate, so we hide it away.

Then we have another code block being generated with let dispatch_impl = quote! which is the enum RuntimeCall and the implementation of Dispatch for Runtime.

Again, due to the quirks of using macros, our RuntimeCall enum will have snake_case variants which exactly match the name of the fields in the Runtime struct.

Macro Assumptions

One of the assumptions programmed into these macros is the existence of the System Pallet. For example, in the execute_block logic, we need access to both system.inc_block_number and system.inc_nonce.

Some macro level assumptions are intentional, and actually define the architectural decisions of the framework designing those macros. This is the case with the System Pallet, since so much of a blockchain framework depends on a consistent meta-layer.

Other assumptions exist just because it is easier to write the macro if the assumption is made.

The main takeaway here is that macros can almost always continue to improve, providing better and better user experiences for developers. It just needs someone to identify what improvements need to be made, and someone else to program those improvements into the low level macro code.

Add the Runtime Macro

Let’s finally go through the steps to add the #[macros::runtime] attribute to your Runtime.

1. In main.rs, add #[macros::runtime] on top of your pub struct Runtime.
2. Remove the entire impl Runtime code block.
3. Remove the entire enum RuntimeCall.
4. Remove the entire implementation of Dispatch for Runtime.
5. Update instances of the RuntimeCall enum to use snake_case:
   • Change RuntimeCall::Balances to RuntimeCall::balances.
   • Change RuntimeCall::ProofOfExistence to RuntimeCall::proof_of_existence.

And that’s it! You have now completed the full tutorial for building a simple rust state machine. 🎉