Installation

You have to install the following dependencies before you can install Trdelnik:

Install Rust (nightly release)
Install Solana tool suite (stable release)
Install Anchor

Trdelnik can be installed using the cargo package manager. To install Trdelnik, run the following command:

cargo install trdelnik-cli

# or the specific version

cargo install --version <version> trdelnik-cli

Motivation

The usual way of testing Anchor programs in the past was to use the testing framework baked into Anchor. Anchor utilizes JavaScript (TypeScript) testing library Mocha to test programs.

The tests are run on solana-test-validator, which is a Solana node running. This is a great way to test programs, but it has some drawbacks:

All recommendations for writing tests say that they should be independent on each other. This is not possible with the current testing framework, because the state of the program is shared between tests.
Tests are run sequentially, which is not optimal for larger programs with many tests as they can take a long time to run.
The tests are written in JavaScript, which can be tedious to debug and write.

With that said the testing framework some really cool features baked in, such as the generated types and methods for interacting with the program. Therefore it is really easy to write tests for programs.

How does Trdelnik solves those problems?

We wanted to provide similar fast and easy way of testing programs, but with the ability to run tests in parallel and with the ability to write them in Rust.

For each test we create a local validator and deploy the program to it. This way we can run tests in parallel and they are independent on each other. We also provide a console to interact with the program, which is really useful for debugging.

use trdelnik_client::{anyhow::Result, *};

#[trdelnik_test]
async fn test_happy_path1() {
    let mut validator = Validator::default();

    validator.add_program("escrow", program_id.pubkey());
    let trdelnik_client = validator.start().await;
    trdelnik_client
        .create_token_mint(&keypair(1), system_keypair(1).pubkey(), None, 0)
        .await?;
}

Trdelnik also generates a instruction helper methods for each instruction in the program. This way you can call similarly to IDL generated methods for TypeScript. The methods are in the .program_client module. This simplyfies the code and makes it easier to write tests.

use trdelnik_client::{anyhow::Result, *};
use program_client::escrow_instruction;
use escrow;


#[trdelnik_test]
async fn test_happy_path1() {
    let mut validator = Validator::default();
    validator.add_program("escrow", &escrow::id());
    let trdelnik_client = validator.start().await;
    let mut fixture = Fixture::new(trdelnik_client);

    // This is a helper method generated by Trdelnik
    escrow_instruction::initialize_escrow(
        &trdelnik_client,
        500,
        1000,
        fixture.alice_wallet.pubkey(),
        fixture.alice_token_a_account,
        fixture.alice_token_b_account,
        fixture.escrow_account.pubkey(),
        System::id(),
        token::ID,
        [fixture.alice_wallet.clone(), fixture.escrow_account.clone()],
    )
    .await?;
}

How to write tests using Trdelnik?

Initialization

Initialize the project with

# navigate to your project root directory
trdelnik init
# it will generate `.program_client` and `trdelnik-tests`
# directories with all the necessary files
trdelnik test
# view all commands
trdelnik --help

Overview

The trdlenik init command generated a skeleton for trdelnik tests. The trdelnik-tests is a rust package that willcontain all the tests. The trdelnik-tests/Cargo.toml contains all the dependencies for the tests.

The trdelnik-tests/tests/test.rs contains a test example.

Then there is the .program_client package that contains all the generated code for the program. Using this you can easily call instructions of your Anchor program. If you add any new instruction or change the derive_id!(...) program pubkey regenerate this package using trdelnik build.

Make sure your program is using a correct program ID in the derive_id!(...) macro and inside Anchor.toml. If not, obtain the public key of a key pair you're using and replace it in these two places.

If there are any dependencies that you need when calling instructions, add them to the .program_client/src/lib.rs at top using the use keyword.

Testing

Here is a sample test that can be run using trdelnik test command. The default behaviour of is to run tests in paralell only if they are in the same file. You can run the tests parallel across multiple files using nextest. For installation instructions, see here. Then just use the this flag while running tests trdelnik test --nextest command.

#![allow(unused)]
fn main() {
// <my_project>/trdelnik-tests/tests/test.rs
// TODO: do not forget to add all necessary dependencies to 
// the generated `trdelnik-tests/Cargo.toml`
use program_client::my_instruction;
use trdelnik_client::*;
use my_program;

#[throws]
#[fixture]
async fn init_fixture() -> Fixture {
  let mut validator = Validator::default();
  // @todo: here you can call your add your program
  // validator.add_program("name", PROGRAM_ID);
  let client = validator.start().await;
  // create a test fixture
  let mut fixture = Fixture {
    client,
    // make sure your program is using a correct program ID
    program: program_keypair(1),
    state: keypair(42),
  };

  // deploy a tested program
  fixture.deploy().await?;
  // call instruction init
  my_instruction::initialize(
    &fixture.client,
    my_instruction::instruction::Initialize { state: fixture.state.pubkey() },
    my_instruction::accounts::Initialize { 
      initializer: fixture.client.payer().pubkey(),
      system_program: System::id(),
    },
    Some(fixture.state.clone()),
  ).await?;
  fixture
}

#[trdelnik_test]
async fn test_happy_path(#[future] init_fixture: Result<Fixture>) {
  let fixture = init_fixture.await?;
  // call the instruction
  my_instruction::do_something(
    &fixture.client,
    "dummy_string".to_owned(),
    fixture.state.pubkey(),
    None,
  ).await?;
  // check the test result
  let state = fixture.get_state().await?;
  assert_eq!(state.something_changed, "yes");
}
}

Let's break down what is happening in the test.

Fixture

First a fixture for test is initialized. The fixture is then injected into the test using the #[future] init_fixture: Result<Fixture> parameter of the test function. This is done using the amazing rstest package.

The fixture can have other fixtures as it's function parameters. You can define some common fixture for initialization and other common tasks and then just use them across all the tests.

In the fixture a local validator is started. The validator is a local Solana cluster that can be used for testing. The validator is started using the validator.start().await command. The validator is stopped when fixture goes out of scope. This allows to run multiple tests in parallel as every validator is indepenent and can process transactions without affecting each other's state.

Test function itself

Then there is the test function itself. The test function is marked with the #[trdelnik_test] macro. This is a standard rust test that can be run using cargo test as well.

You can have multiple test functions in the same file, therefore try to keep the logic inside of the tests simple.

Instructions with custom structures

If you want to test an instruction which has custom structure as an argument

#![allow(unused)]
fn main() {
pub struct MyStruct {
  amount: u64,
}

// ...

pub fn my_instruction(ctx: Context<Ctx>, data: MyStruct) { /* ... */ }
}

You should add an import to the .program_client crate

#![allow(unused)]
fn main() {
// .program_client/src/lib.rs

// DO NOT EDIT - automatically generated file
pub mod my_program_instruction {
  use trdelnik_client::*;
  use my_program::MyStruct; // add this import

// ...
}
}

This file is automatically generated but the use statements won't be regenerated

Skipping tests

You can add the #[ignore] macro to skip the test.

#![allow(unused)]
fn main() {
#[trdelnik_test]
#[ignore]
async fn test() {}
}

Testing programs with associated token accounts

Trdelnik does not export anchor-spl and spl-associated-token-account, so you have to add it manually.

# <my-project>/trdelnik-tests/Cargo.toml
# import the correct versions manually
anchor-spl = "0.24.2"
spl-associated-token-account = "1.0.3"

#![allow(unused)]
fn main() {
// <my-project>/trdelnik-tests/tests/test.rs
use anchor_spl::token::Token;
use spl_associated_token_account;

async fn init_fixture() -> Fixture {
  // ...
  let account = keypair(1);
  let mint = keypair(2);
  // constructs a token mint
  client
    .create_token_mint(&mint, mint.pubkey(), None, 0)
    .await?;
  // constructs associated token account
  let token_account = client
    .create_associated_token_account(&account, mint.pubkey())
    .await?;
  let associated_token_program = spl_associated_token_account::id();
  // derives the associated token account address for the given wallet and mint
  let associated_token_address = spl_associated_token_account::get_associated_token_address(&account.pubkey(), mint);
  Fixture {
    // ...
    token_program: Token::id(),
  }
}
}

The trdelnik init command generated a dummy test suite for you.
For more details, see the complete test implementation.

Fuzz testing usage

Fuzz testing is a type of automated testing that involves providing invalid, unexpected, or random input to a software program in order to test its robustness and detect bugs. Trdelnik provides an easy to use interface for fuzz testing your Solana programs.

Usage

To create a new fuzz test run the

trdelnik fuzz new <fuzz_test_name>

command. This will create a new fuzz test in the trdelnik-tests/fuzz-tests directory. The fuzz test will be named <fuzz_test_name>.rs. The fuzz test will be automatically added to the trdelnik-tests/Cargo.toml for Rust to be able to execute it as binary.

In the trdelnik-tests also add the fuzz testing library using cargo add trdelnik-fuzz.

Getting started with fuzz tests

For the purpose of explaining the fuzz testing, simple anchor-counter smart contract will be used.

#![allow(unused)]
fn main() {
#[account]
pub struct Counter {
    pub count: u64,
}

#[program]
pub mod anchor_counter {
    use super::*;

    pub fn increment(ctx: Context<Update>) -> Result<()> {
        let counter = &mut ctx.accounts.counter;
        counter.count += 1;
        msg!("Current count is {}", counter.count);
        Ok(())
    }

    pub fn decrement(ctx: Context<Update>) -> Result<()> {
        let counter = &mut ctx.accounts.counter;
        counter.count -= 1;
        msg!("Current count is {}", counter.count);
        Ok(())
    }
}
}

The smart contract supports only incrementing and decrementing passed account. It saves the current count as a u64 number. Fuzz testing should be able to find that decrementing the counter below 0 will cause an overflow.

Starting the fuzz tests

The fuzz tests are just a simple binary rust programs, that have the #[trdelnik_fuzz] macro added to the main function. Then the FuzzTestBuilder struct is used to build the fuzz test. The builder allows to add flows, invariants and validators to the fuzz test. The builder also allows to set to set the number of sequences and the number of iterations per sequence.

#[trdelnik_fuzz]
async fn main() {
    FuzzTestBuilder::new()
        .initialize_validator(initialize_validator)
        .add_flow(flow_increment)
        .add_flow(flow_decrement)
        .with_state(TestState {
            counter_account: CopyableKeypair(Keypair::new()),
            count: 0,
        })
        .add_invariant(invariant_check_counter)
        .start(2, 250)
        .await;
}

When fuzz testing the smart contract, there should be some local state that will be just a simple data structure in Rust. The advantage of having such a simple state is that the developer does not need to think about accounts, just about what should be the resulting state after the flow is executed. This local state is then validated against the state on the blockchain.

The fuzz test starts by running the initialize_validator function. This function is used to initialize the validator with smart contract and possibly other accounts.

#![allow(unused)]
fn main() {
fn initialize_validator() -> Validator {
    let mut validator = Validator::default();
    validator.add_program("anchor_counter", PROGRAM_ID);
    validator
}
}

The start function ran specifies how many validators will be ran in parallel and how many flows should be executed per validator. As seen on line 12 of the FuzzTestBuilder.

State

The developer can register his own state structs that will be automatically injected into the flows and invariants. The state gets registered using the with_state function. The state struct needs to implement the Clone and Debug trait.

#![allow(unused)]
fn main() {
#[derive(Clone, Debug)]
struct TestState {
    count: i128,
    counter_account: CopyableKeypair,
}
}

Flows

Flows are functions mutating the state of the smart contract. The ran flow is randomly chosen from the array of registered flows. Flows are registered using the add_flow function.

State was automatically injected into the flow function. The flow function can also take other arguments, but they need to be registered in the with_state function.

Flow functions first mutate the Smart Contract and then synchronize what they did in the local state.

#![allow(unused)]
fn main() {
async fn flow_increment(State(mut state): State<TestState>, client: Client) {
    anchor_counter_instruction::increment(
        &client,
        Increment {},
        Update {
            counter: state.counter_account.0.pubkey(),
            user: client.payer().pubkey(),
            system_program: System::id(),
        },
        vec![client.payer().clone(), state.counter_account.0.clone()],
    )
    .await
    .unwrap();
    state.count += 1;
}
}

This is how the decrement flow looks like.

#![allow(unused)]
fn main() {
async fn flow_decrement(State(mut state): State<TestState>, client: Client) {
    anchor_counter_instruction::decrement(
        &client,
        Decrement {},
        Update {
            counter: state.counter_account.0.pubkey(),
            user: client.payer().pubkey(),
            system_program: System::id(),
        },
        vec![client.payer().clone(), state.counter_account.0.clone()],
    )
    .await
    .unwrap();
    state.count -= 1;
}
}

Invariants

Invariants are functions that check the state of the smart contract. They are ran after each flow. Invariants are registered using the add_invariant function.

Here the invariant just checks that the count in the local state is the same as the count in the smart contract.

#![allow(unused)]
fn main() {
async fn invariant_check_counter(State(state): State<TestState>, client: Client) {
    let counter_account = client
        .get_account(state.counter_account.0.pubkey())
        .await
        .unwrap()
        .unwrap();

    let counter_account = Counter::try_deserialize(&mut counter_account.data()).unwrap();
    assert_eq!(counter_account.count as i128, state.count);
}
}

Running the fuzz tests

The test can be ran using the trdelnik fuzz run <fuzz_test_name> command. The fuzz test will be ran for the specified number of sequences and iterations per sequence. The fuzz test will be ran in a docker container, so the developer does not need to worry about the environment.

Command line tool

The trdelnik command line tool is used to generate, run and debug tests for Solana programs.

Build Command

Creates a program_client crate.

Parameters:
- root: (Optional) Anchor project root. Default value is ./.

Explorer Command

The Hacker's Explorer.

Init Command

Initializes all the necessary files for a Trdelnik testing environment.

KeyPair Command

Gets information about a keypair.

$ trdelnik key-pair program 7

will print information about the key pair received from program_keypair(7).

Test Command

Runs program tests.

Parameters:
- root: (Optional) Anchor project root. Default value is ./.
- nocapture: (Optional) Do not capture and hide output from tests.
- package: (Optional) Package to test.
- nextest: (Optional) Runs all the tests in parallel using nextest. For installation instructions, see here.
- test_name: (Optional) Name of the test to run.

Trdelnik