Using Substrate API Sidecar with Moonbeam¶
Introduction¶
Substrate API Sidecar allows applications to access blocks, account balance, and other information of Substrate-based blockchains through a REST API. This can be useful for exchanges, wallets or other types of applications that need to keep track of account balance and other state changes on a Moonbeam network. This page will describe how to install and run a Substrate API Sidecar for Moonbeam, and the commonly used API endpoints.
Installing and Running Substrate API Sidecar¶
There are multiple ways of installing and running the Substrate API Sidecar. This guide will describe the steps for installing and running it locally through NPM. For running Substrate API Sidecar through Docker, or building and running it from source, please refer to the Substrate API Sidecar Github Repository.
Checking Prerequisites¶
Running this service locally through NPM requires Node.js to be installed.
You need to install Node.js (for this example, you can use v16.x) and the npm package manager. You can download directly from Node.js or in your terminal:
curl -sL https://deb.nodesource.com/setup_16.x | sudo -E bash -
sudo apt install -y nodejs
# You can use homebrew (https://docs.brew.sh/Installation)
brew install node
# Or you can use nvm (https://github.com/nvm-sh/nvm)
nvm install node
You can verify that everything is installed correctly by querying the version for each package:
node -v
npm -v
Installing the Substrate API Sidecar¶
To install the Substrate API Sidecar service locally in the current directory, run this from the command line:
npm install @substrate/api-sidecar@19.2.0
Note
If the current folder does not already have a Node.js project structure, you need to manually created the node_modules
directory by typing mkdir node_modules
.
Substrate API Sidecar v19.2.0 is the current stable version that has been tested to work with Moonbeam networks. You can verify the installation was successful by typing from the installation directory root:
node_modules/.bin/substrate-api-sidecar --version
Setting up the Substrate API Sidecar¶
In the terminal that Sidecar will run, export the environmental variable for the WS endpoint of the network. Examples:
export SAS_SUBSTRATE_URL=wss://wss.api.moonbeam.network
export SAS_SUBSTRATE_URL=wss://wss.api.moonriver.moonbeam.network
export SAS_SUBSTRATE_URL=wss://wss.api.moonbase.moonbeam.network
export SAS_SUBSTRATE_URL=ws://127.0.0.1:9944
Please reference the Public Endpoints page for a full list of Moonbeam network endpoints.
After setting the environmental variable, you can use the echo
command to check that the environmental variable has been set correctly, by typing:
echo $SAS_SUBSTRATE_URL
And it should display the network endpoint you have just set.
Generating the Types Bundle¶
Moonbeam introduces custom types that differ from the standard Substrate types. For API clients like Substrate API Sidecar to properly understand and decode these custom types, you must provide Substrate API Sidecar with the corresponding custom types bundle for the respective network you're interacting with. Generating and associating the custom types bundle with Substrate API Sidecar is quick.
First, ensure that you installed Parity's generate-types-bundle
package:
npm install -g @substrate/generate-type-bundle
Then, navigate to the following directory within your project:
cd ./node_modules/@substrate/api-sidecar/build/src/
Then, run the following command to generate the types bundle for the respective network:
generate-type-bundle -p . -s moonbeam
generate-type-bundle -p . -s moonriver
generate-type-bundle -p . -s moonbase
Note that running subsequent commands will overwrite the existing typesBundle.json.
You'll then need to set the SAS_SUBSTRATE_TYPES_BUNDLE
environment variable as shown below. If you've renamed the typesBundle.json,
ensure you use the correct file name.
export SAS_SUBSTRATE_TYPES_BUNDLE="./typesBundle.json"
After setting the environment variable, you can verify that you set it correctly by using the following echo
command:
echo $SAS_SUBSTRATE_TYPES_BUNDLE
Running Substrate API Sidecar¶
Navigate back to the root directory of the project:
cd ../../../../..
With the network endpoint environmental variable set, and from the installation directory root, run:
node_modules/.bin/substrate-api-sidecar
If the installation and configuration are successful, you should see this output in the console:
v0.40.0: Pulling from moonbeamfoundation/moonbeam
SAS:
π¦ LOG:
β LEVEL: "info"
β JSON: false
β FILTER_RPC: false
β STRIP_ANSI: false
β WRITE: false
β WRITE_PATH: "/temp/node_modules/@substrate/api-sidecar/build/src/logs"
β WRITE_MAX_FILE_SIZE: 5242880
β WRITE_MAX_FILES: 5
π¦ SUBSTRATE:
β URL: "wss://wss.api.moonbeam.network"
β TYPES_BUNDLE: undefined
β TYPES_CHAIN: undefined
β TYPES_SPEC: undefined
β TYPES: undefined
π¦ EXPRESS:
β BIND_HOST: "127.0.0.1"
β PORT: 8080
β KEEP_ALIVE_TIMEOUT: 5000
2024-05-07 11:29:54 info: Version: 18.0.0
2024-05-07 11:29:55 warn: API/INIT: RPC methods not decorated: eth_getBlockReceipts, moon_isBlockFinalized, moon_isTxFinalized
2024-05-07 11:29:55 warn: API/INIT: moonbeam/2902: Not decorating unknown runtime apis: 0xd0399cd053adda2b/1, 0xa33d43f58731ad84/2, 0xba8173bf23b2e6f8/1
2024-05-07 11:29:55 info: Connected to chain Moonbeam on the moonbeam client at wss://wss.api.moonbeam.network
2024-05-07 11:29:55 info: Listening on http://127.0.0.1:8080/
2024-05-07 11:29:55 info: Check the root endpoint (http://127.0.0.1:8080) to see the available endpoints for the current node
Substrate API Sidecar Endpoints¶
Some of the commonly used Substrate API Sidecar endpoints include:
- GET /blocksβ/head β Get the most recently finalized block. The optional parameter
finalized
can be set tofalse
to the get the newest known block, which may not be finalized - GET /blocks/head/header β Get the most recently finalized block header. The optional parameter
finalized
can be set tofalse
to the get the newest known block header, which may not be finalized - GET /blocks/{blockId} β Get a block by its height or hash
- GET /accounts/{accountId}/balance-info β Get balance information for an account
- GET /node/version β Get information about the Substrates node's implementation and versioning
- GET /runtime/metadata β Get the runtime metadata in decoded, JSON form.
For a full list of API endpoints available on Substrate API Sidecar, please refer to the official documentation.
EVM Field Mapping in Block JSON Object¶
Substrate API Sidecar returns Moonbeam blocks as a JSON object. Information related to EVM execution of Moonbeam transactions is under the extrinsics
top level field, where individual extrinsics are organized numerically as nested JSON objects. The nesting structure is as following:
RESPONSE JSON Block Object:
|--extrinsics
|--{extrinsic_number}
|--method
|--pallet: "ethereum"
|--method: "transact"
|--signature
|--nonce
|--args
|--transaction
|--{transaction_type}
|--hash
|--events
|--{event_number}
|--method
|--pallet: "ethereum"
|--method: "Executed"
|--data
|--0
|--1
|--2
|--3
...
Moonbeam EVM transactions can be identify by the method
field under the current extrinsic object, where it is set to:
{extrinsic_number}.method.pallet = "ethereum"
{extrinsic_number}.method.method = "transact"
Transaction Types and Payload¶
The Moonbeam EVM currently supports three transaction standards: legacy
, eip1559
, and eip2930
. These correspond to the transaction type
field in the above JSON object diagram. For each transaction type, the transaction payload contains the following fields:
...
|--eip1559
|--chainId
|--nonce
|--maxPriorityFeePerGas
|--maxFeePerGas
|--gasLimit
|--action
|--value
|--input
|--accessList
|--oddYParity
|--r
|--s
...
...
|--legacy
|--nonce
|--gasPrice
|--gasLimit
|--action
|--value
|--input
|--signature
...
...
|--eip2930
|--chainId
|--nonce
|--gasPrice
|--gasLimit
|--action
|--value
|--input
|--accessList
|--oddYParity
|--r
|--s
...
For more information on the new EIP1559 and EIP2930 transaction types and what each field means, please refer to the respective official Ethereum proposal specs.
Transaction Field Mappings¶
To obtain the EVM sender address, recipient address, and EVM hash of any EVM transaction type, check the events
field under the current extrinsic object, and identify the event where the method
field is set to:
{event_number}.method.pallet: "ethereum"
{event_number}.method.method: "Executed"
The EVM field mappings are then summarized as the following:
EVM Field | Block JSON Field |
---|---|
Chain ID | extrinsics[extrinsic_number].args.transaction.eip1559.chainId |
Nonce | extrinsics[extrinsic_number].args.transaction.eip1559.nonce |
Max priority fee per gas | extrinsics[extrinsic_number].args.transaction.eip1559.maxPriorityFeePerGas |
Max fee per gas | extrinsics[extrinsic_number].args.transaction.eip1559.maxFeePerGas |
Gas limit | extrinsics[extrinsic_number].args.transaction.eip1559.gasLimit |
Access list | extrinsics[extrinsic_number].args.transaction.eip1559.accessList |
Signature | extrinsics[extrinsic_number].args.transaction.eip1559.oddYParity/r/s |
Sender address | extrinsics[extrinsic_number].events[event_number].data[0] |
Recipient address | extrinsics[extrinsic_number].events[event_number].data[1] |
EVM hash | extrinsics[extrinsic_number].events[event_number].data[2] |
EVM execution status | extrinsics[extrinsic_number].events[event_number].data[3] |
EVM Field | Block JSON Field |
---|---|
Nonce | extrinsics[extrinsic_number].args.transaction.legacy.nonce |
Gas price | extrinsics[extrinsic_number].args.transaction.legacy.gasPrice |
Gas limit | extrinsics[extrinsic_number].args.transaction.legacy.gasLimit |
Value | extrinsics[extrinsic_number].args.transaction.legacy.value |
Signature | extrinsics[extrinsic_number].args.transaction.legacy.signature |
Sender address | extrinsics[extrinsic_number].events[event_number].data[0] |
Recipient address | extrinsics[extrinsic_number].events[event_number].data[1] |
EVM hash | extrinsics[extrinsic_number].events[event_number].data[2] |
EVM execution status | extrinsics[extrinsic_number].events[event_number].data[3] |
EVM Field | Block JSON Field |
---|---|
Chain ID | extrinsics[extrinsic_number].args.transaction.eip2930.chainId |
Nonce | extrinsics[extrinsic_number].args.transaction.eip2930.nonce |
Gas price | extrinsics[extrinsic_number].args.transaction.eip2930.gasPrice |
Gas limit | extrinsics[extrinsic_number].args.transaction.eip2930.gasLimit |
Value | extrinsics[extrinsic_number].args.transaction.eip2930.value |
Access list | extrinsics[extrinsic_number].args.transaction.eip2930.accessList |
Signature | extrinsics[extrinsic_number].args.transaction.eip2930.oddYParity/r/s |
Sender address | extrinsics[extrinsic_number].events[event_number].data[0] |
Recipient address | extrinsics[extrinsic_number].events[event_number].data[1] |
EVM hash | extrinsics[extrinsic_number].events[event_number].data[2] |
EVM execution status | extrinsics[extrinsic_number].events[event_number].data[3] |
Note
For Substrate transactions, the "Nonce" and "Signature" fields are under extrinsics[extrinsic_number]
. For EVM transactions, the "Nonce" and "Signature" fields are under extrinsics[extrinsic_number].args.transaction[transaction_type]
, leaving the "Nonce" and "Signature" under extrinsics[extrinsic_number]
to be null
.
A successfully executed EVM transaction will return either succeed: "Stopped"
or succeed: "Returned"
under the "EVM Execution Status" field.
ERC-20 Token Transfers¶
Events emitted by smart contracts such as an ERC-20 token contract deployed on Moonbeam can be decoded from Sidecar block JSON objects. The nesting structure is as following:
RESPONSE JSON Block Object:
|--extrinsics
|--{extrinsic_number}
|--method
|--pallet: "ethereum"
|--method: "transact"
|--signature:
|--nonce:
|--args
|--transaction
|--{transaction_type}
|--hash
|--events
|--{event_number}
|--method
|--pallet: "evm"
|--method: "Log"
|--data
|--0
|-- address
|-- topics
|--0
|--1
|--2
|-- data
...
...
Moonbeam ERC-20 token transfers will emit the Transfer
event which can be decoded as the following:
Tx Information | Block JSON Field |
---|---|
ERC-20 contract address | extrinsics[extrinsic_number].events[event_number].data[0].address |
Event signature hash | extrinsics[extrinsic_number].events[event_number].data[0].topics[0] |
Sender address | extrinsics[extrinsic_number].events[event_number].data[0].topics[1] |
Recipient address | extrinsics[extrinsic_number].events[event_number].data[0].topics[2] |
Amount | extrinsics[extrinsic_number].events[event_number].data[0].data |
Other events emitted by EVM smart contracts can be decoded in a similar fashion, but the content of the topics and data fields will change depending on the definition of the specific event.
Note
The amount transferred is given in Wei and in hexadecimal format.
Sample Code for Monitoring Native Token Transfers¶
The Transfers API page has a code snippet demonstrating how to use Substrate API Sidecar to retrieve and decode native token transfers sent with both Substrate and Ethereum APIs on Moonbeam networks. You can reference that as a starting point to build out backends that utilize Sidecar to listen to transfers on Moonbeam networks.
Calculating Transaction Fees¶
For more detailed information and sample code on how to calculate the transaction fees of Moonbeam transactions using Substrate Sidecar API, please check the Calculating Transaction Fees on Moonbeam page.
| Created: December 9, 2021