The New Decentralized The Graph Network
What are the new features and how to use it
Quite some time has passed since my last post about The Graph. If you don't know what it is and why it's useful, go and read the post. It's still relevant and explains in detail why it's needed and how to use it with the centralized hosted service.
But the tl;dr is: Events on a blockchain are a very efficient way to add data without having to store it in every node which is very expensive. This is done through the use of a bloom filter and a client is able to parse blocks and transactions to quickly find data he is looking for. But this still requires parsing through a lot of blocks, so one alternative is having a server index this data and store it in a database. Put GraphQL on top as a very convenient query language and you have The Graph, but as a centralized service.
In conclusion, The Graph allows querying data from a blockchain in a much more efficient way. This is extremely important when building a front-end and showing data about what happened in the blockchain without having to store the data directly in a smart contract.
Since then The Graph has started a new decentralized network and also added more features. The hosted service will end in Q1 of 2023, so now it's time to learn how the decentralized network works, how to use it and what new features there are for you as a developer.

The Transition Towards Decentralization
Originally The Graph only had a centralized hosted service, but that of course is not what we want in the long term. After all, what's the point of a Dapp if you fully rely on a centralized server for querying data. Actually it's still better than a fully centralized infrastructure, but we can still do better!
To get around the issue, The Graph has its own decentralized network with its own GRT ERC-20 token.
1. Protocol Roles

Consumers: Consumers are the ones sending queries to indexers and paying for this service. It could be end users directly, e.g. in the context of using a Dapp, or middleware services.
Indexers. Indexers are the ones actually providing the service of running servers, indexing events and storing them in a database. As reward they will receive fees from the consumers.
Curators: Curators will identify subgraphs that are worthy to be indexed and put up their own GRT for specific subgraphs. They will earn a share of the query fees based on a bonding curve. This might be developers of a subgraph that want to get their subgraph indexed.
Delegators. Delegators can stake GRT with indexers to earn a share of their query fees.
Consumers can freely choose which indexer to use based on things like up-time and price. And of course a Dapp could even use multiple indexers for the highest security.
2. GRT Token
The GRT token itself is used for staking:
- Indexers staking GRT serves the purpose of sybil resistance as well as allowing to slash them for bad behavior. Indexers are expected to stake a proportion of total staked GRT equal to the proportion of work they have performed for the network. This is not directly enforced, but rather comes from the design of collecting transaction fees and rebating those to participants as a function of their proportional stake and fees, inspired by 0x design.
- Delegators will loan their GRT to indexers and receive a share of the query fees in return. They cannot get their GRT slashed due to indexer misbehavior and there is a limit how much GRT each indexer can receive through delegation.

3. Bootstrapping New Subgraphs
To help in the bootstrapping for new subgraphs that don't have demand yet, GRT staking is inflationary and newly minted tokens are given to indexers which index very low query demand subgraphs. To make sure indexers are actually doing the work of indexing these low-demand subgraphs, there is an extra mechanism called Proof of Indexing (POI). A POI is signature over the subgraph state hash and is needed to receive indexer rewards. POIs are accepted optimistically, but could later be used to slash them. In the first version of the network, an arbitrator set through governance will decide these disputes.
Subgraph configurations (called manifest) are typically uploaded to IPFS. But what can we do when a manifest is simply not available? Then it would be impossible to verify these POIs. For this, there is a Subgraph Availability Oracle. It will look at several prominent IPFS endpoints and if a manifest is unavailable, the subgraph is not available for indexer rewards.
4. Payment Channels
Normal payment channels are a great way to scale payments. When you pay for every query like in The ›Graph, we obviously need something like this. In The Graph there will be an extra layer of security for payment channels: WAVE.

WAVE:
Work. Locked micropayment with a description of the work to be performed.
Attestation. Work + signed attestation which unlocks the micropayment optimistically.
Verification. Verify attestation out-of-channel which may lead to penalties.
Expiration. Locked micropayments can expire.
Query Verification
Now the question is how could you verify The Graph queries for correctness? Initially in The Graph, this is handled through an on-chain dispute resolution process, which is decided through arbitration. Fishermen will look for incorrect queries and submit the attestation to arbitration along with a bond.
In the future, rather than rely on arbitration for settling query disputes, the validity of queries could be guaranteed using cryptographic proofs using techniques such as a polynomial commitment or Merkle trees. Similarly even POIs could be automatically verified using a similar mechanism like Optimistic Rollups. Both combined would completely get rid of the need for the more centralized arbitration role currently in the network.
How to deploy to The Decentralized Graph
The Subgraph Studio is the first step towards The Decentralized Graph, but it still contains centralized components. In its current design, the payments are handled by The Graph team directly and you create an API key to query data. Later once verifiable queries are implemented, the users would pay directly.
If you have a subgraph already on the hosted service, take a look at the migration guide.
- Go to https://thegraph.com/studio/ and click create subgraph.
$ npm install -g @graphprotocol/graph-cli
- Inside an existing project deploy a contract, e.g. $
npx hardhat run scripts/deploy.ts --network rinkeby
$ graph init --studio <created-subgraph-name>
- If the Etherscan contract is not verified, you may also pass the ABI file directly, e.g.
artifacts/contracts/Game.sol/Game.json
- Save the correct schema.graphql and schema.ts.
$ graph auth --studio <my-subgraph-id>
(see Subgraph Studio UI)$ graph codegen && graph build
$ graph deploy --studio <my-subgraph-id>
Now you can click 'Publish' in the Subgraph Studio. To get Rinkeby testnet GRT, head over to The Graph Discord #roles, click 'T' and then go to #testnet-faucet and type in !grt <my-eth-address>. And you can immediately start curating the subgraph. Note that at the time of this writing actual queries on Rinkeby via the decentralized network are not yet supported, but they are for mainnet.
- You can access the testnet explorer at https://testnet.thegraph.com. Here you can also find a subgraph and test the curation for other subgraphs.
- Indexers (and curators + delegators) can be found via https://testnet.thegraph.com/explorer/participants/indexers and you can also test the delegation feature here.
- And the mainnet versions are https://thegraph.com/explorer and https://thegraph.com/participants/indexers.
New Features for Developers
- New AssemblyScript version: The Graph has updated the AssemblyScript used for writing the mappings. If you are used to older versions, you can follow the migration guide.
- Debug Forking: A new feature called debug forking allows to clone a deployed subgraph and change its mapping for a specific block. So if you deploy a new subgraph and it fails, now you can debug it locally from that block rather then re-syncing from scratch which takes a lot of time.
- New Blockchains: While not yet available on the decentralized network, The Graph has added indexing support for completely new blockchains, most notably Cosmos, NEAR and Arweave.
- Subgraph Mapping Unit-testing: A new unit-testing feature called matchstick allows testing subgraph mappings, see below for details.
Unit-testing Subgraph Mappings
To get started with unit-testing in an existing Subgraph:
- Install matchstick:
$ yarn add --dev matchstick-as
- Install Postgresql:
$ brew install postgresql
(Mac) or$ sudo apt install postgresql
(Linux)- or for Windows see docs
- Create testing file inside
/tests
folder. - Run the tests:
$ graph test
On the right you can see an example unit-test for our Bet mapping from our previous The Graph tutorial.
- You can first create mocked events using
newMockEvent
.
2. Then modify the mocked event however you like.
3. Then call you handle event function from the mapping.
4. Assert the new state is as expected.
5. Clean the store afterwards.
import {
assert,
describe,
clearStore,
test,
newMockEvent,
} from "matchstick-as/assembly/index";
import { BetPlaced } from "../generated/Game/Game";
import { Address, BigInt, ethereum } from "@graphprotocol/graph-ts";
import { handleBetPlaced } from "../src/mapping";
function createBetPlacedEvent(
player: string,
value: BigInt,
hasWon: boolean
): BetPlaced {
const mockEvent = newMockEvent();
const BetPlacedEvent = new BetPlaced(
mockEvent.address,
mockEvent.logIndex,
mockEvent.transactionLogIndex,
mockEvent.logType,
mockEvent.block,
mockEvent.transaction,
mockEvent.parameters,
null
);
BetPlacedEvent.parameters = new Array();
const playerParam = new ethereum.EventParam(
"player",
ethereum.Value.fromAddress(Address.fromString(player))
);
const valueParam = new ethereum.EventParam(
"value",
ethereum.Value.fromUnsignedBigInt(value)
);
const hasWonParam = new ethereum.EventParam(
"hasWon",
ethereum.Value.fromBoolean(hasWon)
);
BetPlacedEvent.parameters.push(playerParam);
BetPlacedEvent.parameters.push(valueParam);
BetPlacedEvent.parameters.push(hasWonParam);
BetPlacedEvent.transaction.from = Address.fromString(player);
return BetPlacedEvent;
}
describe("handleBetPlaced()", () => {
test("Should create a new Bet entity", () => {
const player = "0x7c812f921954680af410d86ab3856f8d6565fc69";
const hasWon = true;
const mockedBetPlacedEvent = createBetPlacedEvent(
player,
BigInt.fromI32(100),
hasWon
);
handleBetPlaced(mockedBetPlacedEvent);
const betId =
mockedBetPlacedEvent.transaction.hash.toHex() +
"-" +
mockedBetPlacedEvent.logIndex.toString();
// fieldEquals(entityType: string, id: string, fieldName: string, expectedVal: string)
assert.fieldEquals("Bet", betId, "id", betId);
assert.fieldEquals("Bet", betId, "player", player);
assert.fieldEquals("Bet", betId, "playerHasWon", "true");
assert.fieldEquals(
"Bet",
betId,
"time",
mockedBetPlacedEvent.block.timestamp.toString()
);
clearStore();
});
});
You can find the full example at https://github.com/soliditylabs/the-graph-studio-example.
What do you think of the plans of The Graph? Have you used the hosted service or even the decentralized network already?
Solidity Developer
More great blog posts from Markus Waas
How to use ChatGPT with Solidity
Using the Solidity Scholar and other GPT tips
How to integrate Uniswap 4 and create custom hooks
Let's dive into Uniswap v4's new features and integration
How to integrate Wormhole in your smart contracts
Entering a New Era of Blockchain Interoperability
Solidity Deep Dive: New Opcode 'Prevrandao'
All you need to know about the latest opcode addition
How Ethereum scales with Arbitrum Nitro and how to use it
A blockchain on a blockchain deep dive
The Ultimate Merkle Tree Guide in Solidity
Everything you need to know about Merkle trees and their future
zkSync Guide - The future of Ethereum scaling
How the zero-knowledge tech works and how to use it
Exploring the Openzeppelin CrossChain Functionality
What is the new CrossChain support and how can you use it.
Deploying Solidity Contracts in Hedera
What is Hedera and how can you use it.
Writing ERC-20 Tests in Solidity with Foundry
Blazing fast tests, no more BigNumber.js, only Solidity
ERC-4626: Extending ERC-20 for Interest Management
How the newly finalized standard works and can help you with Defi
Advancing the NFT standard: ERC721-Permit
And how to avoid the two step approve + transferFrom with ERC721-Permit (EIP-4494)
Moonbeam: The EVM of Polkadot
Deploying and onboarding users to Moonbeam or Moonriver
Advanced MultiSwap: How to better arbitrage with Solidity
Making multiple swaps across different decentralized exchanges in a single transaction
Deploying Solidity Smart Contracts to Solana
What is Solana and how can you deploy Solidity smart contracts to it?
Smock 2: The powerful mocking tool for Hardhat
Features of smock v2 and how to use them with examples
How to deploy on Evmos: The first EVM chain on Cosmos
Deploying and onboarding users to Evmos
EIP-2535: A standard for organizing and upgrading a modular smart contract system.
Multi-Facet Proxies for full control over your upgrades
MultiSwap: How to arbitrage with Solidity
Making multiple swaps across different decentralized exchanges in a single transaction
The latest tech for scaling your contracts: Optimism
How the blockchain on a blockchain works and how to use it
Ultimate Performance: The Aurora Layer2 Network
Deploying and onboarding users to the Aurora Network powered by NEAR Protocol
What is ecrecover in Solidity?
A dive into the waters of signatures for smart contracts
How to use Binance Smart Chain in your Dapp
Deploying and onboarding users to the Binance Smart Chain (BSC)
Using the new Uniswap v3 in your contracts
What's new in Uniswap v3 and how to integrate Uniswap v3
What's coming in the London Hardfork?
Looking at all the details of the upcoming fork
Welcome to the Matrix of blockchain
How to get alerted *before* getting hacked and prevent it
The Ultimate Ethereum Mainnet Deployment Guide
All you need to know to deploy to the Ethereum mainnet
SushiSwap Explained!
Looking at the implementation details of SushiSwap
Solidity Fast Track 2: Continue Learning Solidity Fast
Continuing to learn Solidity fast with the advanced basics
What's coming in the Berlin Hardfork?
Looking at all the details of the upcoming fork
Using 1inch ChiGas tokens to reduce transaction costs
What are gas tokens and example usage for Uniswap v2
Openzeppelin Contracts v4 in Review
Taking a look at the new Openzeppelin v4 Release
EIP-3156: Creating a standard for Flash Loans
A new standard for flash loans unifying the interface + wrappers for existing ecosystems
Tornado.cash: A story of anonymity and zk-SNARKs
What is Tornado.cash, how to use it and the future
High Stakes Roulette on Ethereum
Learn by Example: Building a secure High Stakes Roulette
How to implement generalized meta transactions
We'll explore a powerful design for meta transactions based on 0x
Utilizing Bitmaps to dramatically save on Gas
A simple pattern which can save you a lot of money
Using the new Uniswap v2 as oracle in your contracts
How does the Uniswap v2 oracle function and how to integrate with it
Smock: The powerful mocking tool for Hardhat
Features of smock and how to use them with examples
How to build and use ERC-721 tokens in 2021
An intro for devs to the uniquely identifying token standard and its future
Trustless token management with Set Protocol
How to integrate token sets in your contracts
Exploring the new Solidity 0.8 Release
And how to upgrade your contracts to Solidity 0.8
How to build and use ERC-1155 tokens
An intro to the new standard for having many tokens in one
Leveraging the power of Bitcoins with RSK
Learn how RSK works and how to deploy your smart contracts to it
Solidity Fast Track: Learn Solidity Fast
'Learn X in Y minutes' this time with X = Solidity 0.7 and Y = 20
Sourcify: The future of a Decentralized Etherscan
Learn how to use the new Sourcify infrastructure today
Integrating the 0x API into your contracts
How to automatically get the best prices via 0x
How to build and use ERC-777 tokens
An intro to the new upgraded standard for ERC-20 tokens
COMP Governance Explained
How Compound's Decentralized Governance is working under the hood
How to prevent stuck tokens in contracts
And other use cases for the popular EIP-165
Understanding the World of Automated Smart Contract Analyzers
What are the best tools today and how can you use them?
A Long Way To Go: On Gasless Tokens and ERC20-Permit
And how to avoid the two step approve + transferFrom with ERC20-Permit (EIP-2612)!
Smart Contract Testing with Waffle 3
What are the features of Waffle and how to use them.
How to use xDai in your Dapp
Deploying and onboarding users to xDai to avoid the high gas costs
Stack Too Deep
Three words of horror
Integrating the new Chainlink contracts
How to use the new price feeder oracles
Adding Typescript to Truffle and Buidler
How to use TypeChain to utilize the powers of Typescript in your project
Integrating Balancer in your contracts
What is Balancer and how to use it
Navigating the pitfalls of securely interacting with ERC20 tokens
Figuring out how to securely interact might be harder than you think
Why you should automatically generate interests from user funds
How to integrate Aave and similar systems in your contracts
How to use Polygon (Matic) in your Dapp
Deploying and onboarding users to Polygon to avoid the high gas costs
Migrating from Truffle to Buidler
And why you should probably keep both.
Contract factories and clones
How to deploy contracts within contracts as easily and gas-efficient as possible
How to use IPFS in your Dapp?
Using the interplanetary file system in your frontend and contracts
Downsizing contracts to fight the contract size limit
What can you do to prevent your contracts from getting too large?
Using EXTCODEHASH to secure your systems
How to safely integrate anyone's smart contract
Using the new Uniswap v2 in your contracts
What's new in Uniswap v2 and how to integrate Uniswap v2
Solidity and Truffle Continuous Integration Setup
How to setup Travis or Circle CI for Truffle testing along with useful plugins.
Upcoming Devcon 2021 and other events
The Ethereum Foundation just announced the next Devcon in 2021 in Colombia
The Year of the 20: Creating an ERC20 in 2020
How to use the latest and best tools to create an ERC-20 token contract
How to get a Solidity developer job?
There are many ways to get a Solidity job and it might be easier than you think!
Design Pattern Solidity: Mock contracts for testing
Why you should make fun of your contracts
Kickstart your Dapp frontend development with create-eth-app
An overview on how to use the app and its features
The big picture of Solidity and Blockchain development in 2020
Overview of the most important technologies, services and tools that you need to know
Design Pattern Solidity: Free up unused storage
Why you should clean up after yourself
How to setup Solidity Developer Environment on Windows
What you need to know about developing on Windows
Avoiding out of gas for Truffle tests
How you do not have to worry about gas in tests anymore
Design Pattern Solidity: Stages
How you can design stages in your contract
Web3 1.2.5: Revert reason strings
How to use the new feature
Gaining back control of the internet
How Ocelot is decentralizing cloud computing
Devcon 5 - Review
Impressions from the conference
Devcon 5 - Information, Events, Links, Telegram
What you need to know
Design Pattern Solidity: Off-chain beats on-chain
Why you should do as much as possible off-chain
Design Pattern Solidity: Initialize Contract after Deployment
How to use the Initializable pattern
Consensys Blockchain Jobs Report
What the current blockchain job market looks like
Provable — Randomness Oracle
How the Oraclize random number generator works
Solidity Design Patterns: Multiply before Dividing
Why the correct order matters!
Devcon 5 Applications closing in one week
Devcon 5 Applications closing
Randomness and the Blockchain
How to achieve secure randomness for Solidity smart contracts?