Sourcify: The future of a Decentralized Etherscan

Learn how to use the new Sourcify infrastructure today

We all love Etherscan. It's a great tool to interact with contracts, read the source codes or just see the status of your transactions. But unfortunately as great as it is, we should not forget that it's a centralized service. The website could be taken down any day. This kind of defeats the purpose of using decentralized infrastructure in the first place.

But there is no reason for why we need to rely only on this centralized service. All functions can be done in a decentralized way. Some already today, some will come in the future.

Let's explore how...

The problems solved by Etherscan

Obviously Etherscan has evolved to something with many different functions by now. But we only want to focus on the main ones here which is also why I believe that a decentralized Etherscan by no means would imply the end of Etherscan itself. Let's get the core functions done in a decentralized way, because those are the security-critical ones. So what is critical here in my opinion?

Source Code Availability
Interaction via ABI
Transaction Status

Problem 1: Source Code Availability

We want a contract to have a verified source code. Without the source code, we can only see the contract bytecode. While there are decompilers (one even added to Etherscan), they are far, far away from having the actual source code. For trustability users need to be able to verify the source code. Until now this is generally done via Etherscan verifications. Most tools support the verification in this way and it has become the de facto standard way.

This is also the biggest problem with Etherscan. If the site gets taken down, all source code information is lost. Then we have thousands of contracts that we need to trust blindly if we want to use them.

Problem 2: Interaction via ABI

This one goes hand in hand with the first problem, but it's a separate issue on its own. Without having the contract's ABI, we cannot interact with it properly. Once again, to some degree it's possible to recover an ABI just from the bytecode, but not always or only partially. This is due to the function signatures being the last 4 bytes of the function name + arguments. So if you don't know the function names and arguments, you can't create a proper ABI.

Problem 3: Transaction Status

People also use Etherscan to look at the current transaction status. It goes as far as MetaMask directly linking to Etherscan for the status. But we are once again relying on a centralized service here. While MetaMask is using Infura for transaction confirmation notifications, even Infura is a centralized service and has been down just three days ago. There are further centralized services like Blocknative for more advanced transaction status tracking.

There is no reason why all of this infrastructure must be designed in a centralized way. We can do better!

Solving Problem 1 and 2: Sourcify - A Decentralized Alternative

Sourcify has been been a recent collective to build alternatives. There's a great FAQ available here. At its heart Sourcify is a decentralized register for contract metadata. You can browse the register here. By providing this register it sets the foundation for a new infrastructure and tooling around it. You can expect the use of Sourcify to become much more widespread in the future.

Let's take a look how this is achieved.

A. What is the Metadata?

First of all, what is the metadata of a contract? The metadata is not new. I went back to the documentation and the first mention of it dates back to v0.4.7 which was back in December 2016. The latest documentation about it can be seen here.

The metadata contains all the information to securely interact with this contract. And this is actually more than Etherscan verification is providing. It contains

compiler settings
source code
ABI
all code documentation like Natspec and inline comments

While you get the first three with Etherscan, you have absolutely no guarantee that the comments and Natspec documentation matches the one used at deployment time. You would think, how could it? It's just a comment and doesn't affect the code after all.

That's true. But as it turns out the metadata is actually appended to a contract's bytecode on deployment and Etherscan just ignores this. This makes sense for a service like Etherscan as the metadata can differ for various reasons and it would be very difficult (even more than it already is) to verify a contract successfully on Etherscan.

So as it turns out, the code comments could be very different (even malicious) and whoever verifies a contract on Etherscan first is chosen as the correct result.

B. Let's use IPFS to store the Metadata

Now that we know how powerful the metadata file is we 'only' need a way to distribute it securely. It's been 4 years since the metadata exists and yet we only now are starting to find a solution. Can someone explain how this took so long?

If you're not familiar with IPFS yet, check out my previous tutorial on it here. In short for now, it's a decentralized storage service. So it's perfect for our use case of storing the metadata along with the source code files.

Let's look at one example:

Under https://repo.sourcify.dev/contracts/full_match/4/0xA67e9490da7899Dd400973190Ba952557AbE92eF/ you can find the published metadata file and source code. The metadata file contains also the IPFS hash for the source files. You could browse those directly, for example in our example under https://ipfs.io/ipfs/QmdFKz1mJcE8MCfeRnkmEaJ2oF62LAmjxbbrLqzxvAXBTb would be the main source file.

This has been the intended use case for the metadata and is already explained in the docs here. Now there is one major issue with IPFS and that is keeping those files online. Sourcify will ensure this using IPFS pinning.

C. How to publish your contracts via Sourcify

Conceptually Sourcify has two ways to verify a contract.

Upload the metadata file for a deployed contract.
Publish the metadata to IPFS before deployment. A monitor service automatically checks if any new deployments match an existing metadata file.

(I) Manually

Your first option is to just manually upload the metadata file under https://sourcify.dev/. A tool like this is required to be able to add metadata files for already deployed contracts.

Note that you don't have to use the UI, but you can also use the Remix plugin (or CLI which will be released in the future). For details on this see the Remix section below.

(II) Remix

Remix has wide support for metadata uploads.

You can activate the Sourcify plugin. The plugin section is the icon on the bottom. Once activated you will see a screen as shown on the right. The plugin allows you to verify and fetch contract metadata. In particular the fetch will be useful in the future. Want to interact with a contract? Just fetch the data here in Remix and you will get the full source code including all comments and ABI.
We can also use the second way of publishing the metadata pre-deployment. You can either choose to publish to IPFS in the compile tab:

Or most conveniently, just activate the option on deployments.

(III) Truffle

In Truffle the metadata is already created for you. There is no direct support for publishing on IPFS yet, but there's a simple example available here.

And on the right is the example code for uploading metadata files to IPFS inside a Truffle project. As you can see, the metadata can be found in ./build/contracts/MyContractName.json and then under .metadata.

Once we have this data, we can use IPFS (in this case connected to an Infura node) and upload the metadata and source files. In this script we automatically go through all compiled Truffle contracts and upload all relevant metadata and source files.

Now all you have to do is make sure to run this command before the deployment. This will ensure the Sourcify monitor will automatically match the new deployment when it's written to the chain.

const IPFS = require('ipfs-http-client');
const shell = require('shelljs');
const path = require('path');

async function uploadToIPFS(){
  const host = 'ipfs.infura.io';
  const ipfs = IPFS({ host, port: '5001', protocol: 'https' });
  const artifactPaths = shell.ls('./build/contracts/*.json');

  for (let _path of artifactPaths){
    const artifact = require(path.join(process.cwd(), _path));

    console.log(artifact.contractName);

    for await (const res of ipfs.add(artifact.metadata)) {
      console.log(`metadata: ${res.path}`);
    }

    for await (const res of ipfs.add(artifact.source)) {
      console.log(`source:   ${res.path}`);
    }
  }
}

(IV) Hardhat (ex Buidler)

Likewise in Hardhat there is no direct Sourcify support yet. But with the hardhat-deploy plugin, the metadata is already created for you. Unless you change the configuration, you will find the metadata under ./deployments/{networkName}/MyContract.json and then again under .metadata. You can use an upload script to IPFS as shown above.

Or you can wait for better upload support as the hardhat-deploy plugin has this already on their TODO list here.

Going beyond Etherscan

What we've discussed so far is the most pressing issue and likely the first becoming used more.

But we can even go one or actually two steps further.

Integration into wallets

The metadata adds further support for Natspec as explained here. This is good news as Natspec is already widely used for Solidity contracts. Usually as pure developer documentation, the Natspec could now also be used as user documentation.

Remember the last time you confirmed a transaction?

Did it look anything like shown on the right?

Yeah, we all now exactly what this transaction will be doing. This is where the Natspec can come in. Instead of this dialogue, a user might get a description such as

'Swap 100 ETH for at least 45,000 DAI if the trade is executed within the next 100 seconds.'

Now this is quite a bit more readable than the hex data, isn't it?

At the time of this writing, there has been a repo started for MetaMask support for Sourcify available at https://github.com/sourcifyeth/sourcify-metamask-snap. It's currently still empty, but you can expect big things to come.

Transaction Status and Mempools

Now this one is certainly something much further away in the future, yet critically important for a long-term sustainable ecosystem. The dark forest is still a reality. And while Blocknative just raised '5 million to illuminate' it, this is once again a centralized service. Don't get me wrong, Blocknative and other services will be a great solution for the short-term, but they won't be what the ecosystem needs in the long run, at least not as a fundamental tech stack.

A few of the issues to solve here are

How can you reliably find out the transaction status?
How can you properly ensure a transaction is actually mined?
Is there a way to prevent front-running?

Unfortunately Sourcify won't help with this issue. We will have to wait for something else. But I feel optimistic that a solution process can at least be started within the next few years.

Decentralization: One step at a time

As you might have noticed, a lot of the architecture is still very much work in progress. I also feel like we will need a few websites that allow for easy interaction with contracts that have published metadata. Remix is a good start, but make it even simpler. Kind of like the Etherscan Dapp pages, but utilizing Sourcify.

This is a great hackathon project in case you need are in need for a project idea. ;)

If you have not seen the original posts about Sourcify by the main guy behind Solidity, Christian Reitwiessner aka chriseth, check them out here:

Markus Waas

Solidity Developer