The big picture of Solidity and Blockchain development in 2020

Overview of the most important technologies, services and tools that you need to know

Now, I do not know about you, but I remember when I first started with Solidity development being very confused by all the tools and services and how they work in connection with one another. If you are like me, this overview will help you understand the big picture of Solidity development. As I run through the various aspects of Solidity development, some basic web and blockchain knowledge will be required, but nothing else.

I will list only the most important tools, so if you are just getting into it or you have been in it for a while and you are still wondering how some technologies work together, this will be for you! Also check out my tutorial for getting started on Windows.

Technologies

If you are a new developer or not familiar at all with web development, you might be confused how Solidity works together in a Dapp (decentralized application). Dapp is just a fancy word for an application making use of decentralized technology, in our case Ethereum.
Solidity Overview
Structure of a typical application for Ethereum

Backend

The backend is the heart of your Dapp. In a true decentralized Dapp, it will consist only of your Solidity smart contracts. Then it basically acts as the server.

Solidity

Technically Solidity or some other EVM language is the only real technology that is required to write a Dapp for Ethereum. You write your logic in Solidity, deploy the contract on the Ethereum blockchain and you are done. Easy, fast... and super inconvenient for users. And that is why we have all the other parts to it. Interacting with a smart contract can be a pain, however a good Dapp can mitigate this.

Server

Even though you have your smart contracts as backend, often times a Dapp will still have an additional traditional server running. Not always, but probably more often than you think. There are many reasons for running an additional backend server and it will depend on the context. One reason might just be user data management. In general, if you can avoid having a backend server, go for it. Not only is it easier for you to manage, but it is also the truly decentralized solution. On the downside, it might be quite expensive storing all data in a smart contract and you might require features only available on a traditional backend server.

Frontend

The frontend is the part of your application that actually runs on the user's side, typically a web browser. It will allow the user to interact with your smart contracts in a convenient way.

HTML and CSS

HTML stands for Hypertext Markup Language. It creates the structure of a website, while CSS stands for Cascading Stylesheets. It lets you make a website look nicer. If you are not familiar with the concepts, check out this explanation.

Javascript

The code for writing the part of the software that will run as the frontend is written in JavaScript. That language can also be used in the backend with node.js. And it is also the most used language for Solidity development tools, see below for details.

Web3

Web3 is a library that you can use in JavaScript that allows you to communicate with your deployed smart contracts. Generally used in the frontend, but technically you can use it on the backend as well. Alternatively, you could use ethers.js. In the frontend the users sign a new transaction with a wallet like MetaMask (see below), while in the backend you will need some kind of private key management and then sign it via ethereumjs-tx.

Tools

Node.js

You will need node.js as a tool even if you do not need a node.js backend server. It will allow you to run JavaScript code without a browser and a lot of tools for developing Solidity code are written in JavaScript.

Package Manager

Npm or yarn will be abolutely essential for Solidity development. They allow you to manage dependencies, install/upgrade/remove other packages (libraries) that you need. You can search on https://www.npmjs.com/ for available packages. Npm comes already with the node.js installation. In 2020 I would say unless you have a specific functionality that you need from yarn, just go along and use npm.

React & create-eth-app

React is a JavaScript library for building user interfaces. It is by no means the only one available, but the most used one in the Ethereum space. It makes creating interactive UI's very easy. You can design simple views for each state in your application, and React will efficiently update just the right parts when your data changes. On top you can use the great tool create-eth-app to get you started up with a working React application. Along with create-eth-app comes a Subgraph integration which is extremely useful for querying a lot of data from your smart contracts.

Truffle

The most important tool for compiling, managing contract deployments and running automated tests is definitely Truffle. It comes with various boxes to get you started which are alternatives to the create-eth-app, but include a truffle integration. Generally, there are three ways you can go about it:

  1. You can have separate repositories for your Solidity backend and JavaScript frontend. For the backend you would use Truffle and for the frontend you would start with create-eth-app.
  2. You can have both parts in one repository for convenience and for example start with the Truffle React box.
  3. You can focus only on the backend part, either because you only want to learn the Solidity part or because you actually don't need a frontend.

Ganache

Also part of the Truffle suite is Ganache. It will enable you to run a local blockchain on your computer for fast tests. As a rule of thumb, if you want to test your Dapp manually, use the GUI application. If you want to run automated tests with Truffle, use the ganache-cli version and set it up to run automatically before tests.
Ganache UI
MetaMask UI

MetaMask

MetaMask is a wallet for your browser. There are some alternatives especially for mobile wallet applications. However, for desktops MetaMask is still the number 1 tool. It will enable users to store their Ethereum accounts including private keys securely inside the browser. When your frontend applications requires interaction with a smart contract, it can request MetaMask to sign a transaction and MetaMask will forward the request to the user to confirm. After the user confirmation, a transaction is broadcasted using the Infura service (see below for Infura). If you want to support more than just MetaMask, you can integrate Web3Modal.

Etherscan

Etherscan is a website for tracking Ethereum transactions. It is very useful while testing to verify your contracts. Afterwards you will have an easy way to communicate with your contracts and have a nice overview of what is going on. You can also make use of the Dapp Interfaces to avoid having to build your own UI. Simply submit the contract information here.

Remix

Remix is a browser based IDE and can be useful for
  • just trying some code quickly using the JavaScript VM
  • calling, deploying and testing contracts in a testnet (or even mainnet)
  • debugging transactions using the integrated debugger (works best in the JavaScript VM)

Buidler

Buidler is a very specific tool, not essential and thus almost did not make it in this list. However, having full stack traces and a console.log inside your contracts is just too useful to not be mentioned here.

Remix UI

OpenZeppelin

Besides security audits, OpenZeppelin offers reference contracts, an SDK and starter kits
  • Contracts: I highly recommend taking a look at the contracts. They are of high quality and you will find commonly required implementations of things like ERC-20 tokens and SafeMath. They even get occasional audits for all contracts and they are generally conservative putting security over convenience. The next upcoming version 3 is almost ready and will support the latest Solidity v0.6.
  • SDK: Useful if you want to have and manage upgradable contracts. To kickstart your development, you can use a starter kit which is similar to above mentioned Truffle boxes or create-eth-app, but offers an additional SDK integration.

Services

Infura

Infura is a service that connects your application with a public blockchain (mainnet or testnet). In the frontend MetaMask is using it automatically. Otherwise, you will have to connect Web3 or Truffle with it manually. After creating an account on the Infura website and adding a project, you can checkout the API keys to connect Web3 and Truffle on their dashboard. Alternatively, you can run your own node and connect to that one instead, but this can be costly, time-consuming and you will have to worry about availability of the node yourself. However, that would be the preferred solution for true decentralization.

Alternatives? Yes, one of them is QuikNode. They offer professional and reliable access with cheap prices. More providers also mean more decentralization, so let's not all use Infura.

Testnets

You can run smart contracts locally with Ganache or instead you can use a public blockchain. If you want to use the main blockchain, it would be quite expensive to do your testing here. Any transaction will cost you gas which you have to pay for with real ETH. That is why there are several public testnets available:

The Future

While the mentioned tools are still part of the most common setups, some people are going towards other ways already. If you want to try the new stack of Buidler + Ethers + Waffle + Typescript, checkout this introduction.

Bonus: Companies

  • Consensys is probably the biggest blockchain company out there. Founded by Joseph Lubin, the company heavily invested and accelerated the Ethereum blockchain.
  • Ethereum Foundation is a a Swiss non-profit foundation that manages funds to accelerate the Ethereum space.
  • Web3 Foundation is a similar non-profit foundation, but with more links to Parity.
  • Parity is a blockchain company founded by Gavin Wood, the inventor of Solidity. It has since built one of the popular Ethereum clients and is now building the next generation of blockchains with Substrate and Polkadot.

Bonus 2: Events

For a list of upcoming interesting events in the Ethereum space, you can check out my events page. Unfortunately, due to the current COVID-19 situation, they are quite limited and mostly virtual. Let us hope this will not last much longer.

  • Devcon: The biggest and best conference for the Ethereum space. Several days, large amounts of great side events and full of creative and interesting people.
  • Edcon: Basically like a smaller Devcon. The last one at the beginning of April this year had to be cancelled.
  • ETH.global: Hosting several big hackathons all over the world, or these days a virtual one, see the upcoming virtual HackMoney event.

Bonus 3: Learning Resources

Bonus 4: Developer Communities

Bonus 5: Jobs

Jobs are often posted on AngelList, LinkedIn, indeed, Crypto Jobs List, Blocktribe and Gitcoin, but you can also always look the companies websites directly or even Reddit. The job market is growing and well-paid.

What else?

There is still more to be said about scaling technologies on layer 2, mobile wallets, DAO's, DAI and moremuch more.  My aim here was to provide you with the components that comprise the big picture of Solidity development. In following posts I will cover many more tools to help you become a master Solidity developer. Stay tuned!

Let me know what the most important tools are that you use! If you think there are important aspects missing here, please let me know.


Markus Waas

Solidity Developer

More great blog posts from Markus Waas

  • Matic Logo

    How to use Matic in your Dapp

    Deploying and onboarding users to Matic to avoid the high gas costs

    Gas costs are exploding again, ETH2.0 is still too far away and people are now looking at layer 2 solutions. Here's a good overview of existing layer 2 projects: https://github.com/Awesome-Layer-2/awesome-layer-2 . Today we will take a closer look at Matic as a solution for your Dapp. Why Matic...

  • Migrating from Truffle to Buidler

    And why you should probably keep both.

    Why Buidler? Proper debugging is a pain with Truffle. Events are way too difficult to use as logging and they don't even work for reverted transactions (when you would need them most). Buidler gives you a console.log for your contracts which is a game changer. And you'll also get stack traces...

  • Factory

    Contract factories and clones

    How to deploy contracts within contracts as easily and gas-efficient as possible

    The factory design pattern is a pretty common pattern used in programming. The idea is simple, instead of creating objects directly, you have an object (the factory) that creates objects for you. In the case of Solidity, an object is a smart contract and so a factory will deploy new contracts for...

  • IPFS logo

    How to use IPFS in your Dapp?

    Using the interplanetary file system in your frontend and contracts

    You may have heard about IPFS before, the Interplanetary File System. The concept has existed for quite some time now, but with IPFS you'll get a more reliable data storage, thanks to their internal use of blockchain technology. Filecoin is a new system that is incentivizing storage for IPFS...

  • tiny-kitten

    Downsizing contracts to fight the contract size limit

    What can you do to prevent your contracts from getting too large?

    Why is there a limit? On November 22, 2016 the Spurious Dragon hard-fork introduced EIP-170 which added a smart contract size limit of 24.576 kb. For you as a Solidity developer this means when you add more and more functionality to your contract, at some point you will reach the limit and when...

  • EXTCODEHASH

    Using EXTCODEHASH to secure your systems

    How to safely integrate anyone's smart contract

    What is the EXTCODEHASH? The EVM opcode EXTCODEHASH was added on February 28, 2019 . Not only does it help to reduce external function calls for compiled Solidity contracts, it also adds additional functionality. It gives you the hash of the code from an address. Since only contract addresses...

  • Uniswap

    Using the new Uniswap v2 in your contracts

    What's new in Uniswap v2 and how to integrate Uniswap v2

    What is UniSwap? If you're not familiar with Uniswap yet, it's a fully decentralized protocol for automated liquidity provision on Ethereum. An easier-to-understand description would be that it's a decentralized exchange (DEX) relying on external liquidity providers that can add tokens to smart...

  • Continuous Integration

    Solidity and Truffle Continuous Integration Setup

    How to setup Travis or Circle CI for Truffle testing along with useful plugins.

    Continuous integration (CI) with Truffle is great for developing once you have a basic set of tests implemented. It allows you to run very long tests, ensure all tests pass before merging a pull request and to keep track of various statistics using additional tools. We will use the Truffle...

  • Devcon 6

    Upcoming Devcon 2021 and other events

    The Ethereum Foundation just announced the next Devcon in 2021 in Colombia

    Biggest virtual hackathon almost finished First of all, the current HackMoney event has come to an end and it has been a massive success. One can only imagine what kind of cool projects people have built in a 30 days hackathon. All final projects can be seen at:...

  • ERC-2020

    The Year of the 20: Creating an ERC20 in 2020

    How to use the latest and best tools to create an ERC-20 token contract

    You know what an ERC-20 is, you probably have created your own versions of it several times (if not, have a look at: ERC-20 ). But how would you start in 2020 using the latest tools? Let's create a new ERC-2020 token contract with some basic functionality which focuses on simplicity and latest...

  • hiring

    How to get a Solidity developer job?

    There are many ways to get a Solidity job and it might be easier than you think!

    You have mastered the basics of Solidity, created your first few useful projects and now want to get your hands on some real-world projects. Getting a Solidity developer job might be easier than you think. There are generally plenty of options to choose from and often times not a lot of...

  • People making fun

    Design Pattern Solidity: Mock contracts for testing

    Why you should make fun of your contracts

    Mock objects are a common design pattern in object-oriented programming. Coming from the old French word 'mocquer' with the meaning of 'making fun of', it evolved to 'imitating something real' which is actually what we are doing in programming. Please only make fun of your smart contracts if you...

  • React and Ethereum

    Kickstart your Dapp frontend development with create-eth-app

    An overview on how to use the app and its features

    Last time we looked at the big picture of Solidity and already mentioned the create-eth-app . Now you will find out how to use it, what features are integrated and additional ideas on how to expand on it. Started by Paul Razvan Berg, the founder of sablier , this app will kickstart your frontend...

  • Design Pattern Solidity: Free up unused storage

    Why you should clean up after yourself

    You may or may not be used to a garbage collectors in your previous programming language. There is no such thing in Solidity and even if there was a similar concept, you would still be better off managing state data yourself. Only you as a programmer can know exactly which data will not be used...

  • How to setup Solidity Developer Environment on Windows

    What you need to know about developing on Windows

    Using Windows for development, especially for Solidity development, can be a pain sometimes, but it does not have to be. Once you have configured your environment properly, it can actually be extremely efficient and Windows is a very, very stable OS, so your overall experience can be amazing. The...

  • Avoiding out of gas for Truffle tests

    How you do not have to worry about gas in tests anymore

    You have probably seen this error message a lot of times: Error: VM Exception while processing transaction: out of gas Disclaimer : Unfortunately, this does not always actually mean what it is saying when using Truffle , especially for older versions. It can occur for various reasons and might be...

  • Design Pattern Solidity: Stages

    How you can design stages in your contract

    Closely related to the concept of finite-state machines, this pattern will help you restrict functions in your contract. You will find a lot of situations where it might be useful. Any time a contract should allow function calls only in certain stages. Let's look at an example: contract Pool {...

  • Web3 1.2.5: Revert reason strings

    How to use the new feature

    A new Web3 version was just released and it comes with a new feature that should make your life easier. With the latest version 1.2.5 , you can now see the the revert reason if you use the new handleRevert option. You can activate it easily by using web3.eth.handleRevert = true . Now when you use...

  • Gaining back control of the internet

    How Ocelot is decentralizing cloud computing

    I recently came across an ambitious company that will completely redefine the way we are using the internet. Or rather, the way we are using its underlying infrastructure which ultimately is the internet. While looking at their offering, I also learned how to get anonymous cloud machines, you...

  • Devcon 5 - Review

    Impressions from the conference

    I had a lot to catch up on after Devcon. Also things didn't go quite as planned, so please excuse my delayed review! This year's Devcon was certainly stormy with a big typhoon warning already on day 1. Luckily (for us, not the people in Tokyo), it went right past Osaka. Nevertheless, a lot of...

  • Devcon 5 - Information, Events, Links, Telegram

    What you need to know

    Devcon 5 is coming up soon and there are already lots of events available, information about Osaka and more. Here is a short overview: Events Events Calendar Events Google Docs Events Kickback Most events are in all three, but if you really want to see all, you will have to look at all three...

  • Design Pattern Solidity: Off-chain beats on-chain

    Why you should do as much as possible off-chain

    As you might have realized, Ethereum transactions are anything but cheap. In particular, if you are computing complex things or storing a lot of data. That means sometimes we cannot put all logic inside Solidity. Instead, we can utilize off-chain computations to help us. A very simple example...

  • Design Pattern Solidity: Initialize Contract after Deployment

    How to use the Initializable pattern

    There are a few reasons why you might want to initialize a contract after deployment and not directly by passing constructor arguments. But first let's look at an example: contract MyCrowdsale { uint256 rate; function initialize(uint256 _rate) public { rate = _rate; } } What's the advantage over...

  • Consensys Blockchain Jobs Report

    What the current blockchain job market looks like

    Consensys published their blockchain jobs report which you can checkout in their Blockchain Developer Job Kit . The most interesting aspects are Blockchain developer jobs have been growing at a rate of 33x of the previous year according to LinkedIns jobs report Typical salary is about...

  • Provable — Randomness Oracle

    How the Oraclize random number generator works

    One particularly interesting approach by Provable is the usage of a hardware security device, namely the Ledger Nano S. It uses a trusted execution environment to generate random numbers and provides a Provable Connector Contract as interface. How to use the Provable Randomness Oracle? Use the...

  • Solidity Design Patterns: Multiply before Dividing

    Why the correct order matters!

    There has been a lot of progress since the beginning of Ethereum about best practices in Solidity. Unfortunately, I have the feeling that most of the knowledge is within the circle of experienced people and there aren’t that many online resources about it. That is why I would like to start this...

  • Devcon 5 Applications closing in one week

    Devcon 5 Applications closing

    Watch out for the Devcon 5 applications. You only have one week left to apply either as Buidler Student Scholarship Press Devcon is by far the biggest and most impressive Ethereum conference in the world. And it's full of developers! I am especially excited about the cool location this year in...

  • Randomness and the Blockchain

    How to achieve secure randomness for Solidity smart contracts?

    When we talk about randomness and blockchain, these are really two problems: How to generate randomness in smart contracts? How to produce randomness for proof-of-stake (POS) systems? Or more generally, how to produce trusted randomness in public distributed systems? There is some overlap of...