How to enable gzip encoding for ethers.js, web3.js and curl

About Gzip

GNU Gzip is a popular file compression format, for users who often retrieve a huge amount of data from a server, compressing the response with Gzip reduces load time and bandwidth consumption effectively.

Gzip has an outstanding 95% compression rate for JSON objects, which makes it a perfect companion for EVM nodes; because basically, all EVM responses are in JSON format and some can be as big as a few MBs.

How Gzip works with EVM nodes in a nutshell

First of all, you need to check if Gzip is available on your EVM-compatible nodes. Gzip is enabled by default for most Linux machines. In this tutorial, the sample code is developed and tested with a Chainstack endpoint—it is highly recommended for the following reasons:

  • Gzip enabled by default
  • Free 3 million requests.
  • Seamless node deployment.

Simply follow these steps to deploy a node:

  1. Sign up with Chainstack.
  2. Deploy an Ethereum node.
  3. Get the deployed node’s endpoint.

This is how Gzip works in a nutshell:

Figure 1: Gzip interaction process

When a sender sends an RPC request to the node/server, a header is attached. In the header, the sender specifies an attribute “Accept-Encoding: gzip”. When the server receives the request, this header attribute lets it know that the client wants the response to be Gzip encoded.

The server compresses the data and sends it back to the client. The response is attached with “content-encoding: gzip” in its header. This is to let the receiver knows that the data is encoded with Gzip. It needs to be decoded to retrieve the original data.

This is basically how Gzip works. Both client and server need to support this format and they notify each other in the request/response header.

Using Gzip in a browser

If you are developing a web application with ethers.js or web3.js and you are wondering how to enable Gzip at the frontend, there’s nothing to configure actually.
Popular browsers like Chrome and Firefox support Gzip by default. All requests sent are attached with the “Accept-Encoding” header.

Both

<script src="https://cdn.ethers.io/lib/ethers-5.2.umd.min.js" type="application/javascript"></script>
<script>
  const provider = new ethers.providers.JsonRpcProvider(“your RPC endpoint")
  provider.send('eth_getBlockByNumber',['0xE6F190',false])
  .then(function(block) 
    {console.log(block);
}
  );
</script>

And

<script src="https://cdn.jsdelivr.net/npm/[email protected]/dist/web3.min.js"></script>
<script>
    const HTTP_ENDPOINT = 'your RPC endpoint'
    const provider = new Web3.providers.HttpProvider(HTTP_ENDPOINT)
    const web3 = new Web3(provider)
    web3.eth.getBlock('0xE6F190').then(function(block) {
        console.log(block);
    });
</script>

Both work fine. You can check the header in the browser developer’s tool.
Request header:

Response header:

And the response was auto-decoded by the browser.

Using Gzip at the backend

Enabling Gzip at the backend is more complicated because the header must be specified explicitly. At the time of writing, only ethers.js and curl support Gzip, not web3.js.

With Curl:

curl https://your-RPC-endpoint/12345 -X POST --data '{"id":1234,"jsonrpc":"2.0","method":"eth_getBlockByNumber", "params":["0xE6F190",false]}' -H "Content-type: application/json;" -H "Accept: application/json;" -H "Accept-Encoding: gzip" -–compressed

With ethers.js

const options = {
  allowGzip: "true",
  url: "your RPC endpoint",
};
const provider = new ethers.providers.JsonRpcProvider(options)

With web3.js, you can set the header by:

const options = { 
  headers: [ 
    { name: 'Content-Type', value: 'application/json', }, 
    { name: 'Accept-Encoding', value: 'gzip', }, 
  ],
}
const HTTP_ENDPOINT = 'your RPC endpoint'
const provider = new Web3.providers.HttpProvider(HTTP_ENDPOINT, options)
const web3 = new Web3(provider) 
web3.eth.getBlock('0xE6F190').then(function(block) {
  console.log(block);
});

However, you will probably receive an error message:

Refuse to set unsafe header “Accept-Encoding”

This error message is returned by a node module:

 /node_modules copy/xhr2-cookies/dist/xml-http-request.

You can delete this line to force it to attach the header. However, when the response is received, you will receive an invalid response like this.

That is a Gzip encoded data. But the receiving end(web3.js) does not know how to decode this response. Therefore it throws an error.
To summarise:

Figure 2: Libraries support Gzip

Performance

To show how Gzip can help optimize bandwidth consumption, 3 different EVM RPC methods were compared in their response size.

Response size without GzipResponse size with GzipBenchmarking
eth_blockNumber69 Byte45 Byte-34%
eth_call297 Byte97 Byte-64%
eth_getBlockByHash204 KB26 KB-87%

The compression rate increases with the size of the return data. But even for the simplest request, Gzip reduces the data size by more than 30 percent.

Conclusion

This is the end of this tutorial. Thanks for reading.
If you have any questions, feel free to ping me on Twitter/Telegram/Discord.
Happy coding.

Power-boost your project with Chainstack

Have you already explored what you can achieve with Chainstack? Get started for free today.

All we need is trust

Exploring the concept of trust and the world of consortiums and blockchain.

Chainstack
Dec 9
Chainstack uses cookies to provide you with a secure and
personalized experience on its website. Learn more.