Mastering DApp development on Chainstack: Techniques, tools, and best practices

At Chainstack, we’re committed to simplifying the intricate world of blockchain deployment and management for developers. We’ve observed how strategies like HTTP batch requests and multicall contracts have been leveraged by Web3 developers to boost their DApp performance. This comprehensive guide is crafted with an aim to venture deeper into these promising techniques, their features, performance, and practical applications.

In this guide, we’ll also shed light on the management of Ethereum logs via the eth_getLogs RPC method, discussing its limitations and offering best practices to enhance your DApp’s performance and reliability. Not only that, but we’ll also discuss how to monitor transaction propagation in Ethereum Virtual Machine (EVM) networks using Python, an essential aspect of maintaining the integrity and efficiency of the network.

The journey doesn’t end there. For those of you looking to handle real-time data in your DApps, we’ll illustrate the use of WebSocket connections with JavaScript and Python. Lastly, we can’t miss out on the power of multithreading in Python for Web3 requests—a proven methodology that can significantly boost your DApp performance, making it both robust and responsive.

This guide is catered to both experienced Web3 developers and those just starting their journey in the world of DApps. We’re confident that it will offer valuable insights that can make Web3 development smoother and more efficient for you. Let’s get started!

What are HTTP batch requests?

As a Web3 developer, you may often find yourself in circumstances requiring several requests to an Ethereum client. Here’s where HTTP batch requests come in handy. An HTTP batch request enables packing multiple HTTP requests into a single one that gets delivered to the server. This can be particularly beneficial when reducing the load on the server and enhancing the performance of your operations.

For instance, if you’re using Ethereum clients that support batch requests, such as Geth, you can leverage this feature for efficient data fetching. These packed requests get handled by the server in one single round trip, which can drastically enhance DApp responsiveness.

How to implement HTTP batch requests?

To implement an HTTP batch request, send a request with a payload containing multiple request objects in an array as shown below:

[
    {"jsonrpc":"2.0","method":"web3_clientVersion","params":[],"id":1},
    {"jsonrpc":"2.0","method":"eth_blockNumber","params":[],"id":2},
    {"jsonrpc":"2.0","method":"eth_syncing","params":[],"id":3},
    {"jsonrpc":"2.0","method":"eth_chainId","params":[],"id":4}
]

The server will send back the results in one response, arranged in the same order as the requests were received. For example:

[
    {
        "jsonrpc": "2.0",
        "id": 1,
        "result": "Geth/v1.10.26-stable-e5eb32ac/linux-amd64/go1.18.8"
    },
    {
        "jsonrpc": "2.0",
        "id": 2,
        "result": "0x10058f8"
    },
    {
        "jsonrpc": "2.0",
        "id": 3,
        "result": false
    },
    {
        "jsonrpc": "2.0",
        "id": 4,
        "result": "0x1"
    }
]

To run it using curl:

curl 'YOUR_CHAINSTACK_ENDPOINT' \\
--header 'Content-Type: application/json' \\
--data '[
    {"jsonrpc":"2.0","method":"web3_clientVersion","params":[],"id":1},
    {"jsonrpc":"2.0","method":"eth_blockNumber","params":[],"id":2},
    {"jsonrpc":"2.0","method":"eth_syncing","params":[],"id":3},
    {"jsonrpc":"2.0","method":"eth_chainId","params":[],"id":4}
]'

Popular Web3 libraries like web3.js and ethers.js support this feature as well. Below is an example of getting ether balances from multiple accounts at once using web3.js.

const { Web3 } = require("web3");
const NODE_URL =
  "YOUR_CHAINSTACK_ENDPOINT";
const web3 = new Web3(NODE_URL);
const addresses = [
  "0x1f9090aaE28b8a3dCeaDf281B0F12828e676c326",
  "0x2bB42C655DdDe64EE508a0Bf29f9D1c6150Bee5F",
];
async function getBalances() {
  const startTime = Date.now();
  // Create a batch request object
  const batch = new web3.BatchRequest();
  // Array to hold promises for each request
  const promises = [];
  // Loop through each address and add a getBalance request to the batch
  addresses.forEach((address, index) => {
    const request = {
      jsonrpc: "2.0",
      id: index + 1,
      method: "eth_getBalance",
      params: [address, "latest"],
    };
    // Add request to the batch and store the promise
    const requestPromise = batch.add(request);
    promises.push(requestPromise);
  });
  // Send the batch request and wait for all responses
  const responses = await batch.execute();
  // Process responses
  responses.forEach((response, index) => {
    if (response.error) {
      console.error(response.error);
    } else {
      const balance = response.result;
      const timeFromStart = Date.now() - startTime;
      console.log(
        `${addresses[index]} has a balance of ${Number(
          web3.utils.fromWei(balance, "ether")
        ).toFixed(3)} ETH retrieved in: ${timeFromStart / 1000} seconds.`
      );
    }
  });
}
getBalances();

The getBalances function creates a new BatchRequest object using web3.BatchRequest().

The function then loops through each address in the addresses array and creates a new request to get the balance of that address using web3.eth.getBalance.request(). It adds each request to the batch using batch.add().

Finally, the function executes the batch request using batch.execute(). When executed, the requests in the batch are sent to the Ethereum network simultaneously, and the callback functions are executed when the responses are received.

What are multicall contracts?

Comparatively, a multicall contract also takes on multiple function call objects but executes them together. As a Web3 developer, you can use the multicall contract as a proxy to evoke other contracts on Ethereum. It’s essentially a straightforward implementation that uses Solidity’s call function to broadcast contract calls.

The implementation of a multicall contract is straightforward: it uses Solidity’s call function to execute contract calls. This is an example implementation of the multicall’s aggregate function:

function aggregate(Call[] memory calls) public returns (uint256 blockNumber, bytes[] memory returnData) {
        blockNumber = block.number;
        returnData = new bytes[](calls.length);
        for(uint256 i = 0; i < calls.length; i++) {
            (bool success, bytes memory ret) = calls[i].target.call(calls[i].callData);
            require(success);
            returnData[i] = ret;
        }
    }

In summary, this function takes an array of calls, executes each one, and returns an array of the results along with the block number in which the function was called. It is designed to be used as a general-purpose aggregator for calling other contracts on the Ethereum blockchain.

How to implement multicall contracts?

Anyone can deploy their own multicall contract. In this example, we use MakerDAO’s multicall contract on the Ethereum mainnet, deployed at 0xeefBa1e63905eF1D7ACbA5a8513c70307C1cE441.

Below is an example of calling the smart contract using MakerDAO’s helper library multicall.js, which performs the same function as the previous example:

const multicall = require("@makerdao/multicall")
const config = {
    rpcUrl: "YOUR_CHAINSTACK_ENDPOINT",
    multicallAddress: "0xeefba1e63905ef1d7acba5a8513c70307c1ce441"
};
const addressArr = [
  "0x2B6ee955a98cE90114BeaF8762819d94C107aCc7",
  "0x2bB42C655DdDe64EE508a0Bf29f9D1c6150Bee5F"
];
async function main() {
  const startTime = Date.now();
  console.log("Started...");
  const calls = [];
  // Retrieve the Ether balance of each Ethereum address in addressArr using the multicall library.
  for (let i = 0; i < addressArr.length; i++) {
      const callObj = {
          call: [
              'getEthBalance(address)(uint256)',
              addressArr[i]
          ],
          returns: [
              [`ETH_BALANCE ${addressArr[i]}`, val => val / 10 ** 18]
          ]
      };
      calls.push(callObj);
  }
  const result = await multicall.aggregate(calls, config);
  console.log(result);
  const timeFromStart = Date.now() - startTime;
  console.log(`Result received in ${timeFromStart / 1000} seconds`);
}
main();

The main function iterates through each address in the addressArr array and creates a call object for each address. These call objects use the multicall library to retrieve the ether balance for each address.

Once all of the call objects have been created and added to the calls array, the multicall library’s aggregate function is called with the array of call objects and the configuration object. This function aggregates the results of all of the calls into a single object, which is stored in the result variable.

Finally, the code logs the result to the console and calculates the time it took to receive the result, which is also logged to the console.

You will need to install the multicall.js library to run this code.

HTTP batch request vs multicall contract performance comparison

Considering performance, both batch requests and multicall contracts can substantially speed up your DApp. Our tests, based on common use cases such as getting account balance and calling a smart contract, reveal performance improvements with both methods.

In this section, we compare the performance of three different approaches:

1. Sending multiple HTTP requests in parallel
2. Sending a batch HTTP request
3. Using a multicall contract

We will test these methods based on two common use cases:

• Getting account balance
• Calling a smart contract

Getting account balance for 30 distinct addresses

The testing script for batch requests and multicall contracts is included in the previous sections. Below is the code for sending multiple HTTP requests in parallel:

const { Web3 } = require('web3');
const web3 = new Web3('YOUR_CHAINSTACK_ENDPOINT');
var addressArr = [
    "0x2B6ee955a98cE90114BeaF8762819d94C107aCc7",
    "0x2bB42C655DdDe64EE508a0Bf29f9D1c6150Bee5F"
];
async function main() {
    var startTime = Date.now();
    console.log("started");
    for (i = 0; i < addressArr.length; i++) {
        web3.eth.getBalance(addressArr[i]).then(function(result) {
            var timeFromStart = Date.now() - startTime;
            console.log("Result received in:" + timeFromStart / 1000 + " seconds");
        });
    }
}
main();

The test was conducted between a server in Europe and a client in Singapore. A total of 15 measurements were averaged, showing that batch requests outperform multicall and normal requests. Compared with sending single requests in parallel, batch requests reduce the total request time by 38%, and multicall reduces it by 18%.

Getting the owners of BAYC tokens

Below are the testing scripts using web3.js for making smart contract calls. The tests are based on an ERC-721 standard method ownerOf from BAYC’s smart contract.

Sending multiple HTTP requests in parallel:

const { Web3 } = require('web3');
const web3 = new Web3('YOUR_CHAINSTACK_ENDPOINT');
const abi = [{ "inputs": [{ "internalType": "uint256", "name": "tokenId", "type": "uint256" }], "name": "ownerOf", "outputs": [{ "internalType": "address", "name": "", "type": "address" }], "stateMutability": "view", "type": "function" }];
const contract = new web3.eth.Contract(abi, "0xBC4CA0EdA7647A8aB7C2061c2E118A18a936f13D");
async function main() {
    const startTime = Date.now();
    console.log("started");
    for (i = 0; i < 30; i++) {
        contract.methods.ownerOf(i).call().then(function(result) {
            console.log(result);
            var timeFromStart = Date.now() - startTime;
            console.log("result received in: " + timeFromStart / 1000 + " seconds");
        });
    }
}
main();

Sending batch requests:

const { Web3 } = require('web3');
const web3 = new Web3('YOUR_CHAINSTACK_ENDPOINT);
const abi = [
  {
    inputs: [{ internalType: "uint256", name: "tokenId", type: "uint256" }],
    name: "ownerOf",
    outputs: [{ internalType: "address", name: "", type: "address" }],
    stateMutability: "view",
    type: "function",
  },
];
const contract = new web3.eth.Contract(
  abi,
  "0xBC4CA0EdA7647A8aB7C2061c2E118A18a936f13D"
);
async function main() {
  const startTime = Date.now();
  const batch = new web3.BatchRequest();
  console.log("started");
  // Array to hold promises for each request
  const promises = [];
  for (let i = 0; i < 30; i++) {
    const request = {
      jsonrpc: "2.0",
      id: i + 1,
      method: "eth_call",
      params: [
        {
          to: contract.options.address,
          data: contract.methods.ownerOf(i).encodeABI(),
        },
        "latest",
      ],
    };
    // Add request to the batch and store the promise
    const requestPromise = batch.add(request);
    promises.push(requestPromise);
  }
  // Send the batch request and wait for all responses
  const responses = await batch.execute();
  // Process responses
  responses.forEach((response, index) => {
    if (response.error) {
      console.error(response.error);
    } else {
      const ownerAddress = web3.eth.abi.decodeParameter(
        "address",
        response.result
      );
      const timeFromStart = Date.now() - startTime;
      console.log(
        `${index} token owner is ${ownerAddress} received in: ${
          timeFromStart / 1000
        } seconds`
      );
    }
  });
}
main();

Multicall contract:

const multicall = require("@makerdao/multicall");
const config = {
    rpcUrl: "YOUR_CHAINSTACK_ENDPOINT",
    multicallAddress: "0xeefba1e63905ef1d7acba5a8513c70307c1ce441"
};
async function main() {
    var startTime = Date.now();
    console.log("started");
    var calls = [];
    for (i = 0; i < 30; i++) {
        var callObj = {
            target: "0xBC4CA0EdA7647A8aB7C2061c2E118A18a936f13D",
            call: ['ownerOf(uint256)(address)', i],
            returns: [
                ['OWNER_ADDR ' + i]
            ]
        };
        calls.push(callObj);
    }
    const result = await multicall.aggregate(calls, config);
    console.log(result.results);
    var timeFromStart = Date.now() - startTime;
    console.log("Result received in: " + timeFromStart / 1000 + " seconds");
}
main();

The same test was conducted for reading contract calls. The result shows that both batch requests and multicall contracts save around 20% of total request time compared with sending single requests.

HTTP batch request vs multicall contract Q&A

If I package 100 requests into a single batch request, does that count as 1 request or 100 requests on Chainstack?

As an RPC provider, Chainstack counts “request” as RPC calls. After a server receives an HTTP batch request, it “unpacks” the request and processes the calls separately. So from the server’s point of view, 1 batch request of 100 calls consumes 100 requests instead of 1.

If I package 100 calls into a single multicall request, does that count as 1 request or 100 requests?

In this case, even though it is a very heavy call, it counts as a single request.

Is there any hard limit for the number of calls to multicall contracts?

The BAYC testing script stops working with 1,040 calls.

Which approach works better?

Even though tests show that batch requests and multicall contracts improve performance significantly, they do have their own limitations.

Requests in a batch request are executed in order, which means if a new block is received during execution, the subsequent results are likely to be different.

If you want to use a multicall contract, you should probably deploy your own contract for production just to ensure its availability.

Both batch requests and multicall contracts return multiple results in a single response. Both of them require much more computational resources. They can easily trigger “request timeout” errors and “response size too big” errors, which makes them not suitable for complex calls.

However, like every technology, they have their limitations. Requests in an HTTP batch are executed in order, indicating any new block received during execution can potentially alter subsequent results. If you’re planning to adopt multicall contracts, we recommend deploying your own contract for production to secure its availability.

Despite these minor constraints, using batch requests and multicall contracts can be an efficient way to handle multiple requests and enhance your DApp’s performance on Chainstack.

eth_getLogs limitations

If you’re a Web3 developer focusing on decentralized applications (DApps), you’ve likely come across eth_getLogs. It’s an Ethereum JSON-RPC endpoint that is used to query logs based on a filter object from the Ethereum blockchain. It’s an essential tool for auditing and retrieving past events emitted by smart contracts and can be accessed directly or indirectly through libraries like web3.js or ethers.js.

However, like any tool, eth_getLogs has its limitations, particularly when you’re working with EVM-compatible chains. Each of these networks often has different constraints, and understanding them can greatly improve your DApp’s efficiency.

For instance, the eth_getLogs method allows you to choose a range of blocks to fetch events from, but it’s important to use this facility judiciously. Although eth_getLogs is a powerful function, it’s resource-intensive, and a large block range can impact your node’s performance.

We’ve experienced that it’s beneficial to limit your block range queries and follow our recommended block range restrictions for various networks. This trade-off between performance and range limits can enhance your application’s performance while ensuring you receive the logs you’re interested in.

Additionally, you should know the best practices and efficient ways to use eth_getLogs. For instance, consider the following:

Limit the block range: Adhere to the block range limits specific to the network you are working with. This practice helps minimize the chances of receiving an oversized response or encountering a timeout error due to an extended query.

Paginate your queries: If you need to retrieve logs over a range that exceeds the network’s limit, divide your request into multiple smaller queries. This approach is akin to pagination in traditional APIs. Here’s an example way of doing that with Web3JS:

const { Web3 } = require("web3");
const NODE_URL = "YOUR_CHAINSTACK_ENDPOINT";
const web3 = new Web3(NODE_URL);
async function getLogs() {
  const startBlock = 14204533;
  const endBlock = 15204533;
  const range = 5000;
  const address = '0x4d224452801ACEd8B2F0aebE155379bb5D594381';
  const topics = ['0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef'];
  for (let i = startBlock; i <= endBlock; i += range) {
    const fromBlock = i;
    const toBlock = Math.min(i + range - 1, endBlock);
    const filter = {
      fromBlock,
      toBlock,
      address,
      topics
    };
    const logs = await web3.eth.getPastLogs(filter);
    console.log(logs);
  }
}
getLogs();

Filter logs to manage returned data effectively: Retrieving logs from a blockchain can generate a substantial amount of data, especially on a busy network or when querying a large number of blocks. To manage this effectively and avoid unnecessary processing, it is crucial to apply filters to your queries unless you need to retrieve all events at once.

How to filter logs to manage returned data example

Let’s take it a step further and demonstrate how to fetch and store Transfer event logs from the BAYC smart contract. This project stores event logs in a MongoDB instance, providing a good starting point for creating your own BAYC API.

Prerequisites:

• Node.js
• MongoDB Atlas account with database setup and dedicated account
• Chainstack Ethereum node

Setup:

For this example, we will be using Node.js, so let’s set up a project:

npm init -y

Next, install the necessary dependencies:

npm install web3 dotenv mongodb

Now, create .env and main.js files and fill in the following information:

.env

MONGODB_CONNECTION_STRING="YOUR_MONGO_DB_CONNECTION"
CHAINSTACK_URL="YOUR_CHAINSTACK_ENDPOINT"

main.js

require("dotenv").config();
const { Web3 } = require("web3");
const MongoClient = require("mongodb").MongoClient;
async function connectToMongoDB(connectionString) {
	const client = new MongoClient(connectionString);
	try {
		await client.connect();
		console.log("Connected to MongoDB");
		return client;
	} catch (err) {
		console.error(`Failed to connect to MongoDB: ${err}`);
		return null;
	}
}
// Connect to your Chainstack Ethereum node
console.log("Connecting to Ethereum node...");
const web3 = new Web3(
	new Web3.providers.HttpProvider(process.env.CHAINSTACK_URL)
);
console.log("Connected to Ethereum node");
// Set the BAYC contract address
const contractAddress = "0xBC4CA0EdA7647A8aB7C2061c2E118A18a936f13D"; // BAYC contract address
// BAYC contract was created at this block
const startBlock = 12686718;
const batch_size = 5000; // Stay within Ethereum network limits
// Connect to MongoDB
connectToMongoDB(process.env.MONGODB_CONNECTION_STRING).then(async (client) => {
	if (client) {
		const db = client.db("BAYC");
		const collection = db.collection("bayc-logs");
		// Get latest block number
		console.log("Fetching latest block number...");
		const endBlock = await web3.eth.getBlockNumber();
		console.log(`Latest block number is ${endBlock}`);
		// Query logs in batches
		for (let i = startBlock; i < endBlock; i += batch_size) {
			const toBlock = Math.min(i + batch_size - 1, endBlock);
			try {
				console.log(`Fetching events from block ${i} to ${toBlock}...`);
				/* 
                We can use getPastLogs or getPastEvents
                getPastLogs return raw data, getPastEvents will do some formatting on the data returned
                */
				const logs = await web3.eth.getPastLogs({
					fromBlock: web3.utils.toHex(i),
					toBlock: web3.utils.toHex(toBlock),
					address: contractAddress,
					topics: [web3.utils.keccak256("Transfer(address,address,uint256)")],
				});
				console.log(
					`Fetched ${logs.length} logs from block ${i} to ${toBlock}`
				);
				// Process logs and store them in MongoDB
				for (let log of logs) {
					await collection.insertOne(log);
					console.log(`Stored log ${log.id} in MongoDB`);
				}
				console.log(`Stored logs from block ${i} to ${toBlock} in MongoDB`);
			} catch (err) {
				console.error(
					`Error fetching logs from block ${i} to ${toBlock}: ${err}`
				);
			}
		}
		client.close();
	}
});

Once the files are ready, run the script with the following command:

node main.js

If everything is set up properly, you will see the following output:

❯ node main.js
Connecting to Ethereum node...
Connected to Ethereum node
Connected to MongoDB
Fetching latest block number...
Latest block number is 17401394
Fetching events from block 12686718 to 12691717...
Fetched 183 logs from block 12686718 to 12691717
Stored log log_5b71b7bd in MongoDB
Stored log log_22da2c2a in MongoDB
Stored log log_6c6c5129 in MongoDB
Stored log log_9b606429 in MongoDB
Stored log log_de39154c in MongoDB
Stored log log_a49334ff in MongoDB
...
...
Stored logs from block 12686718 to 12691717 in MongoDB
Fetching events from block 12691718 to 12696717...
Fetched 125 logs from block 12691718 to 12696717
Stored log log_e19237a9 in MongoDB
Stored log log_7611f848 in MongoDB
Stored log log_a8e6d7b7 in MongoDB
Stored log log_999b2ede in MongoDB
Stored log log_2f4fb9f4 in MongoDB
Stored log log_51054a82 in MongoDB
...
...

Congratulations, you just set up your own data-processing script!

How to monitor transaction propagation in EVM chains with Python

As a Web3 developer, you must be aware that achieving optimal performance in your DApp involves understanding and monitoring transaction propagation on Ethereum Virtual Machine (EVM) networks. At Chainstack, one of our goals is to ensure the integrity and performance of our network. And to do this effectively, it’s imperative to monitor how transactions propagate.

Monitoring transaction propagation serves several key purposes:

1. Test network performance: Assess the network’s resilience and reaction to multiple transactions issued simultaneously.
2. Identify latency issues: Identify potential causes of propagation delay to enhance network performance.
3. Optimize DApp performance: Feedback from these tests can help optimize your DApp, resulting in reduced transaction time and better user experience.

How can you conduct these monitoring activities?

At Chainstack, we use Python to monitor transaction propagation and have devised a detailed strategy. Here’s a simplified rundown:

1. Broadcast tansactions: Issue a series of transactions to the network.
2. Monitor propagation: Set a network of listening nodes using Python to monitor how these transactions propagate.
3. Measure performance: Track the time each node takes to receive a transaction, from the time of issuance to the time of receipt.

Remember, the aim is not just to get transactions confirmed but to gauge how different parts of the network respond under stress. Look for variations in block time, check if certain sections of the network are slower, and use this feedback to enhance your DApp’s performance. Here’s the full code for an example monitoring tool you can BUIDL with ease:

from web3 import Web3
import time
from web3.exceptions import TransactionNotFound
from concurrent.futures import ThreadPoolExecutor, as_completed
import threading
# Connect to the Ethereum node
w3 = Web3(
  Web3.HTTPProvider(
    'YOUR_CHAINSTACK_ENDPOINT'))
# Ethereum address to monitor
address = 'YOUR_ETHEREUM_ADDRESS'
def pretty_print_transaction(tx):
  def wait_for_confirmation(tx_hash, block_info):
    while True:
      try:
        receipt = w3.eth.get_transaction_receipt(tx_hash)
        if receipt is not None and receipt['blockHash']:
          block_info['blockHash'] = receipt['blockHash']
          block_info['blockNumber'] = receipt['blockNumber']
          break
      except TransactionNotFound:
        pass
      time.sleep(1)
  block_info = {'blockHash': None, 'blockNumber': None}
  # Start a separate thread to wait for the block
  confirmation_thread = threading.Thread(target=wait_for_confirmation,
                                         args=(tx['hash'], block_info))
  confirmation_thread.start()
  print("Transaction details:")
  print(f"  Block hash: {block_info['blockHash']}")
  print(f"  Block number: {block_info['blockNumber']}")
  print(f"  From: {tx.get('from')}")
  print(f"  Gas: {tx.get('gas')}")
  print(f"  Gas price: {tx.get('gasPrice')}")
  if 'maxFeePerGas' in tx:
    print(f"  Max fee per gas: {tx['maxFeePerGas']}")
  if 'maxPriorityFeePerGas' in tx:
    print(f"  Max priority fee per gas: {tx['maxPriorityFeePerGas']}")
  print(f"  Transaction hash: {tx.get('hash').hex()}")
  print(f"  Input: {tx.get('input')}")
  print(f"  Nonce: {tx.get('nonce')}")
  print(f"  To: {tx.get('to')}")
  print(f"  Transaction index: {tx.get('transactionIndex')}")
  print(f"  Value: {tx.get('value')}")
  print(f"  Type: {tx.get('type')}")
  print(f"  Access list: {tx.get('accessList')}")
  print(f"  Chain ID: {tx.get('chainId')}")
  print(f"  v: {tx.get('v')}")
  print(f"  r: {tx.get('r').hex() if tx.get('r') else None}")
  print(f"  s: {tx.get('s').hex() if tx.get('s') else None}")
  # Wait for the confirmation thread to complete
  confirmation_thread.join()
  print(f"Updated block hash: {block_info['blockHash'].hex()}")
  print(f"Updated block number: {block_info['blockNumber']}")
def check_pending_transaction(tx_hash, target_address_lower, w3):
  try:
    tx = w3.eth.get_transaction(tx_hash)
    if tx['from'].lower() == target_address_lower or (
        tx['to'] and tx['to'].lower() == target_address_lower):
      return tx
  except TransactionNotFound:
    pass
  return None
# Function to check if any transactions from/to the target address are in the mempool
def find_mempool_transactions(target_address):
  local_w3 = w3
  transaction_list = []
  target_address_lower = target_address.lower()
  current_block = local_w3.eth.block_number
  pending_transactions = local_w3.eth.get_block('pending')['transactions']
  with ThreadPoolExecutor() as executor:
    futures = [
      executor.submit(check_pending_transaction, tx_hash, target_address_lower,
                      local_w3) for tx_hash in pending_transactions
    ]
    for future in as_completed(futures):
      result = future.result()
      if result is not None:
        transaction_list.append(result)
  return transaction_list
# Main function to monitor the mempool
def monitor_mempool(address):
  seen_transactions = set()
  # Add the print statement here
  print("Mempool monitoring starting...")
  while True:
    current_block = w3.eth.block_number
    pending_block = w3.eth.get_block('pending')
    pending_transactions = pending_block['transactions']
    print(
      f"Current block: {current_block}. Pending transactions: {len(pending_transactions)}"
    )
    # Record the start time
    start_time = time.time()
    transactions = find_mempool_transactions(address)
    if transactions:
      new_transactions = [
        tx for tx in transactions if tx.get('hash') not in seen_transactions
      ]
      if new_transactions:
        # Calculate the time taken
        time_taken = time.time() - start_time
        print(f"\\nTime taken since last check: {time_taken:.2f} seconds\\n")
        print(
          f"Found {len(new_transactions)} new transaction in the mempool involving {address}:"
        )
        for tx in new_transactions:
          pretty_print_transaction(tx)
          seen_transactions.add(tx.get('hash'))
        break
monitor_mempool(address)

How to run a test using the monitoring tool?

To conduct this test, start by executing the Python script. Then, initiate a transaction using MetaMask for simplicity.

Your choice of endpoint depends on your objective. If you want to measure the speed at which a transaction reaches the mempool of your own node, use the same endpoint as in the script. If your goal is to determine the time taken for the transaction to propagate across other nodes in the Ethereum network, use a different endpoint. This will provide a more accurate depiction of transaction propagation times across the network.

In the Python script, input your endpoint and the Ethereum address you wish to monitor. This could be either the sending or receiving address for the transaction, as the script is designed to track the transaction in both cases.

Once the script detects a new transaction in the mempool involving the target address, it will display the transaction’s details in the console. Additionally, it will provide an estimated duration that the script took to locate the transaction. While this figure might not be entirely accurate, it serves as a useful approximation of the transaction’s propagation speed across the Ethereum network.

Steps:

1. Start the script.
2. Send a transaction using MetaMask.
3. You will receive a similar log in the console:

Mempool monitoring starting...
Current block: 3772987. Pending transactions: 84
Current block: 3772987. Pending transactions: 102
Time taken since last check: 2.22 seconds
Found 1 new transaction in the mempool involving 0x8f8e7012F8F974707A8F11C7cfFC5d45EfF5c2Ae:
Transaction details:
  Block hash: None
  Block number: None
  From: 0x8f8e7012F8F974707A8F11C7cfFC5d45EfF5c2Ae
  Gas: 21000
  Gas price: 2181505086
  Max fee per gas: 2181505086
  Max priority fee per gas: 1500000000
  Transaction hash: 0xebbeaa0ee6e787fa3486db9e1b8ad9ccb1e3ab982462c51fca8fa41143be053d
  Input: 0x
  Nonce: 59
  To: 0x7ea178aE883bC78Fa540b15F36b1e2a8Ea90F7F7
  Transaction index: None
  Value: 1000000000000000000
  Type: 2
  Access list: []
  Chain ID: 11155111
  v: 0
  r: 0x1483859043ee02820eead543ce58bf9f5a6ec3cd3b339dc709e1860781aa1e57
  s: 0x045fb5f1bb7caf42cbeb2d480fbb1a3ed1a85408154bcb052fbb17417eab5e84
Updated block hash: 0x2b120a75e3a97ba9b77d3764945c4c3c2a328699c13327538fb6dacc4642ff57
Updated block number: 3772988

Testing transaction propagation in the mempool for EVM nodes is crucial for maintaining a reliable and efficient blockchain network. By simulating realistic scenarios and using a combination of network monitoring and custom tests, developers and infrastructure operators can ensure that nodes process transactions in a timely and secure manner. Regularly conducting propagation tests will help identify potential issues, optimize resources, and contribute to the overall health of your blockchain ecosystem.

How to handle real-time data using WebSocket?

The requirement and expectation of real-time updates is gradually becoming prominent in the sphere of DApp development. And this is where the WebSocket protocol jumps in. It notably stands out when contrasted with HTTP, particularly in situations where the server needs to push updates to the client.

At Chainstack, we’ve navigated this landscape to simplify real-time data management for you, the Web3 developer, using WebSockets in both JavaScript and Python.

What is the WebSocket protocol?

WebSocket is a communications protocol providing full-duplex communication channels over a single TCP connection. In simpler terms, it allows both the client and the server to send data to each other, any time. This makes it an adequate choice for use cases requiring real-time data updates, such as DApp development.

How to set up WebSocket connections?

WebSocket connections uphold an open line of communication between the client and the server. To establish a WebSocket connection in JavaScript, consider utilizing libraries like web3.js or ethers.js. And for Python, you may utilize web3.py, which offers a WebsocketProvider for this purpose.

Here’s a simplified workflow process with JavaScript or Python:

1. Establish a connection with the WebSocket server.
2. Call functions and listen for events in real-time.
3. Handle errors and connection stability.

How to handle WebSocket errors and connection stability

It’s pivotal to remember that connection stability and error handling are critical when using WebSocket. Unstable network conditions can cause your WebSocket connection to drop. Therefore, building a resilient logic that caters to these fluctuations effectively can pay dividends in improving your DApp’s performance.

How to fetch real-time data with WebSocket and Web3JS?

The web3.js library integrates subscription functionalities, enabling you to get real-time data from the blockchain effortlessly.

First, install the web3.js library by running the following command:

npm i web3

The following example demonstrates using the web3.js library to receive real-time block headers, including WebSocket reconnect logic.

const { Web3 } = require("web3");
const NODE_URL = "YOUR_CHAINSTACK_WSS_ENDPOINT";
// Reconnect options
const reconnectOptions = {
  autoReconnect: true,  // Automatically attempt to reconnect
  delay: 5000,          // Reconnect after 5 seconds
  maxAttempts: 10,      // Max number of retries
};
const web3 = new Web3(
  new Web3.providers.WebsocketProvider(NODE_URL, undefined, reconnectOptions)
);
async function subscribeToNewBlocks() {
  try {
    // Create a new subscription to the 'newBlockHeaders' event
    const event = "newBlockHeaders";
    const subscription = await web3.eth.subscribe(event); // Changed to 'newHeads'
    console.log(`Connected to ${event}, Subscription ID: ${subscription.id}`);
    // Attach event listeners to the subscription object for 'data' and 'error'
    subscription.on("data", handleNewBlock);
    subscription.on("error", handleError);
  } catch (error) {
    console.error(`Error subscribing to new blocks: ${error}`);
  }
}
// Fallback functions to react to the different events
// Event listener that logs the received block header data
function handleNewBlock(blockHeader) {
  console.log("New block header:", blockHeader);
}
// Event listener that logs any errors that occur
function handleError(error) {
  console.error("Error when subscribing to new block header:", error);
}
subscribeToNewBlocks();

This script sets up a WebSocket connection to an Ethereum node, subscribes to new block headers, and includes logic to handle reconnections. It logs new block headers and errors to the console.

How to fetch real-time data with WebSocket and ethersJS?

Install the ethers.js and ws libraries:

npm i ethers ws

The following example demonstrates how to establish a WebSocket connection using ethers.js. This script is designed to subscribe to new block headers and includes a mechanism for automatic WebSocket reconnection.

const ethers = require("ethers");
const WebSocket = require("ws");
const NODE_URL =
  "YOUR_CHAINSTACK_WSS_ENDPOINT";
function createWebSocket() {
  const ws = new WebSocket(NODE_URL);
  ws.on("close", () => {
    console.log("Disconnected. Reconnecting...");
    setTimeout(() => {
      provider = new ethers.WebSocketProvider(createWebSocket());
      startListening();
    }, 3000);
  });
  ws.on("error", (error) => {
    console.log("WebSocket error: ", error);
  });
  return ws;
}
let provider = new ethers.WebSocketProvider(createWebSocket());
function startListening() {
  provider.on("block", async (blockNumber) => {
    console.log("New block number:", blockNumber);
    const block = await provider.getBlock(blockNumber);
    console.log("Block details:", block);
  });
}
startListening();

This script sets up a WebSocket connection to an Ethereum node, subscribes to new block headers, and includes logic to handle reconnections. It logs new block numbers and their details to the console, ensuring that the connection is re-established if it drops.

How to fetch real-time data with WebSocket and Python?

Install the websockets library:

pip install websockets

The following example demonstrates how to establish a WebSocket connection using Python. This script is designed to subscribe to new block headers and incorporates an automatic WebSocket reconnection mechanism.

# Import required libraries
import asyncio
import json
import websockets
# Replace with your own Ethereum node WebSocket URL
eth_node_ws_url = 'YOUR_CHAINSTACK_WSS_ENDPOINT'
async def subscribe_to_blocks(ws_url):
    # Continuously try to connect and subscribe
    while True:
        try:
            # Establish a WebSocket connection to the Ethereum node
            async with websockets.connect(ws_url) as websocket:
                # Send a subscription request for the Transfer event logs
                await websocket.send(json.dumps({
                    "id": 1,
                    "method": "eth_subscribe",
                    "params": [
                        "newHeads"
                    ],
                    "jsonrpc": "2.0"
                }))
                # Wait for the subscription response and print it
                subscription_response = await websocket.recv()
                print(f'Subscription response: {subscription_response}')
                # Continuously process incoming logs
                while True:
                    # Receive a log entry and parse it as JSON
                    log = await websocket.recv()
                    log_data = json.loads(log)
                    # Print the log data
                    print(f'New log: {log_data}')
                    print("#"*10)
        # If there's an exception (e.g., connection error), attempt to reconnect
        except Exception as e:
            print(f'Error: {e}')
            print('Reconnecting...')
            await asyncio.sleep(5)
# Execute the subscription function
asyncio.run(subscribe_to_blocks(eth_node_ws_url))

This script establishes a WebSocket connection to an Ethereum node, subscribes to new block headers, and includes logic to handle reconnections. It logs new block headers and any errors to the console, ensuring continuous monitoring even if the connection drops.

Applying WebSocket protocol for real-time data can certainly contribute to the overall performance of your DApp and provide a more interactive and responsive experience for the end-user.

How to master multithreading in Python for Web3 requests?

For a Web3 developer constantly in pursuit of robustness and responsiveness for their DApps, harnessing the technique of multithreading in Python can prove to be a significant advantage. At Chainstack, we realize the potential of this method in handling numerous Web3 requests, and we’re eager to guide you through it.

Python’s multithreading functionality allows you to execute multiple threads in parallel. This means that you can send multiple requests concurrently instead of waiting for each request to complete one after the other which can be quite time-consuming.

Here’s a brief overview of using multithreading for Web3 requests:

1. Start with your requests: Identify the requirements of repeated Web3 calls in your DApp.
2. Create worker threads: Python’s threading library allows you to create multiple threads (or ‘workers’) to execute your requests concurrently.
3. Queue your requests: Utilize thread-safe queues to manage your threads and their respective Web3 calls effectively.
4. Run and monitor: Execute your program and make sure to handle exceptions and events correctly.

Multithreading undoubtedly enhances the performance of your DApp by saving time on each Web3 API call, ensuring that DApp users get a smoother interface with quicker responses.

How to create a simple Web3 script without multithreading?

Let’s start with a simple example of making Web3 requests without multithreading. We’ll write a script to fetch the balance of an Ethereum address at various block numbers.

from web3 import Web3
import time
web3 = Web3(Web3.HTTPProvider("YOUR_CHAINSTACK_ENDPOINT"))
address = "0x1f9090aaE28b8a3dCeaDf281B0F12828e676c326"
start_block = web3.eth.block_number
end_block = start_block - 500
def get_balance_at_block(block_num):
  balance = web3.eth.get_balance(address, block_identifier=block_num)
  print(f"Balance at block {block_num}: {web3.from_wei(balance, 'ether')} ETH")
start = time.time()
for block_num in range(start_block, end_block, -1):
    get_balance_at_block(block_num)
print(f"Time taken: {time.time() - start}")

1. Import necessary modules: We import the Web3 module from the web3 package and the time module to measure the script’s execution time.
2. Set up the Web3 provider: Establish a connection to our Ethereum node using Web3.HTTPProvider.
3. Define the Ethereum address and block range: Specify the Ethereum address and the range of block numbers we want to check.
4. Define the function for fetching the balance: The get_balance_at_block function takes a block number as input, fetches the balance at that block, and prints it.
5. Fetch the balance at each block number: Loop over the range of block numbers and call get_balance_at_block for each one. Requests are made sequentially.
6. Measure and print the execution time: Use time.time() to get the execution time and print it.

How to create a simple Web3 script with multithreading?

Now let’s modify the previous example to use multithreading. This will allow us to make multiple Web3 requests concurrently, potentially speeding up our script.

import asyncio
from concurrent.futures import ThreadPoolExecutor
from web3 import Web3
import time
web3 = Web3(Web3.HTTPProvider("YOUR_CHAINSTACK_ENDPOINT"))
address = "0x1f9090aaE28b8a3dCeaDf281B0F12828e676c326"
start_block = web3.eth.block_number
end_block = start_block - 500
max_workers = 100
def get_balance_at_block(block_num):
  balance = web3.eth.get_balance(address, block_identifier=block_num)
  print(f"Balance at block {block_num}: {web3.from_wei(balance, 'ether')} ETH")
async def main():
  with ThreadPoolExecutor(max_workers=max_workers) as executor:
    tasks = [
      loop.run_in_executor(
        executor,
        get_balance_at_block,
        block_num
      ) for block_num in range(start_block, end_block, -1)
    ]
    await asyncio.gather(*tasks)
loop = asyncio.get_event_loop()
start = time.time()
loop.run_until_complete(main())
print(f"Time taken: {time.time() - start}")

1. Additional imports: We import asyncio and ThreadPoolExecutor from concurrent.futures.
2. Creating the ThreadPoolExecutor: Inside the main function, we create a ThreadPoolExecutor with a maximum of 100 worker threads.
3. Creating tasks: Create a list of tasks, where each task calls get_balance_at_block for a different block number, running in the executor.
4. Running tasks concurrently: Use asyncio.gather(*tasks) to run the tasks concurrently.
5. Running the event loop: Use loop.run_until_complete(main()) to run the event loop until the main() function completes.

How to organize the multithreading script response?

To ensure that the results are displayed in the correct order (by block number), we can modify the code slightly:

import asyncio
from concurrent.futures import ThreadPoolExecutor
from web3 import Web3
import time
web3 = Web3(Web3.HTTPProvider("YOUR_CHAINSTACK_ENDPOINT"))
address = "0x1f9090aaE28b8a3dCeaDf281B0F12828e676c326"
start_block = web3.eth.block_number
end_block = start_block - 500
max_workers = 100
def get_balance_at_block(block_num):
    try:
        balance = web3.eth.get_balance(address, block_identifier=block_num)
        print(f"Balance at block {block_num}: {web3.from_wei(balance, 'ether')} ETH")
    except Exception as e:
        print(f"Error occurred while getting balance at block {block_num}: {e}")
async def main():
    with ThreadPoolExecutor(max_workers=max_workers) as executor:
        loop = asyncio.get_event_loop()
        futures = [
            loop.run_in_executor(
                executor,
                get_balance_at_block,
                block_num
            ) for block_num in range(start_block, end_block, -1)
        ]
        for future in asyncio.as_completed(futures):
            try:
                # This will raise an exception if the thread raised an exception
                result = await future
            except Exception as e:
                print(f"Error occurred in thread: {e}")
loop = asyncio.get_event_loop()
start = time.time()
loop.run_until_complete(main())
print(f"Time taken: {time.time() - start}")

How to handle errors and exceptions in multithreaded architecture?

Proper error handling is crucial in multithreaded applications. Here are some common errors and how to handle them:

1. Rate limit errors: Handle these by catching the appropriate exception in a try/except block and including logic to delay or reduce the rate of requests if you encounter a rate limit error.
2. Thread errors: Limit the number of threads created and catch any exceptions that occur when managing threads.
3. Synchronization errors: Use synchronization primitives like locks, semaphores, or condition variables to prevent race conditions.
4. Unhandled exceptions in threads: Catch and handle exceptions within each thread or use the Future.result() method to handle exceptions in the main thread.

Best practices for multithreaded Web3 requests

• Choose an appropriate number of threads: Balance the number of threads based on the nature of the tasks and the system’s resources.
• Handle exceptions properly: Use try/except blocks to catch and handle exceptions in each thread.
• Manage thread lifecycles: Clean up after your threads when they’re done, especially for long-running applications.
• Avoid shared state when possible: Design your threads to be independent of each other to avoid race conditions.
• Use appropriate synchronization primitives: If shared state is necessary, use locks, semaphores, or other synchronization primitives.
• Don’t ignore the GIL: For I/O-bound tasks like network requests, multithreading can provide significant performance benefits despite Python’s Global Interpreter Lock (GIL).
• Respect the server’s limits: Be aware of the server’s rate limits and avoid making too many requests at once.

By following these best practices, you can ensure that your multithreaded Web3 application is efficient, robust, and easy to maintain.

However, it’s pivotal to note that working with multithreading doesn’t come without its challenges. Correctly managing threads is critical to avoid any unexpected issues and improve your DApp’s performance.

We believe that with time, practice, and the right guidance, mastering multithreading in Python for Web3 requests can become a strength in your Web3 developer arsenal, making your DApps more robust and responsive.

Further reading

• The ultimate guide to building DApps on Chainstack: Explore our ultimate guide covering protocol selection, API authentication, error handling, and more.
• Ethereum DApp developer tools to help you every step of the way: Accelerate your journey from complete beginner to Web3 pro with the most impactful Ethereum development tools.
• Web3 Appchains, the future of DApps: Discover how appchains provide an innovative approach to building Web3 applications, significantly improving scalability and performance.
• Guide to Appchains and how to deploy them on Chainstack: Learn how to deploy Avalanche Subnets, Polygon CDK, zkSync Hyperchains, and Starknet Appchains on Chainstack.
• How to choose the right blockchain network for your DApp: Understand how to choose the right blockchain network for your project and the differences between L1 and L2 networks.
• Make your DApp more reliable with Chainstack: Learn how to use multiple Chainstack nodes with load balancer logic to make your DApp more performant and reliable.

Bringing it all together

Navigating the realm of DApp development involves a plethora of considerations—HTTP batch requests, multicall contracts, understanding eth_getLogs limitations, monitoring transactions propagation in EVM networks, handling real-time data with WebSockets, and mastering multithreading in Python for Web3 requests.

At Chainstack, we empathize with the complexities faced by Web3 developers, thus we’re passionate about simplifying these complex nuances. Our aim with this guide was to offer insights into these advanced methodologies and their practical applications. We intended to provide a framework to understand their potential, challenges, and best practices while optimizing DApps on Chainstack.

Sincerely, we hope these insights will make your journey smoother and more efficient as a Web3 developer, leading to the creation of DApps that are not only high-functioning but also user-centric and time-efficient.

Always remember, each technique presents its own set of opportunities and challenges. The key is to use them judiciously, bearing in mind the specific requirements of your DApp, and your audience.

At Chainstack, we’re consistently learning and growing too. So, if you have any thoughts, discoveries, or feedback, we’d love to hear from you. Until then, happy coding!

Power-boost your project on Chainstack

• Discover how you can save thousands in infra costs every month with our unbeatable pricing on the most complete Web3 development platform.
• Input your workload and see how affordable Chainstack is compared to other RPC providers.
• Connect to Ethereum, Solana, BNB Smart Chain, Polygon, Arbitrum, Base, Optimism, Avalanche, TON, Ronin, zkSync Era, Starknet, Scroll, Aptos, Fantom, Cronos, Gnosis Chain, Klaytn, Moonbeam, Celo, Aurora, Oasis Sapphire, Polygon zkEVM, Bitcoin and Harmony mainnet or testnets through an interface designed to help you get the job done.
• To learn more about Chainstack, visit our Developer Portal or join our Discord server and Telegram group. 
• Are you in need of testnet tokens? Request some from our faucets. Multi-chain faucet, Sepolia faucet, Holesky faucet, BNB faucet, zkSync faucet, Scroll faucet.

Have you already explored what you can achieve with Chainstack? Get started for free today.