Benchmarking Reference Guide

ZelNode Benchmarking Reference Guide

New benchmarking script and metrics released for CPU benchtesting.

Ubuntu 16.04 LTS and 18.04 LTS default to installing different versions of Sysbench. The commands below will install Sysbench v1.0.16. Only this version of Sysbench will properly measure your CPU performance for characterization against the minimum requirements at the end of this page.

Please read: In-depth look at ZelNode specs, dates, etc: Medium Article

Please consider using the Zel DigitalOcean Referral code: https://m.do.co/c/c9c22684c5db

Multiple DigitalOcean Droplets at the BAMF, Super, and Basic requirement levels were tested using two separate OSS benchmarking tools. Remember that the Zel Network is being created to provide enterprise-grade hardware to the ecosystem for future development, and the requirements reflect such.

Tests include:

  • CPU performance via a large prime number search

  • Storage drive speed check

  • Storage drive IOPS check

  • Benchmark tools also display system specs (e.g. total RAM, CPU family/speed) for reference

There is also a test to determine the number of available threads for reference, but you should already know how many threads your system has available, either spec'd by the VPS provider, or resource management on bare metal

System Specifications - Differences between VPS and bare metal/co-located hardware

Benchmarking specs subject to change during testnet investigation. We strive to keep ZelNodes as fair as possible to stand up for the typical Zel holder.

Having both VPS and bare metal options provides greater decentralization of the Zel Computational Network both in ownership and geographic location. It also makes it harder to establish a hardware baseline with so many system configuration and infrastructure variables. We have set standards separately for VPS and bare metal/dedicated, as described below. First, the specifications, then the benchmarking results.

VPS Specifications (based on DigitalOcean Droplets)

Since a VPS user typically can't specify a hardware config (CPUs can vary by luck of the draw), we have provided generic VPS hardware specs as stated below. Benchmarking is based on Droplets spec'd for each ZelNode tier. Use these numbers to check if your preferred VPS provider can meet the minimum requirements to ensure payout after network activation.

VPS hardware specs by tier

  • BAMF | 8vCore | 32GB RAM | 600GB SSD | 6TB total monthly bandwidth @ 100mbps+

  • Super | 4vCore | 8GB RAM | 150GB SSD | 4TB total monthly bandwidth @ 100mbps+

  • Basic | 2vCore | 4GB RAM | 50GB SSD | 2.5TB total monthly bandwidth @ 100mbps+

Drive storage specs are total space of the drive. With the OS and other small necessities being part of overhead, the true usable disk space will likely be slightly under the stated values

Bare metal server specifications

There are a near-infinite number of possible server configurations. We are providing a baseline spec that is known to successfully meet the requirements; there are many other combinations that can meet the benchmarking requirements. Ultimately it is up to you to know if your hardware can pass. If you are unsure, please use a VPS until you are certain of which hardware to buy, this is a completely new type of network and we all will need to grow with it.

Server specs by tier

BAMF:

  • CPU: Intel Xeon E5-2690v1 (circa 2012+) or better

  • RAM: 32GB DDR3 or better (Xeons typically come with ECC)

  • SSD: 600GB (SATA III, 6Gb/s) or better w/ I/O speeds > 200 MB/s

  • Bandwidth: 6TB total up/down transfer w/ speeds of 100mbps+

Super:

  • CPU: Intel i5/i7 with 4+ cores & hyperthreading (circa 2017+) or better

  • RAM: 8GB DDR3/4 or better

  • SSD: 150GB (SATA III, 6Gb/s) or better w/ I/O speeds > 200MB/s

  • Bandwidth: 4TB total up/down transfer w/ speeds of 100mbps+

  • Possible to run in NUC-style configuration

Basic:

  • CPU: Intel i3 2.0GHz, Haswell Gen4 family or better

  • RAM: 4GB DDR3/4 or better

  • SSD: 50GB (SATA III, 6 Gb/s) or better w/ I/O speeds > 200MB/s

  • Bandwidth: 2.5TB total up/down transfer w/ speeds of 100mbps+

  • Possible to run in NUC-style configuration

Configurations other than stated above, or those wishing to run multiple nodes per chassis, should take care to benchmark before committing to a large capital investment. Also consider the costs of co-location, electricity, bandwidth, etc.

Benchmarking of DigitalOcean Droplets

Nench and Sysbench were used for benchmarking purposes.

  • Nench can be called with the curl command below

  • Sysbench can be installed via:\

curl -s https://packagecloud.io/install/repositories/akopytov/sysbench/script.deb.sh | sudo bash ----- apt install sysbench

Instructions are courtesy of haydenjames

Nench tests a whole suite of parameters, we are most interested in the IOPS test for the SSDs.

Sysbench also has many possible tests. We will be using the utility to test the CPU performance. Prime number search tests the outright speed of the CPU and does not utilize things like hardware-accelerated encryption, making it a suitable performance test.

Built-in Linux data duplicator (dd) utility is used to test drive speed and overall write times.

Benchmarking performed using Ubuntu 16.04 | Sysbench v1.0.16

Nench

Nench example
  • Command: (curl -s wget.racing/nench.sh | bash; curl -s wget.racing/nench.sh | bash) 2>&1 | tee nench.log

  • Check: ioping: sequential read speed -> iops

  • Can also check your bandwidth to multiple high speed servers

Sysbench

Sysbench example
  • Command: sysbench --test=cpu --threads=8 --cpu-max-prime=60000 --time=20 runChange --threads to 8 for BAMF, 4 for Super, 2 for Basic tiers

  • Check: events per second

DD (data duplicator)

dd example
  • Command: dd if=/dev/zero of=sb-io-test bs=1M count=10k conv=fdatasync; rm -rf sb-io-test

  • Check: Drive speed

Minimum benchmarking requirements as captured from DO Droplets at each spec. tier.

These minimum numbers are based on statistical analysis performed on roughly 12 different Droplets per tier. The requirements below cover the range of random Droplets out to >1 standard deviations. We will continue to add benchmarks to the sample population during testnet release and adjust requirements if necessary.

These specs are for Sysbench v1.0.16. Installation instructions are above.

BAMF (8vCore, 32GB RAM, 640GB SSD)

  • CPU | Sysbench events per second (60k max prime, 20 seconds): > 500

  • SSD IOPS | Nench total iops: > 700

  • SSD speed | dd speed (count=10k): > 200 MB/s

Super (4vCore, 8GB RAM, 160GB SSD)

  • CPU | Sysbench events per second (60k max prime, 20 seconds): > 250

  • SSD IOPS | Nench total iops: > 700

  • SSD speed | dd speed (count=10k): > 200MB/s

Basic (2vCore, 4GB RAM, 80GB SSD)

  • CPU | Sysbench events per second (60k max prime, 20 seconds): > 130

  • SSD IOPS | Nench total iops: > 700

  • SSD speed | dd speed (count=10k): > 200MB/s