HPCBench Now Open Source

Part of my work for High Performance Computing (HPC) analysis, I wrote a C/MPICH toolkit named HPCBench to evaluate the network performance for Linux-based clusters. Now it becomes an open-source project in SourceForge (http://hpcbench.sf.net).

Overview

Hpcbench is a Linux-based network benchmark evaluating the high performance networks such as Gigabit Ethernet, Myrinet and QsNet. Hpcbench measures the network latency and achievable throughput between two ends. Hpcbench is able to log the kernel information for each test, which includes the CPU and memory usage, interrupts, swapping, paging, context switches, network cards' statistics, etc.

Hpcbench consists of three independent packages that test UDP, TCP and MPI communications respectively. A kernel resources tracing tool "sysmon" is also included, whose output is similar to that of vmstat, but has more information of network statistics.

Programming language: C, MPI.
Recommended OS and compiler: Linux kernel 2.4 and gcc.

Features

UDP communication:

• Microsecond resolution
• Roundtrip time test (UDP ping)
• Throughput test
• Unidirectional and Bidirectional test
• UDP traffic generator (can run in single mode)
• Fixed size and exponential test
• Log throughputs and process resource usage of each test
• Log system resources information of client and server (Linux only)
• Create plot configuration file for gnuplot
• Configurable message size
• Other tunable parameters:
  • Port number
  • Client and server's UDP socket buffer size
  • Message size
  • Packet (datagram) size
  • Data size of each read/write
  • QoS (TOS) type (Pre-defined six levels)
  • Test time
  • Test repetition
  • Maximum throughput restriction (Unidirectional and UDP traffic generator)

TCP communication:

• Microsecond resolution
• Roundtrip Time test (TCP ping)
• Throughput test
• Unidirectional and Bidirectional test
• Blocking and non-blocking test
• Fixed size and exponential test
• Linux sendfile() test
• Log throughputs and process resource usage of each test
• Log system resources information of client and server (Linux only)
• Create plot configuration file for gnuplot
• Configurable message size
• Other tunable parameters:
  • Port number
  • Client and server's TCP socket buffer (window) size
  • Message size
  • Data size of each read/write
  • Iteration of read/write
  • MTU (MSS) setting
  • TCP socket's TCP_NODELAY option setting
  • TCP socket's TCP_CORK option setting
  • QoS (TOS) type (Pre-defined six levels)
  • Test time
  • Test repetition

MPI communication:

• Microsecond resolution
• Roundtrip Time test (MPI ping)
• Throughput test
• Unidirectional and Bidirectional test
• Blocking and non-blocking test
• Fixed size and exponential test
• Log throughputs and process resource usage of each test
• Log system resources information of two processes (nodes) (Linux only)
• Create plot configuration file for gnuplot
• Tunable parameters:
  • Message size
  • Test time
  • Test repetition