HPCBench Now Supports Linux Kernel 2.6.X

I just updated the HPCBench utility last week. It now can work in latest Linux distributions with kernel 2.6.x.

You can visit http://hpcbench.sourceforge.net for more information about HPCBench.

HPCBench Now Open Source

Part of my work for High Performance Computing (HPC) analysis, I wrote a C/MPICH toolkit named HPCBench to evaluate the network performance for Linux-based clusters. Now it becomes an open-source project in SourceForge (http://hpcbench.sf.net).

Overview

Hpcbench is a Linux-based network benchmark evaluating the high performance networks such as Gigabit Ethernet, Myrinet and QsNet. Hpcbench measures the network latency and achievable throughput between two ends. Hpcbench is able to log the kernel information for each test, which includes the CPU and memory usage, interrupts, swapping, paging, context switches, network cards' statistics, etc.

Hpcbench consists of three independent packages that test UDP, TCP and MPI communications respectively. A kernel resources tracing tool "sysmon" is also included, whose output is similar to that of vmstat, but has more information of network statistics.

Programming language: C, MPI.
Recommended OS and compiler: Linux kernel 2.4 and gcc.

Features

UDP communication:

• Microsecond resolution
• Roundtrip time test (UDP ping)
• Throughput test
• Unidirectional and Bidirectional test
• UDP traffic generator (can run in single mode)
• Fixed size and exponential test
• Log throughputs and process resource usage of each test
• Log system resources information of client and server (Linux only)
• Create plot configuration file for gnuplot
• Configurable message size
• Other tunable parameters:
  • Port number
  • Client and server's UDP socket buffer size
  • Message size
  • Packet (datagram) size
  • Data size of each read/write
  • QoS (TOS) type (Pre-defined six levels)
  • Test time
  • Test repetition
  • Maximum throughput restriction (Unidirectional and UDP traffic generator)

TCP communication:

• Microsecond resolution
• Roundtrip Time test (TCP ping)
• Throughput test
• Unidirectional and Bidirectional test
• Blocking and non-blocking test
• Fixed size and exponential test
• Linux sendfile() test
• Log throughputs and process resource usage of each test
• Log system resources information of client and server (Linux only)
• Create plot configuration file for gnuplot
• Configurable message size
• Other tunable parameters:
  • Port number
  • Client and server's TCP socket buffer (window) size
  • Message size
  • Data size of each read/write
  • Iteration of read/write
  • MTU (MSS) setting
  • TCP socket's TCP_NODELAY option setting
  • TCP socket's TCP_CORK option setting
  • QoS (TOS) type (Pre-defined six levels)
  • Test time
  • Test repetition

MPI communication:

• Microsecond resolution
• Roundtrip Time test (MPI ping)
• Throughput test
• Unidirectional and Bidirectional test
• Blocking and non-blocking test
• Fixed size and exponential test
• Log throughputs and process resource usage of each test
• Log system resources information of two processes (nodes) (Linux only)
• Create plot configuration file for gnuplot
• Tunable parameters:
  • Message size
  • Test time
  • Test repetition

Sysmon - A Linux Tool to Monitor System Resources

I need to check system resources very often in my research work. Many tools out there but none satisfies my requirement. Instead of keep searching in Internet, I wrote my own and I am quite happy of what it does. I called it Sysmon.

So what's Sysmon anyway? In short, Sysmon is a lightweight Linux-based system resource tracing tool, as vmstat does, but has more information. The output includes the CPU/memory usage, swapping/paging, interrupts and each network card's statistics, which includes interrupts to kernel, packets and bytes that received and sent in a specified interval. Sysmon can run as a daemon and log the system information for a long period. Help menu:

Usage: sysmon [-hwWbk] [-i interface-name] [-r repeat] [-t test-interval] [-T test-time]
[-h] Printout help messages.
[-w] Write all results to a file. Disable by default.
[-W] Write statistics of each network device to separate files. Disable by default.
[-b] Background (daemon) mode. Only valid when write option is defined.
[-k] Kill the sysmon background process (daemon). Disable by default.
[-i interface-name] Define the network device name (e.g. eth0). Monitor all if no interface defined.
[-r repeat] Repetition of monitoring. 10 times by default.
[-t test-interval] The interval (sample time) between each tracing in seconds. 2 seconds by default.
[-T test-time] The duration of system monitoring in minutes. Valid only write option defined.
[-o output] Specify the output (log) filename. Implies the write option.
Note: Default logfile has format of host-start-time.log if write option (-w) is defined.

Use "ifconfig" to check the network devices in your computer.
Possible names: eth0, wlan0, elan0, etc (defined by NETNAME in util.h). loop is for loopback address.
Example 1 (Monitor all network devices, very long format): % sysmon
Example 2 (Only monitor eth0): % sysmon -r 10 -t 2 -i eth0
Example 3 (Log every 10 minutes for one week, run as daemon): % sysmon -bw -i eth0 -t 600 -T 10080
Output format:
Network information: [interrupts] [recv-packets] [recv-bytes] [sent-packets] [sent-bytes]

Sysmon is written with C. Its source code can be downloaded here.