Infrastructure Solutions

Introduction

Flexible IO Tester (FIO) is a benchmarking and workload simulation tool for Linux/Unix. The   typical use of FIO is to write a job file matching the I/O load one wants to simulate. FIO is used to write data on both block devices and file systems.

Hardware and Software Used

FIO installation Steps in Linux

• Download the latest version of FIO from following link:
               https://git.kernel.dk/cgit/fio/
  

•  Unzip the file:
  
  
• Install libaio-devel (if not already installed) by using the following command:
  yum install libaio-devel
• Build and Install FIO
  ./configure
  Note: If the following error occurs while building, we need to install “Development Tools” package for resolution.
  yum group install "Development Tools"
  make
  make install
  
  This completes the installation of FIO, to verify:
  
  
• Execute FIO
  
  

How FIO works       

Fio –name=Test_fio_run - -rw=randwrite - -direct=1 - -ioengine=libaio - -bs=4k - -numjobs=1 - -iodepth=32 - -rwmixread=0 - -rwmixwrite=100 - -size=10G - -ramp_time=60 –runtime=120 –group_reporting -filename=/dev/mapper/mpathak

The first step in getting FIO to simulate a desired I/O workload is writing a job file describing that specific setup. A job file may contain any number of threads and/or files – the typical contents of the job file is a global section defining shared parameters, and one or more job sections describing the jobs involved. When running, FIO parses this file and sets everything up as described. The following parameters are specified while executing IO over a block device:

--name:  Need to specify the name and location of the FIO run that you will execute.

--rw = randwrite:  The parameter “–rw” decides whether the IO accesses are issued sequentially or randomly. The following options are available for “–rw”:

read – Sequential read

write – Sequential Write

randread – random reading

randwrite – random writing

readwrite , rw – Mixed, sequential workload

randrw – Mixed Random Workload

--direct=1: The use of ” –direct = 1 ” bypasses the page cache and therefore the memory is no longer used. For the use of asynchronous accesses (e.g., via the library libaio) “–direct” is a prerequisite because the page cache cannot be addressed asynchronously.

--ioengine=libaio: libaio enables asynchronous access from the application level. This allows parallel I/O requests to be issued.

--bs=4K: The desired block size can be passed to the parameter.

--numjobs=1: Specifies the number of processes that each generate the defined workload (default 1).

--iodepth=32: This parameter can be used to specify the number of requests.

--rwmixread, --rwmixwrite: These parameters specify different distributions in mix workloads.

--size: Specifies the size of the block device to be used. If this parameter is not specified, the whole block device is used.

--ramp_time=60: Specifies the time to let things settle in after which the operation starts.

--runtime=120: Specifies the time of the operation to be executed.

--group_reporting: When enabled, this option generates a group report for tests with numjobs> 1 (instead of individual job reports).

--filename: Specifies the block device in which FIO is executed. Multiple number of file names also can be specified by separating the names with a “:” For example:                            -filename=/dev/mapper/mpathak:/dev/mapper/mpathaj

Also, we can specify the file name inside a mount point as follows:

              -filename=/ha_test/randwrite_mpath_a.file

FIO run using job file

$ fio [options] [jobfile] ....

              It will start doing what the jobfile tells it to do. You can give more than one job file on the command line, FIO will serialize the running of those files. Internally that is the same as using the parameter described in the parameter section. Following is an example of the job file options, where each option applies to the current job until a new [] job entry is seen.

; -- start job file --

[random-writers]

ioengine=libaio

iodepth=4

rw=randwrite

bs=32k

direct=0

size=64m

numjobs=4

; -- end job file --

Observations of FIO run

The following is an example of output of an FIO run.

 TestCase1_test.B_RanWrite_2k: (groupid=0, jobs=1): err= 0: pid=9530: Mon Jun 26 15:51:49 2023

 write: IOPS=35.7k, BW=69.8MiB/s (73.2MB/s)(40.9GiB/600001msec)

    slat (usec): min=2, max=400, avg=13.89, stdev= 9.87

    clat (usec): min=371, max=52366, avg=875.71, stdev=225.38

    lat (usec): min=392, max=52376, avg=890.84, stdev=227.08

    clat percentiles (usec):

     |  1.00th=[  553],  5.00th=[  627], 10.00th=[  668], 20.00th=[  709],

     | 30.00th=[  742], 40.00th=[  775], 50.00th=[  807], 60.00th=[  857],

     | 70.00th=[  930], 80.00th=[ 1037], 90.00th=[ 1205], 95.00th=[ 1352],

     | 99.00th=[ 1532], 99.50th=[ 1582], 99.90th=[ 1745], 99.95th=[ 2008],

     | 99.99th=[ 2966]

   bw (  KiB/s): min=66364, max=75636, per=100.00%, avg=71580.38, stdev=1357.20, samples=1198

   iops        : min=33182, max=37818, avg=35790.21, stdev=678.58, samples=1198

  lat (usec)   : 500=0.12%, 750=31.99%, 1000=44.99%

  lat (msec)   : 2=22.85%, 4=0.05%, 10=0.01%, 20=0.01%, 50=0.01%

  lat (msec)   : 100=0.01%

  cpu          : usr=19.50%, sys=57.98%, ctx=7402623, majf=0, minf=66

  IO depths    : 1=0.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=100.0%, >=64=0.0%

     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%

     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%

     issued rwts: total=0,21445480,0,0 short=0,0,0,0 dropped=0,0,0,0

     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):

  WRITE: bw=69.8MiB/s (73.2MB/s), 69.8MiB/s-69.8MiB/s (73.2MB/s-73.2MB/s), io=40.9GiB (43.9GB), run=600001-600001msec

Slat (Submission Latency):  How long it takes to submit this I/O to the kernel for processing. “usec” specifies micro sec.

Clat (Completion Latency): This is the time between submission to kernel and time when I/O is completed. Msec specifies Mili seconds.

Lat (Total Latency): This is the time between I/O creation and its completion. This is the sum of “slat” and “clat.”

 After execution of FIO the following parameters are collected:

 IOPS, BW and Avg Latency (Marked in Yellow in above example)

Command Line options

 By executing “FIO” on the command line, we can get the version of FIO and the options for usage. For more information on the usage of FIO please refer the links provided in the “Reference” section below.

Reference

https://github.com/axboe/fio

https://fio.readthedocs.io/en/latest/fio_doc.html