ふと、F2FSだとパフォーマンスが上がるのか？と思ったのでベンチマークしてみた。ext4のファイルシステム上にあるloopファイルでやったので、かなり誤差を含んでいると思う。
ベンチマーク結果
ext4
- write: IOPS=652, BW=2612KiB/s
- write: IOPS=378, BW=1514KiB/s
F2FS
- write: IOPS=1155, BW=4622KiB/s
- write: IOPS=675, BW=2703KiB/s
- write: IOPS=1180, BW=4721KiB/s
[akimasa@alarmpi ~]$ rm testdisk
[akimasa@alarmpi ~]$ truncate -s1G testdisk
[akimasa@alarmpi ~]$ mkfs.ext4 testdisk
mke2fs 1.44.5 (15-Dec-2018)
Discarding device blocks: done
Creating filesystem with 262144 4k blocks and 65536 inodes
Filesystem UUID: 732ef760-3977-4470-8586-cb53b3bcf40d
Superblock backups stored on blocks:
32768, 98304, 163840, 229376
Allocating group tables: done
Writing inode tables: done
Creating journal (8192 blocks): done
Writing superblocks and filesystem accounting information: done
[akimasa@alarmpi ~]$ sudo mount testdisk mountpoint/
[akimasa@alarmpi ~]$ cd mountpoint/
[akimasa@alarmpi mountpoint]$ sudo fio -filename=./testtmp -direct=1 -rw=randwrite -bs=4k -size=800M -numjobs=16 -runtime=20 -group_reporting -name=file1
file1: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1
…
fio-3.13
Starting 16 processes
file1: Laying out IO file (1 file / 800MiB)
Jobs: 13 (f=13): [w(1),(1),w(4),(1),w(2),(1),w(6)][26.5%][w=12KiB/s][w=3 IOPSJobs: 13 (f=13): [w(1),(1),w(4),(1),w(2),(1),w(6)][27.7%][eta 01m:00s] Jobs: 11 (f=11): [w(1),(2),w(3),(1),w(2),(1),w(3),(1),w(2)][18.9%][w=7KiB/s]Jobs: 11 (f=11): [w(1),(2),w(3),(1),w(2),(1),w(3),(1),w(2)][19.7%][eta 01m:4Jobs: 11 (f=11): [w(1),(2),w(3),(1),w(2),(1),w(3),(1),w(2)][20.5%][eta 01m:4Jobs: 11 (f=11): [w(1),(2),w(3),(1),w(2),(1),w(3),(1),w(2)][21.3%][eta 01m:4Jobs: 11 (f=11): [w(1),(2),w(3),(1),w(2),(1),w(3),(1),w(2)][22.0%][eta 01m:3Jobs: 6 (f=6): [(5),w(1),(2),w(1),(1),w(3),(2),w(1)][10.9%][w=20KiB/s][w=5 IOPS][eta 03m:57s]
file1: (groupid=0, jobs=16): err= 0: pid=507: Sun Mar 10 21:39:42 2019
write: IOPS=652, BW=2612KiB/s (2675kB/s)(75.1MiB/29460msec); 0 zone resets
clat (usec): min=74, max=10535k, avg=22279.36, stdev=317342.34
lat (usec): min=76, max=10535k, avg=22281.62, stdev=317342.36
clat percentiles (msec):
| 1.00th=[ 3], 5.00th=[ 3], 10.00th=[ 3], 20.00th=[ 3],
| 30.00th=[ 3], 40.00th=[ 3], 50.00th=[ 3], 60.00th=[ 3],
| 70.00th=[ 4], 80.00th=[ 6], 90.00th=[ 6], 95.00th=[ 6],
| 99.00th=[ 10], 99.50th=[ 651], 99.90th=[ 5403], 99.95th=[ 9463],
| 99.99th=[10537]
bw ( KiB/s): min= 160, max=21141, per=100.00%, avg=9711.92, stdev=420.95, samples=254
iops : min= 40, max= 5285, avg=2427.36, stdev=105.25, samples=254
lat (usec) : 100=0.01%, 250=0.15%, 500=0.18%, 750=0.04%, 1000=0.01%
lat (msec) : 4=71.01%, 10=27.64%, 20=0.06%, 50=0.01%, 100=0.09%
lat (msec) : 250=0.15%, 500=0.03%, 750=0.17%, 1000=0.01%
cpu : usr=0.12%, sys=1.23%, ctx=38554, majf=0, minf=342
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,19237,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
WRITE: bw=2612KiB/s (2675kB/s), 2612KiB/s-2612KiB/s (2675kB/s-2675kB/s), io=75.1MiB (78.8MB), run=29460-29460msec
Disk stats (read/write):
loop0: ios=0/19864, merge=0/0, ticks=0/96576, in_queue=81260, util=89.63%
[akimasa@alarmpi mountpoint]$ sudo fio -filename=./testtmp -direct=1 -rw=randwrite -bs=4k -size=800M -numjobs=16 -runtime=20 -group_reporting -name=file1
file1: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1
…
fio-3.13
Starting 16 processes
Jobs: 14 (f=14): [w(9),(1),w(4),(1),w(1)][37.5%][w=8KiB/s][w=2 IOPS][eta 00m:4Jobs: 14 (f=14): [w(9),(1),w(4),(1),w(1)][38.9%][eta 00m:44s] Jobs: 14 (f=14): [w(9),(1),w(4),(1),w(1)][47.2%][eta 00m:38s]
file1: (groupid=0, jobs=16): err= 0: pid=533: Sun Mar 10 21:40:29 2019
write: IOPS=378, BW=1514KiB/s (1550kB/s)(50.8MiB/34351msec); 0 zone resets
clat (usec): min=59, max=15945k, avg=40995.53, stdev=639574.37
lat (usec): min=60, max=15945k, avg=40998.16, stdev=639574.36
clat percentiles (usec):
| 1.00th=[ 1123], 5.00th=[ 1156], 10.00th=[ 1172],
| 20.00th=[ 1205], 30.00th=[ 1237], 40.00th=[ 1434],
| 50.00th=[ 2212], 60.00th=[ 2245], 70.00th=[ 2311],
| 80.00th=[ 2376], 90.00th=[ 2507], 95.00th=[ 2573],
| 99.00th=[ 6063], 99.50th=[ 608175], 99.90th=[15904801],
| 99.95th=[15904801], 99.99th=[15904801]
bw ( KiB/s): min= 144, max=32681, per=100.00%, avg=12826.52, stdev=835.84, samples=130
iops : min= 36, max= 8168, avg=3205.88, stdev=208.95, samples=130
lat (usec) : 100=0.12%, 250=0.56%, 500=0.07%, 750=0.01%
lat (msec) : 2=42.50%, 4=55.51%, 10=0.35%, 20=0.21%, 50=0.05%
lat (msec) : 750=0.15%
cpu : usr=0.06%, sys=0.26%, ctx=25993, majf=0, minf=331
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,12999,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
WRITE: bw=1514KiB/s (1550kB/s), 1514KiB/s-1514KiB/s (1550kB/s-1550kB/s), io=50.8MiB (53.2MB), run=34351-34351msec
Disk stats (read/write):
loop0: ios=0/12999, merge=0/0, ticks=0/63750, in_queue=67480, util=97.72%
[akimasa@alarmpi mountpoint]$ cd
[akimasa@alarmpi ~]$ sudo umount /home/akimasa/mountpoint
[akimasa@alarmpi ~]$ mkfs.f2fs -f testdisk
F2FS-tools: mkfs.f2fs Ver: 1.12.0 (2018-11-21)
Info: Disable heap-based policy
Info: Debug level = 0
Info: Trim is enabled
Info: Segments per section = 1
Info: Sections per zone = 1
Info: sector size = 512
Info: total sectors = 2097152 (1024 MB)
Info: zone aligned segment0 blkaddr: 512
Info: format version with
"Linux version 4.19.25-2-ARCH (builduser@leming) (gcc version 8.2.1 20181127 (GCC)) #1 SMP PREEMPT Tue Feb 26 01:20:52 UTC 2019"
Info: [testdisk] Discarding device
Info: Overprovision ratio = 6.360%
Info: Overprovision segments = 68 (GC reserved = 39)
Info: format successful
(reverse-i-search)`s': gst-launch-1.0 souphttpsrc location="http://192.168.0.9:40772/api/channels/GR/27/services/1024/stream" ! queue ! t^Cemux ! queue ! faad ! queue ! audioconvert ! audioresample ! queue ! opusenc ! oggmux ! tcpserversink port=10000 host=192.168.0.25
[akimasa@alarmpi ~]$ sudo mount testdisk mountpoint/
[akimasa@alarmpi ~]$ cd mountpoint/
[akimasa@alarmpi mountpoint]$ sudo fio -filename=./testtmp -direct=1 -rw=randwrite -bs=4k -size=800M -numjobs=16 -runtime=20 -group_reporting -name=file1
file1: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1
…
fio-3.13
Starting 16 processes
file1: Laying out IO file (1 file / 800MiB)
Jobs: 16 (f=16): [w(16)][0.9%][eta 46m:07s]
file1: (groupid=0, jobs=16): err= 0: pid=571: Sun Mar 10 21:41:53 2019
write: IOPS=1155, BW=4622KiB/s (4733kB/s)(120MiB/26665msec); 0 zone resets
clat (usec): min=62, max=12800k, avg=13822.79, stdev=374463.88
lat (usec): min=63, max=12800k, avg=13825.05, stdev=374463.89
clat percentiles (usec):
| 1.00th=[ 163], 5.00th=[ 1188], 10.00th=[ 1205],
| 20.00th=[ 1221], 30.00th=[ 1237], 40.00th=[ 1270],
| 50.00th=[ 1319], 60.00th=[ 1418], 70.00th=[ 2343],
| 80.00th=[ 2442], 90.00th=[ 2573], 95.00th=[ 2671],
| 99.00th=[ 2999], 99.50th=[ 4817], 99.90th=[10267657],
| 99.95th=[12817794], 99.99th=[12817794]
bw ( KiB/s): min= 398, max=43200, per=100.00%, avg=27369.00, stdev=856.03, samples=144
iops : min= 98, max=10800, avg=6841.78, stdev=214.04, samples=144
lat (usec) : 100=0.72%, 250=0.45%, 500=0.05%, 750=0.01%, 1000=0.01%
lat (msec) : 2=64.20%, 4=33.95%, 10=0.40%, 50=0.05%, 250=0.05%
cpu : usr=0.18%, sys=0.72%, ctx=61631, majf=0, minf=478
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,30809,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
WRITE: bw=4622KiB/s (4733kB/s), 4622KiB/s-4622KiB/s (4733kB/s-4733kB/s), io=120MiB (126MB), run=26665-26665msec
Disk stats (read/write):
loop0: ios=0/30793, merge=0/0, ticks=0/12368, in_queue=25030, util=93.83%
[akimasa@alarmpi mountpoint]$ sudo fio -filename=./testtmp -direct=1 -rw=randwrite -bs=4k -size=800M -numjobs=16 -runtime=20 -group_reporting -name=file1
file1: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1
…
fio-3.13
Starting 16 processes
Jobs: 16 (f=16): [w(16)][0.5%][eta 01h:19m:15s]
file1: (groupid=0, jobs=16): err= 0: pid=591: Sun Mar 10 21:42:23 2019
write: IOPS=675, BW=2703KiB/s (2768kB/s)(59.3MiB/22466msec); 0 zone resets
clat (usec): min=59, max=20792k, avg=23623.54, stdev=673819.89
lat (usec): min=61, max=20792k, avg=23626.06, stdev=673819.87
clat percentiles (usec):
| 1.00th=[ 1074], 5.00th=[ 1106], 10.00th=[ 1123],
| 20.00th=[ 1139], 30.00th=[ 1156], 40.00th=[ 1205],
| 50.00th=[ 1418], 60.00th=[ 2180], 70.00th=[ 2245],
| 80.00th=[ 2311], 90.00th=[ 2442], 95.00th=[ 2540],
| 99.00th=[ 2835], 99.50th=[ 3425], 99.90th=[17112761],
| 99.95th=[17112761], 99.99th=[17112761]
bw ( KiB/s): min=10432, max=42238, per=100.00%, avg=30327.00, stdev=753.64, samples=64
iops : min= 2608, max=10558, avg=7581.00, stdev=188.38, samples=64
lat (usec) : 100=0.14%, 250=0.47%, 500=0.05%, 750=0.01%, 1000=0.01%
lat (msec) : 2=51.43%, 4=47.53%, 10=0.26%
cpu : usr=0.10%, sys=0.43%, ctx=30358, majf=0, minf=319
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,15183,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
WRITE: bw=2703KiB/s (2768kB/s), 2703KiB/s-2703KiB/s (2768kB/s-2768kB/s), io=59.3MiB (62.2MB), run=22466-22466msec
Disk stats (read/write):
loop0: ios=0/15177, merge=0/0, ticks=0/32370, in_queue=32390, util=95.38%
[akimasa@alarmpi mountpoint]$ sudo fio -filename=./testtmp -direct=1 -rw=randwrite -bs=4k -size=800M -numjobs=16 -runtime=20 -group_reporting -name=file1
file1: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1
…
fio-3.13
Starting 16 processes
Jobs: 16 (f=16): [w(16)][1.0%][eta 44m:38s]
file1: (groupid=0, jobs=16): err= 0: pid=610: Sun Mar 10 21:42:57 2019
write: IOPS=1180, BW=4721KiB/s (4834kB/s)(124MiB/26802msec); 0 zone resets
clat (usec): min=61, max=11993k, avg=13529.91, stdev=358536.31
lat (usec): min=62, max=11993k, avg=13532.38, stdev=358536.32
clat percentiles (usec):
| 1.00th=[ 1045], 5.00th=[ 1074], 10.00th=[ 1106],
| 20.00th=[ 1139], 30.00th=[ 1156], 40.00th=[ 1221],
| 50.00th=[ 2008], 60.00th=[ 2212], 70.00th=[ 2278],
| 80.00th=[ 2343], 90.00th=[ 2442], 95.00th=[ 2540],
| 99.00th=[ 2868], 99.50th=[ 3458], 99.90th=[10536092],
| 99.95th=[12012487], 99.99th=[12012487]
bw ( KiB/s): min= 4896, max=46156, per=100.00%, avg=25290.80, stdev=934.24, samples=160
iops : min= 1224, max=11538, avg=6322.20, stdev=233.59, samples=160
lat (usec) : 100=0.17%, 250=0.30%, 500=0.03%, 750=0.01%
lat (msec) : 2=49.50%, 4=49.51%, 10=0.28%, 500=0.10%
cpu : usr=0.18%, sys=0.77%, ctx=63277, majf=0, minf=436
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,31634,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
WRITE: bw=4721KiB/s (4834kB/s), 4721KiB/s-4721KiB/s (4834kB/s-4834kB/s), io=124MiB (130MB), run=26802-26802msec
Disk stats (read/write):
loop0: ios=0/31618, merge=0/0, ticks=0/14719, in_queue=25080, util=93.89%
[akimasa@alarmpi mountpoint]$ cd
[akimasa@alarmpi ~]$ sudo umount /home/akimasa/mountpoint
[akimasa@alarmpi ~]$