Dear misc@, *## Intro, environment* Find below a comparative benchmark of OpenBSD 6.0 vs Linux 4.7 read speeds on a 3.3Ghz Xeon E3 server with a Samsung 850 Pro 256GB SATA SSD, which is one of the very fastest SSD:s in the sub-1000USD/TB price range. dmesg below.
No dual-disk case was tested. (To be meaningful, that would need to be done on separate SATA ports, as the 600MB/sec bandwidth is per SATA motherboard/controller board plug.) I would guess OpenBSD's throughput would be equal to the results in this test in a dual-test disk case, whereas Linux would have double throughput due to its per-device multiqueue. *## General characteristics of SSD:s, some aha moments* First maybe I should say that this benchmark gave me some aha moments regarding how SSD:s work in general - SSD specsheets are generally adorned with figures between 500MB/sec for SATA and 1500MB/sec for NVME. In actuality, both SATA and NVME SSD:s do 4K single-thread random reading at no more than approx 45MB/sec (!). But, if you pump their queues - i.e. ask the SSD to do more reads concurrently - then you're suddenly getting into the ~400MB/sec range for 4K reads on SATA, and (I didn't test it but when correlating with other benchmarks I would expect) ~900MB/sec on NVME. SATA does 100,000IOPS and NVME 400,000IOPS. Multithread random 4K reading at 400MB/sec equals 100,000IOPS (400 * 1024 * 1024 divided by 4096 is ~100,000), so we see that my SSD actually saturates the SATA bus, when pumped with multithreaded reads. The NVME benchmark I got hold of showed ~900MB/sec at 4K in the same usecase, meaning that the disk performs at ~55% of the bus speed. This means that while NVME drives have better "multi-processing power" than SATA disks, NVME still has the same disk access latency as SATA (if defined as a seek+read operation): This should mean that paying a quadruple price tag for current NVME drive not is worth it for a multiuser database usercase - for multithreaded random read performance, an NVME drive would be worth max +125% (for its 900MB/sec performance vs. 400MB/sec). But in that case, why not simply buy two SATA disks and enjoy the higher performance but get double the storage volume. Finally, for sequential reads, a SATA SSD will do something-like 500MB/sec and an NVME SSD will do something-like 1500MB/sec, and that's of course the figure that they like to show in advertisements. (There are SSD:s with a higher performance profile in the ~~5000 USD/TB or so price range, I didn't study those.) The NVME benchmark referenced to here was http://ssd.userbenchmark.com/Compare/Samsung-950-NVMe-PCIe-M2-256GB-vs-Samsun g-850-Pro-256GB/m38570vs2385 . Now on to the benchmark: *## Benchmark specs* Measures were taken against bias from buffer cache, scheduler, and filesystem specifics, by disabling/confusing the buffer cache, running in unnice mode, and benchmarking only direct block device reading. The benchmark was performed using the disk_io_benchmark_c.c program, inlined below. Both read() and mmap() modes were tested. *## Observations* The observations are, to sum up: - In multithreaded random reads, Linux is way faster, e.g. by ~7X (OpenBSD runs at ~50MB/sec and Linux ~350MB/sec at 4K). The difference should be only due to Linux' having and OpenBSD's not having multiqueuing. - In singlethreaded random reads, Linux and OpenBSD perform similarly (both ~50MB/sec at 4K). - For singlethreaded sequential reads, Linux is ~5x faster than OpenBSD (OpenBSD ~120MB/sec and Linux ~500MB/sec). - Re mmap() vs read() performance: In OpenBSD, same performance in both sequential and random reads. In Linux, in sequential read mode, same performance except for in <128B reads where read()'s performance dumps faster than mmap-reads. In Linux, in random read mode, mmap() performs disastrously compared to read() - e.g. mmap() is, in singlethreaded mode 2.5x slower, and in multithreaded mode 20x slower, than read() on Linux! *## My comments* My comments: - I have a multithreaded random reading usecase e.g. multiuser database, and would very humbly call for performance improvements here! This will happen through implementing per-device multiqueues. Following up on this in subsequent email. - While I don't have particular need for it, I find it strange that the sequential read speed is so much slower on OpenBSD than on Linux - I mean, I guess a sequential reads are executed sequentially on Linux (as they are on OpenBSD), and I guess lower-level mechanisms like SATA controller drivers and DMA logics should be have similar functioning between Linux and OpenBSD, so how come the steep difference? (OpenBSD's buffer cache can deliver up to 1700MB/sec in mmap and ~500MB/sec in read(), so, OpenBSD's sequential reading speed constraint (of ~120MB/sec vs. Linux' ~500MB/sec) seems to lie in the logics that do actual disk work.) - Would any particular sysctl:s or other kernel settings speed up random or sequential reading? - It's interesting to see that an SSD can be made to deliver so much random read performance simply by using multiqueues (e.g. 10X higher). It reflects that SSD:s as devices are largely parallellized internally. - The OpenBSD buffer cache containing a lot of irrelevant data will not affect read speeds negatively (that is on this AMD64 with its 32bit buffer size limit, Theo said otherwise about Sparc64 though) - I conclude this as benchmarks gave the same results with kern.bufcachepercent=5 and kern.bufcachepercent=90 settings (on this system with 16GB RAM on a 256GB SSD and the benchmark being constructed to minimize buffer cache utilization through random lseek()). Details follow below. Best regards, Mikael *## Detail benchmark results* *# OpenBSD results* System: GENERIC.MP, 6.0, AMD64, dmesg below. Sequential reads (single-threaded) [1] are around 120MB/sec all the way down to 128-byte reads, and below it it dumps quickly (33MB/sec at 64byte reads etc.) [2]. Random reads, single-threaded are: 512B/read [3] is ~5.8MB/sec, 4KB/read [4] is ~45MB/sec, and 32KB/read [5] is ~67MB/sec. read() and mmap() give essentially the same performance in these two tests. Random reads, 10-threaded, 20-threaded and 40-threaded, give the same total throughput as the single-threaded results above i.e. there is no benefit at all in running multiple concurrent IO operations: Random reads, 10-threaded are: 512B/read [6] is 5.0MB/sec total (0.50MB/sec per process), 4KB/read [7] is 44MB/sec (4.4MB/sec per process), and 32KB/read [8] is 72MB/sec (7.2MB/sec per process). Random-read 20-threaded yielded somehow similar results (1.79MB/sec per process meaning 35.8MB/sec at 4KB) [9]. Random-read 40-threaded yielded somehow similar results (0.81MB/sec per process meaning 32.4MB/sec - that's [9] but doubled). We also do a silly "10-threaded sequential read" (with all reads originating at different offsets) [10], yielding 3.9MB/sec per process meaning 39MB/sec total, so that's similar to single-threaded sequential read. *# Linux results* System: "Linux version 4.6.0-kali1-amd64 ([email protected]) (gcc version 5.4.0 20160609 (Debian 5.4.0-6) ) #1 SMP Debian 4.6.4-1kali1 (2016-07-21)", installed from https://images.offensive-security.com/kali-linux-2016.2-amd64.torrent . The first observation on Linux is that mmap() performance for random reads, is WAY below that of read(): Singlethreaded random read: 512B: 1.45MB/sec mmap() vs. 6.38MB/sec read(), 4K: 11MB/sec mmap) vs. 49MB/sec read(), 32KB: 86MB/sec mmap() vs. 230MB/sec read(), 10-threaded 4K random read: 1.79MB/sec per process mmap() vs. 35MB/sec per process read() [0]. In sequential reads on Linux however, mmap() performance is a bit better than that of read() - on >=128byte reads, performance is equal, and on reads below 100 bytes, mmap() access speed is much closer to the 128byte performance profile, than that of read(). Proportionally speaking OpenBSD is faster here. All the following tests are done with read(): Sequential reads (single-threaded) [1] are around 525MB/sec all the way down to 128-byte reads, and below it it dumps quickly (around 340MB/sec at 64byte reads etc.) [2]. Random reads, single-thread are: 512B/read [3] is 6.38MB/sec, 4KB/read [4] is 50MB/sec, and 32KB/read [5] is 231MB/sec. Random reads, 10-threaded are: 512B/read [6] is 46MB/sec total (4.6MB/sec per process), 4KB/read [7] is 357MB/sec (35.7MB/sec per process), and 32KB/read [8] is 528MB/sec (52.8MB/sec per process). Random-read 20-threaded yielded somehow similar results (19.8MB/sec per process meaning 396MB/sec at 4KB) [9]. Random-read 40-threaded yielded somehow similar results (10MB/sec per process meaning 400MB/sec at 4KB - that's [9] but doubled). We also do a silly "10-threaded sequential read" (with all reads originating at different offsets) [10], yielding 53.6MB/sec per process meaning 536MB/sec total, so that's similar to single-threaded sequential read. *## Benchmark commands* The main difference between OpenBSD test and Linux tests is that OpenBSD uses "sd0c" while Linux uses "sda". *# OpenBSD benchmark commands* [1] sysctl kern.bufcachepercent=5 nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sd0c 0 0 4096 4096 [2] sysctl kern.bufcachepercent=5 nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sd0c 0 0 64 64 [3] sysctl kern.bufcachepercent=5 nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sd0c 1 0 512 512 [4] sysctl kern.bufcachepercent=5 nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sd0c 1 0 4096 4096 [5] sysctl kern.bufcachepercent=5 nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sd0c 1 0 32768 32768 [6] sysctl kern.bufcachepercent=5 nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sd0c 1 0 512 512 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sd0c 1 0 512 512 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sd0c 1 0 512 512 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sd0c 1 0 512 512 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sd0c 1 0 512 512 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sd0c 1 0 512 512 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sd0c 1 0 512 512 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sd0c 1 0 512 512 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sd0c 1 0 512 512 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sd0c 1 0 512 512 & [7] sysctl kern.bufcachepercent=5 nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sd0c 1 0 4096 4096 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sd0c 1 0 4096 4096 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sd0c 1 0 4096 4096 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sd0c 1 0 4096 4096 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sd0c 1 0 4096 4096 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sd0c 1 0 4096 4096 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sd0c 1 0 4096 4096 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sd0c 1 0 4096 4096 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sd0c 1 0 4096 4096 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sd0c 1 0 4096 4096 & [8] sysctl kern.bufcachepercent=5 nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sd0c 1 0 32768 32768 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sd0c 1 0 32768 32768 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sd0c 1 0 32768 32768 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sd0c 1 0 32768 32768 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sd0c 1 0 32768 32768 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sd0c 1 0 32768 32768 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sd0c 1 0 32768 32768 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sd0c 1 0 32768 32768 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sd0c 1 0 32768 32768 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sd0c 1 0 32768 32768 & [9] sysctl kern.bufcachepercent=5 nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sd0c 1 0 4096 4096 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sd0c 1 0 4096 4096 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sd0c 1 0 4096 4096 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sd0c 1 0 4096 4096 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sd0c 1 0 4096 4096 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sd0c 1 0 4096 4096 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sd0c 1 0 4096 4096 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sd0c 1 0 4096 4096 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sd0c 1 0 4096 4096 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sd0c 1 0 4096 4096 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sd0c 1 0 4096 4096 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sd0c 1 0 4096 4096 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sd0c 1 0 4096 4096 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sd0c 1 0 4096 4096 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sd0c 1 0 4096 4096 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sd0c 1 0 4096 4096 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sd0c 1 0 4096 4096 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sd0c 1 0 4096 4096 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sd0c 1 0 4096 4096 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sd0c 1 0 4096 4096 & [10] sysctl kern.bufcachepercent=5 nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sd0c 0 0 4096 4096 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sd0c 0 0 4096 4096 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sd0c 0 0 4096 4096 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sd0c 0 0 4096 4096 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sd0c 0 0 4096 4096 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sd0c 0 0 4096 4096 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sd0c 0 0 4096 4096 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sd0c 0 0 4096 4096 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sd0c 0 0 4096 4096 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sd0c 0 0 4096 4096 & *# Linux benchmark commands* [0] sysctl vm.drop_caches=3; nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sda 1 1 512 512 sysctl vm.drop_caches=3; nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sda 1 0 512 512 sysctl vm.drop_caches=3; nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sda 1 1 4096 4096 sysctl vm.drop_caches=3; nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sda 1 0 4096 4096 sysctl vm.drop_caches=3; nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sda 1 1 32768 32768 sysctl vm.drop_caches=3; nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sda 1 0 32768 32768 sysctl vm.drop_caches=3; nice -n -20 ./disk_io_benchmark_c 1 0 /dev/sda 1 0 4096 4096 & \ nice -n -20 ./disk_io_benchmark_c 1 0 /dev/sda 1 0 4096 4096 & \ nice -n -20 ./disk_io_benchmark_c 1 0 /dev/sda 1 0 4096 4096 & \ nice -n -20 ./disk_io_benchmark_c 1 0 /dev/sda 1 0 4096 4096 & \ nice -n -20 ./disk_io_benchmark_c 1 0 /dev/sda 1 0 4096 4096 & \ nice -n -20 ./disk_io_benchmark_c 1 0 /dev/sda 1 0 4096 4096 & \ nice -n -20 ./disk_io_benchmark_c 1 0 /dev/sda 1 0 4096 4096 & \ nice -n -20 ./disk_io_benchmark_c 1 0 /dev/sda 1 0 4096 4096 & \ nice -n -20 ./disk_io_benchmark_c 1 0 /dev/sda 1 0 4096 4096 & \ nice -n -20 ./disk_io_benchmark_c 1 0 /dev/sda 1 0 4096 4096 & [1] sysctl vm.drop_caches=3; nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sda 0 0 4096 4096 [2] sysctl vm.drop_caches=3; nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sda 0 0 64 64 [3] sysctl vm.drop_caches=3; nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sda 1 0 512 512 [4] sysctl vm.drop_caches=3; nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sda 1 0 4096 4096 [5] sysctl vm.drop_caches=3; nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sda 1 0 32768 32768 [6] sysctl vm.drop_caches=3; nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sda 1 0 512 512 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sda 1 0 512 512 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sda 1 0 512 512 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sda 1 0 512 512 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sda 1 0 512 512 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sda 1 0 512 512 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sda 1 0 512 512 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sda 1 0 512 512 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sda 1 0 512 512 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sda 1 0 512 512 & [7] sysctl vm.drop_caches=3; nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sda 1 0 4096 4096 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sda 1 0 4096 4096 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sda 1 0 4096 4096 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sda 1 0 4096 4096 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sda 1 0 4096 4096 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sda 1 0 4096 4096 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sda 1 0 4096 4096 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sda 1 0 4096 4096 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sda 1 0 4096 4096 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sda 1 0 4096 4096 & [8] sysctl vm.drop_caches=3; nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sda 1 0 32768 32768 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sda 1 0 32768 32768 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sda 1 0 32768 32768 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sda 1 0 32768 32768 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sda 1 0 32768 32768 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sda 1 0 32768 32768 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sda 1 0 32768 32768 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sda 1 0 32768 32768 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sda 1 0 32768 32768 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sda 1 0 32768 32768 & [9] sysctl vm.drop_caches=3; nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sda 1 0 4096 4096 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sda 1 0 4096 4096 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sda 1 0 4096 4096 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sda 1 0 4096 4096 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sda 1 0 4096 4096 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sda 1 0 4096 4096 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sda 1 0 4096 4096 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sda 1 0 4096 4096 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sda 1 0 4096 4096 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sda 1 0 4096 4096 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sda 1 0 4096 4096 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sda 1 0 4096 4096 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sda 1 0 4096 4096 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sda 1 0 4096 4096 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sda 1 0 4096 4096 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sda 1 0 4096 4096 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sda 1 0 4096 4096 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sda 1 0 4096 4096 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sda 1 0 4096 4096 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sda 1 0 4096 4096 & [10] sysctl vm.drop_caches=3; nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sda 0 0 4096 4096 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sda 0 0 4096 4096 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sda 0 0 4096 4096 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sda 0 0 4096 4096 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sda 0 0 4096 4096 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sda 0 0 4096 4096 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sda 0 0 4096 4096 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sda 0 0 4096 4096 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sda 0 0 4096 4096 & \ nice -n -20 ./disk_io_benchmark_c 1 1 /dev/sda 0 0 4096 4096 & *## Benchmark program disk_io_benchmark_c.c* /* * * Disk IO benchmarking program. Tests sequential and random reads. Supports both read() and mmap() based IO. * MIT license. * * Sequential reads are done to a random position on the disk, to avoid reading the OS buffer cache and hence getting unauthentic, too good results. * * Currently presumes any block device to be 256gB. * * * Compile: gcc -o disk_io_benchmark_c disk_io_benchmark_c.c * * (Can be pasted over terminal and compiled by: cat > disk_io_benchmark_c.c; gcc -o disk_io_benchmark_c disk_io_benchmark_c.c ) * * Use: ./disk_io_benchmark_c benchmark-duration-seconds benchmark-duration-gigabytes file random[1 = random, 0 = sequential] mmap?[1 = yes, 0 = no] block-size [random read alignment multiple] * * We don't just benchmark per seconds but also per gigabytes read, as the OS' buffer cache generally will kick in so the buffer cache must be an * insignificant part of the read operation, for the test result to be reliable. * * Make 4096-byte random reads via file IO, up to 1GB: * ./disk_io_benchmark_c 1 1 /dev/sd0c 1 0 4096 4096 * * ./disk_io_benchmark_c 1 1 /dev/sd0c 1 1 4096 512 * * ./disk_io_benchmark_c 1 1 /dev/sd0c 0 0 4096 * { ./disk_io_benchmark_c 1 15 /dev/sd0c 1 0 4096 ./disk_io_benchmark_c 1 15 /dev/sd0c 0 0 4096 ./disk_io_benchmark_c 1 15 /dev/sd0c 1 0 32096 ./disk_io_benchmark_c 1 15 /dev/sd0c 0 0 32096 } */ #include <stdio.h> // For malloc (and arc4random which we don't use): #include <stdlib.h> // For uint64_t: #include <stdint.h> // For ability to printf uint64_t via PRIu64: #include <inttypes.h> // For struct timespec and CLOCK_MONOTONIC: #include <time.h> // For the errno variable: #include <errno.h> // For the bool type: #include <stdbool.h> // For open etc.: #include <fcntl.h> // For mmap: #include <sys/mman.h> // For memcpy: #include <string.h> // For lseek on Linux too: #include <sys/types.h> #include <unistd.h> uint64_t rand64() { return (((uint64_t) rand()) << 32) + ((uint64_t) rand()); } int main(int argc, char *argv[]) { // Use the nanosecond time within the second as random seed. struct timespec randomseedtime; clock_gettime(CLOCK_MONOTONIC,&randomseedtime); // printf("Random seed is %i\n.",randomseedtime.tv_nsec); srand(randomseedtime.tv_nsec); if (argc != 8) { printf("Bad args.\n"); return -1; } int duration = atoi(argv[1]); uint64_t durationGB = atoi(argv[2]); uint64_t durationB = durationGB * 1024 * 1024 * 1024; char* filename = argv[3]; bool random = atoi(argv[4]); bool dommap = atoi(argv[5]); int blocksize = atoi(argv[6]); uint64_t randomreadalignment = atoi(argv[7]); struct timespec start,at; clock_gettime(CLOCK_MONOTONIC,&start); printf("Reading %i byte blocks from %s for %i seconds/%i GB, %s (random alignment %i), %s.\n", blocksize,filename,duration,durationGB,random ? "randomly" : "sequentially",randomreadalignment,dommap ? "memory mapped" : "via file IO"); int f = open(filename,O_RDONLY); if (f == -1) { printf("Failed to open file %s!\n",filename); return -1; } if (lseek(f,0,SEEK_END) == -1) { // doesn't exist: fsize(f); printf("Failed to seek end of file! (%i)\n",errno); return -1; } uint64_t totalsize = lseek(f,0,SEEK_CUR); if (totalsize == 0) { // printf("I guess this is a block device, treating as 256GB. "); totalsize = 256 * 1000 * 1000; // bytes -> kbytes -> mbytes totalsize *= 1000; // -> gbytes } printf("File is %" PRIu64 " bytes in size (%f GB).\n",totalsize,((float) totalsize / 1024 / 1024 / 1024)); lseek(f,0,SEEK_SET); void* mmaparea = NULL; // = dommap ? mmap() : NULL; if (dommap) { mmaparea = mmap(NULL /* addr - we don't provide any suggestion. */, totalsize, PROT_READ, MAP_SHARED, /* flags - MAP_FILE implied */ f, 0 // offset ); if (mmaparea == MAP_FAILED) { printf("mmap failed, errno = %i.\n",errno); return -1; } } void* buf = malloc(blocksize); int reads = 0; uint64_t atpos = 0; uint64_t totalReadBytes = 0; uint64_t microsecondsspent; do { // Jump, if applicable' // // We start any sequential read with a random jump too, this is to minimize OS buffer cache impact. // Also when reaching the end, we make a random jump. if (random || (atpos == 0)) { bool frombeginning = atpos == 0; uint64_t offset = rand64() % (totalsize - blocksize); uint64_t disksectorsize = 256; atpos = offset = (offset / randomreadalignment) * randomreadalignment; if (frombeginning) printf("Reading from position %" PRIu64 " (%f GB).\n",offset, ((float) offset / 1024 / 1024 / 1024)); if (!dommap) { if (lseek(f, offset, // offset SEEK_SET // whence - relative to the beginning of the file ) == -1) { printf("Failed to make random jump! (%i)\n",errno); return -1; } } } // Make read int r; if (!dommap) { r = read(f, buf, blocksize // nbytes ); } else { void* from = mmaparea + atpos; // printf("mmap-reading from %" PRIu64 ".\n",from); memcpy( buf, from, blocksize ); r = blocksize; } if (r != blocksize) { printf("Failed to read! (asked for %i, result = %i, errno = %i)\n",blocksize,r,errno); return -1; } reads++; atpos += r; totalReadBytes += r; if (!dommap) { if ( atpos // lseek(f,0,SEEK_CUR) > totalsize - blocksize ) { lseek(f,0,SEEK_SET); atpos = 0; // printf("Rewinding file as we reached its end.\n"); } } // Count down clock_gettime(CLOCK_MONOTONIC,&at); microsecondsspent = (at.tv_sec - start.tv_sec) * 1000000 + (at.tv_nsec - start.tv_nsec) / 1000; // printf("Time passed, seconds: %f\n",((float) microsecondsspent) / 1000 / 1000); } while ( !( ( // Milliseconds since start microsecondsspent > ((uint64_t) duration) * 1000000 ) && (totalReadBytes > durationB ) ) ); printf(// "Made %i reads in %i seconds, meaning " "%f MB in total, %i reads/sec and %f MB/sec.\n", // reads, // duration, ((float) reads) * ((float) blocksize) / 1024 / 1024, // Read operations/sec: (int) (((float) reads) / ((float) microsecondsspent / 1000 / 1000)), // MB/sec: // (reads * blocksize should equal totalReadBytes) ((float) totalReadBytes / 1024 / 1024) / ((float) microsecondsspent / 1000 / 1000) ); return 0; } *## dmesg* OpenBSD 6.0 (GENERIC.MP) #2319: Tue Jul 26 13:00:43 MDT 2016 [email protected]:/usr/src/sys/arch/amd64/compile/GENERIC.MP real mem = 17121378304 (16328MB) avail mem = 16598011904 (15829MB) mpath0 at root scsibus0 at mpath0: 256 targets mainbus0 at root bios0 at mainbus0: SMBIOS rev. 2.8 @ 0xde17a018 (20 entries) bios0: vendor American Megatrends Inc. version "P3.30" date 06/04/2015 bios0: ASRock E3C226D2I acpi0 at bios0: rev 2 acpi0: sleep states S0 S4 S5 acpi0: tables DSDT FACP APIC FPDT FIDT SSDT AAFT SSDT SSDT MCFG PRAD HPET SSDT SSDT SPMI UEFI DMAR EINJ ERST HEST BERT acpi0: wakeup devices PEGP(S4) PEG0(S4) PEGP(S4) PEG1(S4) PEGP(S4) PEG2(S4) CIR_(S4) PS2K(S4) PS2M(S4) UR11(S4) UR12(S4) PXSX(S4) BR11(S4) RP01(S4) PXSX(S4) RP02(S4) [...] acpitimer0 at acpi0: 3579545 Hz, 24 bits acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat cpu0 at mainbus0: apid 0 (boot processor) cpu0: Intel(R) Xeon(R) CPU E3-1230 v3 @ 3.30GHz, 3293.27 MHz cpu0: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUS H,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX ,SMX,EST,TM2,SSSE3,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT ,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,LONG,LAHF,ABM,PERF,ITSC,FSGSB ASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,SENSOR,ARAT cpu0: 256KB 64b/line 8-way L2 cache cpu0: smt 0, core 0, package 0 mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges cpu0: apic clock running at 99MHz cpu0: mwait min=64, max=64, C-substates=0.2.1.2.4, IBE cpu1 at mainbus0: apid 2 (application processor) cpu1: Intel(R) Xeon(R) CPU E3-1230 v3 @ 3.30GHz, 3292.66 MHz cpu1: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUS H,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX ,SMX,EST,TM2,SSSE3,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT ,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,LONG,LAHF,ABM,PERF,ITSC,FSGSB ASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,SENSOR,ARAT cpu1: 256KB 64b/line 8-way L2 cache cpu1: smt 0, core 1, package 0 cpu2 at mainbus0: apid 4 (application processor) cpu2: Intel(R) Xeon(R) CPU E3-1230 v3 @ 3.30GHz, 3292.66 MHz cpu2: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUS H,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX ,SMX,EST,TM2,SSSE3,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT ,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,LONG,LAHF,ABM,PERF,ITSC,FSGSB ASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,SENSOR,ARAT cpu2: 256KB 64b/line 8-way L2 cache cpu2: smt 0, core 2, package 0 cpu3 at mainbus0: apid 6 (application processor) cpu3: Intel(R) Xeon(R) CPU E3-1230 v3 @ 3.30GHz, 3292.66 MHz cpu3: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUS H,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX ,SMX,EST,TM2,SSSE3,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT ,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,LONG,LAHF,ABM,PERF,ITSC,FSGSB ASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,SENSOR,ARAT cpu3: 256KB 64b/line 8-way L2 cache cpu3: smt 0, core 3, package 0 cpu4 at mainbus0: apid 1 (application processor) cpu4: Intel(R) Xeon(R) CPU E3-1230 v3 @ 3.30GHz, 3292.66 MHz cpu4: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUS H,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX ,SMX,EST,TM2,SSSE3,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT ,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,LONG,LAHF,ABM,PERF,ITSC,FSGSB ASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,SENSOR,ARAT cpu4: 256KB 64b/line 8-way L2 cache cpu4: smt 1, core 0, package 0 cpu5 at mainbus0: apid 3 (application processor) cpu5: Intel(R) Xeon(R) CPU E3-1230 v3 @ 3.30GHz, 3292.66 MHz cpu5: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUS H,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX ,SMX,EST,TM2,SSSE3,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT ,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,LONG,LAHF,ABM,PERF,ITSC,FSGSB ASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,SENSOR,ARAT cpu5: 256KB 64b/line 8-way L2 cache cpu5: smt 1, core 1, package 0 cpu6 at mainbus0: apid 5 (application processor) cpu6: Intel(R) Xeon(R) CPU E3-1230 v3 @ 3.30GHz, 3292.66 MHz cpu6: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUS H,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX ,SMX,EST,TM2,SSSE3,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT ,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,LONG,LAHF,ABM,PERF,ITSC,FSGSB ASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,SENSOR,ARAT cpu6: 256KB 64b/line 8-way L2 cache cpu6: smt 1, core 2, package 0 cpu7 at mainbus0: apid 7 (application processor) cpu7: Intel(R) Xeon(R) CPU E3-1230 v3 @ 3.30GHz, 3292.66 MHz cpu7: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUS H,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX ,SMX,EST,TM2,SSSE3,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT ,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,LONG,LAHF,ABM,PERF,ITSC,FSGSB ASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,SENSOR,ARAT cpu7: 256KB 64b/line 8-way L2 cache cpu7: smt 1, core 3, package 0 ioapic0 at mainbus0: apid 8 pa 0xfec00000, version 20, 24 pins acpimcfg0 at acpi0 addr 0xf8000000, bus 0-63 acpihpet0 at acpi0: 14318179 Hz acpiprt0 at acpi0: bus 0 (PCI0) acpiprt1 at acpi0: bus -1 (PEG0) acpiprt2 at acpi0: bus -1 (PEG1) acpiprt3 at acpi0: bus -1 (PEG2) acpiprt4 at acpi0: bus 1 (RP01) acpiprt5 at acpi0: bus -1 (RP02) acpiprt6 at acpi0: bus 2 (RP03) acpiprt7 at acpi0: bus 3 (RP04) acpiprt8 at acpi0: bus -1 (RP05) acpiprt9 at acpi0: bus -1 (RP06) acpiprt10 at acpi0: bus -1 (RP07) acpiprt11 at acpi0: bus 4 (RP08) acpiprt12 at acpi0: bus 5 (BR10) acpiec0 at acpi0: not present acpicpu0 at acpi0: C2(200@148 mwait.1@0x31), C1(1000@1 mwait.1), PSS acpicpu1 at acpi0: C2(200@148 mwait.1@0x31), C1(1000@1 mwait.1), PSS acpicpu2 at acpi0: C2(200@148 mwait.1@0x31), C1(1000@1 mwait.1), PSS acpicpu3 at acpi0: C2(200@148 mwait.1@0x31), C1(1000@1 mwait.1), PSS acpicpu4 at acpi0: C2(200@148 mwait.1@0x31), C1(1000@1 mwait.1), PSS acpicpu5 at acpi0: C2(200@148 mwait.1@0x31), C1(1000@1 mwait.1), PSS acpicpu6 at acpi0: C2(200@148 mwait.1@0x31), C1(1000@1 mwait.1), PSS acpicpu7 at acpi0: C2(200@148 mwait.1@0x31), C1(1000@1 mwait.1), PSS acpipwrres0 at acpi0: PG00, resource for PEG0 acpipwrres1 at acpi0: PG01, resource for PEG1 acpipwrres2 at acpi0: PG02, resource for PEG2 acpipwrres3 at acpi0: FN00, resource for FAN0 acpipwrres4 at acpi0: FN01, resource for FAN1 acpipwrres5 at acpi0: FN02, resource for FAN2 acpipwrres6 at acpi0: FN03, resource for FAN3 acpipwrres7 at acpi0: FN04, resource for FAN4 acpitz0 at acpi0: critical temperature is 105 degC acpitz1 at acpi0: critical temperature is 105 degC "INT3F0D" at acpi0 not configured "PNP0303" at acpi0 not configured "PNP0501" at acpi0 not configured "PNP0501" at acpi0 not configured "IPI0001" at acpi0 not configured acpibtn0 at acpi0: PWRB acpibtn1 at acpi0: SLPB "PNP0C0B" at acpi0 not configured "PNP0C0B" at acpi0 not configured "PNP0C0B" at acpi0 not configured "PNP0C0B" at acpi0 not configured "PNP0C0B" at acpi0 not configured acpivideo0 at acpi0: GFX0 acpivout0 at acpivideo0: DD1F cpu0: Enhanced SpeedStep 3293 MHz: speeds: 3301, 3300, 3100, 2900, 2800, 2600, 2400, 2200, 2000, 1900, 1700, 1500, 1300, 1200, 1000, 800 MHz pci0 at mainbus0 bus 0 pchb0 at pci0 dev 0 function 0 "Intel Xeon E3-1200 v3 Host" rev 0x06 xhci0 at pci0 dev 20 function 0 "Intel 8 Series xHCI" rev 0x05: msi usb0 at xhci0: USB revision 3.0 uhub0 at usb0 "Intel xHCI root hub" rev 3.00/1.00 addr 1 "Intel 8 Series MEI" rev 0x04 at pci0 dev 22 function 0 not configured "Intel 8 Series MEI" rev 0x04 at pci0 dev 22 function 1 not configured ehci0 at pci0 dev 26 function 0 "Intel 8 Series USB" rev 0x05: apic 8 int 16 usb1 at ehci0: USB revision 2.0 uhub1 at usb1 "Intel EHCI root hub" rev 2.00/1.00 addr 1 ppb0 at pci0 dev 28 function 0 "Intel 8 Series PCIE" rev 0xd5: msi pci1 at ppb0 bus 1 ppb1 at pci0 dev 28 function 2 "Intel 8 Series PCIE" rev 0xd5: msi pci2 at ppb1 bus 2 em0 at pci2 dev 0 function 0 "Intel I210" rev 0x03: msi, address d0:50:99:78:d0:6f ppb2 at pci0 dev 28 function 3 "Intel 8 Series PCIE" rev 0xd5: msi pci3 at ppb2 bus 3 em1 at pci3 dev 0 function 0 "Intel I210" rev 0x03: msi, address d0:50:99:78:d0:70 ppb3 at pci0 dev 28 function 7 "Intel 8 Series PCIE" rev 0xd5: msi pci4 at ppb3 bus 4 ppb4 at pci4 dev 0 function 0 "ASPEED Technology AST1150 PCI" rev 0x02 pci5 at ppb4 bus 5 vga1 at pci5 dev 0 function 0 "ASPEED Technology AST2000" rev 0x21 wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation) wsdisplay0: screen 1-5 added (80x25, vt100 emulation) ehci1 at pci0 dev 29 function 0 "Intel 8 Series USB" rev 0x05: apic 8 int 22 usb2 at ehci1: USB revision 2.0 uhub2 at usb2 "Intel EHCI root hub" rev 2.00/1.00 addr 1 pcib0 at pci0 dev 31 function 0 "Intel C226 LPC" rev 0x05 ahci0 at pci0 dev 31 function 2 "Intel 8 Series AHCI" rev 0x05: msi, AHCI 1.3 ahci0: port 1: 6.0Gb/s scsibus1 at ahci0: 32 targets sd0 at scsibus1 targ 1 lun 0: <ATA, Samsung SSD 850, EXM0> SCSI3 0/direct fixed naa.5002538840030944 sd0: 244198MB, 512 bytes/sector, 500118192 sectors, thin ichiic0 at pci0 dev 31 function 3 "Intel 8 Series SMBus" rev 0x05: apic 8 int 18 iic0 at ichiic0 sdtemp0 at iic0 addr 0x18: mcp98243 sdtemp1 at iic0 addr 0x1a: mcp98243 spdmem0 at iic0 addr 0x50: 8GB DDR3 SDRAM ECC PC3-12800 with thermal sensor spdmem1 at iic0 addr 0x52: 8GB DDR3 SDRAM ECC PC3-12800 with thermal sensor isa0 at pcib0 isadma0 at isa0 com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo com1 at isa0 port 0x2f8/8 irq 3: ns16550a, 16 byte fifo pckbc0 at isa0 port 0x60/5 irq 1 irq 12 pckbd0 at pckbc0 (kbd slot) wskbd0 at pckbd0: console keyboard, using wsdisplay0 pcppi0 at isa0 port 0x61 spkr0 at pcppi0 wbsio0 at isa0 port 0x2e/2: NCT6776F rev 0x33 lm1 at wbsio0 port 0x290/8: NCT6776F uhub3 at uhub0 port 1 "American Megatrends Inc. Virtual Hub" rev 2.00/1.00 addr 2 uhidev0 at uhub3 port 4 configuration 1 interface 0 "American Megatrends Inc. Virtual Keyboard and Mouse" rev 1.10/1.00 addr 3 uhidev0: iclass 3/1 ukbd0 at uhidev0: 8 variable keys, 6 key codes wskbd1 at ukbd0 mux 1 wskbd1: connecting to wsdisplay0 uhidev1 at uhub3 port 4 configuration 1 interface 1 "American Megatrends Inc. Virtual Keyboard and Mouse" rev 1.10/1.00 addr 3 uhidev1: iclass 3/1 ums0 at uhidev1: 3 buttons, Z dir wsmouse0 at ums0 mux 0 uhub4 at uhub1 port 1 "Intel Rate Matching Hub" rev 2.00/0.05 addr 2 uhub5 at uhub2 port 1 "Intel Rate Matching Hub" rev 2.00/0.05 addr 2 uhub5: device problem, disabling port 6 vscsi0 at root scsibus2 at vscsi0: 256 targets softraid0 at root scsibus3 at softraid0: 256 targets sd1 at scsibus3 targ 1 lun 0: <OPENBSD, SR, 006> SCSI2 0/direct fixed sd1: 243163MB, 512 bytes/sector, 497998407 sectors root on sd1a (6688557e6aa6208d.a) swap on sd1b dump on sd1b
