After a lot of tests on AWS with generic and AWS kernels and on non-AWS bare-metal systems with the AWS and generic kernels I discovered that one gets I/O throttling on AWS instances regardless of using ZFS or not.
I ran a really simple test writing and reading to the raw device and found the *same* I/O throttling characteristics. This is due to a feature known as "EBS I/O throttling", see: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-io-characteristics.html and https://aws.amazon.com/ebs/features/ This is not a zfs issue per se, or a kernel issue, but due to I/O throttling with the cloud provider. Marking this as won't fix. ** Changed in: zfs-linux (Ubuntu) Status: Incomplete => Won't Fix -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to zfs-linux in Ubuntu. https://bugs.launchpad.net/bugs/1883676 Title: ZFS performance drops suddenly Status in zfs-linux package in Ubuntu: Won't Fix Bug description: Run Phoronix Flexible IO / Random write benchmark. After running the benchmark repeatedly multiple times (occasionally it can be 30 minutes or 8 hours), the performance drops suddenly. The only way (I know of) to get back to high performance is reboot. --- $ for i in $(seq 1 1000); do printf "2\n4\n2\n1\n1\n1\nn\n" | phoronix-test-suite run fio | awk '/Average.*IOPS/ {print $2}' >> /tmp/iops; done $ less /tmp/iops ... 2095 2095 2093 783 388 389 ... ProblemType: Bug DistroRelease: Ubuntu 18.04 Package: zfsutils-linux 0.7.5-1ubuntu16.9 ProcVersionSignature: User Name 5.3.0-1023.25~18.04.1-aws 5.3.18 Uname: Linux 5.3.0-1023-aws x86_64 NonfreeKernelModules: zfs zunicode zavl icp zcommon znvpair ApportVersion: 2.20.9-0ubuntu7.15 Architecture: amd64 Date: Tue Jun 16 08:55:01 2020 Ec2AMI: ami-06fd83ae47b05282f Ec2AMIManifest: (unknown) Ec2AvailabilityZone: us-east-1f Ec2InstanceType: c4.xlarge Ec2Kernel: unavailable Ec2Ramdisk: unavailable SourcePackage: zfs-linux UpgradeStatus: No upgrade log present (probably fresh install) modified.conffile..etc.sudoers.d.zfs: [deleted] To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/zfs-linux/+bug/1883676/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : [email protected] Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
