** Tags added: cscc -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1775527
Title: Ubuntu16.04:KVM - xfs_log_force: error -5 returned after stress test running after a day #345 Status in The Ubuntu-power-systems project: Invalid Status in linux package in Ubuntu: Invalid Status in linux source package in Xenial: Invalid Bug description: == Comment: #0 - Chanh H. Nguyen <chngu...@us.ibm.com> - 2016-09-08 15:01:30 == We have installed 16.04.1 on our guest and running IO stress test against via virtual qlogic disk. After running around 8 hrs, this error popping up "xfs_imap_to_bp: xfs_trans_read_buf() returned error -5" then with all of these errors "xfs_log_force: error -5 returned" fill up all the log. Jul 19 19:03:41 tinyg2 kernel: [65047.974604] XFS (vdb9): metadata I/O error: block 0x2f140 ("xfs_trans_read_buf_map") error 5 numblks 32 Jul 19 19:03:41 tinyg2 kernel: [65047.975885] XFS (vdb9): xfs_imap_to_bp: xfs_trans_read_buf() returned error -5. Jul 19 19:03:41 tinyg2 kernel: [65047.976069] XFS (vdb9): xfs_do_force_shutdown(0x8) called from line 3433 of file /build/linux-JQcyrX/linux-4.4.0/fs/xfs/xfs_inode.c. Return address = 0xd0000000037951f4 Jul 19 19:03:41 tinyg2 kernel: [65047.976457] XFS (vdb9): metadata I/O error: block 0x703d54 ("xlog_iodone") error 5 numblks 64 Jul 19 19:03:41 tinyg2 kernel: [65047.976662] XFS (vdb9): xfs_do_force_shutdown(0x2) called from line 1197 of file /build/linux-JQcyrX/linux-4.4.0/fs/xfs/xfs_log.c. Return address = 0xd0000000037a4c00 Jul 19 19:03:41 tinyg2 kernel: [65047.976995] XFS (vdb9): Log I/O Error Detected. Shutting down filesystem Jul 19 19:03:41 tinyg2 kernel: [65047.977114] XFS (vdb9): Please umount the filesystem and rectify the problem(s) follow with all of these error that quickly fill up dmesg log.... Jul 19 19:04:37 tinyg2 kernel: [65103.515386] XFS (vdb9): xfs_log_force: error -5 returned. Jul 19 19:05:07 tinyg2 kernel: [65133.595369] XFS (vdb9): xfs_log_force: error -5 returned. Jul 19 19:05:37 tinyg2 kernel: [65163.679358] XFS (vdb9): xfs_log_force: error -5 returned. Jul 19 19:06:07 tinyg2 kernel: [65193.755383] XFS (vdb9): xfs_log_force: error -5 returned. Jul 19 19:06:37 tinyg2 kernel: [65223.835396] XFS (vdb9): xfs_log_force: error -5 returned. Jul 19 19:06:51 tinyg2 kernel: [65237.091455] EXT4-fs (loop1): mounting ext2 file system using the ext4 subsystem Jul 19 19:06:51 tinyg2 kernel: [65237.093219] EXT4-fs (loop1): mounted filesystem without journal. Opts: (null) Jul 19 19:06:51 tinyg2 kernel: [65237.623893] Process accounting resumed Jul 19 19:07:07 tinyg2 kernel: [65253.919361] XFS (vdb9): xfs_log_force: error -5 returned. Jul 19 19:07:37 tinyg2 kernel: [65283.995393] XFS (vdb9): xfs_log_force: error -5 returned. Jul 19 19:08:08 tinyg2 kernel: [65314.075378] XFS (vdb9): xfs_log_force: error -5 returned. Jul 19 19:08:38 tinyg2 kernel: [65344.155397] XFS (vdb9): xfs_log_force: error -5 returned. Jul 19 19:09:08 tinyg2 kernel: [65374.235367] XFS (vdb9): xfs_log_force: error -5 returned. Jul 19 19:09:38 tinyg2 kernel: [65404.319354] XFS (vdb9): xfs_log_force: error -5 returned. Jul 19 19:10:08 tinyg2 kernel: [65434.395368] XFS (vdb9): xfs_log_force: error -5 returned. == Comment: #1 - Chanh H. Nguyen <chngu...@us.ibm.com> - 2016-09-08 15:02:27 == We recently re-created this bug on our new system "cressg2" which is using a different SAN on kernel -36. My Ubuntu 16.04.1 guest, cressg2, are running the IO and other stress test, and it is hitting "xfs_log_force: error -5 returned" problem after it runs the tests for 15 hours. It is using our lab storage (SVC SAN storage configuation with access to DS8000 Volume) and switch, compare to tinyg3, so it is using different storage and fabric switch. root@cressg2:~# uname -a Linux cressg2 4.4.0-36-generic #55-Ubuntu SMP Thu Aug 11 18:00:57 UTC 2016 ppc64le ppc64le ppc64le GNU/Linux [Wed Sep 7 10:26:48 2016] XFS (vdb8): xfs_log_force: error -5 returned. [Wed Sep 7 10:27:18 2016] XFS (vdb8): xfs_log_force: error -5 returned. [Wed Sep 7 10:27:32 2016] XFS (vdb8): xfs_log_force: error -5 returned. [Wed Sep 7 10:27:33 2016] XFS (vdb8): xfs_log_force: error -5 returned. [Wed Sep 7 10:27:33 2016] XFS (vdb8): xfs_log_force: error -5 returned. [Wed Sep 7 10:27:48 2016] XFS (vdb8): xfs_log_force: error -5 returned. [Wed Sep 7 10:27:48 2016] XFS (vdb8): xfs_log_force: error -5 returned. [Wed Sep 7 10:27:49 2016] XFS (vdb8): xfs_log_force: error -5 returned. [Wed Sep 7 10:27:56 2016] XFS (vdb8): xfs_log_force: error -5 returned. [Wed Sep 7 10:28:18 2016] XFS (vdb8): xfs_log_force: error -5 returned. [Wed Sep 7 10:28:48 2016] XFS (vdb8): xfs_log_force: error -5 returned. [Wed Sep 7 10:29:18 2016] XFS (vdb8): xfs_log_force: error -5 returned. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-power-systems/+bug/1775527/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp