Package: gfs2-tools Version: 3.0.12-2 Severity: critical Tags: squeeze My mail server (cid02) uses NFS shares mounted from a NFS server (sane02) which is exporting GFS2 cluster partition.
On random times my mail server get IO blocked process over NFS shares, when I start to investigate the problem I found it is NOT problem with NFS server, network or Mail server (postfix, imap, dovecot). It appears to be a strange GFS2 filesystem block out. Any help is appreciated. Events happens as follow and messages begin to throtle... On server cid02 file /var/log/kern.log Feb 11 04:57:49 ccsdtfdemcid02 kernel: [93435.428896] nfs: server 10.0.18.35 not responding, still trying On server sane02 file /var/log/kern.log Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168102] INFO: task nfsd:5340 blocked for more than 120 seconds. Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168159] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168214] nfsd D 0000000000000002 0 5340 2 0x00000000 Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168221] ffff88012c19f810 0000000000000046 0000000000000000 ffffffffa036293b Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168228] 0000000000000000 ffffffffa0362495 000000000000f9e0 ffff880121ec1fd8 Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168233] 0000000000015780 0000000000015780 ffff88012c092a60 ffff88012c092d58 Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168239] Call Trace: Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168261] [<ffffffffa036293b>] ? dlm_put_lockspace+0x10/0x1c [dlm] Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168271] [<ffffffffa0362495>] ? dlm_lock+0x135/0x149 [dlm] Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168291] [<ffffffffa04b87c0>] ? gdlm_ast+0x0/0xc2 [gfs2] Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168301] [<ffffffffa04b8731>] ? gdlm_bast+0x0/0x3d [gfs2] Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168309] [<ffffffffa04a1dcc>] ? gfs2_glock_holder_wait+0x0/0xd [gfs2] Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168317] [<ffffffffa04a1dd5>] ? gfs2_glock_holder_wait+0x9/0xd [gfs2] Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168327] [<ffffffff812fae26>] ? __wait_on_bit+0x41/0x70 Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168335] [<ffffffffa04a1dcc>] ? gfs2_glock_holder_wait+0x0/0xd [gfs2] Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168340] [<ffffffff812faec0>] ? out_of_line_wait_on_bit+0x6b/0x77 Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168346] [<ffffffff81064d58>] ? wake_bit_function+0x0/0x23 Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168355] [<ffffffffa04a1dba>] ? gfs2_glock_wait+0x23/0x28 [gfs2] Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168364] [<ffffffffa04a4703>] ? gfs2_glock_nq_init+0x16/0x2a [gfs2] Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168373] [<ffffffffa04a505d>] ? gfs2_inode_lookup+0x16c/0x1d1 [gfs2] Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168382] [<ffffffffa04a46fb>] ? gfs2_glock_nq_init+0xe/0x2a [gfs2] Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168392] [<ffffffffa04b4670>] ? try_rgrp_unlink+0xa3/0xcc [gfs2] Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168401] [<ffffffffa04b5105>] ? gfs2_inplace_reserve_i+0x285/0x640 [gfs2] Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168409] [<ffffffffa049a243>] ? gfs2_dirent_find+0x0/0x7 [gfs2] Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168416] [<ffffffffa0499b50>] ? gfs2_dirent_search+0x13b/0x167 [gfs2] Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168426] [<ffffffffa04a521d>] ? gfs2_createi+0x15b/0x912 [gfs2] Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168430] [<ffffffff81064c4b>] ? bit_waitqueue+0x10/0xa0 Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168438] [<ffffffffa04a1c5f>] ? do_promote+0x15b/0x1f3 [gfs2] Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168447] [<ffffffffa04a2ed4>] ? run_queue+0xa5/0x127 [gfs2] Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168456] [<ffffffffa04ae156>] ? gfs2_create+0x67/0x131 [gfs2] Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168465] [<ffffffffa04a513e>] ? gfs2_createi+0x7c/0x912 [gfs2] Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168472] [<ffffffff810f78b4>] ? vfs_create+0x6d/0x89 Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168484] [<ffffffffa044eb27>] ? nfsd_create_v3+0x20a/0x358 [nfsd] Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168493] [<ffffffffa0457a14>] ? do_open_lookup+0x91/0x240 [nfsd] Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168503] [<ffffffffa0457d7a>] ? nfsd4_open+0x1b7/0x339 [nfsd] Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168533] [<ffffffffa0458db1>] ? nfsd4_encode_operation+0x5b/0x135 [nfsd] Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168548] [<ffffffffa0457716>] ? nfsd4_proc_compound+0x1fd/0x3b3 [nfsd] Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168559] [<ffffffffa0449329>] ? nfsd_dispatch+0xdd/0x1b9 [nfsd] Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168577] [<ffffffffa03894ff>] ? svc_process+0x403/0x627 [sunrpc] Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168588] [<ffffffffa0449772>] ? nfsd+0x0/0x12e [nfsd] Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168598] [<ffffffffa0449857>] ? nfsd+0xe5/0x12e [nfsd] Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168605] [<ffffffff81064a5d>] ? kthread+0x79/0x81 Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168615] [<ffffffff81011baa>] ? child_rip+0xa/0x20 Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168622] [<ffffffff810649e4>] ? kthread+0x0/0x81 Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168628] [<ffffffff81011ba0>] ? child_rip+0x0/0x20 -- System Information: Debian Release: 6.0 APT prefers oldstable APT policy: (500, 'oldstable'), (500, 'stable') Architecture: amd64 (x86_64) Kernel: Linux 2.6.32-5-amd64 (SMP w/8 CPU cores) Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8) Shell: /bin/sh linked to /bin/bash Versions of packages gfs2-tools depends on: ii cman 3.0.12-2 Red Hat cluster suite - cluster ma ii libc6 2.11.2-10 Embedded GNU C Library: Shared lib ii libldap-2.4-2 2.4.23-7 OpenLDAP libraries ii libncurses5 5.7+20100313-5 shared libraries for terminal hand ii libxml2 2.7.8.dfsg-2 GNOME XML library ii psmisc 22.11-1 utilities that use the proc file s gfs2-tools recommends no packages. gfs2-tools suggests no packages. -- no debconf information -- To UNSUBSCRIBE, email to debian-bugs-rc-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
