Hi Anand:
Anand Vaidya wrote:
Dear Beowulfers,
We ran into a problem with large disks which I suspect is fairly common,
however the usual solutions are not working. IBM, RedHat have not been
able to provide any useful answers so I am turning to this list for
help. (Emulex is still helping, but I am not sure how far they can go
without access to the hardware)
Details:
* Linux Cluster for Weather modelling
* IBM Bladecenter blades and an IBM x3655 Opteron head node FC attached
to a Hitachi Tagmastore SAN storage, Emulex LightPulse FC HBA,
PCI-Express, Dual port
* RHEL 4update5, x86_64 kernel 2.6.9-55 SMP and RHEL provided Emulex
driver (lpfc) and lpfcdfc also installed
There is a problem in some parted versions (prior to 1.8.x) where they
munge the partition table on gpt/large disks.
Apart from suggesting a more modern kernel (2.6.22.6 or so), there may
be other things you can do.
* GPT partition created with parted
I presume this is 1.6.19 parted?
There is one 2TB LUN, works fine.
There is a 3TB LUN on the Hitachi SAN which is reported as "only" 2199GB
( 2.1TB) ,
Yup. Sounds like either the parted problem, or a driver issue.
What does
parted /dev/3TBlun/ print
report where /dev/3TBlun is the device containing the 3TB lun?
We had seen this behavior with some 1.6.9 and 1.7.x versions of parted.
The only way to fix it was to rebuild parted with 1.8.x
We noticed that, when the emulex driver loads, the following error
message is reported:
Emulex LightPulse Fibre Channel SCSI driver 8.0.16.32
<http://8.0.16.32>
Copyright(c) 2003-2007 Emulex. All rights reserved.
ACPI: PCI Interrupt 0000:2d:00.0[A] -> GSI 18 (level, low)
-> IRQ 185
PCI: Setting latency timer of device 0000:2d:00.0 to 64
lpfc 0000:2d:00.0: 0:1305 Link Down Event x2 received Data:
x2 x4 x1000
lpfc 0000:2d:00.0: 0:1305 Link Down Event x2 received Data:
x2 x4 x1000
lpfc 0000:2d:00.0: 0:1303 Link Up Event x3 received Data: x3
x1 x10 x0
scsi5 : IBM 42C2071 4Gb 2-Port PCIe FC HBA for System x on
PCI bus 2d device 00 irq 185 port 0
Vendor: HITACHI Model: OPEN-V*3 Rev: 5007
Type: Direct-Access ANSI SCSI
revision: 03
sdb : very big device. try to use READ CAPACITY(16).
This is what our JackRabbit reports ...
sdb : READ CAPACITY(16) failed.
This is not what our JackRabbit reports.
[...]
The problem is with the READ CAPACITY(16) failed, but we are unable to
find the source of this error.
We conducted several experiments without success:
- Tried compiling the latest driver from Emulex (8.0.16.32
<http://8.0.16.32>) - same error
- Tried Knoppix (2.6.19) and Gentoo LiveCD (2.6.19 ) , and CentOS 4.4
- same error
Sounds a great deal like parted.
- Tried to boot Belenix (Solaris 32 bit live), failed to boot completely
(may be unrelated issue)
We have a temporary workaround in place: We created 3x1TB disks and used
LVM to create a striped 3TB volume with ext3 FS. This works fine.
RedHat claims ext3 and RHEL4 supports disks upto 8TB and 16TB
respectively (since RHEL4u2)
... yeah.
I would like to know if anyone on the list has any pointers that can
help us solve the issue.
Please run the parted command as indicated. Lets see what the partition
table thinks it is.
Do you have data on that partition? Can you remake the label on that
device with a new version of parted?
Regards
Anand Vaidya
------------------------------------------------------------------------
_______________________________________________
Beowulf mailing list, Beowulf@beowulf.org
To change your subscription (digest mode or unsubscribe) visit
http://www.beowulf.org/mailman/listinfo/beowulf
--
Joseph Landman, Ph.D
Founder and CEO
Scalable Informatics LLC,
email: [EMAIL PROTECTED]
web : http://www.scalableinformatics.com
http://jackrabbit.scalableinformatics.com
phone: +1 734 786 8423
fax : +1 866 888 3112
cell : +1 734 612 4615
_______________________________________________
Beowulf mailing list, Beowulf@beowulf.org
To change your subscription (digest mode or unsubscribe) visit
http://www.beowulf.org/mailman/listinfo/beowulf
