Anaconda LVM and file system size limitations

[Eric Smith]

I just set up a system with a 1.4 TB RAID (on a 3Ware 7500-8), and
have run into some limitations of Anaconda. It seems that it is
not possible to set up an LVM physical volume or volume group larger
than 1 TB, nor to create ext3 (or jfs) filesystems larger than 1 TB.

These limits seem to be hardcoded in fsset.py, in multiple lines like
aelf.maxSizeMB = 1024 * 1024
It's my undertanding that there are problems with filesystems larger
than 2 TB, but it looks like it should be easy to change the Anaconda
limits to a higher value, such as 1536 MB or 1920 MB. Is there any
known problem with doing so?

If I make these changes to Anaconda, how do I create a modified
Fedora Core 1 binary CD-ROM 1? I can't even figure out from where
on the CD-ROM Anaconda's Python files are being loaded during the
install process. I expected them to either be in the base directory
or the initrd, but I don't see them in either place. Is it actually
getting them from the Anaconda RPM in Fedora/RPMS?

I've tried creating the PV, VG, and LVs from the command line then
installing Fedora Core 1, and Disk Druid does show them correctly.
But after I tell it the mount points it gets an error to the effect that
it couldn't tell the kernel about the sda3 partition (the PV). There's
an option to ignore the error, but of course the install doesn't work
after that. I'm hoping that increasing the fsset.py limits will solve
this problem.

Thanks!
Eric

[Alexandre Oliva]

Post by Eric Smith
If I make these changes to Anaconda, how do I create a modified
Fedora Core 1 binary CD-ROM 1?

Look for updates.img in
/usr/share/doc/anaconda-9.2/install-methods.txt

[Eric Smith]

Post by Eric Smith
I just set up a system with a 1.4 TB RAID (on a 3Ware 7500-8), and
have run into some limitations of Anaconda. It seems that it is
not possible to set up an LVM physical volume or volume group larger
than 1 TB, nor to create ext3 (or jfs) filesystems larger than 1 TB.
These limits seem to be hardcoded in fsset.py, in multiple lines like
aelf.maxSizeMB = 1024 * 1024
It's my undertanding that there are problems with filesystems larger
than 2 TB, but it looks like it should be easy to change the Anaconda
limits to a higher value, such as 1536 MB or 1920 MB. Is there any
known problem with doing so?

Thanks to Alexandre Oliva's message pointing me to the Anaconda docs
for how to use updates.img, I've increased these to 1920 MB (patch
attached) and it seems to solve that problem. Probably this could be
set as high as 2047 MB before causing trouble. At 2048 MB there might
be problems with 32-bit numbers overflowing and going negative, though
I haven't tried it. I'll open an enhancement request in Bugzilla
proposing raising it.

I also increased the default LVM physical extent size to 64MB, because
the 4MB default is ridiculously small (max. 256GB logical volume), and
have attached a second patch for that.

Post by Eric Smith
I've tried creating the PV, VG, and LVs from the command line then
installing Fedora Core 1, and Disk Druid does show them correctly.
But after I tell it the mount points it gets an error to the effect that
it couldn't tell the kernel about the sda3 partition (the PV). There's
an option to ignore the error, but of course the install doesn't work
after that. I'm hoping that increasing the fsset.py limits will solve
this problem.

Unfortunately, changing the limits did NOT fix this problem. More
specifically, the error is "Error informing the kernel about modifications
to partition /dev/sda3 - Invalid arguments". sda3 is my LVM physical
volume. This error happens after the message "moving (1) to step
enablefilesystems" is written to vt 1. Note that this is the same error
I was getting *before* I patched Anaconda, so it is not something I broke.
I'm not sure how to go about debugging this, so suggestions would be very
welcome.

Thanks!
Eric

[Jeremy Katz]

Post by Eric Smith
I just set up a system with a 1.4 TB RAID (on a 3Ware 7500-8), and
have run into some limitations of Anaconda. It seems that it is
not possible to set up an LVM physical volume or volume group larger
than 1 TB, nor to create ext3 (or jfs) filesystems larger than 1 TB.

Problems have been reported with devices larger than a terabyte. Also,
in general, most drivers are only safe up to a terabyte -- some can go
to 2, but it's driver dependent in 2.4.x and so it was deemed better to
err on the side of caution since I don't have hardware to investigate
with.

Post by Eric Smith
These limits seem to be hardcoded in fsset.py, in multiple lines like
aelf.maxSizeMB = 1024 * 1024
It's my undertanding that there are problems with filesystems larger
than2 TB, but it looks like it should be easy to change the Anaconda
limits to a higher value, such as 1536 MB or 1920 MB. Is there any
known problem with doing so?

It wasn't working, that's why it's set to 1 TB.

Cheers,

Jeremy

[Eric Smith]

Also, in general, most drivers are only safe up to a terabyte -- some can
go to 2, but it's driver dependent in 2.4.x and so it was deemed better to
err on the side of caution

That might justify an "are you sure you want to do this" dialog, but it
doesn't justify a hard limit (IMNSHO).

I've used other 2.4.x based distributions with >1 TB without problems,
so at least some drivers (such as 3ware) handle it fine. The 3ware card
as a documented limit of 2 TB. I'm hoping that they release new firmware
to eliminate that limit, but it won't help until we use a 2.6.x kernel.

It wasn't working, that's why it's set to 1 TB.

I suppose that would be OK for Fedora, except that I can't imagine
Red Hat Enterprise Server customers being happy about that (I wouldn't),
and IINM Fedora is supposed to be the testing ground for Enterprise
stuff.

So far with the patched Anaconda it looks like the only problem is in the
parted library, and that problem occurs even with unpatched Anaconda if
there is a preexisting partition >1 TB. So even if you don't need the
Anaconda to create such partitions, you won't be able to upgrade a machine
that has them.

I'm out of time for hacking on it this week, but next week I'll see if
standalone parted has the same problem. If so, that should be much easier
to debug.

Best regards,
Eric