Bug 158687 - vgcreate unable to create large volume groups
vgcreate unable to create large volume groups
Product: Red Hat Enterprise Linux 4
Classification: Red Hat
Component: lvm2 (Show other bugs)
i386 Linux
medium Severity medium
: ---
: ---
Assigned To: Alasdair Kergon
Depends On:
Blocks: 158692
  Show dependency treegraph
Reported: 2005-05-24 16:30 EDT by David Milburn
Modified: 2007-11-30 17:07 EST (History)
4 users (show)

See Also:
Fixed In Version: U2
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2006-03-07 14:36:45 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)

  None (edit)
Description David Milburn 2005-05-24 16:30:12 EDT
From Bugzilla Helper:
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.6) Gecko/20050302 Firefox/1.0.1 Fedora/1.0.1-1.3.2

Description of problem:
After creating 506 PVs using 3.9TB of capacity,
vgcreate fails with the following message:

VG vg_4tb_low metadata writing failed

Customer can always reproduce.

Version-Release number of selected component (if applicable):

How reproducible:

Steps to Reproduce:
1. # pvcreate /dev/sdk1 ... /dev/sdtd1 
2. # vgcreate vg_4tb_low /dev/sdk1 ... /dev/sdtd1

Actual Results:  vgcreate results in the following error message.

VG vg_4gb_low metadata writing failed

Expected Results:  Volume group "vg_4tb_low" should be successfully created.

Additional info:

Hardware info:
 The disk configuration consists of two target on a single path.
 Target: 230000004c7f0761
       LUN0  1.9TB (LUN0+LUN1 exceeds 2TB)
       LUN1  130GB
       LUN2  4GB
       LUN253  4GB

 Target: 210000004c7f0761
       LUN0  4GB
       LUN251  4GB

This is this issue that the *large* volume groups cannot be created by vgcreate.
When this issue occurs, the message "VG vg_4gb_low metadata writing failed" is displayed. I think this message is created in the following code in _vg_write_raw() of format-text.c.

       if (!(mdac->rlocn.size = text_vg_export_raw(vg, "", buf, sizeof(buf)))) {
               log_error("VG %s metadata writing failed", vg->name);
               goto out;

NEC thinks that the cause of this issue is fixed buffer size in _vg_write_raw() of format-text.c.

       /* FIXME Essential fix! Make dynamic (realloc? pool?) */
      char buf[65536];
      int found = 0;
Comment 1 Alasdair Kergon 2005-05-24 16:56:11 EDT
Notes for large VGs:

Firstly, as per pvcreate man page, don't store VG metadata on every PV or you'll
find the tools get very slow - and you really don't need 505 backup copies,
every one of which has to be checked every time you access the VG!  4 or 5
copies should be more than enough.  For the others, use --metadatacopies 0.  The
only thing to be careful about is if you were to use vgsplit in the future,
every VG must contain at least one PV with non-zero metadatacopies.

Secondly, you need to use a larger metadata area.  You've plenty of disk space,
so set it to 1 MB or more.  In a quick test here, each additional PV used up 260
bytes, so 506 would bring you up to the 128KB limit.

Thirdly, yes that buffer needs increasing.  It's not a trivial change to do
properly because there are dependencies on other settings - the default values
of reserved_stack and reserved_memory in lvm.conf.
Comment 2 Alasdair Kergon 2005-05-24 16:58:21 EDT
The easy way out for now would be for me to make the 65536 another tunable
parameter in the config file, say "max_metadatasize".
Comment 5 Alasdair Kergon 2005-06-08 10:03:10 EDT
I've changed the code to use malloced memory (instead of the stack) and to
extend the allocation as required.

Fix is in lvm2 CVS.

Note You need to log in before you can comment on or make changes to this bug.