Bug 1761496
| Summary: | grub2-editenv segfaults when called through grub2-mkconfig | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Product: | Red Hat Enterprise Linux 8 | Reporter: | Renaud Métrich <rmetrich> | ||||||||
| Component: | grub2 | Assignee: | Bootloader engineering team <bootloader-eng-team> | ||||||||
| Status: | CLOSED ERRATA | QA Contact: | Release Test Team <release-test-team-automation> | ||||||||
| Severity: | urgent | Docs Contact: | |||||||||
| Priority: | urgent | ||||||||||
| Version: | 8.1 | CC: | byodlows, fkrska, fmartine, kwalker, pjanda, pkhedeka, scorreia, zagar, zguo, zveleba | ||||||||
| Target Milestone: | rc | Keywords: | TestCaseNeeded, ZStream | ||||||||
| Target Release: | 8.0 | Flags: | pm-rhel:
mirror+
|
||||||||
| Hardware: | x86_64 | ||||||||||
| OS: | Linux | ||||||||||
| Whiteboard: | |||||||||||
| Fixed In Version: | grub2-2.02-83.el8 | Doc Type: | If docs needed, set a value | ||||||||
| Doc Text: | Story Points: | --- | |||||||||
| Clone Of: | |||||||||||
| : | 1836196 (view as bug list) | Environment: | |||||||||
| Last Closed: | 2020-11-04 01:53:54 UTC | Type: | Bug | ||||||||
| Regression: | --- | Mount Type: | --- | ||||||||
| Documentation: | --- | CRM: | |||||||||
| Verified Versions: | Category: | --- | |||||||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||||||
| Embargoed: | |||||||||||
| Bug Depends On: | |||||||||||
| Bug Blocks: | 1836196 | ||||||||||
| Attachments: |
|
||||||||||
100% reproducible
# rm /boot/grub2/grubenv
With "big" GRUB_CMDLINE_LINUX
# grub2-mkconfig -o /etc/grub2.cfg
Generating grub configuration file ...
done
Then with "small" GRUB_CMDLINE_LINUX
Generating grub configuration file ...
free(): invalid next size (normal)
/etc/grub.d/10_linux: line 128: 16037 Aborted (core dumped) ${grub_editenv} - set kernelopts="root=${linux_root_device_thisversion} ro ${args}"
Created attachment 1625634 [details]
Coredump
I'm moving this to Urgent because a customer faces the issue even with "GRUB_ENABLE_BLSCFG=false" in /etc/default/grub, hence the proposed workaround in KCS 4570981 doesn't work at all.
REPRODUCER:
1. Copy sample-tuned.conf (attached) to /etc/tuned/sample-tuned/tuned.conf
2. Restart tuned.service
# systemctl restart tuned
3. Apply profile twice
# tuned-adm profile sample-tuned
# tuned-adm profile sample-tuned
RESULT:
Empty (corrupted) /boot/grub/grubenv
CORE BACKTRACE:
(gdb) bt
#0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
#1 0x00007f566e039b25 in __GI_abort () at abort.c:79
#2 0x00007f566e092897 in __libc_message (action=action@entry=do_abort,
fmt=fmt@entry=0x7f566e19f057 "%s\n") at ../sysdeps/posix/libc_fatal.c:181
#3 0x00007f566e098fdc in malloc_printerr (str=str@entry=0x7f566e19d2e3 "malloc(): corrupted top size")
at malloc.c:5366
#4 0x00007f566e09c445 in _int_malloc (av=av@entry=0x7f566e3d4ba0 <main_arena>, bytes=bytes@entry=4096)
at malloc.c:4119
#5 0x00007f566e09d50e in __GI___libc_malloc (bytes=bytes@entry=4096) at malloc.c:3065
#6 0x00007f566e0889c0 in __GI__IO_file_doallocate (fp=0x5575fd5fb640) at filedoalloc.c:101
#7 0x00007f566e096b70 in __GI__IO_doallocbuf (fp=fp@entry=0x5575fd5fb640) at libioP.h:838
#8 0x00007f566e095dd8 in _IO_new_file_overflow (f=0x5575fd5fb640, ch=-1) at fileops.c:749
#9 0x00007f566e094f6f in _IO_new_file_xsputn (n=1024, data=<optimized out>, f=0x5575fd5fb640)
at libioP.h:838
#10 _IO_new_file_xsputn (f=0x5575fd5fb640, data=<optimized out>, n=1024) at fileops.c:1201
#11 0x00007f566e089e2c in __GI__IO_fwrite (buf=0x5575fd5fc840, size=1, count=1024, fp=0x5575fd5fb640)
at libioP.h:838
#12 0x00005575fbfb3585 in write_envblk (name=0x5575fbff5063 "/boot/grub2/grubenv",
envblk=0x5575fd5fb830) at ../util/grub-editenv.c:198
#13 0x00005575fbfb36f3 in set_variables (name=0x5575fbff5063 "/boot/grub2/grubenv", argc=0,
argv=0x7ffdb3942890) at ../util/grub-editenv.c:230
#14 0x00005575fbfb3a87 in main (argc=5, argv=0x7ffdb3942868) at ../util/grub-editenv.c:333
Created attachment 1687306 [details]
Sample tuned profile to be copied to /etc/tuned/sample-tuned/tuned.conf
Every 2 executions, Valgrind shows the following errors:
==6833== Invalid write of size 8
==6833== at 0x130045: grub_memset (misc.c:538)
==6833== by 0x11EB52: grub_envblk_set (envblk.c:146)
==6833== by 0x10D69D: set_variables (grub-editenv.c:223)
==6833== by 0x10DA86: main (grub-editenv.c:333)
==6833== Address 0x72f3b20 is 0 bytes after a block of size 1,024 alloc'd
==6833== at 0x4C30EDB: malloc (vg_replace_malloc.c:309)
==6833== by 0x12E84F: xmalloc (misc.c:92)
==6833== by 0x10D396: open_envblk_file (grub-editenv.c:155)
==6833== by 0x10D61E: set_variables (grub-editenv.c:212)
==6833== by 0x10DA86: main (grub-editenv.c:333)
The allocated 1024 bytes seem to be grubenv file (which is 1024 bytes).
Source code:
516 void *
517 grub_memset (void *s, int c, grub_size_t len)
518 {
519 void *p = s;
520 grub_uint8_t pattern8 = c;
521
522 if (len >= 3 * sizeof (unsigned long))
523 {
524 unsigned long patternl = 0;
525 grub_size_t i;
526
527 for (i = 0; i < sizeof (unsigned long); i++)
528 patternl |= ((unsigned long) pattern8) << (8 * i);
529
530 while (len > 0 && (((grub_addr_t) p) & (sizeof (unsigned long) - 1)))
531 {
532 *(VOLATILE_CLANG grub_uint8_t *) p = pattern8;
533 p = (grub_uint8_t *) p + 1;
534 len--;
535 }
536 while (len >= sizeof (unsigned long))
537 {
538 *(VOLATILE_CLANG unsigned long *) p = patternl;
539 p = (unsigned long *) p + 1;
540 len -= sizeof (unsigned long);
541 }
542 }
The pointer "p" on line 538 refers to a location outside of that 1024 block, corrupting the next malloc header probably.
==6833==
==6833== Invalid write of size 1
==6833== at 0x130063: grub_memset (misc.c:546)
==6833== by 0x11EB52: grub_envblk_set (envblk.c:146)
==6833== by 0x10D69D: set_variables (grub-editenv.c:223)
==6833== by 0x10DA86: main (grub-editenv.c:333)
==6833== Address 0x72f3b58 is 8 bytes before a block of size 16 alloc'd
==6833== at 0x4C30EDB: malloc (vg_replace_malloc.c:309)
==6833== by 0x12ED00: grub_malloc (mm.c:32)
==6833== by 0x11E877: grub_envblk_open (envblk.c:39)
==6833== by 0x10D41A: open_envblk_file (grub-editenv.c:163)
==6833== by 0x10D61E: set_variables (grub-editenv.c:212)
==6833== by 0x10DA86: main (grub-editenv.c:333)
I found an even easier reproducer for this:
$ grub2-editenv /tmp/grubenv create
$ grub2-editenv /tmp/grubenv set foo="bar=$(for i in {1..500}; do var="a$var"; done; echo $var)"
$ cat /tmp/grubenv
# GRUB Environment Block
foo=bar=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
##########################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################
$ grub2-editenv /tmp/grubenv set foo="bar=$(for i in {1..50}; do var="a$var"; done; echo $var)"
malloc(): corrupted top size
Aborted (core dumped)
$ cat /tmp/grubenv
$ ls -lh /tmp/grubenv
-rw-rw-r--. 1 javier javier 0 May 11 15:40 /tmp/grubenv
(In reply to Renaud Métrich from comment #5) > Every 2 executions, Valgrind shows the following errors: > > ==6833== Invalid write of size 8 > ==6833== at 0x130045: grub_memset (misc.c:538) > ==6833== by 0x11EB52: grub_envblk_set (envblk.c:146) > ==6833== by 0x10D69D: set_variables (grub-editenv.c:223) > ==6833== by 0x10DA86: main (grub-editenv.c:333) > ==6833== Address 0x72f3b20 is 0 bytes after a block of size 1,024 alloc'd > ==6833== at 0x4C30EDB: malloc (vg_replace_malloc.c:309) > ==6833== by 0x12E84F: xmalloc (misc.c:92) > ==6833== by 0x10D396: open_envblk_file (grub-editenv.c:155) > ==6833== by 0x10D61E: set_variables (grub-editenv.c:212) > ==6833== by 0x10DA86: main (grub-editenv.c:333) > > The allocated 1024 bytes seem to be grubenv file (which is 1024 bytes). > > Source code: > 516 void * > 517 grub_memset (void *s, int c, grub_size_t len) > 518 { > 519 void *p = s; > 520 grub_uint8_t pattern8 = c; > 521 > 522 if (len >= 3 * sizeof (unsigned long)) > 523 { > 524 unsigned long patternl = 0; > 525 grub_size_t i; > 526 > 527 for (i = 0; i < sizeof (unsigned long); i++) > 528 patternl |= ((unsigned long) pattern8) << (8 * i); > 529 > 530 while (len > 0 && (((grub_addr_t) p) & (sizeof (unsigned long) - > 1))) > 531 { > 532 *(VOLATILE_CLANG grub_uint8_t *) p = pattern8; > 533 p = (grub_uint8_t *) p + 1; > 534 len--; > 535 } > 536 while (len >= sizeof (unsigned long)) > 537 { > 538 *(VOLATILE_CLANG unsigned long *) p = patternl; > 539 p = (unsigned long *) p + 1; > 540 len -= sizeof (unsigned long); > 541 } > 542 } > > The pointer "p" on line 538 refers to a location outside of that 1024 block, > corrupting the next malloc header probably. > Thanks, I'm looking at this bug. Created attachment 1687649 [details]
[PATCH] envblk: Fix buffer overrun when attempting to shrink a variable value
I've attached a patch that fixes this bug.
Reproduced on released RHEL-8.2 x86_64
grub2-pc-2.02-81.el8
in virtual machine
using steps from Javier
- install system on BIOS machine (not UEFI)
$ grub2-editenv /tmp/grubenv create
$ grub2-editenv /tmp/grubenv set foo="bar=$(for i in {1..500}; do var="a$var"; done; echo $var)"
$ grub2-editenv /tmp/grubenv set foo="bar=$(for i in {1..50}; do var="a$var"; done; echo $var)" # note that this command wants only 50 characters
malloc(): corrupted top size
Aborted (core dumped)
$ ls -lh /tmp/grubenv
-rw-rw-r--. 1 root root 0 May 11 15:40 /tmp/grubenv
in my case magic limit was foo="bar=$(for i in {1..346};
Automated testing is ready, some notes about reproducing the issue: The only package required is grub2-tools-minimal. The machine can use UEFI and doesn't have to be x86_64. *** Bug 1683128 has been marked as a duplicate of this bug. *** verified on RHEL-8.3.0-20200616.0 x86_64 Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (grub2 bug fix and enhancement update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2020:4513 |
Description of problem: When having a long GRUB_CMDLINE_LINUX line already installed, then amending it to make it small, grub2-editenv segfaults corrupting memory. Version-Release number of selected component (if applicable): grub2-pc-2.02-66.el8_0.1.x86_64 How reproducible: Don't know exactly, reproduced twice Steps to Reproduce: 1. Build a Grub menu with following settings /etc/default/grub: -------- 8< ---------------- 8< ---------------- 8< ---------------- 8< -------- GRUB_DISTRIBUTOR="$(sed 's, release .*$,,g' /etc/system-release)" GRUB_DEFAULT=saved GRUB_DISABLE_SUBMENU=true GRUB_CMDLINE_LINUX="crashkernel=auto rd.luks.uuid=luks-569a3783-4ec6-4bd0-a122-d6ca32ff8c23 rd.luks.uuid=luks-ba80a2b3-dd86-4321-8a51-bf65a94c7c9c rd.luks.uuid=luks-e7926073-5ed0-4745-96b3-680835cd542d rd.luks.uuid=luks-fc691f4a-285d-406f-ad0d-c1ea19c4b9b9 rd.luks.uuid=luks-306bc442-b91f-4322-9711-ea73635bd602 rd.luks.uuid=luks-4243c2be-ad56-43ed-9e85-d2af2e550912 rd.luks.uuid=luks-7192e378-d963-43ad-b2fb-93ded37dac5a rd.lvm.vg=rhel console=tty0 console=ttyS0,115200n8" GRUB_DISABLE_RECOVERY="true" GRUB_ENABLE_BLSCFG=true GRUB_TIMEOUT=1 GRUB_TERMINAL_OUTPUT="serial console" GRUB_TERMINAL="serial console" GRUB_SERIAL_COMMAND="serial --speed=115200 --unit=0 --word=8 --parity=no --stop=1" -------- 8< ---------------- 8< ---------------- 8< ---------------- 8< -------- # grub2-mkconfig -o /etc/grub2.cfg 2. Change the GRUB_CMDLINE_LINUX as shown below -------- 8< ---------------- 8< ---------------- 8< ---------------- 8< -------- GRUB_CMDLINE_LINUX="crashkernel=auto console=tty0 console=ttyS0,115200n8" -------- 8< ---------------- 8< ---------------- 8< ---------------- 8< -------- # grub2-mkconfig -o /etc/grub2.cfg Generating grub configuration file ... free(): invalid next size (normal) /etc/grub.d/10_linux: line 128: 12517 Aborted (core dumped) ${grub_editenv} - set kernelopts="root=${linux_root_device_thisversion} ro ${args}" Additional info: Now I cannot reproduce because my /boot/grub2/grubenv file seems corrupted: Generating grub configuration file ... /usr/bin/grub2-editenv: error: environment block too small. # cat /boot/grub2/grubenv # GRUB Environment Block saved_entry=7233c98f39fa4200b6f8a8288b95e0be-4.18.0-80.11.2.el8_0.x86_64 kernelopts=root=/dev/mapper/luks-569a3783-4ec6-4bd0-a122-d6ca32ff8c23 ro crashkernel=auto console=tty0 console=ttyS0,115200n8 boot_success=0 ################################################################################################################################################################################################################################################################################################################################################################################################ boot_success=0 ################################################################################################################################################################################################################################################################################################################################################################################################