Bug 1786497

This will require Fujitsu to look into this bug.

By the way, Which version of crash-ptdump-command are you using?

Hi, it's crash-ptdump-command-1.0.3-4.el8

Fujitsu,

Can you please investigate this issue?

Thanks,
  Dave

Hello Fujitsu,

Using the supplied vmcore-ptdump file, here is the full command output:

  crash> ptdump /root/ptdump
  [0] buffer dump: dump.0
  [0] packet decode: decode.0
  [1] buffer dump: dump.1
  [1] packet decode: decode.1
  [2] buffer dump: dump.2
  [2] packet decode: decode.2
  ptdump: invalid size request: 0  type: "read page for write"
  crash> 
  
The fatal error is coming the readmem() call here in the write_buffer_wrapped() function:

        ...

        idx = start_idx;
        page = pt_info_ptr->buffer_ptr[idx];
        offset = pt_info_ptr->output_off & mask;
        len = offset;

        readmem(page, KVADDR, buf, len, "read page for write",
                FAULT_ON_ERROR);

        ...

So I added this debug patch to display some information that might 
show why a readmem() of size 0 was being attempted:

  --- ptdump-1.0.3/ptdump.c.orig
  +++ ptdump-1.0.3/ptdump.c
  @@ -351,6 +351,9 @@ int write_buffer_wrapped(int cpu, FILE *
   	offset = pt_info_ptr->output_off & mask;
   	len = offset;
   
  +error(INFO, "write_buffer_wrapped: start_idx: %d len: %d output_off: %lx (%lx) page: %lx\n", 
  +	start_idx, len, pt_info_ptr->output_off, offset, page);
  +
   	readmem(page, KVADDR, buf, len, "read page for write",
   		FAULT_ON_ERROR);
   
The debug info output seems normal leading up to the fatal readmem() attempt, other than 
the pt_info_ptr->output_off happens to be on a page boundary:
  
  crash> ptdump /root/ptdump
  ptdump: write_buffer_wrapped: start_idx: 86 len: 512 output_off: 56200 (200) page: ffff937f59456000
  [0] buffer dump: dump.0
  [0] packet decode: decode.0
  ptdump: write_buffer_wrapped: start_idx: 823 len: 2944 output_off: 337b80 (b80) page: ffff937ebaf37000
  [1] buffer dump: dump.1
  [1] packet decode: decode.1
  [2] buffer dump: dump.2
  [2] packet decode: decode.2
  ptdump: write_buffer_wrapped: start_idx: 698 len: 0 output_off: 2ba000 (0) page: ffff937ebaaba000
  ptdump: invalid size request: 0  type: "read page for write"
  crash> 

So then I simply avoided the readmem() attempt in that case, with this
addition to the debug code:
  
  --- ptdump-1.0.3/ptdump.c.orig
  +++ ptdump-1.0.3/ptdump.c
  @@ -351,6 +351,11 @@ int write_buffer_wrapped(int cpu, FILE *
   	offset = pt_info_ptr->output_off & mask;
   	len = offset;
   
  +error(INFO, "write_buffer_wrapped: start_idx: %d len: %d output_off: %lx (%lx) page: %lx\n", 
  +	start_idx, len, pt_info_ptr->output_off, offset, page);
  +	if (!len)
  +		goto done;
  +
   	readmem(page, KVADDR, buf, len, "read page for write",
   		FAULT_ON_ERROR);
   
  @@ -364,6 +369,7 @@ int write_buffer_wrapped(int cpu, FILE *
   		return FALSE;
   	}
   
  +done:
   	free(buf);
   	return TRUE;
   }
  
And then it seems to work as expected:
  
  crash> ptdump /root/ptdump
  ptdump: write_buffer_wrapped: start_idx: 86 len: 512 output_off: 56200 (200) page: ffff937f59456000
  [0] buffer dump: dump.0
  [0] packet decode: decode.0
  ptdump: write_buffer_wrapped: start_idx: 823 len: 2944 output_off: 337b80 (b80) page: ffff937ebaf37000
  [1] buffer dump: dump.1
  [1] packet decode: decode.1
  [2] buffer dump: dump.2
  [2] packet decode: decode.2
  ptdump: write_buffer_wrapped: start_idx: 698 len: 0 output_off: 2ba000 (0) page: ffff937ebaaba000
  [3] buffer dump: dump.3
  [3] packet decode: decode.3
  [4] buffer dump: dump.4
  [4] packet decode: decode.4
  [5] buffer dump: dump.5
  [5] packet decode: decode.5
  [6] buffer dump: dump.6
  [6] packet decode: decode.6
  [7] buffer dump: dump.7
  [7] packet decode: decode.7
  [8] buffer dump: dump.8
  [8] packet decode: decode.8
  [9] buffer dump: dump.9
  [9] packet decode: decode.9
  ptdump: write_buffer_wrapped: start_idx: 264 len: 304 output_off: 108130 (130) page: ffff937f4c108000
  [10] buffer dump: dump.10
  [10] packet decode: decode.10
  [11] buffer dump: dump.11
  [11] packet decode: decode.11
  [12] buffer dump: dump.12
  [12] packet decode: decode.12
  [13] buffer dump: dump.13
  [13] packet decode: decode.13
  [14] buffer dump: dump.14
  [14] packet decode: decode.14
  [15] buffer dump: dump.15
  [15] packet decode: decode.15
  ptdump: write_buffer_wrapped: start_idx: 426 len: 3872 output_off: 1aaf20 (f20) page: ffff937f4d9aa000
  [16] buffer dump: dump.16
  [16] packet decode: decode.16
  [17] buffer dump: dump.17
  [17] packet decode: decode.17
  ptdump: write_buffer_wrapped: start_idx: 598 len: 2192 output_off: 256890 (890) page: ffff937f4e656000
  [18] buffer dump: dump.18
  [18] packet decode: decode.18
  [19] buffer dump: dump.19
  [19] packet decode: decode.19
  [20] buffer dump: dump.20
  [20] packet decode: decode.20
  [21] buffer dump: dump.21
  [21] packet decode: decode.21
  ptdump: write_buffer_wrapped: start_idx: 57 len: 624 output_off: 39270 (270) page: ffff937f4f439000
  [22] buffer dump: dump.22
  [22] packet decode: decode.22
  ptdump: write_buffer_wrapped: start_idx: 723 len: 2736 output_off: 2d3ab0 (ab0) page: ffff937f4fad3000
  [23] buffer dump: dump.23
  [23] packet decode: decode.23
  ptdump: write_buffer_wrapped: start_idx: 117 len: 2000 output_off: 757d0 (7d0) page: ffff937f4fc75000
  [24] buffer dump: dump.24
  [24] packet decode: decode.24
  [25] buffer dump: dump.25
  [25] packet decode: decode.25
  [26] buffer dump: dump.26
  [26] packet decode: decode.26
  [27] buffer dump: dump.27
  [27] packet decode: decode.27
  [28] buffer dump: dump.28
  [28] packet decode: decode.28
  ptdump: write_buffer_wrapped: start_idx: 670 len: 1072 output_off: 29e430 (430) page: ffff937f5169e000
  [29] buffer dump: dump.29
  [29] packet decode: decode.29
  ptdump: write_buffer_wrapped: start_idx: 42 len: 576 output_off: 2a240 (240) page: ffff937f5182a000
  [30] buffer dump: dump.30
  [30] packet decode: decode.30
  [31] buffer dump: dump.31
  [31] packet decode: decode.31
  [32] buffer dump: dump.32
  [32] packet decode: decode.32
  ptdump: write_buffer_wrapped: start_idx: 659 len: 1104 output_off: 293450 (450) page: ffff937f52a93000
  [33] buffer dump: dump.33
  [33] packet decode: decode.33
  [34] buffer dump: dump.34
  [34] packet decode: decode.34
  ptdump: write_buffer_wrapped: start_idx: 516 len: 4048 output_off: 204fd0 (fd0) page: ffff937f53204000
  [35] buffer dump: dump.35
  [35] packet decode: decode.35
  [36] buffer dump: dump.36
  [36] packet decode: decode.36
  [37] buffer dump: dump.37
  [37] packet decode: decode.37
  [38] buffer dump: dump.38
  [38] packet decode: decode.38
  ptdump: write_buffer_wrapped: start_idx: 699 len: 3776 output_off: 2bbec0 (ec0) page: ffff937f542bb000
  [39] buffer dump: dump.39
  [39] packet decode: decode.39
  ptdump: write_buffer_wrapped: start_idx: 44 len: 960 output_off: 2c3c0 (3c0) page: ffff937f5482c000
  [40] buffer dump: dump.40
  [40] packet decode: decode.40
  ptdump: write_buffer_wrapped: start_idx: 740 len: 2208 output_off: 2e48a0 (8a0) page: ffff937f54ee4000
  [41] buffer dump: dump.41
  [41] packet decode: decode.41
  ptdump: write_buffer_wrapped: start_idx: 512 len: 2544 output_off: 2009f0 (9f0) page: ffff937f55200000
  [42] buffer dump: dump.42
  [42] packet decode: decode.42
  ptdump: write_buffer_wrapped: start_idx: 60 len: 1856 output_off: 3c740 (740) page: ffff937f5543c000
  [43] buffer dump: dump.43
  [43] packet decode: decode.43
  crash> 

That's probably not the correct way to fix the bug, but is it legitimate
for the pt_info_ptr->output_off to be on a page boundary?

Adding Takao to the NEEDINFO list.

Takao is the author/maintainer of crash-ptdump-command package:

  http://people.redhat.com/anderson/extensions.html#PTDUMP

Dave,

> Adding Takao to the NEEDINFO list.
> 
> Takao is the author/maintainer of crash-ptdump-command package:
> 
>   http://people.redhat.com/anderson/extensions.html#PTDUMP

Sorry for the delayed response.

I have started investigating the issue. Please wait for a while.

By the way, Indoh moved to another team some years ago and he now doesn't maintain ptdump module.
Could you replace the author information at extensions.html by the following signature?

    HATAYAMA Daisuke <d.hatayama>

Thanks.
HATAYAMA, Daisuke

(In reply to fj-lsoft-rh-dump from comment #6)
> Dave,
> 
> > Adding Takao to the NEEDINFO list.
> > 
> > Takao is the author/maintainer of crash-ptdump-command package:
> > 
> >   http://people.redhat.com/anderson/extensions.html#PTDUMP
> 
> Sorry for the delayed response.
> 
> I have started investigating the issue. Please wait for a while.
> 
> By the way, Indoh moved to another team some years ago and he now doesn't maintain ptdump module.
> Could you replace the author information at extensions.html by the following signature?
> 
>     HATAYAMA Daisuke <d.hatayama>

Ok thanks -- done!

Dave

Dave,

> Ok thanks -- done!

Thanks for the update. I confirmed that.
I'll post the next version of ptdump that includes the fix for this issue in crash devel ML after testing.

Oguma-san,

Could you monitor this bugzilla ticket?

Could you also clone this bugzilla ticket for RHEL7.8?
This issue occurs in the RHEL7 series, too.

Thanks.
HATAYAMA, Daisuke

(In reply to fj-lsoft-rh-dump from comment #8)
Hatayama-san,

> Could you monitor this bugzilla ticket?

OK. I will monitor this ticket.

> Could you also clone this bugzilla ticket for RHEL7.8?
> This issue occurs in the RHEL7 series, too.

For RHEL7 - https://bugzilla.redhat.com/show_bug.cgi?id=1791525

Best Regards,
Yoko Oguma

A patch to address this issue has been posted upstream:

  [Crash-utility] Extensions: ptdump update v1.0.7
  https://www.redhat.com/archives/crash-utility/2020-January/msg00007.html

  Hi Dave,

  The attached file is updated extension module ptdump v1.0.7.
  Please replace old one in extension site [1] with this tarball.

  Changelog:
  - ptdump: fix build warning: warning: this ‘if’ clause does not guard
  - ptdump: fix failure: ptdump: invalid size request: 0 type: "read page for write" 
  - ptdump: fix heap memory and fd leak when fault happens

  md5sum:
  b548afa3c44b6e7f81bce020297a1572

  [1] http://people.redhat.com/anderson/extensions.html#PTDUMP

  --
  Thanks.
  HATAYAMA, Daisuke

Created attachment 1655845 [details]
patch

Dave,

> A patch to address this issue has been posted upstream:
> 
>   [Crash-utility] Extensions: ptdump update v1.0.7
>   https://www.redhat.com/archives/crash-utility/2020-January/msg00007.html

I attached the patch, 0001-ptdump-fix-failure-ptdump-invalid-size-request-0-typ.patch, to fix the issue tracked here.
Could you include it in the crash-ptdump-command package?

Thanks.
HATAYAMA, Daisuke

I've been keeping the RHEL package in sync with the upstream version,
so I've already created the patch.  However, I will add your explanation
to the patch file.

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:1749