Bug 1856959

Summary: NCL error while trying to read GRIB2 files
Product: [Fedora] Fedora EPEL Reporter: Jim E <edgar1196>
Component: gdalAssignee: markusN <neteler>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: epel8CC: alex, devrim, jmlich83, mmahut, oliver, orion, pavel.lisy, pertusus, praiskup, scitech-bugs, volker27
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-08-04 03:37:59 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Jim E 2020-07-14 19:08:23 UTC
Description of problem:

An error occurs while trying to read the contents of a GRIB2-formatted file, either with ncl (via script) or ncl_filedump commands. This behavior doesn't happen with GRIB, netcdf or HDF5 formatted files.

Version-Release number of selected component (if applicable):

ncl-6.6.2-10.el8

How reproducible:


Steps to Reproduce:
 
ncl_filedump -c rap.t00z.awp200f00.grib2

Actual results:

Copyright (C) 1995-2019 - All Rights Reserved
 University Corporation for Atmospheric Research
 NCAR Command Language Version 6.6.2
 The use of this software is governed by a License Agreement.
 See http://www.ncl.ucar.edu/ for more details.
g2_getfld: Request for field number must be positive.
warning:Invalid data field request.

Variable: f
(0) File Missing Value : -1
warning:getfiledimsizes: undefined file variable
Segmentation fault (core dumped)

Expected results:

 Copyright (C) 1995-2019 - All Rights Reserved
  University Corporation for Atmospheric Research
  NCAR Command Language Version 6.6.2
  The use of this software is governed by a License Agreement.
  See http://www.ncl.ucar.edu/ for more details.
   
 Variable: f
 Type: file
 filename: rap.t00z.awp200f00
 path:     rap.t00z.awp200f00.grib2
 file global attributes:
  dimensions:
        ygrid_0 = 94
        xgrid_0 = 108
         lv_ISBL0 = 39
         lv_HTGL1 = 2
         lv_SPDL2 = 6
         lv_HTGL3 = 2
         lv_ISBL4 = 37
         lv_SPDL5 = 3
         lv_HTGL6 = 2
      variables:
 ....


Additional info:

1. Data source:   ftp://ftpprd.ncep.noaa.gov/pub/data/nccf/com/rap/prod/

2. Looking at the core dump file:
]$ coredumpctl info

           PID: 10046 (ncl)
           UID: 1000 ()
           GID: 1000 ()
          Signal: 11 (SEGV)
          Timestamp: Tue 2020-07-14 19:46:45 BST (14min ago)
          Command Line: ncl -n filename="rap.t00z.awp200f00.grib2" nvars=0 coords=1 ftype="GRIB" singleElemDims=(/"none"/) timePeriodSuffix=True /tmp/tmp10036.ncl
           Executable: /usr/bin/ncl
...
          Message: Process 10046 (ncl) of user 1000 dumped core.
                
                Stack trace of thread 10046:
                #0  0x00000000005ce174 _NclIGetVarDims (ncl)
                #1  0x00000000005878c9 CallINTRINSIC_FUNC_CALL (ncl)
                #2  0x000000000059261d _NclExecute (ncl)
                #3  0x000000000046f4da yyparse (ncl)
                #4  0x00000000004657f0 NclDriver (ncl)
                #5  0x00007f262c3386a3 __libc_start_main (libc.so.6)
                #6  0x0000000000461bce _start (ncl)

Comment 1 Orion Poplawski 2020-07-18 03:48:12 UTC
valgrind points to the issue:

==15801== Command: ncl -n filename="rap.t00z.awp200f00.grib2" nvars=0 coords=0 ftype="GRIB" singleElemDims=(/"none"/) timePeriodSuffix=True /tmp/tmp15791.ncl
==15801== 
 Copyright (C) 1995-2019 - All Rights Reserved
 University Corporation for Atmospheric Research
 NCAR Command Language Version 6.6.2
 The use of this software is governed by a License Agreement.
 See http://www.ncl.ucar.edu/ for more details.
==15801== Use of uninitialised value of size 8
==15801==    at 0x6D4708B: g2_getfld (in /usr/lib64/libgdal.so.26.0.4)
==15801==    by 0x56E4C5: ??? (in /usr/bin/ncl)
==15801==    by 0x4F9582: _NclFileCreate (in /usr/bin/ncl)
==15801==    by 0x4C25B9: _NclOpenFile (in /usr/bin/ncl)
==15801==    by 0x5B092C: _NclIAddFile (in /usr/bin/ncl)
==15801==    by 0x5878C8: CallINTRINSIC_FUNC_CALL (in /usr/bin/ncl)
==15801==    by 0x59261C: _NclExecute (in /usr/bin/ncl)
==15801==    by 0x46F4D9: yyparse (in /usr/bin/ncl)
==15801==    by 0x4657EF: NclDriver (in /usr/bin/ncl)
==15801==    by 0xC9796A2: (below main) (in /usr/lib64/libc-2.28.so)
==15801==
==15801==
==15801== Process terminating with default action of signal 11 (SIGSEGV): dumping core
==15801==  Bad permissions for mapped region at address 0x56E0AE
==15801==    at 0x6D4708B: g2_getfld (in /usr/lib64/libgdal.so.26.0.4)
==15801==    by 0x56E4C5: ??? (in /usr/bin/ncl)
==15801==    by 0x4F9582: _NclFileCreate (in /usr/bin/ncl)
==15801==    by 0x4C25B9: _NclOpenFile (in /usr/bin/ncl)
==15801==    by 0x5B092C: _NclIAddFile (in /usr/bin/ncl)
==15801==    by 0x5878C8: CallINTRINSIC_FUNC_CALL (in /usr/bin/ncl)
==15801==    by 0x59261C: _NclExecute (in /usr/bin/ncl)
==15801==    by 0x46F4D9: yyparse (in /usr/bin/ncl)
==15801==    by 0x4657EF: NclDriver (in /usr/bin/ncl)
==15801==    by 0xC9796A2: (below main) (in /usr/lib64/libc-2.28.so)

the issue is that gdal modified its internal copy of the g2clib to change g2_getfld() to use 6 arguments and they inserted that argument in the middle.  So ncl is trying to call the standard 5 arg g2_getfld() and ending up with gdal's 6 arg version which leads to chaos.

Comment 2 Orion Poplawski 2020-07-18 04:49:10 UTC
So, I can work around this in ncl by being sure to link g2clib before gdal.  But gdal really should get fixed to prevent other users from having this issue.

Comment 3 Fedora Update System 2020-07-18 14:18:10 UTC
FEDORA-2020-3d7cbc5979 has been submitted as an update to Fedora 32. https://bodhi.fedoraproject.org/updates/FEDORA-2020-3d7cbc5979

Comment 4 Jim E 2020-07-18 20:37:29 UTC
Indeed, ncl seems to be working alright except for this. Looking forward to the new update. Cheers.

Comment 5 Fedora Update System 2020-07-19 01:27:35 UTC
FEDORA-2020-3d7cbc5979 has been pushed to the Fedora 32 testing repository.
In short time you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2020-3d7cbc5979`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2020-3d7cbc5979

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 6 Fedora Update System 2020-07-19 02:00:15 UTC
FEDORA-EPEL-2020-b7ff36d3e7 has been pushed to the Fedora EPEL 8 testing repository.

You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2020-b7ff36d3e7

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 7 Fedora Admin user for bugzilla script actions 2020-07-28 14:55:49 UTC
This package has changed maintainer in the Fedora.
Reassigning to the new maintainer of this component.

Comment 8 Fedora Update System 2020-07-29 14:38:15 UTC
FEDORA-2020-3d7cbc5979 has been pushed to the Fedora 32 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 9 Fedora Update System 2020-07-29 21:16:42 UTC
FEDORA-2020-3d7cbc5979 has been pushed to the Fedora 32 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 10 Fedora Update System 2020-07-30 13:07:45 UTC
FEDORA-2020-3d7cbc5979 has been pushed to the Fedora 32 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 11 Fedora Update System 2020-07-30 17:50:43 UTC
FEDORA-2020-3d7cbc5979 has been pushed to the Fedora 32 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 12 Fedora Update System 2020-08-04 01:22:09 UTC
FEDORA-EPEL-2020-b7ff36d3e7 has been pushed to the Fedora EPEL 8 stable repository.
If problem still persists, please make note of it in this bug report.