Bug 2208249 - eccodes-2.23.0-1.el7
Summary: eccodes-2.23.0-1.el7
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora EPEL
Classification: Fedora
Component: eccodes
Version: epel7
Hardware: x86_64
OS: Linux
unspecified
unspecified
Target Milestone: ---
Assignee: Jos de Kloe
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2023-05-18 11:24 UTC by Vladimir Meerson
Modified: 2023-05-29 01:14 UTC (History)
1 user (show)

Fixed In Version: eccodes-2.30.0-1.el9 eccodes-2.30.0-1.el7 eccodes-2.30.0-1.el8
Clone Of:
Environment:
Last Closed: 2023-05-29 00:46:57 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Vladimir Meerson 2023-05-18 11:24:03 UTC
Description of problem:

The build  eccodes-2.23.0-1.el7 is compiled without option -DENABLE_AEC=ON (usage of Adaptive Entropy Coding library).

Starting on 27/06/2023 all ECMWF GRIB2 files will be disseminated using a CCSDS defined compression method, so it will be impossible to open these GRIB2 files with eccodes.

Version-Release number of selected component (if applicable):
RHEL 7, EPEL

Steps to Reproduce:

[vladimir@ims-db]# grib_dump z_cams_c_ecmf_20230510120000_test_an_ml137_000_aermr01.grib


***** FILE: z_cams_c_ecmf_20230510120000_test_an_ml137_000_aermr01.grib
#==============   MESSAGE 1 ( length=834501 )              ==============
GRIB {
  # Unknown code table entry (grib2/tables/27/0.0.table)
  discipline = 192;
  editionNumber = 2;
  # European Centre for Medium-Range Weather Forecasts (common/c-11.table)
  centre = 98;
  subCentre = 0;
  # Start of forecast  (grib2/tables/27/1.2.table)
  significanceOfReferenceTime = 1;
  dataDate = 20230510;
  dataTime = 1200;
  # Operational products  (grib2/tables/27/1.3.table)
  productionStatusOfProcessedData = 0;
  # Analysis products  (grib2/tables/27/1.4.table)
  typeOfProcessedData = 0;
  # MARS labelling (grib2/grib2LocalSectionNumber.98.table)
  grib2LocalSectionNumber = 1;
  # Copernicus Atmosphere Monitoring Service  (CAMS, previously MACC)  (mars/class.table)
  marsClass = 19;
  # Analysis (mars/type.table)
  marsType = 2;
  # Atmospheric model (mars/stream.table)
  marsStream = 1025;
  experimentVersionNumber = 0078;
  numberOfDataPoints = 405900;
  # There is no appended list  (grib2/tables/27/3.11.table)
  interpretationOfNumberOfPoints = 0;
  # Latitude/longitude  (Also called equidistant cylindrical, or Plate Carree)  (grib2/tables/27/3.1.table)
  gridDefinitionTemplateNumber = 0;
  # Earth assumed spherical with radius of 6 371 229.0 m  (grib2/tables/27/3.2.table)
  shapeOfTheEarth = 6;
  Ni = 900;
  Nj = 451;
  iScansNegatively = 0;
  jScansPositively = 0;
  jPointsAreConsecutive = 0;
  alternativeRowScanning = 0;
  latitudeOfFirstGridPointInDegrees = 90;
  longitudeOfFirstGridPointInDegrees = 0;
  latitudeOfLastGridPointInDegrees = -90;
  longitudeOfLastGridPointInDegrees = 359.6;
  iDirectionIncrementInDegrees = 0.4;
  jDirectionIncrementInDegrees = 0.4;
  gridType = regular_ll;
  NV = 276;
  # Analysis or forecast at a horizontal level or in a horizontal layer at a point in time  (grib2/tables/27/4.0.table)
  productDefinitionTemplateNumber = 0;
  # Unknown code table entry (grib2/tables/27/4.1.192.table)
  parameterCategory = 210;
  # Unknown code table entry ()
  parameterNumber = 1;
  #-READ ONLY- parameterUnits = 1;
  #-READ ONLY- parameterName = 1;
  # Analysis  (grib2/tables/27/4.3.table)
  typeOfGeneratingProcess = 0;
  generatingProcessIdentifier = 154;
  # Hour  (grib2/tables/27/4.4.table)
  indicatorOfUnitOfTimeRange = 1;
  # Hour (stepUnits.table)
  stepUnits = 1;
  forecastTime = 0;
  stepRange = 0;
  # Hybrid level  (-)  (grib2/tables/27/4.5.table)
  typeOfFirstFixedSurface = 105;
  #-READ ONLY- unitsOfFirstFixedSurface = -;
  #-READ ONLY- nameOfFirstFixedSurface = Hybrid level ;
  scaleFactorOfFirstFixedSurface = 0;
  scaledValueOfFirstFixedSurface = 137;
  # Missing  (grib2/tables/27/4.5.table)
  typeOfSecondFixedSurface = 255;
  #-READ ONLY- unitsOfSecondFixedSurface = unknown;
  #-READ ONLY- nameOfSecondFixedSurface = Missing ;
  scaleFactorOfSecondFixedSurface = MISSING;
  scaledValueOfSecondFixedSurface = MISSING;
  level = 137;
  shortNameLegacyECMF = unknown;
  shortNameECMF = aermr01;
  shortName = aermr01;
  nameLegacyECMF = unknown;
  nameECMF = Sea Salt Aerosol (0.03 - 0.5 um) Mixing Ratio;
  name = Sea Salt Aerosol (0.03 - 0.5 um) Mixing Ratio;
  cfNameLegacyECMF = unknown;
  cfNameECMF = unknown;
  cfName = unknown;
  cfVarNameLegacyECMF = unknown;
  cfVarNameECMF = aermr01;
  cfVarName = aermr01;
  #-READ ONLY- modelName = unknown;
  pv(276) =  {
  0, 2.00037, 3.10224, 4.66608, 6.82798,
  9.74697, 13.6054, 18.6089, 24.9857, 32.9857,
  42.8792, 54.9555, 69.5206, 86.8959, 107.416,
  131.426, 159.279, 191.339, 227.969, 269.54,
  316.421, 368.982, 427.592, 492.616, 564.413,
  643.34, 729.744, 823.968, 926.345, 1037.2,
  1156.85, 1285.61, 1423.77, 1571.62, 1729.45,
  1897.52, 2076.1, 2265.43, 2465.77, 2677.35,
  2900.39, 3135.12, 3381.74, 3640.47, 3911.49,
  4194.93, 4490.82, 4799.15, 5119.9, 5452.99,
  5798.34, 6156.07, 6526.95, 6911.87, 7311.87,
  7727.41, 8159.35, 8608.53, 9076.4, 9562.68,
  10066, 10584.6, 11116.7, 11660.1, 12211.5,
  12766.9, 13324.7, 13881.3, 14432.1, 14975.6,
  15508.3, 16026.1, 16527.3, 17008.8, 17467.6,
  17901.6, 18308.4, 18685.7, 19031.3, 19343.5,
  19620, 19859.4, 20059.9, 20219.7, 20337.9,
  20412.3, 20442.1, 20425.7, 20361.8, 20249.5,
  20087.1, 19874, 19608.6, 19290.2, 18917.5,
  18489.7, 18006.9, 17471.8, 16888.7, 16262
  ... 176 more values
  }
  numberOfValues = 405900;
  packingType = grid_ccsds;
  ccsdsFlags = 14;
  ccsdsBlockSize = 32;
  ccsdsRsi = 128;
  # A bit map does not apply to this product  (grib2/tables/27/6.0.table)
  bitMapIndicator = 255;
  bitmapPresent = 0;
  values(405900) =  {
ECCODES ERROR   :  grib_accessor_data_ccsds_packing: CCSDS support not enabled. Please rebuild with -DENABLE_AEC=ON (Adaptive Entropy Coding library)
 *** ERR=-67 (Functionality not enabled) [grib_dumper_default::dump_values]
ECCODES ERROR   :  grib_accessor_data_ccsds_packing: CCSDS support not enabled. Please rebuild with -DENABLE_AEC=ON (Adaptive Entropy Coding library)
ECCODES ERROR   :  unable to get values as double array (Functionality not enabled)
}  #-READ ONLY- maximum = 0;  # *** ERR=-67 (Functionality not enabled) [grib_dumper_default::dump_double]
ECCODES ERROR   :  grib_accessor_data_ccsds_packing: CCSDS support not enabled. Please rebuild with -DENABLE_AEC=ON (Adaptive Entropy Coding library)
ECCODES ERROR   :  unable to get values as double array (Functionality not enabled)
  #-READ ONLY- minimum = 0;  # *** ERR=-67 (Functionality not enabled) [grib_dumper_default::dump_double]
ECCODES ERROR   :  grib_accessor_data_ccsds_packing: CCSDS support not enabled. Please rebuild with -DENABLE_AEC=ON (Adaptive Entropy Coding library)
ECCODES ERROR   :  unable to get values as double array (Functionality not enabled)
  #-READ ONLY- average = 0;  # *** ERR=-67 (Functionality not enabled) [grib_dumper_default::dump_double]
  #-READ ONLY- numberOfMissing = 0;
ECCODES ERROR   :  grib_accessor_data_ccsds_packing: CCSDS support not enabled. Please rebuild with -DENABLE_AEC=ON (Adaptive Entropy Coding library)
ECCODES ERROR   :  unable to get values as double array (Functionality not enabled)
  #-READ ONLY- standardDeviation = 0;  # *** ERR=-67 (Functionality not enabled) [grib_dumper_default::dump_double]
ECCODES ERROR   :  grib_accessor_data_ccsds_packing: CCSDS support not enabled. Please rebuild with -DENABLE_AEC=ON (Adaptive Entropy Coding library)
ECCODES ERROR   :  unable to get values as double array (Functionality not enabled)
  #-READ ONLY- skewness = 0;  # *** ERR=-67 (Functionality not enabled) [grib_dumper_default::dump_double]
ECCODES ERROR   :  grib_accessor_data_ccsds_packing: CCSDS support not enabled. Please rebuild with -DENABLE_AEC=ON (Adaptive Entropy Coding library)
ECCODES ERROR   :  unable to get values as double array (Functionality not enabled)
  #-READ ONLY- kurtosis = 0;  # *** ERR=-67 (Functionality not enabled) [grib_dumper_default::dump_double]
ECCODES ERROR   :  grib_accessor_data_ccsds_packing: CCSDS support not enabled. Please rebuild with -DENABLE_AEC=ON (Adaptive Entropy Coding library)
ECCODES ERROR   :  unable to get values as double array (Functionality not enabled)
  #-READ ONLY- isConstant = 0;  # *** ERR=-67 (Functionality not enabled) [grib_dumper_default::dump_double]
  #-READ ONLY- getNumberOfValues = 405900;

Comment 1 Jos de Kloe 2023-05-18 14:20:46 UTC
Thanks for your report.
I am working on upgrading this package to version 2.30.0 and will ensure ENABLE_AEC=ON is set.
That should solve this problem.

Comment 2 Fedora Update System 2023-05-20 09:04:01 UTC
FEDORA-EPEL-2023-af32eed15c has been submitted as an update to Fedora EPEL 7. https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2023-af32eed15c

Comment 3 Fedora Update System 2023-05-20 09:04:32 UTC
FEDORA-EPEL-2023-f971004f31 has been submitted as an update to Fedora EPEL 8. https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2023-f971004f31

Comment 4 Fedora Update System 2023-05-20 09:04:58 UTC
FEDORA-EPEL-2023-7657389682 has been submitted as an update to Fedora EPEL 9. https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2023-7657389682

Comment 5 Jos de Kloe 2023-05-20 09:07:16 UTC
For the record, here are some references why this update is needed:

ECMWF will be starting to generate data files soon that can only be accessed using this version.
As stated on this page: https://confluence.ecmwf.int/display/COPSRV/Implementation+of+IFS+cycle+48r1+for+CAMS

"To handle CCSDS compressed fields from 48r1 with ecCodes, version 2.30.0 or newer is recommended."

This version contains some changes needed to decode the new compressed GRIB files.

Apart from this the changes are mostly in the file definition tables that come with this package, which are needed to read the latest fields that have been added in the formal WMO (World Meteorological Organization) table definitions.

Comment 6 Jos de Kloe 2023-05-20 09:11:12 UTC
As can be seen here: https://confluence.ecmwf.int/display/ECC/History+of+Changes
version 2.30 adds 
[ECC-1452] - GRIB2: Update tables/templates with the latest WMO V30.0.0
it also includes this update from 2.28.0:
[ECC-1483] - BUFR: Update tables with the latest WMO V39.0.0
and many smaller additions.

Comment 7 Fedora Update System 2023-05-21 00:23:28 UTC
FEDORA-EPEL-2023-af32eed15c has been pushed to the Fedora EPEL 7 testing repository.

You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2023-af32eed15c

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 8 Fedora Update System 2023-05-21 00:23:36 UTC
FEDORA-EPEL-2023-f971004f31 has been pushed to the Fedora EPEL 8 testing repository.

You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2023-f971004f31

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 9 Fedora Update System 2023-05-21 00:23:49 UTC
FEDORA-EPEL-2023-7657389682 has been pushed to the Fedora EPEL 9 testing repository.

You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2023-7657389682

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 10 Vladimir Meerson 2023-05-21 10:06:49 UTC
I installed the new test version 2.30.0-1.el7 on the test server (RHEL 7) and ran the same command, this time with no errors:

 
vladimir@smt-db-lnx-test ~]# grib_dump z_cams_c_ecmf_20230510120000_test_an_ml137_000_aermr01.grib
***** FILE: z_cams_c_ecmf_20230510120000_test_an_ml137_000_aermr01.grib
#==============   MESSAGE 1 ( length=834501 )              ==============
GRIB {
  # Unknown code table entry (grib2/tables/30/0.0.table)  
  discipline = 192;
  editionNumber = 2;
  # European Centre for Medium-Range Weather Forecasts (common/c-11.table)  
  centre = 98;
  subCentre = 0;
  # Start of forecast (grib2/tables/30/1.2.table)  
  significanceOfReferenceTime = 1;
  dataDate = 20230510;
  dataTime = 1200;
  # Operational products (grib2/tables/30/1.3.table)  
  productionStatusOfProcessedData = 0;
  # Analysis products (grib2/tables/30/1.4.table)  
  typeOfProcessedData = 0;
  # MARS labelling (grib2/grib2LocalSectionNumber.98.table)  
  grib2LocalSectionNumber = 1;
  # Copernicus Atmosphere Monitoring Service (CAMS, previously MACC)  (mars/class.table)  
  marsClass = 19;
  # Analysis (mars/type.table)  
  marsType = 2;
  # Atmospheric model (mars/stream.table)  
  marsStream = 1025;
  experimentVersionNumber = 0078;
  numberOfDataPoints = 405900;
  # There is no appended list (grib2/tables/30/3.11.table)  
  interpretationOfNumberOfPoints = 0;
  # Latitude/longitude (Also called equidistant cylindrical, or Plate Carree)  (grib2/tables/30/3.1.table)  
  gridDefinitionTemplateNumber = 0;
  # Earth assumed spherical with radius of 6 371 229.0 m (grib2/tables/30/3.2.table)  
  shapeOfTheEarth = 6;
  Ni = 900;
  Nj = 451;
  iScansNegatively = 0;
  jScansPositively = 0;
  jPointsAreConsecutive = 0;
  alternativeRowScanning = 0;
  latitudeOfFirstGridPointInDegrees = 90;
  longitudeOfFirstGridPointInDegrees = 0;
  latitudeOfLastGridPointInDegrees = -90;
  longitudeOfLastGridPointInDegrees = 359.6;
  iDirectionIncrementInDegrees = 0.4;
  jDirectionIncrementInDegrees = 0.4;
  gridType = regular_ll;
  NV = 276;
  # Analysis or forecast at a horizontal level or in a horizontal layer at a point in time (grib2/tables/30/4.0.table)  
  productDefinitionTemplateNumber = 0;
  # Unknown code table entry (grib2/tables/30/4.1.192.table)  
  parameterCategory = 210;
  # Unknown code table entry ()  
  parameterNumber = 1;
  #-READ ONLY- parameterUnits = 1;
  #-READ ONLY- parameterName = 1;
  # Analysis (grib2/tables/30/4.3.table)  
  typeOfGeneratingProcess = 0;
  generatingProcessIdentifier = 154;
  # Hour (grib2/tables/30/4.4.table)  
  indicatorOfUnitOfTimeRange = 1;
  # Hour (stepUnits.table)  
  stepUnits = 1;
  forecastTime = 0;
  stepRange = 0;
  # Hybrid level (-)  (grib2/tables/30/4.5.table)  
  typeOfFirstFixedSurface = 105;
  #-READ ONLY- unitsOfFirstFixedSurface = -;
  #-READ ONLY- nameOfFirstFixedSurface = Hybrid level;
  scaleFactorOfFirstFixedSurface = 0;
  scaledValueOfFirstFixedSurface = 137;
  # Missing (grib2/tables/30/4.5.table)  
  typeOfSecondFixedSurface = 255;
  #-READ ONLY- unitsOfSecondFixedSurface = unknown;
  #-READ ONLY- nameOfSecondFixedSurface = Missing;
  scaleFactorOfSecondFixedSurface = MISSING;
  scaledValueOfSecondFixedSurface = MISSING;
  level = 137;
  shortName = aermr01;
  name = Sea Salt Aerosol (0.03 - 0.5 um) Mixing Ratio;
  cfName = unknown;
  cfVarName = aermr01;
  #-READ ONLY- modelName = unknown;
  pv(276) =  {
  0, 2.00037, 3.10224, 4.66608, 6.82798,
  9.74697, 13.6054, 18.6089, 24.9857, 32.9857,
  42.8792, 54.9555, 69.5206, 86.8959, 107.416,
  131.426, 159.279, 191.339, 227.969, 269.54,
  316.421, 368.982, 427.592, 492.616, 564.413,
  643.34, 729.744, 823.968, 926.345, 1037.2,
  1156.85, 1285.61, 1423.77, 1571.62, 1729.45,
  1897.52, 2076.1, 2265.43, 2465.77, 2677.35,
  2900.39, 3135.12, 3381.74, 3640.47, 3911.49,
  4194.93, 4490.82, 4799.15, 5119.9, 5452.99,
  5798.34, 6156.07, 6526.95, 6911.87, 7311.87,
  7727.41, 8159.35, 8608.53, 9076.4, 9562.68,
  10066, 10584.6, 11116.7, 11660.1, 12211.5,
  12766.9, 13324.7, 13881.3, 14432.1, 14975.6,
  15508.3, 16026.1, 16527.3, 17008.8, 17467.6,
  17901.6, 18308.4, 18685.7, 19031.3, 19343.5,
  19620, 19859.4, 20059.9, 20219.7, 20337.9,
  20412.3, 20442.1, 20425.7, 20361.8, 20249.5,
  20087.1, 19874, 19608.6, 19290.2, 18917.5,
  18489.7, 18006.9, 17471.8, 16888.7, 16262
  ... 176 more values
  }
  numberOfValues = 405900;
  packingType = grid_ccsds;
  ccsdsFlags = 14;
  ccsdsBlockSize = 32;
  ccsdsRsi = 128;
  # A bit map does not apply to this product (grib2/tables/30/6.0.table)  
  bitMapIndicator = 255;
  bitmapPresent = 0;
  values(405900) =  {
  1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11,
  1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11,
  1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11,
  1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11,
  1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11,
  1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11,
  1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11,
  1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11,
  1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11,
  1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11,
  1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11,
  1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11,
  1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11,
  1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11,
  1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11,
  1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11,
  1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11,
  1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11,
  1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11,
  1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11
  ... 405800 more values
  }
  #-READ ONLY- maximum = 1.91365e-08;
  #-READ ONLY- minimum = 0;
  #-READ ONLY- average = 2.63339e-09;
  #-READ ONLY- standardDeviation = 2.82749e-09;
  #-READ ONLY- skewness = 1.02402;
  #-READ ONLY- kurtosis = 0.586596;
  #-READ ONLY- isConstant = 0;
  #-READ ONLY- numberOfMissing = 0;
  #-READ ONLY- getNumberOfValues = 405900;

In addition, I compared the results of the command grib_get_data z_cams_c_ecmf_20230510120000_test_an_ml137_000_aermr01.grib for the new version on RHEL 7 and the old version 2.26.0-1.el9 on RHEL 9 (seems to have been built with -DENABLE_AEC=ON option). The results are identical.

Comment 11 Fedora Update System 2023-05-29 00:46:57 UTC
FEDORA-EPEL-2023-7657389682 has been pushed to the Fedora EPEL 9 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 12 Fedora Update System 2023-05-29 00:47:32 UTC
FEDORA-EPEL-2023-af32eed15c has been pushed to the Fedora EPEL 7 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 13 Fedora Update System 2023-05-29 01:14:14 UTC
FEDORA-EPEL-2023-f971004f31 has been pushed to the Fedora EPEL 8 stable repository.
If problem still persists, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.