Description of problem: The build eccodes-2.23.0-1.el7 is compiled without option -DENABLE_AEC=ON (usage of Adaptive Entropy Coding library). Starting on 27/06/2023 all ECMWF GRIB2 files will be disseminated using a CCSDS defined compression method, so it will be impossible to open these GRIB2 files with eccodes. Version-Release number of selected component (if applicable): RHEL 7, EPEL Steps to Reproduce: [vladimir@ims-db]# grib_dump z_cams_c_ecmf_20230510120000_test_an_ml137_000_aermr01.grib ***** FILE: z_cams_c_ecmf_20230510120000_test_an_ml137_000_aermr01.grib #============== MESSAGE 1 ( length=834501 ) ============== GRIB { # Unknown code table entry (grib2/tables/27/0.0.table) discipline = 192; editionNumber = 2; # European Centre for Medium-Range Weather Forecasts (common/c-11.table) centre = 98; subCentre = 0; # Start of forecast (grib2/tables/27/1.2.table) significanceOfReferenceTime = 1; dataDate = 20230510; dataTime = 1200; # Operational products (grib2/tables/27/1.3.table) productionStatusOfProcessedData = 0; # Analysis products (grib2/tables/27/1.4.table) typeOfProcessedData = 0; # MARS labelling (grib2/grib2LocalSectionNumber.98.table) grib2LocalSectionNumber = 1; # Copernicus Atmosphere Monitoring Service (CAMS, previously MACC) (mars/class.table) marsClass = 19; # Analysis (mars/type.table) marsType = 2; # Atmospheric model (mars/stream.table) marsStream = 1025; experimentVersionNumber = 0078; numberOfDataPoints = 405900; # There is no appended list (grib2/tables/27/3.11.table) interpretationOfNumberOfPoints = 0; # Latitude/longitude (Also called equidistant cylindrical, or Plate Carree) (grib2/tables/27/3.1.table) gridDefinitionTemplateNumber = 0; # Earth assumed spherical with radius of 6 371 229.0 m (grib2/tables/27/3.2.table) shapeOfTheEarth = 6; Ni = 900; Nj = 451; iScansNegatively = 0; jScansPositively = 0; jPointsAreConsecutive = 0; alternativeRowScanning = 0; latitudeOfFirstGridPointInDegrees = 90; longitudeOfFirstGridPointInDegrees = 0; latitudeOfLastGridPointInDegrees = -90; longitudeOfLastGridPointInDegrees = 359.6; iDirectionIncrementInDegrees = 0.4; jDirectionIncrementInDegrees = 0.4; gridType = regular_ll; NV = 276; # Analysis or forecast at a horizontal level or in a horizontal layer at a point in time (grib2/tables/27/4.0.table) productDefinitionTemplateNumber = 0; # Unknown code table entry (grib2/tables/27/4.1.192.table) parameterCategory = 210; # Unknown code table entry () parameterNumber = 1; #-READ ONLY- parameterUnits = 1; #-READ ONLY- parameterName = 1; # Analysis (grib2/tables/27/4.3.table) typeOfGeneratingProcess = 0; generatingProcessIdentifier = 154; # Hour (grib2/tables/27/4.4.table) indicatorOfUnitOfTimeRange = 1; # Hour (stepUnits.table) stepUnits = 1; forecastTime = 0; stepRange = 0; # Hybrid level (-) (grib2/tables/27/4.5.table) typeOfFirstFixedSurface = 105; #-READ ONLY- unitsOfFirstFixedSurface = -; #-READ ONLY- nameOfFirstFixedSurface = Hybrid level ; scaleFactorOfFirstFixedSurface = 0; scaledValueOfFirstFixedSurface = 137; # Missing (grib2/tables/27/4.5.table) typeOfSecondFixedSurface = 255; #-READ ONLY- unitsOfSecondFixedSurface = unknown; #-READ ONLY- nameOfSecondFixedSurface = Missing ; scaleFactorOfSecondFixedSurface = MISSING; scaledValueOfSecondFixedSurface = MISSING; level = 137; shortNameLegacyECMF = unknown; shortNameECMF = aermr01; shortName = aermr01; nameLegacyECMF = unknown; nameECMF = Sea Salt Aerosol (0.03 - 0.5 um) Mixing Ratio; name = Sea Salt Aerosol (0.03 - 0.5 um) Mixing Ratio; cfNameLegacyECMF = unknown; cfNameECMF = unknown; cfName = unknown; cfVarNameLegacyECMF = unknown; cfVarNameECMF = aermr01; cfVarName = aermr01; #-READ ONLY- modelName = unknown; pv(276) = { 0, 2.00037, 3.10224, 4.66608, 6.82798, 9.74697, 13.6054, 18.6089, 24.9857, 32.9857, 42.8792, 54.9555, 69.5206, 86.8959, 107.416, 131.426, 159.279, 191.339, 227.969, 269.54, 316.421, 368.982, 427.592, 492.616, 564.413, 643.34, 729.744, 823.968, 926.345, 1037.2, 1156.85, 1285.61, 1423.77, 1571.62, 1729.45, 1897.52, 2076.1, 2265.43, 2465.77, 2677.35, 2900.39, 3135.12, 3381.74, 3640.47, 3911.49, 4194.93, 4490.82, 4799.15, 5119.9, 5452.99, 5798.34, 6156.07, 6526.95, 6911.87, 7311.87, 7727.41, 8159.35, 8608.53, 9076.4, 9562.68, 10066, 10584.6, 11116.7, 11660.1, 12211.5, 12766.9, 13324.7, 13881.3, 14432.1, 14975.6, 15508.3, 16026.1, 16527.3, 17008.8, 17467.6, 17901.6, 18308.4, 18685.7, 19031.3, 19343.5, 19620, 19859.4, 20059.9, 20219.7, 20337.9, 20412.3, 20442.1, 20425.7, 20361.8, 20249.5, 20087.1, 19874, 19608.6, 19290.2, 18917.5, 18489.7, 18006.9, 17471.8, 16888.7, 16262 ... 176 more values } numberOfValues = 405900; packingType = grid_ccsds; ccsdsFlags = 14; ccsdsBlockSize = 32; ccsdsRsi = 128; # A bit map does not apply to this product (grib2/tables/27/6.0.table) bitMapIndicator = 255; bitmapPresent = 0; values(405900) = { ECCODES ERROR : grib_accessor_data_ccsds_packing: CCSDS support not enabled. Please rebuild with -DENABLE_AEC=ON (Adaptive Entropy Coding library) *** ERR=-67 (Functionality not enabled) [grib_dumper_default::dump_values] ECCODES ERROR : grib_accessor_data_ccsds_packing: CCSDS support not enabled. Please rebuild with -DENABLE_AEC=ON (Adaptive Entropy Coding library) ECCODES ERROR : unable to get values as double array (Functionality not enabled) } #-READ ONLY- maximum = 0; # *** ERR=-67 (Functionality not enabled) [grib_dumper_default::dump_double] ECCODES ERROR : grib_accessor_data_ccsds_packing: CCSDS support not enabled. Please rebuild with -DENABLE_AEC=ON (Adaptive Entropy Coding library) ECCODES ERROR : unable to get values as double array (Functionality not enabled) #-READ ONLY- minimum = 0; # *** ERR=-67 (Functionality not enabled) [grib_dumper_default::dump_double] ECCODES ERROR : grib_accessor_data_ccsds_packing: CCSDS support not enabled. Please rebuild with -DENABLE_AEC=ON (Adaptive Entropy Coding library) ECCODES ERROR : unable to get values as double array (Functionality not enabled) #-READ ONLY- average = 0; # *** ERR=-67 (Functionality not enabled) [grib_dumper_default::dump_double] #-READ ONLY- numberOfMissing = 0; ECCODES ERROR : grib_accessor_data_ccsds_packing: CCSDS support not enabled. Please rebuild with -DENABLE_AEC=ON (Adaptive Entropy Coding library) ECCODES ERROR : unable to get values as double array (Functionality not enabled) #-READ ONLY- standardDeviation = 0; # *** ERR=-67 (Functionality not enabled) [grib_dumper_default::dump_double] ECCODES ERROR : grib_accessor_data_ccsds_packing: CCSDS support not enabled. Please rebuild with -DENABLE_AEC=ON (Adaptive Entropy Coding library) ECCODES ERROR : unable to get values as double array (Functionality not enabled) #-READ ONLY- skewness = 0; # *** ERR=-67 (Functionality not enabled) [grib_dumper_default::dump_double] ECCODES ERROR : grib_accessor_data_ccsds_packing: CCSDS support not enabled. Please rebuild with -DENABLE_AEC=ON (Adaptive Entropy Coding library) ECCODES ERROR : unable to get values as double array (Functionality not enabled) #-READ ONLY- kurtosis = 0; # *** ERR=-67 (Functionality not enabled) [grib_dumper_default::dump_double] ECCODES ERROR : grib_accessor_data_ccsds_packing: CCSDS support not enabled. Please rebuild with -DENABLE_AEC=ON (Adaptive Entropy Coding library) ECCODES ERROR : unable to get values as double array (Functionality not enabled) #-READ ONLY- isConstant = 0; # *** ERR=-67 (Functionality not enabled) [grib_dumper_default::dump_double] #-READ ONLY- getNumberOfValues = 405900;
Thanks for your report. I am working on upgrading this package to version 2.30.0 and will ensure ENABLE_AEC=ON is set. That should solve this problem.
FEDORA-EPEL-2023-af32eed15c has been submitted as an update to Fedora EPEL 7. https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2023-af32eed15c
FEDORA-EPEL-2023-f971004f31 has been submitted as an update to Fedora EPEL 8. https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2023-f971004f31
FEDORA-EPEL-2023-7657389682 has been submitted as an update to Fedora EPEL 9. https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2023-7657389682
For the record, here are some references why this update is needed: ECMWF will be starting to generate data files soon that can only be accessed using this version. As stated on this page: https://confluence.ecmwf.int/display/COPSRV/Implementation+of+IFS+cycle+48r1+for+CAMS "To handle CCSDS compressed fields from 48r1 with ecCodes, version 2.30.0 or newer is recommended." This version contains some changes needed to decode the new compressed GRIB files. Apart from this the changes are mostly in the file definition tables that come with this package, which are needed to read the latest fields that have been added in the formal WMO (World Meteorological Organization) table definitions.
As can be seen here: https://confluence.ecmwf.int/display/ECC/History+of+Changes version 2.30 adds [ECC-1452] - GRIB2: Update tables/templates with the latest WMO V30.0.0 it also includes this update from 2.28.0: [ECC-1483] - BUFR: Update tables with the latest WMO V39.0.0 and many smaller additions.
FEDORA-EPEL-2023-af32eed15c has been pushed to the Fedora EPEL 7 testing repository. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2023-af32eed15c See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.
FEDORA-EPEL-2023-f971004f31 has been pushed to the Fedora EPEL 8 testing repository. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2023-f971004f31 See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.
FEDORA-EPEL-2023-7657389682 has been pushed to the Fedora EPEL 9 testing repository. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2023-7657389682 See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.
I installed the new test version 2.30.0-1.el7 on the test server (RHEL 7) and ran the same command, this time with no errors: vladimir@smt-db-lnx-test ~]# grib_dump z_cams_c_ecmf_20230510120000_test_an_ml137_000_aermr01.grib ***** FILE: z_cams_c_ecmf_20230510120000_test_an_ml137_000_aermr01.grib #============== MESSAGE 1 ( length=834501 ) ============== GRIB { # Unknown code table entry (grib2/tables/30/0.0.table) discipline = 192; editionNumber = 2; # European Centre for Medium-Range Weather Forecasts (common/c-11.table) centre = 98; subCentre = 0; # Start of forecast (grib2/tables/30/1.2.table) significanceOfReferenceTime = 1; dataDate = 20230510; dataTime = 1200; # Operational products (grib2/tables/30/1.3.table) productionStatusOfProcessedData = 0; # Analysis products (grib2/tables/30/1.4.table) typeOfProcessedData = 0; # MARS labelling (grib2/grib2LocalSectionNumber.98.table) grib2LocalSectionNumber = 1; # Copernicus Atmosphere Monitoring Service (CAMS, previously MACC) (mars/class.table) marsClass = 19; # Analysis (mars/type.table) marsType = 2; # Atmospheric model (mars/stream.table) marsStream = 1025; experimentVersionNumber = 0078; numberOfDataPoints = 405900; # There is no appended list (grib2/tables/30/3.11.table) interpretationOfNumberOfPoints = 0; # Latitude/longitude (Also called equidistant cylindrical, or Plate Carree) (grib2/tables/30/3.1.table) gridDefinitionTemplateNumber = 0; # Earth assumed spherical with radius of 6 371 229.0 m (grib2/tables/30/3.2.table) shapeOfTheEarth = 6; Ni = 900; Nj = 451; iScansNegatively = 0; jScansPositively = 0; jPointsAreConsecutive = 0; alternativeRowScanning = 0; latitudeOfFirstGridPointInDegrees = 90; longitudeOfFirstGridPointInDegrees = 0; latitudeOfLastGridPointInDegrees = -90; longitudeOfLastGridPointInDegrees = 359.6; iDirectionIncrementInDegrees = 0.4; jDirectionIncrementInDegrees = 0.4; gridType = regular_ll; NV = 276; # Analysis or forecast at a horizontal level or in a horizontal layer at a point in time (grib2/tables/30/4.0.table) productDefinitionTemplateNumber = 0; # Unknown code table entry (grib2/tables/30/4.1.192.table) parameterCategory = 210; # Unknown code table entry () parameterNumber = 1; #-READ ONLY- parameterUnits = 1; #-READ ONLY- parameterName = 1; # Analysis (grib2/tables/30/4.3.table) typeOfGeneratingProcess = 0; generatingProcessIdentifier = 154; # Hour (grib2/tables/30/4.4.table) indicatorOfUnitOfTimeRange = 1; # Hour (stepUnits.table) stepUnits = 1; forecastTime = 0; stepRange = 0; # Hybrid level (-) (grib2/tables/30/4.5.table) typeOfFirstFixedSurface = 105; #-READ ONLY- unitsOfFirstFixedSurface = -; #-READ ONLY- nameOfFirstFixedSurface = Hybrid level; scaleFactorOfFirstFixedSurface = 0; scaledValueOfFirstFixedSurface = 137; # Missing (grib2/tables/30/4.5.table) typeOfSecondFixedSurface = 255; #-READ ONLY- unitsOfSecondFixedSurface = unknown; #-READ ONLY- nameOfSecondFixedSurface = Missing; scaleFactorOfSecondFixedSurface = MISSING; scaledValueOfSecondFixedSurface = MISSING; level = 137; shortName = aermr01; name = Sea Salt Aerosol (0.03 - 0.5 um) Mixing Ratio; cfName = unknown; cfVarName = aermr01; #-READ ONLY- modelName = unknown; pv(276) = { 0, 2.00037, 3.10224, 4.66608, 6.82798, 9.74697, 13.6054, 18.6089, 24.9857, 32.9857, 42.8792, 54.9555, 69.5206, 86.8959, 107.416, 131.426, 159.279, 191.339, 227.969, 269.54, 316.421, 368.982, 427.592, 492.616, 564.413, 643.34, 729.744, 823.968, 926.345, 1037.2, 1156.85, 1285.61, 1423.77, 1571.62, 1729.45, 1897.52, 2076.1, 2265.43, 2465.77, 2677.35, 2900.39, 3135.12, 3381.74, 3640.47, 3911.49, 4194.93, 4490.82, 4799.15, 5119.9, 5452.99, 5798.34, 6156.07, 6526.95, 6911.87, 7311.87, 7727.41, 8159.35, 8608.53, 9076.4, 9562.68, 10066, 10584.6, 11116.7, 11660.1, 12211.5, 12766.9, 13324.7, 13881.3, 14432.1, 14975.6, 15508.3, 16026.1, 16527.3, 17008.8, 17467.6, 17901.6, 18308.4, 18685.7, 19031.3, 19343.5, 19620, 19859.4, 20059.9, 20219.7, 20337.9, 20412.3, 20442.1, 20425.7, 20361.8, 20249.5, 20087.1, 19874, 19608.6, 19290.2, 18917.5, 18489.7, 18006.9, 17471.8, 16888.7, 16262 ... 176 more values } numberOfValues = 405900; packingType = grid_ccsds; ccsdsFlags = 14; ccsdsBlockSize = 32; ccsdsRsi = 128; # A bit map does not apply to this product (grib2/tables/30/6.0.table) bitMapIndicator = 255; bitmapPresent = 0; values(405900) = { 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11, 1.46816e-11 ... 405800 more values } #-READ ONLY- maximum = 1.91365e-08; #-READ ONLY- minimum = 0; #-READ ONLY- average = 2.63339e-09; #-READ ONLY- standardDeviation = 2.82749e-09; #-READ ONLY- skewness = 1.02402; #-READ ONLY- kurtosis = 0.586596; #-READ ONLY- isConstant = 0; #-READ ONLY- numberOfMissing = 0; #-READ ONLY- getNumberOfValues = 405900; In addition, I compared the results of the command grib_get_data z_cams_c_ecmf_20230510120000_test_an_ml137_000_aermr01.grib for the new version on RHEL 7 and the old version 2.26.0-1.el9 on RHEL 9 (seems to have been built with -DENABLE_AEC=ON option). The results are identical.
FEDORA-EPEL-2023-7657389682 has been pushed to the Fedora EPEL 9 stable repository. If problem still persists, please make note of it in this bug report.
FEDORA-EPEL-2023-af32eed15c has been pushed to the Fedora EPEL 7 stable repository. If problem still persists, please make note of it in this bug report.
FEDORA-EPEL-2023-f971004f31 has been pushed to the Fedora EPEL 8 stable repository. If problem still persists, please make note of it in this bug report.