Bug 2016685 - some objects uploaded to noobaa are duplicated in the listing
Summary: some objects uploaded to noobaa are duplicated in the listing
Keywords:
Status: CLOSED INSUFFICIENT_DATA
Alias: None
Product: Red Hat OpenShift Data Foundation
Classification: Red Hat Storage
Component: Multi-Cloud Object Gateway
Version: 4.8
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ---
: ---
Assignee: Nimrod Becker
QA Contact: Elad
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-10-22 16:49 UTC by Michal Minar
Modified: 2023-08-09 16:49 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2022-06-09 08:25:24 UTC
Embargoed:


Attachments (Terms of Use)

Description Michal Minar 2021-10-22 16:49:02 UTC
Description of problem (please be detailed as possible and provide log
snippests):
   the same object (having the same name/key) is listed twice or thrice in the output of aws CLI v2.

Version of all relevant components (if applicable):
  ocs-operator.v4.8.3

Does this issue impact your ability to continue to work with the product
(please explain in detail what is the user impact)? no

Is there any workaround available to the best of your knowledge?
  the listing can be deduplicated with e.g. `sort -u`


Rate from 1 - 5 the complexity of the scenario you performed that caused this
bug (1 - very simple, 5 - very complex)?
  3

Can this issue reproducible? happened to me in 2 out of 2 cases (once with 4.8.1, now with 4.8.3)


Can this issue reproduce from the UI? from SAP Data Intelligence's UI


If this is a regression, please provide more details to justify this:
  it is a regression from 4.6

Steps to Reproduce:
1. Install OCP, OCS operator and SAP Data Intelligence
2. create a noobaa S3 bucket with an OBC
3. configure S3 connection using http://s3.openshift-storage.svc.cluster.local endpoint
4. Run data preparation on an CSV file with the noobaa s3 bucket as a target
5. once finished, list the bucket with AWS CLI or Data Intelligence's web UI, e.g.:
    aws3 --output json ls   s3://sdi-data-lake-346151f9-4251-4368-aa8a- 
  326637018975/

Actual results:
  
  2021-09-14 09:48:49       7996 nile.csv
  2021-09-14 10:52:42      82828 us-500-fix-prepared.csv
  2021-09-14 10:58:52      78406 us-500-fix-prepared2.csv
  2021-09-14 09:51:01      81332 us-500-fix.csv
  2021-09-14 10:52:42      82828 us-500-fix-prepared.csv
  2021-09-14 10:58:52      78406 us-500-fix-prepared2.csv
  2021-09-14 10:52:42      82828 us-500-fix-prepared.csv

Expected results:

  2021-09-14 09:48:49       7996 nile.csv
  2021-09-14 10:52:42      82828 us-500-fix-prepared.csv
  2021-09-14 10:58:52      78406 us-500-fix-prepared2.csv
  2021-09-14 09:51:01      81332 us-500-fix.csv

Additional info:

Comment 4 Michal Minar 2021-10-22 16:58:49 UTC
The aws3 command from Comment 1 is an alias of `aws s3` from awscli-1.20.31:
  # command -v aws3
  alias aws3='/nix/store/figpa2d3qhha8s63i4dag1shdgi6jc7w-awscli-1.20.31/bin/aws s3 --endpoint-url=https://s3-openshift-storage.apps.morrisville.ocp.vslen'

Comment 5 Nimrod Becker 2022-06-09 08:25:24 UTC
No repro steps, doesn't happen on QE runs, closing


Note You need to log in before you can comment on or make changes to this bug.