Bug 2016685

Summary:	some objects uploaded to noobaa are duplicated in the listing
Product:	[Red Hat Storage] Red Hat OpenShift Data Foundation	Reporter:	Michal Minar <miminar>
Component:	Multi-Cloud Object Gateway	Assignee:	Nimrod Becker <nbecker>
Status:	CLOSED INSUFFICIENT_DATA	QA Contact:	Elad <ebenahar>
Severity:	high	Docs Contact:
Priority:	unspecified
Version:	4.8	CC:	etamir, mmuench, ocs-bugs, odf-bz-bot
Target Milestone:	---
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2022-06-09 08:25:24 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Michal Minar 2021-10-22 16:49:02 UTC

Description of problem (please be detailed as possible and provide log
snippests):
   the same object (having the same name/key) is listed twice or thrice in the output of aws CLI v2.

Version of all relevant components (if applicable):
  ocs-operator.v4.8.3

Does this issue impact your ability to continue to work with the product
(please explain in detail what is the user impact)? no

Is there any workaround available to the best of your knowledge?
  the listing can be deduplicated with e.g. `sort -u`


Rate from 1 - 5 the complexity of the scenario you performed that caused this
bug (1 - very simple, 5 - very complex)?
  3

Can this issue reproducible? happened to me in 2 out of 2 cases (once with 4.8.1, now with 4.8.3)


Can this issue reproduce from the UI? from SAP Data Intelligence's UI


If this is a regression, please provide more details to justify this:
  it is a regression from 4.6

Steps to Reproduce:
1. Install OCP, OCS operator and SAP Data Intelligence
2. create a noobaa S3 bucket with an OBC
3. configure S3 connection using http://s3.openshift-storage.svc.cluster.local endpoint
4. Run data preparation on an CSV file with the noobaa s3 bucket as a target
5. once finished, list the bucket with AWS CLI or Data Intelligence's web UI, e.g.:
    aws3 --output json ls   s3://sdi-data-lake-346151f9-4251-4368-aa8a- 
  326637018975/

Actual results:
  
  2021-09-14 09:48:49       7996 nile.csv
  2021-09-14 10:52:42      82828 us-500-fix-prepared.csv
  2021-09-14 10:58:52      78406 us-500-fix-prepared2.csv
  2021-09-14 09:51:01      81332 us-500-fix.csv
  2021-09-14 10:52:42      82828 us-500-fix-prepared.csv
  2021-09-14 10:58:52      78406 us-500-fix-prepared2.csv
  2021-09-14 10:52:42      82828 us-500-fix-prepared.csv

Expected results:

  2021-09-14 09:48:49       7996 nile.csv
  2021-09-14 10:52:42      82828 us-500-fix-prepared.csv
  2021-09-14 10:58:52      78406 us-500-fix-prepared2.csv
  2021-09-14 09:51:01      81332 us-500-fix.csv

Additional info:

Comment 4 Michal Minar 2021-10-22 16:58:49 UTC

The aws3 command from Comment 1 is an alias of `aws s3` from awscli-1.20.31:
  # command -v aws3
  alias aws3='/nix/store/figpa2d3qhha8s63i4dag1shdgi6jc7w-awscli-1.20.31/bin/aws s3 --endpoint-url=https://s3-openshift-storage.apps.morrisville.ocp.vslen'

Comment 5 Nimrod Becker 2022-06-09 08:25:24 UTC

No repro steps, doesn't happen on QE runs, closing