Bug 2015548

Summary: Paging not working as expected in Velero
Product: Migration Toolkit for Containers Reporter: Alay Patel <alpatel>
Component: VeleroAssignee: Alay Patel <alpatel>
Status: CLOSED ERRATA QA Contact: Xin jiang <xjiang>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 1.7.0CC: ernelson, hitpatel, midays, rjohnson, sgoodman
Target Milestone: ---   
Target Release: 1.7.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2022-03-24 06:32:27 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Alay Patel 2021-10-19 13:26:51 UTC 
Description of problem:
Velero 1.7 added a paging feature to ease the pressure on api-server on large list calls during backups. However, paging is not working because the list option that makes the actual paged request was getting overridden by a shadow variable without the options set.

Version-Release number of selected component (if applicable):
1.7

How reproducible:
Always

Steps to Reproduce:
1. Create 550 secrets in a namespace
2. Install 1.7 velero and configure a BSL
3. Take backup of the namespace with the secrets
4. Using oc adm node-logs --role=master --path=kube-apiserver/, look at the most recent audit log
5. Grep for secret list in all the nodes, as an example `$ oc adm node-logs ip-10-0-139-69.us-west-2.compute.internal  --path="kube-apiserver/audit.log" | grep 'list' | grep 'kube:admin'`. This query greps for all list requests coming from kube:admin service account since I ran velero logged in as kube:admin locally. Depending on how velero is running, the service account/user needs to be replaced there.


Actual results:
sh-4.4# cat  audit-2021-10-14T18-10-21.851.log  | grep system:serviceaccount:velero:velero-backup-dev-rcdn-01-server | grep secrets  | jq -cs '.[] | [ .stageTimestamp , .verb , .requestURI ]' 
["2021-10-14T18:01:09.620647Z","list","/api/v1/secrets"]


Expected results:
sh-4.4# cat audit* | grep <my_username> | grep secrets | jq -cs '.[] | [.requestURI]'
["/api/v1/secrets?limit=500"]
["/api/v1/secrets?continue=eyJ2IjoibWV0YS5rOHMuaW8vdjEiLCJydiI6OTE4OTM2NjAsInN0YXJ0IjoiZmVsaXgtdGVzdGluZzQxLXByb2plY3QtaW50ZXJuYWwtaW52YWxpZC1jbHVzdGVyLWluZm8vZGVmYXVsdC10b2tlbi1ra3dualx1MDAwMCJ9&limit=500"]
["/api/v1/secrets?continue=eyJ2IjoibWV0YS5rOHMuaW8vdjEiLCJydiI6OTE4OTM2NjAsInN0YXJ0Ijoib3BlbnNoaWZ0LWNsb3VkLWNyZWRlbnRpYWwtb3BlcmF0b3IvZGVmYXVsdC10b2tlbi1icnB0aFx1MDAwMCJ9&limit=500"]
["/api/v1/secrets?continue=eyJ2IjoibWV0YS5rOHMuaW8vdjEiLCJydiI6OTE4OTM2NjAsInN0YXJ0Ijoib3BlbnNoaWZ0LWt1YmUtYXBpc2VydmVyLW9wZXJhdG9yL2RlZmF1bHQtZG9ja2VyY2ZnLWxjbHdoXHUwMDAwIn0&limit=500"]
["/api/v1/secrets?continue=eyJ2IjoibWV0YS5rOHMuaW8vdjEiLCJydiI6OTE4OTM2NjAsInN0YXJ0Ijoib3BlbnNoaWZ0LW9hdXRoLWFwaXNlcnZlci9idWlsZGVyLXRva2VuLXZzcWg0XHUwMDAwIn0&limit=500"]
["/api/v1/secrets?continue=eyJ2IjoibWV0YS5rOHMuaW8vdjEiLCJydiI6OTE4OTM2NjAsInN0YXJ0Ijoib3M0LW1jbXAtbWlnLTM4L2J1aWxkZXItdG9rZW4ta2ZuMnZcdTAwMDAifQ&limit=500"]


Additional info:
1. https://docs.openshift.com/container-platform/4.5/security/audit-log-view.html
Comment 2 hitpatel 2021-11-18 21:14:40 UTC 
Built in openshift-migration-velero-container-v1.7.0-3
Comment 5 Alay Patel 2022-01-27 18:07:07 UTC 
https://github.com/vmware-tanzu/velero/pull/4358

Was merged upstream in velero 1.7
Comment 8 errata-xmlrpc 2022-03-24 06:32:27 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Migration Toolkit for Containers (MTC) 1.7.0 release advisory), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2022:1043