Bug 2114754 - "gather bootstrap" creates unexpected folder "serial-log-bundle-<timestamp>" beyond "log-bundle-<timestamp>.tar.gz"
Summary: "gather bootstrap" creates unexpected folder "serial-log-bundle-<timestamp>" ...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Installer
Version: 4.12
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: ---
: 4.12.0
Assignee: Rafael Fonseca
QA Contact: Jianli Wei
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2022-08-03 08:36 UTC by Jianli Wei
Modified: 2023-01-17 19:54 UTC (History)
2 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2023-01-17 19:54:14 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift installer pull 6197 0 None open Bug 2114754: gather: properly delete serial log dir 2022-08-03 09:38:41 UTC
Red Hat Product Errata RHSA-2022:7399 0 None None None 2023-01-17 19:54:35 UTC

Description Jianli Wei 2022-08-03 08:36:51 UTC
FYI The epic: https://issues.redhat.com/browse/CORS-2067

Version:
$ openshift-install 4.12.0-0.nightly-2022-08-01-151317
built from commit d304ff4459e5110457ab609fca732e5f90b9b359
release image registry.ci.openshift.org/ocp/release@sha256:2050173e8113faae2faadcff7b77346dab996705a68f2384fd5a2674c6e2a2ff
release architecture amd64

Platform: GCP

Please specify: IPI

What happened?
After "openshift-install gather bootstrap...", except the expected "log-bundle-20220803060910.tar.gz", one more folder "serial-log-bundle-20220803060910" also turns present in the installation directory, which is unexpected, because the log files in the folder are already included in the .tar.gz. 

What did you expect to happen?
Interrupt the installation before bootstrap complete, then run "openshift-install gather bootstrap...".

How to reproduce it (as minimally and precisely as possible)?
Always.

Anything else we need to know?

$ openshift-install version
openshift-install 4.12.0-0.nightly-2022-08-01-151317
built from commit d304ff4459e5110457ab609fca732e5f90b9b359
release image registry.ci.openshift.org/ocp/release@sha256:2050173e8113faae2faadcff7b77346dab996705a68f2384fd5a2674c6e2a2ff
release architecture amd64
$ openshift-install create cluster --dir test2
? SSH Public Key /home/fedora/.ssh/openshift-qe.pub
? Platform gcp
INFO Credentials loaded from file "/home/fedora/.gcp/osServiceAccount.json"
? Project ID OpenShift QE (openshift-qe)
? Region us-central1
? Base Domain qe.gcp.devcluster.openshift.com
? Cluster Name jiwei-0803-01
? Pull Secret [? for help] *****
INFO Creating infrastructure resources...         
INFO Waiting up to 20m0s (until 6:23AM) for the Kubernetes API at https://api.jiwei-0803-01.qe.gcp.devcluster.openshift.com:6443... 
INFO API v1.24.0+a9d6306 up                       
INFO Waiting up to 30m0s (until 6:35AM) for bootstrapping to complete... 
^C                                                                                                                                      
$ gcloud compute instances list --filter='name~jiwei-0803-01'
NAME                           ZONE           MACHINE_TYPE   PREEMPTIBLE  INTERNAL_IP  EXTERNAL_IP    STATUS
jiwei-0803-01-fhmsb-bootstrap  us-central1-a  n1-standard-4               10.0.0.6     34.132.94.172  RUNNING
jiwei-0803-01-fhmsb-master-0   us-central1-a  n1-standard-4               10.0.0.5                    RUNNING
jiwei-0803-01-fhmsb-master-1   us-central1-b  n1-standard-4               10.0.0.4                    RUNNING
jiwei-0803-01-fhmsb-master-2   us-central1-c  n1-standard-4               10.0.0.3                    RUNNING
$
$ openshift-install gather bootstrap --key .ssh/openshift-qe.pem --bootstrap 34.132.94.172 --dir test2
INFO Credentials loaded from file "/home/fedora/.gcp/osServiceAccount.json" 
INFO Pulling VM console logs                      
INFO Pulling debug logs from the bootstrap machine 
INFO Bootstrap gather logs captured here "/home/fedora/test2/log-bundle-20220803060910.tar.gz" 
$
$ ls test2 -lrt
total 1640
-rw-r-----. 1 fedora fedora    172 Aug  3 06:00 metadata.json
-rw-r-----. 1 fedora fedora 307953 Aug  3 06:01 terraform.tfvars.json
-rw-r-----. 1 fedora fedora   3325 Aug  3 06:01 terraform.platform.auto.tfvars.json
drwxr-x---. 2 fedora fedora   4096 Aug  3 06:01 auth
drwxr-x---. 2 fedora fedora   4096 Aug  3 06:01 tls
-rw-r-----. 1 fedora fedora  10237 Aug  3 06:03 terraform.post-bootstrap.tfstate
-rw-r-----. 1 fedora fedora  73707 Aug  3 06:03 terraform.cluster.tfstate
-rw-r-----. 1 fedora fedora 323914 Aug  3 06:03 terraform.bootstrap.tfstate
-rw-r-----. 1 fedora fedora      2 Aug  3 06:03 post-bootstrap.tfvars.json
-rw-r-----. 1 fedora fedora   1572 Aug  3 06:03 cluster.tfvars.json
-rw-r-----. 1 fedora fedora    335 Aug  3 06:03 bootstrap.tfvars.json
drwxr-xr-x. 2 fedora fedora   4096 Aug  3 06:09 serial-log-bundle-20220803060910
-rw-rw-r--. 1 fedora fedora 919352 Aug  3 06:09 log-bundle-20220803060910.tar.gz
$ ls test2/serial-log-bundle-20220803060910/ -l
total 1044
-rw-rw-r--. 1 fedora fedora 155343 Aug  3 06:09 jiwei-0803-01-fhmsb-bootstrap-serial.log
-rw-rw-r--. 1 fedora fedora 304361 Aug  3 06:09 jiwei-0803-01-fhmsb-master-0-serial.log
-rw-rw-r--. 1 fedora fedora 298279 Aug  3 06:09 jiwei-0803-01-fhmsb-master-1-serial.log
-rw-rw-r--. 1 fedora fedora 303279 Aug  3 06:09 jiwei-0803-01-fhmsb-master-2-serial.log
$ 
$ tar xvf /home/fedora/test2/log-bundle-20220803060910.tar.gz
...
$ ls log-bundle-20220803060910/serial/ -l
total 1044
-rw-rw-r--. 1 fedora fedora 155343 Aug  3 06:09 jiwei-0803-01-fhmsb-bootstrap-serial.log
-rw-rw-r--. 1 fedora fedora 304361 Aug  3 06:09 jiwei-0803-01-fhmsb-master-0-serial.log
-rw-rw-r--. 1 fedora fedora 298279 Aug  3 06:09 jiwei-0803-01-fhmsb-master-1-serial.log
-rw-rw-r--. 1 fedora fedora 303279 Aug  3 06:09 jiwei-0803-01-fhmsb-master-2-serial.log
$

Comment 5 Jianli Wei 2022-08-08 08:02:54 UTC
Verified with 4.12.0-0.nightly-2022-08-07-180207.

$ openshift-install version
openshift-install 4.12.0-0.nightly-2022-08-07-180207
built from commit 3f27a2e7bedf151101feaa6f4df92b7e052fcd48
release image registry.ci.openshift.org/ocp/release@sha256:61eaa88c2476f6b1cab9300041f485d4e7af712b1fc5a2f226096722aca118b8
release architecture amd64
$ openshift-install create ignition-configs --dir test2
? SSH Public Key /home/fedora/.ssh/openshift-qe.pub
? Platform gcp
INFO Credentials loaded from file "/home/fedora/.gcp/osServiceAccount.json"
? Project ID OpenShift QE (openshift-qe)
? Region us-central1
? Base Domain qe.gcp.devcluster.openshift.com
? Cluster Name jiwei-0808-03
? Pull Secret [? for help] ******
INFO Ignition-Configs created in: test2 and test2/auth
$ 
$ sed -i 's/jiwei-0808-03/foo/' test2/master.ign                                                               
$ 
$ openshift-install create cluster --dir test2                                                                 
WARNING   Discarding the Bootstrap Ignition Config that was provided in the target directory because its dependencies are dirty and it needs to be regenerated             INFO Master pointer ignition was modified. Saving contents to a machineconfig                                                           
INFO Credentials loaded from file "/home/fedora/.gcp/osServiceAccount.json"                                                             
INFO Consuming Bootstrap Ignition Config from target directory                                                                          
INFO Consuming Master Ignition Config from target directory                                                                             
INFO Consuming Worker Ignition Config from target directory                                                                             
INFO Creating infrastructure resources...                                                                                               
INFO Waiting up to 20m0s (until 7:35AM) for the Kubernetes API at https://api.jiwei-0808-03.qe.gcp.devcluster.openshift.com:6443...     
INFO API v1.24.0+a9d6306 up                                                                                                             
INFO Waiting up to 30m0s (until 7:46AM) for bootstrapping to complete...                                                                
INFO Pulling VM console logs                                                                                                            
INFO Pulling debug logs from the bootstrap machine                                                                                      
ERROR Bootstrap failed to complete: timed out waiting for the condition                                                                 
ERROR Failed to wait for bootstrapping to complete. This error usually happens when there is a problem with control plane hosts that prevents the control plane operators from creating the control plane.                                                                      
INFO Bootstrap gather logs captured here "/home/fedora/test2/log-bundle-20220808074621.tar.gz"                                          
$    
$ tar zxvf test2/log-bundle-20220808074621.tar.gz
...
$ ls log-bundle-20220808074621/ -l
total 28
drwxrwxr-x. 8 fedora fedora 4096 Aug  8  2022 bootstrap
drwxrwxr-x. 5 fedora fedora 4096 Aug  8  2022 control-plane
-rw-rw-r--. 1 fedora fedora  139 Aug  8  2022 failed-units.txt
-rw-rw-r--. 1 fedora fedora 3567 Aug  8  2022 gather.log
drwxrwxr-x. 3 fedora fedora 4096 Aug  8  2022 rendered-assets
drwxrwxr-x. 2 fedora fedora 4096 Aug  8 07:47 serial
drwxrwxr-x. 2 fedora fedora 4096 Aug  8  2022 unit-status
$ ls log-bundle-20220808074621/serial/ -l
total 1896
-rw-rw-r--. 1 fedora fedora 155820 Aug  8 07:46 jiwei-0808-03-jtdb2-bootstrap-serial.log
-rw-rw-r--. 1 fedora fedora 592313 Aug  8 07:46 jiwei-0808-03-jtdb2-master-0-serial.log
-rw-rw-r--. 1 fedora fedora 592303 Aug  8 07:46 jiwei-0808-03-jtdb2-master-1-serial.log
-rw-rw-r--. 1 fedora fedora 592824 Aug  8 07:46 jiwei-0808-03-jtdb2-master-2-serial.log
$
$ openshift-install destroy cluster --dir test2                                                                
INFO Credentials loaded from file "/home/fedora/.gcp/osServiceAccount.json"
INFO Stopped instance jiwei-0808-03-jtdb2-bootstrap
INFO Stopped instance jiwei-0808-03-jtdb2-master-1
INFO Stopped instance jiwei-0808-03-jtdb2-master-2
INFO Stopped instance jiwei-0808-03-jtdb2-master-0
INFO Deleted IAM project role bindings
INFO Deleted service account projects/openshift-qe/serviceAccounts/jiwei-0808-0-openshift-g-zq9tc.gserviceaccount.com
INFO Deleted service account projects/openshift-qe/serviceAccounts/jiwei-0808-0-openshift-c-kjv9l.gserviceaccount.com
INFO Deleted service account projects/openshift-qe/serviceAccounts/jiwei-0808-03-jtdb2-w.gserviceaccount.com
INFO Deleted service account projects/openshift-qe/serviceAccounts/jiwei-0808-0-openshift-i-tpwsw.gserviceaccount.com
INFO Deleted service account projects/openshift-qe/serviceAccounts/jiwei-0808-0-openshift-m-kljfs.gserviceaccount.com
INFO Deleted service account projects/openshift-qe/serviceAccounts/jiwei-0808-0-cloud-crede-h96l2.gserviceaccount.com
INFO Deleted service account projects/openshift-qe/serviceAccounts/jiwei-0808-0-openshift-i-mtj9l.gserviceaccount.com
INFO Deleted service account projects/openshift-qe/serviceAccounts/jiwei-0808-03-jtdb2-m.gserviceaccount.com
INFO Deleted 1 recordset(s) in zone qe
INFO Deleted 2 recordset(s) in zone jiwei-0808-03-jtdb2-private-zone
INFO Deleted DNS zone jiwei-0808-03-jtdb2-private-zone
INFO Deleted bucket object bootstrap.ign
INFO Deleted bucket jiwei-0808-03-jtdb2-bootstrap-ignition
INFO Deleted instance jiwei-0808-03-jtdb2-bootstrap
INFO Deleted instance jiwei-0808-03-jtdb2-master-0
INFO Deleted instance jiwei-0808-03-jtdb2-master-1
INFO Deleted instance jiwei-0808-03-jtdb2-master-2
INFO Deleted disk jiwei-0808-03-jtdb2-bootstrap   
INFO Deleted disk jiwei-0808-03-jtdb2-master-0    
INFO Deleted disk jiwei-0808-03-jtdb2-master-1
INFO Deleted disk jiwei-0808-03-jtdb2-master-2
INFO Deleted firewall rule jiwei-0808-03-jtdb2-control-plane
INFO Deleted firewall rule jiwei-0808-03-jtdb2-etcd
INFO Deleted firewall rule jiwei-0808-03-jtdb2-health-checks
INFO Deleted firewall rule jiwei-0808-03-jtdb2-internal-cluster
INFO Deleted firewall rule jiwei-0808-03-jtdb2-internal-network
INFO Deleted firewall rule jiwei-0808-03-jtdb2-api
INFO Deleted firewall rule jiwei-0808-03-jtdb2-bootstrap-in-ssh
INFO Deleted address jiwei-0808-03-jtdb2-bootstrap-ip
INFO Deleted address jiwei-0808-03-jtdb2-cluster-ip
INFO Deleted address jiwei-0808-03-jtdb2-cluster-public-ip
INFO Deleted forwarding rule jiwei-0808-03-jtdb2-api
INFO Deleted forwarding rule jiwei-0808-03-jtdb2-api-internal
INFO Deleted router jiwei-0808-03-jtdb2-router
INFO Deleted subnetwork jiwei-0808-03-jtdb2-worker-subnet
INFO Deleted target pool jiwei-0808-03-jtdb2-api
INFO Deleted backend service jiwei-0808-03-jtdb2-api-internal
INFO Deleted subnetwork jiwei-0808-03-jtdb2-master-subnet
INFO Deleted instance group jiwei-0808-03-jtdb2-bootstrap
INFO Deleted instance group jiwei-0808-03-jtdb2-master-us-central1-a
INFO Deleted instance group jiwei-0808-03-jtdb2-master-us-central1-c
INFO Deleted instance group jiwei-0808-03-jtdb2-master-us-central1-b
INFO Deleted health check jiwei-0808-03-jtdb2-api-internal
INFO Deleted HTTP health check jiwei-0808-03-jtdb2-api
INFO Deleted network jiwei-0808-03-jtdb2-network
INFO Time elapsed: 4m35s   
$

Comment 9 errata-xmlrpc 2023-01-17 19:54:14 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.12.0 bug fix and security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:7399


Note You need to log in before you can comment on or make changes to this bug.