Hide Forgot
Description of problem: rhsc-log-collector never completes successfully when Y is selected for "About to collect information from <n> hypervisors. Continue? (Y/n):" ---------- [root@vm08 ovirt-engine]# rhsc-log-collector Please provide the REST API username for oVirt Engine (CTRL+D to skip): admin@internal Please provide the REST API password for the admin@internal oVirt Engine user (CTRL+D to skip): About to collect information from 2 hypervisors. Continue? (Y/n): Y INFO: Gathering information from selected hypervisors... INFO: collecting information from vm11.lab.eng.blr.redhat.com INFO: Gluster logs will be collected from vm11.lab.eng.blr.redhat.com INFO: collecting information from vm10.lab.eng.blr.redhat.com INFO: Gluster logs will be collected from vm10.lab.eng.blr.redhat.com ^CERROR: Failed to collect logs from: vm10.lab.eng.blr.redhat.com; Killed by signal 2. ERROR: Failed to collect logs from: vm11.lab.eng.blr.redhat.com; Killed by signal 2. INFO: finished collecting information from vm11.lab.eng.blr.redhat.com Exiting on user cancel. INFO: finished collecting information from vm10.lab.eng.blr.redhat.com You have new mail in /var/spool/mail/root ------- However, if you give n, it completes successfully. See below: ---------- [root@vm08 ovirt-engine]# rhsc-log-collector Please provide the REST API username for oVirt Engine (CTRL+D to skip): admin@internal Please provide the REST API password for the admin@internal oVirt Engine user (CTRL+D to skip): About to collect information from 2 hypervisors. Continue? (Y/n): n INFO: Aborting hypervisor collection... Please provide the password for the PostgreSQL user, postgres, to dump the engine PostgreSQL database instance (CTRL+D to skip): INFO: Gathering PostgreSQL the oVirt Engine database and log files from localhost... INFO: Gathering oVirt Engine information... INFO: Log files have been collected and placed in /tmp/logcollector/sosreport-LogCollector-qaredhat.com-20131011150711-2e68.tar.xz. The MD5 for this file is bef798d0b7ca9b77bf706c75683e2e68 and its size is 7.2M ---------- Version-Release number of selected component (if applicable): ---------------------- [root@vm08 /]# rpm -qa |grep rhsc rhsc-2.1.1-0.0.3.master.el6ev.noarch rhsc-lib-2.1.1-0.0.3.master.el6ev.noarch rhsc-webadmin-portal-2.1.1-0.0.3.master.el6ev.noarch rhsc-sdk-2.1.0.0-0.bb3a.el6rhs.noarch rhsc-log-collector-2.1-0.1.el6rhs.noarch rhsc-tools-2.1.1-0.0.3.master.el6ev.noarch rhsc-backend-2.1.1-0.0.3.master.el6ev.noarch rhsc-cli-2.1.0.0-0.bb3a.el6rhs.noarch rhsc-restapi-2.1.1-0.0.3.master.el6ev.noarch rhsc-dbscripts-2.1.1-0.0.3.master.el6ev.noarch rhsc-branding-rhs-3.3.0-1.0.master.201309200500.fc18.noarch rhsc-setup-2.1.1-0.0.3.master.el6ev.noarch [root@vm10 /]# rpm -qa |grep vdsm vdsm-python-cpopen-4.12.0-178.git42c732a.el6.x86_64 vdsm-gluster-4.12.0-178.git42c732a.el6.noarch vdsm-python-4.12.0-178.git42c732a.el6.x86_64 vdsm-xmlrpc-4.12.0-178.git42c732a.el6.noarch vdsm-cli-4.12.0-178.git42c732a.el6.noarch vdsm-4.12.0-178.git42c732a.el6.x86_64 ---------------------- How reproducible: Always Steps to Reproduce: 1. In engine, execute #rhsc-log-collector It runs for ever when logs from hypervisor is also selected and even though the log files are too small. Actual results: rhsc-log-collector never completes successfully Only following is seen in engine-log-collector.log: ------------- [root@vm08 ovirt-engine]# cat engine-log-collector.log 2013-10-11 14:38:56::DEBUG::hypervisors::129::root:: API Vendor(Red Hat) API Version(3.3.0) 2013-10-11 14:38:59::INFO::engine-log-collector::824::root:: Gathering information from selected hypervisors... 2013-10-11 14:38:59::INFO::engine-log-collector::538::root:: collecting information from vm11.lab.eng.blr.redhat.com 2013-10-11 14:38:59::INFO::engine-log-collector::518::root:: Gluster logs will be collected from vm11.lab.eng.blr.redhat.com 2013-10-11 14:38:59::INFO::engine-log-collector::538::root:: collecting information from vm10.lab.eng.blr.redhat.com 2013-10-11 14:38:59::INFO::engine-log-collector::518::root:: Gluster logs will be collected from vm10.lab.eng.blr.redhat.com ------------- Expected results: rhsc-log-collector should collect the sosreports from all the hypervisors and generate the tar ball. Additional info: Sosreports of engine and 2 nodes can be downloaded from: http://rhsqe-repo.lab.eng.blr.redhat.com/sosreports/1018076/sosreport-cb3Engine.08-20131011142514-d79d.tar.xz http://rhsqe-repo.lab.eng.blr.redhat.com/sosreports/1018076/sosreport-server1.10-20131011142152-2fbe.tar.xz http://rhsqe-repo.lab.eng.blr.redhat.com/sosreports/1018076/sosreport-server2.11-20131011142223-e429.tar.xz
Fixed in version: 2.1.2.0 cb5 --no-hypervisors option is changed to --no-servers
Executing #rhsc-log-collector on cb5 build is throwing some xml output and on completion, it freezes the terminal. See below: ------------------ # rhsc-log-collector INFO: Gathering oVirt Engine information... INFO: Gathering PostgreSQL the oVirt Engine database and log files from localhost... Please provide the REST API password for the admin@internal oVirt Engine user (CTRL+D to skip): <?xml version="1.0" ?> <keyValuePair> <link href="/api/capabilities" rel="capabilities"/> <link href="/api/clusters" rel="clusters"/> <link href="/api/clusters?search={query}" rel="clusters/search"/> <link href="/api/events" rel="events"/> <link href="/api/events;from={event_id}?search={query}" rel="events/search"/> <link href="/api/hosts" rel="hosts"/> <link href="/api/hosts?search={query}" rel="hosts/search"/> <link href="/api/networks" rel="networks"/> <link href="/api/networks?search={query}" rel="networks/search"/> <link href="/api/roles" rel="roles"/> <link href="/api/tags" rel="tags"/> <link href="/api/users" rel="users"/> <link href="/api/users?search={query}" rel="users/search"/> <link href="/api/groups" rel="groups"/> <link href="/api/groups?search={query}" rel="groups/search"/> <link href="/api/domains" rel="domains"/> <special_objects> <link href="/api/tags/00000000-0000-0000-0000-000000000000" rel="tags/root"/> </special_objects> <product_info> <name>Red Hat Storage Console</name> <vendor>Red Hat</vendor> <version major="3" build="0" minor="3" revision="0"/> <full_version>2.1.2-0.21.beta1.el6_4</full_version> </product_info> <summary> <hosts> <total>2</total> <active>1</active> </hosts> <users> <total>1</total> <active>1</active> </users> </summary> <time>2013-10-28T14:55:04.111000+05:30</time> </keyValuePair> <?xml version="1.0" ?> <keyValuePair> <cluster href="/api/clusters/99408929-82cf-4dc7-a532-9d998063fa95" id="99408929-82cf-4dc7-a532-9d998063fa95"> <name>Default</name> <description>The default server cluster</description> <link href="/api/clusters/99408929-82cf-4dc7-a532-9d998063fa95/networks" rel="networks"/> <link href="/api/clusters/99408929-82cf-4dc7-a532-9d998063fa95/permissions" rel="permissions"/> <link href="/api/clusters/99408929-82cf-4dc7-a532-9d998063fa95/glustervolumes" rel="glustervolumes"/> <link href="/api/clusters/99408929-82cf-4dc7-a532-9d998063fa95/glusterhooks" rel="glusterhooks"/> <data_center href="/api/datacenters/5849b030-626e-47cb-ad90-3ce782d831b3" id="5849b030-626e-47cb-ad90-3ce782d831b3"/> <memory_policy> <overcommit percent="100"/> <transparent_hugepages> <enabled>true</enabled> </transparent_hugepages> </memory_policy> <scheduling_policy> <policy>none</policy> </scheduling_policy> <version major="3" minor="3"/> <error_handling> <on_error>migrate</on_error> </error_handling> <virt_service>false</virt_service> <gluster_service>true</gluster_service> <threads_as_cores>false</threads_as_cores> <tunnel_migration>false</tunnel_migration> <trusted_service>false</trusted_service> <ballooning_enabled>false</ballooning_enabled> </cluster> <cluster href="/api/clusters/aa8ab992-7445-47e1-b330-22271f40b593" id="aa8ab992-7445-47e1-b330-22271f40b593"> <name>Delhi</name> <description>Delhi</description> <comment>Delhi</comment> <link href="/api/clusters/aa8ab992-7445-47e1-b330-22271f40b593/networks" rel="networks"/> <link href="/api/clusters/aa8ab992-7445-47e1-b330-22271f40b593/permissions" rel="permissions"/> <link href="/api/clusters/aa8ab992-7445-47e1-b330-22271f40b593/glustervolumes" rel="glustervolumes"/> <link href="/api/clusters/aa8ab992-7445-47e1-b330-22271f40b593/glusterhooks" rel="glusterhooks"/> <data_center href="/api/datacenters/5849b030-626e-47cb-ad90-3ce782d831b3" id="5849b030-626e-47cb-ad90-3ce782d831b3"/> <memory_policy> <overcommit percent="100"/> <transparent_hugepages> <enabled>true</enabled> </transparent_hugepages> </memory_policy> <scheduling_policy> <policy>none</policy> </scheduling_policy> <version major="3" minor="3"/> <error_handling> <on_error>migrate</on_error> </error_handling> <virt_service>false</virt_service> <gluster_service>true</gluster_service> <threads_as_cores>false</threads_as_cores> <tunnel_migration>false</tunnel_migration> <trusted_service>false</trusted_service> <ballooning_enabled>false</ballooning_enabled> </cluster> </keyValuePair> <?xml version="1.0" ?> <keyValuePair> <host href="/api/hosts/dd070e09-4b16-4da1-9333-6f8263f19d96" id="dd070e09-4b16-4da1-9333-6f8263f19d96"> <actions> <link href="/api/hosts/dd070e09-4b16-4da1-9333-6f8263f19d96/iscsidiscover" rel="iscsidiscover"/> <link href="/api/hosts/dd070e09-4b16-4da1-9333-6f8263f19d96/commitnetconfig" rel="commitnetconfig"/> <link href="/api/hosts/dd070e09-4b16-4da1-9333-6f8263f19d96/approve" rel="approve"/> <link href="/api/hosts/dd070e09-4b16-4da1-9333-6f8263f19d96/forceselectspm" rel="forceselectspm"/> <link href="/api/hosts/dd070e09-4b16-4da1-9333-6f8263f19d96/iscsilogin" rel="iscsilogin"/> <link href="/api/hosts/dd070e09-4b16-4da1-9333-6f8263f19d96/deactivate" rel="deactivate"/> <link href="/api/hosts/dd070e09-4b16-4da1-9333-6f8263f19d96/fence" rel="fence"/> <link href="/api/hosts/dd070e09-4b16-4da1-9333-6f8263f19d96/install" rel="install"/> <link href="/api/hosts/dd070e09-4b16-4da1-9333-6f8263f19d96/activate" rel="activate"/> </actions> <name>Server1</name> <link href="/api/hosts/dd070e09-4b16-4da1-9333-6f8263f19d96/storage" rel="storage"/> <link href="/api/hosts/dd070e09-4b16-4da1-9333-6f8263f19d96/nics" rel="nics"/> <link href="/api/hosts/dd070e09-4b16-4da1-9333-6f8263f19d96/tags" rel="tags"/> <link href="/api/hosts/dd070e09-4b16-4da1-9333-6f8263f19d96/permissions" rel="permissions"/> <link href="/api/hosts/dd070e09-4b16-4da1-9333-6f8263f19d96/statistics" rel="statistics"/> <link href="/api/hosts/dd070e09-4b16-4da1-9333-6f8263f19d96/hooks" rel="hooks"/> <address>vm10.lab.eng.blr.redhat.com</address> <certificate> <organization>fedora</organization> <subject>O=fedora,CN=vm10.lab.eng.blr.redhat.com</subject> </certificate> <status> <state>up</state> </status> <cluster href="/api/clusters/aa8ab992-7445-47e1-b330-22271f40b593" id="aa8ab992-7445-47e1-b330-22271f40b593"/> <port>54321</port> <type>rhel</type> <storage_manager priority="5">false</storage_manager> <version major="4" full_version="vdsm-4.13.0-19.gitc2a87f5.el6rhs" build="0" minor="13" revision="0"/> <hardware_information> <manufacturer>Red Hat</manufacturer> <version>6Server-6.4.0.4.el6</version> <serial_number>00000000-0000-0000-0000-00259078DB32</serial_number> <product_name>RHEV Hypervisor</product_name> <uuid>e08eef0c-d7d9-4589-aa46-58472adf22f1</uuid> <family>Red Hat Enterprise Linux</family> </hardware_information> <power_management type="apc"> <enabled>false</enabled> <options/> </power_management> <ksm> <enabled>false</enabled> </ksm> <transparent_hugepages> <enabled>true</enabled> </transparent_hugepages> <iscsi> <initiator>iqn.1994-05.com.redhat:eac9944636ff</initiator> </iscsi> <ssh> <port>22</port> <fingerprint>8a:b0:e1:06:c7:d0:35:c8:ae:39:91:ff:e9:94:61:f0</fingerprint> </ssh> <cpu> <topology cores="1" threads="1" sockets="2"/> <name>Intel Xeon E312xx (Sandy Bridge)</name> <speed>2000.</speed> </cpu> <memory>8387559424</memory> <max_scheduling_memory>7982809088</max_scheduling_memory> <summary> <active>0</active> <migrating>0</migrating> <total>0</total> </summary> <os type="RHEL"> <version full_version="6Server - 6.4.0.4.el6"/> </os> <libvirt_version major="0" full_version="libvirt-0.10.2-18.el6_4.13" build="2" minor="10" revision="0"/> </host> <host href="/api/hosts/9f2fc941-94a9-41ee-8f10-0dbaa01cb72e" id="9f2fc941-94a9-41ee-8f10-0dbaa01cb72e"> <actions> <link href="/api/hosts/9f2fc941-94a9-41ee-8f10-0dbaa01cb72e/iscsidiscover" rel="iscsidiscover"/> <link href="/api/hosts/9f2fc941-94a9-41ee-8f10-0dbaa01cb72e/commitnetconfig" rel="commitnetconfig"/> <link href="/api/hosts/9f2fc941-94a9-41ee-8f10-0dbaa01cb72e/approve" rel="approve"/> <link href="/api/hosts/9f2fc941-94a9-41ee-8f10-0dbaa01cb72e/forceselectspm" rel="forceselectspm"/> <link href="/api/hosts/9f2fc941-94a9-41ee-8f10-0dbaa01cb72e/iscsilogin" rel="iscsilogin"/> <link href="/api/hosts/9f2fc941-94a9-41ee-8f10-0dbaa01cb72e/deactivate" rel="deactivate"/> <link href="/api/hosts/9f2fc941-94a9-41ee-8f10-0dbaa01cb72e/fence" rel="fence"/> <link href="/api/hosts/9f2fc941-94a9-41ee-8f10-0dbaa01cb72e/install" rel="install"/> <link href="/api/hosts/9f2fc941-94a9-41ee-8f10-0dbaa01cb72e/activate" rel="activate"/> </actions> <name>Server2</name> <comment>Server2</comment> <link href="/api/hosts/9f2fc941-94a9-41ee-8f10-0dbaa01cb72e/storage" rel="storage"/> <link href="/api/hosts/9f2fc941-94a9-41ee-8f10-0dbaa01cb72e/nics" rel="nics"/> <link href="/api/hosts/9f2fc941-94a9-41ee-8f10-0dbaa01cb72e/tags" rel="tags"/> <link href="/api/hosts/9f2fc941-94a9-41ee-8f10-0dbaa01cb72e/permissions" rel="permissions"/> <link href="/api/hosts/9f2fc941-94a9-41ee-8f10-0dbaa01cb72e/statistics" rel="statistics"/> <link href="/api/hosts/9f2fc941-94a9-41ee-8f10-0dbaa01cb72e/hooks" rel="hooks"/> <address>vm11.lab.eng.blr.redhat.com</address> <certificate> <organization>fedora</organization> <subject>O=fedora,CN=vm11.lab.eng.blr.redhat.com</subject> </certificate> <status> <state>maintenance</state> </status> <cluster href="/api/clusters/aa8ab992-7445-47e1-b330-22271f40b593" id="aa8ab992-7445-47e1-b330-22271f40b593"/> <port>54321</port> <type>rhel</type> <storage_manager priority="5">false</storage_manager> <version major="4" full_version="vdsm-4.13.0-19.gitc2a87f5.el6rhs" build="0" minor="13" revision="0"/> <hardware_information> <manufacturer>Red Hat</manufacturer> <version>6Server-6.4.0.4.el6</version> <serial_number>00000000-0000-0000-0000-0025907C2CA6</serial_number> <product_name>RHEV Hypervisor</product_name> <uuid>45d3c81a-30de-428b-962c-7a044dab335f</uuid> <family>Red Hat Enterprise Linux</family> </hardware_information> <power_management type="apc"> <enabled>false</enabled> <options/> </power_management> <ksm> <enabled>false</enabled> </ksm> <transparent_hugepages> <enabled>true</enabled> </transparent_hugepages> <iscsi> <initiator>iqn.1994-05.com.redhat:6a6c74afac6</initiator> </iscsi> <ssh> <port>22</port> <fingerprint>57:51:7f:68:f3:f2:55:1a:4e:d9:5d:6c:20:2d:b5:a1</fingerprint> </ssh> <cpu> <topology cores="1" threads="1" sockets="1"/> <name>Intel Xeon E312xx (Sandy Bridge)</name> <speed>2000.</speed> </cpu> <memory>4152360960</memory> <max_scheduling_memory>3747610624</max_scheduling_memory> <summary> <active>0</active> <migrating>0</migrating> <total>0</total> </summary> <os type="RHEL"> <version full_version="6Server - 6.4.0.4.el6"/> </os> <libvirt_version major="0" full_version="libvirt-0.10.2-18.el6_4.14" build="2" minor="10" revision="0"/> </host> </keyValuePair> About to collect information from 2 servers. Continue? (Y/n): y INFO: Gathering information from selected servers... INFO: collecting information from vm11.lab.eng.blr.redhat.com INFO: collecting information from vm10.lab.eng.blr.redhat.com INFO: Gluster logs will be collected from vm11.lab.eng.blr.redhat.com INFO: Gluster logs will be collected from vm10.lab.eng.blr.redhat.com INFO: finished collecting information from vm11.lab.eng.blr.redhat.com INFO: finished collecting information from vm10.lab.eng.blr.redhat.com Creating compressed archive... INFO: Log files have been collected and placed in /tmp/sosreport-LogCollector-20131028145600.tar.xz. The MD5 for this file is 5cac936593986ecac651c2dc57487c18 and its size is 102.5M ------------------ Hence, moving back to assigned.
xml output is fixed in CB9, for console freeze I am still investigating the issue which caused the freeze. As a workaround, when console is freezed, type `reset` and hit enter, which clears/resets the console. (Ref: https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_Linux/3/html/Step_by_Step_Guide/s1-navigating-washing.html)
In rhevm-log-collector, added a fix to prevent hang when collecting sos reports from hosts(Ref: https://bugzilla.redhat.com/show_bug.cgi?id=1022505 and https://access.redhat.com/site/solutions/489813), which causes tty freeze sometimes. I will open separate BZ to track the terminal freeze. For a workaround we can use `reset` command as I mentioned in Comment 4.
Verified. ------------- # rhsc-log-collector INFO: Gathering oVirt Engine information... INFO: Gathering PostgreSQL the oVirt Engine database and log files from localhost... Please provide the REST API password for the admin@internal oVirt Engine user (CTRL+D to skip): About to collect information from 2 servers. Continue? (Y/n): y INFO: Gathering information from selected servers... INFO: collecting information from vm11.lab.eng.blr.redhat.com INFO: Gluster logs will be collected from vm11.lab.eng.blr.redhat.com INFO: collecting information from vm10.lab.eng.blr.redhat.com INFO: Gluster logs will be collected from vm10.lab.eng.blr.redhat.com INFO: finished collecting information from vm10.lab.eng.blr.redhat.com INFO: finished collecting information from vm11.lab.eng.blr.redhat.com Creating compressed archive... INFO: Log files have been collected and placed in /tmp/sosreport-LogCollector-20131221155932.tar.xz. The MD5 for this file is e4529d870d13851cfd693304b4a04655 and its size is 71.1M ------------- I'll open a separate BZ for the terminal freeze issue.
Please review the edited DocText and signoff.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. http://rhn.redhat.com/errata/RHEA-2014-0208.html