Bug 1246653

Summary: RFE: overcloud: It is very difficult to collect sosreport and crm_report without having root accesses between cluster nodes.
Product: Red Hat OpenStack Reporter: Leonid Natapov <lnatapov>
Component: rhosp-directorAssignee: Hugh Brock <hbrock>
Status: CLOSED DUPLICATE QA Contact: Shai Revivo <srevivo>
Severity: medium Docs Contact:
Priority: medium    
Version: DirectorCC: abeekhof, aschultz, jcoufal, jslagle, lnatapov, mburns, oblaut, rhel-osp-director-maint, rlondhe, udayendu.kar
Target Milestone: ---Keywords: FutureFeature, Reopened
Target Release: 10.0 (Newton)   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-09-11 19:08:55 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Leonid Natapov 2015-07-24 19:33:16 UTC
overcloud: There is very difficult to collect sosreport and crm_report without having root accesses between cluster nodes.

We have to enable ssh witth the root access between cluster nodes.

Until then, you'll have to configure ssh as you'd like to see it if you don't
want the default setup.

You can do the following across all the controllers:
- enable ssh root login
- enable password authentication for sshd
- generate ssh keys
- sync the keys (or use a common one)

Comment 3 James Slagle 2015-07-24 19:40:25 UTC
Please clearly describe what you're trying to do and how you need ssh configured to do it.

As things stand, it is not difficult to collect sosreport and crm_report from the nodes.

ssh heat-admin@<ip> sudo sosreport

then another command to copy the generated tarball off. having ssh root access would make that no easier.

Comment 4 Leonid Natapov 2015-07-24 19:53:39 UTC
I am opnening this BZ per your request after this thread.

when running crm_report ,it should log into each controller as root to collect the report.

Here is the thread:
----- Forwarded Message -----
From: "James Slagle" <jslagle>
To: "Ofer Blaut" <oblaut>
Cc: "Asaf Hirshberg" <ahirshbe>, "David Vossel" <dvossel>, "openstack-management-team-list" <openstack-management-team-list>, "Hugh Brock" <hbrock>
Sent: Thursday, July 23, 2015 5:35:41 PM
Subject: Re: sosreport and crm_report in rhos director

On Thu, Jul 23, 2015 at 10:25:40AM -0400, Ofer Blaut wrote:
> Hi
>
> I would like to explain the issue better.
>
> We have 4 hosts:
>
> host 1 - undercloud
> host 2-4 controllers
>
> We have access from 1 to 2-4 with heat-admin without password.
>
> crm/sos are located on hosts 2-4, which can't access each other using heat/root users.
>
> moving root key from 1 to 2-4 will not solve the issue, but will provide access as root without password only
>
> I would like to work out of the box as expected by support, what will happen if we will add host 5-6 as controllers ?

Can you put this into a bugzilla for A1? Perhaps as an RFE.

Until then, you'll have to configure ssh as you'd like to see it if you don't
want the default setup.

You can do the following across all the controllers:
- enable ssh root login
- enable password authentication for sshd
- generate ssh keys
- sync the keys (or use a common one)

While it'd have to be manual, it does work.

Note that running sosreport -o cluster, executes crm_report with -S, which
means generate a report for the single node only.

So I think you'd need to run crm_report manually to get a report for all nodes
in the cluster at once (unless there's a way to run without -S via sosreport).



>
> Thanks
>
> Ofer
>
>
>
>
> ----- Original Message -----
> > Hi James
> >
> > I'm using the system out of the box as we install it
> >
> > 1. using ssh -t heat-admin.2.13 sudo sosreport -o cluster, will not
> > collect the entire cluster information [1], same as manual crm_report
> > command from one of the controllers
> > 2. I don't know what is the practice used for support, but have one place to
> > contain all sos report will be great, the suggested usage will locate the
> > files on the controller and not on the instack
> >
> >
> > Ofer
> >
> >
> >
> >
> > [1] http://pastebin.test.redhat.com/299777
> >
> >
> > ----- Original Message -----
> > > On Wed, Jul 22, 2015 at 12:34:57PM -0400, Ofer Blaut wrote:
> > > >
> > > >
> > > > ----- Original Message -----
> > > > > On 22/07/15 16:52, Mike Orazi wrote:
> > > > > > On 07/22/2015 11:19 AM, Ofer Blaut wrote:
> > > > > >> Hi
> > > > > >>
> > > > > >> While trying to validate a bug [1] we need to provide crm_report .
> > > > > >>
> > > > > >> It seems to be a problem to run crm_report since we can't access
> > > > > >> hosts
> > > > > >> via
> > > > > >> user root [2] ,
> > > > > >>
> > > > > >> crm_report need root access to collect info so crm_report -u can't
> > > > > >> be
> > > > > >> used
> > > > > >> as heat-admin or root
> > > > > >>
> > > > > >> Can sos report collect info for list of hosts ? rhos-log-collector
> > > > > >> is
> > > > > >> not
> > > > > >> supported AFAIK
> > > > > >>
> > > > > >> Access each hosts via heat-admin ( after listing nova list) and
> > > > > >> later
> > > > > >> sudo
> > > > > >> and scp files doesn't scale, this should be done via undercloud host
> > > > > >>
> > > > > >> please advise
> > > > > >>
> > > > > >> Ofer
> > > > > >>
> > > > > >>
> > > > > >>
> > > > > >> [1] https://bugzilla.redhat.com/show_bug.cgi?id=1242339
> > > > > >> [2]  https://bugzilla.redhat.com/show_bug.cgi?id=1225069
> > > > > >>
> > > > > >
> > > > > >
> > > > > > Ofer,
> > > > > >
> > > > > > This does not directly solve your problem, but I added doc_text to:
> > > > > > https://bugzilla.redhat.com/show_bug.cgi?id=1225069
> > > > > >
> > > > > > indicating how one can use virt-customize to add root access to the
> > > > > > overcloud image prior to uploading it to glance.
> > > > > >
> > > > > > It is probably worthwhile to validate that my directions work for you
> > > > > > and see if that enables you to create a crm_report.
> > > > > >
> > > > > > Thanks,
> > > > > > - Mike
> > > > >
> > > > > Ofer, does `sudo sosreport -o cluster` not work here as the heat-admin
> > > > > user? I don't have an env to hand but I was sure this worked fine when
> > > > > I
> > > > > tested it some time ago.
> > > > >
> > > > > Lee
> > > >
> > > > It didn't work
> > >
> > > How doesn't it work? It wfm on an overcloud node directly. I could also run
> > > it
> > > via ssh:
> > >
> > > ssh -t heat-admin.2.13 sudo sosreport -o cluster
> > >
> > > (you have to use -t so you get the prompts asking you to continue,
> > > otherwise,
> > > you can just press Enter a few times).
> > >
> > > Note that you can also set a root password via ssh and heat-admin user:
> > > ssh heat-admin.2.13 sudo passwd
> > >
> > > You need to enable root login (disabled via cloud-init):
> > > ssh heat-admin.2.13 sudo sed -i 's/.*ssh-rsa/ssh-rsa/'
> > > /root/.ssh/authorized_keys
> > >
> > > [stack@instack ~]$ ssh root.2.13
> > > Last login: Thu Jul 23 07:44:12 2015 from 192.0.2.1
> > > [root@overcloud-controller-0 ~]#
> > >
> > >
> > > >
> > > > Ofer
> > > > > --
> > > > >
> > > > > Lee Yarwood
> > > > > Principal Software Maintenance Engineer
> > > > > Red Hat UK Ltd
> > > > >
> > > > > Registered in England and Wales under Company Registration No. 03798903
> > > > > Directors: Michael Cunningham (US), Paul Hickey (Ireland), Matt Parson
> > > > > (US), Charles Peters (US)
> > > > >
> > > > > GPG fingerprint : A5D1 9385 88CB 7E5F BE64  6618 BCA6 6E33 F672 2D76
> > > > >
> > > >
> > > --
> > > -- James Slagle
> > > --
> > >
> >
> >
--
-- James Slagle
--

Comment 5 James Slagle 2015-07-24 19:57:49 UTC
and you've never described what you need, why you need it, why it's difficult, and what would make it easier.

the more information you can provide, the better we'll be able to triage this bug.

asking you to open the bug was just a meaningless action so we could get a bug generated. It was to try and get to the root of what you're asking for and what would make it easier.

Comment 6 Ofer Blaut 2015-07-27 14:26:34 UTC
Hi James

1. We would like to CRM_REPORT tool running in NON -S (single mode )


Currently CRM_report of a cluster will fail , since no access to other hosts by default 

2.We would like SOS reports collected from all hosts into one centralized place 

instead of one sos per host 

Thanks

Ofer

Comment 7 Andrew Beekhof 2015-11-17 00:38:58 UTC
Typically clusters are viewed as a single entity.
When a problem occurs, its rarely useful to gather a sosreport for just one of the nodes.

In other environments, admins are used to being able to run crm_report from one node and have it collect the relevant information for all of them and put it in a single tarball.

I believe the request is they'd like to do the same (either via sosreport or crm_report) in an openstack context.

I guess it's the difference between one command + one tarball vs. potentially quite a few of each.  As someone who collects and analyses a lot of these, I'd like to see it too :)

Comment 9 Mike Burns 2016-04-07 20:43:53 UTC
This bug did not make the OSP 8.0 release.  It is being deferred to OSP 10.

Comment 11 Jaromir Coufal 2016-09-27 03:10:12 UTC
Obsolete bug, please reopen if still requested.

Comment 13 Alex Schultz 2017-09-11 19:08:55 UTC
This seems to be a duplicate request of Bug 1475040.  We have functionality in newer versions that might handle the customer's case. Alternatively, it seems a simple ansible playbook would cover the customer's case as well.  sosreport doesn't handle the cross system transfers so the request is for something that allows the execution of sosreport and transfer over existing network paths.

*** This bug has been marked as a duplicate of bug 1475040 ***