This issue depends on the functionality of the log collector in these circumstances (SSH to each host with it's specific name) and having the sos report on the remote host to function with the remote user.

Host deploy already supports adding a host with a different user name, for 3.3 it is blocked only in UI.

Keith,

Can you please comment on the log collector / sos report issue mentioned in comment #1

(In reply to Barak from comment #4)
> Keith,
> 
> Can you please comment on the log collector / sos report issue mentioned in
> comment #1

The hurdles for using a user other than 'root' as they relate to the Log Collector (and the Red Hat Support Plug-in) are as follows:

1) Both the Log Collector and the RH Support plug-in call SoS when they attempt to collect logs from the Hypervisors.  For many reasons, SoS must be run as 'root' because it vacuums up all sorts of data and log files, config files, etc. that only 'root' would have access to.  

2) You would need to store the actual user name of the non-root user somewhere on the RHEV-M. You need this because the Log Collector and the Red Hat Support plug-in would need to know the actual user name to use when attempting to SSH/SFTP into the designated hypervisor.

3) So that you don't have to type a user ID an password in every time you want to collect logs from N hypervisors, the RHEV-M installer would need to insert a public SSH key into each non-root user's /home/NON-ROOT-USER-HERE/.ssh/authorized_keys file.

4) The RHEV-H installer would need to be modified such that it actually created a non-root user.

5) The non-root user would need to have an entry in /etc/sudoers or console-helper on each RHEL-H or RHEV-H so that it could elevate it's privileges to 'root' and run SoS.  I would recommend consolehelper and have scripts for this.

These hurdles are not impossible but it would take major coordination among many  different RHEV components.  The impacted components that I can think of are as follows:
- RHEV-H installer
- The scripts which get run when the RHEV-M connects/adds a hypervisor (either RHEL-H or RHEV-H).  
- The RHEV-M needs to store the non-root user ID somewhere, and would we allow a different non-root user per-hypervisor.

(In reply to Keith Robertson from comment #5)
> (In reply to Barak from comment #4)
> > Keith,
> > 
> > Can you please comment on the log collector / sos report issue mentioned in
> > comment #1
> 
> The hurdles for using a user other than 'root' as they relate to the Log
> Collector (and the Red Hat Support Plug-in) are as follows:
> 
> 1) Both the Log Collector and the RH Support plug-in call SoS when they
> attempt to collect logs from the Hypervisors.  For many reasons, SoS must be
> run as 'root' because it vacuums up all sorts of data and log files, config
> files, etc. that only 'root' would have access to.  

but let's assume we have a user named kuku that has the same permissions as root,
Will Log Collector succeed collecting the data.


> 
> 2) You would need to store the actual user name of the non-root user
> somewhere on the RHEV-M. You need this because the Log Collector and the Red
> Hat Support plug-in would need to know the actual user name to use when
> attempting to SSH/SFTP into the designated hypervisor.

AFAIR this is done already, Yaniv ?


> 
> 3) So that you don't have to type a user ID an password in every time you
> want to collect logs from N hypervisors, the RHEV-M installer would need to
> insert a public SSH key into each non-root user's
> /home/NON-ROOT-USER-HERE/.ssh/authorized_keys file.
> 
> 4) The RHEV-H installer would need to be modified such that it actually
> created a non-root user.

this will be done as a part of the host deploy (if not done already), Alon ?

> 
> 5) The non-root user would need to have an entry in /etc/sudoers or
> console-helper on each RHEL-H or RHEV-H so that it could elevate it's
> privileges to 'root' and run SoS.  I would recommend consolehelper and have
> scripts for this.

Let's assume this is configured (if not final than we could alter it through host deploy

> 
> These hurdles are not impossible but it would take major coordination among
> many  different RHEV components.  The impacted components that I can think
> of are as follows:
> - RHEV-H installer
> - The scripts which get run when the RHEV-M connects/adds a hypervisor
> (either RHEL-H or RHEV-H).  
> - The RHEV-M needs to store the non-root user ID somewhere, and would we
> allow a different non-root user per-hypervisor.

>> 4) The RHEV-H installer would need to be modified such that it actually
>> created a non-root user.

> this will be done as a part of the host deploy (if not done already), Alon ?

correct, should work for regular host.

RHEV-H will not work as:
1. it has no regular user.
2. vdsm-reg and the UI only support configurating root.

But once we re-write the entire registration of rhev-h we also take that into account.

> > 2) You would need to store the actual user name of the non-root user
> > somewhere on the RHEV-M. You need this because the Log Collector and the Red
> > Hat Support plug-in would need to know the actual user name to use when
> > attempting to SSH/SFTP into the designated hypervisor.
> 
> AFAIR this is done already, Yaniv ?

yes, the username that the user used to deploy the host at first time is saved in vds_static table. currently, as barak said, this is only blocked via the ui field

Arthur, will the SOS report work with the suggested changes ?

Keith(In reply to Keith Robertson from comment #5)
> (In reply to Barak from comment #4)
> > Keith,
> > 
> > Can you please comment on the log collector / sos report issue mentioned in
> > comment #1
> 
> The hurdles for using a user other than 'root' as they relate to the Log
> Collector (and the Red Hat Support Plug-in) are as follows:
> 
> 1) Both the Log Collector and the RH Support plug-in call SoS when they
> attempt to collect logs from the Hypervisors.  For many reasons, SoS must be
> run as 'root' because it vacuums up all sorts of data and log files, config
> files, etc. that only 'root' would have access to. 

Does it validate "root" or it assumes that the user running SOS report has admin privileges ? 
 
> 
> 2) You would need to store the actual user name of the non-root user
> somewhere on the RHEV-M. You need this because the Log Collector and the Red
> Hat Support plug-in would need to know the actual user name to use when
> attempting to SSH/SFTP into the designated hypervisor.

This is ready


> 
> 3) So that you don't have to type a user ID an password in every time you
> want to collect logs from N hypervisors, the RHEV-M installer would need to
> insert a public SSH key into each non-root user's
> /home/NON-ROOT-USER-HERE/.ssh/authorized_keys file.


This is ready

> 
> 4) The RHEV-H installer would need to be modified such that it actually
> created a non-root user.


We can not know in advanced what user the customer wants to use.
So this part is left to the customer + it should give it privileges.

Now what does it need:
- just belong to the root group ?
- additional elevation configuration in the suduers ? 


> 
> 5) The non-root user would need to have an entry in /etc/sudoers or
> console-helper on each RHEL-H or RHEV-H so that it could elevate it's
> privileges to 'root' and run SoS.  I would recommend consolehelper and have
> scripts for this.


Not sure I understand what it means ?

> 
> These hurdles are not impossible but it would take major coordination among
> many  different RHEV components.  The impacted components that I can think
> of are as follows:
> - RHEV-H installer
> - The scripts which get run when the RHEV-M connects/adds a hypervisor
> (either RHEL-H or RHEV-H).  
> - The RHEV-M needs to store the non-root user ID somewhere, and would we
> allow a different non-root user per-hypervisor.

This is done

(In reply to Barak from comment #6)
> (In reply to Keith Robertson from comment #5)
> > (In reply to Barak from comment #4)
> > > Keith,
> > > 
> > > Can you please comment on the log collector / sos report issue mentioned in
> > > comment #1
> > 
> > The hurdles for using a user other than 'root' as they relate to the Log
> > Collector (and the Red Hat Support Plug-in) are as follows:
> > 
> > 1) Both the Log Collector and the RH Support plug-in call SoS when they
> > attempt to collect logs from the Hypervisors.  For many reasons, SoS must be
> > run as 'root' because it vacuums up all sorts of data and log files, config
> > files, etc. that only 'root' would have access to.  
> 
> but let's assume we have a user named kuku that has the same permissions as
> root,
> Will Log Collector succeed collecting the data.

We should allow ssh with regular user and privilege escalation, not duplicate root with different name.
 
> > 2) You would need to store the actual user name of the non-root user
> > somewhere on the RHEV-M. You need this because the Log Collector and the Red
> > Hat Support plug-in would need to know the actual user name to use when
> > attempting to SSH/SFTP into the designated hypervisor.
> 
> AFAIR this is done already, Yaniv ?

Correct.

> > 3) So that you don't have to type a user ID an password in every time you
> > want to collect logs from N hypervisors, the RHEV-M installer would need to
> > insert a public SSH key into each non-root user's
> > /home/NON-ROOT-USER-HERE/.ssh/authorized_keys file.
> > 
> > 4) The RHEV-H installer would need to be modified such that it actually
> > created a non-root user.
> 
> this will be done as a part of the host deploy (if not done already), Alon ?

The ovirt-node should have a user (example: user1) with the following entry in sudoers:

/etc/sudoers.d/50-otopi.conf
---
    Defaults:user1 !requiretty
    user1 ALL=(ALL) NOPASSWD: /bin/sh
---

> > 5) The non-root user would need to have an entry in /etc/sudoers or
> > console-helper on each RHEL-H or RHEV-H so that it could elevate it's
> > privileges to 'root' and run SoS.  I would recommend consolehelper and have
> > scripts for this.
> 
> Let's assume this is configured (if not final than we could alter it through
> host deploy

The entire idea is that host-deploy will run as non root also... so we need this at image, see (4).

(In reply to Alon Bar-Lev from comment #11)
> (In reply to Barak from comment #6)
> > (In reply to Keith Robertson from comment #5)
> > > (In reply to Barak from comment #4)
> > > > Keith,
> > > > 
> > > > Can you please comment on the log collector / sos report issue mentioned in
> > > > comment #1
> > > 
> > > The hurdles for using a user other than 'root' as they relate to the Log
> > > Collector (and the Red Hat Support Plug-in) are as follows:
> > > 
> > > 1) Both the Log Collector and the RH Support plug-in call SoS when they
> > > attempt to collect logs from the Hypervisors.  For many reasons, SoS must be
> > > run as 'root' because it vacuums up all sorts of data and log files, config
> > > files, etc. that only 'root' would have access to.  
> > 
> > but let's assume we have a user named kuku that has the same permissions as
> > root,
> > Will Log Collector succeed collecting the data.
> 
> We should allow ssh with regular user and privilege escalation, not
> duplicate root with different name.
>  

First, let met just submit my vote for the user 'kuku'.  I think that would be a superb addition to both the RHEV-H and RHEL-H.  One could imagine the ensuing hilarity in a support engagement.  

On a serious note, Alon is right you SSH in as non-root and you either use SUDO or console-helper to escalate privileges to root.
 

> > > 2) You would need to store the actual user name of the non-root user
> > > somewhere on the RHEV-M. You need this because the Log Collector and the Red
> > > Hat Support plug-in would need to know the actual user name to use when
> > > attempting to SSH/SFTP into the designated hypervisor.
> > 
> > AFAIR this is done already, Yaniv ?
> 
> Correct.
> 

Really?  Where and is the public key pair for this user installed on the RHEV-M?

> > > 3) So that you don't have to type a user ID an password in every time you
> > > want to collect logs from N hypervisors, the RHEV-M installer would need to
> > > insert a public SSH key into each non-root user's
> > > /home/NON-ROOT-USER-HERE/.ssh/authorized_keys file.
> > > 
> > > 4) The RHEV-H installer would need to be modified such that it actually
> > > created a non-root user.
> > 
> > this will be done as a part of the host deploy (if not done already), Alon ?
> 
> The ovirt-node should have a user (example: user1) with the following entry
> in sudoers:
> 
> /etc/sudoers.d/50-otopi.conf
> ---
>     Defaults:user1 !requiretty
>     user1 ALL=(ALL) NOPASSWD: /bin/sh
> ---
> 
> > > 5) The non-root user would need to have an entry in /etc/sudoers or
> > > console-helper on each RHEL-H or RHEV-H so that it could elevate it's
> > > privileges to 'root' and run SoS.  I would recommend consolehelper and have
> > > scripts for this.
> > 
> > Let's assume this is configured (if not final than we could alter it through
> > host deploy
> 
> The entire idea is that host-deploy will run as non root also... so we need
> this at image, see (4).

I can't comment on the host-deploy as non-root aspect but it seems reasonable that the RHEV-H would have a non-root user preconfigured for this purpose.  Also, lets not forget a RHEL-H install, you'd need an RPM(or something) to do the work of creating the non-root user and configuring either sudoers or console-helper. Not sure what the best way to accomplish this is.

Lastly, the non-root user needs to be known to the RHEV-M and that non-root user's public key needs to be available there so that both the Log Collector and Red Hat Access plug-in can use it to SSH/SFTP to the RHE[V/L]-H with no password prompting. 

For reference, here[1] are the console-helper scripts we use on the RHEV-M to collect SoS reports as the non-root user ovirt.

[1] https://code.engineering.redhat.com/gerrit/gitweb?p=redhat-support-plugin-rhev.git;a=tree;f=utils;hb=HEAD

(In reply to Keith Robertson from comment #12)
> (In reply to Alon Bar-Lev from comment #11)
> > (In reply to Barak from comment #6)
> > > > 2) You would need to store the actual user name of the non-root user
> > > > somewhere on the RHEV-M. You need this because the Log Collector and the Red
> > > > Hat Support plug-in would need to know the actual user name to use when
> > > > attempting to SSH/SFTP into the designated hypervisor.
> > > 
> > > AFAIR this is done already, Yaniv ?
> > 
> > Correct.
> > 
> 
> Really?  Where and is the public key pair for this user installed on the
> RHEV-M?

We have two methods:

1. ovirt-node registration - the PKI trust and SSH trust are established via ovirt-node initiative. if we to support non root account, the SSH trust should be established to this account.

2. ovirt-engine initiated - in this case the account within the ovirt-node should be enabled for password login, or the engine public key should be installed manually for that user. After host-deploy sequence the engine's public key is installed within the user's authorized_keys regardless of how it was initiated.
 
> > > > 5) The non-root user would need to have an entry in /etc/sudoers or
> > > > console-helper on each RHEL-H or RHEV-H so that it could elevate it's
> > > > privileges to 'root' and run SoS.  I would recommend consolehelper and have
> > > > scripts for this.
> > > 
> > > Let's assume this is configured (if not final than we could alter it through
> > > host deploy
> > 
> > The entire idea is that host-deploy will run as non root also... so we need
> > this at image, see (4).
> 

<snip>

> Lastly, the non-root user needs to be known to the RHEV-M and that non-root
> user's public key needs to be available there so that both the Log Collector
> and Red Hat Access plug-in can use it to SSH/SFTP to the RHE[V/L]-H with no
> password prompting. 

The other way around, the engine's public key should be installed at at that user's authorized_keys. This can be done via registration process (to be implemented) or later on by the host-deploy (already implemented).

(In reply to Alon Bar-Lev from comment #13)
> (In reply to Keith Robertson from comment #12)
> > (In reply to Alon Bar-Lev from comment #11)
> > > (In reply to Barak from comment #6)
> > > > > 2) You would need to store the actual user name of the non-root user
> > > > > somewhere on the RHEV-M. You need this because the Log Collector and the Red
> > > > > Hat Support plug-in would need to know the actual user name to use when
> > > > > attempting to SSH/SFTP into the designated hypervisor.
> > > > 
> > > > AFAIR this is done already, Yaniv ?
> > > 
> > > Correct.
> > > 
> > 
> > Really?  Where and is the public key pair for this user installed on the
> > RHEV-M?
> 
> We have two methods:
> 
> 1. ovirt-node registration - the PKI trust and SSH trust are established via
> ovirt-node initiative. if we to support non root account, the SSH trust
> should be established to this account.
> 
> 2. ovirt-engine initiated - in this case the account within the ovirt-node
> should be enabled for password login, or the engine public key should be
> installed manually for that user. After host-deploy sequence the engine's
> public key is installed within the user's authorized_keys regardless of how
> it was initiated.
>

Agree to above.

Will we configure any non-root user on the node or will it always be 'kuku', 'vdsm', whatever...?
  
> > > > > 5) The non-root user would need to have an entry in /etc/sudoers or
> > > > > console-helper on each RHEL-H or RHEV-H so that it could elevate it's
> > > > > privileges to 'root' and run SoS.  I would recommend consolehelper and have
> > > > > scripts for this.
> > > > 
> > > > Let's assume this is configured (if not final than we could alter it through
> > > > host deploy
> > > 
> > > The entire idea is that host-deploy will run as non root also... so we need
> > > this at image, see (4).
> > 
> 
> <snip>
> 
> > Lastly, the non-root user needs to be known to the RHEV-M and that non-root
> > user's public key needs to be available there so that both the Log Collector
> > and Red Hat Access plug-in can use it to SSH/SFTP to the RHE[V/L]-H with no
> > password prompting. 
> 
> The other way around, the engine's public key should be installed at at that
> user's authorized_keys. This can be done via registration process (to be
> implemented) or later on by the host-deploy (already implemented).

Agree to your description of the flow.  I could have worded my description a bit better.

(In reply to Keith Robertson from comment #14)
> Will we configure any non-root user on the node or will it always be 'kuku',
> 'vdsm', whatever...?

Yes.

vdsm - has specific usage, so cannot be.
kuku - oh... well...
admin - might be better.

afaiu, using different user (with full root privileges) should work currently, host-deploy will put engine's pk in the picked user authorized_keys, engine will save the username in db and will use it afterwards on reinstall if needed. 

for ovirt-node we need to configure the user as part of the installation in the node and try to register with it (might be better to split the rfe to ovirt-node and engine sides). 
about the logcollector or other tools that count on root access, I don't understand how we'll configure them to use this user. do we really need the console-helper scripts or just tell those tools to use different user\pass?

(In reply to Yaniv Bronhaim from comment #16)
> afaiu, using different user (with full root privileges) should work
> currently, host-deploy will put engine's pk in the picked user
> authorized_keys, engine will save the username in db and will use it
> afterwards on reinstall if needed. 
> 

Are you suggesting that each hypervisor, RHE[V/L]-H, could have a different non-root user or is it safe to assume that the non-root username will *always* be admin, kuku, whatever?

I am asking because allowing each hypervisor, RHE[V/L]-H, the flexibility to have different non-root usernames will be a problem for the Log Collector and the RHEV plug-in unless there is a mechanism for these tools to look up the username for a designated hypervisor.  IMO, allowing different non-root users per hypervisor is a bad idea which adds unnecessary complexity.

> for ovirt-node we need to configure the user as part of the installation in
> the node and try to register with it (might be better to split the rfe to
> ovirt-node and engine sides). 

ACK

> about the logcollector or other tools that count on root access, I don't
> understand how we'll configure them to use this user. do we really need the
> console-helper scripts or just tell those tools to use different user\pass?

Telling the tools to use different user\pass is insufficient.  Something on the RHE[V/L]-H must install a mechanism for a designated non-root user to execute SoS as root.

== Here are the minimum requirements for the hypervisor ==
- The hypervisor must have a non-root user.
- The non-root user must be given a mechanism to execute SoS (which requires root).  There are two options for this and they are sudo or console-helper.  I don't care which is used but we have approved console helper scripts for this.

== Here are the minimum requirements for the RHEV-M ==
- There must be REST endpoint/attribute which a client can query to determine if a node is using non-root (see [1]).  This is important because during upgrade scenarios customers will likely have mixed configurations with nodes using the old root-only mechanism and new nodes which support the new non-root mechanism.  The log collector must be able to ask the REST service what authentication mechanism a particular node uses before attempting to SSH/SFTP to that node.

- A public key from the RHEV-M must be installed into the authorized key file of the non-root user on the node.  The private key must be known to the Log Collector.  This private/public key pair could be the same one that is currently used or it could be a completely different one.  Don't care.  We just need to know what to use.

[1] 
if ( api.getHost('abc123').getFullVersion() >= 3.4){ 
   doNonRootLogin()
} else{
   doRootLogin()
}

*** Bug 508541 has been marked as a duplicate of this bug. ***

*** Bug 794867 has been marked as a duplicate of this bug. ***

*** Bug 1099591 has been marked as a duplicate of this bug. ***

*** Bug 1134903 has been marked as a duplicate of this bug. ***