Bug 1463421

Summary: [Docs][Planning] Add entropy recommendation for SHE to Planning Guide
Product: Red Hat Enterprise Virtualization Manager Reporter: Sam Yangsao <syangsao>
Component: DocumentationAssignee: Tahlia Richardson <trichard>
Status: CLOSED CURRENTRELEASE QA Contact: Byron Gravenorst <bgraveno>
Severity: high Docs Contact:
Priority: medium    
Version: 4.1.2CC: dcadzow, lbopf, lsurette, mperina, rbalakri, srevivo, syangsao, ykaul, ylavi
Target Milestone: ovirt-4.1.6   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-10-06 03:56:35 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Docs RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Sam Yangsao 2017-06-20 21:00:34 UTC
Description of problem:

rhev 4.1 setup hangs at "[ INFO  ] Creating/refreshing Engine 'internal' domain database schema"

Version-Release number of selected component (if applicable):

# rhev v4.1.2

ovirt-web-ui-0.2.2-1.el7ev.x86_64
ovirt-engine-extension-aaa-jdbc-1.1.4-1.el7ev.noarch
ovirt-engine-websocket-proxy-4.1.2.3-0.1.el7.noarch
ovirt-engine-setup-4.1.2.3-0.1.el7.noarch
ovirt-engine-metrics-1.0.3-1.el7ev.noarch
ovirt-engine-setup-base-4.1.2.3-0.1.el7.noarch
ovirt-engine-dwh-setup-4.1.1-1.el7ev.noarch
ovirt-engine-restapi-4.1.2.3-0.1.el7.noarch
ovirt-engine-setup-plugin-vmconsole-proxy-helper-4.1.2.3-0.1.el7.noarch
ovirt-engine-sdk-python-3.6.9.1-1.el7ev.noarch
python-ovirt-engine-sdk4-4.1.3-1.el7ev.x86_64
ovirt-engine-lib-4.1.2.3-0.1.el7.noarch
ovirt-vmconsole-1.0.4-1.el7ev.noarch
ovirt-iso-uploader-4.0.2-1.el7ev.noarch
ovirt-imageio-proxy-setup-1.0.0-0.el7ev.noarch
ovirt-engine-tools-backup-4.1.2.3-0.1.el7.noarch
ovirt-host-deploy-java-1.6.5-1.el7ev.noarch
ovirt-engine-dwh-4.1.1-1.el7ev.noarch
ovirt-engine-backend-4.1.2.3-0.1.el7.noarch
ovirt-engine-webadmin-portal-4.1.2.3-0.1.el7.noarch
ovirt-engine-setup-plugin-ovirt-engine-4.1.2.3-0.1.el7.noarch
ovirt-log-collector-4.1.1-1.el7ev.noarch
ovirt-host-deploy-1.6.5-1.el7ev.noarch
ovirt-vmconsole-proxy-1.0.4-1.el7ev.noarch
ovirt-engine-setup-plugin-ovirt-engine-common-4.1.2.3-0.1.el7.noarch
ovirt-engine-extensions-api-impl-4.1.2.3-0.1.el7.noarch
ovirt-imageio-proxy-1.0.0-0.el7ev.noarch
ovirt-engine-setup-plugin-websocket-proxy-4.1.2.3-0.1.el7.noarch
ovirt-engine-userportal-4.1.2.3-0.1.el7.noarch
ovirt-engine-tools-4.1.2.3-0.1.el7.noarch
ovirt-engine-4.1.2.3-0.1.el7.noarch
ovirt-engine-cli-3.6.8.1-1.el7ev.noarch
ovirt-engine-dbscripts-4.1.2.3-0.1.el7.noarch
ovirt-engine-vmconsole-proxy-helper-4.1.2.3-0.1.el7.noarch
ovirt-setup-lib-1.1.0-1.el7ev.noarch
ovirt-engine-dashboard-1.1.2-1.el7ev.noarch
ovirt-imageio-common-1.0.0-0.el7ev.noarch

# rhel 7.3

# uname -a
Linux rhevm.lab.msp.redhat.com 3.10.0-514.21.2.el7.x86_64 #1 SMP Sun May 28 17:08:21 EDT 2017 x86_64 x86_64 x86_64 GNU/Linux

How reproducible:

Always

Steps to Reproduce:

Install RHEL 7.3 with the latest bits

# subscription-manager attach --pool=8a85f9833e1404a9013e3cddf95a0599
# subscription-manager repos --disable=*
# subscription-manager repos --enable=rhel-7-server-rpms
# subscription-manager repos --enable=rhel-7-server-supplementary-rpms
# subscription-manager repos --enable=rhel-7-server-rhv-4.1-rpms --enable=rhel-7-server-rhv-4-tools-rpms --enable=jb-eap-7-for-rhel-7-server-rpms
# yum -y install chrony vim
# systemctl enable chronyd
# systemctl start chronyd
# timedatectl set-local-rtc 0
# timedatectl 
# date
# yum -y update; yum -y install rhevm; reboot
# run engine-setup

Actual results:

engine-setup hangs at "[ INFO  ] Creating/refreshing Engine 'internal' domain database schema"

/var/log/ovirt-engine/setup/installation.log shows

2017-06-20 15:10:32 DEBUG otopi.context context._executeMethod:128 Stage misc METHOD otopi.plugins.ovirt_engine_setup.ovirt_engine.config.aaajdbc.Plugin._setupAdminPassword
2017-06-20 15:10:32 DEBUG otopi.plugins.ovirt_engine_setup.ovirt_engine.config.aaajdbc plugin.executeRaw:813 execute: ('/usr/bin/ovirt-aaa-jdbc-tool', '--db-config=/etc/ovirt-engine/aaa/internal.properties', 'user', 'password-reset', 'admin', '--password=env:pass', '--force', '--password-valid-to=2217-05-03 20:10:32Z'), executable='None', cwd='None', env={'pass': '**FILTERED**', 'LESSOPEN': '||/usr/bin/lesspipe.sh %s', 'SSH_CLIENT': '10.15.108.17 59926 22', 'SELINUX_USE_CURRENT_RANGE': '', 'LOGNAME': 'root', 'USER': 'root', 'OVIRT_ENGINE_JAVA_HOME': u'/usr/lib/jvm/jre', 'PATH': '/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin', 'HOME': '/root', 'OVIRT_JBOSS_HOME': '/opt/rh/eap7/root/usr/share/wildfly', 'LANG': 'en_US.UTF-8', 'TERM': 'xterm', 'SHELL': '/bin/bash', 'SHLVL': '1', 'HISTSIZE': '1000', 'XDG_RUNTIME_DIR': '/run/user/0', 'OVIRT_ENGINE_JAVA_HOME_FORCE': '1', 'PYTHONPATH': '/usr/share/ovirt-engine/setup/bin/..::', 'SELINUX_ROLE_REQUESTED': '', 'MAIL': '/var/spool/mail/root', 'XDG_SESSION_ID': '24', 'LS_COLORS': 'rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=01;05;37;41:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.jpg=01;35:*.jpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.axv=01;35:*.anx=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=01;36:*.au=01;36:*.flac=01;36:*.mid=01;36:*.midi=01;36:*.mka=01;36:*.mp3=01;36:*.mpc=01;36:*.ogg=01;36:*.ra=01;36:*.wav=01;36:*.axa=01;36:*.oga=01;36:*.spx=01;36:*.xspf=01;36:', 'SSH_TTY': '/dev/pts/0', 'HOSTNAME': 'rhevm.lab.msp.**FILTERED**.com', 'SELINUX_LEVEL_REQUESTED': '', 'HISTCONTROL': 'ignoredups', 'PWD': '/root', 'OTOPI_LOGFILE': '/var/log/ovirt-engine/setup/ovirt-engine-setup-20170620145735-5o90v4.log', 'SSH_CONNECTION': '10.15.108.17 59926 10.15.108.21 22', 'OTOPI_EXECDIR': '/root'}
2017-06-20 15:42:20 DEBUG otopi.plugins.ovirt_engine_setup.ovirt_engine.config.aaajdbc plugin.executeRaw:863 execute-result: ('/usr/bin/ovirt-aaa-jdbc-tool', '--db-config=/etc/ovirt-engine/aaa/internal.properties', 'user', 'password-reset', 'admin', '--password=env:pass', '--force', '--password-valid-to=2217-05-03 20:10:32Z'), rc=0
2017-06-20 15:42:20 DEBUG otopi.plugins.ovirt_engine_setup.ovirt_engine.config.aaajdbc plugin.execute:921 execute-output: ('/usr/bin/ovirt-aaa-jdbc-tool', '--db-config=/etc/ovirt-engine/aaa/internal.properties', 'user', 'password-reset', 'admin', '--password=env:pass', '--force', '--password-valid-to=2217-05-03 20:10:32Z') stdout:
updating user admin...
user updated successfully

Expected results:

engine-setup should just run through setting up the password quickly

Additional info:

Comment 1 Martin Perina 2017-06-21 07:56:08 UTC
Most probably your host/VM where you install engine doesn't have enough entropy, which is needed to encrypt admin@internal password. In case of a VM please check if you enabled /dev/random passthrough using virtio-rng or in case of a physical host you can install haveged service.

Comment 2 Sam Yangsao 2017-06-21 12:57:45 UTC
(In reply to Martin Perina from comment #1)
> Most probably your host/VM where you install engine doesn't have enough
> entropy, which is needed to encrypt admin@internal password. In case of a VM
> please check if you enabled /dev/random passthrough using virtio-rng or in
> case of a physical host you can install haveged service.

It looks low ..

# cat /proc/sys/kernel/random/entropy_avail 
157

I think we should probably document this somewhere in our installation guide or at least specify a warning on the engine-setup that this may need to be increased if they are using a VM for the RHV manager during setup.

I did install rng-tools and followed this article [1] to increase it on my RHEL 7 VM

# cat /proc/sys/kernel/random/entropy_avail 
3079

[1] https://access.redhat.com/solutions/1395493

Comment 3 Martin Perina 2017-06-23 20:39:44 UTC
Is it possible to add some note about entropy requirement into RHEVM installation guide?

Comment 4 Lucy Bopf 2017-06-26 05:08:42 UTC
(In reply to Martin Perina from comment #3)
> Is it possible to add some note about entropy requirement into RHEVM
> installation guide?

Hi Martin,

Sure, we can raise a docs bug for this; it sounds like it would go well in our upcoming Planning Guide. But we'll need some clearer details first. What is the entropy requirement for the machine hosting RHV-M?

Comment 5 Martin Perina 2017-07-18 07:06:49 UTC
(In reply to Lucy Bopf from comment #4)
> (In reply to Martin Perina from comment #3)
> > Is it possible to add some note about entropy requirement into RHEVM
> > installation guide?
> 
> Hi Martin,
> 
> Sure, we can raise a docs bug for this; it sounds like it would go well in
> our upcoming Planning Guide. But we'll need some clearer details first. What
> is the entropy requirement for the machine hosting RHV-M?

Well, we don't have any exact value which is required for RHV, but according to [1] values below 200 are too low, on my system I usually have the value around 3000.


[1] https://major.io/2007/07/01/check-available-entropy-in-linux/

Comment 6 Lucy Bopf 2017-07-19 04:37:02 UTC
(In reply to Martin Perina from comment #5)
> (In reply to Lucy Bopf from comment #4)
> > (In reply to Martin Perina from comment #3)
> > > Is it possible to add some note about entropy requirement into RHEVM
> > > installation guide?
> > 
> > Hi Martin,
> > 
> > Sure, we can raise a docs bug for this; it sounds like it would go well in
> > our upcoming Planning Guide. But we'll need some clearer details first. What
> > is the entropy requirement for the machine hosting RHV-M?
> 
> Well, we don't have any exact value which is required for RHV, but according
> to [1] values below 200 are too low, on my system I usually have the value
> around 3000.
> 
> 
> [1] https://major.io/2007/07/01/check-available-entropy-in-linux/

Thanks, Martin.

Yaniv, Derek, do you agree with adding this recommendation (entropy value above 200) to the Planning Guide?

Comment 7 Yaniv Lavi 2017-07-24 13:01:52 UTC
(In reply to Lucy Bopf from comment #6)
> 
> Thanks, Martin.
> 
> Yaniv, Derek, do you agree with adding this recommendation (entropy value
> above 200) to the Planning Guide?

Yes, we should, but we will need a recommended path to resolve and generate more entropy. Martin, what are the steps to workaround this?

Comment 8 Martin Perina 2017-07-27 09:02:26 UTC
So for the hosted engine VM this should be solved by BZ1413845 and this is much more regular use case (not having enough entropy inside VM). 

But most of the real hosts have enough entropy (at least I haven't heard of any real hosts entropy issues before this one). Usual solution to add entropy to the host is to install rngd (as mentioned in Comment 2) or install haveged [1]. 

Please bear in mind I'm not an expert in this area, so there may be other solutions.

[1] https://www.digitalocean.com/community/tutorials/how-to-setup-additional-entropy-for-cloud-servers-using-haveged

Comment 9 Sam Yangsao 2017-07-27 13:43:22 UTC
We also have an RFE [1] to have haveged added as a supported package - would love to have some PM magic added to this RFE :)

[1] https://bugzilla.redhat.com/show_bug.cgi?id=1472853

Comment 10 Lucy Bopf 2017-08-08 00:19:37 UTC
Moving to Documentation.

Comment 11 Lucy Bopf 2017-08-08 07:42:37 UTC
Assigning to Tahlia for review.

Tahlia, we should provide the recommendation for entropy, and then link to the RHEL docs for adding entropy if needed.

Comment 15 Byron Gravenorst 2017-10-06 02:28:53 UTC
Reviewed and merged.