Description of problem: See e.g. bug 1352384. We should create the engine vm with better entropy sources, e.g. virtio-rng or haveged. If possible, it's probably better to do that in the appliance, so that it's applicable in every installation based on it. Please clone or move to hosted-engine-setup if needed. Not sure. Version-Release number of selected component (if applicable): Current How reproducible: Always Steps to Reproduce: 1. Deploy a machine based on the engine appliance 2. cat /proc/sys/kernel/random/entropy_avail 3. Optionally "use" some random data, e.g.: sh -xc 'while true; do dd if=/dev/random of=/dev/null bs=1 count=5; cat /proc/sys/kernel/random/entropy_avail; sleep 1; done' Actual results: Entropy is gathered slowly Expected results: Entropy is gathered faster Additional info: As per linked bug, a useful actual metric for success is time for engine-setup completion. I'd say it should be no more than a minute or so more that the time it takes on the physical host.
Didi, adding virtio-rng sounds reasonable. But this would be something which should be done by he-setup. I do not think that we can get better entropy by changing the appliance.
*** Bug 1368027 has been marked as a duplicate of this bug. ***
Checked on ovirt-hosted-engine-setup-2.0.2.2-2.el7ev.noarch I can see that HE VM has rng device: <rng model='virtio'> <backend model='random'>/dev/random</backend> <alias name='rng0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/> </rng> but it does not raise entropy level # time sh -xc 'while true; do dd if=/dev/random of=/dev/null bs=1 count=5; cat /proc/sys/kernel/random/entropy_avail; sleep 1; done' + true + dd if=/dev/random of=/dev/null bs=1 count=5 5+0 records in 5+0 records out 5 bytes (5 B) copied, 20.9678 s, 0.0 kB/s + cat /proc/sys/kernel/random/entropy_avail 153 + sleep 1 + true + dd if=/dev/random of=/dev/null bs=1 count=5 5+0 records in 5+0 records out 5 bytes (5 B) copied, 0.000109729 s, 45.6 kB/s + cat /proc/sys/kernel/random/entropy_avail 90 + sleep 1 + true + dd if=/dev/random of=/dev/null bs=1 count=5 5+0 records in 5+0 records out 5 bytes (5 B) copied, 0.000126877 s, 39.4 kB/s + cat /proc/sys/kernel/random/entropy_avail 27 + sleep 1 + true + dd if=/dev/random of=/dev/null bs=1 count=5 5+0 records in 5+0 records out 5 bytes (5 B) copied, 37.5261 s, 0.0 kB/s + cat /proc/sys/kernel/random/entropy_avail 153 + sleep 1 + true + dd if=/dev/random of=/dev/null bs=1 count=5 5+0 records in 5+0 records out 5 bytes (5 B) copied, 0.000120944 s, 41.3 kB/s + cat /proc/sys/kernel/random/entropy_avail 89 + sleep 1 + true + dd if=/dev/random of=/dev/null bs=1 count=5 5+0 records in 5+0 records out 5 bytes (5 B) copied, 0.000107812 s, 46.4 kB/s + cat /proc/sys/kernel/random/entropy_avail 91 + sleep 1 + true + dd if=/dev/random of=/dev/null bs=1 count=5 5+0 records in 5+0 records out 5 bytes (5 B) copied, 0.000108143 s, 46.2 kB/s + cat /proc/sys/kernel/random/entropy_avail 28 + sleep 1 + true + dd if=/dev/random of=/dev/null bs=1 count=5 ^C1+0 records in 1+0 records out 1 byte (1 B) copied, 16.0376 s, 0.0 kB/s real 1m21.553s user 0m0.007s sys 0m0.015s I believe HE VM just does not run enough time with enough devices, so maybe we can use something like rng-tools to generate the desired level of the entropy
Can you please run cat /sys/devices/virtual/misc/hw_random/rng_current on your engine vm?
Sure # cat /sys/devices/virtual/misc/hw_random/rng_current virtio_rng.0 I checked entropy level after rng-tools install: # yum install -y rng-tools # systemctl start rngd # time sh -xc 'while true; do dd if=/dev/random of=/dev/null bs=1 count=5; cat /proc/sys/kernel/random/entropy_avail; sleep 1; done' + true + dd if=/dev/random of=/dev/null bs=1 count=5 5+0 records in 5+0 records out 5 bytes (5 B) copied, 0.000153841 s, 32.5 kB/s + cat /proc/sys/kernel/random/entropy_avail 3167 + sleep 1 + true + dd if=/dev/random of=/dev/null bs=1 count=5 5+0 records in 5+0 records out 5 bytes (5 B) copied, 0.000143667 s, 34.8 kB/s + cat /proc/sys/kernel/random/entropy_avail 3119 + sleep 1 + true + dd if=/dev/random of=/dev/null bs=1 count=5 5+0 records in 5+0 records out 5 bytes (5 B) copied, 0.000133049 s, 37.6 kB/s + cat /proc/sys/kernel/random/entropy_avail 2982 + sleep 1 + true + dd if=/dev/random of=/dev/null bs=1 count=5 5+0 records in 5+0 records out 5 bytes (5 B) copied, 0.000144017 s, 34.7 kB/s + cat /proc/sys/kernel/random/entropy_avail 3038 + sleep 1 + true + dd if=/dev/random of=/dev/null bs=1 count=5 5+0 records in 5+0 records out 5 bytes (5 B) copied, 0.000114122 s, 43.8 kB/s + cat /proc/sys/kernel/random/entropy_avail 2995 + sleep 1 + true + dd if=/dev/random of=/dev/null bs=1 count=5 5+0 records in 5+0 records out 5 bytes (5 B) copied, 0.000131049 s, 38.2 kB/s + cat /proc/sys/kernel/random/entropy_avail 3025 + sleep 1 + true + dd if=/dev/random of=/dev/null bs=1 count=5 5+0 records in 5+0 records out 5 bytes (5 B) copied, 0.000130214 s, 38.4 kB/s + cat /proc/sys/kernel/random/entropy_avail 2577 + sleep 1 ^C real 0m6.398s user 0m0.006s sys 0m0.017s
(In reply to Artyom from comment #3) > Checked on ovirt-hosted-engine-setup-2.0.2.2-2.el7ev.noarch > > I can see that HE VM has rng device: > <rng model='virtio'> > <backend model='random'>/dev/random</backend> > <alias name='rng0'/> > <address type='pci' domain='0x0000' bus='0x00' slot='0x07' > function='0x0'/> > </rng> > > but it does not raise entropy level Why do you think so? > # time sh -xc 'while true; do dd if=/dev/random of=/dev/null bs=1 count=5; > cat /proc/sys/kernel/random/entropy_avail; sleep 1; done' > + true > + dd if=/dev/random of=/dev/null bs=1 count=5 > 5+0 records in > 5+0 records out > 5 bytes (5 B) copied, 20.9678 s, 0.0 kB/s > + cat /proc/sys/kernel/random/entropy_avail > 153 > + sleep 1 > + true > + dd if=/dev/random of=/dev/null bs=1 count=5 > 5+0 records in > 5+0 records out > 5 bytes (5 B) copied, 0.000109729 s, 45.6 kB/s > + cat /proc/sys/kernel/random/entropy_avail > 90 > + sleep 1 > + true > + dd if=/dev/random of=/dev/null bs=1 count=5 > 5+0 records in > 5+0 records out > 5 bytes (5 B) copied, 0.000126877 s, 39.4 kB/s > + cat /proc/sys/kernel/random/entropy_avail > 27 > + sleep 1 > + true > + dd if=/dev/random of=/dev/null bs=1 count=5 > 5+0 records in > 5+0 records out > 5 bytes (5 B) copied, 37.5261 s, 0.0 kB/s > + cat /proc/sys/kernel/random/entropy_avail > 153 Seems it raised it > + sleep 1 > + true > + dd if=/dev/random of=/dev/null bs=1 count=5 > 5+0 records in > 5+0 records out > 5 bytes (5 B) copied, 0.000120944 s, 41.3 kB/s > + cat /proc/sys/kernel/random/entropy_avail > 89 > + sleep 1 > + true > + dd if=/dev/random of=/dev/null bs=1 count=5 > 5+0 records in > 5+0 records out > 5 bytes (5 B) copied, 0.000107812 s, 46.4 kB/s > + cat /proc/sys/kernel/random/entropy_avail > 91 > + sleep 1 > + true > + dd if=/dev/random of=/dev/null bs=1 count=5 > 5+0 records in > 5+0 records out > 5 bytes (5 B) copied, 0.000108143 s, 46.2 kB/s > + cat /proc/sys/kernel/random/entropy_avail > 28 > + sleep 1 > + true > + dd if=/dev/random of=/dev/null bs=1 count=5 > ^C1+0 records in > 1+0 records out > 1 byte (1 B) copied, 16.0376 s, 0.0 kB/s Perhaps let it run for some more time, and also: 1. Compare with unpatched version 2. Compare with the host it's running on > > > real 1m21.553s > user 0m0.007s > sys 0m0.015s > > I believe HE VM just does not run enough time with enough devices, so maybe > we can use something like rng-tools to generate the desired level of the > entropy I'd not do that without consulting a security expert, and/or making it optional (so that users that care can override).
(In reply to Artyom from comment #5) > Sure > # cat /sys/devices/virtual/misc/hw_random/rng_current > virtio_rng.0 > > I checked entropy level after rng-tools install: > # yum install -y rng-tools > # systemctl start rngd According to http://rhelblog.redhat.com/2015/03/09/red-hat-enterprise-linux-virtual-machines-access-to-random-numbers-made-easy/ there is no need to run rng-tools since 7.1 and it should work transparently.
It seams that now the same VM is fine on the same test also removing the rng-tools package. I suspect that the host entropy was low during the test at https://bugzilla.redhat.com/show_bug.cgi?id=1357246#c3
Target release should be placed once a package build is known to fix a issue. Since this bug is not modified, the target version has been reset. Please use target milestone to plan a fix for a oVirt release.
Created attachment 1203111 [details] host and vm entroy I restarted HE VM(on deploy time, so VM still has virtio_rng.0 device), and checked entropy level for the VM and for the host where VM is running for the long period of time and for me it looks like passthrough of random dev to the VM does not work at all
(In reply to Simone Tiraboschi from comment #7) > According to > http://rhelblog.redhat.com/2015/03/09/red-hat-enterprise-linux-virtual- > machines-access-to-random-numbers-made-easy/ there is no need to run > rng-tools since 7.1 and it should work transparently. Amit, do we still have to start the rngd daemon on a RHEL 7.2 guest to take advantage of virtio-rng?
(In reply to Simone Tiraboschi from comment #11) > (In reply to Simone Tiraboschi from comment #7) > > According to > > http://rhelblog.redhat.com/2015/03/09/red-hat-enterprise-linux-virtual- > > machines-access-to-random-numbers-made-easy/ there is no need to run > > rng-tools since 7.1 and it should work transparently. > > Amit, do we still have to start the rngd daemon on a RHEL 7.2 guest to take > advantage of virtio-rng? For just virtio-rng, rngd is not needed. All logic is handled in the kernel.
(In reply to Amit Shah from comment #12) > For just virtio-rng, rngd is not needed. All logic is handled in the kernel. Thanks Amir, According to https://bugzilla.redhat.com/show_bug.cgi?id=1357246#c10 it seams that we are not fully satisfied of the results on the guest while the host deams fine. Could it depends from virtio-rng configuration? We are just relying on the default values without specifying a different bytes and period rate. The device looks like: <rng model='virtio'> <backend model='random'>/dev/random</backend> <alias name='rng0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/> </rng>
I guess the test in question just reads from /dev/random in the guest? That's triggering the worst-case behaviour: continuously requesting entropy from the host. Real workloads are not going to put such pressure. I suggest you try a more real-world test, like bug 1352384 mentioned in comment 0. It's also OK to use /dev/urandom as source (instead of /dev/random) in both the guest and the host. Reading from /dev/urandom doesn't affect the quality of randomness you get.
According to comment https://bugzilla.redhat.com/show_bug.cgi?id=1357246#c14 I'd suggest to verify this bug simply measuring how long does it take to setup hosted-engine with and without these patches on the same HW and storage. If we got a significative improvement we can set it as verified.
So ovirt-hosted-engine-setup-2.0.2 vs ovirt-hosted-engine-setup-2.0.2.2 without changing any other package.
From the execution of the HE with the different packages, I can see improvement: ovirt-hosted-engine-setup-2.0.2.2-2.el7ev.noarch =============================================== [ INFO ] Hosted Engine successfully deployed real 12m4.439s user 2m40.931s sys 0m11.821s 14:13:16 - Running engine-setup on the appliance 14:15:48 - Engine-setup successfully completed ovirt-hosted-engine-setup-2.0.2.1-1.el7ev.noarch =============================================== [ INFO ] Hosted Engine successfully deployed real 18m7.475s user 2m41.469s sys 0m12.059s 16:14:20 - Running engine-setup on the appliance 16:21:02 - Engine-setup successfully completed Verified on ovirt-hosted-engine-setup-2.0.2.1-1.el7ev.noarch