Bug 1232412

Summary: Unable to start snmp on RHEV-H 7.1
Product: Red Hat Enterprise Virtualization Manager Reporter: Robert McSwain <rmcswain>
Component: ovirt-nodeAssignee: Ryan Barry <rbarry>
Status: CLOSED INSUFFICIENT_DATA QA Contact: cshao <cshao>
Severity: high Docs Contact:
Priority: high    
Version: 3.5.3CC: adevolder, amureini, audgiri, cshao, ecohen, fdeutsch, lsurette, mgoldboi, pstehlik, rbarry, rmcswain, sapandit, ycui, yeylon
Target Milestone: ---Keywords: Reopened
Target Release: ---Flags: mgoldboi: Triaged+
Hardware: x86_64   
OS: Linux   
Whiteboard: node
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Cause: A change in the way snmp changes passwords Consequence: Changing the SNMP password in RHEV-H 7 caused SNMP to cease working. Fix: The snmp password file is now unmounted before changing Result: Changing the SNMP password works
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-12-18 15:07:18 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Node RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
snmp.tar.gz none

Description Robert McSwain 2015-06-16 17:10:08 UTC
Description of problem:

We install our rhev-h systems from RHEV-H 6.6 - 20150512.0.el6ev to the RHEV-H 7.1-20150512.1.el7ev. We install them with PXE-boot and enable it with snmp_password=<password> command in the pxe boot options. But snmp is not enabled.

If I try to enable snmp manually on RHEV-H. I get the following error:

│Command 'snmpusm -v 3 -u root -n "" -l authNoPriv -a SHA -A <password> localhost passwd <password>   │
│<password> -x AES' returned non-zero exit status 1

I use ssh to the rhev-h with the user admin. So I get the tui set the [X] and enter a password to enable snmp.

Why  I can't enable snmp on rhev-h 7.1?

Version-Release number of selected component (if applicable):
RHEV-H 7.1-20150512.1.el7e

How reproducible:
100%

Comment 1 Ryan Barry 2015-06-22 15:36:03 UTC
Robert - I can't reproduce this.

Is the customer using a weak password? SNMP requires at least 8 characters with no whitespace.

QE - can you reproduce?

Comment 2 cshao 2015-06-23 06:28:37 UTC
SNMP can enable by auto install, but the function of SNMP doesn't work.

Test version:
rhev-hypervisor7-7.1-20150609.0
ovirt-node-3.2.3-3.el7.noarch

Test steps:
1. Auto install RHEV-H with below parameters
BOOTIF=eth0 storage_init=/dev/sda adminpw=xxx snmp_password=redhat123 firstboot
2. Reboot
3. Login RHEV-H and check the status of snmp.
4. Check snmp function.
#snmpwalk -v3 -u root -l authPriv -a SHA -A redhat123 -x AES -X redhat123 $ip sysDescr
5. Set new snmp password on TUI.

Test result:
1. Auto install with "snmp" parameter can enable snmp.
2. In SNMP menu, SNMP show as enable status.
3. SNMP function doesn't work.
# snmpwalk -v3 -u root -l authPriv -a SHA -A redhat123 -x AES -X redhat123 10.66.9.251 sysDescr
snmpwalk: Unknown engine ID (Sub-id not found: (top) -> sysDescr)
4. An error occurred while applying the changes:    

│Command 'snmpusm -v 3 -u root -n "" -l authNoPriv -a SHA -A redhat123 localhost passwd redhat123 redhat123 -x AES' returned non-zero exit status 1  


# service snmpd status
Redirecting to /bin/systemctl status  snmpd.service
snmpd.service - Simple Network Management Protocol (SNMP) Daemon.
   Loaded: loaded (/usr/lib/systemd/system/snmpd.service; enabled)
   Active: active (running) since Tue 2015-06-23 06:06:01 UTC; 8min ago
 Main PID: 1232 (snmpd)
   CGroup: /system.slice/snmpd.service
           └─1232 /usr/sbin/snmpd -LS0-6d -f

Jun 23 06:06:01 localhost snmpd[1232]: buffer too small to read octet string (17 < 17)
Jun 23 06:06:01 localhost snmpd[1232]: Turning on AgentX master support.
Jun 23 06:06:01 localhost snmpd[1232]: Cannot rename /var/lib/net-snmp/snmpd.conf to /var/lib/net-snmp/snmpd.0.conf
Jun 23 06:06:01 localhost snmpd[1232]: Cannot unlink /var/lib/net-snmp/snmpd.conf
Jun 23 06:06:01 localhost snmpd[1232]: NET-SNMP version 5.7.2
Jun 23 06:06:01 localhost systemd[1]: Started Simple Network Management Protocol (SNMP) Daemon..

Comment 3 cshao 2015-06-23 06:29:36 UTC
Created attachment 1042104 [details]
snmp.tar.gz

/var/log/*.*
/tmp/ovirt.log

Comment 4 Ryan Barry 2015-06-23 14:13:27 UTC
I'm also not able to reproduce this report. After an autoinstall:

$ snmpwalk -v3 -u root -l authPriv -a SHA -A redhat123 -x AES -X redhat123 192.168.122.166 sysDescr
SNMPv2-MIB::sysDescr.0 = STRING: Linux localhost 3.10.0-229.4.2.el7.x86_64 #1 SMP Fri Apr 24 15:26:38 EDT 2015 x86_64

I see that you can get the traceback by trying to set a new password, though. I'll try that as well...

Comment 5 Ryan Barry 2015-06-23 15:26:13 UTC
So:

Test steps:
1. Auto install RHEV-H with below parameters
BOOTIF=eth0 storage_init=/dev/sda adminpw=xxx snmp_password=redhat123 firstboot
2. Reboot
3. Login RHEV-H and check the status of snmp.
4. Check snmp function.
#snmpwalk -v3 -u root -l authPriv -a SHA -A redhat123 -x AES -X redhat123 $ip sysDescr
5. Set new snmp password on TUI.
6. Test again

Test result:
1. Auto install with "snmp" parameter can enable snmp.
2. In SNMP menu, SNMP show as enable status.
3. SNMP  works.
# snmpwalk -v3 -u root -l authPriv -a SHA -A redhat123 -x AES -X redhat123 10.66.9.251 sysDescr
SNMPv2-MIB::sysDescr.0 = STRING: Linux localhost 3.10.0-229.4.2.el7.x86_64 #1 SMP Fri Apr 24 15:26:38 EDT 2015 x86_64
4. An error occurred while applying the changes
5. SNMP doesn't work
# snmpwalk -v3 -u root -l authPriv -a SHA -A redhat123 -x AES -X redhat123 10.66.9.251 sysDescr
snmpwalk: Unknown engine ID (Sub-id not found: (top) -> sysDescr)

This error appears to occur because the password cannot be changed successfully, which I'll look at. It works correctly in earlier versions of RHEV-H, so this must be a platform change.

Comment 6 Ryan Barry 2015-06-23 17:34:15 UTC
Robert -

The problem with changing the password appears to be because net-snmp now expects to be able to move the file to a backup, and so it never properly creates the user, so snmpusm can't find the object.

I wasn't able to reproduce the "snmp doesn't work with autoinstall" issue on a VM running on EL7, but I was on a VM running on F22, so it may not be a password length issue.

As a workaround (until the patch makes it into a build), they should be able to log in, press F2 to drop to a shell, and run the following commands (in order):

service snmpd stop
unpersist /var/lib/net-snmp/snmpd.conf
net-snmp-create-v3-user -A $password -a SHA -x AES root
service snmpd start
persist /var/lib/net-snmp/snmpd.conf

Comment 7 cshao 2015-07-02 08:33:24 UTC
Returning need info removed by bugzilla.

Comment 8 cshao 2015-09-08 07:09:37 UTC
Hi rmcswain,

I can't reproduce this issue, can you ask customer to help us to verify this bug?

Thanks!

Comment 9 cshao 2015-09-10 09:35:43 UTC
I report a new bug 1261424 for trace #c5's issue, but still can't reproduce the original bug.

Test version:
rhev-hypervisor7-7.1-20150827.1.el7ev
ovirt-node-3.2.3-20.el7.noarch

Thanks!