Bug 1416779

Summary: libvirt-2.0.0-10.el7_3.4.x86_64 made ctdb - public ip is assigned to us but not on an interface - error
Product: Red Hat Enterprise Linux 7 Reporter: lejeczek <peljasz>
Component: sambaAssignee: Guenther Deschner <gdeschner>
Status: CLOSED NOTABUG QA Contact: qe-baseos-daemons
Severity: high Docs Contact:
Priority: unspecified    
Version: 7.3CC: asn, gdeschner, jrivera, jsuchane, madam, rbalakri
Target Milestone: rc   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-02-15 14:24:00 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description lejeczek 2017-01-26 11:32:13 UTC 
Description of problem:

I had a working cluster, running off 3 VMs, very basic, standard. I'm not sure if recent updates broke it.
I see these over & over in log file:

2017/01/24 22:20:05.025164 [recoverd: 3474]: Public IP '10.5.10.51' is assigned to us but not on an interface
2017/01/24 22:20:05.027571 [recoverd: 3474]: Trigger takeoverrun
2017/01/24 22:20:05.053386 [recoverd: 3474]: Takeover run starting
2017/01/24 22:20:05.106044 [ 3309]: Takeover of IP 10.5.10.51/28 on interface eth0

cluster reports:

~]$ ctdb status
Number of nodes:3
pnn:0 10.5.10.55       OK
pnn:1 10.5.10.56       OK (THIS NODE)
pnn:2 10.5.10.57       OK
Generation:91773797
Size:3
hash:0 lmaster:0
hash:1 lmaster:1
hash:2 lmaster:2
Recovery mode:NORMAL (0)
Recovery master:1

is seems that CTDB does not create/assign that public IP.

~]$ ctdb ip -v
Public IPs on node 1
10.5.10.51 node[1] active[eth0] available[eth0] configured[eth0]

I suspect the update, for nothing really changed except:

update to libvirt-2.0.0-10.el7_3.4.x86_64

well... ctdb and the rest of the samba suit updated also, but these same versions(as on VM cluster) are running fine on bare metal.


Version-Release number of selected component (if applicable):


How reproducible:

Should be very easy, my cluster runs off three VM guests on a host: libvirt-2.0.0-10.el7_3.4.x86_64
Set up a basic CTDB, in /etc/sysconfig/ctdb:

CTDB_RECOVERY_LOCK=/0-ALL.DATA/samba/.ctdb.lock
CTDB_PUBLIC_ADDRESSES=/etc/ctdb/public_addresses
# CTDB-RA: Auto-generated by /usr/lib/ocf/resource.d/pawel/CTDB, backup is at /etc/sysconfig/ctdb.ctdb-ra-orig
#CTDB_LOGGING=file:/var/log/log.ctdb
#CTDB_MONITOR_FREE_MEMORY=100
#CTDB_SAMBA_SKIP_SHARE_CHECK=yes
CTDB_MANAGES_SAMBA=yes
CTDB_MANAGES_WINBIND=no

CTDB_PUBLIC_NETWORK="10.5.10.48/28"
CTDB_PUBLIC_GATEWAY="10.5.10.49"

In /etc/ctdb/public_addresses:

10.5.10.51/28 eth0

You can also CTDB_MANAGES_SAMBA=no which I was curious about, but, still fails

Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:
Comment 2 Jaroslav Suchanek 2017-01-26 14:25:30 UTC 
Moving to ctdb component for an investigation.
Comment 3 lejeczek 2017-01-27 13:24:01 UTC 
a bit of an update: If I manually - ip addr add 10.5.10.51/28 dev eth0 - then ctdb quiets down and seems to be ok(?)
I think it's not SE, cannot find an evidence it is.
Comment 4 lejeczek 2017-01-30 10:05:07 UTC 
what happened was that VM ctdb cluster (all Centos 7.3) either:
* rpm update got a few event scripts loose their exec bit, including: 10.interface 11.routing
* pcsd toolkit did when I had ctdb under HA and when pcs resource were removed ctdb scripts did not get bitmask reset.
I'd say: please close the report
Comment 5 Andreas Schneider 2017-02-15 14:24:00 UTC 
Closing, thanks!