Bug 1313675 - Discovery breaks ignore_puppet_facts_for_provisioning global setting
Summary: Discovery breaks ignore_puppet_facts_for_provisioning global setting
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Satellite 6
Classification: Red Hat
Component: Discovery Plugin
Version: 6.2.0
Hardware: Unspecified
OS: Unspecified
unspecified
high vote
Target Milestone: Unspecified
Assignee: Lukas Zapletal
QA Contact: Sachin Ghai
URL: http://projects.theforeman.org/issues...
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-03-02 08:09 UTC by Lukas Zapletal
Modified: 2019-09-26 17:36 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-07-27 09:24:33 UTC
Target Upstream Version:


Attachments (Terms of Use)
Nics under host details when ignore_puppet_facts_for_provisioning set to 'false' (55.05 KB, image/png)
2016-04-15 11:29 UTC, Sachin Ghai
no flags Details
when ignore_puppet_facts_for_provisioning set to 'true' only one interface was created with host (58.71 KB, image/png)
2016-04-15 11:42 UTC, Sachin Ghai
no flags Details


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2016:1501 normal SHIPPED_LIVE Red Hat Satellite 6.2 Capsule and Server 2016-07-27 12:28:58 UTC
Foreman Issue Tracker 12391 None None None 2016-04-22 16:44:18 UTC

Description Lukas Zapletal 2016-03-02 08:09:02 UTC
In Satellite 6.1 and 6.2, the disabling of auto-creation of network interfaces has stopped working. This used to be disabled by setting ignore puppet facts for provisioning to True, which we have set, but we still get network interfaces being created. This is a big problem for us on OpenStack compute nodes, which have many virtual interfaces created, and eventually this ends up with timeouts in puppet and Foreman as there can be many hundreds of interfaces for a host, which also stops us being able to edit the host. The only fix is to delete them all via the API.

This is caused by presence of Foreman Discovery plugin as it monkey patches the code. The solution is to do proper handling of fact import.

Comment 2 Bryan Kearney 2016-03-02 09:03:24 UTC
Moving to POST since upstream bug http://projects.theforeman.org/issues/12391 has been closed
-------------
Matt Jarvis
In poking around in the facts importer in the codebase I spotted this commit https://github.com/theforeman/foreman/commit/379de99adbde93349824d4516b95079bfd970e90
which would partially solve the problem for us. In the meantime I've locally patched our copy to exclude the qbr|qvo|ovs|tap etc interfaces that OpenStack uses which seems to be stopping them being created. 
-------------
Brandon Weeks
Brandon Weeks wrote:
> +1 I can reproduce this.
> 
> Both ignore_puppet_facts_for_provisioning and #11972 are happening "too late." The fact_name table spam is unaffected by their use.  ignore_puppet_facts_for_provisioning doesn't even seem to prevent the nics table entries from being created.


https://github.com/theforeman/foreman_discovery/blob/develop/app/lib/puppet_fact_parser_extensions.rb#L25

Well that's definitely the cause. 
-------------
Brandon Weeks
Thank for spotting that commit Matt, it has been a huge help for us. I've proposed backporting in #11972. 
-------------
Lukas Zapletal
Matt, can you try this patch if it helps? https://github.com/theforeman/foreman_discovery/pull/234
-------------
Anonymous
Applied in changeset commit:foreman_discovery|94474795d581c2703e18087004a818973f7ebb58.

Comment 4 Sachin Ghai 2016-04-15 06:21:07 UTC
@Lukas, Could you please provide verification steps. Assuming host should have multiple interfaces, is it correct ? 

and bug description mentioned about issue with openstack node having multiple interfaces, does it mean QE should test this with openstack ?

Comment 5 Lukas Zapletal 2016-04-15 07:41:27 UTC
QA steps:

1) Install discovery
2) Turn off ignore_puppet_facts_for_provisioning
3) Register a unmanaged host via puppet (not via discovery)
4) It should have interface(s) detected under Host detail - NICs (at least one)
5) Also if you change IP or add new interface, new puppet fact upload should update these details in NICs table
6) Repeat steps 2-4 turning on the flag for a new host, it should NOT detecct new interface or update existing ones on change

Comment 6 Sachin Ghai 2016-04-15 11:28:16 UTC
Verified with sat6.2 beta snap8.1

1) Installed discovery on sat6server where discovery works via default proxy.

]# rpm -qa | grep discovery
foreman-discovery-image-3.1.1-3.el7sat.noarch
tfm-rubygem-foreman_discovery-5.0.0.3-1.el7sat.noarch
rubygem-smart_proxy_discovery-1.0.3-3.el7sat.noarch
tfm-rubygem-hammer_cli_foreman_discovery-0.0.2.1-1.el7sat.noarch

2) ignore_puppet_facts_for_provisioning setting was set to false.


3) Register a unmanaged host via puppet


]# puppet cert sign --all
	Notice: Signed certificate request for cloud-qe-19.idmqe.lab.eng.bos.redhat.com
Notice: Removing file Puppet::SSL::CertificateRequest cloud-qe-19.idmqe.lab.eng.bos.redhat.com at '/var/lib/puppet/ssl/ca/requests/cloud-qe-19.idmqe.lab.eng.bos.redhat.com.pem'


4) all interfaces on host have been detected and can be seen under host details -> NIC tab.

Comment 7 Sachin Ghai 2016-04-15 11:29:42 UTC
Created attachment 1147608 [details]
Nics under host details when  ignore_puppet_facts_for_provisioning set to 'false'

Comment 8 Sachin Ghai 2016-04-15 11:31:35 UTC
5) Now I changed the IP of an interface on existing host and tried to uploaf facts via

1) puppet agent -t
2) service puppet restart

but IP doesn't get updated.

Comment 9 Sachin Ghai 2016-04-15 11:40:21 UTC
Later, I set the ignore_puppet_facts_for_provisioning set to 'true', registered a new host via puppet and there I found only one interface created and that too without mac and IP address.

Also, I tried to create/add more interfaces on host manually and tried updating puppet via 'puppet agent -t'. No new interfaces updated under host details -> NIC tab.

Please see the attached screenshot.

Comment 10 Sachin Ghai 2016-04-15 11:42:35 UTC
Created attachment 1147610 [details]
when ignore_puppet_facts_for_provisioning set to 'true' only one interface was created with host

Comment 11 Sachin Ghai 2016-04-15 12:01:05 UTC
@lzap: 

Case1:
===
when ignore_puppet_facts_for_provisioning set to 'false', on registering a host via puppet adds all host interfaces and can be seen under host details -> Nic tab.  
I updated the IP of one interface and tried to upload facts again with "puppet agent -t" but interface IP was not updated. Is it correct way to update facts ?


and an exception raised in production.log:

2016-04-15 07:58:45 [app] [I] Started GET "/node/cloud-qe-19.idmqe.lab.eng.bos.redhat.com?format=yml" for 10.16.96.112 at 2016-04-15 07:58:45 -0400
2016-04-15 07:58:45 [app] [I] Processing by HostsController#externalNodes as YML
2016-04-15 07:58:45 [app] [I]   Parameters: {"name"=>"cloud-qe-19.idmqe.lab.eng.bos.redhat.com"}
2016-04-15 07:58:45 [app] [I]   Rendered text template (0.0ms)
2016-04-15 07:58:45 [app] [I] Completed 200 OK in 178ms (Views: 0.4ms | ActiveRecord: 7.0ms)
2016-04-15 07:58:46 [app] [I] Started POST "/api/hosts/facts" for 10.16.96.112 at 2016-04-15 07:58:46 -0400
2016-04-15 07:58:46 [app] [I] Processing by Api::V2::HostsController#facts as JSON
2016-04-15 07:58:46 [app] [I]   Parameters: {"facts"=>"[FILTERED]", "name"=>"cloud-qe-19.idmqe.lab.eng.bos.redhat.com", "certname"=>"cloud-qe-19.idmqe.lab.eng.bos.redhat.com", "apiv"=>"v2", :host=>{"name"=>"cloud-qe-19.idmqe.lab.eng.bos.redhat.com", "certname"=>"cloud-qe-19.idmqe.lab.eng.bos.redhat.com"}}
2016-04-15 07:58:46 [app] [W] Action failed
 | ActiveRecord::RecordInvalid: Validation failed: Fact name has already been taken
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/validations.rb:57:in `save!'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/attribute_methods/dirty.rb:29:in `save!'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/transactions.rb:273:in `block in save!'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/transactions.rb:329:in `block in with_transaction_returning_status'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/connection_adapters/abstract/database_statements.rb:199:in `transaction'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/transactions.rb:208:in `transaction'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/transactions.rb:326:in `with_transaction_returning_status'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/transactions.rb:273:in `save!'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/associations/has_many_association.rb:38:in `insert_record'
 |

case2:
=========
Later, for another host, I set ignore_puppet_facts_for_provisioning set to 'true' and registered new host via puppet and can see only one interface though there were multiple interfaces configured on host. 

I think case2 working as expected. But in case1, when I run puppet agent -t to update facts, an exception thrown in logs. Is it related to this bz ?

Comment 12 Lukas Zapletal 2016-04-15 12:11:44 UTC
I believe this is a different error, file new BZ for case 1. This should work I think, puppet expert need to evaluate (set Puppet component).

Case 2 is VERIFIED.

This BZ is VERIFIED. Thanks.

Comment 13 Sachin Ghai 2016-04-15 12:15:14 UTC
Moving this to verified as per comment12 and original issue has been resolved as described in comment11 under case2. Thanks

Comment 14 Sachin Ghai 2016-04-18 04:54:48 UTC
Bz filed for issue pointed in comment11 under case1 :

https://bugzilla.redhat.com/show_bug.cgi?id=1327576

Comment 16 errata-xmlrpc 2016-07-27 09:24:33 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2016:1501


Note You need to log in before you can comment on or make changes to this bug.