This service will be undergoing maintenance at 00:00 UTC, 2017-10-23 It is expected to last about 30 minutes
Bug 1313675 - Discovery breaks ignore_puppet_facts_for_provisioning global setting
Discovery breaks ignore_puppet_facts_for_provisioning global setting
Status: CLOSED ERRATA
Product: Red Hat Satellite 6
Classification: Red Hat
Component: Discovery Plugin (Show other bugs)
6.2.0
Unspecified Unspecified
unspecified Severity high (vote)
: Beta
: --
Assigned To: Lukas Zapletal
Sachin Ghai
http://projects.theforeman.org/issues...
: Triaged
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2016-03-02 03:09 EST by Lukas Zapletal
Modified: 2016-07-27 05:24 EDT (History)
4 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2016-07-27 05:24:33 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
Nics under host details when ignore_puppet_facts_for_provisioning set to 'false' (55.05 KB, image/png)
2016-04-15 07:29 EDT, Sachin Ghai
no flags Details
when ignore_puppet_facts_for_provisioning set to 'true' only one interface was created with host (58.71 KB, image/png)
2016-04-15 07:42 EDT, Sachin Ghai
no flags Details


External Trackers
Tracker ID Priority Status Summary Last Updated
Foreman Issue Tracker 12391 None None None 2016-04-22 12:44 EDT

  None (edit)
Description Lukas Zapletal 2016-03-02 03:09:02 EST
In Satellite 6.1 and 6.2, the disabling of auto-creation of network interfaces has stopped working. This used to be disabled by setting ignore puppet facts for provisioning to True, which we have set, but we still get network interfaces being created. This is a big problem for us on OpenStack compute nodes, which have many virtual interfaces created, and eventually this ends up with timeouts in puppet and Foreman as there can be many hundreds of interfaces for a host, which also stops us being able to edit the host. The only fix is to delete them all via the API.

This is caused by presence of Foreman Discovery plugin as it monkey patches the code. The solution is to do proper handling of fact import.
Comment 2 Bryan Kearney 2016-03-02 04:03:24 EST
Moving to POST since upstream bug http://projects.theforeman.org/issues/12391 has been closed
-------------
Matt Jarvis
In poking around in the facts importer in the codebase I spotted this commit https://github.com/theforeman/foreman/commit/379de99adbde93349824d4516b95079bfd970e90
which would partially solve the problem for us. In the meantime I've locally patched our copy to exclude the qbr|qvo|ovs|tap etc interfaces that OpenStack uses which seems to be stopping them being created. 
-------------
Brandon Weeks
Brandon Weeks wrote:
> +1 I can reproduce this.
> 
> Both ignore_puppet_facts_for_provisioning and #11972 are happening "too late." The fact_name table spam is unaffected by their use.  ignore_puppet_facts_for_provisioning doesn't even seem to prevent the nics table entries from being created.


https://github.com/theforeman/foreman_discovery/blob/develop/app/lib/puppet_fact_parser_extensions.rb#L25

Well that's definitely the cause. 
-------------
Brandon Weeks
Thank for spotting that commit Matt, it has been a huge help for us. I've proposed backporting in #11972. 
-------------
Lukas Zapletal
Matt, can you try this patch if it helps? https://github.com/theforeman/foreman_discovery/pull/234
-------------
Anonymous
Applied in changeset commit:foreman_discovery|94474795d581c2703e18087004a818973f7ebb58.
Comment 4 Sachin Ghai 2016-04-15 02:21:07 EDT
@Lukas, Could you please provide verification steps. Assuming host should have multiple interfaces, is it correct ? 

and bug description mentioned about issue with openstack node having multiple interfaces, does it mean QE should test this with openstack ?
Comment 5 Lukas Zapletal 2016-04-15 03:41:27 EDT
QA steps:

1) Install discovery
2) Turn off ignore_puppet_facts_for_provisioning
3) Register a unmanaged host via puppet (not via discovery)
4) It should have interface(s) detected under Host detail - NICs (at least one)
5) Also if you change IP or add new interface, new puppet fact upload should update these details in NICs table
6) Repeat steps 2-4 turning on the flag for a new host, it should NOT detecct new interface or update existing ones on change
Comment 6 Sachin Ghai 2016-04-15 07:28:16 EDT
Verified with sat6.2 beta snap8.1

1) Installed discovery on sat6server where discovery works via default proxy.

]# rpm -qa | grep discovery
foreman-discovery-image-3.1.1-3.el7sat.noarch
tfm-rubygem-foreman_discovery-5.0.0.3-1.el7sat.noarch
rubygem-smart_proxy_discovery-1.0.3-3.el7sat.noarch
tfm-rubygem-hammer_cli_foreman_discovery-0.0.2.1-1.el7sat.noarch

2) ignore_puppet_facts_for_provisioning setting was set to false.


3) Register a unmanaged host via puppet


]# puppet cert sign --all
	Notice: Signed certificate request for cloud-qe-19.idmqe.lab.eng.bos.redhat.com
Notice: Removing file Puppet::SSL::CertificateRequest cloud-qe-19.idmqe.lab.eng.bos.redhat.com at '/var/lib/puppet/ssl/ca/requests/cloud-qe-19.idmqe.lab.eng.bos.redhat.com.pem'


4) all interfaces on host have been detected and can be seen under host details -> NIC tab.
Comment 7 Sachin Ghai 2016-04-15 07:29 EDT
Created attachment 1147608 [details]
Nics under host details when  ignore_puppet_facts_for_provisioning set to 'false'
Comment 8 Sachin Ghai 2016-04-15 07:31:35 EDT
5) Now I changed the IP of an interface on existing host and tried to uploaf facts via

1) puppet agent -t
2) service puppet restart

but IP doesn't get updated.
Comment 9 Sachin Ghai 2016-04-15 07:40:21 EDT
Later, I set the ignore_puppet_facts_for_provisioning set to 'true', registered a new host via puppet and there I found only one interface created and that too without mac and IP address.

Also, I tried to create/add more interfaces on host manually and tried updating puppet via 'puppet agent -t'. No new interfaces updated under host details -> NIC tab.

Please see the attached screenshot.
Comment 10 Sachin Ghai 2016-04-15 07:42 EDT
Created attachment 1147610 [details]
when ignore_puppet_facts_for_provisioning set to 'true' only one interface was created with host
Comment 11 Sachin Ghai 2016-04-15 08:01:05 EDT
@lzap: 

Case1:
===
when ignore_puppet_facts_for_provisioning set to 'false', on registering a host via puppet adds all host interfaces and can be seen under host details -> Nic tab.  
I updated the IP of one interface and tried to upload facts again with "puppet agent -t" but interface IP was not updated. Is it correct way to update facts ?


and an exception raised in production.log:

2016-04-15 07:58:45 [app] [I] Started GET "/node/cloud-qe-19.idmqe.lab.eng.bos.redhat.com?format=yml" for 10.16.96.112 at 2016-04-15 07:58:45 -0400
2016-04-15 07:58:45 [app] [I] Processing by HostsController#externalNodes as YML
2016-04-15 07:58:45 [app] [I]   Parameters: {"name"=>"cloud-qe-19.idmqe.lab.eng.bos.redhat.com"}
2016-04-15 07:58:45 [app] [I]   Rendered text template (0.0ms)
2016-04-15 07:58:45 [app] [I] Completed 200 OK in 178ms (Views: 0.4ms | ActiveRecord: 7.0ms)
2016-04-15 07:58:46 [app] [I] Started POST "/api/hosts/facts" for 10.16.96.112 at 2016-04-15 07:58:46 -0400
2016-04-15 07:58:46 [app] [I] Processing by Api::V2::HostsController#facts as JSON
2016-04-15 07:58:46 [app] [I]   Parameters: {"facts"=>"[FILTERED]", "name"=>"cloud-qe-19.idmqe.lab.eng.bos.redhat.com", "certname"=>"cloud-qe-19.idmqe.lab.eng.bos.redhat.com", "apiv"=>"v2", :host=>{"name"=>"cloud-qe-19.idmqe.lab.eng.bos.redhat.com", "certname"=>"cloud-qe-19.idmqe.lab.eng.bos.redhat.com"}}
2016-04-15 07:58:46 [app] [W] Action failed
 | ActiveRecord::RecordInvalid: Validation failed: Fact name has already been taken
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/validations.rb:57:in `save!'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/attribute_methods/dirty.rb:29:in `save!'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/transactions.rb:273:in `block in save!'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/transactions.rb:329:in `block in with_transaction_returning_status'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/connection_adapters/abstract/database_statements.rb:199:in `transaction'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/transactions.rb:208:in `transaction'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/transactions.rb:326:in `with_transaction_returning_status'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/transactions.rb:273:in `save!'
 | /opt/rh/rh-ror41/root/usr/share/gems/gems/activerecord-4.1.5/lib/active_record/associations/has_many_association.rb:38:in `insert_record'
 |

case2:
=========
Later, for another host, I set ignore_puppet_facts_for_provisioning set to 'true' and registered new host via puppet and can see only one interface though there were multiple interfaces configured on host. 

I think case2 working as expected. But in case1, when I run puppet agent -t to update facts, an exception thrown in logs. Is it related to this bz ?
Comment 12 Lukas Zapletal 2016-04-15 08:11:44 EDT
I believe this is a different error, file new BZ for case 1. This should work I think, puppet expert need to evaluate (set Puppet component).

Case 2 is VERIFIED.

This BZ is VERIFIED. Thanks.
Comment 13 Sachin Ghai 2016-04-15 08:15:14 EDT
Moving this to verified as per comment12 and original issue has been resolved as described in comment11 under case2. Thanks
Comment 14 Sachin Ghai 2016-04-18 00:54:48 EDT
Bz filed for issue pointed in comment11 under case1 :

https://bugzilla.redhat.com/show_bug.cgi?id=1327576
Comment 16 errata-xmlrpc 2016-07-27 05:24:33 EDT
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2016:1501

Note You need to log in before you can comment on or make changes to this bug.