Bug 1227755 - discoverd edeploy plugin puts too much data in Ironic extra column
Summary: discoverd edeploy plugin puts too much data in Ironic extra column
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-ironic-discoverd
Version: Director
Hardware: All
OS: Linux
high
high
Target Milestone: ga
: Director
Assignee: John Trowbridge
QA Contact: Toure Dunnon
URL:
Whiteboard:
: 1234232 1237058 (view as bug list)
Depends On:
Blocks: 1238779
TreeView+ depends on / blocked
 
Reported: 2015-06-03 12:34 UTC by John Trowbridge
Modified: 2015-08-05 13:52 UTC (History)
13 users (show)

Fixed In Version: ahc-tools-0.1.1-4.el7ost openstack-ironic-discoverd-1.1.0-4.el7ost instack-undercloud-2.1.2-1.el7ost
Doc Type: Bug Fix
Doc Text:
The edeploy plugin for ironic-discoverd collected too much information to store in a SQL blob. Discovery failed when edeploy data was posted to Ironic because the column would overflow. This fix changes the edeploy plugin to stores data in a Swift object on the Undercloud. Discovery no longer fails when using the edeploy plugin.
Clone Of:
Environment:
Last Closed: 2015-08-05 13:52:33 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Launchpad 1461252 None None None Never
OpenStack gerrit 189427 None None None Never
Red Hat Product Errata RHEA-2015:1549 normal SHIPPED_LIVE Red Hat Enterprise Linux OpenStack Platform director Release 2015-08-05 17:49:10 UTC

Description John Trowbridge 2015-06-03 12:34:20 UTC
Description of problem:

The edeploy plugin stores all of the facts it collects in the extra column of the Ironic db. When using real hardware, edeploy collects a large amount of facts and can lead to overflowing that column:

"InternalServerError: Remote error: DBError (DataError) (1406, "Data too long for column 'extra' at row 1") 'UPDATE nodes SET updated_at=%s, properties=%s, extra=%s WHERE nodes.id = %s'"

This causes the inspection ramdisk to fail and drop to an emergency shell.


How reproducible:

This was reported upstream, and I have not personally reproduced the issue. Running extended hardware benchmarks would help reproduce the issue, as that collects even more facts.


Steps to Reproduce:
1. Follow the documentation
2. `openstack baremetal introspection bulk start` step will fail

Actual results:

This error appears on the console of the machine running the ramdisk:

"InternalServerError: Remote error: DBError (DataError) (1406, "Data too long for column 'extra' at row 1") 'UPDATE nodes SET updated_at=%s, properties=%s, extra=%s WHERE nodes.id = %s'"

Then the ramdisk fails and drops to an emergency shell.

Expected results:

The inspection ramdisk does not error and drop to an emergency shell.

Additional info:

Upstream, the idea is to resolve this by storing the edeploy facts in Swift and just putting a reference in the Ironic database.

Comment 4 Alexander Chuzhoy 2015-06-19 18:24:05 UTC
Reproduced on my setup when I had 2 virtual hard drives configured for a host. Once the hard drives were removed - the discovery completed successfully.

Comment 5 John Trowbridge 2015-06-19 18:58:37 UTC
Upstream patches to fix this are all merged:

discoverd (renamed to inspector):
https://review.openstack.org/#/c/189427/

instack-undercloud:
https://review.gerrithub.io/#/c/236369/

ahc-tools:
https://review.gerrithub.io/#/c/236579

Comment 7 Mike Burns 2015-06-22 15:31:02 UTC
*** Bug 1234232 has been marked as a duplicate of this bug. ***

Comment 9 Mike Burns 2015-06-30 10:39:14 UTC
*** Bug 1237058 has been marked as a duplicate of this bug. ***

Comment 10 Dariusz Smigiel 2015-07-06 10:35:46 UTC
Tested on OSP7 RC2 and Director Beta2 and problem no longer occurs

Comment 13 errata-xmlrpc 2015-08-05 13:52:33 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2015:1549


Note You need to log in before you can comment on or make changes to this bug.