Bug 1092106
| Summary: | pacemaker: CIB can be corrupted via pcs commands | ||||||
|---|---|---|---|---|---|---|---|
| Product: | Red Hat Enterprise Linux 7 | Reporter: | Robert Peterson <rpeterso> | ||||
| Component: | pacemaker | Assignee: | Andrew Beekhof <abeekhof> | ||||
| Status: | CLOSED NOTABUG | QA Contact: | Cluster QE <mspqa-list> | ||||
| Severity: | urgent | Docs Contact: | |||||
| Priority: | urgent | ||||||
| Version: | 7.0 | CC: | abeekhof, cluster-maint, dvossel, fdinitto, rpeterso, tis | ||||
| Target Milestone: | rc | ||||||
| Target Release: | --- | ||||||
| Hardware: | Unspecified | ||||||
| OS: | Unspecified | ||||||
| Whiteboard: | |||||||
| Fixed In Version: | Doc Type: | Bug Fix | |||||
| Doc Text: | Story Points: | --- | |||||
| Clone Of: | Environment: | ||||||
| Last Closed: | 2014-07-08 02:00:11 UTC | Type: | Bug | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Embargoed: | |||||||
| Bug Depends On: | |||||||
| Bug Blocks: | 1111381 | ||||||
| Attachments: |
|
||||||
|
Description
Robert Peterson
2014-04-28 18:24:44 UTC
This is not a valid way to manage the cib, but I could see where users coming from rgmanager might actually hit something like this. It would be nice if we handled it in a sane way rather than hosing up the entire cluster. I believe the new cib v2 updates will fix this since all updates will be applied in order via cpg rather than being applied locally and then sent out via cpg. rhel 7.1 is already scheduled to receive that cib update for the performance optimizations it provides. -- Vossel (In reply to David Vossel from comment #2) > This is not a valid way to manage the cib, but I could see where users > coming from rgmanager might actually hit something like this. It would be > nice if we handled it in a sane way rather than hosing up the entire cluster. > > I believe the new cib v2 updates will fix this since all updates will be > applied in order via cpg rather than being applied locally and then sent out > via cpg. rhel 7.1 is already scheduled to receive that cib update for the > performance optimizations it provides. > > -- Vossel Valid or not, I don't think CIB corruption is acceptable in any way IMHO. Can we have an interim fix in 7.0.z that doesn't require a full rebase? I'd need more details about "Errors found during check: config not valid" A crm_report would do it. Even running all those commands on all nodes shouldn't be able to produce an invalid config. I do notice that stonith isn't enabled... that might do it. Created attachment 890759 [details]
crm report
I gave the crm report to dvossel, but here's a copy.
Right, so as I proposed in comment #4, its considered an invalid configuration for reasons entirely unrelated to the cssh session: [10:20 AM] beekhof@f19 ~/Development/sources/pacemaker/rhel-branches ☹ # tools/crm_verify -x ~/Downloads/pe-input-15.bz2 -V error: unpack_resources: Resource start-up disabled since no STONITH resources have been defined error: unpack_resources: Either configure some or disable STONITH with the stonith-enabled option error: unpack_resources: NOTE: Clusters with shared data need STONITH to ensure data integrity Errors found during check: config not valid I do see this in the logs: Apr 28 12:34:49 gfs-i8c-01 cib[3595]: error: xml_log: Invalid attribute id for element rsc_order Apr 28 12:34:49 gfs-i8c-01 cib[3595]: error: xml_log: Element constraints has extra content: rsc_order Apr 28 12:34:49 gfs-i8c-01 cib[3595]: error: xml_log: Invalid sequence in interleave Apr 28 12:34:49 gfs-i8c-01 cib[3595]: error: xml_log: Element configuration failed to validate content Apr 28 12:34:49 gfs-i8c-01 cib[3595]: error: xml_log: Element cib failed to validate content but thats a precursor to: Apr 28 12:34:49 gfs-i8c-01 cib[3595]: warning: cib_perform_op: Updated CIB does not validate against pacemaker-1.2 schema/dtd Apr 28 12:34:49 gfs-i8c-01 cib[3595]: warning: cib_diff_notify: Local-only Change (client:cibadmin, call: 2): 0.18.1 (Update does not conform to the configured schema) Apr 28 12:34:49 gfs-i8c-01 cib[3595]: warning: cib_process_request: Completed cib_replace operation for section constraints: Update does not conform to the configured schema (rc=-203, origin=local/cibadmin/2, version=0.17.1) which is pacemaker rejecting an update because it would have made the configuration inconsistent/invalid. Ie. it shows us actively preventing $subject Can I close? Clearing Zstream request till we get to the bottom of it. |