Bug 1387424

Summary: Rebase clufter component [RHEL 7.4]
Product: Red Hat Enterprise Linux 7 Reporter: Jan Pokorný [poki] <jpokorny>
Component: clufterAssignee: Jan Pokorný [poki] <jpokorny>
Status: CLOSED ERRATA QA Contact: cluster-qe <cluster-qe>
Severity: medium Docs Contact: Steven J. Levine <slevine>
Priority: medium    
Version: 7.4CC: borgan, cfeist, cluster-qe, extras-qa, jpokorny, lkocman, lkuprova, mlisik, nobody, phagara, royoung, rsteiger, rwilliam, salmy, slevine, tlavigne, vpavlin
Target Milestone: rcKeywords: Rebase
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: clufter-0.76.0-1.el7 Doc Type: Rebase: Bug Fixes and Enhancements
Doc Text:
_clufter_ rebased to version 0.76.0 and fully supported The _clufter_ packages provide a tool for transforming and analyzing cluster configuration formats. They can be used to assist with migration from an older stack configuration to a newer configuration that leverages Pacemaker. The `clufter` tool, previously available as a Technology Preview, is now fully supported. For information on the capabilities of `clufter`, see the `clufter(1)` man page or the output of the "clufter -h" command. For examples of `clufter` usage, see the following Red Hat Knowledgebase article: https://access.redhat.com/articles/2810031. The _clufter_ packages have been upgraded to upstream version 0.76.0, which provides a number of bug fixes and new features. Among the notable updates are the following: * When converting either CMAN + RGManager stack specific configuration into the respective Pacemaker configuration (or sequence of `pcs` commands) with the `ccs2pcs*` families of commands, the `clufter` tool no longer refuses to convert entirely valid lvm resource agent configuration. * When converting CMAN-based configuration into the analogous configuration for a Pacemaker stack with the `ccs2pcs` family of commands, some resources related configuration bits previously lost in processing (such as _maximum number of failures before returning a failure to a status check_) are now propagated correctly. * When producing `pcs` commands with the `cib2pcs` and `pcs2pcscmd` families of `clufter` commands, proper finalized syntax is now used for the alert handlers definitions, for which the (default) behavior of single-step push of the configuration changes is now respected. * When producing `pcs` commands, the `clufter` tool now supports a preferred ability to generate `pcs` commands that will update only the modifications made to a configuration by means of a differential update rather than a pushing a wholesale update of the entire configuration. Likewise when applicable, the `clufter` tool now supports instructing the `pcs` tool to configure user permissions (ACLs). For this to work across the instances of various major versions of the document schemas, `clufter` gained the notion of internal on-demand format upgrades, mirroring the internal mechanics of `pacemaker`. Similarly, `clufter` is now capable of configuring the `bundle` feature. * In any script-like output sequence such as that produced by the `ccs2pcscmd` and `pcs2pcscmd` families of `clufter` commands, the intended shell interpreter is now emitted as a first, commented line as also understood directly by the operating system in order to clarify where Bash rather than a mere POSIX shell is expected. This might have been misleading under some circumstances in the past. * The Bash completion file for `clufter` no longer fails to work properly when the `=` character is to specify an option's value in the sequence being completed. * The `clufter` tool now properly detects interactive use on a terminal so as to offer more convenient representation of the outputs, and also provides better diagnostics for some previously neglected error conditions.
Story Points: ---
Clone Of: 1343661 Environment:
Last Closed: 2017-08-01 23:06:21 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1343661    
Bug Blocks: 1381522, 1381531, 1381565    

Description Jan Pokorný [poki] 2016-10-20 20:39:30 UTC
+++ This bug was initially created as a clone of Bug #1343661 +++

Upstream development of clufter is advancing (since version targetting
RHEL 7.3, that is v0.59.5, at least 0.59.6 has been released) so this
asks for a rebase.

So far, clufter received bug fix for [bug 1381522].

Comment 2 Jan Pokorný [poki] 2017-03-24 18:22:50 UTC
Currently, there is a possible issue with clufter downgrade,
should it ever be needed:

Prerequisite:
- install all packages from clufter.srpm in 0.70.0-1 version

# yum downgrade clufter
(yum downgrade clufter python-clufter)
(yum downgrade clufter-cli)
> [...]
> Error: Package: clufter-cli-0.59.5-2.el7.noarch (repo)
>            Requires: python-clufter = 0.59.5-2.el7
>            Installed: python2-clufter-0.70.0-1.el7.noarch (installed)
>                python-clufter = 0.70.0-1.el7
>  You could try using --skip-broken to work around the problem
>  You could try running: rpm -Va --nofiles --nodigest

# yum downgrade --skip-broken '*clufter*'
> [...]
> Packages skipped because of dependency problems:
>     clufter-cli-0.59.5-2.el7.noarch from repo
>     clufter-lib-ccs-0.59.5-2.el7.noarch from repo
>     clufter-lib-general-0.59.5-2.el7.noarch from repo
>     clufter-lib-pcs-0.59.5-2.el7.noarch from repo

# yum remove clufter-{cli,lib-{ccs,general,pcs}}
> [...]
> Removed:
>   clufter-cli.noarch 0:0.70.0-1.el7
>   clufter-lib-ccs.noarch 0:0.70.0-1.el7
>   clufter-lib-general.noarch 0:0.70.0-1.el7
>   clufter-lib-pcs.noarch 0:0.70.0-1.el7
>
> Complete!

# yum downgrade python-clufter
> No Match for available package: python-clufter-0.59.5-2.el7.x86_64
> Nothing to do

# yum downgrade python2-clufter
> Nothing to do

Argh?!

Comment 3 Jan Pokorný [poki] 2017-03-27 19:57:03 UTC
+ another thing I need to do is revert requiring python2-setuptools
back to python-setuptools.  The former is/be currently only provided
in 7.4+, see [bug 1259474], and we don't want to impose such artificial
version-lock simply because the underlying functionality remains the
same.

Comment 4 Jan Pokorný [poki] 2017-03-30 17:50:39 UTC
Both issues from [comment 2] (see also relevant report filed against
yum (+ its depsolving) [bug 1437230]) and [comment 3], respectively,
should be fixed in 0.70.0-2.

Comment 12 Jan Pokorný [poki] 2017-05-30 16:34:26 UTC
All-at-once changelog between v0.59.5 (excluding) and v0.75.0 (including)

$ git log --tags --oneline --simplify-by-decoration v0.59.5...v0.75.0 \
  | cut -d' ' -f1 | xargs -l -- git tag -l -n200 --points-at \
  | sed 's|^\(v[^ ]*\)[ ][ ]*\(.*\)$|\n\n\1\n\2|;s|^    ||;s|rhbz#|bug |'


v0.75.0
enhancement and Python 3 compatibility improving release

- enhancements:
  . when using *2pcscmd commands, one can now specify --set-exec
    option to make executable; note this is explicitly discouraged
    as one should always take precautions prior to running anything
    executable from a semirandom source and this would only make
    it easier to skip that step and accidentally run the generated
    script right away
  . with {cib,pcs}2pcscmd commands, the generated sequence is now
    adapted to whether pcs is able to natively cope with "external/*"
    specification of stonith resources where applicable
  . borrowing directly from pacemaker project, the upgrade between
    1.x and 2.x lines of the schema for a CIB instance is now
    available as fmt-cib-1to2 filter, arranged to be implicitly
    triggered so as to bridge supplied vs. required format
    discrepancies (see first internal enhancement point below)
- feature extensions:
  . {cib,pcs}2pcscmd* commands are now aware of configured ACL
    definitions in CIB and able to emit respective configuration
    commands using pcs tool; thanks to implicit format upgrade
    arrangement with fmt-cib-1to2 filter (see above), it can cope
    with "old ACL style" of configuration at the input as well
    [resolves: bug 1440876]
  . likewise {cib,pcs}2pcscmd* commands now instruct sufficiently
    recent target pcs to perform push of the CIB changes (accumulated
    throughout the generated script) as a differential update rather
    than propagating a complete CIB copy, which remains the default
    otherwise (both unless push-by-push is applied, despite warnings)
- internal enhancements:
  . format hierarchies were noticeably rethought with a notion of
    upgradeability in mind -- something natural for the versioned,
    compatibility-sensitive formats such as CIB, the prime target
    (see also the mentioned upgrade filter); with a proper
    arrangement these upgrades are performed implicitly on the
    format instances being exchanged between the filters, should
    the target filter impose that on validation failure basis
  . Python 3 compatibility received further polishing
  . meta specfile holding the suggested RPM packaging moved "nano"
    fallback editor dependency over to -cli package, resulting in
    a bit more lightweight pcs installation, as that itself depends
    just on python*-clufter
    [resolves: PGissue#1]


v0.70.0
bug fix, enhancement and, foremostly, Python 3 compatibility release

- bug fixes:
  . since v0.59.7, {ccs,pcs}2pcscmd commands were meant to emit
    hashbangs so as to make it clear which shell (e.g. bash if its
    extensions are relied upon) the suggested scripts should be run
    under, but alas, exclamation mark has been accidentally missing
    [related bug 1381531]
  . invocations like "clufter ccs2pcscmd | bash -x", despite being
    discouraged (review is needed for any nontrivial case, anyway),
    if "pcs cluster auth" part interactively prompting credentials
    was part of the run, they would be obtained from the piped script
    under interpreter's execution itself due to the way the descriptors
    are inherited; now there is an explicit stdin/stdout redirect
    against /dev/tty in place to prevent such undesired side-effect
- functional changes:
  . owing to the internal overhaul, *2pcscmd commands now properly
    adapt to whether the output is directed to a terminal or not
    (including piping to another command) with regards to colorizing
    and text-wrapping for reasonable line lengths unless overridden
    with explicit switches
  . commands dealing with XML format (at some internal, not necessarily
    user-facing phase; covers all *2pcs* ones) are now proofed so as
    to preserve XML attribute ordering, leading to consistent results
    (desired way beyond internal test suite)
  . better diagnostics are now provided for some previously neglected
    error conditions
- feature highlight: Python 3 support (-> minor version bump so high)
  . beside Python 2.6, clufter project now strives to support 3.2+
    as well; brand new compatibility with the latter was casually
    and unit tested (the suite was also adapted to support both),
    but some corner cases may have not been hit yet, hence this
    class of bug reports is especially appreciated
  . for the clufter as a library users: under Python 3, "bytestring"
    protocol now, due to a strict dichotomy, indeed means "passed as
    bytes (or compatible) type" (if the contrary is observed, it is
    a bug to be fixed)
  . *2pcscmd commands now state also the Python interpreter in the
    initial informative comment block to ease the troubleshooting
    in the light of newly introduced dual Python 2/3 support
  . so-called external plugins are now implicitly searched also in
    additional, Python interpreter agnostic path intended primarily
    for plugins maintaining dual Python 2/3 compatibility (in the same
    vein as core clufter as of this release)
  . meta-spec prescribing a suggested RPM packaging in a generic way
    now encompasses the above point as well as it offers python3-clufter
    subpackage, and in turn factors some shared files to other new
    subpackages (clufter-bin, clufter-common)


v0.59.8
bug fix release (related to ccs_flatten helper)

- bug fixes:
  . some configuration flags (sort of meta attributes) of how to treat
    services or particular resources by rgmanager were previously
    dropped upon initial resource tree flattening phase (via ccs_flatten
    binary) whereas now, some are selectively restored on the output;
    in particular, omitting __failure_expire_time/__max_failures
    was a downright a bug as they were expected by the filters
    (ccsflat2cibprelude) further in the chain, the others are
    restored for possible future use: __enforce_timeouts and
    __restart_expire_time/__max_restarts
  . ccs_flatten binary received an update of function responsible
    for parsing time-related inputs so as to harden against values
    encoded as unexpectedly long strings or similar exotic cases
    [related rgmanager's bugs: bug 1036652, rhbz#1414139]
- internal enhancements:
  . ccs_flatten binary sources and snippets of related filters
    received some more maintenance care


v0.59.7
bug fix release (bash completion + shebangs, regard resource-agents version)

- bug fixes:
  . output of {ccs,pcs}2pcscmd commands could previously confuse users
    as to which shell interpreter the generated script is suitable for,
    especially when passing --noguidance (or --dry-run) as it would
    generate process substitution, which is a construct beyond POSIX,
    so now the expected (build-time configurable) shell selection
    (POSIX or bash-like) is emitted as a standard shebang ("#!" line)
    [bug 1381531]
  . bash completion would previously expose unexpected error when
    triggering the completion once '=' in a standard "-[-]option=value"
    syntax has been typed, whereas now this is fixed and suitable
    options offer also the switches with a trailing '=' for convenience
    [bug 1381565]
- functional changes:
  . with ccs2pcs* conversions, also the version of resource-agents
    package is now considered, currently with an impact on conversions
    of named rgmanager's resource agent targeting the version that
    has not received the native counterpart yet
- internal enhancements:
  . caching of the internal desparsified facts was fixed (broken
    from beginning, but became exposed only after a new style of
    access had been exercised)


v0.59.6
bug fix release sanitizing lvm RA conversion + pcs alert commands

- bug fixes:
  . ccs2pcs* commands would previously choke when converting
    configuration of lvm resource agent from rgmanager's suite
    into ocf:heartbeat:LVM counterpart anytime lv_name parameter
    was specified -- it turns out this behavior, based on incorrect
    assumptions (https://bugzilla.redhat.com/1286292), was an
    unfounded show-stopper (lv_name is commonly specified even
    if superfluously) so now only misguided configurations using
    different LVs (values of lv_name) pertaining given VG (vg_name)
    are rejected
  . *2pcscmd* commands, particularly and only with pcs commands
    configuring alert listeners, would previously disregard
    "accumulate CIB changes, then push" implicit behavior, shooting
    the changes at live cluster stack right away, and moreover the
    respective syntax did not reflect a slightadjustment it
    received during the final (though a bit belated) fine-tuning
    [resolves: bug 1381522]

Comment 19 errata-xmlrpc 2017-08-01 23:06:21 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:2198