Bug 1594194 - Make vdsm-client schema cache part of the package
Summary: Make vdsm-client schema cache part of the package
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: vdsm
Classification: oVirt
Component: Tools
Version: 4.20.31
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ovirt-4.2.7
: ---
Assignee: Marcin Sobczyk
QA Contact: Pavol Brilla
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-06-22 11:17 UTC by Sandro Bonazzola
Modified: 2018-11-02 14:30 UTC (History)
7 users (show)

Fixed In Version: vdsm-4.20.41
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-11-02 14:30:18 UTC
oVirt Team: Infra
Embargoed:
rule-engine: ovirt-4.2+
rule-engine: blocker+
lsvaty: testing_ack+

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
oVirt gerrit 92908 0 master MERGED packaging: Fix requirements for vdsm-client 2020-07-31 22:00:42 UTC
oVirt gerrit 92939 0 master MERGED osupdater: whitelist vdsm commands for RPM %post to re-exec 2020-07-31 22:00:42 UTC
oVirt gerrit 93212 0 ovirt-4.2 MERGED osupdater: whitelist vdsm commands for RPM %post to re-exec 2020-07-31 22:00:41 UTC
oVirt gerrit 93215 0 master ABANDONED packaging: Add signal to node to execute post step of RPM installation 2020-07-31 22:00:41 UTC
oVirt gerrit 93218 0 ovirt-4.2 MERGED packaging: Fix requirements for vdsm-client 2020-07-31 22:00:41 UTC
oVirt gerrit 93219 0 ovirt-4.2 ABANDONED packaging: Add signal to node to execute post step of RPM installation 2020-07-31 22:00:41 UTC
oVirt gerrit 93903 0 master ABANDONED package: make pickled API schema part of vdsm-api package 2020-07-31 22:00:41 UTC
oVirt gerrit 94193 0 master MERGED package: a script converting yaml schema to pickle 2020-07-31 22:00:41 UTC
oVirt gerrit 94196 0 master MERGED package: make schema pickles part of 'vdsm-api' 2020-07-31 22:00:40 UTC
oVirt gerrit 94246 0 master MERGED automation: add python2-enum34 dependency 2020-07-31 22:00:40 UTC
oVirt gerrit 94248 0 master MERGED vdsmapi: introduce 'SchemaType' enum 2020-07-31 22:00:40 UTC
oVirt gerrit 94249 0 master MERGED vdsmapi: Schema implementation relies on pickles 2020-07-31 22:00:40 UTC
oVirt gerrit 94252 0 master MERGED package: remove obsolete schema definition yamls 2020-07-31 22:00:40 UTC
oVirt gerrit 94333 0 None MERGED build: Add missing python-enum34 rpm dependency 2020-07-31 22:00:40 UTC
oVirt gerrit 94347 0 None MERGED vdsmapi: Cleanup 'Schema' factories' APIs 2020-07-31 22:00:39 UTC
oVirt gerrit 94491 0 ovirt-4.2 MERGED package: a script converting yaml schema to pickle 2020-07-31 22:00:39 UTC
oVirt gerrit 94492 0 ovirt-4.2 MERGED package: make schema pickles part of 'vdsm-api' 2020-07-31 22:00:39 UTC
oVirt gerrit 94493 0 ovirt-4.2 MERGED automation: add python2-enum34 dependency 2020-07-31 22:00:39 UTC
oVirt gerrit 94494 0 ovirt-4.2 MERGED vdsmapi: introduce 'SchemaType' enum 2020-07-31 22:00:39 UTC
oVirt gerrit 94495 0 ovirt-4.2 MERGED vdsmapi: Schema implementation relies on pickles 2020-07-31 22:00:40 UTC
oVirt gerrit 94496 0 ovirt-4.2 MERGED package: remove obsolete schema definition yamls 2020-07-31 22:00:39 UTC
oVirt gerrit 94498 0 ovirt-4.2 MERGED build: Add missing python-enum34 rpm dependency 2020-07-31 22:00:39 UTC
oVirt gerrit 94499 0 ovirt-4.2 MERGED vdsmapi: Cleanup 'Schema' factories' APIs 2020-07-31 22:00:39 UTC
oVirt gerrit 94666 0 ovirt-4.2 ABANDONED build: Add missing python-enum34 build dependency 2020-07-31 22:00:39 UTC
oVirt gerrit 94667 0 master MERGED build: Add missing python-enum34 build dependency 2020-07-31 22:00:39 UTC
oVirt gerrit 94669 0 ovirt-4.2 MERGED build: Add missing python-enum34 build dependency 2020-07-31 22:00:39 UTC

Description Sandro Bonazzola 2018-06-22 11:17:36 UTC 
The vdsm-client sub package in vdsm-4.20.31 is defined as:


%package client
Summary:        VDSM client
BuildArch:      noarch
# A hack for unbreaking external packages that expect vdsm-cli.
Provides: %{name}-cli = %{version}-%{release}
Requires: %{name}-api = %{version}-%{release}
Requires: %{name}-yajsonrpc = %{version}-%{release}
Requires: %{name}-python = %{version}-%{release}
Obsoletes: %{name}-cli < %{version}-%{release}

%description client
Access vdsm API from the command line.

%post client
# Generating api schema caches each installation or update of
# vdsm client package in case the api schema files were changed since last
# installation.
/usr/bin/python -c 'from vdsm.api.vdsmapi import create_cache; create_cache()'


But missing:
Requires(post): %{name}-api = %{version}-%{release}
Requires(post): python


So there's no guarantee that the %post script will be executed correctly since vdsm-client can be installed before python and vdsm-api for example when building oVirt Node.

Marking as regression being vdsm-4.20.23 previously released in oVirt 4.2.3 not affected by this.
Comment 1 Red Hat Bugzilla Rules Engine 2018-06-22 11:17:42 UTC 
This bug report has Keywords: Regression or TestBlocker.
Since no regressions or test blockers are allowed between releases, it is also being identified as a blocker for this release. Please resolve ASAP.
Comment 2 Pavel Stehlik 2018-07-31 13:47:35 UTC 
QE just sanity check.
Comment 3 Raz Tamir 2018-08-16 12:54:33 UTC 
(In reply to Pavel Stehlik from comment #2)
> QE just sanity check.

Any update on this?
Comment 4 Raz Tamir 2018-08-18 20:10:36 UTC 
(In reply to Raz Tamir from comment #3)
> (In reply to Pavel Stehlik from comment #2)
> > QE just sanity check.
> 
> Any update on this?

Adding needinfo
Comment 5 Pavol Brilla 2018-08-20 07:05:14 UTC 
Requested part is included in last 4.20.37 build:

%package client
Summary:        VDSM client
BuildArch:      noarch
# A hack for unbreaking external packages that expect vdsm-cli.
Provides: %{name}-cli = %{version}-%{release}
Requires: %{name}-api = %{version}-%{release}
Requires: %{name}-yajsonrpc = %{version}-%{release}
Requires: %{name}-python = %{version}-%{release}
Obsoletes: %{name}-cli < %{version}-%{release}

# Required to generate schema cache in post step
Requires(post): %{name}-api = %{version}-%{release}
Requires(post): python

%description client
Access vdsm API from the command line.

%post client
# Generating api schema caches each installation or update of
# vdsm client package in case the api schema files were changed since last
# installation.
/usr/bin/python -c 'from vdsm.api.vdsmapi import create_cache; create_cache()'


As Requires(post) is not stored anywhere during/after install, this is sanity only test.
Comment 6 Sandro Bonazzola 2018-08-20 07:15:52 UTC 
Moving back to New: we wanted to get rid of cache generation in post
Comment 7 Martin Perina 2018-08-20 07:45:15 UTC 
(In reply to Sandro Bonazzola from comment #6)
> Moving back to New: we wanted to get rid of cache generation in post

Correct, currently we have fixed only RPM dependency issue, but it doesn't help the issue we have on node. Fix for the node issue should be delivered in 4.2.7
Comment 8 Pavol Brilla 2018-08-20 08:34:41 UTC 
sorry for confusion ...
Comment 9 Pavol Brilla 2018-10-10 07:24:16 UTC 
Can you provide verification steps for QE? this looks like more internal change7
Comment 11 Marcin Sobczyk 2018-10-10 07:44:58 UTC 
There are several aspects of the change that can be verified:
  * previously, vdsm-client created files in '/var/cache/vdsm/schema/' as a post-install step - it was an issue when building oVirt node images and this no longer happens
  * running 'time vdsm-client -h' shows a significant difference (~2x) without the contents of '/var/cache/vdsm/schema' (which could be pruned by OS at any time) for previous version
  * simply running 'vdsm-client -h' for new version verifies that the schema files are read correctly - you can check the exit code of the process or the inclusion of strings such as "StoragePool" or "StorageDomain" in the output
Comment 12 Pavol Brilla 2018-10-11 12:20:52 UTC 
before without existing /var/cache/vdsm/schema:

4.20.39.1-1.el7ev
real	0m1.316s
user	0m1.113s
sys	0m0.183s


after 4.20.42-1.el7ev :
real	0m0.858s
user	0m0.703s
sys	0m0.156s


verified
Comment 13 Sandro Bonazzola 2018-11-02 14:30:18 UTC 
This bugzilla is included in oVirt 4.2.7 release, published on November 2nd 2018.

Since the problem described in this bug report should be
resolved in oVirt 4.2.7 release, it has been closed with a resolution of CURRENT RELEASE.

If the solution does not work for you, please open a new bug report.
Note You need to log in before you can comment on or make changes to this bug.