Bug 1132663 - RFE: pulp-manage-db --noop or --check mode
Summary: RFE: pulp-manage-db --noop or --check mode
Keywords:
Status: CLOSED UPSTREAM
Alias: None
Product: Pulp
Classification: Retired
Component: docker-support
Version: 2.4.2
Hardware: Unspecified
OS: Unspecified
high
medium
Target Milestone: ---
: 2.6.0
Assignee: Jeremy Cline
QA Contact: Irina Gulina
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2014-08-21 19:13 UTC by Mark Lamourine
Modified: 2015-02-28 23:42 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2015-02-19 01:19:00 UTC


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Pulp Redmine 247 None None None Never

Description Mark Lamourine 2014-08-21 19:13:18 UTC
Description of problem:

As an operator I want to be able to determine if the database migrations are up to date without making any change to the database.

When running in kubernetes, I need to be able to probe and poll from the worker containers until the db is ready.  Without a non-destructive way to check the
db status I have to manually sequence and test the DB status before launching workers.

Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1. 
2.
3.

Actual results:


Expected results:


Additional info:

Comment 1 James (purpleidea) 2014-11-10 20:35:20 UTC
This is needed for puppet-pulp.

+1 ;)

Comment 2 Jeremy Cline 2014-11-17 15:23:40 UTC
https://github.com/pulp/pulp/pull/1307

When this is merged, this can be tested as follows:

1. Ensure the database isn't set up:
  a. 'mongo'
  b. 'use pulp_database'
  c. 'db.dropDatabase()'
2. Run 'pulp-manage-db --dry-run'. It should print all the migrations it would have applied.
3. echo $? should return 1.
4. Run 'pulp-manage-db' and make sure all the migrations are successful.
5. Run 'pulp-manage-db --dry-run' again. This time echo $? should return 0.

Comment 3 Chris Duryee 2014-12-23 20:52:58 UTC
fixed in pulp 2.6.0-0.2.beta

Comment 4 James (purpleidea) 2014-12-23 20:59:09 UTC
(In reply to Chris Duryee from comment #3)
> fixed in pulp 2.6.0-0.2.beta

Whattt, awesome.
It looks like from https://github.com/jeremycline/pulp/commit/fe3041bc3bb21a738494b73f982824a8edaf40f4
that it's --dry-run
Any idea if this is an expensive operation, or a quick one? < 3 sec (typically)?

Comment 5 Jeremy Cline 2014-12-23 23:18:46 UTC
(In reply to James (purpleidea) from comment #4)
> (In reply to Chris Duryee from comment #3)
> > fixed in pulp 2.6.0-0.2.beta
> 
> Whattt, awesome.
> It looks like from
> https://github.com/jeremycline/pulp/commit/
> fe3041bc3bb21a738494b73f982824a8edaf40f4
> that it's --dry-run
> Any idea if this is an expensive operation, or a quick one? < 3 sec
> (typically)?

I didn't test it with a large data set, but it just checks to see if the current applied migration version is smaller than the current available migration, so it should be very quick.

Comment 6 Irina Gulina 2015-01-06 15:07:24 UTC
> db.dropDatabase()
{ "dropped" : "pulp_database", "ok" : 1 }
> exit

>> sudo -u apache pulp-manage-db --dry-run

Mongo database for connection is version 2.4.6
Loading content types.
Loading type descriptors [docker.json, rpm_support.json, puppet.json, iso_support.json, nodes.json]
Parsing type descriptors
Validating type descriptor syntactic integrity
Validating type descriptor semantic integrity
Would have created or updated the following type definition: docker_image
Would have created or updated the following type definition: distribution
Would have created or updated the following type definition: drpm
Would have created or updated the following type definition: erratum
Would have created or updated the following type definition: package_group
Would have created or updated the following type definition: package_category
Would have created or updated the following type definition: package_environment
Would have created or updated the following type definition: rpm
Would have created or updated the following type definition: srpm
Would have created or updated the following type definition: yum_repo_metadata_file
Would have created or updated the following type definition: puppet_module
Would have created or updated the following type definition: iso
Would have created or updated the following type definition: repository
Would have created or updated the following type definition: node
Content types loaded.
Ensuring the admin role and user are in place.
Mongo database for connection is version 2.4.6
Would have created the admin role.
Would have created the default admin user.
Admin role and user are in place.
Beginning database migrations.
Applying pulp.server.db.migrations version 1
Would have applied migration to pulp.server.db.migrations version 1
Applying pulp.server.db.migrations version 2
....
Applying pulp_rpm.plugins.migrations version 18
Would have applied migration to pulp_rpm.plugins.migrations version 18


>>  echo $?
1

>> sudo -u apache pulp-manage-db
Mongo database for connection is version 2.4.6
Loading content types.
Loading type descriptors [docker.json, rpm_support.json, puppet.json, iso_support.json, nodes.json]
Parsing type descriptors
Validating type descriptor syntactic integrity
Validating type descriptor semantic integrity
Updating the database with types [docker_image, distribution, drpm, erratum, package_group, package_category, package_environment, rpm, srpm, yum_repo_metadata_file, puppet_module, iso, repository, node]
Content types loaded.
Ensuring the admin role and user are in place.
Mongo database for connection is version 2.4.6
Admin role and user are in place.
Beginning database migrations.
Applying pulp.server.db.migrations version 1
Migration to pulp.server.db.migrations version 1 complete.
Applying pulp.server.db.migrations version 2
...
Applying pulp_rpm.plugins.migrations version 16
Loaded plugin yum_distributor for types: rpm,srpm,drpm,erratum,package_group,package_category,distribution,yum_repo_metadata_file
Loaded plugin export_distributor for types: rpm,srpm,drpm,erratum,distribution,package_category,package_group
Loaded plugin iso_distributor for types: iso
Loaded plugin docker_distributor_web for types: docker_image
Loaded plugin docker_distributor_export for types: docker_image
Loaded plugin puppet_distributor for types: puppet_module
Loaded plugin puppet_file_distributor for types: puppet_module
Loaded plugin puppet_install_distributor for types: puppet_module
Loaded plugin nodes_http_distributor for types: node
Loaded plugin group_export_distributor for types: rpm,srpm,drpm,erratum,distribution,package_category,package_group
Loaded plugin yum_importer for types: distribution,drpm,erratum,package_group,package_category,rpm,srpm,yum_repo_metadata_file,package_environment
Loaded plugin iso_importer for types: iso
Loaded plugin docker_importer for types: docker_image
Loaded plugin nodes_http_importer for types: node,repository
Loaded plugin puppet_importer for types: puppet_module
Loaded plugin yum_profiler for types: rpm,erratum
Loaded plugin puppet_whole_repo_profiler for types: puppet_module
Loaded plugin node for types: node,repository
Loaded plugin yum for types: rpm
Loaded plugin rhui for types: rpm
Migration to pulp_rpm.plugins.migrations version 16 complete.
Applying pulp_rpm.plugins.migrations version 17
Migration to pulp_rpm.plugins.migrations version 17 complete.
Applying pulp_rpm.plugins.migrations version 18
Migration to pulp_rpm.plugins.migrations version 18 complete.
Database migrations complete.

>> sudo -u apache pulp-manage-db --dry-run
Mongo database for connection is version 2.4.6
Loading content types.
Loading type descriptors [docker.json, rpm_support.json, puppet.json, iso_support.json, nodes.json]
Parsing type descriptors
Validating type descriptor syntactic integrity
Validating type descriptor semantic integrity
Content types loaded.
Ensuring the admin role and user are in place.
Mongo database for connection is version 2.4.6
Admin role and user are in place.
Beginning database migrations.
Migration package pulp.server.db.migrations is up to date at version 12
Migration package pulp_puppet.plugins.migrations is up to date at version 2
Migration package pulp_rpm.plugins.migrations is up to date at version 18
Database migrations complete.

>> echo $?
0

Comment 7 Brian Bouterse 2015-02-19 01:19:00 UTC
Moved to https://pulp.plan.io/issues/247


Note You need to log in before you can comment on or make changes to this bug.