Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1092572

Summary:

Puppet install distributors on big puppet repos become unavailable during the time it is getting "recreated"

Product:

[Retired] Pulp

Reporter:

Petter Hassberg <petter.hassberg>

Component:

puppet-support

Assignee:

Barnaby Court <bcourt>

Status:

CLOSED CURRENTRELEASE

QA Contact:

Preethi Thomas <pthomas>

Severity:

unspecified

Docs Contact:

Priority:

medium

Version:

2.3

CC:

bcourt, jrooth, mhrivnak, petter.hassberg, pthomas, rbarlow, skarmark

Target Milestone:

---

Keywords:

Reopened, Triaged

Target Release:

2.5.0

Hardware:

Unspecified

OS:

Unspecified

Whiteboard:

Fixed In Version:

Doc Type:

Bug Fix

Doc Text:

Story Points:

---

Clone Of:

Environment:

Last Closed:

2014-11-24 21:33:46 UTC

Type:

Bug

Regression:

---

Mount Type:

---

Documentation:

---

CRM:

Verified Versions:

Category:

---

oVirt Team:

---

RHEL 7.3 requirements from Atomic Host:

Cloudforms Team:

---

Target Upstream Version:

Embargoed:

Attachments:

Description	Flags
Proposed patch	none

Description Petter Hassberg 2014-04-29 13:35:07 UTC

Description of problem:

When pushing a new puppet module into a puppet repository, it becomes unavailable from the point in time you put the module in the repo, until the repo have been extracted.

If you push a lot of new modules into that repo, foreach new file, it will be recreated. During this time, puppet runs to this repoitory will fail, and if connected to foreman, foreamn will suggest a removal of the previously added but not yet removed puppet modules.


Version-Release number of selected component (if applicable):

2.3.1

How reproducible:
always

Steps to Reproduce:
1. create puppet repo in pulp, and configure as "Puppet Master Distributor"
2. upload several ~ 3165 puppet modules to it.
3. try doing a puppet run while uploading new puppet modules.

Actual results:
it wont work until the Master distributor have done its extracting

Expected results:
it should work regardless if there are new modules coming in or not.

Additional info:

maybe it could extract to an mktmp dir or something, and then finally get renamed and replace old path?

Comment 1 Petter Hassberg 2014-04-30 08:47:49 UTC

OK clarification, its when publishing the repo the problem pops up, and not simply uploading to it.

Comment 2 Michael Hrivnak 2014-05-01 16:07:42 UTC

Can you please provide the specific pulp commands that you are running? We don't fully understand what you are attempting to do.

Comment 3 Michael Hrivnak 2014-05-21 16:01:46 UTC

Please feel free re-open this if you would like to add complete steps to reproduce.

Comment 4 Petter Hassberg 2014-05-22 07:01:01 UTC

OK
The call is for the repo to puppet_install_distributor:


  "/pulp/api/v2/repositories/%s/" % repo_id

This ~ is the payload:

         {
            "id": puppet_install_distributor
        }




What we do when adding a unit to a puppet repo, before being able to import this module into foreman, we do a publish of the puppet repo. 

When publish is coming to the step of the "install distributor" for that repo, not only does it take sometimes a very long time to perform the publish operation, but also since it starts by clearing the destination directory, if target path is used by a puppet master, that whole puppet environment is efficiently unusable until the publish operation is complete. 

        # clear out pre-existing contents
        try:
            self._clear_destination_directory(destination)
        except (IOError, OSError), e:
            return publish_conduit.build_failure_report(
                'failed to clear destination directory: %s' % str(e),
                self.detail_report.report)

        # actually publish
        for unit in units:
               [...]



With the behavior to start the publish operation with clearing the pre-existing contents first is problematic, since if foreman starts an import during the time between the destination dir is cleared and to the point in time where the whole repo is published, it will want to DELETE the classes not yet published on the node.

Also, if the dir is used by a puppet master, puppet will be unable to run intil the whole directory is republished.

hence my suggestion to:
1. only the units which have a newer timestamp than last publish date for repo gets expanded onto the dstination dir (the other ones left as is)

or 

2. destination in this code is a temporary destination onto where the units are published, then as a final step. the new dir gets moved to the "real" destination dir.

Comment 5 Petter Hassberg 2014-06-18 13:50:46 UTC

Created attachment 910020 [details]
Proposed patch

Proposed fix to create a temporary directory for pulp modules while running the install_distributor, so as to minimize the gap in where the module path is unusable.

Comment 7 Barnaby Court 2014-08-05 12:45:56 UTC

PR: https://github.com/pulp/pulp_puppet/pull/128

Comment 8 Randy Barlow 2014-08-13 19:59:09 UTC

This was fixed in the 2.4.1-0.1.alpha build.

Comment 9 Randy Barlow 2014-08-20 03:31:38 UTC

We are rebasing pulp-2.4.1, and so all of these bugs can no longer be considered MODIFIED. We will need to recheck them against the rebased build to make sure the cherry picking strategy did not introduce any regressions.

Comment 10 Chris Duryee 2014-09-30 13:52:28 UTC

build: 2.5.0-0.6.beta

Comment 11 Preethi Thomas 2014-10-17 02:47:23 UTC

verified
pulp-puppet-handlers-2.5.0-0.8.beta.el6.noarch
python-pulp-puppet-common-2.5.0-0.8.beta.el6.noarch
pulp-puppet-plugins-2.5.0-0.8.beta.el6.noarch
pulp-puppet-admin-extensions-2.5.0-0.8.beta.el6.noarch
pulp-puppet-consumer-extensions-2.5.0-0.8.beta.el6.noarch
[root@qe-blade-12 ~]# 
[root@qe-blade-12 ~]#