Bug 1004001

Another example is Oracle - the public repos for Oracle Linux include source RPMs in the same directory as the binary/noarch RPMs, e.g.

http://public-yum.oracle.com/repo/OracleLinux/OL5/latest/x86_64/

When I synced this repo, over half of the space consumed was from the source RPMs:

$ sudo du -Lks /var/lib/pulp/published/http/repos/ol5/x86_64/
7759976 /var/lib/pulp/published/http/repos/ol5/x86_64/
$ sudo du -Lks /var/lib/pulp/published/http/repos/ol5/x86_64/*.src.rpm | awk '{SUM+=$1} END{print SUM}'
3686144

The other use that I would like to see is for cloning local repos - i.e. "syncing" from one repo to another in order to create a "promote to production" process.

One model developed using Pulp v1 is described in a Usenix paper from 2011 [1].  This would enable "less risky" (i.e. the majority of) packages from an upstream distributor to be promoted automatically to the internal production repositories, but packages that require additional testing (e.g. kernel, or applications like mysql or httpd) can be filtered from the automatic sync.  That way, the majority of updates can be pushed automatically to clients in a timely manner (or pulled via a standard "yum update"), while reducing the risk of introducting unexpected issues.  This model enables target package sets to be managed in one place, in the repository itself, rather than through excludes on each individual client.  For example, you might have 3 repositories:

"Live" = repo synced daily from upstream distributor
"Unstable" = repo synced daily from "Live", excluding $risky_pkgs
"Stable" = repo synced daily from "Unstable", excluding $risky_pkgs

$risky_pkgs would be manually promoted from Live -> Unstable -> Stable, for example weekly.

Similarly, different teams might choose to set different policies (maybe they want all kernel updates as soon as they are available, but they want to make sure that Python gets tested with their custom app first), so their $risky_pkgs_teamA list would be different.  It enables all teams within an organization to set their own policies while still inheriting certain organization-wide policy (e.g. that packages must be at least a day old before being installed anywhere).

It is not clear to me why the features of "cloning" and "sync filters" were removed between Pulp v1 and v2.

[1] https://www.usenix.org/legacy/events/lisa11/tech/full_papers/Pierre.pdf

Based on the above use cases, I'd like to see sync filters for at least the following criteria:
 - content type (e.g. RPM, source RPM)
 - architecture (e.g. x86_64)
 - package name match (e.g. 'kernel*' or 'kmod*')
 - date package was added to the repo (e.g. "before 20131001" or "before -7days")

Thanks.

*** Bug 1157857 has been marked as a duplicate of this bug. ***

Can we make a point on this and decide how we want to implement this?
I'd like to see this in Pulp since in my case is a "blocking" feature

Should we follow the old v1 approach? (Create a filter, link a filter to a repo)

Moved to https://pulp.plan.io/issues/206

Since this issue was entered in Red Hat Bugzilla, the release flag has been
set to ? to ensure that it is properly evaluated for this release.

Since this issue was entered in Red Hat Bugzilla, the release flag has been
set to ? to ensure that it is properly evaluated for this release.

The Pulp upstream bug status is at ASSIGNED. Updating the external tracker on this bug.

The Pulp upstream bug status is at POST. Updating the external tracker on this bug.

The Pulp upstream bug status is at NEW. Updating the external tracker on this bug.