Bug 855053

Thankfully, the hooks are very well defined. This is what would be required:

- RepoManager.update_unit_count() needs to take an additional parameter, content_type_id.
- That method needs to store per-type unit counts in a new subdocument where keys are content type ids. This is quite easy, and mongo's increment operator will auto-create the attribute and/or the subdocument if both don't exist.
- associate_unit_by_id, associate_all_by_ids, and unassociate_by_criteria (all on the RepoUnitAssociationManager) need to pass the content type ID to RepoManager.update_unit_count. This is easy, because each of those methods already knows the type.
- If we name the subdocument something besides "content_unit_count", there are a couple of commands where we would need to change this in the default filters.
- Update unit tests.
- optionally present a total content unit count, either by having the client calculate it on-the-fly, or by maintaining a "total" attribute in the subdocument. The latter would require two increment commands instead of one. 

Probably the most complicated part of this would be writing a migration to convert existing repos. The migration would have to load and inspect a tremendous portion of the data in the system. I suspect that would double the development work load of this task.

Since this task was to investigate adding the feature, I think it's done. Jay, can you clarify what should happen from here? Will you open a new issue to actually do the work?

I don't actually think the upgrade script will be all that hard. I think that, per repo, you could do a query to retrieve a list of unique type IDs for content in the repo, then another query per type to do the count for each. Shouldn't be too bad if you just have pymongo do counts.

Given that you're pretty booked this sprint, I'm going to just move this bug to next sprint and morph it into the implementation. Sounds like it's more than an hour or two, so we'll make a story for it.

https://github.com/pulp/pulp/pull/359

build: 2.1.0-0.20.beta

FYI when verifying, keep an eye out on upgraded installs. I'm not sure I can replicate the issue I saw and we did a second look at the code and it looks good, but keep it in mind:


<jdob> so if you have repos in 2.0.8 and then run pulp-manage-db for 2.1.0
<jdob> it'll split that apart: https://gist.github.com/jdob/5103244
<jdob> notice on line 10 there, it still has the old count
<jdob> that's supposed to be removed in the migration
<jdob> and it only shows up in repo search
<jdob> i'm thinking this is a one-off issue that i still have both
<jdob> but fyi, keep an eye out for it
<preethi> ok
<jdob> on second thought, I'll just dig up the RFE and include this note there


$ pulp-admin rpm repo search
+----------------------------------------------------------------------+
Repositories
+----------------------------------------------------------------------+
 
Id: f17
Display Name: f17
Description: None
Content Unit Count: 3584  # shouldn't be here
Content Unit Counts:
Distribution: 1
Package Category: 7
Package Group: 227
Rpm: 3349
Notes:

verified

[root@preethi-el6-pulp ~]# rpm -q pulp-server
pulp-server-2.1.0-0.23.beta.noarch
[root@preethi-el6-pulp ~]# 

Also tested on upgrade from 2.0.8 ->2.1

Id:                  zoo
Display Name:        zoo
Description:         None
Content Unit Counts: 
  Erratum:          4
  Package Category: 1
  Package Group:    2
  Rpm:              32
Notes:               
Importers:           
  Config:           
    Feed URL: http://repos.fedorapeople.org/repos/pulp/pulp/demo_repos/zoo/
  Id:               yum_importer
  Importer Type Id: yum_importer
  Last Sync:        2013-01-31T08:49:04-05:00
  Repo Id:          zoo
  Scheduled Syncs:  
Distributors:        
  Auto Publish:        True
  Config:              
    Http:         False
    Https:        True
    Relative URL: /repos/pulp/pulp/demo_repos/zoo/
  Distributor Type Id: yum_distributor
  Id:                  yum_distributor
  Last Publish:        2013-01-31T08:49:06-05:00
  Repo Id:             zoo
  Scheduled Publishes: 
  Auto Publish:        False
  Config:              
    Http:  False
    Https: True
  Distributor Type Id: export_distributor
  Id:                  export_distributor
  Last Publish:        None
  Repo Id:             zoo
  Scheduled Publishes: 


[root@preethi-el6-pulp ~]# pulp-admin -u admin -p admin  repo list
+----------------------------------------------------------------------+
                              Repositories
+----------------------------------------------------------------------+

Id:                  rh6-epel-live
Display Name:        rh6-epel-live
Description:         None
Content Unit Counts: 
  Erratum:          2023
  Package Category: 3
  Package Group:    208
  Rpm:              8336

Id:                  uploads
Display Name:        uploads
Description:         None
Content Unit Counts: 
  Rpm: 4

Id:                  copy-epel
Display Name:        copy-epel
Description:         None
Content Unit Counts: 
  Rpm: 8336

Id:                  rhel6_3
Display Name:        rhel6_3
Description:         None
Content Unit Counts: 

Id:                  test
Display Name:        test
Description:         None
Content Unit Counts: 

Id:                  zoo
Display Name:        zoo
Description:         None
Content Unit Counts: 
  Erratum:          4
  Package Category: 1
  Package Group:    2
  Rpm:              32


[root@preethi-el6-pulp ~]# 


[root@preethi-el6-pulp ~]# pulp-admin rpm repo search
+----------------------------------------------------------------------+
                              Repositories
+----------------------------------------------------------------------+

Id:                  rh6-epel-live
Display Name:        rh6-epel-live
Description:         None
Content Unit Counts: 
  Erratum:          2023
  Package Category: 3
  Package Group:    208
  Rpm:              8336
Notes:               

Id:                  uploads
Display Name:        uploads
Description:         None
Content Unit Counts: 
  Rpm: 4
Notes:               

Id:                  copy-epel
Display Name:        copy-epel
Description:         None
Content Unit Counts: 
  Rpm: 8336
Notes:               

Id:                  rhel6_3
Display Name:        rhel6_3
Description:         None
Content Unit Counts: 
Notes:               

Id:                  test
Display Name:        test
Description:         None
Content Unit Counts: 
Notes:               

Id:                  zoo
Display Name:        zoo
Description:         None
Content Unit Counts: 
  Erratum:          4
  Package Category: 1
  Package Group:    2
  Rpm:              32
Notes:               


[root@preethi-el6-pulp ~]#

Pulp 2.1 released 


http://www.pulpproject.org/2013/04/05/pulp-2-1-0-released/