+++ This bug was initially created as a clone of Bug #795435 +++ I see several occurrences of this error in /var/log/mongodb/mongodb.log. Thu Jun 30 16:32:19 [conn17] pulp_unittest.packages Btree::insert: key too large to index, skipping pulp_unittest.packages.$description_-1 861 { : "Oauth2 was originally forked from Leah Culver and Andy Smith's oauth.p..." } It's not specific to this one package, there are lots of references to packages.$description as being too large to index. I also see occurrences of repos.$packagegroups, probably because that also has a lengthy description. This has the potential to fill up the mongo log, making it very hard to see errors you might care about, as well as fill up disk space. We should look at excluding these keys from indexing. --- Additional comment from jslagle on 2012-03-19 16:58:11 EDT --- errata.description as well needs to be removed --- Additional comment from jslagle on 2012-03-20 12:55:53 EDT --- cherry picked to pulp_v1 1129a660f14a2df404d01bdfb60eb1b779c7959e
cherry picked to pulp rhui branch
additional commit 65d84916f23eb45e07da255cebf08db3efa8a424
QA: you should not see any of the errors from the description (or similar errors) in the mongo log. these errors previously showed up as a result of syncing a new repo, or syncing a repo when new packages or errata were added.
Created a custom rpm with a longish %descirption in .spec; uploaded to a custom repository and checked the logs... Version: RHEL-6.2-RHUI-2.0.3-20120322.0-Server-x86_64-DVD1.iso Verified: - uploading a custom package with a long %description section doesn't produce reported log ### Screen log ### checking the custom package is OK (tester's machine) [mkovacik@malina bhello]$ su Password: [root@malina bhello]# rpm -Uvh /home/mkovacik/rpmbuild/RPMS/x86_64/bhello-0.1-1.el6.x86_64.rpm Preparing... ########################################### [100%] 1:bhello ########################################### [100%] [root@malina bhello]# bhello Hello, World! [root@malina bhello]# rpm -qa bhello bhello-0.1-1.el6.x86_64 [root@malina bhello]# rpm -qi bhello Name : bhello Relocations: /usr/local Version : 0.1 Vendor: (none) Release : 1.el6 Build Date: Mon 26 Mar 2012 01:14:03 PM CEST Install Date: Mon 26 Mar 2012 01:14:49 PM CEST Build Host: localhost Group : Unspecified Source RPM: bhello-0.1-1.el6.src.rpm Size : 64 License: GPL Signature : (none) Summary : A Hello-World script in BASH Description : This is a loooooongish description in order to validate the Bugzilla https://bugzilla.redhat.com/show_bug.cgi?id=805616 So... Only the fool would take trouble to verify that his sentence was composed of ten a's, three b's, four c's, four d's, forty-six e's, sixteen f's, four g's, thirteen h's, fifteen i's, two k's, nine l's, four m's, twenty-five n's, twenty-four o's, five p's, sixteen r's, forty-one s's, thirty-seven t's, ten u's, eight v's, eight w's, four x's, eleven y's, twenty-seven commas, twenty-three apostrophes, seven hyphens and, last but not least, a single ! [1] - [1]: http://en.wikipedia.org/wiki/Pangram ### Upload on a RHUI instance rhui (repo) => u Select the repositories to upload the package into: - 1 : c_1 Enter value (1-1) to toggle selection, 'c' to confirm selections, or '?' for more commands: 1 Select the repositories to upload the package into: x 1 : c_1 Enter value (1-1) to toggle selection, 'c' to confirm selections, or '?' for more commands: c Enter the location of the packages to upload. If the location is an RPM, the file will be uploaded. If the location is a directory, all RPMs in that directory will be uploaded: ^Z [1]+ Stopped rhui-manager [root@dhcp-31-120 ~]# ls /root/bhello-0.1-1.el6.x86_64.rpm /root/bhello-0.1-1.el6.x86_64.rpm [root@dhcp-31-120 ~]# fg rhui-manager /root/bhello-0.1-1.el6.x86_64.rpm The following RPMs will be uploaded: bhello-0.1-1.el6.x86_64.rpm Proceed? (y/n) y Uploading /root/bhello-0.1-1.el6.x86_64.rpm... ------------------------------------------------------------------------------ rhui (repo) => < ------------------------------------------------------------------------------ -= Red Hat Update Infrastructure Management Tool =- -= Home =- r manage repositories c manage content delivery servers (CDS) s synchronization status and scheduling e create entitlement certificates and client configuration RPMs n manage Red Hat entitlement certificates u manage users i manage identity certificate Connected: dhcp-31-120.brq.redhat.com ------------------------------------------------------------------------------ rhui (home) => s ------------------------------------------------------------------------------ -= Red Hat Update Infrastructure Management Tool =- -= Synchronization Status =- dr display repo sync summary vr view the details of the last repository sync sr sync an individual repository immediately dc display CDS sync summary vc view the details of the last CDS sync sc sync an individual CDS immediately Connected: dhcp-31-120.brq.redhat.com ------------------------------------------------------------------------------ rhui (sync) => sc Select one or more CDS instances to schedule to be synchronized before its scheduled time. The sync will happen as soon as possible depending on other tasks that may be executing in the RHUI. Only CDS instances that are not currently synchronizing are displayed. Last Result Next Sync CDS ------------------------------------------ - 1 : Success 03-26-2012 14:11 dhcp-31-121.brq.redhat.com - 2 : Success 03-26-2012 14:11 dhcp-31-122.brq.redhat.com Enter value (1-2) to toggle selection, 'c' to confirm selections, or '?' for more commands: 1-2 Select one or more CDS instances to schedule to be synchronized before its scheduled time. The sync will happen as soon as possible depending on other tasks that may be executing in the RHUI. Only CDS instances that are not currently synchronizing are displayed. Last Result Next Sync CDS ------------------------------------------ x 1 : Success 03-26-2012 14:11 dhcp-31-121.brq.redhat.com x 2 : Success 03-26-2012 14:11 dhcp-31-122.brq.redhat.com Enter value (1-2) to toggle selection, 'c' to confirm selections, or '?' for more commands: c The following CDS instances will be scheduled for synchronization: dhcp-31-121.brq.redhat.com dhcp-31-122.brq.redhat.com Proceed? (y/n) y Scheduling sync for dhcp-31-121.brq.redhat.com... ... successfully scheduled for the next available timeslot. Scheduling sync for dhcp-31-122.brq.redhat.com... ... successfully scheduled for the next available timeslot. ------------------------------------------------------------------------------ rhui (sync) => dc ------------------------------------------------------------------------------ -= Red Hat Update Infrastructure Management Tool =- -= CDS Synchronization Status =- Last Refreshed: 13:31:22 (updated every 5 seconds, ctrl+c to exit) dhcp-31-121.brq.redhat.com .................................. [ UP ] dhcp-31-122.brq.redhat.com .................................. [ UP ] Next Sync Last Sync Last Result ------------------------------------------------------------------------------ dhcp-31-121.brq.redhat.com In Progress 03-26-2012 08:12 Success dhcp-31-122.brq.redhat.com In Progress 03-26-2012 08:13 Success Connected: dhcp-31-120.brq.redhat.com ------------------------------------------------------------------------------ ------------------------------------------------------------------------------ -= Red Hat Update Infrastructure Management Tool =- -= CDS Synchronization Status =- Last Refreshed: 13:32:28 (updated every 5 seconds, ctrl+c to exit) dhcp-31-121.brq.redhat.com .................................. [ UP ] dhcp-31-122.brq.redhat.com .................................. [ UP ] Next Sync Last Sync Last Result ------------------------------------------------------------------------------ dhcp-31-121.brq.redhat.com 03-26-2012 14:11 03-26-2012 13:32 Success dhcp-31-122.brq.redhat.com 03-26-2012 14:11 03-26-2012 13:32 Success Connected: dhcp-31-120.brq.redhat.com ------------------------------------------------------------------------------ ^Crhui (sync) => < ------------------------------------------------------------------------------ -= Red Hat Update Infrastructure Management Tool =- -= Home =- r manage repositories c manage content delivery servers (CDS) s synchronization status and scheduling e create entitlement certificates and client configuration RPMs n manage Red Hat entitlement certificates u manage users i manage identity certificate Connected: dhcp-31-120.brq.redhat.com ------------------------------------------------------------------------------ rhui (home) => ^Z [1]+ Stopped rhui-manager [root@dhcp-31-120 ~]# grep 'key\ too' /var/log/mongodb/mongodb.log [root@dhcp-31-120 ~]# echo $? 1 [root@dhcp-31-120 ~]# fg rhui-manager p Invalid menu item; type "?" for a list of available commands rhui (home) => r ------------------------------------------------------------------------------ -= Red Hat Update Infrastructure Management Tool =- -= Repository Management =- l list repositories currently managed by the RHUI i display detailed information on a repository a add a new Red Hat content repository c create a new custom repository d delete a repository from the RHUI u upload content to a custom repository p list packages in a repository Connected: dhcp-31-120.brq.redhat.com ------------------------------------------------------------------------------ rhui (repo) => p Choose a repository: 1 - Red Hat Enterprise Linux 6 Server (RPMs) from RHUI (6Server-i386) 2 - Red Hat Enterprise Linux 6 Server (RPMs) from RHUI (6Server-x86_64) 3 - Red Hat Update Infrastructure 2.0 (RPMs) (6Server-x86_64) 4 - c_1 Enter value (1-4) or 'b' to abort: 4 Enter the first few characters (case insensitive) of an RPM to filter the results (blank line for no filter): Only filtered results that contain less than 100 packages will have their contents displayed. Results with more than 100 packages will display a package count only. Packages: bhello-0.1-1.el6.x86_64.rpm zsh-4.3.10-4.1.el6.x86_64.rpm ------------------------------------------------------------------------------ rhui (repo) =>
Technical note added. If any revisions are required, please edit the "Technical Notes" field accordingly. All revisions will be proofread by the Engineering Content Services team. New Contents: C: When a new repository was synchronised, or new packages or errata were added, if the new packages had a very long description, a "key too large to index" error was recorded in the mongodb.log logfile. C: This had the potential to cause the logfile to become quite large, and possibly hide critical or important errors. F: Large description files were removed from indexing. R: Updating or installing packages with long description files no longer causes errors to be recorded in the logfile.
Technical note updated. If any revisions are required, please edit the "Technical Notes" field accordingly. All revisions will be proofread by the Engineering Content Services team. Diffed Contents: @@ -1,4 +1,7 @@ C: When a new repository was synchronised, or new packages or errata were added, if the new packages had a very long description, a "key too large to index" error was recorded in the mongodb.log logfile. C: This had the potential to cause the logfile to become quite large, and possibly hide critical or important errors. F: Large description files were removed from indexing. -R: Updating or installing packages with long description files no longer causes errors to be recorded in the logfile.+R: Updating or installing packages with long description files no longer causes errors to be recorded in the logfile. + +Final: +* When a new repository was synchronised, or new packages or errata were added, if the new packages had a very long description, a "key too large to index" error was recorded in the mongodb.log logfile. This had the potential to cause the logfile to become quite large, and possibly hide critical or important errors. Large description files were removed from indexing, and updating or installing packages with long description files no longer causes errors to be recorded in the logfile. (BZ#805616)
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. http://rhn.redhat.com/errata/RHBA-2012-0539.html