Bug 993644
| Summary: | Applying migration pulp_rpm.migrations.0007_inventoried_custom_metadata failed. | ||||||
|---|---|---|---|---|---|---|---|
| Product: | [Retired] Pulp | Reporter: | Tim Hughes <thughes> | ||||
| Component: | z_other | Assignee: | pulp-bugs | ||||
| Status: | CLOSED CURRENTRELEASE | QA Contact: | Preethi Thomas <pthomas> | ||||
| Severity: | medium | Docs Contact: | |||||
| Priority: | medium | ||||||
| Version: | 2.1.1 | CC: | cduryee, mhrivnak, skarmark, thughes | ||||
| Target Milestone: | --- | Keywords: | Triaged | ||||
| Target Release: | --- | ||||||
| Hardware: | x86_64 | ||||||
| OS: | Linux | ||||||
| Whiteboard: | |||||||
| Fixed In Version: | Doc Type: | Bug Fix | |||||
| Doc Text: | Story Points: | --- | |||||
| Clone Of: | Environment: | ||||||
| Last Closed: | 2014-11-07 23:03:33 UTC | Type: | Bug | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Embargoed: | |||||||
| Attachments: |
|
||||||
the file it is trying to chown is a symlink. This is what it looks like [root@ld4repo02 ~]# ll /var/lib/pulp/working/repos/centos-6_4-os-x86_64/importers/isos/x86_64/CentOS-6.4-x86_64-netinstall.iso lrwxrwxrwx 1 apache apache 122 Jun 20 08:57 /var/lib/pulp/working/repos/centos-6_4-os-x86_64/importers/isos/x86_64/CentOS-6.4-x86_64-netinstall.iso -> ../../../../../../../../../../content/distribution/ks-CentOS-6.4-x86_64/../../isos/x86_64/CentOS-6.4-x86_64-netinstall.iso A little more information. [root@ld4repo02 x86_64]# pwd /var/lib/pulp/working/repos/centos-6_4-os-x86_64/importers/isos/x86_64 [root@ld4repo02 x86_64]# ll total 4 lrwxrwxrwx 1 apache apache 122 Jun 20 08:57 CentOS-6.4-x86_64-netinstall.iso -> ../../../../../../../../../../content/distribution/ks-CentOS-6.4-x86_64/../../isos/x86_64/CentOS-6.4-x86_64-netinstall.iso [root@ld4repo02 x86_64]# ll ../../../../../../../../../../content/distribution/ks-CentOS-6.4-x86_64/../../isos/x86_64/CentOS-6.4-x86_64-netinstall.iso ls: cannot access ../../../../../../../../../../content/distribution/ks-CentOS-6.4-x86_64/../../isos/x86_64/CentOS-6.4-x86_64-netinstall.iso: No such file or directory If i remove several "../" from the start of where the symlink is pointing we can get to the correct file. [root@ld4repo02 x86_64]# ll ../../../../../../content/distribution/ks-CentOS-6.4-x86_64/../../isos/x86_64/CentOS-6.4-x86_64-netinstall.iso -rw-r--r-- 1 apache apache 241172480 Jun 3 11:24 ../../../../../../content/distribution/ks-CentOS-6.4-x86_64/../../isos/x86_64/CentOS-6.4-x86_64-netinstall.iso Going to try fixing the symlinks by hand and see where we get to. [root@ld4repo02 x86_64]# pwd /var/lib/pulp/working/repos/centos-6_4-os-x86_64/importers/isos/x86_64 [root@ld4repo02 x86_64]# rm CentOS-6.4-x86_64-netinstall.iso rm: remove symbolic link `CentOS-6.4-x86_64-netinstall.iso'? y [root@ld4repo02 x86_64]# ln -s ../../../../../../content/distribution/ks-CentOS-6.4-x86_64/../../isos/x86_64/CentOS-6.4-x86_64-netinstall.iso CentOS-6.4-x86_64-netinstall.iso [root@ld4repo02 x86_64]# file CentOS-6.4-x86_64-netinstall.iso CentOS-6.4-x86_64-netinstall.iso: symbolic link to `../../../../../../content/distribution/ks-CentOS-6.4-x86_64/../../isos/x86_64/CentOS-6.4-x86_64-netinstall.iso' [root@ld4repo02 x86_64]# file ../../../../../../content/distribution/ks-CentOS-6.4-x86_64/../../isos/x86_64/CentOS-6.4-x86_64-netinstall.iso ../../../../../../content/distribution/ks-CentOS-6.4-x86_64/../../isos/x86_64/CentOS-6.4-x86_64-netinstall.iso: ISO 9660 CD-ROM filesystem data 'CentOS ' (bootable) [root@ld4repo02 x86_64]# OK , changing the above symlinks allowed the migration to progress to the end. There seems to be something fishy going on. That migration should only ever apply to repositories that have the yum importer associated, but an ISO repo should have only the iso_importer. Repos can only have one importer at a time. Can you run the following command and give us the output for the repo that is experiencing this problem? $ pulp-admin iso repo list --details That will show us which importer is associated. I think this may be related to the centos repository metadata linking to content that is outside the base path of the repository. That strikes me as weird, and I'm not surprised it would cause problems. Have a look here: http://mirror.rackspace.com/CentOS/6.4/os/x86_64/.treeinfo Note the "boot.iso" line. Moving back to NEW as we are waiting on more info from Tim. Tim, can you provide more information for comment 4 and 5? The CentOS 6.6 treeinfo metadata appears to not link to files outside of the repo anymore: http://mirror.rackspace.com/CentOS/6.6/os/x86_64/.treeinfo I am marking this bug as CLOSED/CURRENTRELEASE since we have not had other reports of the same issue. If you are still encountering issues, feel free to re-open this bug. I don't think we need additional info anymore. |
Created attachment 783266 [details] rpm -qa|sort Description of problem: yum upgrade from working pulp-server-2.1.1-1.el6.noarch to pulp-server-2.1.3-1.el6.noarch then ran pulp-manage-db. This took forever as it is doing an individual chown on every file. Then failed with the error in the the Additional info section. The actual file I believe that it is looking for is at: [root@ld4repo02 ~]# ll /var/lib/pulp/content/isos/x86_64/CentOS-6.4-x86_64-netinstall.iso -rw-r--r-- 1 apache apache 241172480 Jun 3 11:24 /var/lib/pulp/content/isos/x86_64/CentOS-6.4-x86_64-netinstall.iso Version-Release number of selected component (if applicable): How reproducible: Steps to Reproduce: 1. 2. 3. Actual results: See error below Expected results: Clean upgrade and working pulp server at the end of the process Additional info: Serevr Details: Linux ld4repo02.ld4.lmax 2.6.32-358.11.1.el6.x86_64 #1 SMP Wed Jun 12 03:34:52 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux Filesystem 1K-blocks Used Available Use% Mounted on /dev/mapper/vg_os-lv_root 15449552 3900852 10763908 27% / tmpfs 1961468 0 1961468 0% /dev/shm /dev/vda1 516040 66880 422948 14% /boot /dev/mapper/vg.pulp-mongodb 51606140 13008048 35976652 27% /var/lib/mongodb /dev/mapper/vg.pulp-pulp 722486360 380351592 305434608 56% /var/lib/pulp Software Installed: See attached file Error: [root@repo02 ~]# pulp-manage-db Beginning database migrations. Migration package pulp.server.db.migrations is up to date at version 4 Migration package pulp_puppet.plugins.migrations is up to date at version 0 Applying pulp_rpm.migrations version 7 Applying migration pulp_rpm.migrations.0007_inventoried_custom_metadata failed. See log for details. 2013-08-05 17:08:20,047 db:CRITICAL: Applying migration pulp_rpm.migrations.0007_inventoried_custom_metadata failed. 2013-08-05 17:08:20,059 db:CRITICAL: [Errno 2] No such file or directory: '/var/lib/pulp/working/repos/centos-6_4-os-x86_64/importers/isos/x86_64/CentOS-6.4-x86_64-netinstall.iso' 2013-08-05 17:08:20,126 db:CRITICAL: Traceback (most recent call last): File "/usr/lib/python2.6/site-packages/pulp/server/db/manage.py", line 79, in migrate_database update_current_version=not options.test) File "/usr/lib/python2.6/site-packages/pulp/server/db/migrate/models.py", line 161, in apply_migration migration.migrate() File "/usr/lib/python2.6/site-packages/pulp_rpm/migrations/0007_inventoried_custom_metadata.py", line 43, in migrate inventory_custom_metadata() File "/usr/lib/python2.6/site-packages/pulp_rpm/migrations/0007_inventoried_custom_metadata.py", line 55, in inventory_custom_metadata migrate_repo(repo) File "/usr/lib/python2.6/site-packages/pulp_rpm/migrations/0007_inventoried_custom_metadata.py", line 100, in migrate_repo content_unit = create_content_unit(ftype_data, relative_path) File "/usr/lib/python2.6/site-packages/pulp_rpm/migrations/0007_inventoried_custom_metadata.py", line 128, in create_content_unit unit_data['_storage_path'] = get_content_storage_path(relative_path) File "/usr/lib/python2.6/site-packages/pulp_rpm/migrations/0007_inventoried_custom_metadata.py", line 143, in get_content_storage_path fix_owner(storage_dir) File "/usr/lib/python2.6/site-packages/pulp_rpm/migrations/0007_inventoried_custom_metadata.py", line 161, in fix_owner _recursive_fix_owner(start_path) File "/usr/lib/python2.6/site-packages/pulp_rpm/migrations/0007_inventoried_custom_metadata.py", line 159, in _recursive_fix_owner _recursive_fix_owner(os.path.join(path, c)) File "/usr/lib/python2.6/site-packages/pulp_rpm/migrations/0007_inventoried_custom_metadata.py", line 159, in _recursive_fix_owner _recursive_fix_owner(os.path.join(path, c)) File "/usr/lib/python2.6/site-packages/pulp_rpm/migrations/0007_inventoried_custom_metadata.py", line 159, in _recursive_fix_owner _recursive_fix_owner(os.path.join(path, c)) File "/usr/lib/python2.6/site-packages/pulp_rpm/migrations/0007_inventoried_custom_metadata.py", line 159, in _recursive_fix_owner _recursive_fix_owner(os.path.join(path, c)) File "/usr/lib/python2.6/site-packages/pulp_rpm/migrations/0007_inventoried_custom_metadata.py", line 159, in _recursive_fix_owner _recursive_fix_owner(os.path.join(path, c)) File "/usr/lib/python2.6/site-packages/pulp_rpm/migrations/0007_inventoried_custom_metadata.py", line 159, in _recursive_fix_owner _recursive_fix_owner(os.path.join(path, c)) File "/usr/lib/python2.6/site-packages/pulp_rpm/migrations/0007_inventoried_custom_metadata.py", line 159, in _recursive_fix_owner _recursive_fix_owner(os.path.join(path, c)) File "/usr/lib/python2.6/site-packages/pulp_rpm/migrations/0007_inventoried_custom_metadata.py", line 154, in _recursive_fix_owner os.chown(path, apache_uid, apache_gid) OSError: [Errno 2] No such file or directory: '/var/lib/pulp/working/repos/centos-6_4-os-x86_64/importers/isos/x86_64/CentOS-6.4-x86_64-netinstall.iso' Database migrations complete. Loading content types. Content types loaded.