Bug 451235

Summary:	repository dependancy errors
Product:	[Fedora] Fedora	Reporter:	Rehan Khan <rehan.khan>
Component:	ocaml-json-static	Assignee:	Richard W.M. Jones <rjones>
Status:	CLOSED NEXTRELEASE	QA Contact:	Fedora Extras Quality Assurance <extras-qa>
Severity:	medium	Docs Contact:
Priority:	low
Version:	8	CC:	herrold, james.antill, lmacken
Target Milestone:	---
Target Release:	---
Hardware:	i386
OS:	Linux
URL:	http://fpaste.org/paste/2773
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2008-09-11 17:04:04 UTC	Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Rehan Khan 2008-06-13 15:37:11 UTC

From Bugzilla Helper:
User-Agent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; Maxthon; .NET CLR 1.1.4322; .NET CLR 2.0.50727; .NET CLR 3.0.04506.648; .NET CLR 3.5.21022; InfoPath.1; FDM)

Description of problem:
I'm not sure if this is the right place to post this bug so accept my apologies in advance.

Using a default install of smart package manager (meaning just the fedora and fedora-updates repos) one can run the command 'smart check --all' (This verifies all the dependancies of any package that is in the smart cache). This (right now) produces the following output: http://fpaste.org/paste/2773 (using the latest repository definition files). This output changes by varying amounts over the life of a release (at least since f6).

(Usually the kmod entries are expected because I think only the the release kernel and the two latest kernels are retained in the main fedora repos and the kmod rpm's are retained for those people who have not upgraded thier kernels.)

It was suggested on #fedora that I open a bug for each of the rpms in the list after installing the rpm and getting a failure log. I don't actually use any of the rpms currently showing up on the list so I don't think it would be any use. This would also be ignoring the real problem of rpms getting into the fedora-updates repository with out verifying that thier dependancies have been or will be met.

I would also like to point out that this is not a repository sync issue as it is not actually looking at the individual rpm files. It is just looking at the repository metadata. The metadata is saying package X needs package Y but package Y is not in the metadata.

Smart can pick this up as it uses a different method to depsolve than yum. I am not sure if yum has a way of doing this as well (at least I can't find a way). I guess it would have to be yum that should do this as rpm does not look at the repository metadata just the local installed rpm data. However if you try and install one of these packages yum will fail with a dependancy error.

Perhaps the createrepo tools can somehow also run the smart check --all command and verify the repository data or perhaps the build system should do this.

I hope I have explained this adequately, as it touches on a number of areas. However I think that solving this issue will make things a little easier on the end-user and remove some mysterious yum failures.


Version-Release number of selected component (if applicable):


How reproducible:
Always


Steps to Reproduce:
1. yum install smart
2. setenforce 0 (works around some possible policy issues in smart, running scripts from /tmp)
2. (as root) smart update
   (say yes to import each repository definition, only fedora and fedora-updates are actually enabled by default)
3. smart check --all



Actual Results:
http://fpaste.org/paste/2773

Expected Results:
Usually only the kmod entries should be there due to the nature of kmods and the fedora repository policy of removing older kernels.

Additional info:
The severity is dependant on which rpm has crept into the repo without correct dependancies. The issue arose on #fedora while trying to work out why a livna kmod rpm was failing a yum install as the kernel rpm it required did not exist in the fedora-updates repo (although I believe it might have been there the previous night - again not a repo sync issue - it was in the metadata previously or livna expected it to be there for some reason). However I have come accross this issue in one form or other since f6.

Comment 1 Rex Dieter 2008-06-13 16:29:12 UTC

createrepo isn't the right component, reassigning to smart (I can't reproduce 
with yum).

Comment 2 Rex Dieter 2008-06-13 16:30:00 UTC

and since fpaste.org isn't guaranteed to stick around forever, for posterity,

Checking relations...
All checked packages have correct relations.
Checking relations...
Unsatisfied dependency: bmpx-extension-0.40.13-8.fc8@i386 requires firefox = 
2.0.0.12
Unsatisfied dependency: html2ps-1.0-0.1.b5.fc8@noarch requires tex(dvips)
Unsatisfied dependency: html2ps-1.0-0.1.b5.fc8@noarch requires tex(tex)
Unsatisfied dependency: kmod-sysprof-1.0.8-1.2.6.23_0.142.rc3.git10.fc8@i686 
requires kernel-i686 = 2.6.23-0.142.rc3.git10.fc8
Unsatisfied dependency: 
kmod-sysprof-PAE-1.0.8-1.2.6.23_0.142.rc3.git10.fc8@i686 requires kernel-i686 
= 2.6.23-0.142.rc3.git10.fc8PAE
Unsatisfied dependency: kmod-sysprof-PAE-1.0.9-1.2.6.24.7_92.fc8@i686 requires 
kernel-i686 = 2.6.24.7-92.fc8PAE
Unsatisfied dependency: ocaml-json-static-0.9.6-4.fc8@i386 requires 
ocaml-json-wheel
Unsatisfied dependency: ocaml-json-static-0.9.6-4.fc8@i386 requires 
ocaml(Asttypes) = 40c6157304bd09f7f56a64264aa4d900
Unsatisfied dependency: ocaml-json-static-0.9.6-4.fc8@i386 requires 
ocaml(Parsetree) = b59a1a6771867acd824bde52e6512b5c
Unsatisfied dependency: ocaml-openin-20070524-3.fc8@i386 requires 
ocaml(Asttypes) = 40c6157304bd09f7f56a64264aa4d900
Unsatisfied dependency: ocaml-openin-20070524-3.fc8@i386 requires 
ocaml(Parsetree) = b59a1a6771867acd824bde52e6512b5c
Unsatisfied dependency: ocaml-pa-monad-1.2.0-4.fc8@i386 requires 
ocaml(Asttypes) = 40c6157304bd09f7f56a64264aa4d900
Unsatisfied dependency: ocaml-pa-monad-1.2.0-4.fc8@i386 requires 
ocaml(Parsetree) = b59a1a6771867acd824bde52e6512b5c
Unsatisfied dependency: ocaml-pgocaml-1.1-2.fc8@i386 requires ocaml(Parsetree) 
= b59a1a6771867acd824bde52e6512b5c
Unsatisfied dependency: sepostgresql-8.2.7-1.281.fc8@i386 requires 
postgresql-server = 8.2.7

Comment 3 Rehan Khan 2008-06-13 18:21:07 UTC

Rex, Sorry. I'm just using smart to highlight the potential problem. It's not a 
smart problem. Smart is able to check the dependancies of every package (by 
design) and so it has the check feature. Yum only checks dependancies just 
prior to install.

The problem is either one of two things:

a) rpm's are being posted to the fedora-updates without the metadata being 
written correctly. e.g ocaml(Parsetree) and ocaml(Asttypes) rpm's are in the 
repository directory but the metadata is not being populated when the repo is 
updated.

or b) rpm's are being passed through the build system without checking that all 
the dependancies are available in the fedora-updates repository (they might be 
in fedora-testing or rawhide but they are not getting moved to fedora-updates 
when those packages that need them are).

At the moment I don't know any other way of highlighting this issue other than 
by the output of smart check --all. Yum or rpm don't seem to be able to check 
the validity of all the packages in repositories without first trying to 
install them (yum). Perhaps there is a tool or option in the createrepo rpm but 
I don't know of it.

Something similar to smart check --all should be run against a fedora-updates 
repository *before* an update is released to the mirrors. If there are packages 
that have got through that have missing dependancies then the issue can be 
highlighted to the packager to fix, possibly in an automated fashion before the 
wider user community reports it while trying to update thier machine.

It's not a yum problem although yum could probably be used to highlight it and 
it's not an rpm problem because rpm does not know much about repositories. The 
closest match is createrepo as this might be a validation test that can be done 
after the repository has had createrepo run against it.

Comment 4 Rehan Khan 2008-06-13 18:29:35 UTC

To duplicate this with yum you have to try to install each package with the 
missing dependancy, for example for ocaml-openin-20070524:

[root@unknown000c29509b54 ~]# yum install ocaml-openin-20070524
Setting up Install Process
Parsing package install arguments
Resolving Dependencies
--> Running transaction check
---> Package ocaml-openin.i386 0:20070524-3.fc8 set to be updated
--> Processing Dependency: ocaml(Longident) = 46fb8aad4fb2c12a0f301b02d8139f07 
for package: ocaml-openin
--> Processing Dependency: ocaml(Queue) = caa3a209bfc63d23a30f573541a88fec for 
package: ocaml-openin
--> Processing Dependency: ocaml(Array) = aa8e3cd5824f9bb40b93fcd38d0c95b5 for 
package: ocaml-openin
--> Processing Dependency: ocaml(Nativeint) = e79cdc4d3575c2ed044955cb7ef49aca 
for package: ocaml-openin
--> Processing Dependency: ocaml(Arg) = 03e86a4154064ea900dc32c05f53e364 for 
package: ocaml-openin
--> Processing Dependency: ocaml(Set) = 7da14e671a035f12386ace3890018ef3 for 
package: ocaml-openin
--> Processing Dependency: ocaml(Warnings) = abcb1589615da86f20f475b0ed3bbabc 
for package: ocaml-openin
--> Processing Dependency: ocaml(Stream) = 21a833e12efd34ea0c87d8d9da959809 for 
package: ocaml-openin
--> Processing Dependency: ocaml(Int64) = f8f7e2e4c0667ead94596040b12e732d for 
package: ocaml-openin
--> Processing Dependency: ocaml(Camlp4_config) = 
cb716b4361f43326c6ad695c7a1bb5c0 for package: ocaml-openin
--> Processing Dependency: ocaml(Location) = eed044ad1204a633caad97bdd9048f8c 
for package: ocaml-openin
--> Processing Dependency: ocaml(Obj) = 5cfae708052c692ea39d23ed930fd64d for 
package: ocaml-openin
--> Processing Dependency: ocaml(String) = 2c162ab314b2f0a2cfd22d471b2e21ab for 
package: ocaml-openin
--> Processing Dependency: ocaml(runtime) = 3.10.0 for package: ocaml-openin
--> Processing Dependency: ocaml(Char) = e98bc9c9e918a84b3c1a5a122d42fac1 for 
package: ocaml-openin
--> Processing Dependency: ocaml(Asttypes) = 40c6157304bd09f7f56a64264aa4d900 
for package: ocaml-openin
--> Processing Dependency: ocaml(Lexing) = b1793496643444d3762dd42bebe2cfe3 for 
package: ocaml-openin
--> Processing Dependency: ocaml(Oo) = d1fd8eab2c1fb52f42b20d2c4fa47731 for 
package: ocaml-openin
--> Processing Dependency: ocaml(Int32) = 711321870c949bd3bbdd092d9bae92e4 for 
package: ocaml-openin
--> Processing Dependency: ocaml(Camlp4) = 2e045826779ef15857e5824a007fea98 for 
package: ocaml-openin
--> Processing Dependency: ocaml(CamlinternalOO) = 
6d0d5b328d6db88f403ca4393b4abd38 for package: ocaml-openin
--> Processing Dependency: ocaml(Hashtbl) = 083f2c94b44ff4e0b3220aaea6a783b4 
for package: ocaml-openin
--> Processing Dependency: ocaml(Parsetree) = b59a1a6771867acd824bde52e6512b5c 
for package: ocaml-openin
--> Processing Dependency: ocaml(Buffer) = f6cef633ea14963b84b79c4095c63dc3 for 
package: ocaml-openin
--> Processing Dependency: ocaml(Pervasives) = 8ba3d1faa24d659525c9025f41fd0c57 
for package: ocaml-openin
--> Processing Dependency: ocaml(Printf) = 5dbbf45a03b54e6dbfcf39178d0d6341 for 
package: ocaml-openin
--> Processing Dependency: ocaml(Format) = 35fe566f7a37d8991a5c822bd1463949 for 
package: ocaml-openin
--> Processing Dependency: ocaml(List) = da1ce9168f0408ff26158af757456948 for 
package: ocaml-openin
--> Running transaction check
---> Package ocaml-openin.i386 0:20070524-3.fc8 set to be updated
--> Processing Dependency: ocaml(Asttypes) = 40c6157304bd09f7f56a64264aa4d900 
for package: ocaml-openin
--> Processing Dependency: ocaml(Parsetree) = b59a1a6771867acd824bde52e6512b5c 
for package: ocaml-openin
---> Package ocaml-camlp4.i386 0:3.10.0-7.fc8 set to be updated
---> Package ocaml-runtime.i386 0:3.10.0-7.fc8 set to be updated
--> Finished Dependency Resolution
Error: Missing Dependency: ocaml(Asttypes) = 40c6157304bd09f7f56a64264aa4d900 
is needed by package ocaml-openin
Error: Missing Dependency: ocaml(Parsetree) = b59a1a6771867acd824bde52e6512b5c 
is needed by package ocaml-openin
[root@unknown000c29509b54 ~]#


As you can see yum throws up the same missing dependancies, so the dependancies 
are not in the metadata either because there was an error during metadata 
creation or the file required is not in the repository.

Comment 5 Rex Dieter 2008-06-13 18:34:55 UTC

ok, I stand corrected.  

Still, bugs should be filed against the pkgs with the broken deps, createrepo
isn't the place.

As for auto repoclosure, etc... that *is* done, and maintainers are already
automatically notified of problems.  (and further discussion on that is offtopic
for bugzilla).

Comment 6 Rehan Khan 2008-06-13 19:40:05 UTC

Not sure what autorepoclosure is but if it is being done and it is the thing 
which is supposed to pick these things up I can't understand how these packages 
get into a repository that *every* fedora user is going to update from 
especially where there is rawhide and fedora-testing for these sorts of 
problems to be sorted out.

As the problem has been highlighted and the reasons why createrepo might be a 
good place to start resolving it I am not sure where else I can report the 
problem. Perhaps you could suggest where I could try to bring this issue up if 
here is not the right place? I have tried #fedora and #fedora-devel.

thanks

Comment 7 Axel Thimm 2008-06-14 06:17:12 UTC

Since this is not a smart bug (smart, apt and the repoclosure checker will all
just diagnose what problems the repo has), I'm reassigning to the first package
that matched the list. The proper thing to do is to file bugs with each broken
package, maybe even cloning this report would make sense.

Comment 8 Richard W.M. Jones 2008-06-14 11:12:12 UTC

Acknowledged -- those deps on Asttypes & Parsetree are real bugs in
the packages.

Fix is simple: Add '-i Asttypes -i Parsetree' to the ocaml-find-requires.sh
command line.  To see a similar example, look at:

http://cvs.fedoraproject.org/viewcvs/devel/cduce/cduce.spec?rev=1.6&view=markup

Unfortunately I'm away for the next 2 weeks, so it'll have to wait unless
someone wants to make this fix.

Comment 9 Rehan Khan 2008-06-26 12:42:06 UTC

There is probably no interest in this on here but I have written a script that 
does what reposclosure does but it's *much* faster, a little more flexible,  
and doesn't lock up your machine (=consumes less resources).

It can be found here: http://tracker.labix.org/issue383

Comment 10 Rex Dieter 2008-06-26 13:03:49 UTC

I'm sure the fedora admin/infrastructure folks would be interested,
http://fedoraproject.org/wiki/Infrastructure

Comment 11 Seth Vidal 2008-06-26 13:07:35 UTC

repoclosure just run on my laptop unplugged running at 800mhz.

for fedora + updates it took 1m to run and used 56M of ram to run in at peak.
Not exactly consuming all your resources and not really locking up your machine
- but the hyperbole is appreciated.

I also noticed when I run smart on the same package set it shows no dependency
issues, but repoclosure reports about 10-12 issues.

Are you sure smart check is checking against ONLY the repositories and not
against the local rpmdb, too?

In terms of flexibility it doesn't seem like smart check can handle checking
other architectures at all.

Comment 12 Rehan Khan 2008-06-27 02:29:34 UTC

Crumbs, thats fast. I just ran repoclosure again with only fedora and updates 
enabled because I thought I had done something screwy the last time. It started 
at 1:59am and finished at 2:20am and this was on an amd64x2 3800 (2ghz), 512mb 
RAM. Perhaps I get slow performance because it is a vmware virtual machine 
using only the one 2ghz core ;) Still ~20x slower is still quite odd but not 
really hyperbole. It's what I have taken the time to check myself. Someone on 
#fedora also said that it wasn't fast on his machine (using a local mirrored 
repository). Any reason I might be getting this?

Regarding the other points, smart check has a number of arguments which can be 
seen using smart check --help. Just doing smart check only checks the local rpm 
database and usually should produce no errors as, in the case of yum, either 
errors out with broken rpms or if used with skip broken option drops the broken 
rpms from the install set and in the case of smart it will attempt to install 
the last version which had all it's dependancies met. Smart check --all will 
check all enabled repositories (generally including the local rpm db as it 
tries to enable this everytime it starts). Smart check --channels= will check 
just a the channels specified. Using smart check --all or smart check --
channels=fedora,updates produces a list equivalent to repoclosure. check --all 
works ok as long as one only installs rpm's from the official repos.

I tried to check arches different to the local machines arch but this did not 
work properly. Checking the smart code, it uses rpm.archscore to determine the 
machines arch so unless an option is added to smart check to not do this, this 
will not work.

Tbh, I have no bias over yum, yum-utils or smart. I just want to use the one 
which works best and that doesn't distract me from getting on with my day by 
throwing up errors. At the moment smart does this for me. yum is faster since 
F6 and beats smart during the install phase. So it's neither here nor there.

The point I wanted to make is that if there are the tools that can check this, 
how come rpms creep into the updates with broken dependancies. Broken 
dependancies are for updates-testing and rawhide. I don't think the quality 
standards should only be met for the release dvd. Perhaps there needs to be a 
staging repository that only allows updates to go through to the updates repo 
when all the dependancies are met (I thought that updates-testing was for 
this?). There must be a way to prevent this time consuming and frustrating 
problem happening before it actually impacts potentially thousnads of people?

Comment 13 Fedora Update System 2008-07-01 13:46:42 UTC

ocaml-pa-monad-1.2.0-5.fc8 has been submitted as an update for Fedora 8

Comment 14 Fedora Update System 2008-07-01 13:46:43 UTC

ocaml-pa-monad-1.2.0-5.fc9 has been submitted as an update for Fedora 9

Comment 15 Fedora Update System 2008-07-01 13:47:45 UTC

ocaml-pgocaml-1.1-3.fc9 has been submitted as an update for Fedora 9

Comment 16 Fedora Update System 2008-07-01 13:47:45 UTC

ocaml-pgocaml-1.1-3.fc8 has been submitted as an update for Fedora 8

Comment 17 Fedora Update System 2008-07-01 13:48:56 UTC

ocaml-openin-20070524-4.fc8 has been submitted as an update for Fedora 8

Comment 18 Fedora Update System 2008-07-01 13:49:29 UTC

ocaml-openin-20070524-4.fc9 has been submitted as an update for Fedora 9

Comment 19 Fedora Update System 2008-07-01 13:49:45 UTC

ocaml-json-static-0.9.6-5.fc8 has been submitted as an update for Fedora 8

Comment 20 Fedora Update System 2008-07-01 13:50:01 UTC

ocaml-json-static-0.9.6-5.fc9 has been submitted as an update for Fedora 9

Comment 21 Richard W.M. Jones 2008-07-01 13:50:47 UTC

I'm highly confused by this bug & in particular why it's assigned
to me, but anyhow I have at least fixed the problem mentioned in
comment 8 above.

Comment 22 Fedora Update System 2008-07-02 06:31:34 UTC

ocaml-pa-monad-1.2.0-5.fc9 has been pushed to the Fedora 9 testing repository.  If problems still persist, please make note of it in this bug report.
 If you want to test the update, you can install it with 
 su -c 'yum --enablerepo=updates-testing update ocaml-pa-monad'.  You can provide feedback for this update here: http://admin.fedoraproject.org/updates/F9/FEDORA-2008-5924

Comment 23 Fedora Update System 2008-07-30 20:00:17 UTC

ocaml-pgocaml-1.1-3.fc8 has been pushed to the Fedora 8 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 24 Fedora Update System 2008-07-30 20:03:26 UTC

ocaml-pgocaml-1.1-3.fc9 has been pushed to the Fedora 9 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 25 Fedora Update System 2008-07-30 20:04:48 UTC

ocaml-json-static-0.9.6-5.fc9 has been pushed to the Fedora 9 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 26 Fedora Update System 2008-07-30 20:06:08 UTC

ocaml-pa-monad-1.2.0-5.fc9 has been pushed to the Fedora 9 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 27 Fedora Update System 2008-07-30 20:07:46 UTC

ocaml-openin-20070524-4.fc8 has been pushed to the Fedora 8 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 28 Fedora Update System 2008-07-30 20:09:46 UTC

ocaml-openin-20070524-4.fc9 has been pushed to the Fedora 9 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 29 Fedora Update System 2008-07-30 20:10:29 UTC

ocaml-pa-monad-1.2.0-5.fc8 has been pushed to the Fedora 8 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 30 Fedora Update System 2008-09-11 17:03:59 UTC

ocaml-json-static-0.9.6-5.fc8 has been pushed to the Fedora 8 stable repository.  If problems still persist, please make note of it in this bug report.