Description of problem: Installing: couchdb x86_64 0.10.0-2.fc12 fedora 513 k Installing for dependencies: erlang x86_64 R13B-01.3.fc12 updates 40 M js x86_64 1.70-8.fc12 fedora 340 k libicu-devel x86_64 4.2.1-7.fc12 updates 616 k tcl x86_64 1:8.5.7-5.fc12 updates 1.9 M tk x86_64 1:8.5.7-3.fc12 updates 1.4 M unixODBC x86_64 2.2.14-9.fc12 updates 378 k tcl *and* erlang? Also, tk in a system daemon is weird, as is pulling in a -devel package. Version-Release number of selected component (if applicable): 0.10.0-2.fc12 How reproducible: 100%
[root@localhost ~]# yum install erlang Loaded plugins: presto Setting up Install Process Resolving Dependencies --> Running transaction check ---> Package erlang.i686 0:R13B-01.3.fc12 set to be updated --> Processing Dependency: tk for package: erlang-R13B-01.3.fc12.i686 --> Processing Dependency: libodbc.so.2 for package: erlang-R13B-01.3.fc12.i686 --> Running transaction check ---> Package tk.i686 1:8.5.7-3.fc12 set to be updated --> Processing Dependency: tcl = 1:8.5.7 for package: 1:tk-8.5.7-3.fc12.i686 --> Processing Dependency: libtcl8.5.so for package: 1:tk-8.5.7-3.fc12.i686 ---> Package unixODBC.i686 0:2.2.14-9.fc12 set to be updated --> Running transaction check ---> Package tcl.i686 1:8.5.7-5.fc12 set to be updated --> Finished Dependency Resolution Dependencies Resolved ============================================================================================================================================================= Package Arch Version Repository Size ============================================================================================================================================================= Installing: erlang i686 R13B-01.3.fc12 updates 40 M Installing for dependencies: tcl i686 1:8.5.7-5.fc12 updates 1.9 M tk i686 1:8.5.7-3.fc12 updates 1.4 M unixODBC i686 2.2.14-9.fc12 updates 381 k Transaction Summary ============================================================================================================================================================= Install 4 Package(s) Upgrade 0 Package(s) Total download size: 44 M Is this ok [y/N]: Couchdb only require erlang, js and libicu-devel, the others dependencies is picked up by erlang.
OK. Moving, then. It's just odd that a desktop microblog comment ends up brigning in erlang, tcl, tk, devel packages, etc.
Make that 'microblog application'.
There is another filed bug about dependencies in couchdb: https://bugzilla.redhat.com/show_bug.cgi?id=527052
Actually, as a future solution, we could consider splitting erlang into sub-packages, since some of these dependencies are not so frequently needed. Erlang package has long story, and it definitely needs love. I'll try to work out this task, but, please, don't expect quick solution.
Knowing nothing about erlang, I suppose the simplest split would be to split out the tk gui stuff much like tkinter is split out of python.
Starting a list of things to split out: * /usr/lib/erlang/lib/wx-* (wxWindows API, from R13B03 onwards) * /usr/lib/erlang/lib/gs-* (tcl/tk) * /usr/lib/erlang/lib/odbc-* (libodbc*) More splitout wishes, anyone? There also remains the question of how to name the packages: * Keep "erlang" as the contains-all package, and add a new "erlang-base" or "erlang-common" subpackage with just the basics? * Make "erlang" the new minimal Erlang package, and add a new "erlang-complete" package containing dependencies on all other subpackages? P.S.: I cannot commit any of this, as I do not have commit access to the Erlang package, so this is just collecting ideas for eventual implementation.
(In reply to comment #7) > Starting a list of things to split out: > > * /usr/lib/erlang/lib/wx-* (wxWindows API, from R13B03 onwards) > * /usr/lib/erlang/lib/gs-* (tcl/tk) > * /usr/lib/erlang/lib/odbc-* (libodbc*) > > More splitout wishes, anyone? * Java interface. * All *.erl sources, almost all *.c and *.java stuff may be excluded completely. > There also remains the question of how to name the packages: > > * Keep "erlang" as the contains-all package, and add a new > "erlang-base" or "erlang-common" subpackage with just the basics? > > * Make "erlang" the new minimal Erlang package, and add a > new "erlang-complete" package containing dependencies on all > other subpackages? I personally prefer just ripping off these bits and packaging them as separate sub-packages. E.g. erlang, erlang-wxWidgets, erlang-gs and so on. W/o erlang-base/common/complete and so on.
Hmm. Ripping out the sources... I am not sure whether that is really such a large amount of data that ripping out makes sense. A really interesting option could be to rip all OTP apps out into separate subpackages. E.g. from erlang-R13B03.tar.gz build erlang-app-jinterface-1.5.2 containing /usr/lib/erlang/lib/jinterface-1.5.2/ erlang-app-kernel-2.13.4 containing /usr/lib/erlang/lib/kernel-2.13.4 erlang-app-stdlib-1.16.4 containing /usr/lib/erlang/lib/stdlib-1.16.4 erlang-app-wx-0.98.4 containing /usr/lib/erlang/lib/wx-0.98.4 and so on. This would definitively solve the issue of too broad dependency. A convenience package erlang-all or erlang-devel depending on all few dozen app packages would help with easy installation of a "complete erlang release". This kind of packaging with proper dependencies, parallel installability of packages, and a few other things could also allow for quite neat packaging of OTP apps with their respective requirements, potentially even with updates of the OTP app's RPM package triggering proper OTP app updates for running processes. That would be really neat, but probably requires a little more thought. Splitting each of the per-app subpackages into -bin and -src subsubpackages would double the package count for little gain, IMHO. Putting all app's sources into a single common -app-sources subpackage with all sources could solve the package number issue, but would then requires installing ALL apps if you just want to have ONE app's source files (due to directory dependencies). So you would end up with -common (with the dir), -bin (ebin/ and (some files from) priv/) and -src, thereby tripling the number of subsubpackages. For these reasons, I am tending very much towards leaving the sources in the per-OTP-app subpackages. This discussion is starting getting to broad for a bugzilla entry. Should we move this to the generic Fedora devel list?
(In reply to comment #9) > Hmm. Ripping out the sources... I am not sure whether that is really such a > large amount of data that ripping out makes sense. No need to guess - let's count! [root@Workplace ~]# du -hs /usr/lib/erlang/ 124M /usr/lib/erlang/ [root@Workplace ~]# find /usr/lib/erlang/ -name "*.erl" -delete [root@Workplace ~]# du -hs /usr/lib/erlang/ 77M /usr/lib/erlang/ [root@Workplace ~]# Except few examples, these *.erl files are useless for running programs. They may be somewhat valuable for erlang developers, so we may put them into subpackage, named as erlang-src or erlang-docs
(In reply to comment #10) > Except few examples, these *.erl files are useless for running programs. They > may be somewhat valuable for erlang developers, so we may put them into > subpackage, named as erlang-src or erlang-docs Correct - they are all compeltely useless for running programs. Some examples as well as some original sources are useful for developers.
I'll take care of this issue
*** Bug 527052 has been marked as a duplicate of this bug. ***
This bug appears to have been reported against 'rawhide' during the Fedora 13 development cycle. Changing version to '13'. More information and reason for this action is here: http://fedoraproject.org/wiki/BugZappers/HouseKeeping
Here is a short status report. I almost finished stripping off some rarely used packages with excessive Requires. Here is a scratch build for F-12: http://koji.fedoraproject.org/koji/taskinfo?taskID=2058924 WARNING - this package is for testing ONLY, and the package's actual contents will be changed in Fedora's official package.
From a first glance it looks much better now! Workplace ~/work: du -hs /usr/lib/erlang/ 146M /usr/lib/erlang/ Workplace ~/work: rpm -U --test ~/Desktop/erlang-R13B-04.2.fc12.i686.rpm Workplace ~/work: sudo rpm -U ~/Desktop/erlang-R13B-04.2.fc12.i686.rpm Workplace ~/work: du -hs /usr/lib/erlang/ 70M /usr/lib/erlang/ Workplace ~/work: -75 Mbytes of disk space + no longer requires odbc and tk. Please, send your reports about regressions.
The current erlang packaging has: # remove unneeded *.erl sources find $RPM_BUILD_ROOT/%{_libdir}/erlang/lib -maxdepth 2 -type d -name *src -exec rm -rf {} \; Which leaves rather a lot still in the package: atropine:~% rpm -qlv erlang | grep erl$ | awk 'BEGIN { i = 0 } { i += $5 } END { print i }' 45456512
Also, there's rather a lot of C header files: atropine:~% rpm -qlv erlang | grep -c \\.h$ 77
erlang-R13B-04.5.fc12 has been submitted as an update for Fedora 12. http://admin.fedoraproject.org/updates/erlang-R13B-04.5.fc12
erlang-R13B-04.5.fc13 has been submitted as an update for Fedora 13. http://admin.fedoraproject.org/updates/erlang-R13B-04.5.fc13
Folks, first version of erlang, splitted into dozens of sub-packages pushed into repositories. Next, I'll try to rebuild all erlang-related packages (except rabbitmq-server and couchdb) with this new feature.
I have to admit this confuses me a little: 1. The rpm subpackages with the OTP apps are all called 'erlang-$name'. The rpm subpackage with the documentation is called 'erlang-doc', and any other rpm subpackages with something else but an OTP app (such as erlang-examples) will also be named just as if it was an OTP app called 'doc' or 'examples'. This a) looks confusing and b) is a naming conflict waiting to happen 2. All rpm subpackages with the OTP apps now have the version number from the main erlang package. Given that every OTP app has its own version number, and that OTP releases depend on OTP apps' releases, wouldn't it make sense to have e.g. the OTP apps wx-1.2.3 and wx-1.3.4 in parallel installable rpm subpackages with the version 1.2.3 and 1.3.4, respectively?
(In reply to comment #22) > 2. All rpm subpackages with the OTP apps now have the version number > from the main erlang package. Given that every OTP app has its own > version number, and that OTP releases depend on OTP apps' releases, > wouldn't it make sense to have e.g. the OTP apps wx-1.2.3 and wx-1.3.4 > in parallel installable rpm subpackages with the version 1.2.3 and 1.3.4, > respectively? That's quite interesting for me! Frankly speaking, I didn't think about this enhancement until you pointed me on it, but I already like this idea. I plan to finish with splitting of packages, rebuilding them with custom dependency generators and ensuring that everything is fully compatible with previous monolithic package, first, and after that we should discuss this feature (parallel Erlang libraries installation, exactly like the kernel packages do). I think we will need a package with common erlang directory layout: %dir %{_libdir}/erlang/ %dir %{_libdir}/erlang/bin/ %dir %{_libdir}/erlang/lib/ %dir %{_libdir}/erlang/man/ %dir %{_libdir}/erlang/man/man1/ %dir %{_libdir}/erlang/man/man3/ %dir %{_libdir}/erlang/man/man4/ %dir %{_libdir}/erlang/man/man6/ %dir %{_libdir}/erlang/man/man7/ %dir %{_libdir}/erlang/releases/ It should be built independently from main erlang package (separate review will be required).
erlang-R13B-04.9.fc13 has been submitted as an update for Fedora 13. http://admin.fedoraproject.org/updates/erlang-R13B-04.9.fc13
erlang-R13B-04.9.fc12 has been submitted as an update for Fedora 12. http://admin.fedoraproject.org/updates/erlang-R13B-04.9.fc12
erlang-R13B-04.9.fc13 has been pushed to the Fedora 13 testing repository. If problems still persist, please make note of it in this bug report. If you want to test the update, you can install it with su -c 'yum --enablerepo=updates-testing update erlang'. You can provide feedback for this update here: http://admin.fedoraproject.org/updates/erlang-R13B-04.9.fc13
erlang-R13B-04.9.fc12 has been pushed to the Fedora 12 testing repository. If problems still persist, please make note of it in this bug report. If you want to test the update, you can install it with su -c 'yum --enablerepo=updates-testing update erlang'. You can provide feedback for this update here: http://admin.fedoraproject.org/updates/erlang-R13B-04.9.fc12
One issue with this, for example: # rpm -qR erlang-megaco | wc -l 594 Large numbers of dependencies like this tend to make yum depsolving a bit slow.
Or, provides: # rpm -q --provides erlang-megaco | wc -l 10718
(In reply to comment #28) > One issue with this, for example: > > # rpm -qR erlang-megaco | wc -l > 594 > > Large numbers of dependencies like this tend to make yum depsolving a bit slow. > > Or, provides: > > # rpm -q --provides erlang-megaco | wc -l > 10718 Unfortunately we can't do much here right now. As a future solution (very-very distant future) we can build some kind of "registry" of erlang functions - which module provides which function, and use module names instead. Right now, I plan to concentrate on more fine graining of current packaging structure (for example some modules provide both cli and gui interface - such modules should be splitted too) and on further user experience improvements (more erlang-related features should work right of the box - for example, emacs)
(In reply to comment #30) > (In reply to comment #28) > > One issue with this, for example: > > > > # rpm -qR erlang-megaco | wc -l > > 594 > > > > Large numbers of dependencies like this tend to make yum depsolving a bit slow. > > > > Or, provides: > > > > # rpm -q --provides erlang-megaco | wc -l > > 10718 > > Unfortunately we can't do much here right now. As a future solution (very-very > distant future) we can build some kind of "registry" of erlang functions - > which module provides which function, and use module names instead. This sounds like the equivalent of having a library package Provide() each of its symbols, and a binary package have: Requires: libc/fgets Requires: libc/printf ... and so on. There's a reason we don't do that for other apps, or for kernel symbol versions, etc. - it's just way too much data to process.
the biggest problem right now - is that erlang* pkgs are creating 33% of all provides for the entire distribution. That means EVERY user - whether they are using erlang apps or not - has to download 1/3rd MORE data for the provides alone than they would have had to otherwise. If erlang issues an update just once this cycle - that number will be nearly 50%. Now, for any given yum transaction, when erlang is NOT involved in the transaction - the user will only feel the impact of having to download that much more data. If erlang* is in the transaction then yum is going to have to traverse all of the provides to make sure none of them have been dropped that were needed. That gets expensive at that size. Honestly, this is just not acceptable.
I don't see problem in downloading a little more data (we're in XXI century, aren't we) and I don't see problem in some more yum parsing too. In fact, I even dodn't realize some overhead on my (pretty ancient today) PowerPC G4. That's my opinion - just for the record. However, I appreciate your difficulty and would like to hear your suggestions on how to properly decrease list of provides/reqires. I can, as you already know, build list of exported functions in every erlang package, already installed, so I can create another addition to erlang-rpm-macros, where I could explicitly list all these "erlang(foo:bar/1) = erlang(packagename)" and grep && uniq them while building dependent packages. Unfortunately, and it's obvious, that I would need to rebuild this registry every time when new package is added into Fedora or existing package is upgraded. So far, I found only one way to do so - each package should store some king of erlang-<packagename>.manifest somewhere in the /usr/lib/rpm/* (or somewhere else). What do you, guys, think?
(In reply to comment #33) > I don't see problem in downloading a little more data (we're in XXI century, > aren't we) and I don't see problem in some more yum parsing too. In fact, I > even dodn't realize some overhead on my (pretty ancient today) PowerPC G4. > That's my opinion - just for the record. However, I appreciate your difficulty > and would like to hear your suggestions on how to properly decrease list of > provides/reqires. > A little more data? the primary metadata BEFORE erlang split and provides is 5M uncompressed the primary metadata AFTER the erlang split is 16M uncompressed. That's a big jump. > I can, as you already know, build list of exported functions in every erlang > package, already installed, so I can create another addition to > erlang-rpm-macros, where I could explicitly list all these "erlang(foo:bar/1) = > erlang(packagename)" and grep && uniq them while building dependent packages. > > Unfortunately, and it's obvious, that I would need to rebuild this registry > every time when new package is added into Fedora or existing package is > upgraded. So far, I found only one way to do so - each package should store > some king of erlang-<packagename>.manifest somewhere in the /usr/lib/rpm/* (or > somewhere else). > Why do you need a registry of every function? We don't have that for any other language. What makes this so needed for erlang? When I add functions for ssl into yum, for example, I know I need to add a dep on pyopenssl, so I do. I don't make yum export a complete list of all imports as requires - nor do I export a complete list of functions and classes as provides.
(In reply to comment #34) > (In reply to comment #33) > > I don't see problem in downloading a little more data (we're in XXI century, > > aren't we) and I don't see problem in some more yum parsing too. In fact, I > > even dodn't realize some overhead on my (pretty ancient today) PowerPC G4. > > That's my opinion - just for the record. However, I appreciate your difficulty > > and would like to hear your suggestions on how to properly decrease list of > > provides/reqires. > > > > A little more data? > > the primary metadata BEFORE erlang split and provides is 5M uncompressed > the primary metadata AFTER the erlang split is 16M uncompressed. > > That's a big jump. > > Now, some of that is other pkgs, too - but I isolated the provides and requires from erlang along and the number is 2.8MB of uncompressed text alone. There's more cost to stick it in a sqlitedb. so to put that in perspective -6% of our size for the entire metadata for f13Beta - is caused by 126 pkgs. so .0075% of the pkgs creates 6% of the size of the metadata.
I do not even see any technical advantage of those per-function provides and depends over just checking what OTP apps are required and in what version. What does depending on the the OTP app's NVR not give us? (For those not so familiar with Erlang: An OTP app is what you find in /usr/lib/erlang/lib/$APPNAME-$APPVERSION).
(In reply to comment #34) > Why do you need a registry of every function? We don't have that for any other > language. What makes this so needed for erlang? Unfortunately, erlang-related packages, are the only packages, those runtime dependencies are calculated properly. Other packages, just doesn't do this at all. > When I add functions for ssl into yum, for example, I know I need to add a dep > on pyopenssl, so I do. Well, it's not enough to properly build a list of dependencies for python. I would like to emphasize, that we should add similar automatic dependency generators for python too (and for java, and so on). For example (I'm sure, you're aware of such situation, but other readers possibly not): python-package1 requires python-package-2 >= %{somever}. What does that mean? Will python-package1 work witn python-package2-%{somever+1} or not? Nobody knows, because there may be some functions and classes removed from python-package2-%{somever+1}. What even more odd, is that you don't know whether or not your python-package1 will work even with python-package2-%{somever}, because python-package1 could import some methods and classes actually missing (or wrongly named, see recent breakage with twisted) from python-package2-%{somever}. What is even more sad here, is that you just said something like "I know, we don't calculate dependencies properly in all other cases, and we're are happy with that - even if some breakage occurs here and were, we save few megabytes of space". I think that there is nothing to be proud of here, and that situation should be fixed too (in the future, because properly parsing python dependencies seems to be a huge task). (In reply to comment #35) > Now, some of that is other pkgs, too - but I isolated the provides and requires > from erlang along and the number is 2.8MB of uncompressed text alone. [sorry, skipped] > so .0075% of the pkgs creates 6% of the size of the metadata. These numbers looks acceptable to me. (In reply to comment #36) > I do not even see any technical advantage of those per-function provides and > depends over just checking what OTP apps are required and in what version. What > does depending on the the OTP app's NVR not give us? > > (For those not so familiar with Erlang: An OTP app is what you find in > /usr/lib/erlang/lib/$APPNAME-$APPVERSION). Folks, I would like to add some more clue forthose of us, who don't use/write erlang applications. Library - means library (just as in other languages), while some erlang modules have the ability to be pre-loaded into Erlang VM just like system init daemons on our machines - these applications we call 'applications'. In fact, many erlang packages do require much more erlang modules than those applications, which are explicitly listed in the respective *.app file. What listed in *app.file is just a requirement for starting *applications* (after start, application may or may not require some additional libraries). As for libraries, we need to dig into *.ebin files for list of imported functions to build a list of requires. This is the only way to be *absolutely* sure, that every runtime requirement will be met (or you will need to explicitly dig through sources and create the list of packages manually). Other methods are just a tradeoffs between building time/speed and/or space savings and/or possible human-related packaging errors. No need to talk - let's look inside one particular typical erlang module - erlang-debugger (which is sometimes used for debuging, as its name suggests): rpm -q erlang-debugger --requires ... erlang(kernel) erlang(stdlib) erlang(gs) ... These three modules is the only explicit erlang applications, which should be pre-loaded before loading debugger. But while inspecting deeper, we could see that this module also has dependency on erlang-wx library (see numerous erlang(wxSomeModule:SomeFunction/Arity) dependencies. Actually, I'm sure that this approach with lists of functions (and not just a libraries) can be extended to other languages too. I'm sure that python applications (just for example) could have great benefit (more precise checking for consistency, and therefore less chance for sudden crash with backtrace after updates) for users, if we will require not only a package names, but also a list of imported classes and functions.
If we extrapolate the numbers out to every pkg in the distro - we'd have Provides and Requires metadata, ALONE - not including anything else - that would be roughly 360MB in size. Pretty sure that number is a non-starter. If we're going to radically change how we do auto prov and deps such that we are dramatically increasing the size or number of the provides/requires then we need to have a chitchat with all the pkging folks first.
(In reply to comment #38) > If we extrapolate the numbers out to every pkg in the distro - we'd have > Provides and Requires metadata, ALONE - not including anything else - that > would be roughly 360MB in size. > > Pretty sure that number is a non-starter. On the other hand - we just don't have proper dependency checker at all. I mean - on one hand we have savings of several mbytes of space, on the other - correct automatic dependency list building. > If we're going to radically change how we do auto prov and deps such that we > are dramatically increasing the size or number of the provides/requires then we > need to have a chitchat with all the pkging folks first. Yes, sure. But this is a major issue, and I 'm sure that many people already mentioned this. Returning to erlang situation - I created correct automatic dependency generator, and all of you, folks, are trying to convince me to disable it. This make me very sad. Of course, if FESCo explicitly denies me (and only in this case), then I will obey, but then we should close this ticket as WONTFIX and return to very beginning, because splitting of erlang packages w/o automatic dependency checking between them won't have any sense.
(In reply to comment #39) > (In reply to comment #38) > > If we extrapolate the numbers out to every pkg in the distro - we'd have > > Provides and Requires metadata, ALONE - not including anything else - that > > would be roughly 360MB in size. > > > > Pretty sure that number is a non-starter. > > On the other hand - we just don't have proper dependency checker at all. I mean This is not true. What we don't have is a tool to automatically generate a purely correct dep list. Once the deps are specified (whether manually or automatically), the dependency checker works just fine. Note that unless I totally misunderstand erlang, even your automatic checker isn't perfect; it lists function names but not types of the expected arguments, return values, or what return values are expected for given inputs. If foo changes from returning an integer to returning a string code will still break. If foo changes from concatenating two strings to comparing them for equality code will still break. > - on one hand we have savings of several mbytes of space, on the other - > correct automatic dependency list building. > You keep saying several mbytes, however, what we're really talking about is time. Time to download the metadata whenever an update is issued. Time to parse the metadata whenever yum is started. Time to regenerate the metadata when mash is run. > > If we're going to radically change how we do auto prov and deps such that we > > are dramatically increasing the size or number of the provides/requires then we > > need to have a chitchat with all the pkging folks first. > > Yes, sure. But this is a major issue, and I 'm sure that many people already > mentioned this. > This is not a major issue. Until now, packagers have simply specified these dependencies manually. > Returning to erlang situation - I created correct automatic dependency > generator, and all of you, folks, are trying to convince me to disable it. This > make me very sad. > Well -- there's a tradeoff here. The automatic dependency generator you propose has these beneficial properties: 1) Can tell when a package is broken because nothing provides a necessary function name. 2) Can tell when a package is not broken by an update to a package it depends on (as long as the function hasn't changed its expected input, output, or behaviour). weighed against those: 1) The metadata that all users must download anytime any package in the repo is updated grows which leads to increased time waiting. 2) The algorithm for these automatic deps still isn't perfect -- as noted, changes to what types of parameters and return values will break things as well as changes to what a function actually does to the inputs to generate the outputs. 4) If other packages follow the example of erlang, we'll quickly have metadata that is too big to be useful. Since there's a tradeoff, it may well be that the increase in dep solving time is not worth the time saved in having to specify Requires (remembering that Provides were done per package so there's no time saved there and BuildRequires still have to be specified manually so there's no time saved there). > Of course, if FESCo explicitly denies me (and only in this case), then I will > obey, but then we should close this ticket as WONTFIX and return to very > beginning, because splitting of erlang packages w/o automatic dependency > checking between them won't have any sense. This might eventually become an FPC issue due to being about packaging but I think you're right that it's something FESCo can look at first since it deals with tradeoffs between two different projects within Fedora -- packaging of erlang/automatic dep generators and yum/depsolving.
(In reply to comment #40) > > On the other hand - we just don't have proper dependency checker at all. I mean > > This is not true. What we don't have is a tool to automatically generate a > purely correct dep list. Yes, exactly what I meant. > Note that unless I > totally misunderstand erlang, even your automatic checker isn't perfect; it > lists function names but not types of the expected arguments, return values, or > what return values are expected for given inputs. Unfortunately, it's not possible at all (except few special cases), since Erlang is a dynamically typed (as Python, too). > > - on one hand we have savings of several mbytes of space, on the other - > > correct automatic dependency list building. > > > You keep saying several mbytes, however, what we're really talking about is > time. Whatever. If more cpu time will need to correctly resove dependencies, we should allow it. > This is not a major issue. Until now, packagers have simply specified these > dependencies manually. ...which led us to numerous API/ABI breakages in the past and will lead to many other issues in the future. > > Returning to erlang situation - I created correct automatic dependency > > generator, and all of you, folks, are trying to convince me to disable it. This > > make me very sad. > > > Well -- there's a tradeoff here. The automatic dependency generator you > propose has these beneficial properties: > 1) Can tell when a package is broken because nothing provides a necessary > function name. > 2) Can tell when a package is not broken by an update to a package it depends > on (as long as the function hasn't changed its expected input, output, or > behaviour). > > weighed against those: > 1) The metadata that all users must download anytime any package in the repo is > updated grows which leads to increased time waiting. Measured in seconds and several megabytes (right now ~ 3 mbytes and few seconds to d/l + fes additional seconds to resolve). > 2) The algorithm for these automatic deps still isn't perfect -- as noted, As I mentioned above this is not a valid objection due to Erlang architecture. > 4) If other packages follow the example of erlang, we'll quickly have metadata > that is too big to be useful. Yes, and that should be done. Right now we just cannot guarantee at all, that your whole system is consistent. I would like to say also, that * metadata isn't a major space consumer in our yum cache directories (my own /var/cache/yum is about 150-200 mbytes and there is also a copy in /var/tmp/yum-* which doubles this size). * new dependency generators won't be written very soon, so HDD space and bandwidth will be even more cheaper than now these days.
(In reply to comment #41) > (In reply to comment #40) > > Note that unless I > > totally misunderstand erlang, even your automatic checker isn't perfect; it > > lists function names but not types of the expected arguments, return values, or > > what return values are expected for given inputs. > > Unfortunately, it's not possible at all (except few special cases), since > Erlang is a dynamically typed (as Python, too). I think it is worth noting that given that any Erlang program can call functions with apply/3 or spawn/3, and those three parameters (module name, function name, parameter list) can be determined by the calling program at runtime using whatever input data and logic it wants, there will always be the need to do manual dependency checking. - Unless you want to hack up Erlang's code loader such that if a module is not found, it looks into a module->rpmpackage table and triggers installation of the required rpmpackage (similar to what the font people are doing or starting to do).
(In reply to comment #42) > I think it is worth noting that given that any Erlang program can call > functions with apply/3 or spawn/3, and those three parameters (module name, > function name, parameter list) can be determined by the calling program at > runtime using whatever input data and logic it wants, there will always be the > need to do manual dependency checking. These are very special cases, and I would like to say few words about them. In fact, * In these cases we frequently can't tell what specific module/function will be used anyway (looking through sources won't help us too). * almost every time, the invocation of these functions guarded by try...catch somewhere (or by "let it fail" methodology with subsequent restart of failed module), and that's a correct usage. If not, then it's an issue, which should be reported upstream. So I wouldn't consider these (and the rest of generating code in real-time) examples as a blocker.
Folks, here is a solution - I created a list of substitutions for each 'Provides: erlang(...)' and substituted and uniq-ed every "Requires: erlang(...)". After that I manually copy+pasted the result into spec-file and disabled my custom autoprov/autoreq generator for now. This greatly reduced the number of provides/requires, while still ensuring that all dependencies will be met. Although it is not perfect solution because manual intervention is still required for building dependency lists, but in fact Erlang will not be upgraded very often (I mean, that dependencies may change only during version upgrades, and very likely not between two minor rebuilds), so it's acceptable, I believe. I will provide new builds very soon. And yes, I still hope that sometimes I will enable back automatic generation of dependencies between Erlang packages.
erlang-R13B-04.10.fc12 has been submitted as an update for Fedora 12. http://admin.fedoraproject.org/updates/erlang-R13B-04.10.fc12
erlang-R13B-04.10.fc13 has been submitted as an update for Fedora 13. http://admin.fedoraproject.org/updates/erlang-R13B-04.10.fc13
Peter, Thanks - erlang-wx - which had the most provides - at 21157 now has 3 provides. that's much more manageable.
erlang-R13B-04.10.fc12 has been pushed to the Fedora 12 testing repository. If problems still persist, please make note of it in this bug report. If you want to test the update, you can install it with su -c 'yum --enablerepo=updates-testing update erlang'. You can provide feedback for this update here: http://admin.fedoraproject.org/updates/erlang-R13B-04.10.fc12
erlang-R13B-04.10.fc13 has been pushed to the Fedora 13 testing repository. If problems still persist, please make note of it in this bug report. If you want to test the update, you can install it with su -c 'yum --enablerepo=updates-testing update erlang'. You can provide feedback for this update here: http://admin.fedoraproject.org/updates/erlang-R13B-04.10.fc13
(In reply to comment #47) > Peter, > Thanks - erlang-wx - which had the most provides - at 21157 now has 3 > provides. > > that's much more manageable. I'm considering this change as a temporary measure, a workaround, and will wait for FESCo decision on whether to move further or gave up. If FESCo will ban automatic dependency generation due to their size (and they will ban, very likely, as I understood from the comments above), when I will not proceed in further dependency narrowing of erlang-related packages, because this should not be done manually.
erlang-R13B-04.10.fc13 has been pushed to the Fedora 13 stable repository. If problems still persist, please make note of it in this bug report.
erlang-R13B-04.10.fc12 has been pushed to the Fedora 12 stable repository. If problems still persist, please make note of it in this bug report.
Folks, first round of erlang fine graining ends now, fortunately. Now, I would like to focus on narrowing other packages dependency chain, and on automatic dependency providing/requiring (I will find another approach - which would be less accurate but also less resource-consuming). Also I'll plan to fix remaining erlang tickets. That's all now.