Bug 435625

Summary: PackageKit should support installing packages from media
Product: [Fedora] Fedora Reporter: Jeremy Katz <katzj>
Component: PackageKitAssignee: Richard Hughes <richard>
Status: CLOSED NEXTRELEASE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: low Docs Contact:
Priority: low    
Version: 13CC: alsadi, bruce.butterfield, dgregor, farrellj, fedoraproject, gholms, hedayatv, john.brown009, marejde, mclasen, rhughes, richard, tcallawa, tla
Target Milestone: ---   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2010-07-08 05:12:44 EDT Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Description Jeremy Katz 2008-03-02 13:44:49 EST
We currently have support in pirut for getting (and preferring) the installation
of packages from media if you did an install from a media repository.  To do
this, we copy the media.repo file from the DVD over to the installed system as a
repo.  The repo has a mediaid which yum parses and uses in passing off to a
mediagrabber helper.  See pirut/__init__.py:pirutCDHandler.py() for an
implementation using this
Comment 1 Richard Hughes 2008-03-03 15:27:34 EST
What about if we enable the local dvd repo when we insert the dvd and disable it
when we remove it? This way people with the dvd in the drive would get the
packages installed without downloading them, and any newer packages can be
installed as normal?
Comment 2 Jeremy Katz 2008-03-03 22:22:54 EST
That doesn't work when people are using CDs or have multiple media-based
sources/repos
Comment 3 Richard Hughes 2008-03-04 06:23:55 EST
Valid point. The way I see this working is:

* user insterts disk (with special id file)
* pk-update-icon used gio to watch for the special volume
* pk-u-i adds a repo for /dev/cdrom0
* pk-u-i asks packagekitd to ServicePack(/dev/cdrom0)
* packagekit passes this to the backend action (yumBackend)
* pk-u-i schedules a GetUpdates
* The UpdateSystem does the action using the inserted disk as the media
* The user unmounts the disk, and pk-u-i gets this via gio
* pk-u-i removes the repo /dev/cdrom0

Other ideas welcome.
Comment 4 Jeremy Katz 2008-03-04 08:07:09 EST
That doesn't really cut it.  We're not talking about updates or service packs or
any such stuff.  

The thing is that when someone installs their system from CD/DVD, they expect to
later install _additional_ packages off of the CD/DVD assuming the packages are
available from there.  This includes actual applications that they choose from
the application installer as well as dependencies for a random app they download
and install a package from a website (like, say, the Amazon MP3 Downloader).

Hence, you really really really have to do the dance of asking the user "please
insert CD n" now because there is no way a priori for the user to know they're
going to need it, much less which one they're going to need.
Comment 5 Martin 2008-03-04 11:42:52 EST
Whichever solution is implemented, I think the use case "Installed a long time
ago, and no longer have the CD/DVD. Or my Internet connection is so fast that I
don't really care about the CD/DVD" is important too. I don't know how to best
accomodate both.

I've seen requiring the installation CD/DVD because it is enabled as a default
repo making it harder for users to install extra software than need be. And they
don't know how to disable it and get all the packages from the internet.

Just my 2€.
Comment 6 Richard Hughes 2008-03-27 17:03:39 EDT
I'm working on this now, but because it will need a new API method, is won't be
pulled into the 0.1.10 branch. Hopefully this will work correctly in 0.1.11.
Comment 7 Matthias Clasen 2008-04-01 02:02:16 EDT
Not F9 material at this point, pretty sure.
Comment 8 Bug Zapper 2008-05-14 01:43:33 EDT
Changing version to '9' as part of upcoming Fedora 9 GA.
More information and reason for this action is here:
http://fedoraproject.org/wiki/BugZappers/HouseKeeping
Comment 9 Richard Hughes 2008-05-23 02:36:58 EDT
*** Bug 447983 has been marked as a duplicate of this bug. ***
Comment 10 Muayyad Alsadi 2008-09-23 22:59:46 EDT
filed at upstream as bug #17749
https://bugs.freedesktop.org/show_bug.cgi?id=17749

please give it higher priority here

I hope it will be solved in F10
Comment 11 Richard Hughes 2008-10-09 12:32:37 EDT
We've got service pack stuff in 0.3.7: http://blogs.gnome.org/hughsie/2008/10/09/service-pack-gui/
Comment 12 Muayyad Alsadi 2008-10-09 13:11:01 EDT
I noticed that nice feature before, and I found it cool thanks.
but this is a different thing, so please consider adding the support for media package sources support
Comment 13 TK009 2009-03-03 20:21:04 EST
This bug has been triaged

-- 
Fedora Bugzappers volunteer triage team
https://fedoraproject.org/wiki/BugZappers
Comment 14 Bug Zapper 2009-06-09 19:39:32 EDT
This message is a reminder that Fedora 9 is nearing its end of life.
Approximately 30 (thirty) days from now Fedora will stop maintaining
and issuing updates for Fedora 9.  It is Fedora's policy to close all
bug reports from releases that are no longer maintained.  At that time
this bug will be closed as WONTFIX if it remains open with a Fedora 
'version' of '9'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version prior to Fedora 9's end of life.

Bug Reporter: Thank you for reporting this issue and we are sorry that 
we may not be able to fix it before Fedora 9 is end of life.  If you 
would still like to see this bug fixed and are able to reproduce it 
against a later version of Fedora please change the 'version' of this 
bug to the applicable version.  If you are unable to change the version, 
please add a comment here and someone will do it for you.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events.  Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

The process we are following is described here: 
http://fedoraproject.org/wiki/BugZappers/HouseKeeping
Comment 15 Muayyad Alsadi 2009-06-11 16:12:32 EDT
> are able to reproduce it  against a later version of Fedora please change the 'version' of this bug to the applicable version. 

yes, it's.

I still think that there is no trivial way in fedora (using any package manager, gui or cli, PK, yum, apt-get, smart, yumex, ...etc.) to make it install from CD/DVD media
Comment 16 Muayyad Alsadi 2009-08-01 07:20:13 EDT
Richard Hughes added support in the API but not in yum backend
I have made good progress, I completed the support in yum backend in 3 implementations
1. HAL (depreciated)
2. GIO (not working due to a bug in GIO because the backend is running as root)
3. a native call to the mount command

I made working rpms for fedora 11 using the 3rd way (links can be found https://fedoraproject.org/wiki/Features/MediaRepo)

some one must continue the work, either fix the bug in GIO or implement it in DeviceKit-disk (and make sure it has no problem to be called as root)
Comment 17 Hedayat Vatankhah 2009-08-20 06:07:25 EDT
Hi Muayyad, would you please report the bug in bugzilla? Maybe it'll get more attention there... :(
Comment 20 Hedayat Vatankhah 2010-02-26 03:25:19 EST
Ping! Any progress in this area? :(
Comment 21 Muayyad Alsadi 2010-02-26 09:08:41 EST
With the help of yumex developer I have finished it.
Now this feature is fully functional in yumex.

For this feature to work in PK, we need to be able to mount the media,
I have implemented this in all suggested ways, including DeviceKit.
But policykit does not allow the PK backend (which runs as root) to talk to devicekit and mount devices,

one of the two things must be done to solve the problem:
1. I need policykit people to add an exception rule for root to allow it to mount devices 
2. or change packagekit api so that backends (like yum) can plug code in the non-privileged part of it (desktop console user who got the rights to mount it) 

it's funny that the root is not sufficient for mounting devices, but I understand that.

both of the solutions are not on my hand.
Comment 22 Hedayat Vatankhah 2010-02-26 15:51:19 EST
Thanks for the progress.

At first, I decided to report a new bug for the mentioned problems but it is already a PackageKit bug. Or a new bug against policykit is needed?!
Comment 23 Muayyad Alsadi 2010-03-25 11:52:19 EDT
I think we should direct a question to Fedora Engineering Steering Committee (or any decision making body) on the way they want to solve this issue, if having an exception to allow root to DeviceKit policy to allow mounting devices is allowed/recommended way of solving this

or using the way which calls mount command or using devices that are already mounted (in other words the message would be: "please insert and mount Disk1" instead of "insert Disk1")
Comment 24 Bug Zapper 2010-04-27 07:55:42 EDT
This message is a reminder that Fedora 11 is nearing its end of life.
Approximately 30 (thirty) days from now Fedora will stop maintaining
and issuing updates for Fedora 11.  It is Fedora's policy to close all
bug reports from releases that are no longer maintained.  At that time
this bug will be closed as WONTFIX if it remains open with a Fedora 
'version' of '11'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version prior to Fedora 11's end of life.

Bug Reporter: Thank you for reporting this issue and we are sorry that 
we may not be able to fix it before Fedora 11 is end of life.  If you 
would still like to see this bug fixed and are able to reproduce it 
against a later version of Fedora please change the 'version' of this 
bug to the applicable version.  If you are unable to change the version, 
please add a comment here and someone will do it for you.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events.  Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

The process we are following is described here: 
http://fedoraproject.org/wiki/BugZappers/HouseKeeping
Comment 25 Hedayat Vatankhah 2010-04-30 08:07:44 EDT
(In reply to comment #23)
> I think we should direct a question to Fedora Engineering Steering Committee
> (or any decision making body) on the way they want to solve this issue, if
> having an exception to allow root to DeviceKit policy to allow mounting devices
> is allowed/recommended way of solving this
> 
> or using the way which calls mount command or using devices that are already
> mounted (in other words the message would be: "please insert and mount Disk1"
> instead of "insert Disk1")    

Considering the long time that this bug is still open, I'd prefer using the last approach (at list as the first official kind of support, may motivate a better solution sooner): calling the mount command directly, or using the already mounted devices (at least in gnome with the default settings, the media is automatically mounted when inserted, so "Please insert Disk1" should suffice).
Comment 26 Muayyad Alsadi 2010-04-30 10:42:57 EDT
as I said, I've made several implementation including the one which calls mount command using os.system
 
http://cgit.freedesktop.org/packagekit/tree/backends/yum/yumMediaManagerOS.py

or if you want DeviceKit with the already mounted devices (because root can't mount them) one can use

http://cgit.freedesktop.org/packagekit/tree/backends/yum/yumMediaManagerDeviceKit.py
the line 
if o.is_removable(): yield o
can be made
if o.is_removable() and o.is_mounted(): yield o

also we may need to catch error raised by
mnt = media.acquire()
in http://cgit.freedesktop.org/packagekit/tree/backends/yum/yumBackend.py

I say may because I don't remember does it raise something when root tries to mount or it just fails gracefully
Comment 27 Hedayat Vatankhah 2010-04-30 10:59:07 EDT
So, will be the first implementation (using the mount command directly) included in the official PackageKit release? What I was trying to say was to activate this approach in the Fedora's packagekit so it'll work with installation media out of the box (probably as an update to fedora 13 or at least for F14).
Comment 28 Muayyad Alsadi 2010-04-30 11:21:00 EDT
> will be the first implementation (using the mount command directly) included in the official PackageKit release?

all of my implementations are included but non of them is used (they are not active) waiting some decision either in PK or in PK (policykit or packagekit)

packagekit maintainer accepted my 4 implementations (yumMediaManagerDeviceKit.py
yumMediaManagerGIO.py yumMediaManagerHAL.py and yumMediaManagerOS.py) as a show case and he will activate one of them if it reached the quality he wants, IIRC he consider calling a system command like mount not the right thing to do.
so this won't be in F13 nor F14
Comment 29 Hedayat Vatankhah 2010-05-09 14:51:36 EDT
So, is it possible to go with the other solution? Assuming that the user mounts the media himself? As noted in Comment #25, usually the media is mounted automatically anyways.
Comment 30 Hedayat Vatankhah 2010-05-09 14:52:27 EDT
So, is it possible to go with the other solution (officially)? Assuming that the user mounts the media himself? As noted in Comment #25, usually the media is mounted automatically anyways.
Comment 31 Muayyad Alsadi 2010-05-09 16:17:43 EDT
it's a matter of decision, not a matter of code.
if Richard Hughes says I'm fine with yumMediaManagerDeviceKit.py that searches for the inserted media on currently mounted devices, then it's up to him.

I wait his comment on that.
Comment 32 Matthias Clasen 2010-05-10 18:41:00 EDT
There are various misleading comments here. PackageKit backends run as root, so they _are_ allowed to use udisks to mount media. PolicyKit grants privileges to uid 0 automatically. But doing so would be a layering violation. The policy decision whether to mount or not should come from the user session. And if you do it in the user session, you can use GIO to do it.
Comment 33 Richard Hughes 2010-05-11 06:17:05 EDT
First of all, this is a really hard problem to solve, and rushing into enabling code or patching daemons isn't the right way to go.

First, MediaManager. This is the right way to go if we are talking about pirut, where the yum instance is being run by the local user as uid 0 in the current active console. It blocks, and mounts devices (albeit as root) in the user session. PackageKit is not like this, where the daemon is being run in the system context, possibly by another user, and possibly unattended.

This last point is important, as we need to be able to update the system automatically, and this includes not prompting to "Insert the DVD" when the user is two hours drive away from the machine. It's also not the right thing for the daemon to mount the disks (as Matthias said, it's a laying violation) -- but it would be permissible for the daemon to emit a MediaChangeRequired() signal to the session so that the mounting could be done by the client process in the user (session) context. This would only be done if the transaction was not being run in a non-interactive mode. So the code we have in MediaManagerFoo that actually mounts the CDROM should probably be removed.

So, if we agree that the daemon should not be monitoring cdroms being opened and closed, and automatically mounting the contents (premise of least surprise) then MediaManager needs to be taught about the new interactions required by PackageKit. The session client (GpkTask for GNOME) needs to be taught about how to offer a prompt for a CDROM (SUSE contributed this code, and are using it, so it probably already works). Note, the yumBackend.py file already sends MediaChangeRequired() so there's not a lot of extra work to do there.

So, lets assume that the dameon just sends MediaChangeRequired() and the client puts up the UI, and then requeues the transaction when the user inserts a disk and presses OK. The session would have mounted the disk (but with a race?), but not in a well known mountpoint like /mnt, but instead something like "/media/Fedora 13 DVD". This means we have to move the "source" of the repo to this new location, perhaps after trying some heuristic to make sure we got the right disk.

Now, to do this sanely and properly, we might need to probably patch yum to do something other than just MediaManager.

What we need to do is:

1. rip the disabled mounting code out of the yum backend
2. detect the new mountpoint and adjust the local source location
3. do not prompt for media if the transaction is non-interactive
4. testing, testing and more testing
Comment 34 Muayyad Alsadi 2010-05-11 08:31:53 EDT
> PackageKit backends run as root, so they _are_ allowed to use udisks to mount media.

what is udisks ? I've made a DeviceKit backend if you mean that.
since the backend (which root) is not the console/session user, it's not allowed by PolicyKit.

> The policy decision whether to mount or not should come from the user session. And if you do it in the user session, you can use GIO to do it.    

there is no place in PackageKit I can plug code to the front-end running as the session user.
BTW: I have made a GIO implementation (but just like DeviceKit it's in the backend code which is running as root not the session user)


In Yumex-NG I have made the code that mounts the device in the front end (running as the session user) because yumex developer gave me "Add code to mount here"

> 2. detect the new mountpoint and adjust the local source location

the way we do it in yumex, is that back-end signals "we need foobar DVD", the front end (running as session user) reply it's on "/media/Fedora 11 DVD" (after mounting it, match it, ...etc.)

there is no local source location
there is no detection (the one who mounts should do the detection)
Comment 35 seth vidal 2010-05-11 12:26:38 EDT
yumex is using the callbacks in yum afaik.

There's no reason PK cannot do that.
Comment 36 Tim Lauridsen 2010-05-13 04:10:20 EDT
yumex backend (yumbase) is setting a self.mediagrapper in yumBase, it will be called we at media change is needed.

The yumex mediagrabber is sending a signal to the yumex frontend running as normal user, there is doing the user interaction / mounting and return a signal to the backend with the local mountpath there is returned back to yum.

The problem with PK in this case is that the PK backend not can be waiting for some action from the frontend and continue where it left of in the backend.
Comment 38 Hedayat Vatankhah 2010-07-08 09:13:40 EDT
Looks nice. Thank you. Hope to see the ability to ask the user to insert the required media when the desired package is in the media and user has decided that it prefers the media.

Good luck