What does your proxy look like? Does PackageKit ever download the correct packages? I'm confused that it downloads the update lists and not the packages.

What do you exactly mean by 'What does your proxy look like?' It's set in the Gnome 'Network Proxy' preferences as an http proxy. I don't know how PK accesses it, but yum gets it through the http_proxy environment variable as I didn't modify yum.conf.

Is there some way I can turn on traces during the download to see if PK tries every mirrors is the mirrorlist when one fails?

Can you try doing (as root):

NETWORK=TRUE /usr/share/PackageKit/helpers/yum/yumBackend.py update-system

and pasting the output. Thanks.

If I run that in a root shell, the download works fine. This must be related to the fact that gnome-session sets the proxy according to the preferences in the http_proxy env var. I checked that gpk-update-icon doesn't have this var in its environment.

If I unset http_proxy in the shell before launching the requested command, I get the same error as in the GUI:

┌──┤root@crx3051                                                     /home/fred 
└# NETWORK=TRUE /usr/share/PackageKit/helpers/yum/yumBackend.py update-system
Loaded plugins: refresh-packagekit
allow-cancel	false
allow-cancel	true
percentage	0
status	running
status	dep-resolve
no-percentage-updates
allow-cancel	true
percentage	10
status	download
error	package-download-failed	Errors were encountered while downloading packages.;initscripts-8.86.3-1.x86_64: failure: initscripts-8.86.3-1.x86_64.rpm from updates: [Errno 256] No more mirrors to try.

What does /system/http_proxy/host have as it's value in gconf editor? gnome-packagekit uses this for the proxy setting.

┌──┤fred@crx3051                                                     /home/fred 
└$ gconftool-2 -a /system/http_proxy
 use_http_proxy = true
 use_authentication = false
 host = localhost
 authentication_user = 
 ignore_hosts = [localhost,127.0.0.0/8]
 authentication_password = 
 use_same_proxy = true
 port = 3128



The proxy is indeed running on localhost, listening on the 3128 port.

Hmm. The mystery deepens. Can you try:

killall gpk-update-icon
gpk-update-icon --verbose

and then paste the debug log to this bug please. Thanks.

I killed gpk-update-icon and relaunched it as requested, but the icon doesn't show up again in the notification area. Thus I can't try to trigger an update from the icon.

The beginning of the log contains that:
TI:14:12:25	TH:0x7f1a3d99c680	FI:gpk-watch.c	FN:gpk_watch_set_proxies_ratelimit,1017
 - polling check
TI:14:12:25	TH:0x7f1a3d99c680	FI:gpk-watch.c	FN:gpk_watch_set_proxies_ratelimit,1022
 - set proxy_http=localhost:3128, proxy_ftp=localhost:3128


which I believe means that gpk-update-icon reads the correct proxy values (which is not totally unexpected as the update lists get downloaded properly, only packages fail). I guess the question is if gpk-status-icon passes the proxy information to all its helper processes (sorry if that's totally off; I don't know much about PackageKit design).

I'm attaching the full 'gpk-update-icon --verbose' log that I currently have it case it helps, and I'll leave it running. Maybe the icon shows up later.

Created attachment 341156 [details]
gpk-update-icon start log

Okay, thanks for that. That shows that the pk_control_set_proxy() function is being called with the correct data.

Could you try this:

sudo NETWORK=TRUE proxy_http=localhost:3128 proxy_ftp=localhost:3128
 /usr/share/PackageKit/helpers/yum/yumBackend.py update-system

I'm also slightly confused that you've got:

host = localhost
ignore_hosts = [localhost,127.0.0.0/8]

Doesn't that mean "don't use proxy when connecting to localhost?"

(In reply to comment #10)
> Okay, thanks for that. That shows that the pk_control_set_proxy() function is
> being called with the correct data.
> 
> Could you try this:
> 
> sudo NETWORK=TRUE proxy_http=localhost:3128 proxy_ftp=localhost:3128
>  /usr/share/PackageKit/helpers/yum/yumBackend.py update-system

Now that's maybe interesting: If I run your command line (replacing proxy_[http|ftp] by [http|ftp]_proxy) then the command just hangs. I didn't wait more than a few minutes though, it might have timeout-ed had I waited.

However, if I try the same command using [http|ftp]_proxy=http://localhost:3128/ instead of only localhost:3128 then it works! Maybe these variables need to be correct URLs for some tools to work?

> I'm also slightly confused that you've got:
> 
> host = localhost
> ignore_hosts = [localhost,127.0.0.0/8]
> 
> Doesn't that mean "don't use proxy when connecting to localhost?"  

Yes it means exactly that (Note that I think [localhost,127.0.0.0/8] is the default value for ignore_hosts). But it's totally coherent: you never use a proxy to access your proxy anyway...

(In reply to comment #11)
> Now that's maybe interesting: If I run your command line (replacing
> proxy_[http|ftp] by [http|ftp]_proxy) then the command just hangs.

My error apologies, I meant http_proxy and ftp_proxy.

Does a proxy have to be a valid URI -- is that standardised somewhere? In this case I'm not sure whether the daemon should error out and fail to accept localhost:3128 as a proxy address, or just to add "http://" as a prefix, and "/" as a suffix if it does not exist in the daemon (or yum backend).

If you add http://localhost:3128/ in the gnome proxy configuration rather than "localhost:3128", does this make PK work okay?

Richard.

(In reply to comment #12)
> If you add http://localhost:3128/ in the gnome proxy configuration rather than
> "localhost:3128", does this make PK work okay?

The proxy preference dialog will prevent me from entering http://... As soon as I type the ':' it empties the textbox. So I can't test that (Anyway I'd need to wait for an update to appear again as my system is up-to-date right now).

I have no idea about the 'standards' that would require that. Maybe poking at the gnome-session guys that did the work on propagating the proxy settings to the user's shell would help?

Fred.

Right, according to danw, the http_proxy is required to be a uri, not a string. I'll look at fixing this now.

commit 7af1423a989cac5af9cada6fbc5972f2b769c56b
Author: Richard Hughes <richard>
Date:   Fri Apr 24 15:42:55 2009 +0100

    bugfix: To fix spawned backends, http_proxy has to be a URI not a bare proxy. Fixes RH#491859

Can you please try http://koji.fedoraproject.org/koji/taskinfo?taskID=1319270 and report success/failure. Thanks.

I'm sorry, but I get this issue on a Fedora 10 box that I can't upgrade to F11 just to test that... I can of course try to upgrade just PackageKit, but that doesn't seem easy looking at the rpm errors:

      	libarchive.so.2 is needed by PackageKit-0.4.6-9.fc11.i586
	libnm_glib.so.0 is needed by PackageKit-0.4.6-9.fc11.i586
	python(abi) = 2.6 is needed by PackageKit-0.4.6-9.fc11.i586

I've rebuilt these for F10 as a scratch build: http://koji.fedoraproject.org/koji/taskinfo?taskID=1319775

When you're done testing, you probably want to uninstall then and re-install the PackageKit from f10-updates. I'm sure the codec helper will be non-functional with the different API versions.

Anyway, if you could test these packages for the proxy error, I would appreciate it. Thanks.

It seems the F10 build isn't functional... I get this when trying to launch a system update with the above PK packages installed:

Error Type: <type 'exceptions.AttributeError'>
Error Value: 'PackageKitYumBase' object has no attribute 'preconf'
  File : /usr/share/PackageKit/helpers/yum/yumBackend.py, line 2828, in <module>
    main()
  File : /usr/share/PackageKit/helpers/yum/yumBackend.py, line 2824, in main
    backend = PackageKitYumBackend('', lock=True)
  File : /usr/share/PackageKit/helpers/yum/yumBackend.py, line 196, in __init__
    self.yumbase = PackageKitYumBase(self)
  File : /usr/share/PackageKit/helpers/yum/yumBackend.py, line 2756, in __init__
    pc = self.preconf

Right, F10 has an older yum. Could you try the F11 pre-release please?

Reporter,  Now that F11 is out have you upgraded and is this still an issue?

-- 
Steven M. Parrish - KDE Triage Master
                  - PackageKit Triager
Fedora Bugzappers volunteer triage team
https://fedoraproject.org/wiki/BugZappers

Upgraded to Fedora 11, but it seems like it's worse than before. The update icon doesn't appear at all when there are updates available, and launching the update tool by hand I get the same error as reported at the beginning of the mail.

Richard, did you happen to commit your fix to a git topic branch that hasn't been released, or should I have that fix in my F11 PackageKit?

Fred.

7af1423a989cac5af9cada6fbc5972f2b769c56b is present in a F11 release, just make sure you've updated to PackageKit-0.4.8-2.fc11 already.

Using a proxy certainly works for me now, but if it doesn't work for you with 0.4.8, then we probably need to do some more debugging.

I have PackageKit-0.4.8-1.fc11.x86_64

That should work fine.

Can you try doing:

sudo NETWORK=TRUE http_proxy=http://localhost:3128/ ftp_proxy=ftp://localhost:3128/
 /usr/share/PackageKit/helpers/yum/yumBackend.py update-system

And then pasting the output? If that works, then we have to work out why the daemon is getting the values wrong. Thanks.

No updates available right now, so not much to test... Do you have any trick so I don't have to wait for the distribution pushing new updates each time I want to test that?

(In reply to comment #26)
> No updates available right now, so not much to test... Do you have any trick so
> I don't have to wait for the distribution pushing new updates each time I want
> to test that?  

sudo yum clean all
sudo NETWORK=TRUE http_proxy=http://localhost:3128/
ftp_proxy=ftp://localhost:3128/
 /usr/share/PackageKit/helpers/yum/yumBackend.py get-updates

(In reply to comment #27)
> sudo yum clean all
> sudo NETWORK=TRUE http_proxy=http://localhost:3128/
> ftp_proxy=ftp://localhost:3128/
>  /usr/share/PackageKit/helpers/yum/yumBackend.py get-updates  

This works.

However, launching gpk-update-viewer from the panel or the command line fails with: "Cannot retrieve repository metadata (repomd.xml) for repository: fedora. Please verify its path and try again"

It doesn't look quite the same issue as I reported initially. My bug report was about PK being able to fetch the updates list, but not the updates themselves. Maybe you fixed this issue, but it looks like it can't get the updates list now...

Okay, can you now do (as root):

killall packagekitd
/usr/sbin/packagekitd --backend=yum --verbose --disable-timer

then try getting the update list using gpk-update-viewer (as a normal user).

When this fails, please attach the packagekitd log. Thanks.

Created attachment 353974 [details]
packagekitd log

This is the packagekitd generated log when failing to get update list.

Looking at the beginning of the log, I've seen it's searching for ProxyHTTP and ProxyFTP in /etc/PackageKit/PackageKit.conf. If I set these 2 values, getting the update list works fine. However, shouldn't these values be gathered from the system preferences settings? (Note that even though the update list retrieval seems to work, I haven't tried to retrieve any package as there're currently no updates for my system)

Okay, I think I've worked out what is going wrong, and it's basically a design problem in PackageKit. First, some background:

ProxyHTTP and ProxyFTP will fix your problem, but that's not getting the setting from GConf, so although it'll get things working for you now, it's not a great fix for upstream.

The issues are threefold:

* The session proxy setting is only set when the gpk-update-icon process is started, or when the gconf keys change
* The packagekitd does not save the proxies when it times out, therefore, on next auto-start it's as if a proxy has never been set
* The proxy is systemwide, not tracked per-session

I suspect, that if you redo the /usr/sbin/packagekitd test in the previous comment, and then do "killall gpk-update-icon && gp-update-icon" then things will work correctly when you use gpk-update-viewer.

So, how to fix this properly. The key issue is that the SetProxy() method is on the main interface, and not the transaction interface. This means that if we have multiple sessions with different proxies, then the proxy used is the last one set, which isn't ideal at all.

So, we have two solutions:

1. Track the session in packagekitd when the proxy gets set, and use ColsoleKit to get the session when we get the transaction method, and choose the proxy based on that.

2. Break the API and move the SetProxy method to the transaction interface, and then set the proxy on each method call. This fixes things, but requires every application using PkClient to also query the proxy settings, which could quickly become tiresome, and also ties us to a GConf dep in an anotherwise GLib library.

1 has the disadvantage of calling into ConsoleKit for every method, which adds some delay.
2 has the disadvantage of breaking API, and requiring normal applications to track the proxy, or adding a GConf dep to the PkControl instance.

I think we should choose option 1 as this avoids breaking API and also doesn't require clients to track the session proxy.

For the short term, I've patched GpkWatch (the part of gpk-update-icon that sets the proxy) to re-set the proxy when packagekitd re-appears on the bus. That should fix things in the single user login case, but will suffer the over-write problem mentioned earlier. I'll add some tracking in the daemon for this case, although this will be 0.5.x only code.

Committed to gnome-packagekit:

commit 8156c44ed9700f69b807c73b5abd276ce4a6ef1b
Author: Richard Hughes <richard>
Date:   Thu Jul 16 13:24:30 2009 +0100

    Ensure we re-set the proxy when packagekitd re-appears on the bus

(In reply to comment #31)
> I suspect, that if you redo the /usr/sbin/packagekitd test in the previous
> comment, and then do "killall gpk-update-icon && gp-update-icon" then things
> will work correctly when you use gpk-update-viewer.

This indeed works, thus your analysis seems correct.

(In reply to comment #33)
> This indeed works, thus your analysis seems correct.  

Right, so I've spent yesterday evening writing up a plan of action, and this morning coding it up.

The implementation notes are here: http://cgit.freedesktop.org/packagekit/plain/docs/setting-the-proxy.txt and the security implication notes are here: http://cgit.freedesktop.org/packagekit/plain/docs/security.txt

This was coded in the following commits, onto git master:

commit f17797bbce189fd96b13dbe9476c6a06741dc766
Author: Richard Hughes <richard>
Date:   Fri Jul 17 12:21:20 2009 +0100

    Ensure we set the correct proxy for the transaction according to the uid and session. Fixes rh#491859

:100644 100644 b83030c... 8b8a51b... M	src/pk-engine.c
:100644 100644 2fad8fd... aacbeee... M	src/pk-transaction.c

commit 5b7350701a22272ad5a8a64c77f0bf1282581f3c
Author: Richard Hughes <richard>
Date:   Fri Jul 17 12:09:39 2009 +0100

    Only return true when pk_transaction_db_get_proxy() finds valid data

:100644 100644 3d5cbe7... a1e515e... M	src/pk-transaction-db.c

commit 2f742a809a4d8915a4e692b8eb920f18fd4f0258
Author: Richard Hughes <richard>
Date:   Fri Jul 17 11:09:37 2009 +0100

    Make saving the proxy data secure, and add some notes to the security document

:100644 100644 f5a3ef8... 4da7c73... M	docs/security.txt
:100644 100644 4dece35... 7406545... M	docs/setting-the-proxy.txt
:100644 100644 0605b90... 28ba20c... M	src/pk-dbus.c
:100644 100644 1f99997... 3d5cbe7... M	src/pk-transaction-db.c

commit e59ba218c6f2eec37f26f6cab5f60f00b2cdbcc5
Author: Richard Hughes <richard>
Date:   Fri Jul 17 10:51:11 2009 +0100

    Ensure the transaction database is not world readable

:100644 100644 ac88c9b... b6815cc... M	data/Makefile.am

commit b35824314e3cb479bd2d476e700d28dcc4ddbdf1
Author: Richard Hughes <richard>
Date:   Fri Jul 17 10:17:59 2009 +0100

    Add a document explaining how we map the proxy internally

:000000 100644 0000000... 4dece35... A	docs/setting-the-proxy.txt

commit 90b9108256e1ab557dc96e98ef7d89a657d7ef87
Author: Richard Hughes <richard>
Date:   Fri Jul 17 10:09:00 2009 +0100

    Add uid and session to proxy mapping in the database

:100644 100644 52e3ae2... 1f99997... M	src/pk-transaction-db.c
:100644 100644 8dfdbb8... 5f47cc7... M	src/pk-transaction-db.h

commit 3e5c4f2576aa3b536545f2292c682146ac5fabc2
Author: Richard Hughes <richard>
Date:   Fri Jul 17 10:00:13 2009 +0100

    Check the length of the proxy is less than 1024 bytes before saving it

:100644 100644 d5ab3ef... b83030c... M	src/pk-engine.c

commit 3006f391dc311447b68d267f5d315407618dd55e
Author: Richard Hughes <richard>
Date:   Fri Jul 17 09:59:04 2009 +0100

    Add the ConsoleKit functionality into PkDbus

:100644 100644 2a0f519... 0605b90... M	src/pk-dbus.c

Richard can this be CLOSED  CURRENTRELEASE?

-- 
Steven M. Parrish - KDE Triage Master
                  - PackageKit Triager
Fedora Bugzappers volunteer triage team
https://fedoraproject.org/wiki/BugZappers

(In reply to comment #35)
> Richard can this be CLOSED  CURRENTRELEASE?

Yup, thanks.