Description of problem: hal polls the cd drive every 2 seconds regardless of the usage of the system. 2 seconds is very frequent in terms of power savings (it prevents hardware from going to sleep) and also keeps the cpu more busy than needed. If no user is logged into to GUI, this 2 second polling easily can be 10 seconds without any user experience degredation at all.
Created attachment 135426 [details] Patch to make the hal storage poll happen at the start of every other second The patch in the attachement changes the hal-storage addon code slightly to not blindly sleep 2 seconds, but to sleep until exactly the start of the second that is two seconds away. This new behavior makes the wakeup and the poll coincide with other similar wakees (including the gnome clock but many more soon) so that the processor and the rest of the system don't wake up all the time, but generally only at the top of the second for all such events in that entire second at once. This patch is independent of the actual merit of the 2 second poll; it just optimizes the poll to take place at the best time within the second. The poll still happens every two seconds: (output from strace -tt to show the new behavior) 14:15:29.007623 close(4) = 0 14:15:29.008353 nanosleep({1, 991667000}, NULL) = 0 14:15:31.000171 open("/dev/hdb", O_RDONLY|O_NONBLOCK|O_EXCL) = 4 14:15:31.005705 ioctl(4, CDROM_DRIVE_STATUS, 0x7fffffff) = 1 14:15:31.007658 close(4) = 0 14:15:31.008389 nanosleep({1, 991626000}, NULL) = 0 14:15:33.000088 open("/dev/hdb", O_RDONLY|O_NONBLOCK|O_EXCL) = 4 14:15:33.005625 ioctl(4, CDROM_DRIVE_STATUS, 0x7fffffff) = 1 14:15:33.007593 close(4) = 0 14:15:33.008326 nanosleep({1, 991694000}, NULL) = 0 14:15:35.000151 open("/dev/hdb", O_RDONLY|O_NONBLOCK|O_EXCL) = 4 as the strace shows, the wakeups indeed happen within the first few hundred usecs of every 2nd second.
FYI, newer SATA supports "asynchronous notification". AN is a feature specifically designed for one purpose: to eliminate CD/DVD polling. AN will deliver a hardware interrupt to the kernel when userland should be notified. The kernel will need to be updated to support Async Notification, and hal must similarly be updated to poll /only/ for non-AN devices.
re comment 2: Sounds awesome. What is the user space interface going to look like? Would need an easy way to both a) determine whether the sata device supports AN; and b) some ioctl or pollable sysfs file to get the event without polling. Thanks.
Adding jgarzik since I asked a question in the previous comment 3. Thanks.
Two answers :) 1) We don't know yet, no userland interface planning has been done AFAIK. 2) Your suggestions towards a userland interface are good ones.
Also, I know from krh that some Firewire card readers, at least, are capable of this. Also, I know that some MMC/SD hosts can do this too. As such it would be nice to have a general abstraction in the kernel, e.g. a file /sys/block/sda/media_detection (better name is needed though) that gives 0 resp. 1 iff the drive supports async media change notification as well as being poll()'able. Obviously the file would be absent if the removable file is 0. Thanks for considering.
Actually, thinking some more, the kernel already provides notifications for hardware that is hotplug aware, via /sbin/hotplug. So we just need to standardize on a way to tell userspace not to poll.
Also, hal could get smarter about polling. Maybe you could check utmp or ask X about active users, as the bug summary suggests, and increase the polling time? And you could apply opposite logic as well: decrease the poll time if you know the CD was used "recently." Just thinking out loud.
That might be interesting too.. so the kernel would emit "add" / "remove" uevents for the child block devices appropriately. Sadly it won't work for optical drives as the kernel don't generate child block devices for sessions / partitions on optical discs. That I've always considered a bug, but I suspect there's a great deal of resistance because it might break user space. If this happened I'd be happy to update hal so at least GNOME and KDE would cope with such a change. (btw, /sbin/hotplug is long gone; modern distros, including Fedora, sets /proc/sys/kernel/hotplug to "" in early user space and instead rely on udevd to handle uevents over the netlink socket. To avoid the fork-bomb that was /sbin/hotplug.)
HAL definitely needs to get smarter about polling and the way we're going to solve this is through ConsoleKit, http://gitweb.freedesktop.org/?p=ConsoleKit.git;a=summary to learn about logged in users since utmp/btmp/wtmp isn't reliable at all. This will hopefully happen for F7 as we want the same infrastructure for fast-user-switching. Other items on the HAL TODO list includes - Provide mechanism for tuning the readahead size. For example, a DVD player application (such as GNOME's Totem) might want to read ahead several hundreds of megabyte so the drive only needs to spin up every e.g. 20 minutes while watching a DVD movie. This enables the kernel driver to put the drive in a low-power mode to get substantially better battery life. - Provide mechanism for making an application inhibit polling of a drive. This is useful combined with the TODO right above. It will probably mean that the eject button on the drive is rendered useless but that is justified by the DVD player application displays a soft button for ejecting the media. and I bet the readahead item for DVD watching will require some kernel assistance to pin the readahead buffers so they're not lost.
And comment 10 raises the question of whether SATA optical drives can deliver eject button notifications async too? Today we catch these in HAL through polling, deliver them async to the desktop and the desktop then unmounts and ejects the media.
For the record, HAL is now smarter about things (thanks to ConsoleKit) and reduces polling when no users are logged in / sessions are idle to every 16th second instead of every 2nd. http://gitweb.freedesktop.org/?p=hal.git;a=commit;h=d11a896c7cf1edd2d1d1e46647abdbdc53651224 Something like what the patch in comment 1 does is probably useful to synchronize all wakeups to happen at the top of every Nth seconds.
Hi David - I implemented a mechanism to notify HAL if media is inserted/ejected for SATA ATAPI drives - can you check out the user interface and comment on whether this looks like something that you can use to help reduce HAL polling? http://marc.info/?l=linux-kernel&m=117512562919961&w=2 Thanks, Kristen
Hi Kristen, > This support is exposed to user space via a file in sysfs (/sys/block/sr*) > called "media_change_events". If the drive supports AN, this file will > read 1, otherwise 0. User space can disable polling for new media if this > file reads 1. When new media is inserted into the ATAPI drive, the ahci > driver will send a KOBJ_CHANGE event. I think this sounds good; I suppose the KOBJ_CHANGE event will be on the /block/sr* object. An alternative interface would be to make the file media_change_events poll()'able; this would allow easier distinction of events in the event that KOBJ_CHANGE is used for other things in the future; for example eject button notification. (btw, is eject button notification covered by SATA 2.5 and AHCI 1.1?)
So, the AHCI spec does say "This feature allows an ATAPI device to send a signal to the host when media is inserted or removed and avoids polling the device for media changes", and in my personal experience, disk manufacturers are sending AN for both media insert and removal. So in the code I sent I did not put the event associated with /block/sr*, but I am working on a revision that will do that. the way I envisioned the media_change_events file used would be at init time, HAL would check this file to see if the disk supported AN. If media_change_events == 1, then hal would disable polling on that drive. When a KOBJ_CHANGE event is received on that particular drive, HAL can check for media_changed the way it always has to see if media has changed. I can probably change the call to use kobject_uevent_env to pass some additional information (like "media_change_event" or something) that would indicate what the event was for in case we ever use KOBJ_CHANGE for other things if you think this will work for you.
(In reply to comment #15) > So, the AHCI spec does say "This feature allows an ATAPI device to send a signal > to the host when media is inserted or removed and avoids polling the device for > media changes", and in my personal experience, disk manufacturers are sending AN > for both media insert and removal. Cool. I guess to properly implement this, I would need access to hardware. I have an eSATA II Expresscard adapter; any chance you know of any external eSATA optical drives that support this? I also have a desktop system with a SATA controller so I guess I could buy an internal SATA optical drive as well. > So in the code I sent I did not put the event associated with /block/sr*, but I > am working on a revision that will do that. the way I envisioned the > media_change_events file used would be at init time, HAL would check this file > to see if the disk supported AN. If media_change_events == 1, then hal would > disable polling on that drive. When a KOBJ_CHANGE event is received on that > particular drive, HAL can check for media_changed the way it always has to see > if media has changed. I can probably change the call to use kobject_uevent_env > to pass some additional information (like "media_change_event" or something) > that would indicate what the event was for in case we ever use KOBJ_CHANGE for > other things if you think this will work for you. This sounds perfect to me. I'm adding Kay Sievers, the udev maintainer, to this bug to get his comments as well. Thanks!
For the record, I've also added a hal-disable-polling(1) tool to HAL and this will hit Rawhide tomorrow. There's a man page too that hopefully sums up why we poll and how/why it may break stuff. It's a stop gap solution really. http://gitweb.freedesktop.org/?p=hal.git;a=commitdiff;h=c003685ace0936d4c813bf1ba422d521c72d4d62 It's very simply to use though [root@zelda ~]# ps aux|grep hald-addon-storage|grep polling root 14711 0.0 0.0 3128 956 ? S 16:09 0:00 hald-addon-storage: polling /dev/scd0 (every 2 sec) [root@zelda ~]# hal-disable-polling --device /dev/scd0 Polling for drive /dev/scd0 have been disabled. The fdi file written was /etc/hal/fdi/information/media-check-disable-storage_model_DVD_R___UJ_857.fdi [root@zelda ~]# ps aux|grep hald-addon-storage|grep polling [root@zelda ~]# cat /etc/hal/fdi/information/media-check-disable-storage_model_DVD_R___UJ_857.fdi <?xml version="1.0" encoding="UTF-8"?> <deviceinfo version="0.2"> <device> <match key="info.udi" string="/org/freedesktop/Hal/devices/storage_model_DVD_R___UJ_857"> <merge key="storage.media_check_enabled" type="bool">false</merge> </match> </device> </deviceinfo> [root@zelda ~]# hal-disable-polling --device /dev/scd0 --enable-polling Polling for drive /dev/scd0 have been enabled. The fdi file deleted was /etc/hal/fdi/information/media-check-disable-storage_model_DVD_R___UJ_857.fdi [root@zelda ~]# ps aux|grep hald-addon-storage|grep polling root 14730 0.0 0.0 3128 956 ? S 16:09 0:00 hald-addon-storage: polling /dev/scd0 (every 2 sec) Hope this helps. Thanks.
David - just wanted to let you know that we are working with the drive manufacturer to get some drives for you to develop with - they need to fix their firmware first to solve some problems we've found, but claim they will deliver to us this week. I'll verify the drives work as expected and send out to you ASAP.
Please send one my way too, please. I don't have any 'AN' hardware.
Readjusting the Summary from hal polls the cd drive every 2 seconds even when no user is logged in to hal polls storage devices since we, starting from Fedora 7, actually only poll every 16th second when no users are logged or all sessions are idle.
David - finally have a drive for you, will get it sent out to you on Monday. Unfortunately, we only have one that we can send.
Hi David - did you get the drive I sent you? Geoff left it on your desk. You will need to use a -mm kernel to try out the patches. Let me know what you think and if there is something else we need to do to help hal disable polling.
Hi Kristen, I've been out traveling the past two weeks. The drive is now on my desk and I plan to play around with it tomorrow; I'll report progress here - stay tuned! Thanks!
Hi Kristen, I'm now trying the drive with a recent Rawhide kernel (kernel-2.6.23-0.29.rc0.git6.fc8) which I believe is 2.6.22-git6 + a few RH patches. The drive is identified and works fine scsi 4:0:0:0: CD-ROM MTK 0617 MTK DVD-ROM 1309 A100 PQ: 0 ANSI: 5 sr1: scsi3-mmc drive: 4x/48x cd/rw xa/form2 cdda tray sr 4:0:0:0: Attached scsi CD-ROM sr1 sr 4:0:0:0: Attached scsi generic sg2 type 5 However the capabilities file is # cat /sys/block/sr1/capability 19 which doesn't include GENHD_FL_MEDIA_CHANGE_NOTIFY (defined as 4) as per your patch here http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commitdiff;h=86ce18d7b7925bfd6b64c061828ca2a857ee83b8;hp=352823160613b65fdaa558be486720a71f75ed86 I mean, I'm supposed to see bit 2 set here if the drive supports AN, right? For the record I'm using a SIIG Expresscard eSATA adapter with sata_sil24 and connecting the drive to an eSATA enclosure. Could that be a problem? Do I need to use a newer kernel or some patch not yet in Linus' tree? Or is the firmware wrong? Thanks!
first, you do need a patch that has not been merged yet. It is located here: http://www.kernel.org/pub/linux/kernel/people/kristen/patches/SATA/AN/ I'm still working on getting it acceptable for merging - at this point it looks like it will not be making it into 2.6.23. Secondly, you need to be using the ahci driver for this patch to work. I would suggest not using the adapter and just plugging the drive into a port directly on your motherboard. Make sure your BIOS is configured to use native AHCI mode for the disks.
OK, great, thanks for the pointers. I managed to get this to work # uname -a Linux localhost.localdomain 2.6.22-git6 #1 SMP Wed Jul 18 17:05:48 EDT 2007 i686 i686 i386 GNU/Linux [root@localhost ~]# cat /sys/block/sr0/capability 1d [root@localhost ~]# ps aux|grep hald-addon-storage root 2132 0.0 0.0 3312 984 ? S 17:19 0:00 hald-addon-storage: polling /dev/scd0 (every 2 sec) root 2660 0.0 0.0 4004 732 pts/1 S+ 17:21 0:00 grep hald-addon-storage [root@localhost ~]# kill 2132 [root@localhost ~]# ps aux|grep hald-addon-storage root 2662 0.0 0.0 4004 728 pts/1 S+ 17:21 0:00 grep hald-addon-storage [root@localhost ~]# udevmonitor udevmonitor will print the received events for: UDEV the event which udev sends out after rule processing UEVENT the kernel uevent UEVENT[1184793703.997005] change /block/sr0 (block) UDEV [1184793704.128738] change /block/sr0 (block) [root@localhost ~]# ps aux|grep hald-addon-storage root 2679 0.0 0.0 4000 724 pts/1 S+ 17:22 0:00 grep hald-addon-storage I'll teach hal about this tomorrow.
Created attachment 159561 [details] combined patches Btw, the patches had to be tweaked slightly (main thing was a collision for the ATA_DFLAG_AN SYMBOL; looks like some ACPI bits beat you to it); am attaching a rolled up patch that I'm using against Rawhide's kernel-2.6.23-0.29.rc0.git6.fc8.
I've added support for SATA AN capable drives in the hal code http://gitweb.freedesktop.org/?p=hal.git;a=commitdiff;h=1113ce0b76bae9d48bbc894248a167f783c00428 If the kernel says the device can report media change events, we avoid starting the hald-addon-storage process for these drives and instead rely on the "change" uevent from the kernel. I'll keep this bug open until this is in Rawhide; the hal bits should land on Monday and I'll talk to davej about the kernel bits.
OK - but keep in mind that the kernel interface may change depending on what gets merged upstream.
Hi David, I've released a new version of the patches that send the uevent through the related scsi device instead of through the block device. Can you give it a test and see if this interface will work? It is in the latest -mm release if you want to give that a try. BTW - if you want to get wider testing for this feature, I've discovered that the DVD that comes on the latest Intel Desktop SDP is AN capable. Thanks, Kristen
*** Bug 415091 has been marked as a duplicate of this bug. ***
Here from bug #415091. I had the impression that this bug was now purely about the async notifications allowed by shiny new sata drives, but I guess I was wrong. I filed the bug because I'm not sure what is an acceptable number of wakeups from hald-addon-storage and because I wondered if there's anything more that can be done to make things better for existing and/or older hardware.
Based on the date this bug was created, it appears to have been reported against rawhide during the development of a Fedora release that is no longer maintained. In order to refocus our efforts as a project we are flagging all of the open bugs for releases which are no longer maintained. If this bug remains in NEEDINFO thirty (30) days from now, we will automatically close it. If you can reproduce this bug in a maintained Fedora version (7, 8, or rawhide), please change this bug to the respective version and change the status to ASSIGNED. (If you're unable to change the bug's version or status, add a comment to the bug and someone will change it for you.) Thanks for your help, and we apologize again that we haven't handled these issues to this point. The process we're following is outlined here: http://fedoraproject.org/wiki/BugZappers/F9CleanUp We will be following the process here: http://fedoraproject.org/wiki/BugZappers/HouseKeeping to ensure this doesn't happen again.
Removing annoying triage alias and putting back in ASSIGNED.
Changing version to '9' as part of upcoming Fedora 9 GA. More information and reason for this action is here: http://fedoraproject.org/wiki/BugZappers/HouseKeeping
F9: hal-0.5.11-1.fc9.x86_64 kernel-2.6.25.3-18.fc9.x86_64 I found a problem if the polling is DISABLED. The media size does not get updated which leads to various errors after changing DVD-RW media incl.: # mount /mnt/cdrom kernel: attempt to access beyond end of device kernel: sr0: rw=0, want=68, limit=4 kernel: isofs_fill_super: bread failed, dev=sr0, iso_blknum=16, block=16 Common upstream problem, never saw a solution: http://tinyurl.com/4os6dd Longer media inserted after shorter media gets an early cut on `cat /dev/cdrom' and it ever errors while trying to read iso9660 files located at the end. I expect my drive does not give a media change (not speaking about AN discussed above) so the kernel keeps its media size from the previous inserted media. At least I have never seen any media change message in `/var/log/*'. Commands "eject" or "hdparm -f" do not fix it but HAL polling fixes it right. Are you aware which HAL polling action fixes the drive? Thanks for an info. 3 action points after your approval: (1) Upstream kernel should probably do some media size detection more often - such as on an each device open. (2) Fedora or even upstream powertop should no longer suggest such dangerous setting or suggest it with another caution notice. powertop-1.9-3.fc9.x86_64 Suggestion: Disable 'hal' from polling your cdrom with: hal-disable-polling --device /dev/scd0 'hal' is the component that auto-opens a window if you plug in a CD but disables SATA power saving from kicking in. (3) The man page hal-disable-polling(1) should discourage users from using it: Some drives not giving media notification will be unable to read new media after changing them. The only drive tested - internal Lenovo T60 DVD-RW: Vendor_info : 'HL-DT-ST' Identification : 'DVDRAM GSA-4083N' Revision : '1.08' Tested on DVD-RW media: # dvd+rw-mediainfo /dev/sr0 INQUIRY: [HL-DT-ST][DVDRAM GSA-4083N][1.08] GET [CURRENT] CONFIGURATION: Mounted Media: 14h, DVD-RW Sequential Media ID: MCC 01RW4X Current Write Speed: 4.0x1385=5540KB/s Write Speed #0: 4.0x1385=5540KB/s Write Speed #1: 2.0x1385=2770KB/s GET [CURRENT] PERFORMANCE: Write Performance: 2.0x1385=2770KB/s@[0 -> 125951] 4.0x1385=5540KB/s@[125952 -> 2298495] Speed Descriptor#0: 02/2298495 R=3438KB/s W=5540KB/s Speed Descriptor#1: 02/2298495 R=3438KB/s W=2770KB/s READ DVD STRUCTURE[#10h]: Media Book Type: 00h, DVD-ROM book [revision 0] Legacy lead-out at: 2298496*2KB=4707319808 READ DVD STRUCTURE[#0h]: Media Book Type: 33h, DVD-RW book [revision 3] Last border-out at: 2045*2KB=4188160 READ DISC INFORMATION: Disc status: blank Number of Sessions: 1 State of Last Session: empty "Next" Track: 1 Number of Tracks: 1 READ FORMAT CAPACITIES: unformatted: 2297888*2048=4706074624 00h(800): 2297888*2048=4706074624 10h(10): 2297888*2048=4706074624 15h(10): 2297888*2048=4706074624 READ TRACK INFORMATION[#1]: Track State: invisible incremental Track Start Address: 0*2KB Next Writable Address: 0*2KB Free Blocks: 2297888*2KB Track Size: 2297888*2KB READ CAPACITY: 0*2048=0
This message is a reminder that Fedora 9 is nearing its end of life. Approximately 30 (thirty) days from now Fedora will stop maintaining and issuing updates for Fedora 9. It is Fedora's policy to close all bug reports from releases that are no longer maintained. At that time this bug will be closed as WONTFIX if it remains open with a Fedora 'version' of '9'. Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, simply change the 'version' to a later Fedora version prior to Fedora 9's end of life. Bug Reporter: Thank you for reporting this issue and we are sorry that we may not be able to fix it before Fedora 9 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora please change the 'version' of this bug to the applicable version. If you are unable to change the version, please add a comment here and someone will do it for you. Although we aim to fix as many bugs as possible during every release's lifetime, sometimes those efforts are overtaken by events. Often a more recent Fedora release includes newer upstream software that fixes bugs or makes them obsolete. The process we are following is described here: http://fedoraproject.org/wiki/BugZappers/HouseKeeping
Fedora 9 changed to end-of-life (EOL) status on 2009-07-10. Fedora 9 is no longer maintained, which means that it will not receive any further security or bug fix updates. As a result we are closing this bug. If you can reproduce this bug against a currently maintained version of Fedora please feel free to reopen this bug against that version. Thank you for reporting this bug and we are sorry it could not be fixed.