+++ This bug was initially created as a clone of Bug #344851 +++ Description of problem: The juju firewire stack lacks proper full support for ohci 1.0-based firewire controllers. Most glaringly, the lack of isochronous I/O support means dv cameras are non-functional with 1.0-based controllers. When we follow the call chain from dvgrab, it traces down to the call ohci_allocate_iso_context() in drivers/firewire/fw-ohci.c, where we have: /* FIXME: We need a fallback for pre 1.1 OHCI. */ if (callback == handle_ir_dualbuffer_packet && ohci->version < OHCI_VERSION_1_1) return ERR_PTR(-EINVAL); As Stefan mentioned in bug 328011, and as suggested by the above code, its a known issue, listed at both http://wiki.linux1394.org/JujuMigration and http://wiki.linux1394.org/ToDo as a top priority item. 1.0 controllers appear to be quite a bit more prevalent than first thought (in my own systems, they outnumber 1.1 controllers 5 to 1), so we really need to get proper support for 'em sooner than later. Attached is a patch from Stefan to at least log the reason for the failure in the interim. -- Additional comment from jwilson on 2007-10-21 22:35 EST -- Created an attachment (id=233701) Stefan's patch to have firewire stack log something about not supporting 1.0 controllers -- Additional comment from stefan-r-rhbz.de on 2007-10-22 03:38 EST -- > 1.0 controllers appear to be quite a bit more prevalent than first > thought (in my own systems, they outnumber 1.1 controllers 5 to 1) My own sample: 1.0 chip types : 1.1 chip types = 3 : 2 1.0 cards : 1.1 cards = 5 : 3 All acquisitions during the past 12 months (2 new, 1 used) happened to be 1.0. It shows that I don't do anything isochronous. -- Additional comment from nyh.ac.il on 2007-10-28 11:35 EST -- I have just installed Fedora 7 on a new computer, and dvgrab from firewire doesn't work, and all signs point to this bug (apparently, despite being a new card, it is "OHCI 1.0" which isn't supported by the "Juju" patch that Fedora used in the kernel). To me, this looks like a major Fedora bug (more serious than the "medium severity" assigned to it). It prevents people from using their hardware, and worst of all, and is quite difficult to fix by manual compilation (requires recompiling the kernel, and applications like dvgrab or libraries lib libraw1394 which were also modified by Fedora to fit the Juju patch). I was most saddened by the rumors I've been reading (but correct me if I'm wrong) that this situation won't be fixed in Fedora 8. I wonder, if half the people (or whatever - see statistics in above comments) with firewire cannot use it AT ALL with this juju patch, how on earth can it be considered an improvement over the old ieee1394 stack? What is preventing fedora from *temporarily* but *immediately* reverting back to kernels with the old stack, and programs like libraw1394 unpatched, and only returning to this juju version when it actually works for more than half the people? Since people have bought their firewire card for a good reason (usually, to capture video), often the card will be more important to them than their choice of distro, and many will switch to some other distro. That would be a real shame, and simply unnecessary if Fedora only reverted to the working version of the kernel. -- Additional comment from stefan-r-rhbz.de on 2007-10-28 12:51 EST -- > is quite difficult to fix by manual compilation (requires recompiling > the kernel, and applications like dvgrab or libraries lib libraw1394 > which were also modified by Fedora to fit the Juju patch). AFAIU dvgrab is unchanged. Changed were kernel modules, the low-level libraries libraw1394 and libdc1394, installation tools, and kernel-releated stuff like initrd, perhaps hal (I'm not sure of the latter). I can't speak for the Fedora project (I don't use Fedora and don't develop for it, I'm just lurking here as co-maintainer of kernel.org's FireWire drivers), but better than reverting to old drivers would be to get someone to implement the missing functionality. Alas I can't lend a hand at the moment due to lack of spare time. > I wonder, if half the people (or whatever - see statistics in above > comments) with firewire cannot use it AT ALL with this juju patch, > how on earth can it be considered an improvement over the old > ieee1394 stack? The number of OHCI 1.0 installations and the fact that even newly sold cards and mainboards may contain OHCI 1.0 chips has alas been underestimated by those involved, including me. Those who use FireWire only for storage devices are not affected by this, BTW. -- Additional comment from stefan-r-rhbz.de on 2007-10-28 13:44 EST -- From the don't-do-this-at-home department: Krzysztof Halasa found that VIA VT6307 can be programmed to support either OHCI 1.0 or OHCI 1.1: http://marc.info/?l=linux1394-user&m=119292265030931 My impression though is that most VT6307 are already sold programmed in OHCI 1.1 mode. Somebody has yet to try whether this also works with VIA VT6306; VIA's documentation is ambiguous in that regard. It seems to me that all VT6306 are sold as OHCI 1.0 controllers. -- Additional comment from nyh.ac.il on 2007-10-28 17:25 EST -- Hi Stefan, according to lspci, I have a "FireWire (IEEE 1394): VIA Technologies, Inc. IEEE 1394 Host Controller (rev 46)" - I don't know whether or not it's a "VT6306", and even if it were, I'm not likely to try something its own author says "might blow up your machine" and "use at your own risk". I don't even know for sure that my card is "OHCI 1.0", or how to check it. Anyway, I understand why ultimately, the "firewire" ("juju") drivers are better than the ieee1394, and why they shouldn't be completely abandoned. But I completely fail to understand why, if the new drivers don't actually work for the majority of the people, Fedora doesn't issue an update to the kernel and libraw1394 (or whatever needs to be changed) returning the old drivers - which the rest of the Linux world besides Fedora - appears to be using. When the missing "OHCI 1.0" support is returned - hopefully more sooner than later - a new update to the kernel and libraw1394 can return back the improved Juju kernel. While this unfortunate situation (where half the users of firewire digital-video grabbers can't use them on Fedora) has lasted for half a year now, I believe that it still can be rectified (by returning the old drivers) in a very short time, and I hope that the powers that be decide to do so. -- Additional comment from stefan-r-rhbz.de on 2007-10-28 18:10 EST -- > I don't even know for sure that my card is "OHCI 1.0", or how to check it. "dmesg | grep firewire_ohci" will display the OHCI revision, unless there were already too many other kernel messages after firewire-ohci's initialization message. In that case, you can repeat the initialization with "modprobe -r firewire-ohci && modprobe firewire-ohci && dmesg | grep firewire_ohci". (Grep needs an underscore _ in the driver name.) The VIA rev 46 controller is most certainly a VT6306 which I believe does not implement OHCI 1.1 and I suspect cannot be reprogrammed to do so. The problem with the new drivers is: In OHCI 1.1, there exist three modes for isochronous reception, all operating on series of memory buffers: Packet-per-buffer, which puts not more than one packet into each buffer; buffer-fill, which concatenates packet in the buffers; and dual-buffer, which is like buffer-fill but with two strings of buffers. With dual-buffer, the controller takes each packet apart and puts all the first parts in the first string of buffers and the second parts into the second string. So, regarding functionality, dual-buffer is a superset of buffer-fill. It is useful to separate protocol headers from payload data. In OHCI 1.0, only packet-per-buffer and buffer-fill were specified. (This was only discovered after the fact because the OHCI 1.1 spec doesn't say what's new, and the OHCI 1.0 spec is nowhere available for download.) The firewire-ohci driver is specialized on dual-buffer mode. What needs to be done is to use buffer-fill mode instead, at least if an OHCI 1.0 chip is encountered. This requires someone with (a) knowledge of DMA programming in the Linux kernel, (b) some insight into isochronous FireWire applications and libraries, (c) time to implement it and hardware to test it. At the moment I lack at least (b) and (c). -- Additional comment from stefan-r-rhbz.de on 2007-10-28 18:33 EST -- > So, regarding functionality, dual-buffer is a superset of buffer-fill. This could mean that with a bit of luck... > This requires someone with (a) knowledge of DMA programming in the > Linux kernel, (b) some insight into isochronous FireWire applications > and libraries, (c) time to implement it and hardware to test it. ...(b) might not be a strict requirement, and (a) could be done the monkey-see-monkey-do way based on the existing dual-buffer code and the OHCI 1.1 spec (see download links at http://wiki.linux1394.org/Links/Specifications). Unless there is something more to it of which I am not yet aware of. -- Additional comment from jwilson on 2007-10-28 23:10 EST -- I believe (b) should be mostly unnecessary, and yeah, I think (a) can mostly be figured out by inspecting the dual-buffer code. I've got (c), and have starting trying to whip something up, simultaneous with digesting the specs -- initially, looking at doing a buffer-fill iso receive for ohci 1.0. I've talked to Kristain a little bit about all this as well, and will be picking his brain as I try to get this working. Sounds like the most fun might be dissecting the packet headers from the data in software (essentially doing what dual- buffer does for us in hardware). -- Additional comment from stefan-r-rhbz.de on 2007-10-29 03:02 EST -- > the most fun might be dissecting the packet headers from the data in > software (essentially doing what dual-buffer does for us in hardware) Does the stack actually have to do so if serving software written for raw1394? -- Additional comment from jwilson on 2007-10-29 12:00 EST -- Good question. At a glance, I'm not seeing the dissection being done in-kernel in the old stack, but I've not looked thoroughly. I'll peruse both the old stack and the libraw1394 support for the old stack to see if I can't figure that out... Certainly sounds reasonable to just hand off the raw captured data to libraw1394 though. -- Additional comment from stefan-r-rhbz.de on 2007-10-29 12:46 EST -- Does libraw1394's Juju backend let the kernel split off anything at all? I guess the answer lies in the values provided with struct fw_cdev_create_iso_context.header_size (and struct fw_cdev_iso_packet.control's HEADER_LENGTH part). -- Additional comment from jwilson on 2007-10-29 17:21 EST -- Kristian suggests that libraw1394 shouldn't have to know what IR method we're using, and we ought to present everything to libraw1394 the same way, regardless of whether we had to go to a fallback IR method for ohci 1.0. Thus far, I've added analogs to ohci_queue_iso_receive_dualbuffer and handle_ir_dualbuffer_packet, unimaginatively named ohci_queue_iso_receive_bufferfill and handle_ir_bufferfill_packet, wired them up, and at least got the controller receiving and acknowledging packets. The primary thing that is left (and still a bit over my head) is splitting up the headers and data, and sending them out to userspace in an intelligible form. -- Additional comment from jwilson on 2007-11-03 23:31 EST -- Just a quick status update... We made some pretty good progress this past week on ohci 1.0 support, and for the most part, all that is left is properly sorting out the headers and payloads into the right buffers where userspace expects them (so that we're presenting exactly the same thing to userspace, regardless of the controller being ohci 1.1 or 1.0). Don't want to promise anything, but I'm hoping to be grabbing video off my cable box w/my ohci 1.0-equipped Mac Mini by next weekend... :) -- Additional comment from jprenaud on 2007-11-12 05:15 EST -- Any update on this? Also, would that be made available for fedora 7 too? -- Additional comment from jwilson on 2007-11-12 09:58 EST -- Still working on it, and inching closer to completion. I believe I've got the packet parsing working correctly now (including all the fun cases like when a packet overflows to another buffer and/or page), but we still need to wire up the bits to actually feed the split headers and payloads out to userspace. Should be ready Real Soon Now(tm)... :) And yes, once working, this will be added to a Fedora 7 update kernel as well. -- Additional comment from choke on 2007-11-12 11:01 EST -- Awesome to head that, Jarod. Looking forward to it. -- Additional comment from jwilson on 2007-11-13 16:12 EST -- Still some rough edges to finish up, but with (a lot of) help from Kristian this afternoon, we just got 'dvgrab -d 5' to capture a 5 second clip using an OHCI 1.0 controller. :) -- Additional comment from jprenaud on 2007-11-13 16:45 EST -- Great news! Thanks for your hard work. I am looking forward to kino working on F7. -- Additional comment from jwilson on 2007-11-14 00:43 EST -- Created an attachment (id=257711) FireWire OHCI 1.0 Iso Receive support patch I've got a scratch kernel build with an initial cut of OHCI 1.0 Isochronous Receive support working its way through the build system right now: http://koji.fedoraproject.org/koji/taskinfo?taskID=240966 Would appreciate testing feedback from folks (i.e., does 'dvgrab -d 5' give you a 5-second clip that looks and sounds sane, and results on how kino behaves -- still need to try that myself). I'm already anticipating that people are going to see dvgrab spitting errors about buffer underruns, as I've seen that myself on most of my test captures thus far. Unfortunately, it seems this is largely due to this not being a particularly efficient implementation, which is a result of trying to make it feed the exact same results back to userspace as OHCI 1.1 controllers do (so that userspace isn't required to know which one we're doing). We actually ended up scrapping buffer-fill IR mode for now, instead using packet-per-buffer for OHCI 1.0, as it turned out to be next to impossible to make bufferfill feed the same thing as the OHCI 1.1 method (dual-buffer) out to userspace. I'm still hoping we can streamline the packet-per-buffer code a bit more to help improve the performance... After a bit of further poking, testing and hopefully, improving, I'll rediff against upstream, throw an SOB or two on it and ship an official patch toward linux1394-devel. -- Additional comment from zimny_browar on 2007-11-15 05:43 EST -- when you finish this fix? I still in Fedora 8 have this: dvgrab -autosplit -format raw -size 0 -noavc -timestamp foo- Found AV/C device with GUID 0x08004601044380f6 ioctl call failed, retval = -1 ieee1394io.cc:460: In function "virtual bool iec61883Reader::StartReceive()": "iec61883_dv_fb_start( m_iec61883.dv, channel )" evaluated to -1 ieee1394io.cc:460: errno: 38 (Function not implemented) "" 0.00 MB 0 frames Capture Stopped -- Additional comment from jwilson on 2007-11-15 17:08 EST -- (In reply to comment #21) > when you finish this fix? There was a scratch build to test out linked above in comment #20. There should be an updates-testing kernel relatively soon carrying this patch, though the resulting video captures are not perfect and I'm still trying to figure out why. -- Additional comment from wwoods on 2007-11-16 13:26 EST -- Woo! I can confirm that 'sudo dvgrab -d5 -f raw' works on an OHCI 1.0 controller. There are occasional buffer underrun messages - but it doesn't happen every time. And I guess you need mpeg2 support to play back the resulting video, so it's slightly tricky to test in a pure-Fedora environment. There are some artifacts in the video. Sometimes it fails to find the camera. And occasionally I get this message in dmesg: firewire_ohci: context_stop: still active (0x40000411) But other than that, it works exactly how I'd expect it to. I've never had this working before now, so I'm not sure if those things are normal or not. Can someone who uses this stuff in more heavy/interesting ways give that test kernel a shot? -- Additional comment from jwilson on 2007-11-20 12:20 EST -- Will's experience in comment #23 is about on par with what I see myself (though I also occasionally hit bug 370931 as well). The first odd thing I've noticed about the artifacts is that when they show up in a given frame, its usually in the same places within the frame as it is for other frames in a given capture. (i.e., the artifacts always show up at the same x/y offsets in the video when they do show up). The other odd thing about the artifacts is that they're often either obvious residual data from the prior frame, or a lighter or darker version of what should be in the current frame. I've triple-checked the code, and thus far, have come up empty for an explanation. Its definitely not arch-specific or a cpu power thing either, since I get the same results on an x86_64 quad 3GHz core xeon and an i686 1GHz Via C3 EPIA... -- Additional comment from updates on 2007-11-26 13:50 EST -- kernel-2.6.23.8-34.fc7 has been pushed to the Fedora 7 testing repository. If problems still persist, please make note of it in this bug report. If you want to test the update, you can install it with su -c 'yum --enablerepo=updates-testing update kernel' -- Additional comment from jwilson on 2007-11-26 14:00 EST -- Nb: the updates-testing kernel comes with the version described in earlier comments. Its not perfect, but better than nothing, and we're working on fixing its deficiencies. -- Additional comment from ahziem1 on 2007-11-27 10:57 EST -- I am trying 2.6.23.8-34.fc7. When running as non-root, dvgrab and Kino find no camera. Kino complains about libraw1394 not loaded or can't read/write /dev/raw1394. Actually, that does not exist. Log files for non-root: Nov 27 07:22:51 z kernel: firewire_ohci: Added fw-ohci device 0000:00:0b.0, OHCI version 1.0 Nov 27 07:22:51 z kernel: firewire_core: created new fw device fw0 (0 config rom retries, S400) Nov 27 08:39:55 z kernel: firewire_ohci: Added fw-ohci device 0000:00:0b.0, OHCI version 1.0 Nov 27 08:39:55 z kernel: firewire_core: created new fw device fw0 (0 config rom retries, S400) Nov 27 08:40:54 z kernel: firewire_core: phy config: card 0, new root=ffc1, gap_count=5 Nov 27 08:40:54 z kernel: firewire_core: phy config: card 0, new root=ffc1, gap_count=5 Nov 27 08:40:55 z kernel: firewire_core: created new fw device fw1 (0 config rom retries, S100) When running "dvgrab sony-" as root, it operates the camera and creates a file, but the captured data is corrupt. The file is not recognized by Kino, vlc, gmplayer, or file. Here are the logs for root: Nov 27 08:51:17 z kernel: firewire_ohci: context_stop: still active (0x40000411) Nov 27 08:51:17 z kernel: dvgrab[3937]: segfault at b66fc004 eip 00ba83ec esp bf9b1148 error 4 # lspci 00:0b.0 FireWire (IEEE 1394): VIA Technologies, Inc. IEEE 1394 Host Controller (rev 80) -- Additional comment from ahziem1 on 2007-11-27 11:00 EST -- Sorry. After waiting for the file to finish, I can play it. Video looks OK, but I hear a clicking in the audio track. I'll test more later. -- Additional comment from stefan-r-rhbz.de on 2007-11-27 11:47 EST -- Re comment #27: The inability to use kino and dvgrab as unprivileged user is very certainly a matter of device file permissions: $ ls -l /dev/fw* These device files are used by the juju-enabled libraw1394 instead of /dev/raw1394. Any error messages referring to /dev/raw1394 are just remnants of the old IEEE 1394 kernel interface. Ideally, a facility like udev or hal would adjust the permissions of /dev/fw* files which belong to nodes which have AV/C units or IIDC units so that users who are for example members of a video group or something like that could read and write to these device files. Of course this would need explicit support by udev or hal. The permissions problem has also been mentioned in https://bugzilla.redhat.com/process_bug.cgi#c23 It can be seen as successor of bug 191670. As for the segfault: Another dvgrab segfault is mentioned in bug 370931. When I tested dvgrab with an OHCI 1.1 controller on a vanilla x86-32 kernel, it always segfaulted on exit but wrote a valid video file before that. When I tested dvgrab with an OHCI 1.0 controller on a vanilla x86-64 kernel with Jarod's patch for OHCI 1.0 iso reception, dvgrab did not segfault. Re comment #28: When captured by dvgrab via OHCI 1.1, video and audio are fine. When captured by dvgrab via OHCI OHCI 1.0, video is defective like described by Jarod in comment #24. Audio appeared to be fine. I tested with a JVC miniDV camcorder. I have less success with Kino. Except for a single lucky occasion, it always segfaults in raw1394_iso_recv_start as soon as I attempt to enter the Capture tab, regardless whether a camera is connected or not. -- Additional comment from jwilson on 2007-11-30 13:19 EST -- (In reply to comment #29) > The inability to use kino and dvgrab as unprivileged user is very certainly a > matter of device file permissions: > $ ls -l /dev/fw* > These device files are used by the juju-enabled libraw1394 instead of > /dev/raw1394. Any error messages referring to /dev/raw1394 are just remnants of > the old IEEE 1394 kernel interface. > > Ideally, a facility like udev or hal would adjust the permissions of /dev/fw* > files which belong to nodes which have AV/C units or IIDC units so that users > who are for example members of a video group or something like that could read > and write to these device files. Of course this would need explicit support by > udev or hal. hal is supposed to be taking care of access permissions after a user logs into the desktop, and does do so on a freshly installed F8 system for me. I've seen other situations where it doesn't though -- my originally-fc6-upgraded-to-f7-then-to-f8 Mac Mini doesn't get the access permissions right either, though that box is running ratpoison for its window manager (maybe no hal stuff loading at login then?). > Re comment #28: > > When captured by dvgrab via OHCI 1.1, video and audio are fine. When captured > by dvgrab via OHCI OHCI 1.0, video is defective like described by Jarod in > comment #24. Audio appeared to be fine. I tested with a JVC miniDV camcorder. New patch coming shortly, which on every card I've tried outside of Via VT6307 ones, produces flawless captures using an improved ohci 1.0 IR method. I got no less than five different firewire chipsets that all captured perfectly with this forthcoming version, but the VT6307 simply doesn't want to work for reasons I've yet to figure out. > I have less success with Kino. Except for a single lucky occasion, it always > segfaults in raw1394_iso_recv_start as soon as I attempt to enter the Capture > tab, regardless whether a camera is connected or not. A lot of the kino code is identical to dvgrab code, so I suspect the kino segfault is rather similar in nature to the dvgrab one, and could be fixed in a similar fashion. -- Additional comment from jwilson on 2007-11-30 14:13 EST -- Created an attachment (id=274231) Updated FireWire OHCI 1.0 Iso Receive patch for 2.6.23.x -- Additional comment from jwilson on 2007-11-30 14:14 EST -- Created an attachment (id=274241) Upstream version of FireWire OHCI 1.0 Iso Receive patch
Patch for RHEL5 coming soon...
Created attachment 274301 [details] FireWire OHCI 1.0 Isochronous Receive support This may well already be covered for RHEL5.2 inclusion via bug 370421, but just the same, here's the support backported to the stack as shipped in 5.1. Successfully tested against kernel 2.6.18-53.el5 on an x86_64 system w/an Agere FW323 OHCI 1.0 controller and juju-ized dvgrab bits rebuilt from Fedora.
Created attachment 275981 [details] Updated FireWire OHCI 1.0 Iso Receive support Updated version remedies a regression introduced on OHCI 1.1 controllers introduced by the prior version of this patch.
Committed to 2.6.24-rc5: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commitdiff;h=a186b4a6b22fdc96a1ed63da483d267b5d00839e
This request was evaluated by Red Hat Product Management for inclusion, but this component is not scheduled to be updated in the current Red Hat Enterprise Linux release. If you would like this request to be reviewed for the next minor release, ask your support representative to set the next rhel-x.y flag to "?".
The patch that I added for bz#370421 should have covered most of this. With it, kernel-2.6.18-84.el5 works with my DV camcorder and dvgrab-3.0-1.el5 and my iSight and coriander-0.2007.07.03-0.2. Please open a new bug if there are specific problems with the 5.2 kernel and OHCI 1.0 controllers.