Description of problem: The juju firewire stack lacks proper full support for ohci 1.0-based firewire controllers. Most glaringly, the lack of isochronous I/O support means dv cameras are non-functional with 1.0-based controllers. When we follow the call chain from dvgrab, it traces down to the call ohci_allocate_iso_context() in drivers/firewire/fw-ohci.c, where we have: /* FIXME: We need a fallback for pre 1.1 OHCI. */ if (callback == handle_ir_dualbuffer_packet && ohci->version < OHCI_VERSION_1_1) return ERR_PTR(-EINVAL); As Stefan mentioned in bug 328011, and as suggested by the above code, its a known issue, listed at both http://wiki.linux1394.org/JujuMigration and http://wiki.linux1394.org/ToDo as a top priority item. 1.0 controllers appear to be quite a bit more prevalent than first thought (in my own systems, they outnumber 1.1 controllers 5 to 1), so we really need to get proper support for 'em sooner than later. Attached is a patch from Stefan to at least log the reason for the failure in the interim.
Created attachment 233701 [details] Stefan's patch to have firewire stack log something about not supporting 1.0 controllers
> 1.0 controllers appear to be quite a bit more prevalent than first > thought (in my own systems, they outnumber 1.1 controllers 5 to 1) My own sample: 1.0 chip types : 1.1 chip types = 3 : 2 1.0 cards : 1.1 cards = 5 : 3 All acquisitions during the past 12 months (2 new, 1 used) happened to be 1.0. It shows that I don't do anything isochronous.
I have just installed Fedora 7 on a new computer, and dvgrab from firewire doesn't work, and all signs point to this bug (apparently, despite being a new card, it is "OHCI 1.0" which isn't supported by the "Juju" patch that Fedora used in the kernel). To me, this looks like a major Fedora bug (more serious than the "medium severity" assigned to it). It prevents people from using their hardware, and worst of all, and is quite difficult to fix by manual compilation (requires recompiling the kernel, and applications like dvgrab or libraries lib libraw1394 which were also modified by Fedora to fit the Juju patch). I was most saddened by the rumors I've been reading (but correct me if I'm wrong) that this situation won't be fixed in Fedora 8. I wonder, if half the people (or whatever - see statistics in above comments) with firewire cannot use it AT ALL with this juju patch, how on earth can it be considered an improvement over the old ieee1394 stack? What is preventing fedora from *temporarily* but *immediately* reverting back to kernels with the old stack, and programs like libraw1394 unpatched, and only returning to this juju version when it actually works for more than half the people? Since people have bought their firewire card for a good reason (usually, to capture video), often the card will be more important to them than their choice of distro, and many will switch to some other distro. That would be a real shame, and simply unnecessary if Fedora only reverted to the working version of the kernel.
> is quite difficult to fix by manual compilation (requires recompiling > the kernel, and applications like dvgrab or libraries lib libraw1394 > which were also modified by Fedora to fit the Juju patch). AFAIU dvgrab is unchanged. Changed were kernel modules, the low-level libraries libraw1394 and libdc1394, installation tools, and kernel-releated stuff like initrd, perhaps hal (I'm not sure of the latter). I can't speak for the Fedora project (I don't use Fedora and don't develop for it, I'm just lurking here as co-maintainer of kernel.org's FireWire drivers), but better than reverting to old drivers would be to get someone to implement the missing functionality. Alas I can't lend a hand at the moment due to lack of spare time. > I wonder, if half the people (or whatever - see statistics in above > comments) with firewire cannot use it AT ALL with this juju patch, > how on earth can it be considered an improvement over the old > ieee1394 stack? The number of OHCI 1.0 installations and the fact that even newly sold cards and mainboards may contain OHCI 1.0 chips has alas been underestimated by those involved, including me. Those who use FireWire only for storage devices are not affected by this, BTW.
From the don't-do-this-at-home department: Krzysztof Halasa found that VIA VT6307 can be programmed to support either OHCI 1.0 or OHCI 1.1: http://marc.info/?l=linux1394-user&m=119292265030931 My impression though is that most VT6307 are already sold programmed in OHCI 1.1 mode. Somebody has yet to try whether this also works with VIA VT6306; VIA's documentation is ambiguous in that regard. It seems to me that all VT6306 are sold as OHCI 1.0 controllers.
Hi Stefan, according to lspci, I have a "FireWire (IEEE 1394): VIA Technologies, Inc. IEEE 1394 Host Controller (rev 46)" - I don't know whether or not it's a "VT6306", and even if it were, I'm not likely to try something its own author says "might blow up your machine" and "use at your own risk". I don't even know for sure that my card is "OHCI 1.0", or how to check it. Anyway, I understand why ultimately, the "firewire" ("juju") drivers are better than the ieee1394, and why they shouldn't be completely abandoned. But I completely fail to understand why, if the new drivers don't actually work for the majority of the people, Fedora doesn't issue an update to the kernel and libraw1394 (or whatever needs to be changed) returning the old drivers - which the rest of the Linux world besides Fedora - appears to be using. When the missing "OHCI 1.0" support is returned - hopefully more sooner than later - a new update to the kernel and libraw1394 can return back the improved Juju kernel. While this unfortunate situation (where half the users of firewire digital-video grabbers can't use them on Fedora) has lasted for half a year now, I believe that it still can be rectified (by returning the old drivers) in a very short time, and I hope that the powers that be decide to do so.
> I don't even know for sure that my card is "OHCI 1.0", or how to check it. "dmesg | grep firewire_ohci" will display the OHCI revision, unless there were already too many other kernel messages after firewire-ohci's initialization message. In that case, you can repeat the initialization with "modprobe -r firewire-ohci && modprobe firewire-ohci && dmesg | grep firewire_ohci". (Grep needs an underscore _ in the driver name.) The VIA rev 46 controller is most certainly a VT6306 which I believe does not implement OHCI 1.1 and I suspect cannot be reprogrammed to do so. The problem with the new drivers is: In OHCI 1.1, there exist three modes for isochronous reception, all operating on series of memory buffers: Packet-per-buffer, which puts not more than one packet into each buffer; buffer-fill, which concatenates packet in the buffers; and dual-buffer, which is like buffer-fill but with two strings of buffers. With dual-buffer, the controller takes each packet apart and puts all the first parts in the first string of buffers and the second parts into the second string. So, regarding functionality, dual-buffer is a superset of buffer-fill. It is useful to separate protocol headers from payload data. In OHCI 1.0, only packet-per-buffer and buffer-fill were specified. (This was only discovered after the fact because the OHCI 1.1 spec doesn't say what's new, and the OHCI 1.0 spec is nowhere available for download.) The firewire-ohci driver is specialized on dual-buffer mode. What needs to be done is to use buffer-fill mode instead, at least if an OHCI 1.0 chip is encountered. This requires someone with (a) knowledge of DMA programming in the Linux kernel, (b) some insight into isochronous FireWire applications and libraries, (c) time to implement it and hardware to test it. At the moment I lack at least (b) and (c).
> So, regarding functionality, dual-buffer is a superset of buffer-fill. This could mean that with a bit of luck... > This requires someone with (a) knowledge of DMA programming in the > Linux kernel, (b) some insight into isochronous FireWire applications > and libraries, (c) time to implement it and hardware to test it. ...(b) might not be a strict requirement, and (a) could be done the monkey-see-monkey-do way based on the existing dual-buffer code and the OHCI 1.1 spec (see download links at http://wiki.linux1394.org/Links/Specifications). Unless there is something more to it of which I am not yet aware of.
I believe (b) should be mostly unnecessary, and yeah, I think (a) can mostly be figured out by inspecting the dual-buffer code. I've got (c), and have starting trying to whip something up, simultaneous with digesting the specs -- initially, looking at doing a buffer-fill iso receive for ohci 1.0. I've talked to Kristain a little bit about all this as well, and will be picking his brain as I try to get this working. Sounds like the most fun might be dissecting the packet headers from the data in software (essentially doing what dual- buffer does for us in hardware).
> the most fun might be dissecting the packet headers from the data in > software (essentially doing what dual-buffer does for us in hardware) Does the stack actually have to do so if serving software written for raw1394?
Good question. At a glance, I'm not seeing the dissection being done in-kernel in the old stack, but I've not looked thoroughly. I'll peruse both the old stack and the libraw1394 support for the old stack to see if I can't figure that out... Certainly sounds reasonable to just hand off the raw captured data to libraw1394 though.
Does libraw1394's Juju backend let the kernel split off anything at all? I guess the answer lies in the values provided with struct fw_cdev_create_iso_context.header_size (and struct fw_cdev_iso_packet.control's HEADER_LENGTH part).
Kristian suggests that libraw1394 shouldn't have to know what IR method we're using, and we ought to present everything to libraw1394 the same way, regardless of whether we had to go to a fallback IR method for ohci 1.0. Thus far, I've added analogs to ohci_queue_iso_receive_dualbuffer and handle_ir_dualbuffer_packet, unimaginatively named ohci_queue_iso_receive_bufferfill and handle_ir_bufferfill_packet, wired them up, and at least got the controller receiving and acknowledging packets. The primary thing that is left (and still a bit over my head) is splitting up the headers and data, and sending them out to userspace in an intelligible form.
Just a quick status update... We made some pretty good progress this past week on ohci 1.0 support, and for the most part, all that is left is properly sorting out the headers and payloads into the right buffers where userspace expects them (so that we're presenting exactly the same thing to userspace, regardless of the controller being ohci 1.1 or 1.0). Don't want to promise anything, but I'm hoping to be grabbing video off my cable box w/my ohci 1.0-equipped Mac Mini by next weekend... :)
Any update on this? Also, would that be made available for fedora 7 too?
Still working on it, and inching closer to completion. I believe I've got the packet parsing working correctly now (including all the fun cases like when a packet overflows to another buffer and/or page), but we still need to wire up the bits to actually feed the split headers and payloads out to userspace. Should be ready Real Soon Now(tm)... :) And yes, once working, this will be added to a Fedora 7 update kernel as well.
Awesome to head that, Jarod. Looking forward to it.
Still some rough edges to finish up, but with (a lot of) help from Kristian this afternoon, we just got 'dvgrab -d 5' to capture a 5 second clip using an OHCI 1.0 controller. :)
Great news! Thanks for your hard work. I am looking forward to kino working on F7.
Created attachment 257711 [details] FireWire OHCI 1.0 Iso Receive support patch I've got a scratch kernel build with an initial cut of OHCI 1.0 Isochronous Receive support working its way through the build system right now: http://koji.fedoraproject.org/koji/taskinfo?taskID=240966 Would appreciate testing feedback from folks (i.e., does 'dvgrab -d 5' give you a 5-second clip that looks and sounds sane, and results on how kino behaves -- still need to try that myself). I'm already anticipating that people are going to see dvgrab spitting errors about buffer underruns, as I've seen that myself on most of my test captures thus far. Unfortunately, it seems this is largely due to this not being a particularly efficient implementation, which is a result of trying to make it feed the exact same results back to userspace as OHCI 1.1 controllers do (so that userspace isn't required to know which one we're doing). We actually ended up scrapping buffer-fill IR mode for now, instead using packet-per-buffer for OHCI 1.0, as it turned out to be next to impossible to make bufferfill feed the same thing as the OHCI 1.1 method (dual-buffer) out to userspace. I'm still hoping we can streamline the packet-per-buffer code a bit more to help improve the performance... After a bit of further poking, testing and hopefully, improving, I'll rediff against upstream, throw an SOB or two on it and ship an official patch toward linux1394-devel.
when you finish this fix? I still in Fedora 8 have this: dvgrab -autosplit -format raw -size 0 -noavc -timestamp foo- Found AV/C device with GUID 0x08004601044380f6 ioctl call failed, retval = -1 ieee1394io.cc:460: In function "virtual bool iec61883Reader::StartReceive()": "iec61883_dv_fb_start( m_iec61883.dv, channel )" evaluated to -1 ieee1394io.cc:460: errno: 38 (Function not implemented) "" 0.00 MB 0 frames Capture Stopped
(In reply to comment #21) > when you finish this fix? There was a scratch build to test out linked above in comment #20. There should be an updates-testing kernel relatively soon carrying this patch, though the resulting video captures are not perfect and I'm still trying to figure out why.
Woo! I can confirm that 'sudo dvgrab -d5 -f raw' works on an OHCI 1.0 controller. There are occasional buffer underrun messages - but it doesn't happen every time. And I guess you need mpeg2 support to play back the resulting video, so it's slightly tricky to test in a pure-Fedora environment. There are some artifacts in the video. Sometimes it fails to find the camera. And occasionally I get this message in dmesg: firewire_ohci: context_stop: still active (0x40000411) But other than that, it works exactly how I'd expect it to. I've never had this working before now, so I'm not sure if those things are normal or not. Can someone who uses this stuff in more heavy/interesting ways give that test kernel a shot?
Will's experience in comment #23 is about on par with what I see myself (though I also occasionally hit bug 370931 as well). The first odd thing I've noticed about the artifacts is that when they show up in a given frame, its usually in the same places within the frame as it is for other frames in a given capture. (i.e., the artifacts always show up at the same x/y offsets in the video when they do show up). The other odd thing about the artifacts is that they're often either obvious residual data from the prior frame, or a lighter or darker version of what should be in the current frame. I've triple-checked the code, and thus far, have come up empty for an explanation. Its definitely not arch-specific or a cpu power thing either, since I get the same results on an x86_64 quad 3GHz core xeon and an i686 1GHz Via C3 EPIA...
kernel-2.6.23.8-34.fc7 has been pushed to the Fedora 7 testing repository. If problems still persist, please make note of it in this bug report. If you want to test the update, you can install it with su -c 'yum --enablerepo=updates-testing update kernel'
Nb: the updates-testing kernel comes with the version described in earlier comments. Its not perfect, but better than nothing, and we're working on fixing its deficiencies.
I am trying 2.6.23.8-34.fc7. When running as non-root, dvgrab and Kino find no camera. Kino complains about libraw1394 not loaded or can't read/write /dev/raw1394. Actually, that does not exist. Log files for non-root: Nov 27 07:22:51 z kernel: firewire_ohci: Added fw-ohci device 0000:00:0b.0, OHCI version 1.0 Nov 27 07:22:51 z kernel: firewire_core: created new fw device fw0 (0 config rom retries, S400) Nov 27 08:39:55 z kernel: firewire_ohci: Added fw-ohci device 0000:00:0b.0, OHCI version 1.0 Nov 27 08:39:55 z kernel: firewire_core: created new fw device fw0 (0 config rom retries, S400) Nov 27 08:40:54 z kernel: firewire_core: phy config: card 0, new root=ffc1, gap_count=5 Nov 27 08:40:54 z kernel: firewire_core: phy config: card 0, new root=ffc1, gap_count=5 Nov 27 08:40:55 z kernel: firewire_core: created new fw device fw1 (0 config rom retries, S100) When running "dvgrab sony-" as root, it operates the camera and creates a file, but the captured data is corrupt. The file is not recognized by Kino, vlc, gmplayer, or file. Here are the logs for root: Nov 27 08:51:17 z kernel: firewire_ohci: context_stop: still active (0x40000411) Nov 27 08:51:17 z kernel: dvgrab[3937]: segfault at b66fc004 eip 00ba83ec esp bf9b1148 error 4 # lspci 00:0b.0 FireWire (IEEE 1394): VIA Technologies, Inc. IEEE 1394 Host Controller (rev 80)
Sorry. After waiting for the file to finish, I can play it. Video looks OK, but I hear a clicking in the audio track. I'll test more later.
Re comment #27: The inability to use kino and dvgrab as unprivileged user is very certainly a matter of device file permissions: $ ls -l /dev/fw* These device files are used by the juju-enabled libraw1394 instead of /dev/raw1394. Any error messages referring to /dev/raw1394 are just remnants of the old IEEE 1394 kernel interface. Ideally, a facility like udev or hal would adjust the permissions of /dev/fw* files which belong to nodes which have AV/C units or IIDC units so that users who are for example members of a video group or something like that could read and write to these device files. Of course this would need explicit support by udev or hal. The permissions problem has also been mentioned in https://bugzilla.redhat.com/process_bug.cgi#c23 It can be seen as successor of bug 191670. As for the segfault: Another dvgrab segfault is mentioned in bug 370931. When I tested dvgrab with an OHCI 1.1 controller on a vanilla x86-32 kernel, it always segfaulted on exit but wrote a valid video file before that. When I tested dvgrab with an OHCI 1.0 controller on a vanilla x86-64 kernel with Jarod's patch for OHCI 1.0 iso reception, dvgrab did not segfault. Re comment #28: When captured by dvgrab via OHCI 1.1, video and audio are fine. When captured by dvgrab via OHCI OHCI 1.0, video is defective like described by Jarod in comment #24. Audio appeared to be fine. I tested with a JVC miniDV camcorder. I have less success with Kino. Except for a single lucky occasion, it always segfaults in raw1394_iso_recv_start as soon as I attempt to enter the Capture tab, regardless whether a camera is connected or not.
(In reply to comment #29) > The inability to use kino and dvgrab as unprivileged user is very certainly a > matter of device file permissions: > $ ls -l /dev/fw* > These device files are used by the juju-enabled libraw1394 instead of > /dev/raw1394. Any error messages referring to /dev/raw1394 are just remnants of > the old IEEE 1394 kernel interface. > > Ideally, a facility like udev or hal would adjust the permissions of /dev/fw* > files which belong to nodes which have AV/C units or IIDC units so that users > who are for example members of a video group or something like that could read > and write to these device files. Of course this would need explicit support by > udev or hal. hal is supposed to be taking care of access permissions after a user logs into the desktop, and does do so on a freshly installed F8 system for me. I've seen other situations where it doesn't though -- my originally-fc6-upgraded-to-f7-then-to-f8 Mac Mini doesn't get the access permissions right either, though that box is running ratpoison for its window manager (maybe no hal stuff loading at login then?). > Re comment #28: > > When captured by dvgrab via OHCI 1.1, video and audio are fine. When captured > by dvgrab via OHCI OHCI 1.0, video is defective like described by Jarod in > comment #24. Audio appeared to be fine. I tested with a JVC miniDV camcorder. New patch coming shortly, which on every card I've tried outside of Via VT6307 ones, produces flawless captures using an improved ohci 1.0 IR method. I got no less than five different firewire chipsets that all captured perfectly with this forthcoming version, but the VT6307 simply doesn't want to work for reasons I've yet to figure out. > I have less success with Kino. Except for a single lucky occasion, it always > segfaults in raw1394_iso_recv_start as soon as I attempt to enter the Capture > tab, regardless whether a camera is connected or not. A lot of the kino code is identical to dvgrab code, so I suspect the kino segfault is rather similar in nature to the dvgrab one, and could be fixed in a similar fashion.
Created attachment 274231 [details] Updated FireWire OHCI 1.0 Iso Receive patch for 2.6.23.x
Created attachment 274241 [details] Upstream version of FireWire OHCI 1.0 Iso Receive patch
F8 kernels w/the updated patch are building as I type: http://koji.fedoraproject.org/koji/taskinfo?taskID=267566
Created attachment 275941 [details] v3 of FireWire OHCI 1.0 Iso Receive support Version 3 of this patch fixes a regression on OHCI 1.0 introduced by v2.
Created attachment 275971 [details] Fixed upstream version of FireWire OHCI 1.0 Iso Receive patch Upstream version of FireWire OHCI 1.0 Isochronous Receive patch with OHCI 1.1 regression remedied.
Correction to comment #34... v3 fixes a regression on OHCI 1.1 (not 1.0) that was introduced by v2. Changes have been added to F7, F8 and rawhide kernel trees, should trickle into builds soon...
OHCI 1.0 support added to rawhide, will show up in next kernel build. Also added to F8 kernels 2.6.23.9-78.fc8 and later, F7 kernels 2.6.23.9-44.fc7 and later. F8 builds available for testing here: http://koji.fedoraproject.org/packages/kernel/2.6.23.9/78.fc8/
When 2.6.23.9-78 kernel be in official f8 repo?
Not sure exactly when it'll land in the official updates repo, but not for a little while. Fedora kernel updates that aren't security fixes typically get pushed to updates-testing first. Should get over to updates-testing relatively soon though. In the mean time, you can simply pull them down from the link in comment #37 to test immediately.
kernel-2.6.23.8-34.fc7 has been pushed to the Fedora 7 stable repository. If problems still persist, please make note of it in this bug report.
The new-and-improved version isn't in 2.6.23.8-34.fc7, its in 2.6.23.9-44.fc7 for F7 and 2.6.23.9-78.fc8 for F8, as well as in current rawhide.
Still no luck with my setup (used to work on old stack): # uname -a Linux zaphod.theander.dk 2.6.23.9-78.fc8 #1 SMP Mon Dec 3 14:50:36 EST 2007 x86_64 x86_64 x86_64 GNU/Linux # dvgrab -f raw dippo- Found AV/C device with GUID 0x0000f0000b5a35ba "" 134215792.00 MB 0 frames Capture Stopped Error: no DV Version is: dvgrab 3.0 Camcorder starts playback, runs for a while and stops. No file is created. Kino (v.1.1.1) says: >> AV/C Enabled >>> Using iec61883 capture >>> iec61883Reader::StartThread on port 0 Kino experienced a segmentation fault. Dumping stack from the offending thread Obtained 10 stack frames. kino [0x435252] /lib64/libc.so.6 [0x3ff2c30f30] /usr/lib64/libraw1394.so.8 [0x3005603c52] /usr/lib64/libraw1394.so.8 [0x3005603d9a] /usr/lib64/libraw1394.so.8(raw1394_iso_recv_start+0x20) [0x3005603dd0] kino(_ZN14iec61883Reader12StartReceiveEv+0x20) [0x467a40] kino(_ZN14iec61883Reader11StartThreadEi+0xa9) [0x4667c9] kino(_ZN11PageCapture12CheckDevicesEv+0x11c) [0x47a91c] kino [0x47aa95] /lib64/libpthread.so.0 [0x3ff3806407] Done dumping - exiting. Device is: 00:0a.0 FireWire (IEEE 1394): VIA Technologies, Inc. IEEE 1394 Host Controller (rev 46) Kernel says: firewire_core: created new fw device fw2 (0 config rom retries, S400) firewire_core: phy config: card 0, new root=ffc1, gap_count=5 firewire_ohci: context_stop: still active (0x40000400) firewire_core: phy config: card 0, new root=ffc1, gap_count=5 If I strace dvgrab it ends in a bunch if ioctl, read and epoll_wait and just quits. Nothing is ever written on stdout??? Anything else I can provide?
(In reply to comment #42) > Device is: > 00:0a.0 FireWire (IEEE 1394): VIA Technologies, Inc. IEEE 1394 Host Controller > (rev 46) Ah. A Via VT6306 chipset. Crap. [...] > Anything else I can provide? Nope, that says it all. Both Stefan and I have encountered issues with the Via VT6306 and VT6307 in OHCI 1.0 mode that mirror your symptoms. I have a VT6306 PCI card that works fine in one box, but does what you're seeing in another, as well as onboard VT6307 controllers in two other systems that also have this problem. All other manufacturer's OHCI 1.0 controllers that I've been able to test (Agere FW323, NEC OrangeLink and various other TI and NEC 1.1 controllers forced into 1.0 mode) work just fine, so it seems to be a quirk with Via chipsets in 1.0 mode that we have yet to isolate, but attempting to do so is on my TODO list. :\ I think we should actually open a new bug to track the Via firewire controller problem -- I'll do so shortly and add you to the cc list. Anyone else with a Via controller is welcome to tag along, of course...
Okay, I've opened bug 415841 to track the problem with Via controllers.
Re comment #42: The Kino crash is unrelated to the issue with VIA cards. I get it also with cards which work with dvgrab. Maybe I'll find something but this may take a while since I never really worked with the libraries' code yet.
I guess VT6306 could be switched to OHCI 1.1 as well. We just need someone with a spare VT6306 card to confirm.
> I guess VT6306 could be switched to OHCI 1.1 as well. > We just need someone with a spare VT6306 card to confirm. I shall try this eventually... Today somebody posted at libdc1394-devel that he has got a VT6306 card in OHCI 1.1 mode, so it should indeed be possible.