Created attachment 605705 nsBuiltinDecoder* based implementation tl;dr: I'm attaching a new, simpler patch that adds gst decoding support to firefox, based on the recent(ish) gecko nsBuiltinDecoder* interfaces. Hopefully this version will be easier to review, merge and maintain.
I'm a gst developer and I deal with embedded devices a lot. I recently started working on this patch as I'm interested in porting b2g to new devices. Also more generally, i'd like to see firefox do media playback as efficiently as possible on embedded devices, making use of gst's hw support. Two weeks ago, after the GStreamer SDK effort was announced (http://is.gd/tteunv), I decided to pick up the existing firefox gst patch. After getting familiar with it and the gecko media code, and after an half failed rebase attempt (content/media/ has changed significantly and the patch has bitrotten a bit), I decided to write a new gst decoder based on the recent nsBuiltinDecoder* interfaces in gecko. My approach is deliberately simpler than the older patch. My goal was to just get hw decoding support through gst, delegating the rest of the work (i/o, synchronization, rendering, state management etc) to the existing media code like the WebM and OGG decoders do. The code is self contained and 99% of it is in the nsGStreamerReader class in content/media/gstreamer/nsGStreamerReader.cpp. In terms of lines of code and complexity, it's comparable to nsWebMReader and nsOggReader. Internally it uses the following gst pipeline: appsrc ! decodebin2 name=d d. ! ffmpegcolorspace ! video/x-raw-yuv,format=I420 ! appsink d. ! audioconvert ! audioresample ! appsink. In english: it feeds data from gecko to gstreamer using appsrc, decodes using the decodebin2 element and feeds a/v back to firefox using appsink elements. The patch adds an --enable-gstreamer opt-in switch to configure. When enabled, it currently takes over WebM and OGG decoding and adds H264 support (see my closing note efore starting flames on this please). Eventually i think i'd like to make it takeover WebM and OGG decoding depending on runtime prefs. I tested the patch on OSX. I've got accelerated h264 decoding (supported by the gst applemedia plugin) working but it requires a colorspace conversion currently, which I plan to fix soonish in the plugin. After that, my goal is to make firefox mobile do hw accelerated decoding on android using the gst-omx plugin. There are a few known bugs at this point, the major one being that I need to implement efficient buffer allocation. Also the the bytes-read, frames-parsed/decoded stats are not implemented yet. Being my first firefox patch, there are probably also plenty of stylistic fixes needed. Anyway, overall I think it's good for a first review. Please note: i don't want to get into the argument of whether firefox should support h264 decoding or not. The patch currently adds h264 decoding only so that I could test and demo accelerated decoding on OSX. I think decoding using gst has merits regardless of whether h264 ends up being supported or not. I'm more than willing to remove h264 support from the patch if that makes it easier to get it merged. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/412647 Title: Firefox is not able to play mp4 <video> tags To manage notifications about this bug go to: https://bugs.launchpad.net/firefox/+bug/412647/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
