VP8 Decoder Implemented In Flash Using Alchemy

An anonymous reader writes "Mozilla's Chris Double has an interesting post on his blog about a port of the VP8 decoder to Flash. He writes, 'Ralph Hauwert has been posting on twitter about work he's done on getting WebM decoding to work by compiling the libvpx source code using Adobe's Alchemy technology. Alchemy is a research project that allows compilation of C and C++ libraries into code that runs on the ActionScript virtual machine used by Flash.' Of course, it's very slow and Adobe says that they'll bring native VP8 support to Flash in due course, but implementing a VP8 decoder in ActionScript is an interesting project nonetheless."

So let's see:

[commodore64_love]

WebM is the container
--- VP8 is the video
--- Vorbis is the audio

Why didn't Google use Ogg for the container? I see Google's also developed a WebP format for pictures, also based on VP8.

Re:

[Samantha Wright]

I'm too busy to look up the details exactly, but two possibilities:

1. The container, as well as the codecs, are potentially patent-encumbered, so it's easier to steer clear.

2. The container isn't part of the patent concern, but there are companies who aren't sure of that (see #1). So it's easier to get adoption if they steer clear.

Re:

[Sylak]

OGG container is patent-free, but it's probably the issue with most open-source adoption, where because it's pre-existing and has is open-source, they don't trust it

Re:

[davester666]

Well, you can say it's "patent-free", but that's a meaningless phrase.

To legally be able to say "X is patent-free", you would need to, in every different patent/legal jurisdiction[the matrix of where a patent can be registered and the areas where different laws apply to that patent], you proactively go through court proceedings and prove EVERY single current and pending patent that hasn't expired doesn't apply to X, and have the judge rule in your favor, then go through all appeals, if any.

Assuming of cours

Re:

[commodore64_love]

MPEG4/h264 vs. VP8 comparison (h264 slightly better): http://compression.ru/video/codec_comparison/h264_2010/vp8_vs_h264.html [compression.ru]

HE-AACplus vs. Vorbis (HEAAC wins): http://listening-tests.hydrogenaudio.org/sebastian/mf-48-1/results.htm [hydrogenaudio.org]

WebP vs. JPEG (WebP wins): http://englishhard.com/2010/10/01/real-world-analysis-of-googles-webp-versus-jpg/ [englishhard.com]

Re:

[commodore64_love]

by russryan
Also, http://x264dev.multimedia.cx/archives/377 [multimedia.cx] for a VP8 tear down.

"Overall, VP8 appears to be significantly weaker than H.264 compression-wise. The primary weaknesses mentioned above are the lack of proper adaptive quantization, lack of B-frames, lack of an 8×8 transform, and non-adaptive loop filter. With this in mind, I expect VP8 to be more comparable to VC-1 or H.264 Baseline Profile than with H.264."

Ouch. You're a brave man to publish that on /.

Re:

[RocketRabbit]

Notice that your comment and the post you're referring to are both at 0 now.

Slashdot: The Unofficial PR Branch of Google(TM).

Re:

[arose]

It's not a tear down, it's a "this is how it's similar, but worse than H.264". This little gem makes it quite clear:

Update: Alt-ref frames can apparently be used to partially replicate the lack of B-frames. It’s not nearly as good, but it can get at least some of the benefit without actual B-frames.

AKA, I didn't look at them or pondered what they might be fore and now that I've been pointed out that they exist I won't consider them on their own merits, but just as an inferior B-frame.

Re:So let's see:

[Anonymous Coward]

I take issue with the HEAACplus vs Vorbis comparison you chose.

At the sort of bitrates that are actually used in streaming HD video all lossy encoders are essentially transparent, even old war horses like MP3. Performance at lower bitrates is academic at best. Vorbis beats all comers in the 80-96kbps range, and HEAACplus beats all comers under 64kbps, but neither is relevant to HD streaming video.

Re:

[cpu6502]

Performance at lower bitrates is academic at best.

Not if you're on dialup or a cellphone. You want the lowest bitrate possible while still keeping good quality. For example the radio station I listen to at work streams at a mere 12kbit/s HEAAC and sounds... well not great but close to FM quality. Vorbis sounds like it was squeezed through a shortwave radio.

"Vorbis beats all comers in the 80-96kbps range"

It doesn't beat HE-AAC (AACplus) which still has better sound on the high frequencies.

Re:

[arose]

And yet JPEG is good enough (that and JPEG2000 blurs like no tomorrow) to the extent that none of the propositions have been given a second look for widespread web usage. And this is the biggest evidence for why quality is not that critical when picking what to use for web video.

Re:

[AvitarX]

I don't know why .ogg was not used, as according to Matroska.com this is where .ogg excels.

MKV is definitely superior to OGM though (Matroska is modest on this point int their faq).

look at the table here: http://www.alexander-noe.com/video/documentation/containers.pdf [alexander-noe.com] to see MKV vs OGM

4.3 in the linked PDF is probably why WebM was created (a pure subset of Matroska I believe).

I can't find any support, but I believe I read somewhere that OGM was week in seeking vs MKV (don't know how this compares to WebM).

Re:So let's see:

[BZ]

http://lachy.id.au/log/2010/05/webm [lachy.id.au] under "Benefits of Matroska" describes the seeking issues in some detail. The summary is that ogg requires you to read more separate bits to seek correctly, and each separate bit ends up having to be a separate HTTP request in the context of web streaming. So your latency starts to bite you when seeking. It's not an issue if you don't have to seek or if you have the whole file already. But for the youtube use case, neither is true, typically.

Re:

[Goaway]

"WebM" is not really the container, it's an overall name for VP8 and Vorbis in a container that is a subset of the MKV format.

Also, they probably did not choose Ogg because it is a bit of a horrible mess and programmers in general don't like it at all.

Re:

[Bobakitoo]

http://www.matroska.org/news/webm-matroska.html

webm stands for Web Media (not Web Matroska). It is a subset of Matroska with only the necessary features required for video (and audio) to play over the web. As of today, it is supported by web browsers like Chrome, Firefox and Opera. Some video websites like YouTube are already streaming 360p and 720p in the webm format using the HTML5 feature of these browsers. Our tools, like mkvalidator and mkclean, already support webm files; the new version of Haali's Media Splitter can handle webm files; and support in mkvtoolnix has already been added to the next release.

So it is simpler to implement on lighter devices like phone perhaps..

Re:

[badkarmadayaccount]

To put it shortly - Ogg sucks, and Matroska rules. Compare the two formats and you will see why. Matroska allows encoding almost any data and synchronizing it in the stream completely forward and backwards compatibly, with no need to develop bit-field mappings, unlike Ogg.

Re:

[Rockoon]

VP8 isnt a standard.

Google has had a year to submit VP8 to a standards organization, but hasn't done so, so its Googles fault that it isnt even on the road to being a standard.

Instead, VP8 is a "here is the source" codec.

It isnt the FORMAT that is patented, but instead the METHODOLOGY.

Essentially, ONLY that reference implementation is owned by Google. Make a change that accelerates decoding (while remaining compatible) and you might just land smack dab in the middle of not "free." Have a look at the

Re:

[tuppe666]

As far as I am aware W3C the organisation that looks after web standards has given no codec the green light. I cannot think of a better way of showing a standard than a working implementation...Something OOXML that standard that got through by corruption and still has no working implementation. To be fair though the vast majority of browsers using the Video tag will be WebM which is next month. Which is what I would call a internet standard.

Re:

[Rockoon]

Except that the method is the format. Without the format conversion, all you have are semi-random bits.

Not a programmer, are you? You do realize that there is more than one way to implement something identical, right? No? Yeah... thats how I know that you arent a programmer.

That DOES NOT MAKE IT NONSTANDARD.

Nobody said that it did. You, however, seem to think that source code makes a standard.

Plus, I suspect that you're wrong. There are several implementations of the standard.

If they are not identical, then only one of which is owned by Google and as such only one of which is covered by Googles "we wont sue you, unless.." terms.

The other could contain patented algorithms not owned by Google and STILL DECODE THE SAME STR

Re:

[tuppe666]

Thats wonderful news. So your saying that the code can be optimised, by hardware and software companies using different methods including patented ones.

Are you saying that if the FUD about WebM a format that predates H.264 really were to come into being it could be coded around ;)

They have licensed it BSD, because thats how you get adoption of the format. Thats what they want. Thats the only way they can succeeded...are you really trying to imply that Google is going to sue anyone. I suspect that with H.264

Re:

[Rysc]

Essentially, ONLY that reference implementation is owned by Google

ffmpeg has an independent implementation of the decoder, but for the encoder you are correct.

The main problem with vp8 isn't that it hasn't been submitted to a standards body. That's trivial detail. The problem is that it's not meticulously documented, which means that conforming implementations are difficult. If I implement an encoder in a way which contradicts the reference implementation but does not contradict the documentation, does that make my implementation wrong, the documentation wrong, or the ref

Re:

[Rockoon]

ffmpeg has an independent implementation of the decoder, but for the encoder you are correct.

..which may, or may not, violate patents not covered by the ones Google acquired.

Some people seem to be having trouble with this concept, so I'll use a little analogy.

I can patent a sorting method.. lets call it RockoonSort(). I can develop a compression algorithm that requires sorting and use my sorting method in it.

I cannot patent sorting itself, so you could go ahead and avoid my patent by using qsort(), heapsort(), mergesort(), bogosort(), ...

All of these video codecs use arithmetic coding, and

Re:

[Rockoon]

I'm not quite getting your point here. Is your argument that at least with H.264 we know we're fucked? That we should just give up on open and royalty-free standards and open source until the U.S. gets a patent reform?

Sigh.. I'm saying that any re-implementation that isnt the reference is very likely to infringe on patents.

Are you just too stubborn to admit it to yourself that you have to pretend to struggle with this idea?

Take a look at H.264's patent pool sometime. Many methods of implementing the same equivalent thing are in there for a reason, so that implementors have options. VP8 gives no options.

Why is this so hard?

Re:If video tag meant H.264, internet dies.

[makomk]

Google has had a year to submit VP8 to a standards organization, but hasn't done so, so its Googles fault that it isnt even on the road to being a standard.

The relevant standards bodies have no interest in something like VP8. VP8 is designed not to be patent-encumbered, whereas standards bodies like the MPEG group aim to create video codecs that are as technically advanced as possible and that are encumbered by as many of their members' patents as possible. (The patent encumbrance isn't just a side-effect of wanting the best codec possible - member organisations deliberately try and make sure that their patents are an unavoidable requirement of implementing the standard, because they directly benefit from this.)

Standards bodies are not a panacea, and they have known issues. This is one of those issue.

Essentially, ONLY that reference implementation is owned by Google. Make a change that accelerates decoding (while remaining compatible) and you might just land smack dab in the middle of not "free."

That's true of basically everything in the software industry, including h.264 itself. Remember that the h.264 patent consortium does not even guarantee it owns every patent that affects h.264, let alone patents covering particular implementations. It's still better than h.264 though - that's designed so that you can't write a compliant decoder or encoder at all without infringing patents, because the specification was written by the patent owners.

Re:

[Rockoon]

If only one of these patent pools would patent multiple methods of doing the same thing so that implementors would have a choice of algorithms likely to be efficient on their hardware.

Oh.. wait.. MPEG-LA did exactly that.

Re:

[TheSync]

The relevant standards bodies have no interest in something like VP8.

SMPTE [wikipedia.org] is an individual membership organization. I'm sure if an individual brought VP8 before SMPTE for standardization it would happen. SMPTE has a mixture of broadcast vendors and broadcast users of all kinds. There are some folks in SMPTE who are from companies with compression IP, but there are also plenty of other vendors represented in SMPTE that don't.

Of course, it would take time (like when SMPTE Standardized Windows Media Video

Re:

[tuppe666]

Google, Opera, Firefox religious zealots OMG when did this happen. I thought they made these choices for business reasons. Please tell me more about this religion is there handshakes involved. Tell me there is lots of SEX. To be honest those other companies that only allow H.264 format that they make money off the small hardware/software companies as a barrier of entry...are they zealots too!? Tell me more.

Re:

[tuppe666]

They aren't forbidding WebM. So, no, I don't have to tell you more, since you've already walked your own way to idiocy.

Then I look forward to Microsoft and Apple announcing there full backing of WebM using the Video tag rather than funny[sic] bolg posts or silence.

Video tag meant most used codec

[SuperKendall]

If video tag meant H.264, internet dies since you can't participate without the consent of a third party (the patent holders).

The video tag always meant, that we should be able to assume the most widely used codec was in place.

Currently that is h.264, primarily because of hardware support. But in the future it COULD have been VP8, once there was hardware in place and wider support for it.

Instead killing off the video tag means we are LOCKED INTO h.264 in a way that would not have been true if the video ta

Re:

[Draek]

The video tag always meant, that we should be able to assume the most widely used codec was in place.

Erm, no. The video tag always meant that we could assume that at least the codec specified in the standard itself (ie, Theora before Apple's bitchfest) was in place. It never was a popularity competition, and now that Apple managed to remove Theora from the standard and no particular codec is specified, it's all as it was before the days of Flash: a random coin toss as to whether it'll work or not, regardless of which codec(s) you use.

Instead killing off the video tag means we are LOCKED INTO h.264 in a way that would not have been true if the video tag really took off. It means we are LOCKED INTO flash players in a way that would not have been true if the video tag took off.

Don't like h.264? Then you of all people should be more angry at Google than I am, because they ensured the marginalization of WebM and VP8.

No, he should be angry at Apple, as without them there would have been a

Re:

[SuperKendall]

Apple only brought up the point that in order for web designers to accept the tag it had to support a widely used codec.

You can blame Apple all you like but the video tag was doomed when it was Theora only; Apple tried to save it. Google has curb-stomped it real good now, so we can forget about that part of HTML5.

On an amusing sidenote, in Chrome now using Youtube, the WebM player uses twice as much CPU than the native h.264 HTML5 player did, and the Flash player also uses less CPU than the WebM player!

Re

Re:

[arose]

Ah, revisionism. The video tag was at no point "Theora only", as you claim, Theora was suggested as a baseline. That is you could use H.264 or whatever else as you please, just as long as you had the ability to play Theora at all. Apple basically said, "we are not doing that no matter what", and it was dropped because an unimplemented standard is pointless.

Talk to Draek

[SuperKendall]

Ah, revisionism. The video tag was at no point "Theora only", as you claim, Theora was suggested as a baseline.

Whoa there, you are responding to the wrong person. I never claimed that, I merely accepted what Draek said at face value. Correct him if you must correct someone.

The fact that Apple said "we are not doing that" simply goes back to my point that they noted people would not use a codec that didn't already have wide support. They also pointed out the madness of having to require a codec that had z

You don't understand...

[SuperKendall]

Letting Safari play Theora videos would have in no way affected their precious ability to play back H.264, you don't need widespread support to merely implement something.

What happens is you kill battery life in mobile devices if you mandate support for a format no-one supports. It's just a bad idea.

I realize you care not a whit for actual USERS; the ones we all write software for? Someone had to. That's where Apple came in.

A new standard lacking hardware implementations? Inconceivable!

Actually it *is* i

Re:

[Draek]

Sooo... according to you, the reason Apple didn't accept the HTML5 standard was that web developers would be unsure that a codec whose support was required by all HTML5 implementation would be supported by all HTML5 implementations? I'd love it if you could give us a source for that, I could use a laugh.

Sorry, but outside your broken logic, Apple is the reason we're in this mess and it's Google, along with Opera and Mozilla, who are trying to fix the damn problem they created.

And BTW, I've never said that t

Re:

[SuperKendall]

Sooo... according to you, the reason Apple didn't accept the HTML5 standard was that web developers would be unsure that a codec whose support was required by all HTML5 implementation would be supported by all HTML5 implementations?

Yes, because if you think about it that's exactly why you would do that.

Source: reality. The reality is that at that point h.264 was in wide use, and had good hardware support. If you want a standard to be adopted, which path is better - to select a baseline option that no-one

Re:

[bhcompy]

Now that Google has dropped h.264 support from Chrome, the new reality is that the <video> tag in HTML5 is dead, and pretty much all desktop video will be served in Flash players.

Except that Chrome has rather small share of the browser market.

Re:

[arose]

Why not serve WebM over Flash or Theora via Java to people who don't care about such things?

Re:

[petermgreen]

Why not serve WebM over Flash
did adobe ever implement that? They said they would implement VP8 but i've found no evidence of them actually doing so and afaict they only talked about implmenting VP8 not the rest of webm.

Re:

[xouumalperxe]

Did you even read the title for the article you're posting on?

Re:

[petermgreen]

heh I think I glossed over the article and then came back sometime later to read the comments without remembering what the article was.

However reading the article 1.5FPS does not seem to make for a usable video player, sure you could crank down the resoloution and maybe optimise the code a bit but I'd still be surprised if this ever performs acceptablly.

Re:

[arose]

They haven't yet, but then again, we have an article here about someone who did. I wasn't suggesting everyone should start doing it right now, just that it should be an option, probably by the time the first hardware is rolled out as well. They didn't specify implementing WebM, but the announcement is pretty clear that VP8 will be implemented to support it, it's probably the bulk of the WebM package.

Re:

[tuppe666]

It's not that small anymore, and Firefox doesn't support h.264 either. Between the two of them, they make up a very large segment. The basic problem you have is: FF/Chrome - no h.264 IE/Safari - no WebM

This means you have a strict division of support, and in order to satisfy more than 50% of the market, you'd have to offer both formats. If this doesn't change, unfortunately the video tag will likely die.

Not true. Your trying to imply that their is a 50% split in the market rather than the the uptake over time of the video tag. Its more complicated for a start it doesn't cover Video Content on the net YouTube is going to support WebM lets face it and is the largest Video streaming company on the net.

Now for browsers *currently* the only browsers ignoring beta's that support the video tag are chrome and safari...giving only support for webm to chrome users leaving chrome+safari for h.264, and YouTube servin

Re:

[icebraining]

Now for browsers *currently* the only browsers ignoring beta's that support the video tag are chrome and safari...

No true, Firefox 3.6 and Opera 11.0 (and Chrome) all support the video tag with Ogg Theora.

Re:

[tuppe666]

Sorry that means *currenty* WebM to H.264 split of about 2:1 ratio, and the gap widening further for native support to 3:1 next month and the gap widening further.

I don't think I have made any other glaring errors. I left Opera out as it was complex enough as it was it needs a labelled graph.

Re:

[arose]

No, you very secificaly said this:

Now for browsers *currently* the only browsers ignoring beta's that support the video tag are chrome and safari...

And proceeded with:

and YouTube serving its videos in H.264 format.

If you are going to ignore betas, then ignore the fucking betas, YouTube only serves HTML5 video in beta. Furthermore, it serves both H.264 and WebM there, so....

Re:

[tuppe666]

I've reread my first statement it is perfect. as it states browsers not web sites. I use the connective "and" to describe the content available for them, which primarily is YouTube

Oddly I did not mention it did both video formats were available in YouTube, but going forward it will be in WebM in a video tag with a fallback to WebM in Flash which could be as soon as Feburary.

Re:

[Desler]

So if Market share was the only indication WebM is a long way in the lead...and with YouTubes backing!?

But YouTube is going to be forced to continue supporting H.264 because of the iPhone and Android devices. That is unless Google is planning on either replacing all Android devices with ones that have a VP8 decoder ASIC or making Android phones battery life abysmal by forcing the use of a software VP8 decoder.

Re:

[Shikaku]

http://gs.statcounter.com/#browser-ww-monthly-201012-201012-bar [statcounter.com]

Really? It's 3rd yes, but it's bigger than everyone under it combined.

Re:

[tuppe666]

Now that Google has dropped h.264 support from Chrome, the new reality is that the <video> tag in HTML5 is dead, and pretty much all desktop video will be served in Flash players.

Except that Chrome has rather small share of the browser market.

Chrome has about 10% of the Market though lets be honest its a 50% Split between companies(firefox opera are WebM) who support WebM vs H.264, and a massive majority if you only include those that are cutting edge browsers with supported platforms. With IE9 still with no release date and not working on XP. That and the fact the LARGEST video streaming site on the net Youtube will support it. Ignoring the fact that OS's can provide safe fallbacks. So no not really.

Re:Flash players everywhere thanks to Google

[goombah99]

I just implemented Google Chrome in Flash using Alchemy. Next I'm going to implement Flash in Flash using Alchemy. Then Alchemy in Flash using Alchemy. All on my Amiga console!

Re:

[tuppe666]

Now that Google has dropped h.264 support from Chrome, the new reality is that the <video> tag in HTML5 is dead, and pretty much all desktop video will be served in Flash players.

So it's good they are getting a head start on getting the VP8 codec tuned in Flash, although the practical reality is that for full support in all browsers all you'll have to do is encode in h.264 and call it a day; thus that's all most companies will ever do. You have to encode in h.264 to support video playback on iOS devices which is still a huge segment of the mobile market that uses the internet.

After all Adobe owns Flash, and they have no reason to remove h.264 support now that web designers are being forced to use of Flash players for desktop. So the new steady state for the system is h.264 in Flash players everywhere except for systems that can play h.264 directly.

Your very fortunate. Google still offer support for WebM through Chrome as do Firefox and Opera. Hopefully Microsoft and Apple can get behind this. They have already said its better to speak the same language...and both love openstandards. As you have already stated Safari ;) and IE9(not XP Microsoft have not given them a choice for IE) have native OS fallbacks, Although as you can see those devices that give users the choice of Flash can get WebM in Flash as well.

Re:

[moderatorrater]

What the fuck are you talking about? Take a good hard look at the codecs [wikipedia.org] supported by the big three browsers. Even before Chrome's announcement, there wasn't a single column that was green all the way down. This fragmentation started long before Chrome removed h.264 support.

Interesting

[VincenzoRomano]

as much as useless timewasting project.
I would have preferred spending those resources into making a better/faster/leaner VP8 decoder!

Turing!

[Dan East]

Wow! Flash is Turing Complete! Who woulda thunk it? Now if only it could be implemented in Javascript so as to also be unusable within another interpreted / bytecode environment....

Re:

[drinkypoo]

Wow! Flash is Turing Complete! Who woulda thunk it? Now if only it could be implemented in Javascript so as to also be unusable within another interpreted / bytecode environment....

I see you have a problem using google [ajaxian.com]

Re:

[Shikaku]

Newsflash: companies have self-interests.

In other news, water is wet.

Re:

[iammani]

water is wet.

[Citation Needed]

Re:

[VGPowerlord]

*throws a bucket of water at iammani*

Re:

[Desler]

They won't support H.264 because its proprietary but they went out of there way to support Flash.

And yet they are forced to support it via streaming from Youtube lest they wreak all sorts of havoc on Android devices. That is unless they want to kill the battery usages of Android devices by forcing them to use a software VP8 decoder instead of the H.264 ASIC built into the phone.

Erm...

[interval1066]

Actually, its not that interesting. "Engineer implements random protocol with miscellaneous sdk." The fact that a programmer gets a toolset to do something shouldn't be news. Even if the toolset is used for something it wasn't intended, a well designed toolset should handle unforeseen scenarios with some dexterity anyway. A mildly interesting hack, sure. Headline worthy? No.

What about Silverlight?

[aitan]

I think that one of the features of Silverlight was that it was possible to write a codec in c# and it was fast enough to process video, so that could be an interesting task for someone with enough time: Testing what performs better, decoding vp8 in flash or silverlight, without native support in neither platform

Just a thought

[rawler]

I'm sure someone has already reflected on this, but the thought hit my slow brain at first today:

  [li] A project like Firefox could never have succeeded in a web-landscape where license-payments were needed to implement a web browser.

  [li] Without Firefox, we would most likely still be stuck with IE.

The Meat comes from LLVM, not Alchemy

[tyrione]

http://labs.adobe.com/wiki/index.php/Alchemy:FAQ [adobe.com]

What does Alchemy include?

The Alchemy tool chain includes a set of tools for building, testing and debugging C/C++ projects, example projects and documentation. It is based largely on the LLVM compiler project.

What is LLVM?

LLVM is a set of tools for creating and manipulating "low level virtual machine" bytecode. LLVM includes gcc/g++ based front-ends for converting arbitrary C and C++ code to LLVM bytecode. (http://llvm.org)