Browser Vendors Force W3C To Scrap HTML 5 Codecs 640
snydeq writes "Major browser vendors have been unable to agree on an encoding format they will support in their products, forcing the W3C to drop audio and video codecs from HTML 5, the forthcoming W3C spec that has been viewed as a threat to Flash, Silverlight, and similar technologies. 'After an inordinate amount of discussions on the situation, I have reluctantly come to the conclusion that there is no suitable codec that all vendors are willing to implement and ship,' HTML 5 editor Ian Hickson wrote to the whatwg mailing list. Apple, for its part, won't support Ogg Theora in QuickTime, expressing concerns over patents despite the fact that the codec can be used royalty-free. Opera and Mozilla oppose using H.264 due to licensing and distribution issues. Google has similar reservations, despite already using H.264 and Ogg Theora in Chrome. Microsoft has made no commitment to support <video>."
Apple? (Score:4, Interesting)
Re:Apple's concern (Score:5, Interesting)
No, if something being royalty-free were a downside they would not have included a BSD userspace with OS X. While Ogg Theora is royalty free, there are no -known- patent violations. As I recall back when Vorbis was getting off the ground, the implication was made that people with patents wouldn't care unless it got off the ground and then they would start looking for violations.
Basically, Theora and Vorbis are huge unknowns with potential patent bombs in them, regardless of what the developers and /. thinks. And all it takes is someone with a patent and the muster to enforce it and everyone who implemented them in their browser suddenly has a huge problem.
Re:Why do the vendors have a say? (Score:4, Interesting)
I agree. Mozilla have supported Ogg Vorbis and Theora as of 3.5 and it works pretty good from the demos I have used. The W3C needs to ignore everyone and push forward with Ogg support in the spec. If hardware acceleration is a problem then work with companies to get it supported in hardware. I know it won't be easy but saying "ugh that is gonna be too hard, lets just drop it from the spec" is stupid, work with Nvidia and ATI and Intel, etc. to get h/w support for Ogg. I am not a specialist so I have no idea how hard it would be to get h/w support for Ogg up and running but I know that my iRiver H10 mp3 player had Ogg support back in 2003 or so, so I am sure it is possible without _too_ much work.
A solution: system codecs. (Score:5, Interesting)
How about making the browser use system (DirectShow on Windows, whatever-it's-called on Linux) codecs, so everybody could be using whatever codec they want. Look, a lot of media players on Windows (like WMP and MPC) use DirectShow, so thew users can install additional codecs.
Why they want to include the codecs in the browsers. This way is worse. If system codecs were used, then the sites could choose whether to use h.264, ogg or some other codec, like XviD.
Also, this way all of the patent/license/whatever issues for the browser vendors would go away. And if the users are watching video files on their computers they most likely have codecs already installed.
Then html5 wont exist (Score:3, Interesting)
If no browser will support the codecs then webmasters wont use html 5 and stick with html4. When IE owned a significant marketshare a couple of years ago the web evolution slowed down to a halt. Firefox can't adopt H.264 because its patented and Firefox can be shutdown if a lawsuit over infringement takes place.
And Firefox does not have a significant enough marketshare for developers to care about Ogg Vorbis/Theora. Besides all the professional tools do not support it so it wont ever be used. It wont ever be used because professional tools do not support. Its a catch-22 just like Microsoft Windows and Office. You can't ever leave the platform.
If silverlight and flash work on 95% of the market why switch?
Re:Apple? (Score:2, Interesting)
Because Apple wants to create a monopoly with file formats. Supporting Theora would lower barriers to entry for competitors running on Windows to compete with them.
Re:Why do the vendors have a say? (Score:3, Interesting)
In fact I wouldn't mind if California politicians learned this lesson too, since they seem to have trouble in the compromise area a lot.
The real reasons... (Score:5, Interesting)
Vendors never actually mean what they say. Here are the real reasons:
Apple won't support a codec that's incompatible with its huge installed base of ipods and iphones. They don't care about royalty fees because most Safari users pay for an OS X licence, and they want the free browsers to look sub-par compared with theirs.
Microsoft won't support a codec that makes the web more reliable for non-Windows users - especially Linux users. They don't care about royalty fees because all IE users pay for a WIndows licence, and they want the free browsers to look sub-par compared with theirs.
Google, Opera and Mozilla won't support anything that puts them at risk of needing to pay royalties on the huge number of free downloads they give away.
Nobody actually cares about end users or developers. If you think they do, you're kidding yourself.
Re:Things to learn from the Open Source model (Score:5, Interesting)
To be fair, Google is also refusing to switch YouTube to Ogg because of its lower quality per bitrate than h.264.
As was argued by the original author, you're left in a situation where if Ogg were specified in the standard, you'd have folks who followed the standard at a disadvantage in quality and/or bitrate.
Besides, W3C doesn't say which image file formats are allowable, why should it specify a codec?
Fuck Apple, but why Java? (Score:5, Interesting)
Fuck you Microsoft. Die already!
Fuck you Adobe. Die already!
Fuck you Java. Die already!
Fuck you too Realnetworks. Just because.
Not "Just because". Fuck Real for producing crappy software that doesn't fit in anywhere at makes it annoyingly non-trivial to download things I want to watch.
Fuck Adobe for Flash. Seriously, I don't need vector graphics in my web browser. I'd love to have embedded .wmv/.avi/.mpeg files, whatever, because I can play those with mplayer which DOES NOT SUCK. As opposed to flash.
Fuck Microsoft for being the great browser market retardant. And in general for writing shitty software which doesn't do what I want it to (heck, I can't even get XP to install; epic fail).
And fuck Apple for being such control freaks. Well, first, fuck 'em for not helping fix this browser shit. Secondly, fuck them for being a worse control freak than Microsoft could ever be. I recently played with an iPhone (display/sales demo); among the top 25 apps in the store is one that displays scantily clad women, which are "as naked as Apple will let us get away with". FFS, Apple. Don't decide whether I'm going to watch porn on my phone. And you include a web browser---is that porn-filtered too? Assholes.
But don't fuck with Java. It's free software. It works for what it does: sorting algorithm animations and interactive Rubik's cube algorithm display. Java is OK, when used in moderation.
Flame on ;-)
Re:Apple's concern (Score:4, Interesting)
> While Ogg Theora is royalty free, there are no -known- patent violations.
The exact same argument can be made for the BSD base Apple uses for OSX. It doesn't matter that BSD went through a long copyright case way back when; both because that case was about copyrights rather than patents, and because unknown patent violations can easily have crept into the code base since then. In fact, I can safely go out on a limb and guarantee that every non-trivial piece of software (including everything Apple has) is violating software patents. Software patents are handed out by the USPTO like Bibles are handed out in prison.
Apple's argument that they won't use Theora because of potential patent problems rings completely hollow. I'm not going to speculate on their motives, but the one they gave is nonsense.
Re:Things to learn from the Open Source model (Score:3, Interesting)
uh? Lots of companies are not stupid about proprietary crap being obsolete. They are moving away by the droves, and such publication is deliberately hidden from the media due to Microsoft essentially owning the media (you'd be surprised).
Once licensing is done for, for many products, you'll see lots of people switching. Examples of this software are things like Office 2K7. If there's a version released after that, everyone will swap to openoffice as they've already been planning/preparing at an enterprise level. Give it about 5 years and there won't be much proprietary left. Companies that are established understand "in house" costs vs "pay an enterprise and fork over tons extra" in the long run, and moreso due to the economy at large.
Remember, it is not IT, even CIO's, are not the people that make the decisions, it is the business sector. Every part of a company makes the argument of "we're losing money every second we don't do XYZ" but when you can say "we can put things with existing costs down to 0 and make ourselves no longer legally liable" thats when you start speaking to the businessfolks (and get a job in consulting).
Re:A solution: system codecs. (Score:3, Interesting)
You might be interested in http://weblogs.mozillazine.org/roc/archives/2009/06/directshow_and.html [mozillazine.org]
Re:Things to learn from the Open Source model (Score:2, Interesting)
Why?
iPhone = 2Gb data at $60 a month
celphone = unlimited data at $15 a month
Now, I am in Canada so maybe it is different here but to me it is a no-brainer.
Re:Things to learn from the Open Source model (Score:3, Interesting)
>Apple doesn't want to be the deep pocketed commercial implementation of Ogg that ends up having to pay patent
>trolls. That's why it is going with the ISO/MPEG standard, which pools patents together from everyone.
The patent pools provide ZERO protection against patent trolls.
Several people got sued DESPITE paying for patent licenses to the MPEG patent pool. The MPEG LA provides no guarantee they cover all patents applying to their technologies.
Audio video codecs are outside the scope of HTML (Score:3, Interesting)
Audio video codecs are outside the scope of HTML. Whatever it says in the HTML 5 spec about video codecs, that will not magically change the last 20 years of digital audio video away from MPEG to something else.
The current audio video standard is ISO MPEG-4 (2001) and the codecs are H.264/AAC. Supporting this standard is not an academic issue because the world is full of content as well as hardware and software players and authoring tools that conform to this standard. It's also the video in Flash and in YouTube, which is considered the de facto standard in "Web video". When people talk about Web video they usually mean YouTube or something very like it. They are talking about MPEG-4.
The MPEG-4 content that you find in the world and on the Web today includes:
- every song ever offered for sale in or purchased from iTunes Store
- every song ripped from a CD by iTunes since 2002
- every video ever made on a cell phone (3GPP is part of MPEG-4) including the iPhone's recent shoot, edit, upload to YouTube feature which is H.264/AAC
- every video on YouTube is stored as MPEG-4 (no matter what format you originally uploaded)
- almost all of the video that runs in Adobe Flash, excluding 320x240 movies which may be the old codec
- all of the consumer video shot on solid state storage, and most of it from a few years before that
- all Podcast video is H.264 and most Podcast audio is AAC
- Blu-Ray
Nobody has explained how all of this content would be transcoded to Ogg or other non-standard format in order to be published on the Web. Where would the computing time come from? How would it be practically done? What are you going to tell someone who wants to upload a video from their camera or phone directly to the Web? That they should transcode it into a non-standard audio video codec first?
The players are very important also, because they have H.264/AAC decoding HARDWARE, which enables them to work efficiently enough to run on batteries. You can't drop a new software codec into these, you have to drop in a replacement audio video decoder chip. These include:
- every iPod and all of their competitors, except for the ones that only play MP3 which is part of MPEG-2
- every PC with a recent NVIDIA GPU can decode H.264/AAC without breaking a sweat or busting its batteries because it happens in the GPU
- Internet set-top boxes such as AppleTV and Netflix
- PlayStation3 and other game boxes
- even the Zune has MPEG-4 hardware in it, although somewhat underutilized from what I hear
Even software players cannot so easily be modified to support a non-standard codec, because of the scope of the MPEG-4 support. We're talking about every Mac and every PC in the world, because they all have one or both of these:
- every QuickTime/iTunes since 2002 is MPEG-4
- every Adobe FlashPlayer version 9 or 10 is MPEG-4
The reason those 2 match both each other and all the hardware players is because of the benefits of standardization, which took place almost a decade ago for MPEG-4 and goes back further to previous MPEG versions. If you, or Mozilla, or anyone, wants to make an audio video player, they only need to conform to the MPEG-4 standard to enable their player to play all of the content from QuickTime/iTunes and Flash. You can come along in 2009 and decide to get your feet wet in audio video players and simply by following a published ISO specification you can have instant equality with QuickTime and Flash and others. Again, the benefit of standardization.
A very important consideration that is often completely ignored by Web-centric people as they talk about audio video is the authoring tools! People who make audio and video all day long also want to publish their work on the Web. MPEG-4 is standardized QuickTime, so there is not just 8 years of MPEG-4 authoring tools right now, there is almost 18 years of digital audio video practice realized in MPEG-4. A key feature here is that these tools must not make content that has a "content tax" on it, like
Re:Things to learn from the Open Source model (Score:2, Interesting)
W3C doesn't say which image formats are allowed? (Score:2, Interesting)
src = uri [CT]
This attribute specifies the location of the image resource. Examples of widely recognized image formats include GIF, JPEG, and PNG.
Now true, that doesn't say that any formats are recommended, well at least not until you head to the W3C PNG specification:
http://www.w3.org/Graphics/PNG/ [w3.org]
They also have a nice section on SVG:
http://www.w3.org/Graphics/SVG/ [w3.org]
Re:Why do the vendors have a say? (Score:3, Interesting)
I suspect that the problem is that companies like Apple, Microsoft, and Adobe have enough influence on the W3C to kill something like this.
The W3C is irrelevant. The WHATWG didn't start out as part of the W3C, and if the W3C tried to push it around it could just break off again. The contents of the HTML 5 spec are determined solely by Ian Hickson, currently employed by Google. His only oversight is a steering committee. I can't find who's on the steering committee, but I'm very certain that it includes no one from Microsoft or Adobe, and Mozilla plus Opera almost certainly have more votes than Apple.
The fact is, the HTML 5 standard is not meant to dictate anything, because that doesn't work. It's a forum for browser vendors to coordinate new features, and it documents the features that are agreed upon. If implementers refuse to implement it, the spec doesn't include it. That's how it works for everything, not just video. Try subscribing to the whatwg mailing list to see how it works.
Re:H.264 Theora: a demo (Score:3, Interesting)
Well, it's not like YouTube sells more ads with better looking video, and I doubt 90% of the uploads get watched more than a dozen times. They probably have some pretty deep metrics about the watts/cents per minute of video encoding and tune for that.
YouTube is also really only a good example of YouTube, since they're a massively money-losing operation funded by a very rich company. No one else does it like YouTube, and ever other video site is going to average a lot higher views/clip, so they can afford more CPU time to improve quality.
Or maybe they're just not very good at video compression :).
Beyond B-frames, they're not using 8x8 blocks or CABAC entropy coding, both of which can offer substantial efficiency improvements.
Re:Things to learn from the Open Source model (Score:3, Interesting)
Re:Things to learn from the Open Source model (Score:3, Interesting)
If closed source was ever done fully, we'd all be using IE 6 or something, no wouldn't we?
Wait, what?
Re:H.264 Theora: a demo (Score:5, Interesting)
Well, YouTube has three sets of settings:
Low bitrate H.263 + MP3
HQ bitrate H.264 + AAC-LC
HD bitrate H.264 + AAC-LC
The low bitrate, for whatever reason is keeping to the specs they've been using since launch, which are using the xvid implementation of old Sorenson Spark H.263 v1/MPEG-4 Part 2 Short Header. Maybe for device compatibility? Anyway, That's a codec about as old as the Theora bitstream, so we wouldn't expect it to be much better.
But I don't know that YouTube thinks it's "good enough" - they're offering higher quality modes, and that's what you get by default on the iPhone and other platforms. For whatever reason they're keeping around a legacy version, likely backwards comaptibility with some clients that don't do H.264 for whatever reason.
For the their high quality streams, Theora isn't competitive in quality. And for the highly compatible streams, Theora isn't competitive in compatibility.
So YouTube saying that Theora doesn't make sense for them makes sense to me. Therora doesn't an advantage in quality or compatibility for the streams they're doing.
Also, Big Buck Bunny isn't the best clip to extrapolate from, as it's really high quality lossless animation. To really see what YouTube needs to handle, try some lousy webcam, DV, and VOB rips. That's where H.264's in-loop deblocking filter give it a big advantage over other codecs, because it just gets smoother intead of blocky as the content gets more challenging.
Not to dismiss the excellent development work Xiph has done on Theora. The posts have been a fascinating read. But it's not plausible to me that anyone can make a business case for Theora over H.264, VC-1, or ASP licensing is available; the reduced bandwidth costs would be bigger than the actual real-world licensing fees for the real world examples I've thought of.
Theora's sweet spot would be in cases where MPEG-LA codec licenses simply aren't available for whatever reason. I imagine a fully refined Theora decoder would need fewer MIPS/pixel than H.264 High Profile, and perhaps even Baseline. But even in those cases, VC-1 Main Profile will probably offer similar performance with significantly better efficiency.
Re:Fuck Apple, but why Java? (Score:3, Interesting)
No, but they have an advantage: they're done right :)
(is my first, not-so-thorough impression)
Re:H.264 Theora: a demo (Score:3, Interesting)
You sure do put a lot of energy into slagging Ogg, and you consistently neglect mention the advantage Ogg has over H.264: it is unencumbered by patents and therefore free for anybody to encode and/or play, on any hardware they wish.
Oh, I don't have anything against Theora per se, nor Ogg in general. It's just people keep having highly unrealistic hopes for what it can do in terms of compression efficiency and ecossytem.
Codecs are hard, and it does no one any benefit to assume they're capable of things they simply aren't.
The Xiph blog posts on their optimization process for Theora have been excellent reading, and they've done really good work. But the bitstream itself simply isn't capable of what modern codecs are capable of already. I'm sure Theroa will continue to improve, but I don't see any reaosn why H.264 won't see improvements at least as quickly.
And H.265 is already in development, targeting new bistream features that will add further substantial efficiency improvements.
I for one, am perfectly happy to burn a little extra bandwidth for that, and anyway I not buy your assertion that Ogg cannot close the bandwidth gap over time. .
After all, you are a Microsoftie with a vested interest in keeping video proprietary
Eh, I work on Silverlight, where we have the Raw AV pipeline for managed code decoders. It's be trivial for any customer add Theora support to Silverlight if they want it. If anything, Theora would be a competitive advantage for Silverlight.
Also, I don't think anyone is talking about propritary codecs here, except for perhaps VP6. VC-1 and H.264 are both international standards, with licensing handled by MPEG-LA. They are patent encumbered, but are not propritary any more than MP3 or ASP are.
Re:Things to learn from the Open Source model (Score:3, Interesting)
Ogg use on the internet is a rounding error at best;
This is totally irrelevant. HTML5 is a new standard and can use whatever is best for the job, not what was popular for whatever reason before. Because otherwise we should use the crappy Sorenson H.263 (old flash codec), it's probably still the #1 codec on the web.
The fact is that for something so important and so widely used as the web, it is indispensable that the standard can be freely used by everyone, and not controlled by whoever happens to have the patents for some video format. Freedom of the web is much much much more important than picture quality. Let alone that if ogg is used in HTML5, it will attract a lot of research and very quickly we'll have a high quality, free to use video format.
Btw, your codec list is misleading. Flash does support H.264 but still its old format (H.263) is more widely used. Moreover it supports HTTP and 99% of the video sites (including youtube) use HTTP to serve videos. RTMP is used for "real" streaming with Flash Media Server (Adobe's streaming server that few people use due to high price) and RTMPe that you mention, is the encrypted version of RTMP, used by very very few.
Re:Things to learn from the Open Source model (Score:3, Interesting)
This is totally irrelevant. HTML5 is a new standard and can use whatever is best for the job, not what was popular for whatever reason before. Because otherwise we should use the crappy Sorenson H.263 (old flash codec), it's probably still the #1 codec on the web.
Not if you care about playback on devices like phones, media players, and Netbooks which really count on having hardware decode for media playback.
The fact is that for something so important and so widely used as the web, it is indispensable that the standard can be freely used by everyone, and not controlled by whoever happens to have the patents for some video format.
Well, I don't think H.264 or any MPEG-LA codec can be said to be "controlled" - it's RAND licensing and available to anyone. It's not free to implement, but it's not propritary either. Lots of ISO standards are like that.
Freedom of the web is much much much more important than picture quality. Let alone that if ogg is used in HTML5, it will attract a lot of research and very quickly we'll have a high quality, free to use video format.
Well, there's plenty of companies who care a whole lot about quality and delivery costs. Theora is certainly capable of high quality; it can do that already with sufficient bitrates.
The real challenge is in compression efficiency, and that's fundamentally constrained by the bistream syntax. Optimizations can converge on the theoretical limits of a codec, but those are going to be a lot lower than H.264, and lower yet than H.265 in a few more years. And while Theora may get a lot of attention and tuning, H.264 is already getting LOTS of that from multiple vendors and groups competing to build the best implementations. H.264 is getting better at least as quickly as Theora is.
But certainly, Theora is already more than good enough for plenty of tasks. Wikipedia is committed to using it, and for short clips of smaller frame size, it'll be fine. It's going to be better than MS MPEG-4v1 and the original RealVideo codec that I had to use at the birth of web video, and there's much more bandwidth available as well.
But H.264 High Profile will be able to deliver equivalent quality to Theora at a half to a third the bitrate, and broader compatibilty with existing devices. How important those considerations are will vary by market, but are very important to some big markets. YouTube and Hulu certainly aren't going to double their bandwidth budget and end-user connection speed requirements.
Btw, your codec list is misleading. Flash does support H.264 but still its old format (H.263) is more widely used. Moreover it supports HTTP and 99% of the video sites (including youtube) use HTTP to serve videos. RTMP is used for "real" streaming with Flash Media Server (Adobe's streaming server that few people use due to high price) and RTMPe that you mention, is the encrypted version of RTMP, used by very very few.
Now that YouTube is using H.264, I'm sure that the eyeball-hours of H.264-in-Flash are a lot higher than H.263-in-Flash today. VP6 versus H.264 is the more interesting competition. YouTube hasn't used it, but most of the big Flash media sites like Hulu skipped H.263 entirely and started with VP6.