SIGGraph and Open Source

SeanCier writes "The SIGGraph 2004 conference showed off a lot of trends: high-dynamic-range (HDR) displays and video, suddenly ubiquitous general-purpose GPU programmability (it's not just for polygon shading anymore), 3D and high-colour displays, ever-more-refined fluid dynamics, crowd animation, and point-based graphics, to name just a few. But there was an unspoken undercurrent, a trend that's waiting to happen in the visual effects community, and happen in a big way: Open Source." Read on for more.

There are plenty of examples of open source and the graphics community getting along grandly: Gimp and CinePaint (aka FilmGimp), ILM's OpenEXR, and projects like Open Scene Graph. Linux, in particular, has made spectacular inroads: nearly everybody uses it for rendering, and many (most?) use it as their desktop OS of choice. In the RenderMan user's group (I'll get into RenderMan more in a minute), for example, somebody asked how many people used Linux as their main OS. Plenty of hands, and some approving chuckles all around. Mac OS X? A few hands, and woots. Windows? No hands at all -- and moreover, an handful of boos, followed by everybody cracking up as they realized the whole community was abandoning Microsoft wholesale.

But then there's the other side. All the major visual effects and animation studios -- ILM, Pixar, Dreamworks, Digital Domain, Blue Sky, Disney, and so on -- have a team of programmers in-house. Five, ten, two dozen, or more. They're the ones that'll write the software that does special rendering algorithms for Shrek 2, or an animation control system for Mr. Incredible, or produce massive crowd simulators for Lord of the Rings. Things that commercial software doesn't quite do -- or that nobody else has tried to do, or even thought of. Things they need to do just so. Things they need to do now.

Everybody has a ton of custom software written -- often good software, with flexible frameworks and clever hacks. Moreover, they don't want to rely any more than necessary on commercial software, because if ILM finds a bug in Maya that holds them up or slows them down, they best they can do is pay Alias to fix it fast (i.e. weeks) and then have hundreds of animators waste thousands of hours time working around it for weeks. And worse, if Digital Domain buys Alias and decides they'll keep new versions of Maya to themselves, ILM is simply screwed, in a big way. If they want to get a particular feature in Maya, and a plugin won't cut it? Well, that's even harder -- and involves more money and more time.

So ILM writes their own stuff whenever they have to, and whenever they can. And Digital Domain writes their own stuff. And Dreamworks writes their own stuff. And Disney writes their own stuff.

And most of it is all the same stuff. Fluid dynamics? Hair? Subsurface scattering? Muscle-and-skin systems? Crowd control? Dozens of topics -- and every studio pretty much has pretty similar, rather redundant code to do 'em all.

These studios aren't in the business of writing software, they're in the business of making movies. So why are they spending their time and money writing software? Because they have to; it's a Necessary Evil.

So, what if they all worked on Open Source stuff instead? Look at what I just wrote. Every word is a reason to go Open Source. No drawbacks, all upside: no lock-in, you can fix stuff, you can add stuff, you don't have to wait on anybody else, and plus, you can do all this while also using what others have written.

The knee-jerk reaction that may be some executives' first objection: our code is a strategic advantage, giving it away would be throwing away money. If we can do hair and our competitors can't, we'll make better films then they can (and, if it's a visual effects studio, we'll win contracts based on that unique ability).

Bull honkus. If your competitors need hair, they'll write hair software, no problem. Another quote from the Pixar RenderMan user's group, this one by a RenderMan developer (paraphrased): "this is based on the subsurface scattering papers from a couple years ago. Everybody does this, based on those papers." Nope, I don't see strategic advantage there: I see waste.

It is, as they say, a win-win scenario; the studios contribute their code to Open Source projects, and everybody helps make that code better. ILM started it in a small way, with OpenEXR, and it worked: OpenEXR is *the* format for high-dynamic-range images, no questions asked. Did it benefit ILM? You betcha: major packages everywhere (Photoshop, RenderMan, etc) either import/export OpenEXR now, or will soon. Pixar even contributed new compression code.

So, a great scenario, and proof that it works. Why hasn't it happened in a bigger way yet? Fear of the unknown. But listen close, and you'll hear a flood coming that could change the landscape -- and it's hard to divert a flood.

That leaves only one question: how will it start? Well, it could begin with open source projects becoming valuable to studios, as started happening with Gimp (though here I'm talking more about advanced 3D animation, simulation, and rendering; Blender's great for what it does, but medium-to-large studios aren't its intended audience; it's not going to displace Maya any time soon, because it doesn't offer anything that Maya lacks as far as the studios are concerned). Or it could start with a studio making a bunch of their custom in-house software Open Source (like ILM did with OpenEXR). Either way, it's up to us as a community -- either to write the software or to sell the concept.

I'd suggest that a great place for all this to start would be with Pixar's PRMan (PhotoRealistic RenderMan, these days often called just RenderMan). And note I say this as a shareholder. Selling RenderMan and related software accounts for less than 5% of Pixar's revenue; the real reason -- the *only* business reason -- they still develop it is for the other 95% of the company to use. If open-sourcing it would bring in collaboration and improvements that would make them just 5% more efficient in generating movie revenue, doesn't that justify the decision right there? And of course that's not counting those who would still pay for service contracts, or the reduction in development costs that could come from the rest of the community helping with their R&D (the budget for which, BTW, surpasses their software revenue). RenderMan has always been a product ahead of its time, and that's why -- despite Pixar's belligerent and hostile use of patents and close-held IP -- it's still the golden standard in this industry. The RenderMan protocol and API was intended fifteen years ago to be a renderer-independent standard, the PostScript of the 3D world. That dream died because of Pixar's unwillingness to release IP: it became difficult or impossible for others to implement that standard officially, or at all, because Pixar grasped the it so tightly (case in point, ExLuna: their lawyers summarily killed what was the best chance in years of having a RenderMan-compliant renderer with new and different functionality, complementary to PRMan). But the renderer -- PRMan -- doesn't have to die through the same mistake, even in the face of an ever-shrinking market share and competitors with the advanced global illumination algorithms PRMan lacks.

But that's not to say Pixar is the only -- or even the best or most likely -- option here. They most certainly don't hold all the cards. So, don't sit back and wait for Pixar or another studio to start the ball rolling: we need to give it a push.

Did You Go To The Panel?

[jakeweston]

There was a panel about the role of custom software development in VFX houses. Though it seems like a good idea on the surface, none of the four big houses represented seemed particularly keen to move towards open source. Most of reasons come down to competition - sure they are all building the same things, but the differences between how well and rapidly they build them determine whether they win contracts over their competitors. Simple business, just as all the big competing auto manufacturers are building the same type of components, but they're not rushing to share their designs...

And the benefits are not as clear as it would seem - the best case seems to be OpenEXR, but the ILM guy was disappointed by the lack of community contributions, that most of the work on the new version had to come from within ILM, and the initial packaging work had cost them more than expected.

Also mentioned were the risks associated by opening their source, particularly the patent issues. I'm sure SCO has persuaded a few companies not to open sources just in case they get involved in that kind of opportunistic farce.

So, in some idealistic collaborative future, a lot could be done with open source, but in the real competitive one, it will be slow progress...

SIGGRAPH 2004 Overview and Open Source in VFX

[bhouston]

A few of us from Frantic Films Software [franticfilms.com] wrote up summary of SIGGRAPH 2004 for CgChannel [cgchannel.com] this past Thursday. It touches on many of the same topics in a slightly different light -- although not at all on open source in the industry.

I understand that open source is a hard sell for VFX companies. Most specifically while at SIGGRAPH I heard Steve Sullivan from ILM [ilm.com] speak (at a discussion panel [siggraph.org]) about how even though they have had many users of OpenEXR [openexr.com] and wide community adoption of the technology they have had very few people from other VFX companies contribute back to its future development. Steve said that ILM pretty much had to write version 2.0 of OpenEXR by themselves. Thus in effect they have had the problem of many people free riding on their large effort.

Thus for us [franticfilms.com], while we do plan on releasing smaller tools open source (similar to some of my past open source projects: ExoEngine [exocortex.org] and Exocortex.DSP [exocortex.org]), ILM's experience with a large costly open source endeavor scares me away from trying this with a larger project -- at least for the time being.

-ben

Re:Microsoft is not going down without a fight

[Anonymous Coward]

Technology Review Magazine has a good look at the advancements of the MS Beijing lab in its June 2004 issue.

It has been said before [slashdot.org]:

I wouldn't put a whole lot of faith in what Technology Review has to say. With a quick look at their

staff [technologyreview.com] you will see where their priorities lay. They have one fact checker and 26 people involved in marketing and advertising.

They may have once been a reputable magazine, but since Bruce Journey [technologyreview.com] took over, they are more concerned with selling magazines than quality reporting. Mr. Journey used to work for such rags as Time and TV Sports. When appointing Mr. Journey to lead Technology Review, William Hecht said [mit.edu]:

"Technology Review has long been highly regarded for its editorial excellence," Mr. Hecht said. "It is now time for MIT to invest in its commercial potential. With the appointment of Mr. Journey, we have begun the effort to secure a prominent place for Technology Review in the competitive world of commercial publishing."

I would not be surprised in the least if that article was literally a Microsoft-sponsored advertisement.

Umm...

[boomgopher]

The best bet is to get a group of people together and create their own open source version of it.

Pixie [sourceforge.net],
Aqsis [aqsis.com],
jrman [jrman.org],
et al.

Problem with Open Source in VFX, and a solution

[Thagg]

The problem is patents.

It's an unmitigated disaster. If I was to release the color correction tools I use at Hammerhead as Open Source software, for instance, there is no small chance that I would be sued by somebody, or more likely several somebodies, for infringements on their color correction patents. This kind of stuff is patented out the wazoo, and (unfortunately) the only thing that keeps the patent monster at bay is the fact that everybody does work secretly. That, and the fact that Hammerhead is so small that it's not worth suing. Note well that studios are sued over almost every successful movie they make by people alleging the most tenuous copyright infrigement. A typical example is here. [groklaw.net]

Publishing open source software does have a tremendous advantage, though, in that it is a perfect vector for publishing information that could be used as prior art when trying to defend against other patents -- so open-source is a two-edged sword (or maybe a sword that is honed sharp at both ends.)

Perhaps, just perhaps, there is a solution. It might not be impossible to have anonymous open-source, with guaranteed anonymity provided much the same way the Cypherpunks' MixMaster remailer network works. That way, one could contribute to open-source projects, and share the benefits of your work with others, without exposing yourself to patent suits.

I'm not sure how one would do this, and the network of visual effects studios might be too small -- and the coding styles of the few hundred programmers might be too distinctive -- to have this work, but it could be interesting.

Thad Beier

Re:New display tech at Siggraph

[oquigley]

Basically, there are colors that the eye can perceive, but that a monitor just can't display. Really intense flourescent green, or deep, dark, saturated violets for example.
Every display device can only show some subset of the range of visible colors. CRT Monitors can show some colors that LCD monitors can't, so you can describe the gamut (basically the achievable range) of colors of the CRT monitor as being larger than that of an LCD monitor.
The thing to keep in mind with 32bit color, is that even if it was structured to encompass the entire range of perceptable color (which I understand it's not, but that's outside my expertise), you still have to show the color on a monitor eventually.

As an aside, I've read that some women have tetrachromatic vision. In addition to having red, green and blue cones like normal people, they have another receptor equadistant from red and green (yellow?). It doesn't give them the ability to see colors that others can't, but it does give them much finer discrimination along that color range...The kicker is that the mutation is correlated with the genes that cause red-green colorblindness in men, so it's as if the women swipe a cone from their male relations ;-)

Umm...Paint by numbers.

[Anonymous Coward]

You forgotten a couple.

Toxic [sourceforge.net] Lucille [sourceforge.net] Pane [fsu.edu] Sunflow [sourceforge.net] R.I.S.E. [sourceforge.net] RenderPark [kuleuven.ac.be] Radiance [lbl.gov] and of course. POV-Ray [povray.org]

Re:You just don't get it.

[dhess]

furiousgeorge wrote:

(FYI - ILM considers OpenEXR to be a big failure. They've gotten pretty much zero contributions back from anybody. It's only take take take. It still helps ILM because they're getting most other packages to implement the format so they can make their pipeline more unified, but whether that was more or less effort that open sourcing the package in the first place is subject to debate).

Speak for yourself, it is simply not true that ILM considers OpenEXR to be a failure of any kind. We have received contributions from the open source community. The initial version of OpenEXR didn't support Win32, for example, yet 3 days after we released it, there was a port to Win32 which we later incorporated into the main code base.

Billy Biggs has written a useful collection of OpenEXR tools [scanline.ca] and made them available as open source.

Cinepaint supports the format and there's at least one other open source project which, last I talked to them, is rewriting its entire image processing pipeline to deal with floating-point pixels, inspired in part by OpenEXR.

Pixar donated code for a new compressor to the project and made it available under our modified BSD-like license.

I will admit that I would have liked to see more VFX houses following Pixar's lead and making contributions, esp. in the form of plugins for various commercial packages, but overall I'm very happy with the support we've gotten from the community in general. Many commercial packages support the format now, or will in their next version, so that's basically a moot point now, anyway.

OpenEXR's success as an open source project isn't judged solely on the number of contributions made, either; it's really all about its acceptance in the industry, and it's doing pretty well in that category. There were several goals in releasing OpenEXR as open source. The main one, from ILM's perspective, was to get support from commercial packages so we didn't have to write and maintain our own plugins. That's already happening, and that alone will save more developer time in the long run than it took to package OpenEXR as an open source project.

Another positive, yet unforseen, outcome that's shaping up is interest in using OpenEXR as an exchange format between post houses. This is something that ILM is currently working on, with valuable input from the community. There was a BOF covering this topic at SIGGRAPH; the initial proposal can be found here [openexr.com]. In today's climate of multiple post houses working simultaneously on movie productions, exchanging files and managing color information between houses is a big PITA. There's a lot of excitement about using OpenEXR for this and, in the process, preserving HDR data, which is not possible with DPX (not to the extent that it is with OpenEXR, anyway). Something like this wouldn't have been possible if we hadn't open-sourced OpenEXR.

So, in summary, it's simply not true to say that ILM considers OpenEXR to be a "big failure." We regard it as a pretty big success.

-dwh-