Open Source And Closed Standards?

jaaron writes "Can open source and closed standards work together? That's the question asked by Kevin Bedell in his O'Reilly weblog article. The issue springs from questions on an OSI mailing list, hinting that Sun Microsystems is looking for an open source license that would require derivatives to maintain test suite compatibility. Under such a scheme Sun could maintain control of the Java API but allow open implementations."

Bah

[ThoreauHD]

I'm gonna stay out of this one flame war. When diversity means less options, then I'm all for closed. Until then.. Darwin is in control.

Open source + Closed standard = Closed

[mind21_98]

By definition, anything created to satisfy a closed standard leaves very little room for improvement. If you have to build around a crappy API, you can't improve the API. In order to have a fully open source application, you must build around open standards as well. Otherwise you'd have some very nasty license issues.

Open Source Works with Closed Standards:1 Caveat

[reporter]

Open source can work with closed standards under 1 caveat: the open-source programmers may need to rename a variant on the closed standards.

The situation is analogous to building a chip that runs an instruction set architecture (ISA) owned by a competitor. The ISA is a closed standard in the sense that the company owning the ISA has trademarked its name. For example, MIPS technology trademarked the name "MIPS". A competitor, Lexra, then implemented a subset of the MIPS ISA, omitting 2 instructions. Lexra said that its chip is MIPS ISA compatible. MIPS sued and won. If Lexra had, instead, labeled its chip "MIPS ISA flavored", not "MIPS ISA compatible", then there would be no legal problems.

Another good analogy is Microsoft incorporating the Java runtime environment in its browser. The environment was not fully compatible with Sun's closed-standard for the Java runtime environment. Sun sued and won. If Microsoft had claimed that the browser was equipped with a "Java flavored runtime environment" or "JavaPlus[tm] runtime environment" (and trademarked "JavaPlus"), then there would be no legal problems.

I do not see a problem here.

Open source is now a credible movement. The open-source development lab (OSDL) and the free software foundation (FSF) have sufficient clout that if any team of talented programmers created a language called "JavaPlus", derived from and mostly (but not entirely) compatible with the closed-standard Java, there is the strong likelihood that JavaPlus would come to dominate the market for Java. Then, Sun would need to kiss OSDL's or FSF's ass. Sun would be forced to alter the Java standard to make it compatible with JavaPlus.

Sweet. Sweet revenge.

This is BS

[Anonymous Coward]

All Sun has to do is to publish a standard Solaris OS specification (similar to whats done with SPARC) and then GPL the OS.

GPL offers protection to individual developers that they will be able to freely benefit from improvements to their contributions anyone makes without having to fork over money. The problem with BSD is that if a company uses your source in a closed product you dont get any future benefit and if you want to use their product you have to pay.

Open source developed Solaris OS'es should have the ability to claim compliance to whatever revision of Solaris.

Sun can charge a fee to do compatibility testing. In theory though it should be possible for third parties to also engage in the certifying compliance business .. if they do a half ass job ..they just won't be trusted.
They will have to have their own logo though and not be allowed to use a Sun certified Solaris specification compliant logo

Re:Open Source Works with Closed Standards:1 Cavea

[Anonymous Coward]

Microsoft already tried that with J++.

It lasted about, I donno, 3 months before they were sued out the ass...

I think it's brilliant

[yaphadam097]

Maybe it's just because I've been doing Java development exclusively for the past three years. Or, maybe it's because I've been doing Extreme Programming exclusively for the last two and this gels extremely well with the idea of Customer Tests which are at the core of what I do. But, I think this is absoulutely brilliant.

Essentially, "Do whatever you want, but you can't call it Java unless it passes our compatibilty suite." Thus the core vision of "write once run anywhere" is preserved but the community is given the freedom (And, yes, I do know what that word means) to enhance and bugfix. BTW, it is already pretty easy and wouldn't become any harder to expand beyond core java by adding additional libraries. The difference would be that you could distribute the whole thing under a single open source license.

The one thing you couldn't do would be to change the language itself. But then, maybe I'm missing something, but if you don't care about compatibility why use java in the first place?!? It's not like there aren't good alternatives out there that will let you do whatever you want (Perl, Python, C++, etc.) The whole advantage of Java is that it is so prolific, and it is so because of it's rigorously maintained compatibility/portability (And strong advocacy by Big Blue among others... who like it because of it's portability across the many platforms they offer and support.)

Re:Why even bother open sourcing Java then?

[Anonymous Coward]

Depends. There are lots of things that you could do, depending upon the form of the license. You could reimplement the guts of java in the language of your choice, such as C#, pascal, or ada. You could add functionality to the JVM or language, if the license allowed it. You could optimize the compilers for different purposes. You could develop instrumented JVMs. Lots of things.

And don't forget, the reason for Java is compatabilty. If you don't care about that, then it really isn't Java. Just roll your own and insert whatever you want.

Doesn't really mix

[joe_plastic]

If Bob Scheifler had read the Open Source definition [opensource.org] he would have noticed that maybe criteria 8 and 10 contends with what he wanted to accomplish.

8. License Must Not Be Specific to a Product: The rights attached to the program must not depend on the program's being part of a particular software distribution.
His test suite would be another program.

10. License Must Be Technology-Neutral:No provision of the license may be predicated on any individual technology or style of interface.
The environment to be tested might not support all of the I/O that his suite might need in order to pass. IE maybe it has some combination of no writable filespace, no gui, no network connection, no terminal....

I wish the definition was more clear that the license itself shouldn't restrict the kinds of modifications that can occur. If that is impied then criteria 3 is abused as well.
The license must allow modifications and derived works, and must allow them to be distributed under the same terms as the license of the original software. You are allowed to make modification as long as the md5sum of the resultant file is cc4e48a5fe0ba15b13a98b3fd34b340e ;->

Isn't That What Trademarks Are For?

[femto]

Surely the correct solution is to license all the copyrightable stuff under the GPL then reserve access to the "Java" trademark for implementations which comply with Sun's (open) standards?

The idea of a trademark is to make is difficult to pass of an inferior clone as the original, which seems to be precisely what Sun is trying to prevent.

Why Open Sourcing Java worries me.

[Yaztromo]

I like Java. I maintain an Open Source project [jsyncmanager.org] coded in Java. I particularily apperciate the fact that Java applications can be easily made completely portable across platforms.

Here's what concerns me. Open Source has never really shown that it's terribly interested in ensuring API and binary compatibility across releases. Native binaries tend to be somewhat tightly compiled for their specific distro. To get around this, many packages are distributed as source so you can compile them specifically against your platform of choice.

All well and good, but take a look at how the sources accomplish this: via pre-compiler directives to ensure things compile correctly on different platforms, or via complex makefiles to build specific sources on specific platforms.

Currently, I don't typically have to worry about such things with Java. There are no pre-compiler directives, and there is no need to use them: one codebase compiles on every platform.

Here's where my concern comes in. As soon as you Open Source Java, someone is going to want to put in pre-compiler directives because they're used to them from the C/C++ world. Around the same time, someone is going to create a Java fork which isn't 100% compitable in some area.

Java developers, wanting to target as many platforms as possible, are going to start using the pre-compiler directives in order to work around implementation-specific bugs. Maintainers are going to start worrying less-and-less about API compatibility issues because developers are going to have pre-compiler directives to work around them (as we've already seen many times over the years in the C/C++ world). All of which is going to help reduce Java's platform neutrality, and make my job as a Java developer more complex than it is currently, reducing incentive to use it in the first place.

My biggest interest as a Java developer would be to ensure that all Java runtimes conform to a single, standardized testsuite as Sun seems to want. And I don't care that the testsuite could be buggy -- so long as any API bugs that do exist are consistant across platforms. At the same time, there are some amazing things the Open Source world could do with all the other parts of the Java Runtime Environment -- for example, making the HotSpot Compiler Open Source could allow for some pretty massive JIT research to be consolidated in one place for the benefit of everyone.

Much of this could be solved if Sun put the Java API and other technologies through an official standardization process, and then made their implementation Open Source. The former has worked well for other languages (Ada comes to mind), where a tight standardization process long helped to ensure source compatibility between platforms. The latter works extremely well for enhancing the adoption and development of a given technology. But to make it work, you couldn't just go with some form of defacto standard that most Open Source projects use/create/adopt. Unfortunately, I'm not quite sure what benefit Sun would see from doing something like this (not that I personally care anything about wether or not Sun were to get anything out of doing this -- I just realize they're going to need to see some sort of benefit before they ever decide to do such a thing).

Yaz.

Open Source has nothing to do with it

[Wesley Felter]

There are already plenty of not-quite-compatible Java hacks out there. The MS VM was the most obvious one (you don't really need JNI, do you?), but consider stuff like GCJ, Kaffe, Pizza, etc. The Java community has survived them; it can survive open source.

Re:Why even bother open sourcing Java then?

[gehrehmee]

I think you're sort of mis-reading Sun's intention on this. I don't believe they have any interest in restricting what we, the open-source community can do with it.

What they want desperately to avoid is being screwed the same way they've been screwed so many times before: Microsoft swings in, take what Sun (or the W3C in the case of HTML&Friends) and shattering it into independant & incompatible implementations that eliminate one of the project's main goals: Interoperability.

I believe Sun is trying very hard to let the open source community take the code and run with it as its done with so much other software, but without letting MS tie it to a You-Require-Windows-To-Work-In-The-Real-World business model.

Re:Sun will Wither Away

[thisgooroo]

"Can open source and closed standards work together?

No they can't really

tell me, how open are posix, ANSI C, or the internet standards? are linux, *bsd, apache open source or not?

"Under such a scheme Sun could maintain control of the Java API but allow open implementations."

Sun never learns. When they got into fight over java with Mircrosoft the result was MS making .NET.

MS put out an incompatible java in yet another attempt to control the internet. in order to prevent that, sun had to do something. so MS didn't like the outcome and decided to do its own standard. fine, but at least we can pretty much rely on the java we have installed on our systems run whatever claims to be java

Re:Why even bother open sourcing Java then?

[Chandon Seldon]

The "Trademark Java and don't let people call things Java that aren't Java" plan works perfectly. If it doesn't adhere to Sun's Java standard, there's no reason anyone should be calling it Java anyway.

Re:Maybe a bit off topic

[Minna Kirai]

What if many test suites are flawed

RTFS. The whole point of Sun's proposal is that there can be only one [geocities.com] test suite. They want to release Java code including a test suite, so that whoever recieves that code can't redistribute it unless that test suite works. Not some random test suite, but the specific test code included by the original author.

Open Standards are Huge

[alanbs]

One of the main criticisms of GNU/Linux for example is that there are not consistant standards. I know that this has recently been fixed to a large degree with the specification created a few weeks ago by all the big distros and important people, but this is a great example of the more general situation. Linux people out of all people are for open implimentations, but there was still a need for large collaberation in interface. As a result, some freedom was taken away, but I think most would agree that this is a good thing. Sometimes what you need is a good benevolent dictator. Is Sun benevolent? I don't know. That has been a point of recent contraversy.

before flamming sun remember...

[the-build-chicken]

...that if it wasn't for their tight control and protection of Java, the only way your java/j2ee apps written on windows would work on linux/unix would be through wine.

And vice versa through cygwin.

Re:This is BS

[psetzer]

It would be nice if Sun decided to open their intellectual property up, but right now they are in dire straits. Asking them to throw away all of their software sales in exchange for entering a different market is asking them to really risk their livlihoods and their business on something that may not pan out. Every time the open source community tells Sun to make the leap, I wonder if they've thought about the consequences and are willing to support Sun, or if they just want to cannibalize Solaris and Java, and just let the company falter.

that doesn't make sense

[jeif1k]

The primary purpose of open source licenses is to give users control over the software platform that they use--to allow them to adapt it to their own needs, instead of the business needs of some mega-software-corporation. This includes removing or replacing poorly conceived portions of a platform and adding incompatible extensions. An "open source" implementation for a closed standard under the control of Sun doesn't allow this, hence it doesn't achieve the goals of open source.

Furthermore, requiring formal test suite compatibility means that such a project simply cannot meet the definitions of an open source project.

I don't get it.

[mcrbids]

What is the big deal?

How many implementations are there of Java? Perl? PHP? Python? Ruby?

Yep. Just one, each. That hasn't stopped each from growing to enterprise-grade capacity. (Yahoo has bet the farm on PHP, etc)

If Sun Opens up Java, but requires that forks retain a baseline compatability, what is wrong with that?

There are numerous implementation of Structured Query Language (SQL) and the fact that they are all "mostly" compatable is a developer's nightmare. You either don't bother with all the fancy foreign keys/rules/triggers stuff, or you choose AN implementation of SQL (Say, PostgreSQL, or Oracle, etc) and develop for it alone (or you have lots off time/money to burn). You can only develop cross platform appliccations in a fairly simple way, because of all the potential gotchas between SQL implementations. Anytime you need true ACID compliance, and true data integrity guaranteed at the database level, you're stuck.

How different this would be if ANSI SQL came not just with a specification (does this, that, etc) but also came with an extensive set of statements and results that could be compared so that there was a strong, capable baseline that would be cross platform.

Then, establish a non-profit agency that could enforce the standard, providing three degrees of certification: ANSI SQL based, ANSI SQL compliant and ANSI SQL certified.

"Based" would refer to an incomplete implementation of the spec. The only difference between "compliant" and "certified" being that "compliant" could execute the statements to get the desired result, and "certified" has been independently tested by the enforcement agency to ensure that the test runs right. (and a fee has been paid)

This would be a total BOON to developers who could write ANSI-compliant statements and be confident that their stuff would work everywhere.

There are DOZENS of gotchas in SQL. For example, if you define a UNIQUE constraint across two fields where one of the fields could be null, does the UNIQUE constraint apply to rows where one of the fields is null? According to the spec, no. But numerous database engines will enforce the uniqueness despite NULL being one of the values.

OOPS! NULL is not a value, it's a non value. It's an "I don't know". How can that be considered in an evaluation of unique?

Usually not important if you're running a BLOG. Can be critically important if you're keeping track of financial transactions!

I would *LOVE* it if ANSI treated SQL like Sun proposes dealing with JAVA....

Re:Comparing

[Ogerman]

Sun's problem is that they know that people want to produce non-conformant implementations. They feel they have to stop them doing that. This goal is, by its very nature, incompatible with an open source license. No amount of clever wording is going to change that.

Perhaps this is exactly why Sun has been so reluctant to even approach open source licenses with Java up until now?

Perhaps a better question is this: Why does Sun think that it needs complete control over the Java specification in order to remain a relevant player? It's pretty obvious that this is the real issue. I think they're afraid that Java will be forked into new (maybe better) languages and those languages will detract from the marketshare of Java. Since they don't make any money off the Java language itself, this is a *branding* issue. (Its value is in driving customers towards Sun as a trusted source for hardware and not-for-free software) From a business perspective, the possibility of losing this brand recognition is a valid concern. Nevertheless, it is now an unavoidable threat. There is now plenty of competition to Java, both from MS and from the Open Source world. As great as it is, Java is not the end-all-be-all programming language. (nor is any other language..) IMHO, Sun's best move would be to GPL their Java implementation, strictly enforce the Java trademark based on compliance, and ride out the momentum generated by the Java brand as long as it lasts. (And frankly, this will be quite a long time, given the installed base of mission-critical Java software, where compliance and reliability is king)

So what would happen if an Open Source, non-Java(tm) fork were to make a desireable but incompatible improvement? Simply include it in the next revision of the official Java(tm) spec! There is nothing to say Sun could not pull innovations back out of derivative works for inclusion in the trademark-protected Java specification. This would instantly reduce such forks to experimental projects which, while valuable, would not gain widespread acceptance. In the end, most people would trust that which is certified Java-compliant because most people would prefer stability to the bleeding edge of the language's evolution.

C'mon Sun. We welcome you to become benevolent dictator over a Subversion or Arch tree of GPL'ed Java code. Lets fight our common enemy together. (:

Open Source means No Control

[Jay Carlson]

Open Source means No Control.

That's the core of the various free software guidelines.

You never need to ask permission before taking some 3D library and stuffing it into an SNMP monitoring tool, and then posting it on freshmeat---where some person on the other side of the world finds it and hacks up a web interface.

You never have to be captive to a copyright owner. If you think RMS is making poor technical decisions [sourceforge.net] in FSFmacs, or XFree86 does silly things with licenses, or some guy neglects [sf.net] his [sourceforge.net] hobby projects [sf.net] (ahem), you can go off on your own without begging anybody. All you have to lose is the previous name and its reputation.

On the other hand, there are about a zillion Linux distros out there that nobody's heard of. The ultimate penalty for doing a bad fork is being ignored.

Re:This is BS

[Eivind Eklund]

You are repeating a lie.

As a developer with the FreeBSD project, I can say with certainity that there do come benefits from proprietary derivates of BSD-

Specific examples:

• The entire SCSI subsystem of FreeBSD (CAM) comes from a proprietary derivate
• The entire netgraph subsystem (network transformation system) comes from a proprietary derivate
• Many of our core developers are employed by companies making proprietary derivates
• The mpd multilink PPP daemon came from a company that made a proprietary derivate

We've also got a ton of other submissions, but bugfixes and feature enhancements.

Now, we can have an economics debate about which license results in the most contributions - but claiming that "if a company uses your source in a closed product you dont get any future benefit" is plain misinformation.

Eivind.

Re:Maybe a bit off topic

[Tony-A]

"New bugs for Old!"

We can be reasonably assured that there will be bugs in that one test suite. Eventually they will become known bugs. Well technically, an obviously buggy implementation will pass the official test suite.

A different test suite will cover those now known bugs, but introduce some unknown bugs. If you are risk-averse, then trading known bugs for unknown bugs is not a good idea. I think that some of the stuff that Sun does involves things that are very risk averse.

Changing standards is a very awkward and time-consuming process. Look how long it has taken the USA to switch to the metric system.

Re:there is a precedent for this

[Tony-A]

As for the versions that many distributions ship based on GCC 3.x, they generally don't pass all the tests.

Which could help explain why Sun is so sensitive about it.

If you have to depend on something, you need to be able to depend on that something. A fixed test suite help assure that at least the bugs don't keep changing on you. As things get more complicated, faster, and more out of sight, everything really has to be better just to break even.

Re:Why even bother open sourcing Java then?

[Ollierose]

This Javascript thingy that some browsers support is nothing to do with either Java(tm) or Microsofts J++... Java (the language) and Javascript (the browser thingy) are completely different

The golden bridge of a good license.

[Combuchan]

To that end, we have the Lesser GPL [wikipedia.org], which would allow compiled applications themselves to be closed-source.

It's funny you mention readline, because I seem to recall there being some disagreement about it being GPL'd as opposed to LGPL'd. In the FSF's [gnu.org] opinion on the subject [gnu.org], "releasing it under the GPL and limiting its use to free programs gives our community a real boost. At least one application program is free software today specifically because that was necessary for using Readline."

It is paradoxical here that the FSF dims their own light on behalf of their own greater good. In the case of readline, and I'm sure a possibly GPL'd java class library, is that when the license which inhibits adoption by closed-source folks who wish to develop in their own manner, these same closed-source folks will instead opt for a more restrictive/liberal alternative (depending on your point of view) that enables them to continue to do what they've been doing all along--develop closed-source software. This slows the overall adoption rate of open-source software.

Open-source seems to grow best when it's not forced down our throats or dangled in front of us like an unattainable carrot. The ideal solution should be to showcase the power of open-source in combination with the freedom to do what you want with it, including using it in closed-source. This greatly assists open-source's wide-scale adoption. Lao Tsu said it best: "Build your enemies a golden bridge to retreat across."

Look at GCC and friends--closed source software built on GPL'd and LGPL'd libraries released for open-source platforms such as Linux increases that platform's market share. Regardless of how you perceive Flash and RealPlayer1, they are both closed-source applications that help Linux be a better desktop OS because you can easily view a good chunk of the WWW with it without having to learn about swfdec or mplayer/xine, respectively. And people all over can move their IIS/Oracle/ASP application letter by letter to LAMP2--all because interoperability with the closed and open is possible.

In summary, let the open and the closed comingle, because the open will certainly prevail.

1. It should also be noted that once in the open-source world, people will be more prone to ditch those closed-source holdovers in favor of the aforementioned open-source (and many times superior) alternatives.

2. Substitute L for F, N, O, H, as applicable. The P can stand for whatever the hell you want--I'm not getting into that tonight. ;)

--sean

It depends on what you call "open" standards.

[Anonymous Coward]

I don't think that "open standards" mean that you can change it whatever way you want. If that would be the case, "open standards" would be a contradiction. To be a standard, it has to be well defined and respected. Otherwise, applications would not be compatible and that's the very goal of standards. "Open standards" should mean that the specifications are public and free to use, not that you can change them just like that, there still has to be some organisation controlling a standard or it simply won't work.

Re:Its called a trademark silly

[k98sven]

That's a misunderstanding. It's not about how you distribute it. It's about the technical aspects of library-linking.

The FSF is very clear [gnu.org] on this matter, and their stance is that linking to a GPL-ed library constitutes a 'derived work' under the GPL, and thus the resulting executable must be GPL-licenced.

Merely running your code on Linux or using a GPL-ed interpreter to run your program does not constitute a 'derived work' under this definition.
Linus Torvalds himself has clearly delineated where this border is with the kernel: User space programs are not derived works, neither are kernel modules.

There is no legal significance in saying that something is a 'system requirement'.