Somehow Amazon's Open Source Fork of ElasticSearch Has Succeeded (infoworld.com) 23
Long-time open source advocate Matt Asay writes in InfoWorld:
OpenSearch shouldn't exist. The open source alternative to Elasticsearch started off as Amazon Web Services' (AWS) answer to getting outflanked by Elastic's change in Elasticsearch's license, which was in turn sparked by AWS building a successful Elasticsearch service but contributing little back. In 2019 when AWS launched its then Open Distro for Elasticsearch, I thought its reasons rang hollow and, frankly, sounded sanctimonious. This was, after all, a company that used more open source than it contributed. Two years later, AWS opted to fork Elasticsearch to create OpenSearch, committing to a "long-term investment" in OpenSearch.
I worked at AWS at the time. Privately, I didn't think it would work.
Rather, I didn't feel that AWS really understood just how much work was involved in running a successful open source project, and the company would fail to invest the time and resources necessary to make OpenSearch a viable competitor to Elasticsearch. I was wrong. Although OpenSearch has a long way to go before it can credibly claim to have replaced Elasticsearch in the minds and workloads of developers, it has rocketed up the search engine popularity charts, with an increasingly diverse contributor population. In turn, the OpenSearch experience is adding a new tool to AWS' arsenal of open source strengths....
As part of the AWS OpenSearch team, David Tippett and Eli Fisher laid out a few key indicators of OpenSearch's success as they gave their 2022 year in review. They topped more than 100 million downloads and gathered 8,760 pull requests from 496 contributors, a number of whom don't work for AWS. Not stated were other success factors, such as Adobe's earlier decision to replace Elasticsearch with OpenSearch in its Adobe Commerce suite, or its increasingly open governance with third-party maintainers for the project. Nor did they tout its lightning-fast ascent up the DB-Engines database popularity rankings, hitting the Top 50 databases for the first time.
OpenSearch, in short, is a bonafide open source success story. More surprisingly, it's an AWS open source success story. For many who have been committed to the "AWS strip mines open source" narrative, such success stories aren't supposed to exist. Reality bites.
The article notes that OpenSearch's success "doesn't seem to be blunting Elastic's income statement." But it also points out that Amazon now has many employees actively contributing to open source projects, including PostgreSQL and MariaDB. (Although "If AWS were to turn forking projects into standard operating procedure, that might get uncomfortable.")
"Fortunately, not only has AWS learned how to build more open source, it has also learned how to partner with open source companies."
I worked at AWS at the time. Privately, I didn't think it would work.
Rather, I didn't feel that AWS really understood just how much work was involved in running a successful open source project, and the company would fail to invest the time and resources necessary to make OpenSearch a viable competitor to Elasticsearch. I was wrong. Although OpenSearch has a long way to go before it can credibly claim to have replaced Elasticsearch in the minds and workloads of developers, it has rocketed up the search engine popularity charts, with an increasingly diverse contributor population. In turn, the OpenSearch experience is adding a new tool to AWS' arsenal of open source strengths....
As part of the AWS OpenSearch team, David Tippett and Eli Fisher laid out a few key indicators of OpenSearch's success as they gave their 2022 year in review. They topped more than 100 million downloads and gathered 8,760 pull requests from 496 contributors, a number of whom don't work for AWS. Not stated were other success factors, such as Adobe's earlier decision to replace Elasticsearch with OpenSearch in its Adobe Commerce suite, or its increasingly open governance with third-party maintainers for the project. Nor did they tout its lightning-fast ascent up the DB-Engines database popularity rankings, hitting the Top 50 databases for the first time.
OpenSearch, in short, is a bonafide open source success story. More surprisingly, it's an AWS open source success story. For many who have been committed to the "AWS strip mines open source" narrative, such success stories aren't supposed to exist. Reality bites.
The article notes that OpenSearch's success "doesn't seem to be blunting Elastic's income statement." But it also points out that Amazon now has many employees actively contributing to open source projects, including PostgreSQL and MariaDB. (Although "If AWS were to turn forking projects into standard operating procedure, that might get uncomfortable.")
"Fortunately, not only has AWS learned how to build more open source, it has also learned how to partner with open source companies."
Used more open source than it contributed (Score:5, Insightful)
This was, after all, a company that used more open source than it contributed
Is there some other kind of company? Who doesn't use more open source than they contribute? Yeah, I'm sure such companies exist. But a *company* is literally in business to make money, so when they contribute to open source code, it's not just out of the goodness of their hearts.
Re: Used more open source than it contributed (Score:5, Insightful)
Actually it may well be out of the goodness of their hearts. But even then, this metric is full fucking retarded. I use Linux, right? I could code a thousand lifetimes and not "contribute more open source than I use". Sure, a company can buy a thousand man-lives of hours, but why? One of the main goals of open source is to leverage the fact that many put in a little and all pull out a lot.
Re: (Score:3)
when they contribute to open source code, it's not just out of the goodness of their hearts.
It could be they contribute to save money; if they e.g. use some sort of utility software, maintaining an independent fork or a set of patches and producing well-tested binaries is more trouble than sending bugfixes upstream.
Re: (Score:1)
Yes, saving money certainly would be a reason for a profit-seeking company to contribute.
Re: Used more open source than it contributed (Score:2)
Exactly. I've seen multinationals that have no love of anything except money contribute serious amounts of work to open source projects for entirely selfish reasons. Everyone gains.
Understanding how much work it takes (Score:5, Interesting)
I didn't feel that AWS really understood just how much work was involved in running a successful open source project,
We're talking about a major software vendor, and one that the author actually worked for. I'm confused about why he would think such a company wouldn't know how much work it takes to run a software project, open source or otherwise.
Re: Understanding how much work it takes (Score:1)
I mean, "New World" shows they have no fucking idea how anything else works so why would this be am exception?
Re: (Score:3)
Yeah, I know, right! They've never built anything significant, like, say, a phone or tablet OS, or, you know, the #1 cloud infrastructure platform (which requires a whole lot of software development). They clearly have no idea how to build software.
Re: (Score:2)
> They clearly have no idea how to build software.
Nobody ever made such a claim.
Re: (Score:2)
Did you even read the comment I replied to? I'd say cfalcon pretty much made exactly that claim.
Re: (Score:2)
I mean, "New World" shows they have no fucking idea how anything else works so why would this be am exception?
I think it hammered home that being a pioneer in one thing doesn't mean you can just pivot into whatever you'd like and succeed. A lot of hubris being cut down to size with the movie and game divisions.
Re: (Score:3)
I didn't feel that AWS really understood just how much work was involved in running a successful open source project,
We're talking about a major software vendor, and one that the author actually worked for. I'm confused about why he would think such a company wouldn't know how much work it takes to run a software project, open source or otherwise.
I think this answers itself, really. Author being honest: he knew his company (management / "culture", processes) and didn't think they could pull it off.
Re: (Score:3)
Perhaps that's how the author *felt* about Amazon. But the hard facts say otherwise. Amazon has managed to develop many, many successful projects, including open source projects. Over 30% of cloud-based software runs on Amazon software and hardware. https://en.wikipedia.org/wiki/... [wikipedia.org] You don't get there just by being able to write programs, you have to be able to engineer software at a massive scale. The supposition that Amazon doesn't know how to "pull off" a successful software project is kind of ridiculou
Re: (Score:2)
I've only played around with AWS (most of my experience is using Azure) but I can tell you that at the very least, their UI is slick and really sells you on their services.
What's going on behind the scenes? Who knows heh But yeah, I hear you and agree
This is the most important nerd news since... (Score:2, Insightful)
comparison (Score:3)
posting since I looked it up:
https://bigdataboutique.com/bl... [bigdataboutique.com]
I missed the drama - the Amazon product is the open source one now.
That's why it's a successful open source project (obviously?).
Re: comparison (Score:2)
Forking projects SOP for Amazon? (Score:3)
This was literally a response to a childish action from Elasticsearch developers, one which was quite a black mark on open source community and which Slashdot condemned when it came up (you don't get to cry just because a big company uses your open source project). Why would this become Standard Operating Procedure for Amazon. It's not something they wanted to do in the first place.
Re: (Score:1)
Originally Apache licensed. (Score:3, Informative)
I love to hate on Amazon, but when I first heard of the original dispute and "strip mining open source" I assumed it was over a GPL clause. Au contraire. This is all nothing but sour grapes from Elastic.
As long as Amazon didn't and do not instigate patent litigation (and other clauses), they were and are within their rights of the Apache license.
Hopefully they fork Prometheus (Score:1)
The dev team over there spent years fighting the use of *environment variables*. As in, setting environment variables to pass config to the service at startup. Bloody mental.
What? (Score:2)
While I agree Amazon has been far from a good open source citizen, this sentence just seems weird:
"This was, after all, a company that used more open source than it contributed."
Isn't that what exactly what we want? It's sort of the whole point of open source. I mean, it's pretty literally impossible for most companies to contribute more than they use. Very, very, very few companies that say, use Linux, can hope to make Linux-sized contributions back, and those companies shouldn't be just using Linux.