Catch up on stories from the past week (and beyond) at the Slashdot story archive

 



Forgot your password?
typodupeerror
×
Software Supercomputing Apache

Hadoop 1.0 Released 38

darthcamaro writes "There has been a tonne of hype about Big Data and specifically Hadoop in recent years. But until today, Hadoop was not a 1.0 release product. Does it matter? Not really, but it's still a big milestone. The new release includes a new web interface for the Hadoop filesystem, security, and Hbase database support. '"At this point we figured that as a community we can support this release and be compatible for the foreseeable future. That makes this release an ideal candidate to be called 1.0," Arun C. Murthy, vice president of Apache Hadoop, said.'"
This discussion has been archived. No new comments can be posted.

Hadoop 1.0 Released

Comments Filter:
  • What is Hadoop? (Score:5, Informative)

    by Anonymous Coward on Wednesday January 04, 2012 @05:02PM (#38589606)

    From Wikipedia:

    Apache Hadoop is a software framework that supports data-intensive distributed applications under a free license.[1] It enables applications to work with thousands of nodes and petabytes of data. Hadoop was inspired by Google's MapReduce and Google File System (GFS) papers.

    Hadoop is a top-level Apache project being built and used by a global community of contributors,[2] written in the Java programming language. Yahoo! has been the largest contributor[3] to the project, and uses Hadoop extensively across its businesses.[4]

    Hadoop was created by Doug Cutting,[5] who named it after his son's toy elephant.[6] It was originally developed to support distribution for the Nutch search engine project.[7]

    ...in case you're as ignorant as I am. Post anonymously to avoid karma whoring.

  • by Anonymous Coward on Wednesday January 04, 2012 @05:06PM (#38589664)

    Um, what the heck is Hadoop? A filesystem? Linux distro? Database software? Something to do with web servers? Throw me a bone here, man. Why does this 'Big Data' need capitalization?

    And most importantly, why did they go with the British spelling for 'tonne'? Is this a product of the UK?

    What, read the article? Are you mad?

    • by Alioth ( 221270 )

      A tonne and a ton are different things. A tonne is a metric measure, 1000kg, a ton may be a short or long ton, and is some odd number of lbs (in the order of 2000 lbs or so).

  • by abigor ( 540274 ) on Wednesday January 04, 2012 @05:09PM (#38589692)

    It was actually released over a week ago, but I guess the announcement got lost over the holidays. I am actually a bit surprised they did a 1.0 version before solving the "NameNode is a single point of failure" problem with HDFS. I know for a fact that big companies (one of which was a client) are sometimes hesitant to deploy Hadoop because of this.

    In theory, you can also use Hadoop with purportedly more robust distribute file systems, like KFS (Kosmos File System, I think it's called). I've never seen this in the wild though.

    • by Anonymous Coward

      Something else I thought that got missed over the holidays was this:

      HPCC Systems From LexisNexis Breaks World Record on Terasort Benchmark [msn.com]

      Pretty amazing when you consider how little code it took to run the 100GiB sort, never mind that it was faster than hadoop using 1/5 of the hardware. Being able to read in from disk, perform network calls, compute the sort, and write back down to disk just over 1 gigabyte a second is BLAZING fast.

  • by Anonymous Coward

    Now it's released 1.0. it can increase Mozilla style.

  • "There has been a tonne of hype about Big Data and specifically Hadoop in recent years. But until today, Hadoop was not a 1.0 release product. Does it matter? Not really

    Wasn't there a slogan about "news for nerds, stuff that matters" around here somewhere?

    • Wasn't there a slogan about "news for nerds, stuff that matters" around here somewhere?

      Key word being was. Or is it still around?

  • by Anonymous Coward on Wednesday January 04, 2012 @06:14PM (#38590358)

    Seems a fair number of you are unaware of what Hadoop is.

    Hadoop is a platform that enables distributed computing. Specifically, it supports map/reduce programming in a manner similar to Google's App Engine, except that it is open source. It supports distributing data for redundancy and/or scalability (in other words, you can have multiple copies of each data item on multiple computers, or you can split a data set across multiple computers, or both, with the data set sharded across multiple machines but with copies of each shard on multiple machines).

    There is a distributed filesystem built on top of hadoop called HDFS. There is a distributed key/value store (somewhat analogous to a database...actually, scratch that, it's a distributed hash map) called HBase. There are also a number of distributed computing libraries built on top of Hadoop, like Mahout (for machine learning), Hive (for ad-hoc querying of large data sets), and Pig (another distributed computing model that some consider to be easier than map/reduce).

    The whole setup provides a distributed computing model similar to Google's distributed environment, supporting very large clusters, map/reduce programming, and distributed storage of very large and/or spare matrices and tables.

    • ....

      There is a distributed filesystem built on top of hadoop called HDFS.

      ...

      I would say that Hadoop runs on top of HDFS. That is, the Hadoop layer expects to use a distributed filesystem which has some location awareness (allowing processing tasks to be sent to the nodes containing the data) and a level of resilience to individual datanodes failing (there is at least 2-way replication of data assumed in a typical HDFS configuration). The HDFS API expected by Hadoop can be implemented by other filesystems, such as Kosmos File System, mentioned above.

      Apos for posting this comment

  • ZooKeeper is a subproject of Hadoop ( and BookKeeper a sub-subproject, so to say ). I have been using both for a while now, and must say I am astonished about their resilience. Great products. Moreover, ZooKeeper is a valiant attempt at solving one of computer science's oldest standing problems: leader election in a ring. Hooray Hadoop, keep the good work going !
  • Amazing that nobody mentioned Cascalog https://github.com/nathanmarz/cascalog [github.com]

To the landlord belongs the doorknobs.

Working...