Follow Slashdot stories on Twitter

 



Forgot your password?
typodupeerror
×
Data Storage Databases

Enthusiasts Convene To Say No To SQL, Hash Out New DB Breed 423

ericatcw writes "The inaugural NoSQL meet-up in San Francisco during last month's Yahoo! Apache Hadoop Summit had a whiff of revolution about it, like a latter-day techie version of the American Patriots planning the Boston Tea Party. Like the Patriots, who rebelled against Britain's heavy taxes, NoSQLers came to share how they had overthrown the tyranny of burdensome, expensive relational databases in favor of more efficient and cheaper ways of managing data, reports Computerworld."
This discussion has been archived. No new comments can be posted.

Enthusiasts Convene To Say No To SQL, Hash Out New DB Breed

Comments Filter:
  • by KingPin27 ( 1290730 ) on Thursday July 02, 2009 @04:57PM (#28565105)
    Just use flat text files --- no need for expensive db's .... think of the freedom!
    • Re:Quit Whining (Score:4, Insightful)

      by Anonymous Coward on Thursday July 02, 2009 @05:03PM (#28565157)

      The horrible lag I get when using address completion in Firefox 3 makes me wish more people thought that way!

    • This is one of the main objectives of ReiserFS, to make such things easy, a project which unfortunately has run into some difficulty of late.
      • Re: (Score:3, Funny)

        This is one of the main objectives of ReiserFS, to make such things easy, a project which unfortunately has run into some difficulty of late.

        I wonder if I could sneak Hans an eeepc inside a birthday cake...

      • I"ve lost data in two filesystems thanks to the Slasher's shoddy work.

        • Re: (Score:3, Insightful)

          by phantomfive ( 622387 )
          You didn't learn to backup after the first time?
          • Re: (Score:3, Insightful)

            by fooslacker ( 961470 )
            I have no idea what rubycodez's experience was but just because I can recover from backup doesn't mean data wasn't lost by the filesystem. More importantly external risk management schemes (i.e. backup) don't relieve the file system of the obligation not to kill the copy of my data I've entrusted it with.
        • by Paradise Pete ( 33184 ) on Thursday July 02, 2009 @06:40PM (#28566195) Journal

          I"ve lost data in two filesystems thanks to the Slasher's shoddy work.

          Have you looked near Redwood Regional Park? On the side of a hill?

    • Re:Quit Whining (Score:4, Insightful)

      by CyberLife ( 63954 ) on Thursday July 02, 2009 @08:39PM (#28567265)

      Flat files are a perfectly viable option in some circumstances. Not everything requires data uniformity or the ability to run complex ad-hoc queries, nor does everything need information to be controlled by a separate process running on a different machine. Not every system integrates multiple applications through a shared data-store. The NoSQL crowd isn't arguing that SQL is bad, just overused. There are a great many situations where something like flat files or Berkeley DB is more than sufficient, and yet people still use relational technology. In my experience it's generally because that's all they know. In their mind, if one needs to store data one uses SQL. They don't select the right tool for the job because they honestly don't know there are other tools.

      • Re:Quit Whining (Score:5, Insightful)

        by jadavis ( 473492 ) on Thursday July 02, 2009 @10:17PM (#28567855)

        One of the reasons is because RDBMSs offer a lot of tools, like atomicity, durability, backup/restore, centralization, point-in-time-recovery, etc. Many application developers need these things without actually needing the abstraction of a relational system.

      • You can use SQL with flat files.

        SQL is going to be around for a long time, because it's useful as an "API" - as a protocol or layer of abstraction.

        Programmers can write all sorts of programs in all sorts of programming languages and then use SQL to talk to the DB. If the DB changes a bit, they can often use the same SQL or modify it slightly.

        You often see lots of grumbling and cursing in various companies because people actually end up doing that and companies end up with lots of stuff hooked to the DB - MS
  • by Marillion ( 33728 ) <ericbardes@nosPAM.gmail.com> on Thursday July 02, 2009 @05:01PM (#28565143)
    There is a time and place for SQL. There is a time and place to avoid SQL.
    SQL is great for financial data. SQL is terrible for genetic data.
    • It would be interesting to hear why this is.

      • by Carewolf ( 581105 ) on Thursday July 02, 2009 @05:36PM (#28565521) Homepage

        Design an efficient table relating a tree structure. Then design queries to answer questions such as:
        * Find the nodes in the subtree under B.
        * Find all ancesters of G
        * Find the nearest common ancestor of D and H

        Trees is a wellknown problem of SQL, but the fact is that SQL can't handle most datastructures and complex relations, only very simple one dimensional ones.

        • by E IS mC(Square) ( 721736 ) on Thursday July 02, 2009 @06:25PM (#28566043) Journal
          >> Trees is a wellknown problem of SQL, but the fact is that SQL can't handle most datastructures and complex relations, only very simple one dimensional ones.

          Sorry, that's not true. Have you tried analytical functions? You would be amazed how complex scenarios can be handled easily with them. And they are part of ANSI SQL standards. And db providers (Oracle etc) have taken the concept and improved a lot on it.

          I think the anti-sql 'movement' has more to do with new (internet era) languages and their developers than so called 'lack' of features. In my limited experience, I have observed people coming from C (and such) background have no problem with sql, while java developers (and this is probably true for most developers working on web-based applications) are the worst kind when it comes to understanding even basics of sql. All they want is their objects.

          I strongly believe that a competent programmer designing/developing system which includes data and data-storage should at least know normalization, indexes, and what does it mean by 3NF. Programming language is one thing, database is another, and knowledge of both is required to build a decent system.
          • by raddan ( 519638 ) * on Thursday July 02, 2009 @08:10PM (#28567017)
            And to expand on that a little, I think each part of the MVC idiom has it's own domain-specific language because those languages are well-suited to those applications. An imperative language with an emphasis on objects (e.g., Java) just doesn't do the same thing that a declarative set-theoretical language (SQL) does. Well, it can, but doing so is a royal pain in the ass. That same imperative language is also total overkill for defining a layout. HTML does that job beautifully and simply.

            There are certainly common CS themes running between all three. We have three languages not because people haven't thought about those things, but because they make our lives easier.

            Whenever I hear people bitching about 'doing away' with SQL, I always wonder what they think is wrong with it. SQL certainly has some limitations, don't get me wrong, but it is a great language for the vast, vast majority of cases. If your application is so specialized that SQL isn't appropriate, well, bravo, but that does not mean that the relational database concept is flawed. Personally, I think if people spent a few moments doing some formal analysis before they built their databases (imagine that, thinking before doing?!), they would find that SQL is a beautiful thing. If your implementation of SQL doesn't cut the mustard, maybe you just need a better query optimizer?
          • by TheNarrator ( 200498 ) on Thursday July 02, 2009 @09:13PM (#28567481)
            I think the main problem that the web 2.0 dynamic language crowd has with RDBMs is that:
            • 1. Relational data is strongly typed. You cannot easily add new fields to a table or store arbitrary types in a column and expect acceptable performance.
            • 2. Migrating large amounts of relational data to a new structure takes a very very long time. Constant refactoring of data models is to be avoided. You have to get it right the first time or at least very early in the development cycle to avoid major headaches...
            • 3. Databases are hard to mock in a testing context. Automated tests can be significantly slowed down with even a test database..
            • 4. Error in database architecture are very difficult to correct due to 1 and 2, especially when used with a dynamically typed language..
            • 5. It's difficult to maintain the data integrity that RDBMSs take for granted in highly scalable distributed systems and have acceptable performance.

            The only real show stopper and a real reason to replace RDBMSs is #5. All the others can be worked around by just deeper study of data modeling techniques. Data modelling is not something most developers can figure out intuitively. There is a lot of theory to be learned to do it right and it can very easily be done badly leading to severe performance problems and an unmaintainable application.
            With regards to # 5: I went to a presentation at Javaone where some Ebay engineers explained that they do not use transactions in any of their database operations. They just leave junk rows around in the db if a transaction half completes and as long as they aren't reachable they don't consider it a big deal. They have to very carefully organize the order in which they manipulate data to avoid data corruption ,but that lets them get around # 5,

          • Re: (Score:3, Insightful)

            by pvera ( 250260 )

            As a web programmer, I wanted to take offense at your statement, but something that happened to me only a few weeks ago is making me have a hell of a lot less faith about the available pool of web programmers out there:

            During a round of interviews, we sent out a take-home quiz. We mostly wanted to know if the candidates either knew the actual answers, or could at least google it. One of the questions involved simple aggregates in SQL. Given a table with a unique id and a date of birth, I wanted a query that

          • Cloudscape/Derby (Score:3, Interesting)

            by Kupfernigk ( 1190345 )
            Yes, and Java persistence systems (Hibernate) suck dreadfully; they are a solution for which there is no problem. By the time you've learned the mess that is Hibernate, you can have learned SQL and the Java Collections well enough to be able to knock up any persistence model you need in no time flat.

            Derby 10.5, meanwhile, still has a tiny footprint, and can do most if not all of the SQL you will ever want for a typical Java application, along with features like the ability to do live backups, live table com

        • by diamondmagic ( 877411 ) on Thursday July 02, 2009 @07:58PM (#28566911) Homepage

          Design an efficient table relating a tree structure.

          Huh? Tree structures are best handled by relational databases, as it is far faster then recursion. Give row a unique ID and a parent ID, and in addition, a left hand and right hand number, the root node having a left-hand value of 1 and a right hand value of (number rows * 2), the first child node has a left-hand value of one more than the parent's, the right-hand value is one less then the left-hand of a younger sibling.

          Then design queries to answer questions such as:
          * Find the nodes in the subtree under B.

          SELECT * FROM rows WHERE left > [left hand value of B] AND right < [right hand value of B]

          * Find all ancesters of G

          SELECT * FROM rows WHERE left < [left hand value of G] AND right > [right hand value of G]

          * Find the nearest common ancestor of D and H

          SELECT * FROM rows WHERE left < [lowest left hand value from D,H] AND right > [highest right hand value from D,H] ORDER BY right LIMIT 1

          Trees is a wellknown problem of SQL, but the fact is that SQL can't handle most datastructures and complex relations, only very simple one dimensional ones.

          Are you saying trees are easy or hard? And for more complex systems, that is what JOINs are for. SQL is by far the most powerful way and often the fastest way to manipulate data that I know of. The only time I can recall that I had to use a non-SQL solution that was faster then the SQL solution was a matrix operation.

          • by lawpoop ( 604919 ) on Thursday July 02, 2009 @09:43PM (#28567643) Homepage Journal
            For anyone wondering, parent is talking about a preorder tree traversal algorithm:
            Link 1 [mysql.com]
            Link 2 [sitepoint.com]

            And parent it right. I was doing an adjacency list in MySQL for a while, because I thought that preorder trees were just a little too complicated, but they are *way* easier and more intuitive.
          • Re: (Score:3, Interesting)

            by aztracker1 ( 702135 )

            >> Find all ancesters of G

            > SELECT * FROM rows WHERE left [right hand value of G]

            Now promote a tree node so it's before G...

        • by mcrbids ( 148650 ) on Friday July 03, 2009 @04:00AM (#28569475) Journal

          Design an efficient table relating a tree structure. Then design queries to answer questions such as:...

          I don't know, but I recall reading that Postgres 8.4 is now out and includes support for recursive queries. [slashdot.org] (trees) Not sure about the reputation of the blog in question, but you may have heard of it?

          the fact is that SQL can't handle most datastructures and complex relations, only very simple one dimensional ones

          You are kidding, right? Just today I cooked up a 7-table query including 2 subselects, and a left outer join to a meta table consisting of 2 inner joined tables. Total of some 11 tables comprising a highly complex data set. Don't know what you mean by "very simple one dimensional ones" but 11 tables each joined in either a one-to-many or many-to-many mapping provides at least 11 dimensions. (more if you self-join tabls, often needed) And this isn't particularly hard for me - often I have joins combining 12 or more very large tables with unrestrained combinations somewhere in the billions to trillions of possibilities that all somehow seem to parse just a few seconds thanks to a few well-placed indexes and a well-structured query.

          Methinks you don't really understand SQL?

      • Re: (Score:3, Insightful)

        It would be interesting to hear why this is.

        My guess would be that because SQL is a Structured Query Language it is best used for handling structured data. If you have serial, unstructured data you have to invent your own format for it to use inside the database, and then the query language isn't helping you.

        • by Marillion ( 33728 ) <ericbardes@nosPAM.gmail.com> on Thursday July 02, 2009 @06:10PM (#28565879)
          Right, I went into a little detail on another post. When I said, "genetic," I mean genes - DNA. There are four main Nucleic Acid types in DNA: Adenosine, Cytosine, Guanine, and Thymidine. Abbreviated ACGT. So you could store a gene sequence as ACGCCTGCAATC. But in other populations, Asian for example, the same gene is more commonly found as ACTCCTGCAATC. (The third nucleotide is different) Exact string matches won't find matches between different population groups. So they create wild-card letters that represent either G or T -> K. So ACKCCTGCAATC would match either the both of sequences commonly found in western and eastern populations. Data of this nature has no business being in a relational database. For that matter, it doesn't belong in these pseudo databases either.
          • Re: (Score:3, Insightful)

            Our own nervous system works as a pretty good associative database, though (in my case at least) it seems to be designed to associate places with objects, ie, it is intended to answer the question "what happened the last time I was here?". So as we develop new applications we tend to develop spatial or geographical models for our data.

            The genetic data you describe is not too different from other things we all have to deal with. Trace or log data. Video streams. Sequences of real time events from practical
        • by smittyoneeach ( 243267 ) * on Thursday July 02, 2009 @08:52PM (#28567355) Homepage Journal
          It seems like a huge chunk of all programming is like being Gerardus Mercator [wikipedia.org].
          You've got a bunch of information with one shape, but you really need it in another shape.
          The code is about dealing with the embarrassment of it all.
          If you've got tabular stuff, and you so very often do, the relational model is fantastic.
          If you're dealing with some kind of graph, you're hating life.
          People coming in complaining that their aircraft makes a poor submarine are initially amusing, but become tedious.
      • Re: (Score:3, Interesting)

        by Marillion ( 33728 )
        Genetic sequences are long strings alphabetic characters. One of the most common representations is the FASTA [wikipedia.org] which deals with the most common type of nucleotide polymorphisms. You can't use exact string searching to find a match which makes BLOBS and CLOBS useless. That said, the meta-data of genetic data is reasonably structured and does load into relational databases fairly well.
    • by julesh ( 229690 ) on Thursday July 02, 2009 @07:11PM (#28566523)

      SQL is great for financial data.

      Actually, this isn't true either. See this article [dbmsmag.com] for pointers to some of the failings of SQL in dealing with financial data, particularly time series (e.g. sales figures, share prices, etc.). Here's another take on the problem [nyu.edu], which essentially is that SQL doesn't recognise that there can be relationships between the rows of a table (e.g., "this happened after this").

      • by raddan ( 519638 ) * on Thursday July 02, 2009 @08:25PM (#28567135)
        A basic premise of the relational model is that there is no relationship between rows. So it isn't surprising that SQL can't see any. Maybe you need to organize that data differently? You can solve a lot of problems in SQL using triggers, temporary tables, and the built-in aggregate and sorting functions.
      • by Kjella ( 173770 ) on Friday July 03, 2009 @12:39AM (#28568637) Homepage

        1) 1996 called, they want their arguments back. For example, most RDBMS have ranking functions now.
        2) Even in 1996, he doesn't know SQL worth shit

        SELECT (prev.sales+now.sales+next.sales)/3 three_day_average
        FROM sales prev,
                      sales now,
                      sales next
        WHERE prev.day_number = now.day_number-1
        AND next.day_number = now.day_number+1

        Easy as pie making most of the calculations he wants. Maybe he should ask someone knowledgable in SQL?

  • by Anonymous Coward on Thursday July 02, 2009 @05:01PM (#28565145)

    When you get a lot of morbidly obese nerds with no life to program for you.

    Meanwhile SQL users get laid.

  • by ChoboMog ( 917656 ) on Thursday July 02, 2009 @05:02PM (#28565149)
    Go fork yourself!
    • Re: (Score:3, Funny)

      It seems an idiot has modded you down because they don't understand very basic database expressions.

      No need to get mad at Slashdot's mod point system, because, after all, if they outlaw giving mod points to stupids, then only stupid outlaws will have mod points... or something like that.
  • by Anonymous Coward on Thursday July 02, 2009 @05:03PM (#28565151)

    Seems to be a silly thing to be against. Relational databases and the stuctured query language may not be perfect, but I bet these people could die in their 90's and people will still be using relational dbs and sql.

    If you want to tout open or cheap dbs and more lightweight types of storage/db servers, then they might have some points, but being against sql is just plain dumb.

    • by Qzukk ( 229616 ) on Thursday July 02, 2009 @05:24PM (#28565391) Journal

      SQL isn't the only way possible to query relational databases. It's nice and does a really good job for even mildly complex queries and I would not want to ditch it just yet, but seriously... who hasn't had a business need for multiple levels of aggregates (eg averages of sums across multiple groupings, say "average across all customers' total balances") As it is, you end up splitting the logic between the database and the application, or creating a view of the first level of aggregation, then querying against that and hoping that the performance doesn't suck total ass.

      • Re: (Score:3, Insightful)

        by profplump ( 309017 )

        I agree, there are problems SQL doesn't solve well. But I think it's unlikely that other, better solutions to those problems will also be superior to SQL where it *does* perform well. As such, "no SQL" is probably not the right plan any more than "SQL only".

      • by Strudelkugel ( 594414 ) on Thursday July 02, 2009 @06:09PM (#28565861)
        OLAP [wikipedia.org] was designed to answer that type of question. MDX [wikipedia.org] is the language used to perform multi-dimensional queries.
      • who hasn't had a business need for multiple levels of aggregates (eg averages of sums across multiple groupings, say "average across all customers' total balances")

        Funny you should mention that. Window functions in the SQL:2003 standard address that need, and there was an article on Slashdot earlier today about PostgreSQL 8.4 being released with support for them. Oracle has for a while now.

      • Re: (Score:3, Interesting)

        by TheRaven64 ( 641858 )
        I was at an HPC talk a few years ago, where the speaker said:

        I don't know what kind of language people will be using for HPC programming in 20 years. I don't know the features it will have. I do know that it will be called Fortran.

        I wouldn't be surprised if the same applies to SQL. The language has evolved a lot over the years, to better express different kinds of data. In 20 years time, I wouldn't be surprised if the most commonly-used subset of SQL is nothing like the subset currently popular, but I'd be surprised if the thing to replace SQL isn't called SQL.

  • Flat Earth (Score:3, Insightful)

    by Seumas ( 6865 ) on Thursday July 02, 2009 @05:06PM (#28565199)

    I've seen strong reactions from various camps with regard to concern over saying no to SQL. I'm not sure why people freak out over it. First, you have to strike out toward new things if you want to progress the world. Second, SQL hasn't caused people to stop using spreadsheets or Access databases. Third, there are groups that get together to dispute that the earth is round; insisting that it is flat. Or that gray aliens are visiting earth regularly and probing our anuses.

    Bring on the next fascinating data technology. SQL will continue to have a major place for many years to come, no matter what happens.

    • Re:Flat Earth (Score:4, Interesting)

      by syzler ( 748241 ) <david@@@syzdek...net> on Thursday July 02, 2009 @05:17PM (#28565321)

      I've seen strong reactions from various camps with regard to concern over saying no to SQL.. Third, there are groups that get together to dispute that the earth is round; insisting that it is flat.

      Corporations represented in this group included the likes of Google, Last.fm, Amazon, and Facebook. Hardly the same caliber of people who claim the earth is still flat. I'm inclined to listen to engineers from these companies if they say that an SQL database does not scale well for vast amounts of data.

      • Re:Flat Earth (Score:5, Insightful)

        by MightyMartian ( 840721 ) on Thursday July 02, 2009 @05:23PM (#28565385) Journal

        And yet where the other corporations; the oil companies, the banks, large merchant conglomerates. In IT we seem to have this sort of myopic view that if it isn't an IT company of some kind, it doesn't exist. Google, as compared to the huge companies that use tools like Oracle, is a bit player. I know that's hard for all of us who have sucked at the teat of silicon valley for so long have a hard time dealing with, but a significant amount of data that has nothing to do with social networking and finding pr0n goes on and does use tools like SQL.

        • by hachete ( 473378 )

          I work for a financial company and if the rest use their bright shiny oracle databases like we do - and I don't think we're atypical - then, no, they have no idea how to use a database. Or build applications. At all. Not a clue.I can't begin to describe the inability, the sheer awesome crap-ness of what they do. The amount of work-arounds that the programmers implement to short-circuit the crap-ness. Really, you have no idea what you're talking about.

        • by kraut ( 2788 )

          Actually, the oil companies almost certainly have huge amounts of non-SQL data; I'm not sure whether seismology data comes in HDF, but it certainly doesn't come in SQL ;) Ditto banks have enormous amounts of non-SQL market data in specialised tick databases. That doesn't stop them from also having other important systems using SQL.

          Vice versa, I'm pretty sure that while Google doesn't store its petabytes of web indexing info in a relational database (why on earth would you?), I'm equally sure that its b

        • by syzler ( 748241 )

          In IT we seem to have this sort of myopic view that if it isn't an IT company of some kind, it doesn't exist.

          I understand that not all companies that maintain large data sets are technology companies. My only point was that when a group of companies known to manage large sets of data say that SQL does not always fit the bill, then I am inclined to listen rather than calling them nuts.

        • by fabs64 ( 657132 )

          I look to the oil companies to innovate in drilling technologies.
          I look to financial companies to hopefully not innovate too much anywhere :-)
          I look to IT companies to innovate in IT.

          I dunno about you, but I've seen an incredible amount of money spent in the last 10 years or so attempting to change those massive relational databases into formats that can be reported on, as well as huge amounts of energy put into moving from one relational schema to another.

          Pretending the big conglomerates present the best a


      • I'm inclined to listen to engineers from these companies if they say that an SQL database does not scale well for vast amounts of data.

        This statement, taken as a whole is pure nonsense. "Databases" scale quite well for "vast" amounts of data. There's retailers that store millions of transactions a day on relational databases that would be out of business if they didn't.

        If I had to guess, I'd say that relational databases might not be a great solution for a quickly evolving web company with possibly consta

        • by leenks ( 906881 )

          There's retailers that store millions of transactions a day on relational databases that would be out of business if they didn't.

          He said vast...

    • Re: (Score:3, Interesting)

      The whole thing is just reactionary mumbo-jumbo. There are kinds of data that relational databases are fantastic for, and kinds of data they're not, and sometimes none of it is exactly perfect. SQL is actually a pretty damned good, single-purpose language. It's not hard to learn, and once you learn it, the differences between RDBMS implementations becomes a little like Javascript, just something you have to put up with, not that a lot of people actually have to worry all that much about writing fully-por

    • Re: (Score:3, Funny)

      You're not going to get many page hits with an attitude like that...

    • Agreed. SQL is a generalized solution that works well for a lot of different things and works extremely well for a subset of those thing. For other applications (like indexing the internet), more specialized solutions are going to kick its ass. It's the same way as any programming you do: the easier and more general the tool, the more you sacrifice for it in terms of speed, efficiency, scalability, whatever.
  • I thought DEC RDB was a pretty good query language. I never got into SQL as a result. I am glad people are thinking about alternatives.
  • by presidenteloco ( 659168 ) on Thursday July 02, 2009 @05:14PM (#28565289)

    The problem is the performance of transactions and persistence and distribution of data techniques, not
    whether we are using a logic-like STRUCTURED QUERY LANGUAGE to ask for data matching certain conditions.

    The latter is still, and will continue to be, very useful.

    It's just that now that we can assume local clusters and WANs worth of co-operating data stores, there
    are probably better, more performant ways of implementing persistence, replication, distribution of data
    than traditional RDBMS implementations.

    The two concerns: The logical model of how we QUERY for data (or combine it in bulk), which is the core of SQL,
    and how we persist it and retrieve it quickly, now have more options for being separated.
     

    • by oGMo ( 379 ) on Thursday July 02, 2009 @06:17PM (#28565955)

      It's just that now that we can assume local clusters and WANs worth of co-operating data stores, there are probably better, more performant ways of implementing persistence, replication, distribution of data than traditional RDBMS implementations.

      You can also assume magical fairy dust and free energy, but that doesn't make it so. You can ask if there are better ways, but you can't assume it, and in the end you will find there is no magic.

      Clusters and replication are NOT NEW. Not even remotely new. There is, in fact, nothing new architecturally at all that would indicate some new capability that hasn't already been repeatedly analyzed and tried. That doesn't mean you can't tweak something for a situation, or that you need a giant Oracle database for everything, but "the web" and "cheap hardware" change the equation by precisely nothing.

      What has changed the equation is cheap, unimportant data, which covers the majority of the web. "Real" applications, where data integrity is important (like say, your bank account), and immediate accuracy guaranteed, require the main thing you use a database for: data integrity. Your facebook page, your google search, that blog entry, or some video on youtube: these don't matter. If it's a little slow, or doesn't update immediately, or you get an error, no one is losing money. No one cares.

      In essence, if a reliable database isn't important for your app, your app isn't really handling important data. This may be fine; in the mainstream, there's a lot of noncritical stuff. But this doesn't make databases unimportant.

  • by JobyOne ( 1578377 ) on Thursday July 02, 2009 @05:21PM (#28565357) Journal
    It's pretty easy to say "yes" to alternatives without saying "no" to SQL.

    Just because a crowbar can pull out a stubborn nail better doesn't mean they should replace all the hammers. Then what would we put nails in with? Different tools for different jobs.
    • Most nails are put in with nailguns. Hammers these days are mostly used for demolition of various sorts, including pulling nails. T
  • by Anonymous Coward on Thursday July 02, 2009 @05:22PM (#28565371)

    If I was to read the article, I bet somewhere someone would be wittering on about Key Value Datastores.

    The brainchild of a generation brought up on high level collections, they learn one (in this case Map) and apply it to everything.

    Sadly SQL, and RDBMS, works for most people. It maps object data well (oh whaaaa, i have to do foreign keys - GROW SOME FUCKING BALLS YOU LAZY GRADUATE!) and it is well understood. And with abstractions like LINQ to query them, even the lazy dumb Windows .NET programmer doesn't have to strain their brain to learn SQL.

    And when you have terabytes of specific unique data, you clearly should go away to work out how best to store it. Even a RDBMS/SQL solution is too generic for all problems.

    • Well, file systems, databases, object inheritance trees, etc, they all are based on the incomplete concept of hierarchical trees and maps. While in reality, everything can be generalized trough graphs. Generic graphs. Of course everyone got its own poor fix for this. File systems have links, databases have foreign keys, and OO languages have interfaces or multiple inheritance. It's a mess, because it is an afterthought.

      I stopped using all those approximations of data structures, and use my own high-performa

    • I'd say LINQ is significantly harder to use than SQL most of the time. The only real exception is when you need to convert the value of a subquery into a comma-delimited list
    • by fabs64 ( 657132 ) <beaufabry+slashdot,org&gmail,com> on Thursday July 02, 2009 @06:09PM (#28565869)

      Saying RDMS's map object data well is a bit of a stretch, they map relational data well and that's it.

      http://www.codinghorror.com/blog/archives/000621.html [codinghorror.com] for some good background on the problems.

      For me using an RDMS as the persistence layer for an object-oriented application has ALWAYS felt like a bit of a kludge. Like we're using it just because it's what we have, rather than the best tool for the job.

  • by SendBot ( 29932 ) on Thursday July 02, 2009 @05:24PM (#28565389) Homepage Journal

    I'm not seeing anything that offers a real advantage over using advanced features like one finds in postgres combined with memcached. Some of my program likes to think of its data as a structured object while other parts like seeing that data as rows in a table (they even link up to other tables through foreign keys!).

    • The main problem with relational databases is that they use a completely different storage scheme than your program does. Databases are organized into tables, rows and columns, but programs are organized into random access variables, structs, and classes. Thus, to use data from a relational database in your program, you need to have a conversion layer that converts from tables to random access, and back. These guys are saying it would be nice if we had a way to store this stuff that didn't require a conve
      • A conversion layer is wasteful when there's only one way to look at your data. In that case, key value pairs can perform better, no question.

        The problem lies in situations where you need to look at the data in 5 different ways. or 50. Then, a single object model for your data is a whole lot less practical than having a conversion layer, and have the data in a very flexible format, like a relational model.

        • Honestly, I don't even care about performance; relational databases perform reasonably well......I just hate all the time I have to spend actually WRITING the conversion layer.
      • by SendBot ( 29932 )

        Well, I have a conversion layer to create the object my program uses, but I can't think of a need to convert it back. All the things that make it what it is are a result of all the little things that interact with the db. Using triggers, it knows when to update parts of itself. The parts that interact with the db often don't care about the object, even if it's being used as in input to those parts.

        When I DO need to care about the object, replicate it, or maintain persistence, then I use...

        DUN DUN DUN!

        memcac

  • go back to flat files aka DAT files.

    Use the old DBase III standard DBF files?

    Use the old Lotus 123 WK1 files?

    Use MS-Office MS-Access MS-Excel etc files?

    Use comma separated values files?

    SQL set a standard for relational databases, a structured query language that almost any database can use and then build extensions to it.

    Will the Post-SQL age begin, and will it be object oriented and a fifth generation language?

  • by syousef ( 465911 ) on Thursday July 02, 2009 @05:37PM (#28565535) Journal

    Saying no to SQL and relational databases is just fine if you've got something better to replace it with. However I know of no such thing. The reason they're popular is that they are so powerful for data storage. If something better came along you wouldn't even need to say no to SQL. You'd just say yes to the newer better rival.

  • by j. andrew rogers ( 774820 ) on Thursday July 02, 2009 @05:47PM (#28565627)

    SQL is not a database, it is a standard interface to a feature set commonly associated with relational models. Before everyone standardized on SQL, there were other relational query languages. The "No" part of "NoSQL" refers to the fact that some basic elements of relational implementations cannot be usefully expressed using a much simpler distributed hash table model.

    All the "NoSQL" does is eliminate all the parts of traditional relational databases that do no scale -- discarding the bottleneck rather than fixing it. These are things like joins and external indexing. Unfortunately, discarding those things means you discard a lot of very important functionality as a practical matter, notably the ability to do fast, complex analytics. Adopting the NoSQL architecture runs contrary to the trend toward more real-time, contextual analytical processing. There are a great many analytical applications that are not amenable to batch-mode pattern-matching, and the NoSQL model is a lot less applicable than I think some people want to acknowledge. In its domain, it is a great tool but it has many, many prohibitive limits. We are essentially trading power for scale.

    That said, do not take this as an endorsement of traditional SQL relational databases either, as they have a number of serious limitations themselves. As just mentioned, a number of the core analytical operations those models support are based on algorithms that scale poorly. The SQL language itself has mediocre support for many abstract data types (e.g. spatial) and data models (e.g. graph), which in part reflects the inadequacies of the assumed underlying database algorithms (e.g. B-trees) that are implicit in SQL. The inability to efficiently do event-driven/real-time applications is also more a reflection of the access methods used in databases than any intrinsic weakness in SQL; SQL may be clunky for that purpose, but that is not the real limiter.

    A truly revolutionary deviation from SQL would usefully implement a superset of the features SQL supports, not take them away. Of course, we would need access methods more capable than hash tables and B-trees to useful implement those features, which is a lot more work than discarding features that scale poorly. NoSQL is a stopgap technical measure for that small subset of applications where the serious tradeoffs are acceptable.

  • by kpharmer ( 452893 ) on Thursday July 02, 2009 @05:54PM (#28565701)

    Note that most of these solutions come from the interwebs, social networks, etc. And it isn't so much anti-sql as it is anti-relational database (sql != rdb).

    The basic premise is that we need different solutions that: can scale very high for very narrowly scoped reads & writes, don't need to perform ranged queries / reporting /etc, and don't need ACID compliance. And that may be the case. Sites like slashdot, facebook, reddit, digg, etc don't need the data quality that ebay needs.

    On the other hand, ebay achieves scalability AND data quality with relational databases. And when I've worked with architectures that scale massively and avoid the relational trap for better solutions - they inevitably later regret the lack of data quality and complete inability to actually get trends and analysis of their data. It *always* goes like this:
        Me: So, is this thing (msg type, etc) increasing?
        Developer: No idea.
        Me: Ok, so lets find out.
        Developer: How?
        Me: I don't know - typical approach - lets query the database.
        Developer: It'll take four+ hours to write & test that query and then days to run. And when it's done we might find that we wrote the query wrong.
        Me: What?!?
        Developer: We had to do it this way, you can't report on 10TB databases anyhow
        Me: What?!? Are you on crack? there are dozens of *100TB* relational databases out there that people are reporting on
        Developer: well, we probably don't need to know what that trend is anyhow
        Me: I'm outta here

    • Re: (Score:3, Informative)

      by GryMor ( 88799 )

      Yah, good luck with that. Unless the index already exists to do nearly exactly what you want, queries against multi terabyte production oracle tables have this bad tenancy of never completing, if you are lucky, and (effectively, from the perspective of the app running on top of it) taking down the database if you are not so lucky.

      If the index doesn't exist, good luck adding in less than a week or two.

      For the most part, novel trend and behavior information is trivial to instrument in the service layer as a s

  • by 4to6Offshore ( 594235 ) on Thursday July 02, 2009 @06:09PM (#28565865)

    First: my mantra: Data belongs to the organization, not the application... if the app fails and data is accessible then we all go on - if the data fails or is locked away - what was the point of the app again?

    In a SQL database then data is understood by the organisation, DBAs and data architects. If left to app developers taking an app-centric approach to data... I get nervous quickly.

    So long as the data is just as definable and accessible as current SQL databases then all good - give me an app with some odd-ball storage then it is bye-bye.

  • could we see a rise in the use if tree/hirearchial Databases like LDAP?

  • by 93 Escort Wagon ( 326346 ) on Thursday July 02, 2009 @08:00PM (#28566919)

    So a bunch of Excel users got together for dinner in San Francisco - why is this news?

  • Seriously misguided (Score:3, Interesting)

    by Stu Charlton ( 1311 ) on Thursday July 02, 2009 @11:33PM (#28568357) Homepage

    Trash SQL in favour of coding all your data access needs. Welcome back to 1973, guys.

    It's not like [wikipedia.org] we could do parallel SQL in the 1980's. Or that you can't do parallel SQL in a compute cloud [vertica.com] today.

    No, It basically seems like they don't want to pay software vendors any money for database technology. That's mostly what the arguments boil down to. Oracle RAC is very scalable, arguably easier to do at massive scale than MySQL - but you have to pay Oracle money. For an Internet startup, I can understand why you'd take your chances with "roll your own". For an enterprise... I think not.

  • by Lord Bitman ( 95493 ) on Friday July 03, 2009 @03:12AM (#28569301)

    SQL syntax sucks, is inconsistent, and just non-standard enough at its corners that it's completely annoying to write anything for more than one DB. Also lacks various features which logically _should_ be there, because of the relational back-end. SQL is a toy, and though I'm the guy everyone in the office turns to if they want to write a query that does more than SELECT * FROM sometable, that doesn't mean I have to like it.

    But that's not the fault of relational databases. The relational logic makes sense, and we'll be seeing it referenced in countless "new ideas" that come along for years, just as ideas which Lisp already had in 1970 will be touted a new features on for the next millennium (you hear? PHP can do Lambda functions as of yesterday!)

    SQL sucks, but SQL is NOT what makes something relational.

Fast, cheap, good: pick two.

Working...