Want to read Slashdot from your mobile device? Point it at m.slashdot.org and keep reading!

 


Forgot your password?
Close
typodupeerror
Databases Programming Software Data Storage IT

The Future of Databases 315

Posted by timothy from the your-flight-status-is-'maybe' dept.
gManZboy writes "Ever wonder where database technology is going? This is something that Turing award winner Jim Gray from Microsoft has given a lot of thought to. He recently published an article in which he looks at the many forces pushing database technologies forward, and what those new technologies will look like. Gray writes, 'the greatest of these [research challenges] will have to do with the unification of approximate and exact reasoning. Most of us come from the exact-reasoning world -- but most of our clients are now asking questions that require approximate or probabilistic answers.'"
This discussion has been archived. No new comments can be posted.

The Future of Databases More

The Future of Databases

Comments Filter:

  • Turing award winner? (Score:5, Funny)

    by Anonymous Coward on Monday May 02, 2005 @08:15PM (#12414737)
    As in, he passed the Turing test?

  • approximate answers.. (Score:5, Funny)

    by ShaniaTwain ( 197446 ) on Monday May 02, 2005 @08:17PM (#12414764) Homepage
    most of our clients are now asking questions that require approximate or probabilistic answers.'"

    • 42..ish
    • > > most of our clients are now asking questions that require approximate or probabilistic answers.'"
      >
      > * 42..ish

      Larry: Of course, he only had the two arms and the one head, and he called himself Jim Gray.

      Melinda: But you must admit, he did turn out to be from another planet.

      Larry: By my yacht! Melinda Gates!

      it.slashdot.org: Infinity minus 1. Improbability sum now complete.

      Larry: What are you doing here?

      Melinda: With a degree in human-computer interaction and another i

    • More seriously, this means something like + Lucene[1] (or, more likely, lucene4c [2])

      [1] http://lucene.apache.org/ [apache.org]
      [2] http://incubator.apache.org/lucene4c/ [apache.org]

  • Why complicate things so much? (Score:5, Interesting)

    by bigberk ( 547360 ) <bigberk@users.pc9.org> on Monday May 02, 2005 @08:20PM (#12414785)
    How many times have we heard of huge sites going down because databases become corrupt or unrecoverable, or of the huge resource strain (memory and CPU) from a large database?

    In my opinion, the future of databases is nothing so complicated as pitched here -- but rather a move to simpler, more reliable back ends where the filesystem is the database. This is certainly the vision pitched by Hans Reiser and reiserfs [namesys.com], which aims to put more database like intelligence within the filesystem. So you eliminate extra unnecessary layers that just eat up resources and create fragile databases.

    • Re:Why complicate things so much? (Score:4, Informative)

      by fireboy1919 ( 257783 ) <rustyp&freeshell,org> on Monday May 02, 2005 @08:29PM (#12414859) Homepage Journal
      Ah yes. Harken back to the earlier days, when databases were just files on a file system, and did not distribute the resourses at all.

      Certainly that's not going to lead to more crashes.

      Certainly it's a better idea than, for example, distributing the databases and using load-balancing and regularly scheduled back-ups to ameliorate the loss of the least realiable portions of a databases design - the harddrives.

      When you've only got a hammer, everything seems like a nail...what does Hans Reiser do? He could be right. Microsoft is jumping on the filesystem-database wagon with their new filesystem, and we all know that if anyone knows and cares about reliability it's Microsoft.

    • Re:Why complicate things so much? (Score:5, Insightful)

      by dioscaido ( 541037 ) on Monday May 02, 2005 @08:34PM (#12414910)
      This random example just server to clarify what you mean -- How implement a airline database that has entries for 1,000,000 customers, 150,000 flights a year, and 12,000,000 reservations a year? and what would a query look like to find an open flight on a particular date range, and register a reservation? And how would doing all this on a ReiserFS be any less prone to data corruption than an often backed up database?
      • Sorry, I was a bit unclear in my post. I meant that I'd like the original poster to clarify what they mean. Given the example I pose, how exactly would you 'go simple' and use reiserfs instead of a full SQL system that handles this problem perfectly?

    • Atomicity in filestores is a great benefit (Score:2, Informative)

      by Anonymous Coward
      The parent makes a good point, and it's pretty easy to see why if one holds off the usual anti-Reiser reactions and thinks it through a bit.

      Databases require a mechanism for atomicity to create their transactions, and because no common operating system has ever provided such, they need to implement it themselves at application level. It's like the bad old days before PCs provided networking, and you had to run up your own networking stack if your application needed comms.

      Well reiserfs has the goal of pro
      • Does reiserFS support atomicity at the group level? Can I edit a group of 30 files, and only once the modifications are done for those 30 files do we commit it to the file system, and in any other case none of the files change? That is a major feature of a transactional database, where you can modify various tables simultaneously and if at any point there are issues, all the data is easily restored by doing a roll-back.
        • ... oh, and the file system should also verify the integrity of the files, and the system as a whole -- make sure that your changes are "allowed" (both state-constraints and transition-constraints), make sure that everything works together (imagine your FS making sure that your changes to your mail server config match up with your changes to the user list?) ... ... oh, and allowing multiple users to modify files at the same time, and know enough about the file formats to reconcile possible conflicts (not st
    • Hans Reisers vision is about unifying namespaces (filesystem, relational database, XML, etc...) by providing the functionality in a filesystem to make this reasonable. In otherwords, making the file system better than current databases.

      Do we evolve the file system into a database (Reiser approach) or evolve a database into a file system (Microsoft WinFS approach)?
    • Databases such as oracle used to do this, use the underlying disks directly without an extra layer, i.e. use raw devices. In a sense, the database and filesystem were one. However there has ben a move away from this: using a separate filesystem below makes it easier to manage and move around, while costing only about 1% performance.

      Note: huge sites don't go down because of 1% performance loss due to an extra filesystem layer.

      And I doubt very much that applications that use filebased storage instead of a r

    • Well, I suppose we could go back to the original UNIX way of doing things:

      Everything is a string of bytes on the hard disk.

      No file system at all.

      Somehow I don't think that's going to take off. Grep is great - but it's not THAT great.

    • How many times have we heard of huge sites going down because databases become corrupt or unrecoverable, or of the huge resource strain (memory and CPU) from a large database?

      How many times? Not all that many, in my experience. How many times when the sites were running off a hardy RDBMS like Oracle, rather than something in the MySQL range? Even fewer.

      Of course, "websites going down" is not exactly the best indicator of database reliability in the first place...

      While you're proposing making database

  • A real problem comes full circle (Score:5, Interesting)

    by Anonymous Coward on Monday May 02, 2005 @08:22PM (#12414804)
    Data is data. Just data. Save it, read it, sort it how you like. Efficient results mean having rapid, low-latency access to data.

    Add code to it, and you have data+code.

    OF course, code is data, and thus data can be treated as code, and handled by other code. LISP does this moderately well.

    But you can't avoid the fact that, as it stands, databases are just engines for keeping your data structures outside your code, or when you add code to them, engines for reading your data structure for you so that you don't have to think about how to do it. ... except that you still do, because SQL isn't a way of avoiding logical errors. ... and that they still don't save time. At best, they allow for some parallelism, external access to the data, and a separation of concerns.

    I'm getting rather tired of the fad that databases should be tacked on to everything, ranging from a shopping list to guidance systems. When did adding overhead become the mark of skill?

    • Re:A real problem comes full circle (Score:4, Insightful)

      by zappepcs ( 820751 ) on Monday May 02, 2005 @08:52PM (#12415063) Journal
      I think I agree with the parent. Databases are methods of storing and retrieving data. Trying to make queries fuzzy, or less structured is just wrong.

      If you want to be able to ask probablistic type queries of a database, you need to add some code between you and the database.

      More to the point, the fuzzier your logic is, the higher the probability that your database will not contain all of the answers on its own, and you will have to cross reference your data to the data owned by someone else or gathered from a different disparate source.

      It sounds like M$ is going to try to re-invent data warehousing? and then of course, patent it.

      Trying to make the database do everything is not right and simply doesn't make sense. The code that accesses the data for you needs to do the fuzzy probablistic stuff.

      P.S. I have no faith that M$ (no matter who they hire) can effectively provide the code required to make it work in the idealistic manner spoken of... mostly because they would have to patent accessing other people's data before they could do it.

      Just my thoughts
    • When did adding overhead become the mark of skill?

      The second it became profitable to market it as such.

      KFG

    • . . . I no longer have to debunk the sweeping claims of an AC--I can simply provide a link [wikipedia.org].

  • Great Article (Score:5, Insightful)

    by Spaceman40 ( 565797 ) <blinks AT acm DOT org> on Monday May 02, 2005 @08:23PM (#12414810) Homepage Journal
    The requirements for a database today aren't too much different from those twenty years ago - except for what we want to get out of them.

    Now that data mining is a $[insert large number here]million industry, databases are being asked to do a lot more processing with this data than before. For example: old database query = get these attributes from tuples that match this pattern. New database query = determine how likely a user who has accessed 30 or more times this last month is to subscribe to the second-level pay service within the next ninety days, with or without an email advertising said service.

  • In other words ... (Score:5, Insightful)

    by Daniel Dvorkin ( 106857 ) * on Monday May 02, 2005 @08:26PM (#12414835) Homepage Journal
    ... MBA's want the magic glowy box to do their thinking for them.

    Fortunately, Microsoft will be there to take their money.

  • $article_title by $blowhard. (Score:5, Funny)

    by Anonymous Coward on Monday May 02, 2005 @08:28PM (#12414855)
    $techology is dying. It will be replaced with $replacement. Insert 4000 more words sprinkled with $random_buzzwords. I am so smart! The end.

  • Accountable bitemporal DBs (Score:4, Informative)

    by G4from128k ( 686170 ) on Monday May 02, 2005 @08:32PM (#12414891)
    The rise of Sarbanes-Oxley [slashdot.org] highlights a key insecurity in the accountability of enterprise systems. Although the high-level applications can do a good job of tracking who did what to the financial data, the core DB may be open to tampering. If a DB admin with the right password can manually diddle a field in a database, they can change the financials of the company.

    In contrast, a secure bitemporal DB would record not only the date of the what the data refers to (e.g., the purchase order was entered on March 3rd, 2004) but also the date(s) of any modifications of the data (the quantity and total was changed on December 31, 2004, Uh-Oh!).

    This is more than just securing the DB with a hierarchy of privileges, it means that no one can overwrite the old data or change any data without creating an audit trail. This, of course, also means changes in the DB, OS or file system to make critical data only accessible through a secure DB layer that tracks changes (e.g., no accessible plain-text DB data structures). These same concepts could be used (probably are, for all I know) for OSS version control to track who did what and when to the code.
    • As long as the data's there, it can be changed. The only thing you can do about it is turn to verified cryptographic solutions. As long as the data is there, someone will have the password necessary to change it. If you can't do it from within the database program, you can edit the file on disk and change it that way.

    • Re:Accountable bitemporal DBs (Score:4, Insightful)

      by TopSpin ( 753 ) * on Monday May 02, 2005 @09:46PM (#12415493) Journal
      The rise of Sarbanes-Oxley highlights a key insecurity in the accountability of enterprise systems.

      Yeah, I've heard that one too. Reality has a way of factoring out the ambiguity of such abstract, open-ended claims.

      On way to deal with the problem of DBAs and their ability to access/modify financial data is to register them with the exchange, just like the finance and executive types. Now they're Sarbanes-Oxley insider compliant! That's what has been done where I earn my living.

      Thus, we may dispense with elaborate schemes of secure data version control using unspecified, hypothetical systems, paid for with budgets that don't exist. Next!

      Until some future revision of Sarbanes-Oxley begins to specify the design and implementation of electronic finance systems, no one can claim a database is more or less susceptible to malfeasance than a locked filing cabinet. That's why the auditors stop once they've concluded you're changing your password with adequate frequency.
    • In contrast, a secure bitemporal DB would record not only the date of the what the data refers to ... but also the date(s) of any modifications of the data

      You mean your RDBMS doesn't have full auditing capabilities?

      What are you using SQL Server?

      Any "enterprise" RDBMS worth it's salt has had such features for 20 years.

      Of course, before you enable full auditing, you'd better double your IO capacity, well as increasing your CPU capacity.
    • The rise of Sarbanes-Oxley

      You misspelled 'unholy birth'.

  • I predict... (Score:4, Interesting)

    by rainman_bc ( 735332 ) on Monday May 02, 2005 @08:38PM (#12414949)
    Better indexing, faster lookups...

    That's... about... it...

    Object relational was the "new thing" that didn't really take off as well as they'd hoped.

    Hell, I work with people who still can't handle compound keys and joins well...
    • All the industry db experts should shift their focus to working with MySQL. You look at oracle and it's just so bloated.

      The worst is every oracle db runs on some expensive production environment hardware. Every MySQL db runs on a cheap PC. Until this is changed, oracle is stuck as the industry standard.

      • The worst is every oracle db runs on some expensive production environment hardware. Every MySQL db runs on a cheap PC. Until this is changed, oracle is stuck as the industry standard.

        What are you smoking? I've seen oracle running on compaq proliant pentium pro systems, and i've worked with mysql installations running on sun enterprise 5000's. Oracle is not the industry standard because it runs on bigger hardware. Oracle is the industry standard because even though it is the biggest pain in the ass to
        • it's a little bit insane to suggest that all the experts should quit working on real databases and work on MySQL instead.

          You're right. They should use PostgreSQL instead. ;)

          But seriously, an Oracle DBA (who's not dependent on GUIs) would feel at home with PostgreSQL 8.0.
    • Object relational was the "new thing" that didn't really take off as well as they'd hoped.

      You're thinking of Object databases, which indeed did not take off at all.

      However Object-Relational systems are EVERYWHERE. There's hardly a big database anymore that doesn't have several object-relational mapping systems between it and code...

      Object->Relational mappers have taken off in a big way, which is good in a way since the databases can remain the nice placid solid systems they've always been and you ca

  • I want clustered databases for high-availability (Score:5, Interesting)

    by SpecialAgentXXX ( 623692 ) on Monday May 02, 2005 @08:45PM (#12415005)
    The "next great advancement" in databases will be when I can setup 2 or more linux servers and have them act as a single database server. Our database server is the most expensive item in our datacenter because it's an N-way IBM server.
    • You certainly can do it, it's just not easy. I agree, though, that the first group that can make this easy enough (run program, login to DB network with DB servers, access DB network just as single DB is accessed), will become very popular.

    • Re:I want clustered databases for high-availabilit (Score:5, Informative)

      by kpharmer ( 452893 ) * on Monday May 02, 2005 @10:09PM (#12415661)
      > The "next great advancement" in databases will be when I can setup 2 or more linux servers and have
      >them act as a single database server. Our database server is the most expensive item in our datacenter
      >because it's an N-way IBM server.

      lol, IBM has supported *exactly* what you are talking about for at least five years.

      That is, you can spread your db2 database across 10,100, or 1000+ linux commodity boxes (ideally blades). Or you can use windows, or aix, or solaris, or hp-ux, etc. Of course, those individual boxes can be SMPs in their own right - so a thousand 8-way aix boxes is certainly possible, if not cheap.

      Oracle is now in this game as well - oracle 10g can certainly support 32, and maybe 64 individual linux boxes in a cluster. The techniques are different between the two - oracle might be better at transactional systems. db2 is definitely better at data warehousing, data mining, etc.

      Of course, there are still benefits to a big smp: a single P570 16-way will cost you $250k. But each of those 16 cpus is multi-code (and far faster than intel or amd), and with its micro-partitioning - it can run at least 150 linux or aix lpars (logical partitions). These lpars can grow or shink as they need - so you aren't always over-buying for size, buying new hardly-used hardware, or having to colocate apps on a busy server - when a different os would be preferable. Not to say everyone should go this way - but there are definite benefits.
      • Digital did this over ten years ago. One of the things that Oracle inherited when they bought RdB from Digital was the cluster support. However it seems they tool a long time to get the technology into their own RDMS.
        • > Digital did this over ten years ago. One of the things that Oracle inherited when they bought RdB
          > from Digital was the cluster support. However it
          seems they tool a long time to get the technology
          > into their own RDMS.

          I've got fond memories of an 800 gbyte billing & customer data warehouse on rdb around 1995 - giving sub-second access and running on a vms quad. That was such a slow system compared to what we've got now - but it sure handled a ton of data well.

          On the other hand, I don't re
    • Who moderated this interesting? "-1 Clueless" or "-1 ill-informed" is more like it.

      2 servers acting as a single database server has been available for many years...e.g., Oracle 9i RAC, Oracle 10g, DB/2's something or other, etc.

      • My original post said:

        The "next great advancement" in databases will be when I can setup 2 or more linux servers and have them act as a single database server.

        i.e. Out of the box, I can setup a database cluster. I'm talking about costs for HA. If I can dump the big iron for 2- or 4-way x86 servers, I'd save money. But if I need to pay a lot for support for Oracle's RAC, or a custom setup/installation, etc., then I'm not saving money. It all comes down to the bottom line.

  • It warms my heart... (Score:3, Interesting)

    by Baldrson ( 78598 ) * on Monday May 02, 2005 @08:50PM (#12415039) Homepage Journal
    To see Bill Gates' organization of "really smart people" spinning their wheels so energetically really warms my heart.

    I can see they've no hope of being any competition at all come the real db revolution.

  • Well, I should no longer expect news from Slashdot but :

    this is almost the same content as his SIGMOD 2004 speech,
    which is available since April 2004 :
    http://research.microsoft.com/research/pubs/view. a spx?tr_id=735 [microsoft.com]

    How is the refurbishing of an one year old article news?

    (And, BTW, I find the keynote speech better structured then the refurbishment)
    • Yes yes, but, despite the "news for nerds" tagline, Slashdot is really more of a dicussion site than a news site. If you really just want the latest in tech news, try someplace like http://freshnews.org/ [freshnews.org] where you can get everything consolidated into a nice digest for you, including the ancient slashdot headlines. If you want a discussion forum for geek chatting, stupid memes, repetitive jokes and overall fun, then read slashdot.

      Personally, despite some of the crap on slashdot, I still stick around, mostly

  • That is what SAS is for... (Score:4, Insightful)

    by the eric conspiracy ( 20178 ) on Monday May 02, 2005 @09:13PM (#12415249)
    most of our clients are now asking questions that require approximate or probabilistic answers

    What are my chances of getting laid tonight...
    What are the odds of my winning the lottery...
    What are the chances that my boss will find out about that phoney dinner reciept...

    Seriously, SAS stat analysis software does exactly what this numbskull is talking about. You don't need a new kind of database, merely somebody with training in stats.

    • Back when I was a kid, people used a Magic-8 ball to answer questions like this.

  • Hmmm Databases (Score:5, Interesting)

    by Chitlenz ( 184283 ) <chitlenz@@@chitlenz...com> on Monday May 02, 2005 @10:09PM (#12415668) Homepage
    As a 15 year DBA, currently we are working with some of the would-be far reaching (to most people) concepts described in this paper. The idea of a TRUE SQL Debugger is like, so big it's sick. Quest offers some tools that kinda sorta do this for Oracle systems, but a true realtime debugger would save me YEARS of work during my career as a SQL coder. For an Idea of scale, The last replication project I wrote for an employer propogated over Oracle DB_LINKS via triggers to synchronize a dataset in two cities, log it, and do something with errors. Because this particular system was a Peoplesoft installation, it was a subset of 6800 tables and 15k lines of code give or take some triggers, with NO debugger. OMG, it's like a "finally" moment to have someone even claim to be fixing this soon in their architecture.

    Next, there was some inane reference to reiserfs above, which clearly ignores what a database fundamentaly both is, and is becoming. It really began (and I hate to admit this as a former Solaris/Oracle admin) with SQL Server 7 and Oracle 8, and the concept that a database should be object programmable. Reiser is not going to be streaming still frames of image data fast enough to a remote client to rebuild seamlessly into a movie, for instance. Or recalculate all of a company's business logic for point of sale systems so that, for instance, the wrong type of credit card gets rejected, or so a supply chain gets populated, the list is endless. Reiser, and for that matter VFS and the other myriad of database enhanced filesystems, are tools. Good ones, but tools...

    It's interesting to note that MS has finally figured out that the "n-tier" was a dumb idea. It's almost like, well you take all this shit, then sell it through a middle man, but expect to not have to pay him anything for brokering. Like, duh. We actively benchamrked this process, in fact, and discovered that it does, not suprisingly, take time to pass data through an extra server.

    Workflow is life. It's what make this page exist (SD is I believe run in MySQL). The idea of publishing-subscribers with atomic transactions is hardly new, but I agree with the authors that this is the direction of the market, simply because businesses now are getting spread all over. Read - If your job just went to India, learn to be a DBA, cuz when all that shit they sent over there comes back, you can bet its going to be a mess (and is a mess actually already, which is why, in particular, people in ERP fields that intertwine with mine(as a DBA) demand and recieve very large salaries, 200$US an hour is not unusual). The reason this particular ramble is relevant, is because lots of global companies are either looking at, or are already implementing, the idea of data grids, where all the data servers inside a global network stay in sync. Suzy the secretary checks out a document in Baltimore, and that document flags as in use in Madrid through transactional replication within a kind of database trust-relationship network. It's a very very good way for companies with lots of data to keep it all together, but today it's still a pain in the ass to manage.

    Vertical partioning is pretty much worthless except to data warehousing installations, most of whom are probably running on strong equipment already (to have that much data). Not to mention, I believe (I'd have to check, since it's not a feature I'd really use) Oracle's 10G product allows for this already if you really want it. Materialized views is another point here that raises my hackles. This guy is writing about the wonder of materialized views and column partitions, which ARE a cool performance cheat in large systems, but make no mistake that by the time you get to this point, you are probably rearranging deck chairs on the titanic anyway. Essentially Materialized views precache SQL resultsets into a temporary table which gets constantly updated so it can always provide a full resultset without having to parse the parent table. This is processor and space expensive. Vertical par
    • > XML SUCKS. PASS IT ON.

      agreed - as a human-readable way of persisting data it stinks. as a way of persisting data it stinks. But it isn't bad as an over-the-wire protocol.

      > vertical partitioning...

      Hmmm, don't see much vertical partitioning in data warehouses any more. Used to on oracle years ago, but can't even remember why. But I am finding that both mean range & hash partitioning work well together. The range is cheaper & easier to implement but only gives a performance benefit when

      • Re:Hmmm Databases (Score:4, Interesting)

        by julesh ( 229690 ) on Tuesday May 03, 2005 @09:33AM (#12419332)
        [XML] isn't bad as an over-the-wire
        protocol.

        Yes, it is. I've worked on a project that allowed offline modification of a database by replicating a copy to user's PCs, and it originally used XML as the format for data transfer. We got a 30% speedup by switching to tab-separated variables with a line of metadata at the start of each chunk of the stream. Any technology that costs that much in overhead and provides little or no perceivable benefit is a waste of time. (Of course, if your data isn't relational, this is probably not much use to you, but then... what are you storing it in? XML documents?)

        The only justification for XML is that there are a lot of tools out there that work with -- I use it is an intermediate interchange format between different environments because the libraries available make it easy with just about anything I want to access the data with.
    • there isn't a +6(Interesting).

      The big end of the database world has always seemed strange to me. Your post provides some interesting views on that area.
  • From a practical point of view, the kind of databases that people like Gray have made a career out of have been very useful up to a point, no question.

    On the other hand, those databases have already been pushed far beyond their limits: people have been using them inappropriately in many applications. Much of the "hot" recent stuff Gray mentions is not new technology: smart people have been proposing it and using it for years, only to be beaten down in the market by the relentless push behind relational te

  • Google is a good example... (Score:3, Insightful)

    by dantheman82 ( 765429 ) on Monday May 02, 2005 @10:34PM (#12415847) Homepage
    I'd personally ask a Google employee where the future of databases is heading. The Google FS really shows where databases are moving...

    I give Gray a lot of respect in most cases because he's a really smart guy. But the math and computationally-intensive parts should be focused in the probabilistic searches.

    In one sense, though, Gray is quite right. And this is the direction of speech recognition. I might add that the Speech Server beta out by Microsoft is quite good...even at this stage.
  • I see the future now and it will happen in two phases: getting rid of SQL and then replacing it with something half-way decent (like a properly implemented relational algebra.)
  • Picture this... memory nowdays is a hell lot cheaper than a full Oracle Licence. So, instead of investing on a DBMS why not buy massive quantities of ECC memory and keep all instances of your data in-memory for near instant access?

    Crazy idea, huh? What if I said that this can be as fast as 8000 times faster than Oracle? And 3000 times faster than MySQL!

    Crash recovery? No big deal, keep a serialized version of your in-memory-objects, and a transaction log and you're set!

    Read more at:
    http://www.prevayler.org/ [prevayler.org]
    • > So, instead of investing on a DBMS why not buy massive quantities of ECC memory and keep all
      > instances of your data in-memory for near instant access?

      because a *well-tuned* relational database with a 1:4 ratio of memory to disk is almost as fast as an in-memory database - due to efficient caching

      because some queries require an enormous amount of temp space. supporting them can easily double your space requirements - which have to be purchased in memory.

      because if you just want to run your datab

    • Re:The future of databases is... no Database at al (Score:5, Insightful)

      by rossifer ( 581396 ) * on Tuesday May 03, 2005 @12:53AM (#12416661) Journal
      [misc drivel] Read more at:
      http://www.prevayler.org/ [prevayler.org]

      Oh my dear god. You've never actually used Prevayler have you? Prevayler isn't nearly as useful on actual data problems as Prevayler's worshippers would have you believe.

      I know this because I tried to use it. If you'd ever tried to use it, you'd know how unbelievably poorly it performed when attempting to implement real world queries. You have to implement every query in Java, and Java is a particularly poor implementation choice for creating complex queries.

      What if I said that this can be as fast as 8000 times faster than Oracle?

      This "performance comparison" that the Prevayler group trots out is particularly funny as their test uses a single ArrayList of objects as in-memory "storage" and then "queries" it by index. Not exactly a realistic problem. Try a query across four classes with a few million instances of each class and you'll quickly discover what relational databases are good for.

      Regards,
      Ross

    • Bullshit artist... (Score:2, Informative)

      by Anonymous Coward
      Prevayler exposed [pyrasun.com].
  • The only force that can change the nature and architecture of current database technology is a fundamental change in the way they are used. Change the requirements and the technology will change to meet the new requirements. Change the requirements in a radical way and you will get radically new technology.

    The use of a database as a file system will require radical new technological advances in database theory as the current methods break down under the new requirements. The functionality of the file sys

    • Pray for WinFS.

      Like, one assumes, one would pray for a sick friend?

      What is now considered the traditional file system API is not well designed for databases, but there have been other ones that might be better used in the past: an API that does for databases what the UNIX API (after all, virtually all file system APIs these days are based on it) did for files is needed.
  • At one time, I though object oriented databases were going to be the next big thing.

  • regex (Score:2)

    by yagu ( 721525 )

    Regex! As processors get faster, memory gets cheaper.... I wouldn't be surprised to see more better, faster, etc. implementations of regex that allow doing what full blown databases do today. Of course that's in a read/only context, but I've implemented full blown "database" applications centered around the regex. And some will point out regex doesn't deal with integrity and data management issues, I would point out many databases are implemented in overkill mode where data integrity and management are h


  • This notion of "active databases" seems to me to be interesting but fraught with problems.

    Not least of which is the old bugaboo - documentation. How do you document a system composed of myriad triggers scattered on myriad tables in myriad databases communicating over the Net?

    All I know from trying to decipher ONE Oracle Forms application at City College of San Francisco is that it is nearly impossible to get a handle on what happens where when. There appears to have been NO effort made by Oracle to enab

  • To answer the question... (Score:3, Funny)

    by jim_v2000 ( 818799 ) on Tuesday May 03, 2005 @02:55AM (#12417413)
    "Ever wonder where database technology is going?"

    Yeah, all the time.

  • You mean in the last 25 years? Nowhere. (Score:4, Insightful)

    by Qbertino ( 265505 ) <moiraNO@SPAMmodparlor.com> on Tuesday May 03, 2005 @03:38AM (#12417581)
    Honestly, folks, databases are like crutches: Pathetic, but you when you need them, there's hardly an alternative. They are the living proof that abstract concepts and computer simulation of those on real world hardware need the strangest type of hacks to be mended together.

    On top of that - and this is the worse part - what we call databases today is nohing much more of a historically grown apocalyptic chaos. With one of the crappiest programming languages ever as a cornerstone of its technology. A weedy mumbojumbo of wanna-be virtual machines, wanna-be server daemons, makeshift security layers, obstrusive user management and pseudo operating systems and a bazillion proprietary variants of said programmin language. With features bolted on left right and center. This basically is the case with any current DB in widespread use, be it MySQL, Oracle or anything inbetween.
    And if you look at the core of it Database technology and how long it has been that way there isn't much hope that DB's will go anywhere anytime soon.

    Then again, if you want to get a glimpse of a possibly brighter future, I'd actually recomend Zope [zope.org]. I consider it's object relational DB a working proof of avantgarde "database" concepts and a prototype of what DBs generally could look like in the future if anyone were interested.

  • Most of our clients... (Score:3, Insightful)

    by cardpuncher ( 713057 ) on Tuesday May 03, 2005 @05:46AM (#12418024)
    ... are now asking questions that require approximate or probabilistic answers

    I suspect that may translate as "most of our clients want to be given easy answers to difficult questions".

    I'm sure there'd be a big market for a database system that stored flight bookings and could answer the question "which of our customers is a terrorist?". You don't address that market with new technology, though, but by developing new sources of snake oil.

Slashdot Top Deals

Quantity is no substitute for quality, but its the only one we've got.

Close