Slashdot Log In
MySQL Falcon Storage Engine Open Sourced
Posted by
kdawson
on Tue Jan 02, 2007 03:52 PM
from the stooping-to-conquer dept.
from the stooping-to-conquer dept.
An anonymous reader writes "The code for the Falcon Storage Engine for MySQL has been released as open source. Jim Starkey, known as the father of Interbase, is behind its creation; previously he was involved with the Firebird SQL database project. Falcon looks to be the long-awaited open source storage engine that may become the primary choice for MySQL, and along the way offer some innovation and performance improvements over current alternatives." This is an alpha release for Windows (32-bit) and Linux (32- and 64-bit) only, and is available only in a specially forked release of MySQL 5.1.
This discussion has been archived.
No new comments can be posted.
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
Full
Abbreviated
Hidden
Loading... please wait.
Please explain (Score:3, Interesting)
Isn't MySQL already open source? If so, how does the Falcon storage engine differ from the "regular" storage engine that comes with MySQL?
Re: (Score:3, Funny)
Re: (Score:3, Insightful)
Here's what gets me about MySQL. They say they have "pluggable" storage engines, but there's no clean abstraction. Each engine supports some things and not others.
Pluggable engines might be useful if the only differences are in the implementation, storage requirements, performance, and other administrative aspects. However, a constraint violation that will cause an error in one storage engine passes right through another storage engine. So, it's not like you can just swap one
Re: (Score:3, Funny)
Re:Please explain (Score:5, Informative)
MySQL has no "native" way to store or obtain data - everything goes through plugins, some of which ship with MySQL some don't.
MyISAM - the most common and fastest. But no transactions, no ACID, etc. Good for many read-only or non critical tables.
InnoDB - licensed from InnoSoft (now oracle). GPL for non commercial, extra dollars for commercial. Transactions, ACID, but a bit slow.
Parent
Re: (Score:3, Interesting)
Well you can use GPL version for commercial projects and The Other license for totally uncomercial projects.
Re:Please explain (Score:4, Insightful)
This is easy and one of the tenants of so-called dual licensing setups...
Basically, if you don't want to pay to use the software, you are bound to the terms of the GPL. If you don't want to be bound to the terms of the GPL, you gotta pay.
Parent
Re: (Score:3, Informative)
Re:Please explain (Score:5, Informative)
Falcon has been specially developed for systems that are able to support larger memory architectures and multi-threaded or multi-core CPU environments. Most 64-bit architectures are ideal platforms for the Falcon engine, where there is a larger available memory space and 2-, 4- or 8-core CPUs available. It can also be deployed within a standard 32-bit environment.
The Falcon storage engine is designed to work within high-traffic transactional applications. It supports a number of key features that make this possible:
* True Multi Version Concurrency Control (MVCC) enables records and tables to be updated without the overhead associated with row-level locking mechanisms. The MVCC implementation virtually eliminates the need to lock tables or rows during the update process.
* Flexible locking, including flexible locking levels and smart deadlock detection keep data protected and transactions and operations flowing at full speed.
* Optimized for modern CPUs and environments to support multiple threads allowing multiple transactions and fast transaction handling.
* Transaction-safe (fully ACID-compliant) and able to handle multiple concurrent transactions.
* Serial Log provides high performance and recovery capabilities without sacrificing performance.
* Advanced B-Tree indexes.
* Data compression stores the information on disk in a compressed format, compressing and decompressing data on the fly. The result is in smaller and more efficient physical data sizes.
* Intelligent disk management automatically manages disk file size, extensions and space reclamation.
* Data and index caching provides quick access to data without the requirement to load index data from disk.
* Implicit savepoints ensure data integrity during transactions.
Parent
VACUUM? (Score:3, Interesting)
So if Falcon uses MVCC, does it require something like PostgreSQL's VACUUM? Or does it have some other way to detect and remove dead tuples?
Also, has anyone looked at making PostgreSQL a storage plugin for MySQL? :-)
Re:VACUUM? (Score:4, Funny)
Parent
Re: (Score:2)
Re: (Score:2)
Re: (Score:3, Insightful)
Re: (Score:3, Informative)
The Falcon engine is from a renowned database developer, and as such has all sorts of neat features [mysql.org].
Re: (Score:3, Insightful)
Re:Please explain (Score:5, Interesting)
Parent
Re: (Score:3, Interesting)
Re: (Score:2)
Re: (Score:2)
Re: (Score:3, Informative)
Re: (Score:3, Insightful)
MySQL's counter-attack to Oracle's advances (Score:5, Informative)
InnoDB (Score:2)
Re: (Score:3, Insightful)
Get developed by a company that doesn't hate MySQL, for starters.
Re: (Score:3, Insightful)
MVCC: Multi-Version Concurrency Control (Score:5, Informative)
Here is a good explanation of PostgreSQL's MVCC [onlamp.com].
Parent
Re: (Score:3, Insightful)
InnoDB is licensed under the GPL. InnoDB is no more dependent on Oracle then Linux is on Red Hat. MySQL could if need be develop InnoDB themselves.
Except that a large chunk of MySQL AB's revenue would disappear as they'd no longer be able to sell a commercial version of their database with InnoDB support. And believe me, few people are going to buy the commercial version if the preferred storage engine is MyISAM. MySQL pay InnoSoft for the right to distribute the InnoDB engine as part of their commercia
Sweet... but one of those long-term things :-( (Score:2)
Why not PostgreSQL? (Score:2, Insightful)
Re:Why not PostgreSQL? (Score:5, Interesting)
1) Get better performance on multiprocessor systems
2) Get a decent storage engine that is not controlled by MySQL competitors
As far as I can tell there is nothing in it that you can't get in Postgresql.
Postgresql already performs better than the standard MySQL on multiprocessor systems.
It remains to be seen if Falcon will be better than Postgresql once its production
ready. Well, there is one thing, Falcon compresses data while Postgresql doesn't. can't help wondering what this will do to performance.
On the other hand there seam to be a lot missing from Falcon that you find in Postgresql.
If you read the Falcon limits page on the mysql site you find that it lacks e.g:
- SELECT FOR UPDATE
- No online backup
- No foreign keys
All in all, I would say Postgresql would be a better choice, if your web hosting company allows you to use it.
MySQL have a tendency to slow down on many concurrent or complex queries. Postgresql is far better at handling triggers and can be programmed in many different programming languages. Support for domains and much more. MySQL also lacks EXCEPT, this makes some types of queries (relational division) much more complex than they have to be.
Still for people that aren't free to choose their database, it is nice to see that MySQL
makes some progress. Besides a little competition never hurts.
Parent
It might have potential... (Score:2)
for smaller databases, but limiting the tablespace to a single file per database/schema doesn't sound very flexible, and won't allow DBAs to maximize their disk throughput.
I am guessing that this is more of a MyISAM replacement than an InnoDB replacement, so it's not really a shot across Oracle's bow (as some comments make it sound like).
Re: (Score:3, Informative)
That isn't necessarily the case. Recently it has become popular to aggregate spindles into single stripe/mirror volumes with large stripe widths. This spreads I/O operations uniformly across disks. All disks contribute their IOPS capacity to all operations. Large stripe widths attempt to leverage high sequential IO bandwidth.
Oracle ca
Oh, please (Score:4, Informative)
Come on. Give me PostgreSQL [postgresql.org] any day. After fiddling with MySQL at work for a few custom developments (both versions 4 and 5.0.2) i'm ready for anything else. Gave PSGSQL a shot at home and it runs very nice, with lots of advanced features. Anyone with real-world deploying feedback to share?
Re: (Score:3, Informative)
Don't underestimate this one (Score:5, Informative)
We're talking in-memory MVCC here. This means you can add 1000 records, do a rollback, and the harddisk hasn't been accessed. Even if you commit, performance will eventually be magnificent compared with on-disk MVCC systems. You can run larger systems on one server with this, than you would be able to run on a cluster with other database systems.
This system has been designed to provide very good performance improvements for those who do know how to create SQL statements, but probably even better performance improvements for those who don't. And we don't have a tradeoff between performance and transactions any more - transactions and better performance are both included.
Also, please note that this technology will make MySQL a trustworthy data storage for many commercial applications out there, giving added value to their apps and their businesses. It will also enable small but very skilled development teams able to use MySQL as a trustworthy database for specialized applications - previously only Firebird and Postgresql were able to provide this for free, and even though Firebird has a very high deployment in USA's top 500 companies, postgresql seems to be very much *nix only in deployment statistics.
I have been programming database applications for more than 20 years, and have been programming Oracle, MSSQL, MySQL, postgresql, Firebird, dBase, Paradox, Access and other databases. I see Jim's contributions to MySQL as extremely important for the database market. Instead of having "just" a transaction layer on top of a storage layer, MySQL now provides mechanisms that give this design an advantage over those database systems where the transactions are stored on disk (like Firebird, Postgresql).
And - by the way - this has NOTHING to do with "optimizing for web applications". Web applications are just as diverse as GUI applications and other systems, and GUI applications will benefit from this as much as web applications.
Re: (Score:2, Informative)
Gotcha. (Score:2)
Re:Eh. MySQL user, actually. (Score:5, Interesting)
It's just a function of how easy MySQL is to set up. It's trivial to set up, but a lot of the default decisions are generally bad for an SQL database, and the documentation -- while good -- never encourages you to go beyond the defaults.
It's like hearing someone say they can design websites, and then finding out they mean with FrontPage.
Drupal is one example of something that works great with MySQL. I can argue about MySQL's faults over and over, but at the end of the day it's easy to use and it's good enough for most people. CMS systems and forums are where MySQL really shines.
Parent
Noobs and Access, oh my! (Score:3, Funny)
These guys who've set up MySQL are gurus by all comparisons
Re: (Score:3, Insightful)
Re:Please explain part 2 (Score:5, Funny)
Mysql maximizes rich channels and empowers cross-platform convergence letting you drive mission-critical niches whereas Falcon utilizes scalable initiatives by scaling end-to-end networks for reintermediate granular platforms (win32 and linux 32/64).
Parent
Re:MySQL versus PostgreSQL (Score:5, Insightful)
But in something as mission-critical as a database, of all things, reliability trumps everything. I don't think they could have developed PostgreSQL any other way and still supported its primary goal of safety.
What gave you the (wrong) impression that PostgreSQL folks have been sitting around twiddling their thumbs? Version 8.2 just came out within the month and includes several performance boosts that make it fly on our production systems.
Parent
Re: (Score:3, Informative)
That depends on the database.
Re:New Microsoft Sql Server (Score:5, Insightful)
This is a nice step forward for MySQL, but for the most part it is just a means for catching up to the other commercial DBs and PostgreSQL. ACID compliance, granular locking, MVCC, and multithreading are *not* differentiating features in the database world.
Parent
Re: (Score:3, Informative)
Re: (Score:3, Insightful)
Re: (Score:3, Funny)
Re:It's only a matter of time (Score:4, Informative)
MySQL *have* done so with Falcon. MySQL *do* own Falcon.
[Jim has worked on other RDBMS in the past] != [MySQL do not own the one he's currently working on]
Parent
Re: (Score:3, Informative)
SELECT * FROM foo LEFT OUTER JOIN y ON foo.bar = y.x AND y.z > 4 WHERE y.x IS NULL
But I'm sure there are nasty cases where you can't substitute joins readably or maybe at all.