CNet on WinFS 466
Weston writes "CNet has posted an article about WinFS, or more specifically, what Bob Muglia (a VP at Microsoft) said about it in a recent interview. According to Muglia, the new filesystem will not replace NTFS, but will incorporate feratures of NTFS, SQL, and XML all into a filesystem which, accoring to Microsoft, will open up a whole new world of information availability. He goes on to describe such a filesystem as the 'holy grail' that is sought by developers. WinFS is slated for release in 2005/06 as part of the Longhorn OS."
Re:What about those of us (Score:1, Informative)
See, C:\Foo\Bar1.txt and C:\Foo\Bar2.txt right now do not have to live next to each other on the hard disk. In order to display the contents of C:\Foo\ the system has to search the entire File Allocation Table to look for files that reside in C:\Foo\. This is why a directory that has 1000 entries can sometimes take nearly forever to display in Windows, and even longer if a networking situation is involved.
The basic idea here to replace the simple FAT, or even NTFS, with a database that has a lot of indexes, so that any time you make a request of the file system, the answer is either already ready or very close to being so. So yes, even you should see a speed improvement.
Re:What about those of us (Score:3, Informative)
In order to display the contents of C:\Foo\ the system has to search the entire File Allocation Table to look for files that reside in C:\Foo\.
This is wrong. To look up the files in C:\FOO is first looks up the FOO entry on C: off the root drive, and then it loads the FOO directory information directly. No searching the file allocation table, just searching the root directory entry and then the FOO directory entry. With 1000s of files under FAT, then some 1000s of FAT entries do need to be traversed, but definately not the entire table.
NTFS works far more similarly to the way traditional UNIX file systems look up files. It starts with a root node, looks for the directory list of that node, finds the entry it's looking for in that list, and heads off to that node to pick out the list of files in the given directory.
Granted, this is simplifying the matter a bit, but the fact remains that the entire FAT is not searched to look up a single file or even a directory.
The real reason that directories under Windows Explorer take a long time to display is that after Windows Explorer loads the directory it then loads the metadata about the file and in some cases scans the file to check which icon it should display. It's this action that causes the system to go slowly and causes network shares to slow down, not the file system. If you go to the CMD prompt and run a DIR on a given directory, you should notice that it goes by quite speedily with little disk action. Browse to it in Windows Explorer and it could take a while longer depending on the type of files stored in the directory.
Re:What about those of us (Score:3, Informative)
Nope. You'll have to search the FAT for the clusters in which the directory information is stored. This is similar to walking a single-linked list, if you get cluster numbers that don't belong to the directory (list), your filesystem is corrupt.
The basic idea here to replace the simple FAT, or even NTFS, with a database that has a lot of indexes, so that any time you make a request of the file system, the answer is either already ready or very close to being so. So yes, even you should see a speed improvement.
What is the difference between a directory as-stored-on-disk and a file index? So what happens if you build big indexes that enumerate all files on a disk for quick searching? You have created what comes down to one big directory, containing all files. If you create multiple indexes, that comes down to having files in different directories at the same time; keeping multiple indexes consistent (when files are deleted) will cost overhead, and only speeds up access times if you know in what index to search, if you don't know it will be slower as searching a hierachical file system (because there will be more duplicates, meaning bigger indexes to search).
Re:Been saying it for years (Score:3, Informative)
One of the first things IBM tried after inventing the relational database was to replace the file system with it. You can tell how far that went.
What do you mean? To this day IBM uses a database AS the filesystem in the AS/400. Ever heard of DB/2?
Re:Been saying it for years (Score:3, Informative)
However, even on the 486-33mhz 16mb Systems that NT 3.1 and 3.51 were released on showed that NTFS's performance was at the same level and sometimes much faster than less featured FS technologies like FAT, etc.
Ironically, Journaling is still seen as a major overhead on OSes like OSX - which baffles me, considering MS was able to offer great journaled performance on a 486 platform, and on modern systems is transparent and offset by the advanced MFT and data structuring in NTFS.
and I don't need journaling
You don't? Do you even know what it is? Journaling allows the OS FS to track and protect from read/write file corruption. Especially in cases of power loss or hardware failure.
This is why CHKDSK screens on NT running NTFS are very rare - the journaled NTFS takes care of these problems without letting the FS or the information on the FS get corrupted by an interrupted write for example.
File Systems are a database variant, whether people like it or not. Why not add the advanced indexing and heuristics of a database technology onto the basic NTFS structure (especially since NTFS was designed to be extended in this direction).
And no, a simple indexing server is not going to cut it, there are simple indexing systems in Win2k and XP already, they prove inefficient and do not expose non-FS storage data mechanisms to applications.
Also your Metadata example of the MP3s is already a moot point. NTFS using Metadata ALREADY, just because it is not that obvious to users, does not mean that NTFS is not ALREADY doing this. This is part of the design of NTFS, the ability to store more than name, date, and location of file structures. It can store virtually any amount of additional data about files on the volume, and already does with many files types. People just don't realize that this is already happening, and because NTFS was designed to do this, there is already NO performance degradation.
Put fast information retrieval technology that exists in Database servers behind this, and not only will you add functionality for users, but it could in theory also increase the file access performance of the NTFS file system itself.
Database server algorithms are very fast and efficient and offer relational constructs, something a modern OS just might be able to use - especially when you stop seeing files and documents on a volume as closed entities and instead realize that they are data constructs that often relate to other data on the system. (i.e. A very easy basic example would be Mail Folder Files)