FLAC Joins The Xiph Family 319
Ancipital writes "Xiph.org (of Ogg Vorbis fame) have today announced that the FLAC (Free Lossless Audio Codec) project has joined the Xiph rebel alliance. The full story and press release can be found at the Xiph site. (FLAC is nice, because it gives you pristine lossless audio at roughtly 50% size reduction over uncompressed WAVs- you can store them on your hard drive/wherever and then transcode down to a lossy format when you need portability, yum!)"
Missing the Point (Score:5, Insightful)
So the point isn't that FLAC is new... the point is that FLAC is OSS, and has joined forces with an organization backing such efforts. The SHN codec is not OSS.
Re:Missing the Point (Score:2, Informative)
And I'd laugh my guts out if it were.
FLAC isn't new.
the point is (Score:2)
Most people don't use OSS because it is open, they use it because it is better. The problem with FLAC is that shorten is established as the standard and there are shorten encoders/decoders available at no cost. The only way FLAC is going to be able to thrive is if it does something better. I was seriously considering moving my 500 GB SHN collection to FLAC, but after doing a few test encodes, I concluded that it wasn't worth the hassle. If FLAC were to give me 75% compression ratio that would free up 125 GB for me and would definitely be worth the hassle. But as it stands now, I'm not going to spend a few days converting my SHN collection just for the sake of it using an open file format and this is coming from someone who uses OSS for just about everything (except Photoshop and Sound Forge) .
Re:Agreed (Score:2)
Yawn.. (Score:5, Funny)
If you take your LPs out of the cardboard sleeves you easily save over 50% space.
FLAC streaming (Score:5, Informative)
-adnans (*plug*!)
Re:FLAC streaming (Score:4, Insightful)
I'm not sure your ISP hope so, though.
Re:FLAC streaming (Score:2)
If you want to send the music to someone, ftp or whatever.
If you want to stream music radio-style, then compressed is 'good enough' - surely?
Great... but what about 3rd party support? (Score:4, Interesting)
I've been contemplating a digital audio player like the Turtle Beach AudioTron [turtlebeach.com] for awhile now, and while the AT has better support for a variety of formats than most, it's missing both FLAC and OGG (and the developers have stated it's not coming due to lack of CPU power).
I'd love to encode all my CDs onto a central server and have several units around the house playing from that. But I'd rather not rip around 1000 CDs more than once. And it's still not cost effective to just store them as WAVs - using FLAC would double the capacity.
Yeah, I know... Samba can translate files on the fly now, but that requires a good bit of horsepower. The Celeron 300A in the server just isn't going to be capable of transcoding FLAC->anything in real time, much less do it for 2 or 3 streams at once.
I guess the question is, what's holding back consumer electronics companies from implementing OGG and FLAC support? Is it technical, financial, or what? And what can Xiph do to help them in this?
uh... (Score:2)
Re:Great... but what about 3rd party support? (Score:5, Interesting)
Re:Great... but what about 3rd party support? (Score:2)
Re:Great... but what about 3rd party support? (Score:2, Interesting)
There's a list on the sidebar of the FLAC homepage.
I guess the question is, what's holding back consumer electronics companies from implementing OGG and FLAC support?
In the case of the AudioTron, they are getting the full court press from Microsoft to do WMA lossless, and knowing Microsoft this will be to the exclusion of all others. I made a good case on their mailing list that with moderate work (and I was willing to help), the AT would be able to decode FLAC natively. Their response has always been "we tried it and it's not fast enough", despite the fact that identical hardware in other devices (Rio Receiver, PhatBox) can decode FLAC fine.
But no matter, other manufacturers are providing a choice, and the list is growing. Recently the ReQuest guys have added FLAC support to their ARQ boxes. So time will tell what consumers really want.
Re:Great... but what about 3rd party support? (Score:2)
Not much. Xiph offers ported and optimized fixed-point code for Ogg Vorbis to anyone who wants it on royalty-free terms (BSD or similar license IIRC). Moreover Xiph has been actively promoting its standards and codebases directly to the involved businesses.
Nevertheless, the trick is actually getting the end-product companies to make Vorbis/FLAC support a requirement for their next product/revision. I suspect it'll be Apple or a similar market leader that forces the hand of the competitors, since it seems that most marketing departments can't see further past their noses than their competitor's feature lists.
Re:Great... but what about 3rd party support? (Score:2)
Very very cool (Score:5, Informative)
In the end, I settled on FLAC for four reasons:
* It's completely lossless.
* Gapless playback
* If you save the TOC from the source CD, you can burn an exact copy, pregaps and all, from your FLACs.
* I can reencode to Ogg, MP3 or whatever lossy format I want at any time. Nice for when I want to make a MP3 disc to play on my MP3 walkman, and I don't lose quality like I would if my source material was in Ogg.
Hopefully, we'll see wider support for FLAC come from this partnership. Not too many players support FLAC, though the FLAC developers have made plugins for XMMS and WinAmp.
Oh, and some people have been tossing the '50% compression' thing around already. It really depends on the music. I have managed up to 70% compression on some sparse music, (mainly ambient and classical) while my death metal and noise encoded around 30%. It seems that the more dense the source is, the less it compresses.
Re:Very very cool (Score:2)
I wonder if FLAC could be adapted do this too, so you could 'head --bytes 1000000' to get a lossy version. Okay, maybe not quite such good quality as an Ogg Vorbis file of the same size, but it might be good enough.
Re:Very very cool (Score:2, Informative)
Progressive jpegs.
Re:Very very cool (Score:2)
flac -c -d flac_file.flac | lame - >mp3_file.mp3
It takes under a minute to reencode a typical 4 or 5 minute track this way on my 1.7ghz P4. Just decoding the track takes around 7 seconds, and reencoding it with lame takes 50 seconds.
I suppose it could be faster, but it works for me.
Re:Very very cool (Score:2)
HOWEVER, it's possible that Ogg could be adapted the other way, adding another layer to it to make it lossless (probably by computing the difference between the lossy result and the actual source, and compressing the resulting stream). The result would be larger and MUCH more computation-demanding to play than FLAC, but it would be lossless and peelable.
I don't have any idea how much larger or more computation-dependant it would be.
-Billy
Algorithms? (Score:4, Interesting)
One idea that would be really cool is if they could get acheive lossless compression by noting the differences between the original and the
Re:Algorithms? (Score:5, Informative)
For the most part, linear prediction. This uses a linear combination of past sample values to predict the next sample value. The difference between the prediction and the actual is Golomb-Rice encoded. Golomb-Rice codes are used when the probability of an integer occurring is geometric (i.e., the value N+1 is 1/R times as likely as the value N, for some R > 1). This is a pretty good assumption for audio, since the predicted values tend to be quite close to the real ones. Some other lossy compression algorithms also use linear prediction, but they quantize the predicted values to reduce the bitrate even further. The quantization is the lossy step.
MP3 and OGG, on the other hand, work differently. They first transform a block of audio using the MDCT, and apply a psychoacoustic model to the resulting spectral envelope. This eliminates a lot of subbands that are "inaudible." At that point the remaining subband energies are quantized and entropy-coded. To decode, the encoded energies are decoded and the spectral envelope is reconstructed, then transformed back into the time domain to become "audio" again.
It would be a serious feat to integrate FLAC and OGG. They are totally different systems.
Re:Algorithms? (Score:2, Informative)
Not so, they are already integrated, i.e. you can already encode to raw FLAC or Ogg FLAC with the command-line flac encoder. FLAC packets are embeddable in an Ogg container just as easily as Vorbis ones.
Yeah (Score:2)
What the original poster was suggesting would be to encode the file losslessly, and then FLAC encode the residual produced by subtracting the encoded waveform from the unencoded one. This is a very cool idea, but it won't work. The residual signal is going to be very noise-like, so it would be resistant to FLAC compression (FLAC uses a "verbatim" mode when it sees noise -- the verbatim mode does no compression at all). It isn't that you couldn't do it, but I very highly doubt you'd gain anything by it.
Re:Algorithms? (Score:3, Informative)
FLAC already has a very good wrapper. OGG is very small, and adds as little as possible to the size of the raw data making up the media stream, but some decisions were made that make OGG useless to me as a wrapper.
As an example, how do you seek in a file?
In FLAC's native format, you read the Metadata Block Seektable [sourceforge.net] which gives you a mapping between points in time, and points in the file.
In QuickTime, you read the Sample Table Atom [apple.com] which does basically the same thing.
In OGG [xiph.org]? It appears ( from vorbisfile.c [xiph.org]) that you have to seek through the whole stream, reading the headers of every page to find the locations of all of the absolute granule position markers and regenerate the same information that other formats spend a few hundred bytes to store in a table.
Needing to read the whole file before being able to seek might not seem like much, but when you are dealing with files of moderate size (6 hours or so) stored on a media where the transfer rate between the file and the player is close to the bitrate of the audio, it becomes extremely annoying.
Re:Algorithms? (Score:2)
FLAC and Ogg are already integrated. Ogg is simply a container file format, it has nothing to do with audio compression. You are thinking of Vorbis.
See the FAQ [vorbis.com].
Re:Algorithms? (Score:2)
The predicted value isn't what gets sent. They send the difference between the prediction and the actual. On the decoding side, they make the same prediction, then add the "delta" back in, to recover the original sample. The procedure is entirely lossless.
The high level of compression comes from the fact that the deltas are distributed geometrically (because audio is highly correlated in time). Golomb-Rice codes are the optimal way to encode geometric distributions, so that's what they chose.
Yeah, there are plenty of pedants on /. just waiting to pounce on your slightest mis-wording or mistake...
Re:Algorithms? (Score:2)
Being lossless FLAC is going to care about a bunch of little details that OGG won't.
Actually I'm going to bet that the Vorbis noise floor management may hurt when FLAC has to deal with it. But FLAC is designed to expect alot of crap to sneak by its main predictors, and into the rice (coding).
So it beats me. I'd like to see the difference with various instruments.
Re:Algorithms? (Score:5, Interesting)
For more info see here [sourceforge.net]
One idea that would be really cool is if they could get acheive lossless compression by noting the differences between the original and the .OGG, and appending that to the .OGG. Then if you can just strip off the added info when you make copies to restricted-space devices. The only question is whether this can be done with a competitive compression ratio.
This has been suggested before, but would require all Vorbis decoders to decode to the exact same result, which is not practical (Vorbis decodes to float samples).
Re:Algorithms? (Score:2)
2) Isn't there some integer decoder everyone was talking about a while back? I think they could use that to deal with this problem.
3) I think there are other serious barriers of entry. That is, the discrepancies between a vorbis file's output and the original might be more difficult to encode than the audio in the first place, and the resulting filesizes might be bigger than FLAC (Or hell, have no advantage over uncompressed audio in the first place.)
4) I don't know anything about Ogg Vorbis.
Re:Algorithms? (Score:2)
Not true.
I don't know if this has been tried before, but the limitation you propose does not exist. This would not require all OGG decoders to produce the same exact result. It would require just all FLAC decoders to produce the same ogg output for the ogg data in the flac file, which is possible.
You'r
I believe the problem that is actual has nothing to do with practicality, but that the residual wave form that is produced via the diff between the
If someone found the "trick" to compress this noise (because this noise will definately have a pattern to it, depending on which codec you used to encode the origional
Is this REALLY a solution? (Score:2, Interesting)
Why can't we develop a codec which is "almost lossless" and works well at higher bitrates? Ogg and MP3 do okay at 320kbps, but the quality increase isn't 3 times a 128kbps mp3.
A good test for encoding quality is to encode new age (enya, enigma) or classical music as they tend to have many subtle, yet distinct instrumental sounds (bells, small symbols, synthesized effects) in the background. Listen to them using a pair of good quality headphones (seinheisser or bose) - you're not listening for artifacts (at high bitrates, you should't find any) - instead listen for the subtle background sounds. THEN, make the decision if lossless really is better. Personally, I prefer 192kbps OGG for my encoding, as it provides reasonably good quality without sucking up my entire drive.
Re:Is this REALLY a solution? (Score:2, Insightful)
Why can't we develop a codec which is "almost lossless" and works well at higher bitrates? Ogg and MP3 do okay at 320kbps, but the quality increase isn't 3 times a 128kbps mp3.
---
That's fine. But lossless compression is important for people that trade and distribute music. Having an *exact* copy is what we want and lossless compression schemes do that for us. In the trading circles that I'm a part of, mp3 and ogg are the product of the devil.
Re:Is this REALLY a solution? (Score:2)
That's good, since it should only be 2.5 times better.
Re:Is this REALLY a solution? (Score:4, Insightful)
Yes, it is.
There are many musicians who want portability. Try encoding some wav to mp3/ogg at home, decoding it in the studio, mix it, encode it again to mp3/ogg and go home to your homestudio.
Then try that 20 times, and see what remains of the soundquality.
Then sure, you can also carry wavfiles if it matters that much to you, but 50% savings can be a lot.
Re:Is this REALLY a solution? (Score:2)
with successive reencoding. In theory, mp3
has psychoacoustic model to eliminate frequencies
you can't hear. So once that is done on the first
pass, the successive passes should find nothing
more to eliminate. Where am I wrong?
Re:Is this REALLY a solution? (Score:2)
1) your suggestion might be technically possible, but impractical. For instance, there is no practical reason to build a codec that has this capability if it will cost more to build than to just use lossless codecs.
2) You still don't have a way to archive the ORIGIONAL SOUND. Would you store your master copy on a record? surely not. The same should be true for lossless v lossy compression/storage.
3) After you decode an mp3, and edit it, then re-encode it, you are no longer re-encoding the origional mp3's output, so you cannot predict its re-encodeability, EVEN IF you did build a codec that could do what you propose.
An example of #3:
have you ever tried to run a
Re:Is this REALLY a solution? (Score:5, Informative)
Most poeple, experts included, cannot tell the difference between a 320kbps mp3 and an original 44.1khz pcm sample. I mean, the vast, huge majority of experts simply cannot tell the difference.
But there IS a difference. We know there is a difference because it's lossy compression. We know that when we take an original CD and use flac on it, we end up with an exact copy of the original. That's why lossy compression exists.
If you are simply listening to something on the headphone jack of your computer with medium or low quality headphone (like Bose or most of the Sennheiser line (medium) or the normal crap you buy in any store (low)), you don't have a chance of hearing the difference between a high bitrate mp3 and the original.. there is too much noise from the computer, and not enough power from the headphone jack.
On the other hand, if you are using a clock stabilized external output from a good external soundcard with a proper mixer, running through a good class-A headphone amp and into a good pair of headphones (Sennheiser HD580, HD600, Grado RS-1, RS-2, SR325), in a quiet room built for listening, and if you have good ears, and are used to listening for detail, you may hear a difference.
Re:Is this REALLY a solution? (Score:4, Interesting)
Re:Is this REALLY a solution? (Score:5, Informative)
Say you've got your collection of CDs at home, and you're just about to encode them all for your iPod. "Okay", you figure, "I'm going to pick.. umm.. 192kbps MP3s, since that's pretty good and I'm going to be listening to them over cheap headphones on the train on the way to work."
So you go ahead and encode your entire 600-album collection to 192 kbps MP3s. And you put them on your iPod, and everything's fine... until you decide you want to listen to them at work as well, and 192kpbs just isn't good enough for listening in the quieter environment in your cube.
Now you've gotta take your 600 CDs and re-encode them at 320 kbps, because if you were to do something silly like extract your 192 kbps MP3s to wave files and re-encode to 320 kbps, you'd just end up with inflated 192 kbps MP3s.
Better yet, say you want (vbr) ogg files at work; or Apple (heaven forfend) finally comes out with a portable player with ogg support. You still need to go back to your original CDs (are they scratched yet? Did you lend 'em to your friend and forget he had it before he left for Maryland? Did your wife take your favorite disc to work with her, where one of her students used it for an art project?) and re-encode everything.
Now, say instead you use FLAC (or SHN, or even APE which I've never personally used).
You take your collection to work; turns out your servers are slightly too small for the FLAC files, so you expand to wave and encode to 320 kpbs MP3s using a simple shell script for the entire collection.
You want ogg files for your new next-generation iPod; great, just run a slightly different shell script to expand to wave and encode to ogg.
Your apartment is broken into and your entire 600 CD collection is stolen, including that ultra-rare CD you got from that band that was once part of that other band but split off when the original drummer OD'd, but they only burned 300 copies of their indie CD and besides they haven't been together since '94. No problem, you've still got the FLAC files and can at least burn yourself a virgin, bit-for-bit exact copy (depending on how carefully you originally extracted it, of course) of the audio -- your artwork and individually-numbered disc are still gone, sorry.
And that's not to mention new compression algorithms, media formats, etc. MP3 and any other lossy compression algorithm doesn't handle future-readiness very well.
Re:Is this REALLY a solution? (Score:5, Informative)
Visit etree.org [etree.org]. The big benefit of lossless compression is it makes for better distribution of live recordings. The short of it is that demanding recordings in a losslessly compressed audio format, along with verification using checksum files, guarantees no loss in fidelity.
There are many alternate live-music trading scenarios which cause a loss in fidelity. Two of the most common: 1) CD Audio->CD Audio copies are not perfect (unless you use a specialized tool like EAC - Exact Audio Copy); 2) trading lossily-compressed audio tends to lead to loss of fidelity through inevitable decompression, writing to CD, reripping, and reencoding.
Re:Is this REALLY a solution? (Score:2, Insightful)
It's a go! (Score:2, Interesting)
With all the news on Microsoft's "new" TabletPC (old idea), I am quite intrigued that Microsoft doesn't have any innovative technology to bundle with their TabletPC; Xiph.org has it! The Opensource "revolution" is crumbling many barriers, including the proprietary ones put up just as a "distraction" (yes, inter-operability with Microsoft's proprietary software is a distraction from good programmers to design and implement better software and standards).
Come to think of it, Microsoft has nothing innovative in the audio and video world. Their AVI format, its many subspecies (wsf, wmf, wma, etc), and the general proliferation thereof are a justified (and quite notable) example of how media standards is not as crucial element in a company's survival. Bill Gates (yes his statment still stands as being verry impressive and of his accurate observation) generally stated that Microsoft's goal is to extend itself to its competitors by ussurping them to use Microsoft software. I just saw a black cat, the same one, walk by twice. XIPH has technology that Microsoft wants; loss-less audio. We know S3's S3TC is a loss-less standard of computer graphics and it is the only standing technology that is keep the DRI project [sourceforge.net] from being able to objectionably compete as an opensource platform. So now, where does Microsoft think its going today? Microsoft has no software forcing anyone to use it now; the better of the software is opensourced and freely available.
In the immortal words of Nelson... "Hah ha!"
Re:It's a go! (Score:3, Informative)
Ya know, up to that point, you had a good Anything But Microsoft rant going there. But you missed something. Microsoft Windows Media 9 already has a lossless codec. [microsoft.com]
Lossless compression is a joke (Score:3, Interesting)
On top of this, you are still limited by the response of the equipment you are playing it on. Maybe this would help a little if you had an optical connection to a good amp, but computer speakers will provide more interference than compression any ol day.
Re:Lossless compression is a joke (Score:5, Insightful)
If the CD is lost or destroyed by scratches (many of mine are allready), you still have the original recording that you can compress with lossy compression of the day for your daily use. Conversion between lossy codecs is meaningless, but compressing from a lossless format to a lossy format is OK.
So, if Ogg Vorbis 2.0 is better than 1.0, you can make 2.0 files from your lossless compressed files.
Re:Lossless compression is a joke (Score:3, Informative)
Not true! While this is true for the Xing encoder which a lot of bad CD-ripper applications use, LAME or the Fraunhofer mp3 encoders do not mangle the sound as much. Try and get a good compile of LAME from HA [hydrogenaudio.org] and do a "lame --alt-preset standard file.wav" and open the resulting wave file in Cool Edit and have it show you the frequency stats of the created mp3. You shouldn't use this for a comparison of sound quality though - your ears are still the only useful tool for that, but what it will show you that you are wrong.
You might have heard of the Nyquist theorem before, which states that when storing audio data using PCM (Pulse Code Modulation) the highest frequency you can have it half the samplerate. Thus there are no frequencies higher than 22.05kHz on a CD (because their samplerate is 44.1kHz).
How does it work? (Score:2)
I pretty much know how lossy compressions work and how gzip/zip/etc work, but what does this do (that must be specific to audio) that zip doesn't?
Fast commandline editting? (Score:2)
Editting 30-minute audio files on Linux is quite slow going using the GUI programs. The best I've yet found is GLAME, which at least lets me select a region then resize the region to get things right, but it takes an age. It would be much quicker to pinpoint the start and end points by listening in xmms, then use a commandline.
Re:Fast commandline editting? (Score:2)
Here is a FLAC to MP3 script (Score:2, Informative)
#
# Converts FLAC to MP3 preserving tags
# License: GPLv2
# Home: http://www.GuruLabs.com/downloads.html
#
use MP3::Info;
foreach $file (@ARGV) {
if (!($file =~
print "Skipping $file\n";
next;
}
undef $year; undef $artist; undef $comment; undef $album; undef $title; undef $genre; undef $tracknum;
if ($tag = get_mp3tag($file)) {
$year = $tag->{YEAR};
$artist = $tag->{ARTIST};
$comment = $tag->{COMMENT};
$album = $tag->{ALBUM};
$title = $tag->{TITLE};
$genre = $tag->{GENRE};
$tracknum = $tag->{TRACKNUM};
chomp($year, $artist, $comment, $album, $title, $genre, $tracknum);
$tracknum = sprintf("%2.2d", $tracknum);
} else {
print "Couldn't get id3v1 tag for $file.\n";
}
if (($artist) && ($title) && ($tracknum)) {
$outfile = "$tracknum" . "_-_" . "$title.mp3";
`flac -c -d "$file" | lame --alt-preset standard --ty $year --ta "$artist" --tc "$comment" --tl "$album" --tt "$title" --tg "$genre" --tn $tracknum - "$outfile"`;
} else {
$outfile = $file;
$outfile =~ s/\.flac$/.mp3/;
`flac -c -d "$file" | lame --alt-preset standard - "$outfile"`;
}
}
Re:Here is a FLAC to MP3 script (Score:3, Insightful)
There's a security/correctness problem with that script:
`flac -c -d "$file" | lame --alt-preset standard --ty $year --ta "$artist" --tc "$comment" --tl "$album" --tt "$title" --tg "$genre" --tn $tracknum - "$outfile"`;
Don't use this script unless you trust the creator of the .flac file. In other words, definitely not on anything you downloaded off gnutella. You can execute arbitrary commands through carefully crafted ID3 tags. (Not even that carefully; an artist of '"; rm -rf /; echo "' minus the single quotes would do it.)
Normally I'd say to use the system('program', 'arg1', 'arg2', ...) form instead, since it doesn't go through the shell. In this case, that's impossible because there's a pipe. You could accomplish the same thing with fork()/exec() and some manual pipe manipulation. It's a pain, so maybe someone's written a CPAN module to do it for you.
Or, you could set the ID3 tags of the .mp3 afterward with the same MP3::Info module that you used to read the ID3 tags from the .flac. That way might even be less code. But you still need to make sure $outfile is safe.
If all else fails, you could use a regexp to sanify those variables. But I don't like that approach for a couple reasons:
etree (Score:2, Informative)
I personally prefer lzip! (Score:5, Funny)
Some impressive stuff from the FAQ that made me leave that Monkey-compression-thingy once and for all:
"We're talking about a constant-time algorithm that can reduce a file down to 0% of its original size. What's not to like?"
---
"You will most likely experience a feeling of euphoria or lightheadedness as you watch your free disk space cascade upwards to 100%."
---
Are there any drawbacks?
"Not that we know of. Occasionally, in the pre-1.0 days, someone would compress a file down to 0K and it would be lost for good. But that has been happening less and less frequently, and these days it has been a long time since we received any complaints from the people who reported this originally."
---
I'm especially impressed by their complex PLACeBO and Lessiss-Moore algorithms.
And don't forget to read their Free-Object Oriented License [sourceforge.net] (or simply "FOO"):
Only 50% compression? Try harder. (Score:2)
Start by converting stereo to A+B and A-B form. This is lossless, but the A-B track often has less variation than the A+B track; anything that's on both channels doesn't affect the A-B track much.
Then convert those two tracks to deltas from the previous sample. This reduces small changes to small numbers, which compress well in later steps. When the source material doesn't have high frequencies, you'll have runs of similar numbers.
Then reorder the bytes so that samples are sequential, not interleaved. That way, runs of similar deltas are sequential.
Then run gzip, which is very good at compressing runs of similar bit sequences.
This is completely reversible. Try it and see how well it compresses. It should do especially well on instrumental classical music.
Re:Only 50% compression? Try harder. (Score:5, Insightful)
Except that they go even further than your naive scheme, and use a predictor to get even smaller deltas than your scheme (e.g., assume waveform is locally quadratic/cubic/quartic then extrapolate the next sample). A signal can be varying rapidly and yet still be highly predictable. Your simplistic scheme wouldn't handle it.
Then they use Rice-Golomb coding to encode the deltas. This does FAR better than gzip ever could, because it is designed SPECIFICALLY to handle the geometric distribution of the deltas, whereas gzip is a generic dictionary algorithm.
I really doubt you've even tried what you are suggesting. You're on the right track, but the FLAC team beat you to the punch. Sorry.
Re:Only 50% compression? Try harder. (Score:3, Insightful)
Umm, you try it. Presumably they've spent a lot of time on designing a format. You should take a little bit of effort before claiming you can do much better, especially since verifying the compression of such a simple format should be easy. Making the claim without even taking that much effort is insulting.
Besides, FLAC has important features [sourceforge.net] your format does not. In particular, FLAC is seekable. As anyone who has tried to quickly extract a single file from a large .tar.gz knows, gzip is not.
Open source lossless auido compression? Been Done! (Score:3, Funny)
Its called gzip. Try it yourself and see the results!
gzip -c9 audiofile.wav > audiofile.gz
Re:huh? (Score:5, Interesting)
I've backed-up about 325 music CDs to CD-R using FLAC. It works as advertised. If you want lossless compression, use FLAC. It even has a XMMS plugin -- I use it all the time.
Sie of FLAC files? (Score:4, Interesting)
(I mean 'gigs' as in Gigabytes, not the head-banging kind
I am actually more than curious, I am very interested to know. I have a 500+ CD collection that I have never ripped. I think it's time to begin backup-ing everything digitally, but I can't decide between mp3 and ogg (and I don't want to rip more than once either). So FLAC looks like the right thing for me. Just wondering about the size of files.
Re:Sie of FLAC files? (Score:4, Informative)
If it takes 650Megs to store 74 minutes of music, at 50%, it should only take 325Megs.
325M x 325CD's = 105625Megs, or 105.625Gigs
This of course is an approximation, since sometimes the compression is better/worse..
Re:Sie of FLAC files? (Score:4, Informative)
Calculated, 650 MB per CD times 165 CDs is about 107 GB.
As soon as I can scrape the cash together, I'm gonna buy a large hard drive and keep them on-line for XMMS. Right now I have them compressed to 256MB MP3s with LAME.
Re:huh? (Score:4, Insightful)
Re:huh? (Score:5, Informative)
In short: id3 (especially id3v2) sucks and should just DIE as soon as possible. Foobar 2000 [hydrogenaudio.org] even goes as far as to completely forgo id3v2 support on ideological reasons. Honestly, I think they are on to something.
Re:huh? (Score:3, Informative)
Minor point, but the tags are not part of the Ogg container. FLAC implements tags the same way as Vorbis does, as one of the initial packets, so they are available in raw FLAC as well as Ogg FLAC.
Re:huh? (Score:5, Informative)
Here's a little meat for you:
2. RESTRICTIONS. Notwithstanding any provisions in this agreement to the contrary, Licensee may not (a) make, use or load into temporary memory any unapproved copies of the Licensed Materials without the appropriate license(s) for use on additional CPUs; (b) distribute the Licensed Materials; (c) modify, transmit, rent, lease or sublicense the Licensed Materials; (d) reverse-engineer, decompile or disassemble the Licensed Materials, except to the extent required to be permitted by applicable law; (e) disclose any source core or performance characteristics of the Licensed Materials to any person or entity; (f) use the Licensed Materials in a service bureau or "application service provider" environment or for the benefit of third parties; or (g) at any time do or permit to be done anything which shall adversely affect SoftSound's right, title or interest in the Licensed Materials. If the Licensed Materials are used within a country of the European Community, nothing in this Agreement shall be construed as restricting any rights available under the EC Council Directive 14 May 1991 on the legal protection of computer programs.
Emmett Plant [mailto]
CEO, Xiph.Org Foundation [xiph.org]
huh? (Score:2, Funny)
Try saying that out loud, see what your co-workers do.
Re:sweet (Score:2)
Re:New? (Score:4, Informative)
Re:New? (Score:5, Informative)
You can think of the relationship between FLAC and MAC similarly to the relationship between Vorbis and MP3. It's a slightly strained analogy, but works to a first approximation.
Re:New? (Score:5, Informative)
Near as I can tell, Monkey's Audio still doesn't work anywhere but on Windows (though support for other platforms is promised). FLAC works on Windows, MacOS9/X, Linux, BSD and even has hardware support. That single platform limitation makes Monkey's Audio difficult to justify for any serious audio preservation.
Microsoft Windows won't be around forever. (Score:2, Insightful)
Windows is likely to disappear from the face of the earth sometime in 2004
MS-DOS lived from 1981 to 2002. It is no longer maintained; instead, a GPL clone [freedos.org] is maintained by the community.
The first good version of Microsoft Windows (Windows 3.x) appeared around 1990. I don't see the product surviving past 2020, let alone the 2080's when the copyrights begin to expire.
Re:Lossless format (Score:2, Insightful)
For example, dictionary-style compression (what
Zip is living proof that lossless compression exists - you COMPRESS your text files, without LOSING any of the data in it when you DECOMPRESS.
That's what lossless compression is.
For more info, look into the entropy [data-compression.com] of data, which helps to determine the lossless compressability.
An example of LOSSY compression is mp3 or JPEG, where you (usually) sacrifice some quality for increased compression. Part of what makes mp3 work so well, is that it throws out the parts of the audio signal that are out of the human range of hearing.
--
Re:Lossless format (Score:2, Redundant)
However, the compression mostly happens for highly idealized data (such as written text); binary files or music/image files where the randomness of successive bits is very high, and thus lossless compression can't happen.
I don't disagree that a lossless compression scheme that gets 50% compression is highly questionable, but it's definitely not out of the realm of possibility.
Shannon Limit (Score:2)
But for any given encoder, there is always at least one source that will result in output at least one bit more than the input.
Any given content will have a Shannon Limit, which is the maximum theoretical compressibility. The more entropy/randomness in the content, the higher the Shannon Limit will be. A completly random series of numbers will have a a Shannon Limit of the size of the file - it's impossible to compress*.
*and yes, it's cheating to know what the random number algorithm is and working back to the seed number. When I say "truly random" it means "truly random."
So what if I enter random bits? (Score:2, Interesting)
PS: This post is a user interface question. I understand the entropy stuff :)
Fractal compression is lossy and patented (Score:2)
One example? You can compress some of the most beautiful images in the world in a few lines of code and some parameters.. It's called a fractal. I don't have to send you a PNG; I give you the parameters, and you generate it, given the 'specialized compression algorithm'..
Generic fractal compression works only with fractals created by hand. It cannot compress a scene captured with a camera.
So Barnsley came up with a special case of fractal compression called the fractal transform [wikipedia.org]. Guess what: It's 1. patented with no available royalty-free license, and 2. not as efficient as JPEG 2000.
Re:Better than ZIP? (Score:3, Informative)
I wish Jeff Gilchrist would add a unix/linux section to the Archive Comparison Test (ACT) page (http://compression.ca). He does have a corpus of sound files, so check it out to see how other compressors are doing (FLAC is not included, though).
FLAC vs. the World (Score:2)
check it out to see how other compressors are doing (FLAC is not included, though).
If you want to see how FLAC performs vs. Shorten, Monkey's Audio Codec, and other popular lossless waveform codecs, read this page [sourceforge.net].
Re:Better than ZIP? (Score:2, Informative)
Re:Better than ZIP? (Score:2, Insightful)
Re:ADPCM? (Score:5, Informative)
Re:ADPCM? (Score:2)
41.184 MB wave source file.
36.780 MB using gzip with -9 (11% compression).
32.937 MB using bzip2 with -9 (20% compression).
25.713 MB using FLAC (38% compression).
5.221 MB using LAME with -V3 (87% compression).
Yeh i know the mp3 is lossy, but this one is essentially indistinguishable by the human ear so I thought I would at least add it to the above list.
Re:ADPCM? (Score:2, Interesting)
'Perceptual' means that the method has some model of human hearing, which means that it can more easily discard data which the human ear can't hear.
'Lossy' means that the encoded data is not an accurate representation of the original.
Generally, non-perceptual lossy audio codecs represent an old generation of technology -- they take up less processing power than perceptual codecs, but cannot compress audio as efficiently as perceptual codecs.
ADPCM is lossy. (Score:2)
ADPCM is lossy, so I'm not sure what you're trying to get at...
I can assure you that FLAC and codecs specifically modelled around sound compression will -- in general -- outdo something general like lz77+huffman.
Re:Oh, boy, yet another codec.... (Score:2, Funny)
They are working on a lossless codec. This means that your sound quality would be exactly the same as the source, whether it be CD, DVD, or something else. MP3, Ogg, and all the other commonly used codecs are lossy, which means that they are of lower quality then the source file.
Whether or not the world needs another lossless codec is another matter entirely, but this project has a different goal then producing yet another MP3 competitor.
(Yes, that may have been a troll, but someone reading this probably managed to get confused in one way or another)
Re:Oh, boy, yet another codec.... (Score:5, Informative)
I'm sure everyone here would welcome any successes you have in researching this.
Storing in one format and then have to convert to another all of the time just not an option. Maybe when memory is a dollar a gigabyte (and I mean RAM!), them this might be a choice - but I am hoping for something better.
Maybe you're missing the point. FLAC is a replacement for WAV. That is, a lossless way to store sound, and still be able to use it, via. direct playability in XMMS and WinAmp.
If you want small, then use mp3 or ogg, which is for small but lossy files. If, after encoding to mp3, you still keep your old WAV files, in order to be able to re-encode into any other lossy format, then FLAC is useful to convert your WAV collection to -- not as a replacement for mp3 or ogg.
Re:Oh, boy, yet another codec.... (Score:2)
I didn't mean any offense. But I perceive your remark as the typical slashdot complain about something, but do nothing. FLAC's lossless compression might be just about as good as it gets.
It is just a 50% decrease over WAV is not enough to trouble switching to FLAC and re-encoded all of my WAV files to FLAC. Or adding to mountain of codecs IN MY HOME. I don't care what codecs you use - knock yourself out. Just don't lay your bum trip on me.
This depends, perhaps on two things: (1) how large your collection is, and (2) how easily you can automate the conversion.
I actually prefer my software (and hardware devices) to have support for as many formats as possible (except WMA), to give me greatest compatibility. (Not having WMA is my choice.) I still might only keep my files encoded in a (very) limited number of formats.
Re:Oh, boy, yet another codec.... (Score:5, Informative)
This is why (or at least a major reason) lossless audio compression is so hard. There just isn't enough repitition at the sample level to produce a dictionary for your traditional compression algorithms (gzip, bzip2 etc)
Now, if music was as repetative as you thought, we'd be able to compress 90% of the music released in the last 5-10 years to about 1kbyte ;)
Re:Oh, boy, yet another codec.... (Score:2)
Re:Oh, boy, yet another codec.... (Score:5, Interesting)
There is a real market for such a codec in the professional audio industry - have you any idea how much space backing up a 48-track studio recording takes, especially now the industry is moving towards 96Khz/24bit recording?
Respected (at least until Apple bought them!) music software giant Emagic will sell you a program called ZAP which make about a 35% space saving and costs about $100, so free software that beats that is definitely good news for some people.
Not a trivial feat... (Score:5, Insightful)
The problem comes with the word "lossless".
Music does indeed have a *lot* of repetition, at a high level. If you look at an audio waveform, you can see very regular-looking patterns in the data, that change every now and then but can go on for thousands of samples with only slight variation. At a low level, however, music has a *huge* amount of noise (not noise as in clicks and artifacts, mind you, noise as in stronly leptokurtic Gaussian deviations from what the waveform "should" look like), and even extremely regular plosives just destroy any sort of adaptive prediction-based encoding. For reference, "huge" means on the order of 5 to 6 bits out of 16 (even local nonlinear methods give a RMS error of at best 40ish, but getting that low means storing a lot of parameters of the prediction model, RBF centers and weights as an example).
If you want and extremely high level of compression that you can *almost* call lossless, use FLAC (or Shorten, or Monkey's, or whatever) *after* running your sound through a trajectory-based nonlinear noise reduction filter. You'll see the compression go from 50% to 25% or better (for reference, "archive quality" VBR OGG only gets down to 20-25%). But, you can't *truly* call that lossless anymore, because even though you might not consider the "noise" as part of the music, people *can* tell the difference and usually prefer the version with noise (and, as I mentioned, such a filter blunts plosives, which *should* stay in the music, so you'd need to detect those and add them back in to avoid a noticeable degredation of quality).
Trust me, lossless audio compression does *not* count as a "toy" problem, nor one that people have already "solved" optimally (for example, just about every well-understood time series prediction/analysis technique out there depends on a property called "stationarity", which music very strongly lacks... You can still use such methods, but they give suboptimal results in the best case, and exhibit serious instability in the worst cases). For another problem, *almost all* research on time series analysis has focused on out-of-series error and stability. This lets you do things like predict stock values and the weather. It doesn't, however, necessarily give the best *in-series* error, which matters in an application like audio compression, since you already know the entire extent of the data you need to predict (postdict?). In AI, this has a close analogy to the idea of "overfitting" a neural net - if you train a neural net too long, it learns too many subtleties of the training data and loses its generalization power. Except, in audio compression, you don't *care* about the generalization power, you care about it learning as much about the training data as possible.
Re:finally (Score:2, Interesting)
300 "audiophiles" disagree with you. [r3mix.net]
I don't see a need for lossless compression. LAME MP3 with the r3mix preset sounds perfect to me, and anyone who thinks that they can tell a difference between that and CD audio is only saying that to impress people.
Please - no r3mix.net links (Score:4, Interesting)
Re:Please - no r3mix.net links (Score:2, Informative)
No proof for this criticism is given (AFAIK -- but I've only read about 4 or 5 threads on it) so it seems to be to be more of a forum vs. forum flamewar. Sort of like K5 is better than Slashdot and vice versa. Maybe what we all need is for SNUH [geocities.com] to get involved and fix the attitudes of both groups. There's no way I will register at boards with those attitudes, though.
I'd love for anyone to point out what is incorrect on the R3Mix site. I'm prepared to defend CDs over vinyl, though, so it had better be better than that.
99% of the bitching seems to be that the r3mix setting doesn't produce the very best audio. Well, of course it doesn't! It's promoted as the best acoustical compromise between space and quality. Even if it isn't the best, the fact still remains that 256 & 320 kbits MP3 are unintelligable to the original by normal human ears, which is what the original poster wants people to know. I suppose if you have abnormal hearing you might tell the difference, you know, like you're a Ferrengi or something.
Re:Please - no r3mix.net links (Score:2)
Re:Please - no r3mix.net links (Score:5, Funny)
Remember kids, you're only allowed to have ONE source of information! You must choose a site, and take everything stated there as the word of God!
Re:Please - no r3mix.net links (Score:4, Funny)
Re:finally (Score:2)
I'd like to use a lossless audio codec for video capture and editing for this reason. Unfortunately, it doesn't appear that any of these lossless compressors are set up so they'll work as audio codecs inside video editors. I can use Huffyuv [berkeley.edu] to compress video without loss; why aren't FLAC and its competitors available as ACM codecs so that content compressed with them can get stuffed inside a WAV or AVI?
Until then, I guess I'll just have to stick with uncompressed audio...when 1 hour of Huffyuv-compressed D1 video takes 20-25GB, the 605MB that one hour of CD-quality audio takes isn't much by comparison.
Re:difference high bitrate lossy lossless (Score:2)
This type of codec probably won't be used by the majority of people out there; probably just the most dedicated of audiophiles, and those who make/mix music.