The Technology Behind Last.fm 125
CNET's Crave has up a detailed interview with Last.fm's Matthew Ogle, the company's head of Web development. Reader CNETNate notes that Last.fm has streamed 275,000 years of audio around the world. From the interview: "We stream all music directly off our servers in London. We have a cluster of streaming nodes including a bunch of powerful machines with solid-state hard drives. We have a process that runs daily which finds the hottest music and pushes those tracks on to the SSDs streamers that sit in front of our regular platter-based streaming machines. That way, if someone is listening to one of our more popular stations, the chances are really good that these songs are coming off our high-speed SSD machines. They're fast because every song is sitting in memory instead of being on a slow, spinning platter." The interview is actually on two pages but pretends it's on three.
SCSI 4 LIFE! (Score:1, Offtopic)
Re: (Score:2)
But these days people don't use run-of-the-mill insults and actual technical data to argue the issue, they prefer to use insults like "shitkocks", "butfaget" and "cawksokker" and they'd rather accuse each other of having deviant sexualities than argue over unimportant technical details (who cares what technology is best suited for a task when it is obvious that all SSD users are mactalibanfaggets because you can buy macs with SSD drives?). (Yes, I miss the days when usenet was still alive and useable).
/Mika
275,000 years? Wow. (Score:1, Funny)
Last.fm has streamed 275,000 years of audio around the world.
I'd love to know how much of that was stuff like Britney Spears.
Re: (Score:1, Funny)
Re: (Score:2)
I was interested to find out a couple of years ago that I had a current and active last.fm account. Turns out that my original audioscrobbler plugin on one of my computers was still alive and kicking and sent updates to the same servers (prefs/plugin were in App Data or something and got copied over with every reformat)
Comment removed (Score:5, Funny)
Re: (Score:2, Interesting)
They have detailed week-by-week charts going back to 2005. Lady Gaga is in fifth place this week is at 1,923,168 plays by 92,208 listeners.
Muse, The Beatles, Radiohead, and Coldplay precede her, but that's likely due to the fact that Last.fm is based in the UK and the majority of their users from the UK* and that those bands are much much better
Re: (Score:3, Funny)
English, you insensitive clod.
Re: (Score:1, Funny)
Hardly. 'English', 'Scottish', and even 'Irish' people are all really Welsh; they're just too embarrassed to admit it. Something about the silly place names...
Re: (Score:2)
Hardly. 'English', 'Scottish', and even 'Irish' people are all really Welsh; they're just too embarrassed to admit it. Something about the silly place names...
And those poor Welsh souls are really wannabe Cornish!
ah the cornish (Score:2)
in perpetual denial of the true fount of all civilization and culture in the british isles: the isle of man
Re: (Score:2)
He's got a point - it's where Mark Kermode came from as well as umm, err... umm...
They've got a cool annual TT race too.
Re: (Score:2)
Wrong, the U.K. was formed in 1800.
Re: (Score:1)
Re: (Score:1)
What do you call someone from the UK? I wanted to say British but that excludes Northern Ireland.
'British' is used for UK residents, not just the residents of Great Britain. It therefore includes Northern Ireland. For example 'British Government' is a term often used by the UK government.
Re:275,000 years? Wow. (Score:4, Informative)
What do you call someone from the UK? I wanted to say British but that excludes Northern Ireland.
Only if you're a republican; plenty of northern irish identify themselves as "british".
Re: (Score:1, Interesting)
Only if you're a republican; plenty of northern irish identify themselves as "british".
So why is the name of the country the United Kingdom of Great Britain and Northern Ireland? That name does suggest that Northern Ireland isn't part of Great Britain.
Not trolling, just curious.
Re: (Score:2)
Great Britain's the name of the island comprising England, Scotland and Wales. The UK is all that (and the surrounding smaller islands) plus Northern Ireland. The adjective used to cover all the UK is British. The island of Ireland is geographically part of the British Isles.
Re: (Score:2, Funny)
Re: (Score:2)
That's OK, we prefer it that way.
Re: (Score:2, Funny)
What do you call someone from the UK? .
a subject
Re: (Score:1)
It is funny that the 2nd most scrobbled artist is "The Beatles" but the do not have any of their songs :(
Re: (Score:3, Informative)
They haven't really decided. Rule of thumb, all the U.K. areas except England tend to go by their own name, and England goes by British about 50-50, depending on age, politics, etc. But what do I know? That is just my guess from observing some Wikipedia disputes over this issue.
The "demonym" for the U.K. is "British". That includes Northern Ireland... an awkward situation. Of course, we have "Americans" meaning just the U.S. And back in the olden days, you either called
Re: (Score:1)
discludes
You mean excludes?
Re: (Score:2)
Why is that? I didn't realise that ones taste in music could be such a defining characteristic.
Are you also this snobbish about peoples choice of software, clothing, transport?
Re: (Score:3, Insightful)
It's a generational thing I suspect. Many people of my generation (I'm mid forties) very much define people by their music, especially what they listened to in their youth. The particular sub culture you belonged to as a teen was strongly related to your musical tastes and general mindset. These days, fashion/tribe is still important but the music side less so - you can have kids who dress the same but have
Re: (Score:3, Insightful)
>>>This marks them out to the rest of us as being a bit mindless, easily led etc.
I've found that those who listen to "alternative" music have the same flaws, but are merely following a different type of peer pressure (the pressure to listen to non-popular music).
Re: (Score:2)
I've found that to be less true for people who listen to alternative music, and thus had to actively seek out their music, than those who listen to the music considered mainstream. If the peer-pressure is to listen to non-popular music, I suspect you've confused causation with correlation: people who listen to non-mainstream music seek out different peers to be "pressured" by.
Re: (Score:3, Funny)
You're new here?
Re: (Score:1)
Many of those people did everything right, stayed out of trouble, never questioned authority, and are generally conscientious but lacking in personality and interesting life experience.
Shit, at least my friends wait until they're really drunk before they break out the Toby Keith.
Re: (Score:1)
at least my friends wait until they're really drunk before they break out the Toby Keith.
Who?
Re: (Score:2)
It still confuses me why their "psytrance radio" keeps pulling in Britney Spears.
Re: (Score:2)
If they can't appreciate Sonic Youth I don't want to know them.
Re: (Score:2)
>>>Last.fm is definitely a way to feel awkward with friends. Some of my acquaintances are well-read, well-dressed, well-spoken people, the sort who really seem to have it all together, but then you can never really manage the same level of respect for them after you've seen their Last.fm profile is nothing but Justin Timberlake and Lady Gaga.
>>>
That's because (1) a lot of the so-called "better" music that people have recommended to me is actually boring shit, and (2) I'm not looking for bo
Re: (Score:3, Insightful)
Call me picky but I wouldn't be interested in a woman that liked that sort of music.
Re: (Score:2)
Re: (Score:2)
I feel quite comfortable dumping someone for having generally (what I consider) bad taste in music, art, literature etc. Tastes reveal a lot more than you give credit for. (Granted, that's broader than "disliking one's favorite musician.")
Re: (Score:1)
Re: (Score:3, Insightful)
Being familiar with music from hotties like Timberlake and Daughtry makes you popular with the ladies. Saying, "Those guys are crap" is only going to get you dumped
By "ladies" you of course mean: girls under 18, or fat old women who wear Betty Boop/Disney pajamas and form Twilight fan clubs.
Me and my girlfriend spent many a night bonding over Slayer and Mike Patton, which suited me just fine. I'd actually be very frightened if she listened to Justin Timberlake (and whoever "Daughtery" is), since there is s
Re: (Score:1, Insightful)
Don't judge them too harshly. Unless you went to school for music, or otherwise spent some quality time cultivating an appreciation for the organization of sound, you probably don't have any taste. Music seems to be the last area where anyone bothers to develop taste nowadays.
In this age when everyone is concerned with breaking down the barriers and bucking elitism, maybe listening to pop trash makes your well-heeled friends feel folksy and good about themselves.
I did go to school for music, and did play
Re: (Score:2)
Re: (Score:2)
I think a lot of people educated in Western music over-emphasize the appreciation of the melodic line and phrase over rhythm. Western art music only caught up with polyrhythms used in other musical traditions in the beginning of the 20th century, and even now, few really do well with microtones. I did have a background in music theory and appreciation (and performance), but it took a course in Hindustani classical music for me to start to realize what I was missing.
Re: (Score:2)
If the unschooled person hears something more than the lyrics, it's usually only the highest and lowest pitches at any given moment (the relationship between the bass and melody). All that western harmony in the middle spectrum is really lost on them. That's what my music cognition friends have to say about it, anyway.
I wouldn't considered myself really "schooled" in music. I played an instrument for 8 years during my pre-college days, but it was nothing special.
But I'm in the exact opposite, I listen to the music's melody and harmony and almost ignore the lyrics entirely. I admit that I don't have the grasp of the complexity as someone truly schooled in music, but I don't listen to the lyrics much save for a few singers.
Heck, I used to laugh at myself because there were a few rap songs with decent music that I liked a
Re: (Score:2)
With you on that, it's very rare the lyrics get me, I'm more of a rhythm and timbre man myself... which accounts for my liking hip-hop and death metal amongst other things lol :)
Re: (Score:2)
You know, I was right there with you knocking Lady GaGa, but there's really more to her than she lets on...
http://vigilantcitizen.com/?p=2737 [vigilantcitizen.com]
Here she is pre-fame performing at a NYU talent show: http://www.youtube.com/watch?v=NM51qOpwcIM [youtube.com]
And a Metafilter comment that expresses what I want to say very well: http://www.metafilter.com/86769/Norah-Jones-Look-Out#2827870 [metafilter.com]
I considered Lady Gaga a guilty pleasure until this last video came out. I think it epitomizes all of her potential as a star and artist. All of it, down to the intentionally cheesy Europop backing, is commentary. And all of it is a mask, revealing practically nothing about the person at the helm. That's a really difficult feat to pull off, because stars tend to be insecure enough to want to be liked and respected as an icon but also as a person.
Gaga is basically trying to keep up the illusion that there is no person under there, or at least not what we are used to thinking of as a person. There is something trans-human about her ambition that I think is perfectly timely. She's a smart person having a lark, taking it a million times farther than anyone in their right mind would attempt, and making other pop stars look like the fools and relics that they actually are. I'm surprised more people don't get or appreciate that.
posted by hermitosis at 6:41 AM on November 18 [132 favorites ]
Re: (Score:2)
Since I have seen more examples of her signing without autotune in other videos and signing without autotune WHILE playing piano in the linked video, I have even more respect. Sure she does some crazy shit for attention but I bet she is making a killing doing so and will have no problem dropping into some more respectable "professional" mus
Re: (Score:2)
Yeah, but the whole Gaga schtick, she's playing a character. She's rubbing all of our faces in it, too.
The way I think about it: Kurt Cobain killed himself because he was completely distraught over how his music was distributed and digested. He wrote songs about cliques and fads and how shallow and empty and stupid they are... just so see those same cliques he was mocking empty-headedly became his fans and turned his band into a fad, a trend.
He was aiming for the audience, but it sailed right over their
Re:275,000 years? Wow. (Score:4, Informative)
56,904,147 plays (1,246,583 listeners) [www.last.fm]
Re: (Score:1, Interesting)
A lot of her scrobbles get deleted though:
http://playground.last.fm/unwanted
Re:275,000 years? Wow. (Score:4, Funny)
"I'd love to know how much of that was stuff like Britney Spears."
Unfortunately, you'd probably have to measure that metric in Libraries of Congress.
Re: (Score:2)
Re: (Score:1)
Re: (Score:2)
I hope that's an SI unit, else you run the risk of offending the metric nazi's ;)
Naw, that's not offensive.
Try: One Kilo-Libraries of Congress is 1024 Libraries of Congress. Tremble before my usurpation of your precious, precious prefixes!
RTFA (Score:5, Informative)
Reader CNETNate notes that Last.fm has streamed 275,000 years of audio around the world
Where did the submitter get that impression? Certainly not from the article. It mentions that they scrobbled 275,000 years of audio. Scrobbling is what Last.fm's client does when it takes a song you are playing from another source and uploads the meta data to them. Clearly that uses much less bandwidth than streaming a song
So now even the submitters aren't reading TFA anymore? I know, I know... its slashdot. /sigh
Re: (Score:3, Informative)
Re: (Score:2)
If anyone has asked me (prior to reading the above comment) if I'd ever "scrobbled" on the internet, I'd be turning my webcam away while in front of my computer a lot more.
Re: (Score:2)
So now even the submitters aren't reading TFA anymore? I know, I know... its slashdot. /sigh
You must be new here.
Memory instead of platters...? (Score:5, Insightful)
I really hope that was an attempt to dumb it down for the article. It's a pretty poor way of describing the difference between HDDs and SSDs. After all, HDDs are a form of non-volatile memory too. They just happen to have a mechanical aspect.
In fact, the only way in which they could stream music without having it all in memory first is if they were using a microphone and a live band. Sure, it might make for an entertaining data center, but it's not very scalable.
Re: (Score:2)
... or they could have just filled each 1u server with 8-16 GB of ram and run 1-8 memcached daemons on each cheaper than ssd, same result.
Memcached for MP3 music? I think not. (Score:2)
... or they could have just filled each 1u server with 8-16 GB of ram and run 1-8 memcached daemons on each cheaper than ssd, same result.
How does your suggested setup using memcache get around the problem of storing/fetching objects greater than 1MB? That is, without implementing custom interfaces for handling multiple chunks.
From the memcache wiki page:
http://code.google.com/p/memcached/wiki/WhyNotMemcached [google.com]
"Memcached is terrific! But not for every situation...
You have objects larger than 1MB.
Memcached is not for large media and streaming huge blobs.
"
Cheers.
Re: (Score:2)
Any sane person would just split an mp3 file in 256-512KB chunks of data.
This is a non-issue anyway... changing the default memcached unit size is a matter of editing a header file and recompiling the daemon server.
Re: (Score:2)
Memory used to be much cheaper and (quality) SSD drives more expensive , only a few months ago I bought a performance 4GB kit for around 60$ (plus taxes in my country as I'm in EU). In addition, a company like last.fm would probably buy at least 1000 memory modules at a time and at such scale you would get a significant discount - 30$ or less for a 2 GB module would not surprise me.
You would have to keep in mind that SSD drives also die, probably 1-3% of them each year, so maybe it makes more sense to just
Re: (Score:2)
I was assuming that they really were referring to memory. At 4mb per song you can hold 4000 songs in 16GB of ram which isn't a huge amount for a server. With 4000 songs per server for popular servers you shouldn't need to touch the disk so having fast SSD drives seems like a waste of money. I would put fast SSD drives for less popular songs since SSD is cheaper than ram in terms of storage capacity and then for very low popularity I would use hard disks if cost is that much of an issue.
MogileFS (Score:2)
Beyond that, our streamers are all running Linux and using MogileFS -- which is an open-source distributed file system, which is a little bit like a software RAID system.
OMG Files a lot of files served with MogileFS.
No thanks, last.fm (Score:1, Interesting)
This is precisely why I rarely listen to radio, whether it's streamed or broadcast over the air. They place too much weight on "the hottest new music", and this causes otherwise good music, which may not be "today's hottest new music" to be buried in the background noise. Not to mention that "the hottest new music" then gets played over and over, 100's of times a day on popular radio stations. This get boring and monotonous really quick. While radio can be a good way to discover new bands, I rarely listen t
Re: (Score:2)
Re: (Score:2)
Last.fm isn't exactly radio the way you describe. You give it a few seeds, [...] and it can play related music.
Yes, but no. You can 'tune' it to anything, hot or obscure as it may be. It'll play something that sounds like your seed, then something that sounds like that, then something that sounds like that, which unfortunately sounds nothing like your original seed.
The upshot is that I do listen to Last.fm, but too frequently I need to skip and block and my few well-used stations don't ever seem to learn. Pandora does this very much better but, again unfortunately, they offer even less international access (without
Re: (Score:2)
But it doesn't do that, "playing something that sounds similar". It groups things according to tastes of its listeners, nothing more, nothing less. It outputs "people who listen to what you specified also like this"
Which, for me, works much better than Pandora approach. The latter is so...sensible, logical...predictable. When its recommendations aren't disliked by me, it gives something which I already know.
Last.fm tends to give quite a bit more of nice, new things which I've never heard before. They rarely
Re:No thanks, last.fm (Score:5, Informative)
Last.fm's definition of "hottest" is what people actually listen to. It's not a handful of artist names handed down from MusicMegaCorpCoLLC to be digested by the uninformed masses ;)
I suggest looking at what Last.fm actually is. It has helped me find new music frequently. It also made me spend lots of money, which is the only real drawback. Anything you play is recorded, and musical compatibility with other members is compared to give suggestions. There might not be samples of everything on their site, but I usually find samples somewhere (Spotify is the weakest, iTunes and eMusic usually has it).
Re: (Score:3, Insightful)
Last.fm's definition of "hottest" is what people actually listen to. It's not a handful of artist names handed down from MusicMegaCorpCoLLC to be digested by the uninformed masses ;)
The masses' musical tastes are still mostly decided by "MusicMegaCorpCoLLC", even if not directly through Last.fm.
The same general pool of artists is popular on Last.fm as is popular on radio.
Re: (Score:2)
Really? You have at your place a popular radio which plays mostly Muse, The Beatles, Radiohead, Coldplay, The Killers, Red Hot Chili Peppers, Metallica, Kings of Leon, Nirvana, Green Day, Pink Floyd...and so on? ( http://www.last.fm/charts/artist [www.last.fm] ) Now?
Re: (Score:2)
The same general pool of artists is popular on Last.fm as is popular on radio.
So are you suggesting that because popular music is...popular that it is inherently "bad music" and that once anything becomes "popular" that it was due to the "uninformed masses"? Yes, there is horrible music being created and promoted that gets to the top, but do you have an inherent dislike for something just because it reaches a certain level of notoriety?
Re: (Score:1)
There might not be samples of everything on their site, [...].
That has become the main problem for me, aside from the same 30 songs they play everytime I start a station, nothing new ever comes out.
In order for it to work better, you must listen to whatever genres they cover better. I guess alternative, grunge, pop and rock-pop may be ok. But for heavy metal they do not have a lot of diversity, OR their recommendation algorithm is broken.
Re: (Score:2)
You know, they can't just take any music they like and stream it. First it ahs to be put there by owners.
Anyway, "text recommendations" work OK regardless.
Re:No thanks, last.fm (Score:5, Insightful)
I'm not sure if you've really missed the point, or are just trolling (making me an idiot for replying..).
The idea of putting the "most popular" tracks on SSD is to make it more efficient to stream the tracks that are more likely to be requested.
It's optimising the efficient use of their hardware. It doesn't have anything to do with last.fm's suggestions algorithms and does not at all mean last.fm will force these tracks on you.
You're amusingly uninformed considering you're throwing around terms like "sheeple".
Re: (Score:2)
uninformed = uniformed
Or something.
Re: (Score:1)
Re: (Score:2)
Because the part that everybody other are sheeple and you are soooo special is just self-delusion.
There are certainly many people on Last.fm with similar music tastes to yours. But...their music library has some artists that yours doesn't, and which still likely fit. That's the data on which Last.fm builds recommendations.
Re: (Score:3, Interesting)
I do use Last.fm, unfortunately they do not have a wide range of artists, at least not of my liked genre.
Every time I start a station with say the "Satriani" artist tag, I get the exact same 20 songs (in random order), before something completely unrelated start playing. I have the same results with "Kamelot", "Stratovarius" and "Dream Theater".
I liked it more when you could specify two or three artists. That would give you a bit of more breadth on the pool of music to listen.
Regarding alternatives, I have
Re: (Score:3, Informative)
I liked it more when you could specify two or three artists. That would give you a bit of more breadth on the pool of music to listen.
But you can:
http://www.last.fm/listen#pane=multiArtistTab [www.last.fm]
I agree, though, at some point it will either expand its scope, loop the playlist, or just stop (saying it ran out of appropriate stuff to stream). Frankly, though, if given a narrow topic (and a finite music library), what else could it do?
Re: (Score:3, Interesting)
Re: (Score:2)
Our "compatibility" - for example - is SUPER (my last.fm username is the same as here), but just glancing through the charts, it seems we only have the whole prog-melodic-symphonic metal area in common (if I can lump those together for the purpose of this exercise), but that's only about half of what I listen to.
It never seems to recommend anything worthwhile based on, for example, Gogol Bordello; and it's not like they are obscure or any
Re: (Score:2)
They also have a lot of Saxon
Re: (Score:3, Insightful)
Uh, the only impact being the "hottest new music" has is being served from one of their SSD hosts instead of the normal streaming cluster. This is completely transparent to the user. It doesn't limit what you can listen to or even make it more likely you'll hear "hot new" music from an SSD host. It's caching of the most popular files, plain and simple.
Re: (Score:2)
You know, there was an easy way you could make sure if what you're writing makes sense...
http://www.last.fm/charts [www.last.fm]
The most presented list, Weekly artist and track charts:
1. Muse
2. The Beatles
3. Radiohead
4. Coldplay
5. Lady GaGa
6. The Killers
7. Red Hot Chili Peppers
8. Metallica
9. Michael Jackson
10. Kings of Leon
Heck, even hyped artists and tracks (which simply show the increase in actual listening) seem passable...
Memory (Score:3, Insightful)
They're fast because every song is sitting in memory instead of being on a slow, spinning platter."
Aren't the HDDs (the one's with platters) still considered memory?
http://en.wikipedia.org/wiki/Computer_memory [wikipedia.org]
Computer memory refers to devices that are used to store data or programs (sequences of instructions) on a temporary or permanent basis for use in an electronic digital computer.
Re: (Score:2)
Hmm, your dumb question masks a more interesting one: aren't SSDs still considered a storage device as far as the OS is concerned?
In which case, doesn't the data from the SSDs still pass through the server's DIMMs before making its way onto the network? Or is there some fancy pipework which avoids this bottleneck?
Re: (Score:2)
See, my questions seemed non-dumb to me, especially since I'm a biology major and rely on questions concerning computer hardware/software classification specifics, to learn.
When I "attempt" to answer your Biology related questions I'll try not to do it with such a pompous response as you so kindly did.
G'day.
Re: (Score:2)
Please forgive me, I (wrongly) assume everyone here is a computer nerd. At least I didn't call you a pedantic jackass.
There is a fundamental conceptual difference between memory and storage, which is independent of the actual material used. When a program runs, its code is first loaded into memory (probably from storage) where the CPU acts on it directly. The program can read from and write to storage, which is assumed to be more permanent, but possibly orders of magnitude slower to access. Sometimes the me
Re:Memory - yes and no (Score:2)
The Wiki article would translate to what you said. But there is another view that needs to be thought about. That is "how it is connected?".
ROM and RAM are normally connected directly to address and data buses of CPU and thought today more as memory. The disk are normally connected though an interface (some times another computer/controller) and thought as storage. This all about the PHYSICAL connection. It is also why other connections to "disks" can be created like RAM disks, iSCSI and tape-drive-disks
Re: (Score:2)
They're fast because every song is sitting in memory instead of being on a slow, spinning platter."
So does that mean that the songs sound like a 45 played at 78? Or that the ones from the hard drives sound like a 45 played at 33?
Of course, half the /.ers here won't know what I'm talking about. :-)
Last.fm complements Spotify (Score:1)
Re: (Score:2)
I find it quite easy to use last.fm without Spotify.
Hmmm... (Score:2)
From TFA:
We give the labels a breakdown of which artists should be accruing what royalties, so they have fairly good information on what they should be paying who.
Given what we've heard about record labels, who wants to bet that when this "fairly good information" gets to the record label it is printed on nice soft paper, cut into individual sheets and then placed in the lavatories?
Duh (Score:1)
Re: (Score:1)
Or hierarchical storage management (HSM). Really not new idea.
Take heed - Last.fm can run servers (Score:4, Insightful)
Of all the so-called "social" sites whose services I use, Last.fm probably has the best uptime and overall availability. I think I've only seen the main Last.fm site down once or twice in over two years, and I've never seen the Scrobbling service go offline. On top of that, they can actually run a database - unlike Facebook, with its oft-inaccurate or missing data, all of my Last.fm profile is always there. Kudos to these guys for sticking to it and figuring out how to manage high loads properly instead of just whining about how inadequate the tools they have to work with are.
--- Mr. DOS
Re: (Score:2)
Re: (Score:2)
Or just not use the radio service (since that isn't the only service they provide).
Besides, $3 per month? That's peanuts for all you can hear streaming music! Its not as if streaming music over the Internet is exactly a low-bandwidth job.
Re:NOT $3 (Score:2)
So, no, in terms of risk, I'd be paying far m