Monday, January 30, 2006

Pandora and Last.fm: Nature vs. Nurture in Music Recommenders

Over the past week, there has been some blog talk (Fred Wilson, TechCrunch, David Porter) comparing music-recommendation services Pandora and Last.fm. I’ve been using both for the past couple months, making notes along the way. The idea was that I’d eventually have something to say. That might as well be now.

Both services allow you to specify a favorite artist, based on which you immediately receive an Internet audio stream of similar music. When I tell people this is possible—that you can have a personalized streaming radio station—most are astonished. So let’s start by saying that what these and similar services do is cool. How Pandora and Last.fm do it is an interesting compare-and-contrast.

Nature versus Nurture

Algorithmically, Pandora versus Last.fm is something like the nature versus nurture debate. Taking the nature side, Pandora’s recommendations are based on the inherent qualities of the music. Give Pandora an artist or song, and it will find similar music in terms of melody, harmony, lyrics, orchestration, vocal character and so on. Pandora likes to call these musical attributes “genes” and its database of songs, classified against hundreds of such attributes, the “Music Genome Project.”

On the nurture side (as in, it’s all about the people around you), Last.fm is a social recommender. It knows little about songs’ inherent qualities. It just assumes that if you and a group of other people enjoy many of the same artists, you will probably enjoy other artists popular with that group.

Like Last.fm, most music-discovery systems have been social recommenders, also known as collaborative filters. Although much of the academic work in the area has focused on improving the matching algorithms, Last.fm’s innovation has been in improving the data the algorithms work on. Last.fm does so by providing users an optional plug-in that automatically monitors your media-player software so whatever you listen to—whether it came from Last.fm or not—can be incorporated into your Last.fm profile and thus be used as the basis for recommendations. Compared to relying on users to manually provide preferences, this automatic and comprehensive data capture leads to far better grist for the data mill.

A side note: In my years of analytics and data mining, a recurring theme is that better algorithms are nice but better data is nicer. That’s because a large number of smart people have evolved the best data-mining algorithms for various scenarios; thus, further improvements tend to be incremental. By contrast, whatever data you happen to be using in a project has probably had no priming for analytical use. Thus, improving how you acquire, clean, and transform that data can have disproportionately large benefits. The catchphrase for the negative version of this is “garbage in, garbage out,” although one could just as easily say, “the more signal in, the more signal out.”

Surfacing New Artists

Pandora and Last.fm are both about helping people discover new music, so let’s consider their approaches in terms of discovering truly “new” music—that is, artists who are just appearing on the music scene. If we assume that both services put new artists into their database at the same rate, Last.fm will be slower in surfacing them as recommendations. This is due to the “cold start” problem that afflicts social recommenders: Before something new can become recommendable, it needs time to accumulate enough popularity to rise above the system’s noise level. In contrast, because Pandora is only comparing songs’ inherent qualities—not who they’re popular with—it should be able to recommend a new artist the first day that artist is in the system. That said, I wouldn’t be surprised if Pandora did a little biasing of recommendations by popularity, which it measures as people use the service.

Partisans of Last.fm might retort that, in practice, Pandora will be slower at getting new artists and music into its database because of Pandora’s classification bottleneck—that is, the time necessary for a Pandora employee to classify each song on hundreds of musical attributes. With that bottleneck, Pandora can’t just classify everything as it comes in the door. By contrast, Last.fm does not need to do manual classification. With its software plug-in continually updating people’s preferences, Last.fm has a virtual army of talent scouts constantly finding new things, which Last.fm can integrate into its database automatically.

(Leaky) Locked Loops

Pandora people might counter that Last.fm’s army of talent scouts is compromised by its relative uniformity. That is, a social recommender tends to reward people who are like those who already use the system. If there are already many people in Last.fm with similar tastes to you, you’ll get good recommendations; if not, then maybe not. And if you don’t get good recommendations, are you going to keep feeding the system data? Probably not, and thus we have a self-perpetuating in-group/out-group situation. The result is a “locked loop,” whereby a social recommender gets stuck in certain genres and styles.

But with a social music recommender, a truly locked loop is unlikely. The reason is “leakage”: A population that shares the same core musical tastes will have enough variance in secondary tastes to allow for a continually expanding spectrum, albeit with much slower expansion in certain genres than others. Here’s an example of the problem. When I checked Last.fm’s similar artists to the reggae legend Bob Marley, first on the list was James Brown, followed by The Chemical Brothers, then Aerosmith. (If you’re reading this well after January 30, 2006, beware that Last.fm’s system is continually evolving, so the lists these links point to will probably have changed.) Other reggae acts appear further down, but the unlikely top choices suggest that Marley has been brought into the system more as a distant secondary choice than as a primary choice with other acts in his genre. A quick check of Aerosmith’s similar artists confirms this: Marley is 41st on the list, way behind various likelier suspects.

While better non-reggae recommendations are easy to imagine for Marley, they probably won’t appear until Marley’s primary fans are better represented on Last.fm. Then the quality non-reggae choices can emerge from his core fans’ secondary choices.

For the sake of comparison, when I put Marley into Pandora, I got something like a reggae radio station at first, which then drifted into other stuff over time.

Why versus What

Pandora is less subject to the echo chamber of overly like minds, but it has its own fundamental challenge in its reliance on matching songs’ “genes.” This rules out connections between songs or artists that don’t fit Pandora’s modeling and matching of musical qualities—which, in turn, puts enormous pressure on Pandora’s specific approach to be correct. In other words, Pandora’s success hinges on a theory, and a specific implementation of that theory, about why music recommendations work. By contrast, Last.fm simply describes what goes together according to its audience and then makes relatively simple inferences from that. So if there are hidden factors that Pandora isn’t explicitly capturing, Last.fm is at least capturing them indirectly.

It’s not hard to find cases where Pandora’s approach runs aground, although the system’s lack of transparency makes it difficult to know where the problem lies. For example, it’s hard to explain Pandora’s initial choices for Gary Numan (he of “Cars” fame). With Numan as the seed, Pandora gave me syrupy pop tunes by Orchestral Maneuvers in the Dark and the Human League. Yes, each artist’s most famous material was from the same time and was primarily electronic, but the latter two really miss the Numan aesthetic, which is more like supercooled liquid metal than warm syrup. Pandora went on to do somewhat better, but not great, with subsequent tunes.

In comparison, Last.fm immediately delivered Numan-appropriate songs from Assemblage 23, Killing Joke, Kraftwerk, and Skinny Puppy, eventually drifting into less relevant territory. Still, Pandora partially redeemed itself with an inspired connection: “Out of Control” by Ric Ocasek (former leader of The Cars), an obscure cut from an artist that is far from obvious as a connection for Gary Numan.

Last.fm’s Delivery versus Pandora’s Promise

I raise the Numan example because it exemplifies my experiences with Last.fm and Pandora. Having used a wide range of artists as seeds, I found Last.fm better than Pandora at delivering songs that I liked or at least didn’t feel compelled to skip, which is the most important thing when I’m listening while doing something else. The exception was when the seed artist had not hit critical mass in the Last.fm system, per the Marley example. Meanwhile, Pandora had more misses but was more likely to surface something truly out of left field, as with the Ric Ocasek example.

As a result, both Pandora and Last.fm have maintained a place in my music-listening world. However, ultimately I think Pandora has greater promise because it is far easier for Pandora to incorporate Last.fm’s functionality than the other way around. This point is important because, just as with the nature versus nurture argument, the best answer is likely to involve elements of both camps. That said, Pandora’s advantage comes at a significant cost to its business, with all the manual work it entails. At this point, Pandora is not delivering proportionally more benefit for that cost—which is why I used the word “promise” above.

Pandora Possibilities

The key to Pandora’s changing the game is to take better advantage of its exclusive, hard-to-replicate metadata about music. Users may never be able to objectively judge the quality of recommendations among different services, but they can definitely tell the difference between services with unique ways of getting to recommendations. For example, I’d like to see Pandora expose some of its internal attributes as dials for the user to control. If I put in the singer Paul Westerberg (former leader of The Replacements), I’d like to tell the system to match more strongly along his lyrical style rather than by the fact he has a “gravely male voice” (which is one of the things Pandora said it was matching on). It’s easy to picture many other creative uses of Pandora’s metadata, both in terms of a recommender and other applications.

Finally, I wonder why Pandora continues to employ hundreds of attributes. In the world of modeling preferences, hundreds of variables typically can be consolidated down to a much smaller number with nearly the same predictive power. Typically, you start with a large number of variables as a kind of fishing expedition and then, over time, reduce the set down to those that are doing most of the work. The reduced set can be part of the original set and/or new variables derived specifically for predictive power. For a labor-intensive business like Pandora’s, being able to cut the number of variables in half (or a lot more) would help contain the costs. And if there’s good reason not to consolidate attributes, I would still be wondering how to innovate in streamlining the production process just as much as how to innovate in the customer-facing part of the business.

Bowling or Batting?

A final thought: What Last.fm and Pandora do is hard. The people who built these services deserve a lot of credit. Given the ambitious scope, it’s easy to find examples where each of the services comes up short. However, it’s worth considering what the yardstick should be. Should we expect spot-on recommendations like a pro bowler expects a strike every time? Or is this more like the baseball batter, who is happy to get a hit one in three times? Whatever the metaphor, the fact that these services do enough right to retain a substantial number of users is good news, because the features and quality will only get better. So when you try Last.fm and/or Pandora, be sure to give them enough time—and enough different starting points—to show their best stuff.

78 comments:

  1. Implementation is also critical - as of right now (the first time I've checked out either service), Pandora is loading & working beautifully whereas Last.fm cannot serve even static pages in less than 30 seconds.
    For the moment, at least, this means that I'll play with Pandora but cannot even try out Last.fm ... which I may never visit again.
    Steve, I'm sure you recall that Firefly originally started as Ringo, a collaborative-filtering based music recommendation site in the mid-nineties. It's taken a long time for these ideas to find an implementation that can meaningfully draw users.

    ReplyDelete
  2. I think your insight is spot on, Steve. I also think that implementation is extremely important, as noted above. I believe ease of use for the end user is just as important as the content you receive. Pandora is a very slick, web based app that any user can jump right into, immediately. Last.FM has a confusing interface that requires more thought, and they ask me to install their proprietary software on my machine. Simple and good will always beat complex but slightly better. (or in this case, arguably better)
    I'm a musician myself whose spent many years trying to market my music. It's really tough in the music industry, especially for unknowns like my band 'Atomic Mint'. But the good news is that companies like Pandora and Last.FM in conjunction with the accessibility the Internet provides, will lead to a revolution of sorts.
    I think it's also important to realize that there is lots of competition in this space today. Companies like Last.FM and Pandora are beginning to emerge as the first players in the personal digital radio market, but what model will really grab the public by the horns?

    I just started a new music community with a few friends called BlueO2. We are still in Beta at the moment, please check us out! http://www.blueo2.com

    What's clear is this much. There are many evolving technologies in the music space. That much, coupled with a general public dissatisfaction with traditional music distribution models tells me, change is coming. In fact, it's already here! Who ever can figure out how to simply give quality recommendations to music lovers AND solve the distribution piece of the puzzle at the same time will come out on top.

    ReplyDelete
  3. I cannot do anything but agree to this brilliant review of these services. I haven't tried last.fm, but will definately be toying with it.
    The Pandora drawbacks are precisely those that I encountered in trying to "teach" it my musical likes and dislikes. Often Pandora would give me something that is theoretically similar to what I seed it, but it lacks the inherent qualities I listen for in Music.
    Hence I haven't really been able to construct a channel in Pandora that won't have me skip more than half the songs.
    Somehow this all reminds me of Zen and the Art of Motorcycle Maintenance. The concept of what one percieves as Quality cannot be readily defined, and thus never captured by a mere algorithm. This means that in the short run Last.fm's approach makes more sense. In the long run I cannot but agree that it should be a property of Pandora's approach in order for Pandora to "learn" about what humans percieve as quality music.

    ReplyDelete
  4. Hey!
    I have used last.fm for quite a while now (http://last.fm/user/thul) and I'm quite satisfied with the service! they have given me a lot of new inputs and most of them have been right on target!

    ReplyDelete
  5. Great article, which made me try out Pandora (I already use Last.fm)
    So,I loaded up Pandora for the first time and decided to create a station based on JJ72 (fairly obscure band). Woo it found them!
    But their profile says 'aggressive female vocalist'. Their lead singer is a MAN! Admittedly, a somewhat high-pitched singer, but still definitely of the male variety.
    Then I try a more mainstream artist: Manic Street Preachers. Again their description is problematic. Although their last two albums fit the 'harmony, emphasis on production' profile, their first four albums are utterly different, with a noisy raw punk influenced sound.
    You can use all the fancy coding that you like, but if the basic data is wrong then the whole thing is pointless!

    ReplyDelete
  6. I'va also been using both services for a while. Where I've come out is that I'm using last.fm more than Pandora, but because of the "tag" based stations. One stupid example was over Christmas, I was able to get a great channel going on the last.fm service, playing a succession of tunes from Frank Sinatra's version on White Christmas through to Slade's "Merry Christmas Everybody". No musical link between the two that could get caught by Pandora but a link which makes perfect sense in terms of my personal use of the service.

    ReplyDelete
  7. I am afraid that commerce will have to say a lot about which model will prevail. My guess is the last.fm model. Record companies will pay to have "matches" inserted into the listeners streams, which they hope will create interest with some listeners. Once a song gets known, the last.fm model will do the rest in making it a hit.

    ReplyDelete
  8. Regarding the lock loop issue that you mention above, there is another trend which could possibly help Last.fm. Specifically, people with non-mainstream/eclectic tastes wanting to show how eclectic their tastes are.

    ReplyDelete
  9. I am rather curious, why not include the best service there is like this right now -- Yahoo's Launchcast? I have been using that service for years and it's helped me to find lots of new bands that I would never have found otherwise.
    Algorithmically, I can't say whether it's better or worse than any of the others... but it works, and quite well.

    ReplyDelete
  10. Great article, lets hope that Pandora and Last's people are reading too :-)

    ReplyDelete
  11. I found that Pandora works better if you give it tons of seeds. I initially entered "Bjork", got one Bjork song and then 8 out of 10 songs that I did not like. Then I browsed Yahoo Launch for my other favorite artists and began seeding Pandora with them. After about 10 seeds, I started getting more songs that I liked.

    ReplyDelete
  12. I prefer pandora to last.fm, for a number of reasons; It loads faster, it falls over less often, and it doesn't keep playing the &^%^%"!" Dave Matthews Band no matter which artist I plug in.

    ReplyDelete
  13. I read about both these programs and having tried them both I would agree with your prognosis, although I feel the music gene idea is a little dubious in itself.
    http://www.liveplasma.com/
    This is another interesting take on the music reccomendation idea.

    ReplyDelete
  14. Wow, great analysis.
    I found the same thing in Last.FM that trying contemporary artists was spectacular, but stuff like Reggae and Jazz was not nearly as accurate.
    I did a similar analysis in December (also including Soundflavor as another option, and an AllMusic.com project called Tapestry that I've been involved with).
    More Music Nerdery here:
    http://datawhat.blogspot.com/2005/12/places-to-discover-music-online.html

    ReplyDelete
  15. I'd never heard of Last.fm until I read this article, but you seem to have analyzed the differences very fully. I agree that Pandora would benefit from allowing users to dial in the influence certain musical "genes" should have on its recommendations. While I might, for example, like Phil Collins for his vocals, I might be more interested in Bowling for Soup on the basis of their lyrics or energy.

    ReplyDelete
  16. I have been a user of Pandora now for 6 months. At first I started using pandora just because the whole idea of a music "genome" is very interesting; and in fact a few years back a friend of mine and I had discussed the possibilities that could come from a system that could ascertain musical relationships based on the music itself, and not its popularity...especially in realtionship to music copyrights.
    Anyhow, back to using Pandora, I began to realize that it really depended on the starting artist that you choose when you create your station and I firmly believe (like with your Newman expierence) that the closer you can get to a song that has exactly what your looking for by a certian artist, the better the system matches it with other artists.
    Example...just putting in "The Cure" the system will pull a random song by that artist and start the "comparisons". However by actually stating "Fire In Cairo" as opposed to "lullaby", it comes up with different results (as well as some crossovers). That is where I think Pandora shines, because then at least 1 - 2 in 5 songs is a song you have never heard or an artist you never heard of.
    If it was Last.fm, no matter what song I put in there by "The Cure", the whole of "the cure"'s songs are used, therefore you get the same recomendations, very rarely any new stuff...which is good when I want to hear all the usualls in more like a "underground late 70's early 80's" station.
    Just my thoughts, I'm not saying that Pandora is better then Last.fm, just in my case I prefer a little mix up that Pandora seems to provide easier than Last.fm.

    ReplyDelete
  17. Another "service" similar to this that works on your own music collection is the Predixis MusicMagic Mixer (http://music.predixis.com/). This "fingerprints" your music by analysing the actual music itself rather then manually entering variables. This may or may not be better but it would certainly be less man power intensive. I have had good luck with them, however, it does have the downside of never introducing you to new artists. I don't mean artists who are new on the scene today, I mean anyone who is not already in your personal music collection. The advantage of MagicMusic Mixer is that you can take your mixes with you on your personal music player. I have suggested to Pandora that they allow users, especially paying users, to create playlists in addition to just streaming.

    ReplyDelete
  18. Some very good points -- especially liked the bowlers v. batters and nature v. nurture analogies. But one set of opposing theories was left out: taxonomy v. "folksonomy".
    Pandora and Last.fm are on opposite sides of this divide, too. Last.fm has implemented (unfortunately not very well) tagging of the music which members enter. Used properly, along with the massive amounts of data collected, this could conceivably provide some of the filtering which you find lacking in Pandora. Pandora, however, has built its service on a fixed taxonomy, and, if "lyrical style" is not in their taxonomy, you'll probably not get to filter on that attribute. Now, of course, their analysis of their "genetic" data may turn up some unsuspected correlation between attributes which accurately predicts and delivers musical style, but that is hardly assured.
    Finally, even if Pandora could adapt Last.fm's data model more easily, Last.fm seems to be a more complete service, incorporating so many other aspects of community into the site that complement and extend the music recommendation core of the service.
    Now, excuse me, but I've got the Replacement's "Can't hardly Wait" stuck in my head and I've gotta go listen!

    ReplyDelete
  19. Thanks for that, I've been using Pandora for a while and even emailed them to suggest they go to a collaborative filtering approach -- I wasn't aware of LastFM. (I wrote a collaborative filtering system several years ago which I sold to IdeaLab! for it's short-lived RecoMentor website.)
    Another point of comparison: Pandora is far, far more listenable than 99% of internet radio stations in my experience.
    -Ralph

    ReplyDelete
  20. Just pointing out a quick mistake:
    "(If you're reading this well after January 30, 2005, beware that Last.fm's system is continually evolving, so the lists these links point to will probably have changed.)"
    It's not 2005 any more =]

    ReplyDelete
  21. I've been using Pandora for about a month now, but I started at a different place than you; I entered a SONG that I liked as the seed and worked from there. Pandora's ratings are based on a song's "genes", and supposedly doesn't seem to care about performer's relationships too much. So I think I would expect your Numan results if he ever recorded anything that was "warm syrup." Songs that you may not like by that artist get used as seed songs when you enter an artist name.
    By entering single songs I have found that I have much less variation in my recommendations, and I can gradually increase the variation by adding slightly different single songs.

    ReplyDelete
  22. I've been using Last.fm for about a week. I found it when I bought my Squeezebox from Slim Devices. There is a plugin to Slim's server software that will report your listening habits to Last.
    Anyway, I went to Pandora and entered in two artists whom I like - "Daniel Amos" and "Phil Keaggy". These are both fairly popular artists in the Christian music arena. Pandora did not know either of them, while Last provides an extensive list of tracks played, albums, and similar artists.
    So in my case, having musical interests that fall outside of what Pandora's controllers have chosen to enter, Last.fm wins hands down.
    So, perhaps contrary to your "critical mass" example above, Last.fm works better because if your musical tastes are "esoteric" you at least have a chance that others with similar tastes have signed up, where with Pandora one is completely at the mercy of the operators.
    - Jasen.

    ReplyDelete
  23. I completely agree with your viewpoint on Pandora. I do enjoy the loading time and fluid response in music playback. However, I also have had serious problems with the way it connects certain groups. With selections from Pearl Jam or Moe. I have found various times that I would have to skip 5 to 6 songs to get to one good song.
    This can be extremely annoying when I am doing something else on my PC or in my office. Especially if Pandora plays 3 consecutive songs that match the taste of sound I want to hear, followed by 5 songs that have no connection to the style of music outside of certain guitar rifts or artist "influences". Last.FM is a great engine for people who want to listen to music while multi-task, but Pandora seems to be more of the gem at finding new artists.
    There are times where Pandora has delivered several groups that are new to my ears and pleasing. Thoug h I agree some sort of reform for it is needed. I would prefer, maybe not everyone, to see a better pattern in its presentation of artists I enter in. If a music channel I define is supposed to have Pearl Jam as the main group, I would like to hear more consecutive tracks of that artist. I do like that it makes the connection from Pearl Jam to Mad Season (a band that consisted of members of Pearl Jam/Alice in Chains/Soundgarden collaboration), and I hope it still makes those type of connections.
    In time, I can see Pandora expanding from user feedback, and I hope the constructive criticism your article provided as well as others helps in doing that.

    ReplyDelete
  24. Hi,
    I find your article excellent, and very much to the point as for the observations you have made on the pros & cons of these two recommender systems. However as a musician, I must point out that neither of the two systems satisfy me truely (although I use them with pleasure, and am comvinced of their success for 95% of the listenners).
    Pandora is somewhat frustrating, as you pointed out, since you cannot refine your liking of a song/artist, i.e. I like mild syncopations, but I don't like accoustic arrangements. As a musician I was frustrated by Pandora, not so much by it's poor choice (which I found often "good enough to listen to"), but because I quickly found I couldn't give accurate enough indications about what music I liked. The best way I found to imrpove this was by adding multiple songs and artists to a channel, rather than just having one artist that I critiqued a lot.
    Last.fm (which I have just discovered through your article-thx) really has a HUGE cold-start problem. Having used it now for the whole afternoon (and having critiqued at least 50 songs) I am still getting 9 bad songs for 1 good one (according to my taste). However the tag-system works really well, which is not surprising: tags combine a mix of "user-preference" and "content value". Is this the future of recommendation systems? ...

    ReplyDelete
  25. Curiously, Steve didn't notice the most conspicuous thing of all about Pandora - how ridiculously AMERICAN all the music is !!!
    That's the real flaw in the Pandora model - they can only think of themselves, their own tastes, and their own extremely limited worlds.
    As good as Pandora is, it's deeply flawed until they start to understand and address the inexhaustibly rich music universes outside Fortress America.
    Get it?

    ReplyDelete
  26. that's funny, pandora.com isn't working for me "it's taking longer than expected ... bla bla bla" (and i tried several artists and tracks) whilst last.fm is and has always been working for me (they had a one day streaming downtime because of an upgrade)

    ReplyDelete
  27. The Bob Marley example is striking, but I do not think the problem is lack of critical mass, but overabundance of critical mass, so to speak. Since Bob Marley is the one reggae artist hugely popular among people who wouldn't touch most other reggae with a 10 foot pole, the recommendations actually make a warped kind of sense: Bob Marley is mainstream, and so is James Brown, but not so much, for instance, Lee Perry.
    That is also why The Beatles have a tendency to show up everywhere, at least on other hugely popular artists' entries, like Coldplay and U2.
    I've given up worrying about it: last.fm has features which let you get around all that very easily: you can set the stream to only play music you've never heard before, and there's an obscure vs. popular slider.
    And re: John Hart: An unfortunate coincident had Last.fm upgrading its servers yesterday, which may explain your outage.

    ReplyDelete
  28. I am a user of Last.FM (formally Audioscrobbler) and I did try Pandora whilst it was in beta. I have to say the big point of Last.fm is to create a community (user buy-in I think they call it), where I keep going back to Last.FM to check the groups/forums and nose around the site. I have the choice to check out the charts and see what my (musical) neighbours are listening to.
    I do agree on the quality of data being submitted to Last.fm it looks like a headache to people that run it and they ask that users run all their tracks trough the Musicbrainz software which straightens out any incorrect metadata.
    I will continue to use Last.fm I like what they do and they seem like a nice bunch. Oh and it's free.

    ReplyDelete
  29. It's worth noting that Last.fm gives paying members priority when it comes to loading pages and delivering streams. There was also a radio upgrade going on around the time the first comment was left according to their news page.
    I prefer Last.fm for two main reasons:
    1) I've never stumped it with an artist. No matter how obscure I've tried to be somebody else has already listened to that band and put them in the system. While Pandora frequently responds to my inputs by saying "who?"
    2) The social aspects. In last.fm I can read the journals of my neighbors (users with similar tastes according to their algorithms) and participate in forums based on my general criteria. Such as "Radiohead is not our top artist" or "we love Radiohead" or even "Firefox users." And each of those groups get their own recomendations and their own radio stations.
    So for your Bob Marley example you could've listened to the radio station for one of the Bob Marley groups or the radio station attached to a related tag such as "Marley" or "reggae"

    ReplyDelete
  30. I like the randomness of Pandora--I've been using it for about three months. My tastes happen to focus on bands that aren't widely known so I think pandora is a better fit for me--Plus I have heard some awesome music I would have never heard on Lastfm just because it's so "underground".
    I don't have anything super important to say except that you wrote "january 30th 2005" instead of 2006. You might wanna change that.
    Liked the article, it was a good read.

    ReplyDelete
  31. One thing I've noticed about Pandora, is that it does much better when you use specific songs as seeds rather than artists.

    ReplyDelete
  32. Good article. Have been using last.fm for a while now (with no problems at all in page loading times). Might check out Pandora too after this.
    Btw, you put 2005 in there (just after the bob marley link) when I assume you meant 2006. Might want to fix that. :)

    ReplyDelete
  33. I think you are missing the reason many people use Last.fm -- not primarily for music recommendation, but for its data collection and presentation abilities. Each user gets a public page where the music they like is presented within an attractive, well-designed web page. Many people would like to add information on what music they are currently listening to as well as music they most like to their blog, but are either too lazy or lack the technical competence to do so. Last.fm makes this easy and automatic.

    ReplyDelete
  34. Steve-
    this is the first time i have been to your blog. based on this post i will be sure to look at some previous posts and checking back for the stuff in the future.
    I just recently have been exposed to last.fm and Pandora (listening to Pandora right now), I found Pandora to be much better at playing songs that "matched" what i was looking for. I would not say that it has exposed me to much new music though. I am not sure about all the techinical stuff that goes on behind the curtin to make it all work, but i do know that if i want music to play with out much work from me to get songs that i will not want to skip i will listen to Pandora. It works for me.
    Thanks for the comparison and all the technical background.

    ReplyDelete
  35. I tried Pandora but ditched it as too many of the 'predictions' were misses for me.. Their algorithms didn't seem to work very well for some electronica subgenres. I will try last.fm.
    Thanks for the write-up though, nicely done.

    ReplyDelete
  36. AMG, the people behind allmusic.com have just launched a similar service demo called "Tapestry" at http://tapestry.allmusic.com/ . Besides doing such mundane tricks as finding similar songs, albums, or artists, Tapestry uniquely creates playlists bases on "Themes", "Tones", and "Styles". One can choose from giant lists of Themes such as "Road Trip", "Spring", "Dinner Ambiance", and "Divorce". After playing with various services extensively, I can say with confidence that, in my humble experience, nothing even gets close to Tapestry in the accuracy and relevance of the playlists that are generated. Tapestry is getting around the trials, tribulations, and inaccuracies of other approaches such as collaborative filtering by using the Allmusic.com editors to hand review each song using a list of more than 6200 descriptors (according to the Tutorial documentation). Hopefully AMG will come out with a commercialized service for on-demand streaming radio based on Tapestry since it would instantly make subscription streaming music services hugely popular.

    ReplyDelete
  37. Well, some of this has to do with the concept of "new" and that has to do with the "state" or "intent" of the listener.
    For example, when I was a college dj, in 1980-82, and punk/new wave was cool, I liked to find 60's classics that mapped back to similarities. So, I played things like the "nuggets" collection and garage band rock opposite new wave.
    Today, a 13 year old would find classic Elton John "new" and a system that mapped back from chronologically new, to all things similar and maybe obscure might be more what the listener was looking for.

    ReplyDelete
  38. Each service seems to answer a different question. Pandora answers "What other songs sound like this song/arist?". Last.fm asks "Assuming my taste in music is not based just on the inherent qualities of music, but on my personal demographics (age, where I grew up, where I've lived, etc.), what other songs are associated with the demographics of my life?".
    Sometimes people a prefer song because it reminds them of other songs, while other times they like a song because it reminds them of other times/people/things/experiences.

    ReplyDelete
  39. Well... Last.fm requires a LOT of software, and doesn't work with my music store of choice, Rhapsody. Not only that, the Last.fm people actually DISCOURAGE writing plugins for streaming media services.
    OTOH, Pandora is beautifully simple, what a breath of fresh air compared to trying to get Last.fm to work. I've been listening to Pandora for an hour and can't get myself off of it to go do stuff that needs to get done!

    ReplyDelete
  40. You really hit my main issue with Pandora!
    They need to expose more of the prperties.
    If I give a song a Thumbs Down because of lyrics or vocal style, but really liked the beat or the melody, there is no way to distinguish, thereby diluting the value of the "Personalized Station"

    ReplyDelete
  41. Update: Per several comments, lest I be accused of living in the past, I fixed a date reference to 2005 that was supposed to be 2006. Also, in addition to this post's comments section, there are more than 100 comments on Slashdot:
    http://slashdot.org/articles/06/02/01/0553233.shtml
    Thanks for the kind words and interesting opinions.

    ReplyDelete
  42. Just a quick note - you've written the date wrong, I presume - 2006, not 2005, right?
    Personally, I've been using Last.fm as a tool, mostly because I like the fact it records my tunes, more than using it as a recommnedations engine - I have VERY wide rangnig tastes, so find Pandora too closed.
    Great post though!
    K

    ReplyDelete
  43. Steve this is an incredibly thoughful and articulate post. Thanks for taking the time to consider all of this and contribute to the conversation that Fred kicked off a couple of weeks ago.
    We've just tonight pushed out a big new Pandora release, so I've been very caught up in the logistics around that. As a result, I've have been kind of watching all of this from the sideline. Maybe that's for the best; it's been fascinating to watch and learn from the passions, observations, and critiques that are fueling the discussion.
    Suffice to say that while I'm the CTO at Pandora, I'm also a fan of what the gang at last.fm are doing; I'm a paying last.fm subscriber and I maintain a musical profile there. At least in my case, the services scratch two very different itches.
    Thanks again for the insightful post.
    Tom
    CTO @ Pandora

    ReplyDelete
  44. Very nice breakdown and comparison. I personally use pandora, and have been for quite some time. I think there are two distinct sides to the classification system: The good side is that it is genre ignorant. Sometimes I really like that, since I listen to most musical styles. The bad side is that it is genre ignorant. Some genre combinations just aren't suitable on the same station. It would be nice if they added "omg no more of this genre on this station" buttons for rating. I've also found that as I add seeds to the stations they get become appropriately refined.
    I would also say that I think the commercialization aspect could end up being a bad thing. With pandora's algorithmic model, you're more likely to get things fed off pure data. With last.fm, I could see the easier potential for preference seeds to be bought (kinda like they are for radio now) - which means that in the end you're really just hearing what record companies want you to.
    Overall, I've been much happier with pandora and discovered good new music from it. After reading this I will give last.fm another chance, but I don't see pandora leaving my regulars (especially if they get to the point of opening an API for me to develop against).

    ReplyDelete
  45. Here's another music recommendation service which I think its worth to mention it, its called Foafing the Music http://foafing-the-music.iua.upf.edu
    BTW, I tried (in the demo box) with artists: Bob Marley, Gary Numan and Paul Westerberg and the results are quite interesting to have a look at!

    ReplyDelete
  46. I wonder how many comments to this entry were placed by viral marketing firms- a good half of them triggered my B.S. filters.

    ReplyDelete
  47. last.fm works fantastically for very fringe artists. Bob Marley was used effectively to make a point, but everyone's heard of him. Hence, since reggae is under-represented, you get people who mostly listen to aerosmith, and some Bob Marley on the side. However, if you search for Autechre you get a very effective selection including Aphex Twin, Squarepusher and Boards of Canada. People who aren't into electronic music probably won't have heard of Autechre, so the results are 'undiluted'.
    As long as last.fm keep a Google-style independence (no preference-for-money), they will interest me...but have they already sold-out to the Dave Matthews' Band?
    Pandora don't interest me as much: their classification system is too obvious. I love music which is non-obvious and hard to second-guess.
    I would love it these tools could suggest interesting comparisons: for example Amy Winehouse's October Song and Sarah Vaughan's Lullaby of Birdland. I have no idea how this could be done. I guess that's what real DJs are for!

    ReplyDelete
  48. I agree with Andy Farnsworth on his mention of the Taxonomy/Folksonomy comment. After becoming a fan of the tagging idea as modelled by Flickr and Del.icio.us - I went on an active search of musical folksonomies - and so I was sort of glad to discover Last.fm. But also a little disappointed - because the tagging feature only seems bolted on as an afterthought. I suspect that most users don't bother tagging things with any diligence. And who can blame them - it's not a very prominent feature of the site - nor is it very easy to navigate to enter lots of tags for a diverse playlist. What I wanted & still want - is a way of searching for music by a large variety of criteria: time-signature, instruments, country-of-origin (and language?), subject-matter, and of course genres and sub-genres... (How else can I find a Hungarian Rock-n-roll Czardas Ballad featuring Hurdy-Gurdy? (Ok - there probably aren't too many of those - but if they're out there - then I do want to find them!)

    ReplyDelete
  49. I've taken a slightly different approach to comparing the two services. I review them both from an end-user, web-developer, and entrepreneur perspective on my blog here - http://www.streampad.com/blog/?p=24 . In the end, I prefer last.fm

    ReplyDelete
  50. Great posts. Use both services for different things. To me, one subject not really discussed is that music is first and foremost an emotional experience, not a technical one, a fact which makes any attempt to categorize music on this grand a scale so inherently difficult and ultimately flawed.
    Just because two songs use similar sounds, structures, or instrument type, that doesn't mean both songs are good...certainly doesn't mean I will like both. Those technical aspects are useful, they're just not the most valid predictor of what I will like. Emotion is. So while I appreciate what they're tying to do w/ Pandora -- I'll continue to try it -- I just wonder if they're barking up the wrong tree.
    I think Last.fm has a better shot at recommending something I'll like because it relies on a vast network of my peers' listening habits. Peers who don't care about structure or instrument as much as they care about what they respond to. Simple as that. I mean, when I play a song 22 times in a week, I'm not thinking, "I just can't get enough of this mild rhythmic syncopation and major key tonality," I'm thinking, "Man, I F#!**** LOVE this song!" And very possibly my close "musical neighbors" will as well.

    ReplyDelete
  51. See also my Pandora bakeoff here: http://ymusicblog.com/blog/2006/02/10/yahoo-music-personalization-primer-pandora-lastfm-soundflavor/

    ReplyDelete
  52. Hey great post reviewing both services...thank you!
    This whole technology niche is sooo cool in a TIVO sort of way. Power to the viewer..or listener!
    I was introduced to last.fm via my squeezebox/Slimdevices.com
    I enjoyed last.fm but have found an over reliance on certain groups for my musical preferences (eg too much Phish and Dave Matthews).
    I have been exposed to a couple new groups but again it is because they are in heavy rotation.
    I really like pandora!! Maybe after a few more days I will draw a different conclusion, but at this point I find its interface and subsequent 'tune surfing' more conducive to finding new music I like based on the preferences I have fed it.
    Maybe over time I may prefer getting what I expect with last.fm versus the (appropriate) new and unknown I seem to get with pandora.
    At this point I see last.fm as fine tuned top 40 radio. Pandora as link to my unexplored musical tastes.
    Both have a role at this point.
    Thanks!

    ReplyDelete
  53. Hey thanks for the interesting discussion and the mention of two interesting services. Lots of potential in this type of service, IMHO
    Unfortunately, neither one fits me well so far, since I tend to listen to VERY different types of music for different occasions.
    Has anyone found a service that better supports one user having different profiles?
    You might think - and I was hoping - that Pandora's stations would support this (it seems like a step in the right direction).
    However they have only one list of favorites per person for example, and seems not so easy to managage stations, and as others have noted, Pandora doesn't work quite so well (yet) if/when your interests are out of the mainstream.
    So no luck for me so far. Any suggestions of services that support multiple user defined profiles or genres per user, and separate sets favorites etc. within each of the profiles?

    ReplyDelete
  54. I enjoyed the post and I found some interesting follow up points. Pandora is a data mining tool that focuses on finding new tracks that resemble some user input. The whole point is to discover new music. I may or may not like the next track, but I appreciate the fact that it is new. It would be annoying if Pandora kept playing the offensive track after I gave it a negative score, but not before!

    ReplyDelete
  55. Hans Cz. JørgensenMarch 8, 2006 at 7:10 AM

    The obvious thought that strikes me is: 'would it be likely that last.fm and Pandora could join forces?'
    I wonder what such a site would look like. Is it even feasible?

    ReplyDelete
  56. I'm a big fan of Pandora, and it's probably my favorite. Steve - great comparison of the two sites. Hopefully when the Pandora team get on top of all the new music 'suggestions' Pandora really will be a force to be reckoned with.
    The next issue of Music Monthly magazine (19th March) is going to run a feature on '20 essential music-related websites'. Hopefully we'll see more comparison there.

    ReplyDelete
  57. One of the most interesting new music finding techniques I'm using at the moment is MySpace. Most of the bands I like are pretty obscure. Lots of them have MySpace pages. The bands who are influenced by them have invariably befriended them. So by friending those bands, and putting a note on my page saying 'hey, if you play music like the bands I like, let me know', I'm getting a steady stream of email from bands saying 'you might like us'. And indeed I might.

    ReplyDelete
  58. Thanks for excellent piece! I have to say I am partial at this point to Pandora because 1. This is a new/fresh/interesting way of presenting music to me, which I am always open to, and 2. I am a little tired of simply being linked to things based on what others like. This is across the board with media: "well people who liked this also liked this, or people who bought this also bought this...". It doesn't always capture the nuances of my moods or tastes. Pandora definitely has a way to go (I admit to having a few bizare recommendations come my way) but this approach is very cool.

    ReplyDelete
  59. I just tried last.fm, for the first time, entering the jazz guitarist Bill Frisell — not mainstream, but not obscure either. What I got was : "Error : There is not enough content to play this station."
    If a mainstream choice gives least-objectionable programming, and a targeted choice no result, maybe Last.fm is made only for popular-but-not-too-much artists ?
    Pandora recognized it and produced a very nice set of artists…

    ReplyDelete
  60. TechCrunch has an article today on a Pandora/Last.fm mashup:
    http://www.techcrunch.com/2006/03/29/pandora-and-lastfm-togethersort-of/

    ReplyDelete
  61. I'd never heard of Last.fm until I read this article, but you seem to have analyzed the differences very fully. I agree that Pandora would benefit from allowing users to dial in the influence certain musical "genes" should have on its recommendations. While I might, for example, like Phil Collins for his vocals, I might be more interested in Bowling for Soup on the basis of their lyrics or energy.

    ReplyDelete
  62. Aparently the way we understand, use and consume music is suffering tremendous and interesting changes. Ever since the digital revolution, the music industry has been seriously afected, since napster and limewire facilitated mp3 files exchange, the way we listen and buy music is not the same.
    Then comes personalized radio wich is very welcomed, since my music library, exceeds 5 thousand legaly purchased tracks, and creating playlists that suit my different moods becomes very important, but it can be a full time job!
    Then services like Last.FM and Pandora are great when you have a party or when working on you computer, its music you like with little effort.

    ReplyDelete
  63. I enjoyed your review.
    I was interested in your mention of the wide scope of attributes in Pandora as a potential drawback.
    I think of it rather in the context of granularity. While in a purely analytical context the bubbling up effect into smaller and smaller aggregates, my be meaningful, in the context of the sensory experience, without increasing granularity nuance is obviated.
    The low resolution of mid seventies monochrome displays versus high definition colour displays of today is a good example.
    The larger the canvas the greater the capacity for revealing the otherwise inexpressible.
    Another way of expressing the difference between these two "systemic" experiences is the one is artist centric, the other is listener centric.
    While the one explores the complexity of the artist by combing different attributes into new aggregates, the other merely normalises the listener's experience toward the mundane.
    In all art, the acquisition of "taste" is always to some degree initially a little bitter or a little sweet, a little sour or insipid (unclear).

    ReplyDelete
  64. So to bring this a little more up to date since the review was created very early in the year.
    It would appear that Pandora has indeed started to adopt some of the aspects of Last.FM, allowing you to exercise preferences on music. I assume that this means that they are weighting the information in the background so that if you do look to listen to Mesuggah you can tell it that "Gravely Male Vocals" really aren't part of the appeal.
    But then you can't listen to Pandora for very long without runing into a "please sign up" prompt, which appears to be tracked by IP. Not sure how this would work in a multi-user environment behind a proxy server, but that seem to be what they've adopted.
    Last.FM on the other hand, has started a streaming media page that appears to let you listen to music for hours upon hours without having to sign up for an account. I suspect that will change at some point since its erally only a disadvantage from their perspective, but for now its good.
    Last.FM also shows potential for abuse, at least thats the only reason I can see for creating a station based on a tag search for Ministry and ending up having Britney Spears pop into the playlist.
    These changes are of course meaningless in the long run, what you really want are extensive playlists and a juge artitst pool. Pandora just ain't got it.
    It can be said of me that I am difficult when it comes to music. I expect everyone to at least be familiar with what I listen to. Pandora wasn't that familiar with my selections, but last.FM had at least heard of 5 of 7 artists (vs 3 of seven for Pandora). Sure I didn't expect to see The Peyote Kings show up in the playlists, but The Great Lukeski could only be found scrobbled in last.fm, Pandora drew a blank.
    Of course if you consider being teased (as last.FM will do "Oh, yeah! That artist is mentioned in the database! We don't actually have it but it was scrobbled!") a bad thing, stick with Pandora.

    ReplyDelete
  65. Both, last.fm and pandora are incredible. At the moment I listen a lot to pandora, thats why I use a firefox plugin, which includes my listened songs from pandora to last.fm (http://code.yerblog.com/lastfm/)... I think thats a great opportunity.

    ReplyDelete
  66. Steve, thanks for excellent analysis. I'm curious if your thinking changed in any way since you wrote this post a year ago.
    Now that Pandora powers MSN Radio, Last.fm received some funding, and Ticketmaster liked iLike, it seems that there will be some acquisitions this year, which usually slows the pace of innovation. How will we all find what to listen to next?

    ReplyDelete
  67. LastFM is brilliant - but completely different to Pandora. We use it in all of our posts at http://www.trackfeeder.com . Its great for bloggers. Personally i think Pandora is a bit of a let down . You have to sit through hours and hours of rubbish music. Try http://www.finetune.com . I can't see it mentioned is these comments. Its brilliant

    ReplyDelete
  68. I agree that Pandora's music is a bit substandard. I much prefer listening to last.fm's radio as it has more surprises. Let's see how the CBS buyout effects the music they play and the artists that they push.
    On a different front of music recommendation I have recently been enjoying using The Filter. This software effectively makes Pandora/Lastfm radio stations from within your own music collection - which is a nice way to rediscover long lost songs and listen to your music in a refreshing sense. I recommend it as a free download from http://www.thefilter.com

    ReplyDelete
  69. I know its been a little while since this post. I just want to say how much last.fm's recommendation system has improved. Obviously there's been a lot more users & music added to the database, but to me, its obvious their algorithm has been significantly improved. I joined in Jan 07; Lastfm's recomendations to me in say April 07 were delivering good recommendations about 5 or 6 times out of 10. These days its 8 or 9 times out of ten. A fantastic hit rate, and the recommendations are not only popular artists. I get a lot of spot-on recommendations for artists with only a couple of hundred plays.
    I now found myself wanting to play my recommendations in preference to music I already know. If you haven't listened recently, its worth checking out again.

    ReplyDelete
  70. I haven't been happy with either Pandora or Last.fm. The stuff they think I'll like gets rapidly worse towards the tail -- they work fine for the popular stuff I like, but not so much for the rarer stuff. Also, being a mac user, loading a web page really doesn't mesh with the way I listen to music on my computer. I was using iLike which is a downloadable app, and it's OK, but recently I've switched to Additune. The recommendations are simply amazing and it works with my iTunes library. I don't have to rate stuff, I don't have to tell it what I want to listen to -- I just select a playlist from my library and tell it to give me more songs like the ones in my playlist. It's still a beta and doesn't yet do all the stuff some of the other guys do, but it's getting there. The GUI looks like something out of Leopard and again the recs are just so spot on it's uncanny.

    ReplyDelete
  71. I agree that Pandora could be improved upon. Specifically, as you have mentioned, by allowing users to shape their stations based on 'genes'. However, Pandora is far more than 'promising' when compared to the stagnant and unoriginal Last-FM which apart from its 'skyping' capabilities offers nothing more than the same old method of receiving recommendations from the ignorant masses with their inappropriate and misspelled tags. It especially falls short for those with obscure tastes, only playing a few tracks before running out of choices. Pandora provides great recommendations no matter how idiosyncratic the seed, as my Glass Eye and Khaleel stations attest.

    ReplyDelete
  72. a great aticle..have been using lastfm and pandora for years now an both are excellent for me,as i'm into all kinds of music/

    ReplyDelete
  73. I started using Pandora and Last.fm around the same time. After a while I gave up Last.fm and stayed with Pandora. The problem was that I was finding it more difficult to "customize" my listening experience on last.fm. IMHO it wasn't quite "learning" my tastes fast enough. Pandora, on the other hand has created the ultimate stations. It takes time, but my listening experience is much better on Pandora.
    Some may disagree, but let me point out one thing I discovered about Pandora. It's fairly easy to end up "in the weeds" if you throw too much information all at once. If I throw too many songs and artists at it all at once without letting it "learn" you end up messing up your station because songs and artists have different sounds and Pandora gets "confused" so to speak. Take it slow. Add a little at a time.

    ReplyDelete
  74. Superb article Mr. Krause and some insightful observations about the way it is and the way it could be. Thank you.
    After reading this article and several of the ensuing comments, there is no way I'll be making the trip to LastFM. Pandora is (and will continue to be) by personalized streaming radio station of choice. The notion of LastFM feeding me a playlist based on how my specific music tastes are similar to everyone else is precisely the homogenized, herd-mentality, lowest-common-denominator drek that has diluted artistic expression (and appreciation) since the dawn of culture.
    While Pandora is pioneering the exploration of the essence of music, LastFM is mining the mass media, mass culture, artistically stagnant, be-like-everyone-else vein.
    Which - given the nature of the web and contemporary culture - means LastFM will be wildly popular and rich while Pandora gets shunted to an academic back-water as a theoretical curiosity.
    Sorry. Feeling a bit snarky today. The point is... I would much rather discover new music based on the MUSIC's composition (even if I don't necessarily like it) than based the statistical average of people who think Surivor and The Bachelor are haute culture, and who haven't the courage to embrace an emotional challenge.
    feh... I sound like a frothing aesthete. Do what you want, listen to what you want... but if you consume the musical equivalent of junkfood, don't whine when your soul collapses in puddle of flaccid, self-indulgent, twaddle.

    ReplyDelete
  75. You can have the best of both worlds, however. There's a service accessible through LastFM that lets you listen to Pandora and tag/love/ban the tracks and artists at LastFM while scrobbling what the tracks you hear. Thus you not only get Pandora's (eventually) better recommendations but the tracking and retention capabilities of LastFM. I get mostly garbage from LastFM listening to their "stations". Pandora actually applies my "yeas" and "nays" to what it feeds me next.
    When you can make stations based on whether you're in the mood for potato chips or Quiche Lorraine, and refine them to your tastes, and still look up your favorite artist from a month ago, that's how music software should work.

    ReplyDelete
  76. I agree that last.fm s getting much better the last year. And as more jou train your station the better the better it sounds.

    ReplyDelete
  77. One aspect of Last.fm that is more compelling is the Social Network aspect. The services are fairly equal in many ways, but I prefer Last.fm just because I find myself hitting the "Ban" button fewer time. And once I developed a group of friends, many from my "neighborhood" well at that point it wasn't only about the music, it was about visiting these new and often distant friends whose musical taste turns out to be a quite a predictor of other shared perspectives...Pandora's social efforts fell short and I hardly ever go back...
    john
    www.very.fm

    ReplyDelete
  78. There is nothing more that I could add to the debate which has not already been said, except for that Last.fm has one distinct advantage over its competitors - It is available outside of the U.S, unlike Pandora who had its global operations restricted to just American territories because of the copyright board rulings in 2007 and were required to restrict its use.
    Unfortunately, for this reason I have never been able to expereince Pandora for myself, but have read about it numerous times, which is quite a shame. It appears that the biggest concern for personalised radio stations is the outrageous music licensing which is bordering on the absurd. If only the music companies realised that these tools for music discovery could be used to boost exposure for their artists, it could create a new econmic model not based on expensive and complicated individual licensing deals.
    Gerd Leonhard offers an insight into what th future of music could be like in his e-book "music 2.0" which does raise some interesting questions about copyright with digital music, if a bit lacking in detail in parts, which I suggest people should check out if they're interested in the topic.

    ReplyDelete