Is Predicting Hit Songs Futile?

I recently covered Columbia Professor Duncan Watts’ “cumulative advantage” experiment, in which similar groups of people started with the same selection of songs but ended up with different choices for which songs were hits. If people were just judging the songs on content, the groups’ choices for hits should have been similar. However, there was also a social factor: Except for a control group, each group’s members could see the popularity of songs within their group but not within other groups.

Professor Watts proposed that the divergent choices for hits were due to each group’s piling-on to whatever happened to be initially popular within that group. See the original post for details.

In my praising the experiment, I held back on some questions about the strongest claim in Professor Watts’ New York Times Magazine article. In essence, he claimed that predicting hits was futile due to the inherent randomness of social systems like the word-of-mouth that affects entertainment choices:

Because the long-run success of a song depends so sensitively on the decisions of a few early-arriving individuals, whose choices are subsequently amplified and eventually locked in by the cumulative-advantage process, and because the particular individuals who play this important role are chosen randomly and may make different decisions from one moment to the next, the resulting unpredictability is inherent to the nature of the market.

This effect was true of Professor Watts’ experiment, but is it realistic to have the early-arriving individuals “chosen randomly”? Isn’t there a relatively small percentage of people who act as tastemakers: people who are into new stuff first and whose knowledgable opinions influence others? If these people have non-random qualities, shouldn’t there be a lot more predictability?

The Limits of an Individual Influential

After some email back-and-forth with Professor Watts, I was surprised to find that the role of “influentials” is potentially a lot less than is commonly believed. In a draft of a paper due for publication later this year, Watts and collaborator Peter Dodds detailed their mathematical simulations of various scenarios involving influentials. The results were summarized in the Harvard Business Review’s Breakthrough Ideas for 2007:

Our work shows that the principal requirement for what we call “global cascades”—the widespread propagation of influence through networks—is the presence not of a few influentials but, rather, of a critical mass of easily influenced people, each of whom adopts, say, a look or a brand after being exposed to a single adopting neighbor. Regardless of how influential an individual is locally, he or she can exert global influence only if this critical mass is available to propagate a chain reaction.

To be fair, we found that in certain circumstances, highly influential people have a significantly greater chance of triggering a critical mass—and hence a global cascade—than ordinary people. Mostly, however, cascade size and frequency depend on the availability and connectedness of easily influenced people, not on the characteristics of the initiators—just as the size of a forest fire often has little to do with the spark that started it and lots to do with the state of the forest.

The researchers’ forthcoming paper makes a compelling case for these conclusions, exploring influentials’ role under many different scenarios. However, its various social-network models all start with the single “spark” of an individual discovering and communicating something. It does not consider a scenario where a large number of simultaneous and non-random sparks occur throughout the network. That is, if a single, random spark can cause a forest fire under the right conditions, how about a bunch of sparks purposely set at once, across that same forest?

The Potential of Coordinated Influence

The coordinated, multi-spark scenario matters because it is how certain social-marketing companies supposedly work: unleashing a small army of “on message” people to tell their friends about some great new thing. One might argue that a favorable newspaper review, radio airplay, or other one-to-many media do something similar, simultaneously “sparking” many consumers at once.

The key point: Instead of having a single line of sentiment that needs to propagate enough times to reach critical mass, the multi-spark scenario has many lines propagating, each of which could randomly run into other lines, thereby accelerating toward a critical mass.

Bringing this all back to the original New York Times Magazine article and its assertion that hits cannot be predicted, the multi-spark scenario is a way for hits to be predicted. In essence, it increases the prediction reliability by manipulating the system.

You may say this is unfair, like loading the dice, but it’s how entertainment marketing works. Companies spend marketing dollars in proportion to what they think will be popular, thereby making what they think will be popular more popular. Economically, the question is whether the cost of manipulating the word-of-mouth system is worth the increased probability of a hit.

Note that predicting a hit doesn’t mean being right all the time; it just means that across many attempts the gain is greater than the cost. Thus, even if you only went from a 3% hit rate to a 5% hit rate, predicting was worthwhile if it cost less than the benefit from those extra two percentage points.

So, I’m not ready to conclude that it’s futile for entertainment companies to predict hits. If the companies were merely acting as pure observers, then Professor Watts’ case would be strong enough for me. However, because entertainment companies’ predictions are often entangled with manipulating the systems being predicted, there may still be reason to try to pick winners.

Whether the benefits outweigh the costs...well, that’s another experiment to do.

Do You Like What You Like Because You Like What I Like?

An experiment:

• Have a large group of people rate songs they’ve never heard before. Each person listens and rates privately so no one knows what others have done. If a person likes a song, he or she can download it. Call this group the “independent group.”
• Now have another large group of people do the same thing with the same songs, except members of this group can see how popular the songs are with others. Call it the “social-influence” group.
• Split the social-influence group into eight subgroups (“worlds”). Every world has the same songs, but a song’s popularity is counted only within that world. Thus, the social-influence group is split into eight parallel popularity contests.

The 4/15/2007 New York Times Magazine had a piece by Duncan Watts, professor of sociology at Columbia University, about this experiment. It was conducted via the Web with more than 14,000 participants. Professor Watts’ summary of the expectations and results follows.

First, if people know what they like regardless of what they think other people like, the most successful songs should draw about the same amount of the total market share in both the independent and social-influence conditions—that is, hits shouldn’t be any bigger just because the people downloading them know what other people downloaded. And second, the very same songs—the “best” ones—should become hits in all [eight] social-influence worlds.

What we found, however, was exactly the opposite. In all the social-influence worlds, the most popular songs were much more popular (and the least popular songs were less popular) than in the independent condition. At the same time, however, the particular songs that became hits were different in different worlds....

So does a listener’s own independent reaction to a song count for anything? In fact, intrinsic “quality,” which we measured in terms of a song’s popularity in the independent condition, did help to explain success in the social-influence condition. When we added up downloads across all eight social-influence worlds, “good” songs had higher market share, on average, than “bad” ones. But the impact of a listener’s own reactions is easily overwhelmed by his or her reactions to others. The song “Lockdown,” by 52metro, for example, ranked 26th out of 48 in quality; yet it was the No. 1 song in one social-influence world, and 40th in another. Overall, a song in the Top 5 in terms of quality had only a 50 percent chance of finishing in the Top 5 of success.

And why did this happen?

[W]hen people tend to like what other people like, differences in popularity are subject to what is called “cumulative advantage,” or the “rich get richer” effect. This means that if one object happens to be slightly more popular than another at just the right point, it will tend to become more popular still. As a result, even tiny, random fluctuations can blow up, generating potentially enormous long-run differences among even indistinguishable competitors—a phenomenon that is similar in some ways to the famous “butterfly effect” from chaos theory. Thus, if history were to be somehow rerun many times, seemingly identical universes with the same set of competitors and the same overall market tastes would quickly generate different winners: Madonna would have been popular in this world, but in some other version of history, she would be a nobody, and someone we have never heard of would be in her place.

I’ve quoted at length because I think it’s an ingenious and compelling experiment, well explained by Professor Watts. Although we all intuitively know the bandwagon effect, this experiment quantifies its importance in judging unfamiliar music. In this context, the results suggest we—the notorious average “we”—are quick to let what’s popular tell us what’s good.

The Joshua Bell Experiment

As “a test of whether, in an incongruous context, ordinary people would recognize genius,” The Washington Post deployed violin virtuoso Joshua Bell as an anonymous street musician in a Washington, DC, commuter plaza. At his feet, open for donations, was the case of his $3.5 million Stradivarius.

In 43 minutes, Bell played six classical masterpieces. 1,070 people passed by with little to no affect. Seven people stopped for at least a minute. 27 people donated a total of $32, not counting the twenty-dollar bill Bell got from the single person who recognized him.

A fine piece of writing, The Post article describes the experiment and ruminates at length about what it might mean. I can’t improve on that, but I’ll add some commentary about a few of the numbers.

First, and this isn’t pretty, Bell’s response rate of around 2.5% is similar to response rates for direct-mail solicitations of credit cards, loan refinancings, and such.

Second, the article tells us that Bell plays concerts where the cheap seats go for $100. It’s easy to read that detail as an implied value for his commuter-plaza performance, as if the 97.5% of people who ignored him might as well have been walking past a hundred-dollar bill on the ground.

This presumes that because some people would pay $100 to see Joshua Bell, then that’s the value. It’s not. It’s the value to the people who paid $100, not the average passer-by on the street. Based on the experiment, the value of seeing Joshua Bell to the average passer-by was roughly three cents. (Don’t believe me? Divide the $32 Bell made by the 1,077 people that passed by.)

Of course, the people paying $100 are doing so for a formal performance, at a concert hall, with an admission fee, at a convenient time, knowing that Joshua Bell is the player. All of that is missing from the experiment. So how surprised should we be that most people ignored him?

Quibbles aside, the article is still a good, thought-provoking read. It’s gotten a lot of play in the blogosphere, suggesting that if the public can’t recognize anonymized genius, it can at least recognize interesting commentary about the public’s inability to recognize anonymous genius.

Trojan Goldfish

We interrupt this blog for a special investigatory report.

Time is running out. To what, we don’t know. But a 45-year trail of clues is telling us something.

1962: Inspired by a fish-shaped cheese cracker she saw in Switzerland, Pepperidge Farm founder Margaret Rudkin “returns with the recipe” and introduces Goldfish snack crackers in the United States.

Unanswered in the historical record is where this “recipe” originally came from. Visual observation indicates that a Goldfish cracker is a three-way genetic crossing of a Cheez-It, an oyster cracker, and a goldfish. But is that really all?

This question matters because over time, Goldfish crackers have evolved—as if by some mysterious genetic code—from their original ecological niche as a cocktail cracker to the snack cracker of choice for small children. Along the way, Goldfish have spawned multiple variants that display emotions, personality characteristics, and the latent capability to influence a generation.

1973: Co-discoverer of DNA Francis Crick, with Leslie Orgel, propose the theory of directed panspermia, suggesting that the seeds of life may have been purposely spread by an advanced extraterrestrial civilization.

Although the typical interpretation of directed panspermia is about the origin of life on Earth, what if a group of Swiss scientists in 1959 came across recently arrived seeds of life, courtesy of comet debris still frozen after impact in the Alps? And what if careful analysis revealed that the ideal host for this new type of life was a baked-goods consumer product?

1987: Speaking to the United Nations General Assembly, U.S. President Ronald Reagan says: “I occasionally think how quickly our differences worldwide would vanish if we were facing an alien threat from outside this world. And yet, I ask you, is not an alien force already among us?”

One year later, Goldfish crackers go into space aboard the Space Shuttle Discovery.

1997: Goldfish crackers appear with a smile stamped on them, the first change since their introduction. They become “the snack that smiles back.”

With this, Goldfish become more than passive objects of consumption. Goldfish become friends to their little consumers. Ingratiating their way into relationships by simply smiling back, are Goldfish setting the stage for something more than smiles?

1998-2004: Pepperidge Farm introduces Goldfish product variants such as Flavor Blasted Goldfish, Goldfish Colors, Giant Goldfish, Baby Goldfish, Goldfish Sandwich Snackers, and Goldfish Crisps.

Consistent with the evolutionary theory of punctuated equilibrium, Goldfish speciation occurs in an explosive six-year period following a 35-year period of stasis. The new variants replicate the primitive emotional apparatus of “smiley,” albeit for different market segments. 

2005: Pepperidge Farm announces that “Americans will be smiling even more as they get to know [Goldfish] in a whole new way as the fun-shaped snack comes to life in three dimensions.”

Embodied in the animated character Finn, Goldfish now have a figurehead to actively influence young minds. More than a year of market research shaped Finn to leverage the already “significant emotional connection with the brand” that Goldfish had attained.

2005: According to Pepperidge Farm, nearly half of U.S. households with children under 18 purchase Goldfish snack crackers annually.

While perhaps true, this statistic masks the well-known fact that 100% of children under age five eat Goldfish crackers on a near-continuous basis. The few parents that have tried to resist—such as those who sought refuge from Goldfish’s cheddary goodness by living in former nuclear-missile silos—still found their children innocently enjoying handfuls of Goldfish while watching Teletubbies.

In other words, while Goldfish were evolving their emotional and communicative capabilities, they were also accumulating market share, invited into American homes like little Trojan Horses.

2006: Pepperidge Farm announces a new ad campaign featuring Finn and three new Goldfish friends, Gilbert, Brooke and X-treme. Steve White, Vice President, Youth Snacks, commented: “We see this new campaign as a tool to begin to help teach important lessons and help instill values in kids in ways they understand and identify with, without being preachy or patronizing. The Goldfish characters’ distinct personalities and tales of everyday life are things every child—and adult—can relate to in an optimistic way.”

Having built the infrastructure for its own mini-religion, with a devoted following of millions, what “lessons” and “values” will be forthcoming? What panspermic messages did those Swiss scientists transfer into the Goldfish genetic code that have yet to be expressed? And given Goldfish’s recent rate of evolution, how long will it be until Goldfish are capable of human-like intelligence, and perhaps superhuman emotional, brand-building characteristics?

The stakes are high. We could preemptively try to negotiate with them now, before they turn America’s children against us. If so, do we take the obvious route and negotiate with Finn, or do we try to turn his new sidekicks against him?

The 85 billion Goldfish crackers produced each year are forward-positioned in diaper bags, pantries, and other strategic locales throughout the world. We don’t know their next move. What will be ours?

[Note to readers who arrive here from a search after April 1, 2007, when this was written. If you are unfamiliar with April Fools (or All Fools) Day, then be aware that the above is not entirely reliable, and thus you should not use it as a primary source for your term paper.]