Jim Lucchese, CEO of Echo Nest, is giving a talk on the future of music, which he says is in the hands of app developers.

Echo Nest analyzes music tracks (16M so far), looking at many, many parameters. It makes that information available to developers of apps.

MTV uses Echo Nest to figure out who is listening to what, how the audio sounds, and what they’re saying about it on the Web, in order to build a personalized station. More interactive, more web-connected, more personalized, and more engaging, he says. Shifts in how we interact with and experience music are occurring every day. “Music apps are thriving” he says, referring to IOS (iPhone, iPad). The bad news is that most of the thousands of developers reshaping music are locked out of the business. They have to navigate all of the rights issues, and get access to the players. Echo Nest has a community of 6,000 developers, but many of the apps are sitting on the shelf because they can’t get access.

The aim of Echo Nest is to build a machine learning system that understands music, but does it at Web scale. It analyzes music and finds the pitch, tempo, etc. Pandora does this by hand, and has analyzed about 800,000 tracks; it doesn’t scale to 10M tracks. Echo Nest combines this with cultural understanding, which it gathers by crawling the Web. Out of this comes “a ton of data”: similar artists, how popular, tag clouds, hotness, bios, song structure, “fanalytics” (demographics of who is listening, psychographics, etc.) They make this info available to developers, who have made 120 apps, including visualizers, targeted marketing apps, etc.

Many were built during music hack days (weekend coding fests). E.g., more granular control over a Pandora-like app. Or, provide detailed info about artists and tracks. Or, Six Degrees of Black Sabbath: find connections between any two artists. Or, a social trivia app (name the tune, identify the fake band, etc.). Or, turn any tune into a swing tune using Echo Nest’s audio manipulation tools. Or, Audio Kicker: location-aware social music discovery act (uses tastes of a group in the same room).

But, there’s an industry chokepoint. The transactions costs are too high for dealing with a lot of developers. So, Echo Nest is working on open content API’s. If the artist is comfortable with more open models, Echo Nest makes the content available to developers. E.g., the DMCA allows streaming within some limits, e.g., no more than two tracks per album per hour. IF you comply, you can pay a compulsory license and not have to first negotiate the rights. Nest Echo lets developers access DMCA streaming of 10M tracks (because Echo Nest has done a deal — Seven Digital in the UK — with a license to those 10M for DMCA streaming). This approach means we don’t have to wait for copyright reform, it lowers the tansaction costs, and provides a filtering mechanism for content owners.

Q: The CEO of Pandora says that Pandora’s survived because humans do the music analysis.
A: It comes down to the quality of the results. We’re powering personalized radio for MTV, Mog, Thumbplay, Spottify, and for an enormous catalog of tracks. There are humans in our system as well: we’re aggregating what people say on the Web. Pandora has problems. E.g., if they want a Klezmer channel, they need about 5,000 tracks, and they can’t afford to put an army of Klezmer musicians to work finding and analyzing tracks. There are also problems with purely machine analysis: it can be hard to tell low-fi punk vs. country, Christian rock vs. heavy metal.

Q: Is your adio analysis violating copyright?
A: We don’t sell directly. As for copyright, there are a couple of cases. Gracenote (nee CDDB) uses a fingerprint to identify tracks. There’s been no litigation around whether what they or we are doing are derivative works. Our agreement with the holders is that we’re deriving facts that are not copyrightable.

Q: Among your developers, which countries are represented?
A: We just did a survey, but we made the mistake of letting the enter a free text answer to “Where do you live?” So, I’ll get back to you in a year. But there have been music hack days in Europe, Sao Paolo, maybe one coming in India…

Q: What’s the backend?
A: For audio analysis, we send out a lightweight binary that will analyze an audio track in about 2 seconds. We also offer that as a web service. We make the analysis data available for about 16M tracks. On the cultural analysis side, it’s highly customized, uses some open source (SOLR, Lucene), web crawlers.

Q: Business model?
A: We’re a data analysis company. Open API is for noncomercial use. If you’re a ommercial developer, we’ll charge a monthly fee and take a piece of your app’s revenues. If you’re MTV, you’re willing to pay a great license fee and don’t want to share as much with us. But if you’re developing, say, a jogging music stationthat matches the beat to your jogging tempo, we charge much less.

Q: Scholarly interest in analyzing your data?
A: Yochai Benkler was interested in the activity data, especially around artists who are giving away their music: we have data on playcount and how people are trending.

Q: Apps do well on the IOS, but is it just a few apps?
A: There was more churn than we expected. We looked at the top 100 music apps per month for a year, categorized them, and look at the number of new names. Streaming apps had 34 different apps in the top 100 in a year. No consolidation yet. (We don’t have access to the long tail of apps.)

Q: What will happen to copyrighted music?
A: Cloud based access is the answer to peer-to-peer sharing. If Spottify etc. offer a better experience than going through a file sharing network, that’s what people will do. But that will change the model: A user’s interaction with a track on Pandora is worth much less to an artist than the user buying a CD.

Q: Cost?
A: The apps are often free, but it costs maybe $10/month to get access to the music. The digital music market was about $4.5B last year. RPU in England is about $55/yr. If that goes to $120/yr, that’ll be a much bigger music. But maybe it won’t be $10/month, especially if you do a deal for a subset of tracks. Or an ISP opt-out plan for $5/month; the opt-out wold make the penetration rates much higher. Too early to tell. Most of the services are just beginning. Spotify, though, has grown to a million subscribes in Europe in over a year.

Q: Access from car?
A: We’re working with some companies. But, if you have a mobile phone and a car with audio in, you’re there. OTOH, the biggest music subscription company in the US is probably XM Radio.

Q: Selling your service to advertisers?
A: Record labels buy data from us to help them understand their market. One company is using our music data as a way to figure out how to target consumers for non-music products.

A: We are matchmakers between developers and large brands. The brands want apps built.

Q: How big is a catalog of 10M tracks?
A: 10M x 3.5mb ? Warner Music has to update hundreds of repositories and catalogs every week. There ought to be one centralized catalog. It’ll happen someday. Every time so far it’s been muli-year industry efforts among players who don’t want to standardize on a competitor’s standard. We’re very interested in opening up music metadata. We think there’s a commons approach. Problem: 50 ways to spell Guns ‘n’ Roses [sp]. We’re a text analysis company, so we do that. Every collection has its own ID sets. We released an open service called Rosetta Stone that maps among them. A free service. We’ve released an open source audio fingerprinter and do lookups against our database of tracks for free; if you’re compiling additional fingerprints you have to share them (and we share them, too). (We don’t download the tracks when we analyze them.)

Categories: copyright, culture dw