Spotify slashes staff to move faster into AI – and Wall Street loves it

Shares of parent company Spotify Technology SA are up more than 30% over the last six months and up more than 135% year to date.

The company joins other tech firms in retrenching as pandemic-era demand has dried up. It also has to make up for the more than $1 billion it spent on podcasting, much of which went toward deals with celebrities to make podcasts that never materialized and acquiring podcast studios that it later shuttered.

“Economic growth has slowed dramatically and capital has become more expensive. Spotify is not an exception to these realities,” Ek wrote in a letter to staff posted to the company’s website.

Jumping on the AI gravy train

In November, Spotify unveiled a partnership with Google Cloud to overhaul how the platform recommends audiobooks and podcasts through its use of one of Google Cloud’s language models, Vertex AI Search.

Large language models like ChatGPT are computer programs trained on large sets of data that can recite human-like text and information back to users based on what the program “knows.”

Spotify introduced an “AI DJ” in February and began using OpenAI’s “Whisper” voice translation tool to translate select episodes of English podcasts into Spanish, French and German.

A representative for Spotify said in an email to CNN that the company plans to expand the technology in the future pending creator and audience feedback. They also pointed to some comments made by Ek during the company’s third-quarter earnings call, where the word “efficiency” was used more than 20 times.

“The primary way you should think about these  (AI) initiatives, (is that it creates) greater engagement and that greater engagement means we reduce churn,” he said during Spotify’s October earnings call. “Greater engagement also means we produce more value for consumers. And that value to price ratio is what then allows us to raise prices like we did this past quarter with great success.”

In a research note, Douglas Anmuth, managing director and internet analyst at JP Morgan, said that along with investments into advertisements by artists, investments into podcasts have the potential to drive engagement over the long term.

So how does personalization work?

Spotify has hyper-personalized its experience for users for about a decade. It was able to add that personal touch once it acquired music analytics firm, The Echo Nest Corp, in 2014, to combine machine learning and natural language processing.

Spotify’s technology builds a database of songs and artists by recognizing musical pitches and tempos and connecting the works of artists within a shared cultural context.

Metadata like release date and metrics like volume, duration and how likely a song is to get someone dancing also go into determining which songs fit a user’s taste.

From here, playlists like “Daily Mix” and “Discover Weekly” are born. So-called Time Capsules and “On Repeat” playlists gather a user’s most-listened to songs, to either keep users hooked to what they’re already listening to or revisit songs they haven’t heard in a while.

In an email to CNN, Anil Jain, global managing director of strategic consumer industries at Google Cloud said that its Vertex AI Search allows media and entertainment companies to build content discovery capabilities across video, audio, images and text. Jain did not comment on any details of the deal with Spotify.

Vertex AI Search considers a range of factors when recommending content for users such as real-time user behavior, content similarity and content related to what users are searching for.

Challenges and opportunities

Reece Hayden, senior analyst at ABI Research, expressed confidence that large language models (LLMs) could work to increase engagement across Spotify’s platform.

“Large language models can enhance personalization, improve recommendations, and ensure recommendations are more reflective of user interests by understanding entire text/video rather than utilizing keywords/metadata,” he said in an email to CNN.

He added that unlike keyword/metadata dependent “basic predictive models,” LLMs can understand and interpret podcasts to see if they match user interests and can gain a deeper understanding of user preferences by analyzing all user data to determine their preferences.

But that comes at a cost.

“Running LLMs to understand all podcasts/audiobooks is resource intensive and may add limited value compared to basic predictive models … LLMs bring additional data privacy and cost/resource challenges which will be significant,” he said.

He expressed faith in Whisper to help translate podcasts, but admitted mistakes may be made in the form of flubbed sentences or phrases as generative AI learns.

“Given the availability of data points, different language translations models like Whisper will quickly improve, ensuring a high degree of accuracy,” he said. “The downside of whisper is that its core competency is translating from other languages to English … Most podcasts are recorded in English and therefore it cannot be applied effectively across the board.”