Resume
Back to projects
Resume
Back to projects
NYU: Applied UX Research
(2023)
(
Identify areas of improvement in Spotify's music recommendation feature through algorithmic analysis and user research.
SOLUTION
KEYWORDS
Spotify,Youtube Music, music recommendations, user experience, algorithmic transparency, personalization
PARTNER
Riola Musoke-Lubega
MY ROLE
Co-UX Researcher and UX Designer conducting literature reviews, user interviews, usability tests, and data analysis. Collaborated on solution proposals and final report documentation.
Abstract
Spotify’s personalized music recommendations and streaming service model has played a large role in restructuring the music industry. This research study aims to identify the areas of improvement within Spotify’s music recommendation feature by establishing a framework that understands Spotify’s algorithmical function in music suggestions through literature reviews, and uncovers users’ thoughts and sentiments through interviews and usability tests. In furthering our understanding of user preferences in Spotify’s music recommendation feature, we analyze the practicality of certain design features, such as the Spotify radio ‘Like’ and ‘Hide’ button, as well as the ‘Enhance’ button.
We researched the recommendation systems and models that are currently used in the Spotify platform, like “Collaborative playlists” and “Filter bubbles,” and we evaluated the potential impact they have on the listener’s enjoyment and musical development. These current models overtime can create a drop in user engagement and overall retention for the lack of diversity and accuray. Through interviews and usability tests, we defined how the design of the user interface plays a crucial role in the transparency and customization of algorithmic recommendations.
Listen to Your Heart designed by Dani Lam
Introduction
Spotify is the number one music streaming service in the world, with 489 million users and 205 million subscribers. The platform’s personalized music recommendations and streaming service model has played a large role in restructuring the music industry. Spotify’s unique features include collaborative playlists and the ability to see the music other Spotify users are listening to in real time.
The purpose of this research is to investigate how the current algorithm-driven listening through recommendations is associated with reduced consumption diversity. This research has the potential to expand Spotify’s knowledge of different listening habits and user-types. An expansion in Spotify’s understanding of these pain points will attract new users and ensure user retention over time.
Spotify’s current music recommendation system uses the ‘filter bubble’, which
recommends users music similar to what they are already familiar with and listen to, and makes no attempt to introduce users to new music. And Collaborative filter which recommends music between users that share related preferences. Over time, this filter risks “getting over-specialized recommendations,” in other words recommendations become more niche, which do not allow further analysis of the user changes in behavior and need for discovery. This system could lead to a drop in user engagement and overall retention.
If we can identify the user’s pain points with Spotify’s music recommendation system, we can use this information to build models that enhance user engagement and retention within the platforms. The research questions we set out to answer are:
- What can we do to counteract recommendation bias?
- How can song recommendations grow with a user’s interests and habits over time?
- What motivates users to use the recommendation feature?
- What makes a song recommendation feature successful?
Literature review
User Intents and Satisfaction with Slate Recommendations - Spotify Research
Spotify has conducted research on the significance of user intent and user interactions in relation to Spotify’s ‘Search’ function. Through this research, they identified 8 key intentions a user might have when using the app. They make a case for “ the need for grouping user sessions into intent groups for predicting satisfaction, but also for shared learning across all intents.”
- “User interaction with such systems is often motivated by a specific need or intent, often not explicitly specified by the user, but can nevertheless inform on how the user interacts with, and the extent to which the user is satisfied by the recommendations served. We hypothesize that user interactions are conditional on the specific intent users have when interacting with a recommendation system, and highlight the need for explicitly considering user intent when interpreting interaction signals.”
- “Indeed, the interpretation of signals varies with goals; for example, scrolling can indicate negative experience when the goal is to quickly listen to music now, but can also indicate a positive experience when the goal is to browse the diverse collection of music the system has to offer.”
Modeling Users According to Their Slow and Fast-Moving Interests - Spotify Research
Spotify has conducted research on the topic of creating accurate and diverse models. For this study, researchers categorized user’s listening habits into ‘slow’ and ‘fast’ features. ‘Slow’ features are the consistent listening preferences established over time, while ‘fast’ features are the newer listening preferences which are oftentimes subject to change.
Fairness in Question: Do Music Recommendation Algorithms Value Diversity?
Spotify recommender systems are biased and unfair to specific groups of both users and artists from datasets used to train the algorithms for consumptions to algorithmic bias, which “might recommend popular content even to users who are not interested in popular content.” It also goes over the details of how Spotify uses feedback loops and filter bubbles in their recommendation feature.
How to get the most out of your YouTube Music subscription.
YouTube Music uses the search and watch, and location history from your Google and YouTube data to train the algorithm for better recommendations. Youtube Music gives the users the opportunity to Pause watch history, Pause search history, Pause activity-based recommendations and clear recommendations to reset the recommendations to some degree. Youtube Music gives the users access to clear the details settings on your favorite artists when you first open the app.
Methodology
On average, interview sessions ranged between 15 to 30 minutes. A total of 13 participants were recruited, mainly Spotify users with an age range from 20 to 37 years old. The data collected from the Surveys received responses from participants across the globe. The survey had 23 responses.
Phase I
Quantitative Method
- Surveys: We conducted a survey over the last two weeks focused on Spotify’s impacts on the user’s listening experience and user’s interest in diversified music recommendations within the platform and other music streaming services.
Phase II
Qualitative Method
- User Interviews: We conducted user interviews focused on understanding user sentiments.
- Usability Testing: We conducted user observations to see how participants found new music and how they categorize music within the platform.
Phase III
Complexity Theory
Digital Engagement Model - Exploring Users' compromise to Spotify
- Combining findings from Phase 1 and Phase 2 to understand why user’s stay with Spotify despite difficulties
- Step 1) Tool Use: Examines the relationship between the individual and the Spotify platform. This involved examining how intensely the individual used Spotify's platform and how they embedded it in their life.
- Step 2) Motivation: Look at what drives the individual to engage with Spotify platform. Temporary benefits of pleasures and their consequences.
Phase IV
Competitive Analysis
- Youtube Music, a subsidiary of Google, has access to exclusive tracks, live performances, curated playlists based on liked songs in YouTube Music. Youtube Music combined with Youtube Premium has 50 million subscribers and has only been in the market for over five years now. This analysis method aims to examine Youtube Music users' behaviors and expectations with these features to identify areas of improvement in the Spotify platform.
Methodology Limitations
- For complexity theory, self-reported data may not be an accurate representation of the user's thoughts.
- User Interviews cannot explain every user behavior.
- In-person surveys and interviews have human bias (participant’s responses can be affected by the interviewers presence)
Ethics
- We protected participants' data and privacy. We conducted ethical User Research and disclosed all necessary information to participants. We asked for their consent before recording interviews, and explained how the data will be used.
Results
Phase I: Quantitative Method: Survey
Applied UX Research: Spotify Survey Responses
- 30 % of participants use other music streaming services like Youtube and Youtube Music
Phase II: Qualitative Method
Applied UX Research: Spotify Data Compilation
Despite having similar use cases of Spotify, people interact with the platform in various styles. Spotify must be adaptable to the needs of individual users.
Many of the participants create playlists to listen to music, but that does not mean their listening habits are the same.
-
“I'm big on sequencing and how a song transitions. I want to be the one to manually do that. This is my love language, so I want to be very hands on, I don't want that to be automated.”- P #7
-
“I create playlists by adding recommendations at the beginning. I typically only have five songs that fit the vibe I want, so I use the recommendations to see if I can find new songs that I don't know of.” - P #12
They feel they’re accurately “labeled” to a profile that won’t allow them to restart their musical journey
- “Spotify it's not surprising me with new stuff usually surprised me with things that I forgot? Spotify can put me under a stereotype because I'm from South America, I should like pop music from Latin America. So it's like, if I need to find my own ways of getting new music.” -P#3
-
“I’m just curious where they get the feed from. And sometimes I feel like it's not always accurate. Like say I had a good week and listened to a happy song. Next week, it's like sad, like I mean, I'm pulling from memory so I can't really say exactly, but I remember having that like contrasting impression.”-P#2
- “I expect if anything, probably tell me like, why it recommends me that specific music. Because you listened this, we recommend this...”
User Interviews
From the user interviews, we found that users preferred to choose a track directly from their personal library and use the “Radio” to find recommendations that were similar to that track. Users mainly explored through artist’s playlists to find unbiased or “fresh” recommendations. Users also had to type in a specific genre in the search bar to find curated playlists for those genres
Phase III: Complexity Theory
During usability tests, many participants argued that one of the main reasons why they kept using Spotify is to share music among their peers. One participant expressed that they started using other music streaming platforms over Spotify for its failure and bias in the recommendation feature, but that they still used the platform to share music with friends. Overall, participants enjoy using the Spotify interface and appreciate the various ways they can catalog their music. We discovered that user engagement wasn’t affected by promotional music content and that users’ musical discovery could be influenced by other community spaces in other platforms.
Phase IV: Competitive Analysis:
The question ‘What do you use to listen to music’ yielded similar results as the ones from the survey. Many spotify users supplement their listening with time spent on Youtube or Youtube Music, to listen and discover new songs.
Interviews
- Majority of participants feel the need to use YouTube and YouTube Music to find diverse music outside of Spotify.
- “I feel like finding unique or more rare artists is a lot easier on YouTube than Spotify. Spotify kind of recommends the mainstream stuff that's used a lot, because that's how I assume their algorithm works. On YouTube, I feel like I could find a lot more obscure things.” - P #12
- “If something I'm listening to is like only on a record. And if someone uploaded it to YouTube, I would look at that person's channel. And then most likely, they also upload other records from different artists, or they have other playlists, and then that's how I would do it.” - P #4
Additional Literature Review
Youtube has more data of users to curate a playlist of their interest and expose them to diverse selections of music.
How your listening habits affect your YouTube Music experience
- “If you don't like a song that YouTube Music recommended, skip or dislike it. Similarly, you should like songs that you enjoy.”
- “Searching for songs trains the algorithm and teaches it your preferences as well. If you're willing to go out of your way to find and listen to a song, you must like it, right? YouTube Music also uses the search and watch history from your Google and YouTube (the video one) data to provide better recommendations.”
- “Your recommendations and search results are also based on videos that you’ve liked and playlists that you’ve created in YouTube Music and YouTube. You can remove liked videos and edit or delete playlists to influence your recommendations and search results”
Location
- “Privacy and location page, look at the Google Play Music history options. If you used Google Play Music before Google pulled the plug, consider turning on Use Google Play Music history to import your listening habits from the old app and make your recommendations more accurate.”
Search History
- “To clear YouTube Music Recommendations on mobile, tap your profile picture in the upper-right corner to summon your account options, navigate to Settings > Privacy and location and find the Manage watch history, Manage search history, and Manage location history options. Each of these needs to be deleted manually. After opening each of them, find the Delete option and select the period for which you wish to remove your listening history.”
Settings
- “YouTube Music asks for details on your favorite artists when you first open the app (much like Spotify), and this information is the first step in letting the algorithm know what type of music you want to hear.”
- “Don't panic if you didn't select all your favorites. You can return to the Pick some artists you like page by navigating to Settings, scrolling to the bottom of the page, and tapping Improve your recommendations. You can also update your favorite artists as your taste changes to further refine your recommendations.”
Sources:
How to get the most out of your YouTube Music subscription
Youtube Music Help - Customize Your Music
Discussion
We have an opportunity to restructure the way people find and listen to new music. Users are motivated to use the recommendation feature to discover new music and share their findings with peers. To understand the success of the music recommendation feature, we need to analyze the user’s current interaction with the feature. From our research, we learned that the majority of participants prefer to skip rather than use the ‘Hide’ button when they are evaluating songs on Spotify’s music radio. Therefore, we could argue that users do not find these functions necessary and are unaware of their purpose. We noticed a similar situation with the ‘Enhance’ button (The ‘Enhance’ button is a feature that adds recommended songs into a user's playlist. It is a feature that can be toggled on or off) . With this knowledge, it is clear that the user’s relationship to the music recommendation feature must change to one that is seamless and intuitive. Multiple participants expressed that they are less likely to repeatedly engage with the recommendation feature when they feel that they are being recommended the same songs or artists that they disliked in the past. Participants also felt that they needed to use other resources, outside of the Spotify platform, to find “fresh” music recommendations.
The potential impact of this research is creating a stronger commitment to Spotify within users. Beyond user retention, the impact of this research is opening up Spotify’s potential to be one of the most reliable places, if not the only one where users go to find new music. In the user interviews, participants mentioned music competitors such as Youtube/ Youtube Music, SoundCloud and Tiktok. What’s notable about the mention of Youtube and Tiktok is that both of these platforms are social media sites. Spotify has positioned itself to be both a music streaming service as well as a social media platform with its ‘Friend Activity’ feature and its ‘Collaborative Playlists’. Multiple participants mentioned the importance of social engagement on Spotify, and spoke about how intimate the experience of making a playlist for another person can be. Because music is often a communal experience, Spotify must consider the ways it can strengthen user’s desire for personal connection within the app.
Through the user interviews, we noticed that many participants wanted to understand the metrics behind the machine learning algorithm and how their data was being used in the recommendation features. Furthermore, the participants felt that they were accurately “labeled” to a profile that wouldn’t allow them to restart their musical journey. We can counteract recommendation bias by providing users access to their history of music preferences over time. Since users could have more initiative and critical analysis of their interests and habits over time, this could also be a human-centered redesign, and its transparency could also help build trust between the platform and its users.
When it comes to exposing users to diversified music recommendations, many participants clarified that they are interested in having more control over their exploration journey. How might we help them transition to new music at their own pace? We’re proposing two features. The first feature, the Mix Match, lets users input tags within categories that include genre, location, and year. And the second feature consists of a “Jukebox” that includes a catalog of human-made playlists with assorted musical tracks. This could also be a human-centered redesign since users are able to expand their emotional palettes and create a sense of belonging or new social connections opportunities.
Personalization
Spotify’s interface should make very few assumptions about how users intend to use the application, and provide users with flexible user interfaces that aid specific issues users may face. Despite having similar use cases of Spotify, people interact with the platform in various styles. Based on the user interviews we conducted, it is clear that users want more flexibility from Spotify. Spotify must be adaptable to the needs of individual users.
Many of the participants create playlists to listen to music, but that does not mean their listening habits are the same. Some participants create many short playlists for specific purposes, while other participants make really one long playlist that they use daily.
Algorithmic Transparency
Most people want to understand the metrics behind the machine learning algorithm. They feel they’re accurately “labeled” to a profile that won’t allow them to restart their musical journey
Competitive Analysis
The majority of participants feel the need to use YouTube and YouTube Music to find diverse music outside of Spotify. Spotify offers a vast library of music, it may not have every single song or artist that a user may be interested in. Additionally, some artists may choose not to release their music on Spotify, or their music may only be available on YouTube.
Conclusion and Reflection
Key Takeaways
- Users want to comprehend and trust the results and output created by machine learning algorithms.
- Users want to have more control over their exploration journey. How might we help them transition to new music at their own pace?
In hindsight, we may have benefited from a more holistic view of Spotify as a service. If we had started the research project with the understanding that Spotify acts as both a music streaming service and social media site, maybe the direction our research went would have been different.
We could also further attempt to get more recent information about Spotify’s recommendation algorithms, like the Natural Language Processing, or NLP which analyze popular topics on social media, and use the top keywords with descriptive adjectives to make similar recommendations to that class. We could have revised the ethics behind the 2021 Spotify speech recognition technology patent, which captures live audio to more accurately “identify the listener’s mood” and analyze its current use, to see if it regulates or enhances emotions of the users.
Another input tag that we could have dug into for the Mix Match proposal were gender groups tags. We didn't consider this topic of conversation during our interviews, but based on our research it could be another issue in the recommendation feature. We could have analyzed if this was affecting our participants and how the database could be trained to get an equal number of gender groups in the recommendation playlists to fill in the gaps between both groups.
Limitations
Due to the time constraints of this project, we conducted our research within a very short time limit. We were only able to interview 13 subjects, and if we were to iterate on this project, we would have more subjects. Also, due to limited resources as students, the age range of our participants is specifically looking at the sentiments of users from the ages 20 to 37. To have a more comprehensive understanding of Spotify users and their music recommendations, we would expand our age range to include a wider spectrum of users. Many of the participants were associated with New York University in some capacity, so that creates limitations within our data pool. Except for one participant, all of the participants were people we knew. It’s likely our data would be different if our participants were strangers.
Future Research
We recognize that user interviews are attitudinal, not behavioral. To have a deeper understanding of how Spotify listeners engage with the music recommendation feature, in the future we would like to conduct ethnographic field studies and diary studies that track the listening habits of Spotify users while they conduct daily tasks (grocery shopping, going to the gym, walking to work) to define how users might use the recommendation feature naturally. For example, one person might listen to a 10 song playlist while running errands, and we would monitor the person’s decisions when listening to recommended songs that come on when that playlist ends (Do they skip, ‘favorite’, or ‘hide’ songs?) The diary studies could be conducted with the same participants, and allow the participant to explain, from memory, their thoughts and feelings about using the recommendation features on Spotify during the ethnographic field study.
Relevant Documents
From manual to assisted playlist creation: a survey
How to get the most out of your YouTube Music subscription.
Inside YouTube’s plan to win the music-streaming wars - Protocol
4 Tips to Improve YouTube Music Recommendations — Make 'Discover' & 'Your Mix' So Much Better « Smartphones
Customize your music - YouTube Music Help.
Assessing the diversity-accuracy dilemma of Spotify’s recommendation systems
Fairness in Question: Do Music Recommendation Algorithms Value Diversity?
Spotify is first music streaming service to surpass 200M paid subscribers
Mint.com Usability Study Report
A UX/UI Case Study on Spotify
Spotify – How data is used to enhance your listening experience.