GRAMMY Award for Best New AI: Audio Analysis

This year's Grammy Awards were a showcase of the latest and greatest in music and audio from Billie Eilish to Abba. 

Music presents itself to various applications within AI understanding the context and sentiment of audio. This is how Spotify creates custom playlists based on your moods.  AI is also used for automatic speech recognition (ASR), which powers Siri, Alexa, and Google Home. During the GRAMMYs, AI was used in a multitude of ways. In fact, some of GRAMMYs winners may have been aided in popularity by algorithms like Spotify. Let's dive into how AI analyzes sound and how IBM used AI to improve the GRAMMY experience.

How does AI analyze music & Sound ?

There are a few different ways. The most common is to use a neural network that has been trained on a large dataset of music for audio data analysis. This type of AI can identify features in audio files such as tempo, pitch, and timbre. It can also identify the genre of a song and the mood it evokes.

This is the type of AI that Spotify uses along with its recommendation algorithms which recognize and recommend music similar to yours. It likely used signals such as average listen duration, number of "likes," along side the song's relation to your personal algorithm to drop a new song in the shuffle.

AI is also used for sound recognition. This is how your smartphone knows to turn off when you're in a movie theater. Sound recognition can also be used for contextual advertising.

How was AI Used for GRAMMY Insights

IBM was the sponsor of the 64th GRAMMY awards. In partnership with the GRAMMYs, IBM built GRAMMY Insights in 2020, which was designed to drive engagement by utilizing natural language processing.

This is the first time that IBM has used its Watson technology at the GRAMMYs.

Watson analyzed the red carpet video and audio in real-time and matched it with data scanned from news articles and blogs to present additional facts about the artist or about the conversation topic.

For example, if you were watching a red carpet interview with Billie Eilish, Watson would be able to present additional information about her such as her age, hometown, and recent awards.

This is just one way that IBM used AI to improve the GRAMMY experience. It will be interesting to see how else AI is used in the future to improve our experience with music and audio.

How Contextual Algorithms are using Audio

As we've seen, AI is being used in a number of different ways surrounding sound. But one way that you may not have thought of is how contextual advertising can use sound recognition as a signal for ad placement and sentiment.

Similar to GRAMMY Insights, CatapultX is using sound recognition to identify the context of what's happening in the content or video. At its lowest level, if the dialogue in a video is regarding the Superbowl, the algorithm may choose to show you an ad for Football swag.

But it can also identify things like whether someone is happy, sad, or even angry in the video. In combination with visual recognition to gain an understanding of the video, this allows for a more targeted ad experience. For example, if you're watching a cooking show and the chef seems to be struggling, an ad for a new kitchen gadget that makes cooking easier may be played.

On-Stream ads, served contextually, can be directly integrated into videos to provide a seamless experience. Contextual AI-powered advertising is the next Olivia Rodrigo for video publishers and advertisers.

Let us know who your favorite artist this year is, and if you're interested in AI-powered contextual advertising or monetization solutions, drop us a line.

‍