In 2018, Apple closed its $400 million acquisition of music recognition app Shazam. It is now bringing Shazam’s audio recognition capabilities to app developers in the form of the new ShazamKit. The new framework will allow developers, including those on Apple and Android platforms, to create apps that can identify music from Shazam’s massive database of songs, or even from their own custom catalog of pre-recorded audio.
Many consumers are already familiar with the Shazam mobile app, which allows you to press a button to identify what song you are listening to and then perform other actions, such as viewing lyrics, adding the song to a playlist and exploring music trends. Shazam, which first launched in 2008, was already one of the oldest apps in the App Store when Apple acquired it.
The company is putting Shazam to better use than just as a music identification utility. With the new ShazamKit, developers will now be able to leverage audio recognition capabilities to create their own app experiences.
The new framework consists of three parts: Shazam catalog recognition, which allows developers to add song recognition to their apps; custom catalog recognition, which performs an on-device comparison with arbitrary audio; and library management.
Shazam catalog recognition is what you probably think of as the current Shazam experience. The technology can recognize the song being played in the environment and then look up its metadata, such as title and artist. The ShazamKit API will also be able to return other metadata such as genre or album art, for example, and it can identify where in the audio the match occurred.
When matching music, Shazam does not actually match the audio itself. Instead, it creates a representation of it, called a signature, and matches it. This method greatly reduces the amount of data that must be sent over the network. Signatures also cannot be used to reconstruct the original audio, which protects the user’s privacy.
The Shazam catalog comprises millions of songs and is hosted in the cloud and maintained by Apple. It is regularly updated with new tracks as they become available. When a customer uses a developer’s third-party app for music recognition through ShazamKit, it may be desired to save the song to their Shazam library. This is found in the Shazam app, if the user has it installed, or can be accessed by long pressing the Music Recognition Control Center module. The library is also synchronized between devices.
Apple suggests that apps inform their users that recognized songs will be saved in this library, as no special permission is required to write to the library.
Meanwhile, ShazamKit’s custom catalog recognition feature could be used to create synchronized activities or other second-screen experiences in apps by recognizing the developer’s audio, not Shazam’s music catalog.
This could enable educational apps where students follow a video lesson, where a portion of the audio from the lesson could drive an activity to begin in the student’s companion app. It could also be used to enable mobile shopping experiences that popped up while watching their favorite TV show.
ShazamKit is updated in beta on iOS 15.0+, macOS 12.0+, Mac Catalyst 15.0+, tvOS 15.0+ and watchOS 8.0+. On Android, ShazamKit comes in the form of an Android Archive (AAR) file and also supports custom music and audio.