Artificial intelligence makes image search smarter: Bavarian State Library develops AI-based image similarity search

The Bavarian State Library has just launched a new image-based similarity search for bavarikon, Bavaria's digital treasure trove. The development from the in-house Munich Digitisation Centre (MDZ) searches over three million image segments from photos, graphics, paintings and illustrations recognised by machine learning in the portal.

At www.bavarikon.de/imagesimilarity, those interested in culture can now search for motifs quickly and efficiently or enjoy drifting from image to image.

For the new image-based similarity search, the MDZ has trained a system based on artificial intelligence (AI) using selected examples. The AI system can transfer this acquired "knowledge" to new, previously unseen examples. In the case of bavarikon, the system was taught to recognise and classify individual images and image sections within the extensive stock of images and books. Relevant image segments are recognised by the trained neural network - an "artificial brain" - and categorised based on the training, e.g. as an illustration, miniature, coat of arms or person.

The Bavarian State Library has created a new high-performance technical infrastructure for training the AI. It is able to carry out the computationally intensive processes for the more than 3 million images available in bavarikon in an adequate period of time.

Image-based research becomes an inspiring experience with this new tool, as it complements the classic text search and goes far beyond it. There are three search options to choose from: searching for similar images to an image you have uploaded yourself, searching for similar images to an existing bavarikon object and various curated search examples that lead into the depths of the bavarikon content.

By applying various filters it is possibl to refine the search results. In this way, images or details of images can also be found, especially if it might not be possible to do so at first glance in the more than 450,000 digital cultural objects in bavarikon.

The next step will be really exciting: the Bavarian State Library will soon be unleashing the new image-based similarity search on its four million digital collections. These contain more than 60 million (!) image elements, which will then be searched for similarities in a matter of seconds.

Director General Dr Klaus Ceynowa: "The newly developed AI-based bavarikon image similarity search is unrivalled among the online tools of German cultural institutions and is therefore unique for a regional portal in Germany. At the same time, it is a good practical example of how AI can be put to good use in the cultural sector in particular."

You are very welcome to try our new search tool: https://www.bavarikon.de/imagesimilarity