Description

imgs.ai is multimodal, semantic search engine for media theorists and art historians indexing circa 2,5 million images from datasets of art-historical value.

Background

As Leonardo Impett and Fabian Offert argue in their paper “There is a Digital Art History”, the advent of multimodal embedding networks ushers in a new era of digital art history, by “facilitat

\[ing\]

a completely new form of image retrieval … based on para-visual concepts of arbitrary complexity” (192–3). Moreover, the discipline of digital art history appears to be uniquely well-positioned to address the entanglement of model and dataset, or ‘way of seeing’ and object: “in the age of foundation models, we can never quite isolate the neural network from the object of study” (202).

Use

imgs.ai allows researchers to use four embedding models to query four public domain datasets of art-historical significance.

Datasets

  • Rijksmuseum
  • Metropolitan Museum
  • Smithsonian
  • Getty

Models

  • CLIP (OpenAI)
  • VGG19 (Oxford University)
  • poses
  • raw

More datasets and functions can be accessed by requesting an account. The hosted version is restricted to non-commercial use, though the source code is available on Github.

Technical

The deep visual imgs.ai search engine grew out of Fabian Offert’s work at UCSB and was written by him, Peter Bell, and Oleg Harlamov. Over the course of the AI Forensics project, it was significantly rewritten to make use of the highly performant k-NN algorithm courtesy of Spotify’s annoy library.