April 25, 2018
The Artificial Intelligence revolution is underway and how we as a society are interacting with images is changing drastically in response. Computer vision technology is increasingly becoming the standard vehicle through which we navigate the visual world on our devices. From the identification of persons in photographs in social media or shopping items as we search online, to the vision technology that enables self-driving cars or some medical surgeries, image recognition and retrieval tools have become very powerful, silently guiding the visual culture of our digital worlds. While developments in this area have to date been led by academics in computer science and large companies that utilize machine-learning techniques (a subset of AI) to automatically predict and direct visual search results, often with the aim of consumer consumption, it is time that image recognition and retrieval technology be recognized as having far too great a universal value and social impact to not question how its assumptive curatorial processes are being directed. And to whom should this responsibility be bestowed? To all those who want to partake in a visual culture dedicated to the accurate and responsible representation of knowledge through images.
Unfortunately, the humanities, the most seeming ally given the curatorial quandary posed by the machine-learned image landscape of the digital world, have an often resistant approach to alliances with STEM; such a partnership has been interpreted more as an arranged marriage forced by budget-starved administrators than as the growth of a natural humanistic pursuit. The external push to bring a science agenda into the humanities has been justifiably met with much skepticism, but at the same time, let’s not forget the long and fruitful partnership between the arts and sciences that has blossomed for centuries, organically, through a shared spirit of curiosity for our place in the natural world. At this moment we are presented with an unprecedented opportunity for the arts to take a leading role in the development of the most cutting-edge AI-based technologies that are building our digital, visual worlds either way. Shouldn’t the humanities want to be a part of this endeavor that is shaping our very societal identity and aesthetic interactions with images?
Instead, most computer vision scientists are exploring aesthetic perception as terra incognita when one of the most basic questions dealt with in the field of art history is What does it mean to see? While this philosophical inquiry is arguably at the origin of all undertakings in the visual arts, its meaning has taken on radically different signification in the world of big data and image analytics empowered with machine learning. For art historians, there is a community-based responsibility not only to help curate the digital landscape of images, but also to harness and direct the use of these technologies.
Although the digital humanities movement has fostered much interest in the use and interpretation of digital tools and materials in the arts, its failure to adequately recognize the significance of the AI revolution suggests that it may be a poor conduit for the advancement of research in machine learning and the humanities. When I conducted an academic survey with my collaborator in computer science on the perceptions held by art historians and computer scientists on the uses of computer vision technology in the arts four years ago, it was a positive sign that the majority of participants thought well of the potential collaborations between the fields. It was disappointing, however, to learn that the developments in image analysis in particular were little known outside of the sciences. Not surprisingly, when asked whether the implementation of AI-based technology in the humanities would signal the beginning of a positive paradigm shift in academia, the art historians overwhelmingly responded that it would not, and provided ample comments indicating their distress about this type of technological import.
Four years later, doubts endure in the humanities while computer vision technology and applied machine learning is disrupting all the major industries. Unfortunately, art history has yet to fully benefit from our new abilities to make smart searches for visual information through machine-empowered sight. In response to this notable incongruity, a major symposium organized by The Frick Collection and Art Reference Library has addressed the advantages of employing image recognition and retrieval tools for the arts. Entitled “Searching Through Seeing: Optimizing Computer Vision Technology for the Arts,” the symposium took place at the museum April 12-13, 2018 and brought together art historians, curators, computer scientists, technologists, business leaders, and the investment community to address the use and construction of new tools for the study of art.
In the age of AI, we are quickly adapting to the investigative possibilities of information in non-traditional forms. The Frick symposium has anticipated a societal turn toward image-based search queries and the tremendous research potential of this technology in all sectors, if developed responsibly to accurately represent knowledge. It also reopens the philosophical debates on aesthetic perception and asks what it means to visually behold an image in a technocratic world that is extending our conception of what it means to see. If the future of art history relies on its partnership with science, on these grounds it could perhaps be fruitfully consecrated.
Francesco del Cossa (ca. 1436–1477/78), Saint Lucy, ca. 1473. Tempera on poplar panel, 30 3/8 x 22 1/16 in. National Gallery of Art, Washington D.C.