Whilst the photographic image has become a ubiquitous feature of digital culture, it has undergone far-reaching transformations through computational systems. This has meant that understandings of the photograph in culture are radically different from understandings of the photographic image as data.
In partnership with the Centre for the Study of the Networked Image, London South Bank University, we appointed Nicolas Malevé as a Doctoral Researcher in 2015 to further unpack how machines see the world, exploring the politics behind processes of image annotation, and identifying the challenges it poses for photographic culture. Nicolas Malevé’s thesis “Algorithms of Vision: Human and machine learning in computational visual culture” will be published in 2021, and draws upon an early experiment conducted in 2007 at Caltech by Fei Fei Li, initiator of ImageNet, one of the most popular visual datasets.
It offers a map of the entanglement of computer vision and photography and documents, Variations on a Glance, a series of re-enactments Nicolas staged in the Gallery and elsewhere, based on the Caltech experiment in order to generate new questions concerning the politics, labour and temporality of decoding images for machines.
Nicolas Malevé's Research Abstract
Algorithms of Vision: Human and machine learning in computational visual culture
Current computer vision algorithms largely depend on the availability of images labelled by human annotators at very high speed. The mode of production of these annotations strongly resonates with an early experiment conducted in 2007 at Caltech by Fei Fei Li, initiator of ImageNet, one of the most popular visual datasets. In a laboratory, the subjects were asked to describe photographs shown for a few milliseconds and to filter them through a taxonomy. The Caltech experiment is used, in the thesis, to engage with the photographic elaboration of computer vision: the model of vision, the photographic alignments and the micro-temporal rhythm that subtend the modes of production of labelled data and the labour behind it.
The written and practice components of the submission elaborate a novel method and document the path towards it. The method has developed in the context of practice-led research in collaboration with The Photographers' Gallery and crystallised into a project, Variations on a Glance, a series of re-enactments based on the Caltech experiment. The original experimental protocol is submitted to several variations, called re-experiments, exploring its potential to produce a time-critical model of vision and collective visual interpretations. The experimental protocol is re-designed iteratively to explore specific configurations of micro-temporal vision and different configurations of collectives of human and non-human participants. The thesis examines the dynamics of these collectives, in particular how they reach consensual interpretation, and how the taxonomic practices of the lab interfere in this process.
The contribution of this research is a mapping of the entanglement of computer vision and photography and a method embedded in practice that does not attempt to resolve the differences and tensions between photography and computer vision but provides a device to explore the texture of their relation. The research complements and complicates the recent critiques related to bias and discrimination in machine learning and the exploitative work conditions it relies on. Finally it offers to the photographic institution and its public a mode of intervention into the making of computer vision.
Exhibiting ImageNet (excerpt)
Selected Projects, Talks & Publications
Horizontal Humans (2 Oct 2015-10 Jan 2016), A commissioned Media Wall project by ScanLAB Projects
Nicolas Maleve (2016) Contours of the discontinuous in: Loose Associations 2.2
Nicolas Maleve (2016): “The cat sits on the bed”, Pedagogies of vision in human and machine learning
Search by Image, live (Lena/Fabio) by Sebastian Schmeig (7 October 2016 - 29 January 2017. A commissioned Media Wall Project
Nicolas Maleve & Katrina Sluis: Curating Machines lecture series.
Katrina Sluis (2017) Machine Literacies in the Photographic Museum, Ways of Machine Seeing Symposium, Cambridge University, CoDE and Cambridge Big Data, 26 – 28 Jun
Geoff Cox (2016) Ways of Machine Seeing
Decision Space, an online commission by Sebastian Schmeig (7 Oct 2016 - 5 Feb 2017) For further documentation, see: http://decision-space.com
Robot Vision Geekender (2016), featuring work by Katriona Beales, Terence Broad, Luba Elliott, Lynn Hershman Leeson, Ryo Ikeshiro, Carlos Molinero, 3D Scanbot, Foxall Studio, South Bank Collective & Superflux
Nicolas Maleve & Sebastian Schmieg (2017) The politics of image search - A conversation with Sebastian Schmieg in: Unthinking Photography.
Nicolas Maleve (2019): An introduction to image datasets in: Unthinking Photography
Gaia Tedone (2019) From Spectacle to Extraction. And All Over Again in: Unthinking Photography
Data/Set/Match programme, which includes:
- Exhibiting ImageNet, Media Wall Project, 01 July – 13 September 2019
- What does the Dataset Want?, A symposium on 14 September 2019 with speakers including Zach Blas, Heather Dewey-Hagborg, Nicolas Maleve, Anna Ridler, Daniel Rubinstein, Katrina Sluis
- New commissions on the Media Wall from Heather Dewey-Hagborg (How Do You See Me?), Mimi Onuoha (The Future Is Here!), Anna Ridler (Laws of Ordered Form), Philipp Schmitt, xtine burrough & Sabrina Starnaman and Everest Pipkin
- Working with Datasets, an online round table 15 October 2019
A Cat, A Dog, A Microwave... Cultural Practices and Politics of Image Datasets (2023, edited by Nicolas Malevé and Ioanna Zouli) critically investigates the development and impact of visual datasets from the perspective of machine learning, while exploring their artistic possibilities in the contemporary image culture.
