Motivation
This user story describes some of the current challenges facing digital comics research, a highly interdisciplinary community that includes researchers from media, literary and cultural studies, art history, computer and cognitive science. Over the last ten years, computational methods have become increasingly important for comics research, which in itself is a young but rapidly growing and institutionalizing field of research. In addition to specific annotation languages and software, several digital corpora have been constructed for Japanese manga, serial American comics books and English-speaking graphic novels. However, most of these corpora, either in part or completely, feature comics that are protected by copyright. Indeed, given the recent history of the medium, which only comes into its own in the late nineteenth and matures throughout the twentieth century, any corpus-driven history or systematic description of comics will have to find a way to share corpora of copyright-protected works. Without such solutions, neither can research results be made available in transparent fashion and reproduced, nor can research data be shared and reused. A particular challenge lies in the multimodal quality of comics, almost all of which combine images and text. Solutions for the legal sharing of copyright-protected text have been proposed elsewhere, making use of token-level annotations and word frequencies. These will be of central importance for digital comics research but can only be fully effective, if similar technical solutions are implemented for the images with which comics text is closely entwined.
Objectives
What are the specific elements that could contribute to solving the challenges I just described? The most basic concerns an annotation standard for comics that includes a formal description of its baseline visual components. While a similar proposal, known as „Comic Book Markup Language“ and authored by John Walsh (Indiana), exists for comics text, an annotation standard for comics image needs to bring together scholars who may use different disciplinary terms to describe visual objects and even basic graphic elements, such as the drawn borders that often separate individual visual scenes within a page. Spatial coordinates can be used for the description of graphic layout, including the placement of text. But how can visual scenes, including faces, objects, color, drawn lines and artistic techniques such as shading and crosshatching be described with the help of visual features that are rich enough in information to be of use to scholars from different disciplines but not subject to legal restrictions under copyright law? Machine-learning models trained on specific corpora might offer a potential pathway, but only if the features shared by researchers do not enable the reconstruction of the copyrighted material.
As a consequence, detailed and continuing legal advice and approval of the technical solutions would seem to be a necessary basis for progress in this area. While an informal group of scholars made first steps towards discussing an annotation standard for comics at a workshop in 2018, further work on this subject is urgently needed, ideally in coordination with the TEI community. Close cooperation with experts in computer vision will also be needed to define the visual features that can further uptake by a wider community. Sustainable implementation of data sharing demands input from the area of research data management.