Transcribing and annotating wartime letters. How can it help us and what is lost in the process?

Tess Schijvenaars, a Research Master’s student at Utrecht University, prepared the ground last year by investigating how we can datafy historical documents such as these, what is lost in the process, and how this simultaneously opens up a wealth of new research possibilities. In this blog post, she writes about her research and findings.

As part of the Research Seminar course, students taking the Research Master’s in History at Utrecht University are assigned to several research projects, to give them insight into how historical research is done in practice. I joined the NIOD project and got working on a collection of 26 wartime letters, all addressed to members of the Kleinrock family. This Jewish family lived in Vienna, Austria, during the Second World War. Its members were arrested by the Nazis and transported to various labour and concentration camps in Europe. None of them survived the war.

How can this tragic story, recounted in the 26 letters, be translated into data? Can ‘datafied’ versions of these letters add anything to historical research, and are certain elements lost in the process? These are the questions that I set out to address.

Datafication is the conversion of unstructured text, such as letters, into structured data. We add structure, for example, by noting (annotating) information in the letter about the writer, or the date and place of writing. This makes the digitised archival documents suitable for computer-based data analysis, and makes it easier for researchers to find them and search them. The two steps that I took to datafy the letters were to transcribe them and then to annotate them with structural and textual metadata, using Transkribus.

Transkribus is software developed by the University of Innsbruck, which uses artificial intelligence and handwritten text recognition to read and automatically transcribe historical documents (written or typed).

Figure at the left is a transcription of a war letter in Transkribus. At the right is structural metadata of a war letter in Transkribus.

In addition to automatic transcription, Transkribus can also be used to annotate historical documents by adding structural and textual metadata. Metadata is often defined as ‘data about data’. It describes the properties of a particular document. War letters, for example, frequently include a date, copy and salutation, and are divided up into a number of paragraphs; they follow and describe a certain structure. This is what we call structural metadata.

Certain terms and words in the text itself can also be seen as metadata. Textual metadata describe the semantic properties of a text. Within the context of the war letters project, for example, personal names, place names and surnames are important terms that are often mentioned in the letters. Textual metadata can also include underlined or italicised words. These are all features of the text that matter, and that can be annotated.

What can we gain from the datafication of war letters, in particular the Kleinrock letter collection? Which elements are lost in the process, and what hidden structures are revealed?

First of all, I made a model of the structural and textual annotations, which I used for all 26 war letters. This ‘metamodel’ could potentially function as a controlled vocabulary, and could be applied to other collections of war letters in order to make them more accessible and easier to describe.

In addition, a datafied version of the Kleinrock war letters could form the starting point for further research. Using annotations and metadata tags, elements such as place names, dates and people are recorded in Excel and XML files, for example. These files could be used to create visualisations of the paths taken by the various members of the Kleinrock family during the war. Which member of the Kleinrock family was where and when? Textual tags such as personal names and the names of organisations could also be used to link the story of the Kleinrock family to other war sources and collections, making it part of a larger overarching history.

However, many elements are also lost during the datafication process. For example, it is not possible to reconstruct the full text and context of the letters using the meta data alone. Elements of the syntax, tone and emotional content of the text can potentially be lost. Also consider the material on which a letter is written, the texture of the paper and the smell of the document; all of these are missing from a datafied letter.

The datafication of historical documents has great potential, but it is not the holy grail. Quantitative research methods can reveal hidden structures and links, but qualitative methods such as discourse analysis and close reading then give meaning to these structures and links. One need not exclude the other; the richest historical interpretations emerge from a combination of the two.

Tess Schijvenaars

Research student Universiteit Utrecht