Risk
Rebecca D. Frank
“A trustworthy digital repository will understand threats to and risks within its systems. Constant monitoring, planning, and maintenance, as well as conscious actions and strategy implementation will be required of repositories to carry out their mission of digital preservation. All of these present an expensive, complex undertaking that depositors, stakeholders, funders, the Designated Community, and other digital repositories will need to rely on in the greater collaborative digital preservation environment that is required to preserve the vast amounts of digital information generated now and into the future.”
Introduction
The preservation of digital information is fundamentally an exercise in risk management. Whether digital information remains accessible long-term depends on many different stakeholders engaging in a coordinated effort to identify, understand, prevent, and respond to potential risks. The work of managing risk in order to preserve and provide access to many types of valuable digital information takes place within the social and organizational contexts of digital repositories, libraries, archives, and museums.
For example, in December of 2021 a software malfunction during a routine backup permanently deleted approximately 77 terabytes of data from the supercomputing center at Kyoto University in Japan. Scholars such as Perrow and Rijpma have theorized that accidents such as this – normal accidents - are inevitable in complex technical environments. Digital repositories, which consist of complex arrangements of people and technology, are a prime example of the sociotechnical systems in which normal accidents can occur.
In cases such as this, there is often a disconnect between the people who choose to take on risk, and those who will be most affected by a disaster. In digital repositories, even when data can be restored from a backup, it can take substantially more time an effort than repository staff expect. For example, in the process of conducting research about risk and disaster planning, I interviewed someone from a large data repository in the United States who decided to run disaster recovery exercises in their organization. One of those exercises was to restore their collections from the tape backups. After a month of work, the IT staff of the repository were eventually able to recover all of the data, using their own equipment and systems. However, given the difficulties that they experienced, the repository leadership remained skeptical about the prospect of data recovery if their organization experienced a disaster event in which the technical infrastructure was lost or damaged in addition to the data itself. An event that would result in the loss of staff members with specialized knowledge or expertise could also limit or prevent a repository’s ability to recover lost data, because much of the work involved relies on tacit knowledge rather than well-documented processes and procedures.
Standards and best practices in digital preservation tend to treat risk as an abstract concept, requiring people in organizations to identify and name risks, and to explain policies and plans that will mitigate those risks. The types of risk that are identified, and the mitigation strategies developed in response to those risks, tend to reflect the perspectives of those in positions of authority and/or power rather than those who will be directly affected. In short, abstract notions of risk prompt ideas about policy and planning that do not always reflect the pragmatic realities of digital preservation work and the ongoing maintenance of digital repositories.
In this chapter, I argue that while it is important to be able to identify and understand potential sources of risk for digital preservation, it is how people behave in response to those risks that will determine the longevity of digital information. People with different backgrounds and experiences interpret and respond to risk information in ways that are influenced by their unique perspectives and lived experience. Because behavior in response to risk information is influenced by social and organizational factors, I argue here and elsewhere that we must regard risk as a social construct.
Risk as a Social Construct
Classical definitions of risk consist of two elements in regard to an adverse event: the likelihood of that event, and the magnitude of its consequences. In other words, they ask how likely it is that a disaster will happen, and how bad things will be if it does. Many disciplines, including digital preservation and computer science, which both contribute to the long-term preservation of digital content, assume that knowable values can be assigned to these elements in order to assign a calculable value to the concept of risk.
Risk has a long history that is linked to gambling, financial markets, and economic endeavors that depend upon natural phenomena (e.g., weather, maritime conditions, etc.). One of the key commonalities across all these areas is uncertainty about the outcome of events, when factors that cannot typically be substantially controlled or influenced by people. Over time understandings of risk shifted from something of concern to specific groups of people whose work or living conditions involved exposure to risk, to a view of risk as a condition of contemporary life, as exemplified by Beck’s Risk Society.
In line with the classical definition, ISO 31000, a standard for Risk Management, describes risk as the effect of uncertainty on objectives, and specifies that risk consists of a source, an event, and the consequences, balanced with the likelihood of that event. The view of risk put forth in this standard treats risk as calculable. ISO 31000 explains that risk management is a process for organizations to address risks. Underlying this explanation is an assumption that different people within organizations will behave predictably in response to the same information – that they will be rational actors. This assumption is problematic. Indeed, “rational actor theories may be well-suited to describe individual actions under uncertainties but fail to provide satisfactory explanations for collective risk actions or decisions.”
Despite the common assumption that people will behave in predictable ways when presented with risk information, researchers across many disciplines have found that risks are socially constructed. That is, people are not perfectly rational actors with regard to risks, but rather “risks are created and selected by human actors.” Indeed, Slovic argued in 1987 that “the concept ‘risk’ means different things to different people.” Theories of risk perception hold that different people interpret the probability and adverse consequences of events in different ways, and that these differing understandings are the result of social, organizational, and/or political factors. Approaches to managing risk which rely on the notion of a rational actor “[presume] a world that does not exist.
Rather, there are several factors that have been found to influence the construction of risk in digital preservation. Published in a 2020 article in the Journal of the Association for Information Science and Technology, the Model for the Social Construction of Risk in Digital Preservation that I developed includes eight factors: communication, complexity, expertise, organizations, political culture, trust, uncertainty, and vulnerability.
Communication: Perceptions of risk vary depending on the way in which information about those risks is communicated, including the source, method, channel, and means of communication. These elements can either amplify or attenuate perceptions of risk for different individuals and groups.
Complexity: High levels of complexity can make identification difficult with regard to hazards, probabilities, and consequences. Complexity in systems can also lead to unexpected interactions between component parts, often leading to increased levels of risk.
Expertise: Both expertise and lack of expertise can influence perceptions of risk. Experts may have particular knowledge that allows them to understand risk in a particular area, but they have been found to have a narrow focus based on their specialized knowledge, which can influence their perception of risk. Individuals who lack expertise in a particular area may not have the same nuanced understanding of particular areas that experts do, but they have been found to have a greater sense of the broad social context within which they are operating.
Organizations: Organizations both produce and manage risk, and perceptions of risk vary for people depending on their position within an organization. Risk assessment and management activities take place within the context of organizations, and are therefore influenced by the organizations themselves as well as the roles of the individuals within the organizations who participate in those activities.
Political Culture: National context influences how risks are defined. Perceptions of risk are shaped not only by the political culture within which individuals exist, but also by their place or role within that culture. These factors can elevate or reduce perceptions of risk depending on the position of an individual within the culture. Decisions about how to manage and respond to risks are shaped by political culture as well.
Trust: Organizations and processes that involve cooperation by people and groups with different types of knowledge and expertise require trust among those actors. Perceptions of risk can vary depending on the amount of trust that these individuals and groups have for one another.
Uncertainty: In many situations it can be difficult to determine and understand risk and its components (hazard, probability, consequences). People and groups operating under conditions of uncertainty may perceive risks differently depending on their level of uncertainty.
Vulnerability: Risk exposure, or vulnerability, influences perceptions of risk. People and groups who are able to limit their risk exposure may have different perceptions about risk than those who lack the ability to manage their exposure to risks. Greater vulnerability has been shown to increase perceptions of risk, while privilege and the ability to limit or select risk exposure has been shown to decrease perceptions of the severity of risks.
A social constructionist approach to risk, which accepts that social factors such as those listed above influence how people perceive and construct their understanding of risk, embraces the idea that risk will have different meanings for different people. This means that the same set of information can have multiple meanings as it is interpreted by people with different characteristics and experiences. Baxter argues that this social constructionist stance regarding multiple interpretations of risk information is not a statement about the absolute legitimacy of all interpretations, but rather that effective risk reduction strategies can only be created once we consider how people construct their own understandings of risk.
In the context of digital preservation, this means that we shouldn’t immediately accept every possible idea or interpretation of potential risk, but rather that it is important to understand where all of the repository stakeholders are coming from in order to create risk mitigation strategies that will be successful. I agree with this perspective and argue in my own research that in digital preservation a social constructionist view of risk highlights the shortcomings of a classical definition of risk because practical digital preservation outcomes depend not just on identifying threats and calculating their probability, but also on the actions that people take in response to their perceptions of those threats.
Risk in Digital Preservation
Despite the prominence of the word risk in literature about digital preservation, very few sources provide a definition of this term. Those that do tend to rely on a classical definition. For example, in research proposing a risk management-based approach to the design and assessment of digital preservation environments, Barateiro et al. state that “risk is defined as the combination of the probability of an event and its consequences.” A risk management solution that assumes everyone will perceive the same risks and will respond to them in the same way fails to consider the fact that there are many factors which can influence perceptions of risk which, in turn, can influence the ways in which they respond to the risks that they perceive.
A common thread across risk-related scholarship in digital preservation is the judgment of experts. I have argued above that an approach that considers risk as a social construct must take into account the fact that different people will perceive risk differently based on a number of social factors. In digital preservation specifically, there is a strong tendency to assume that the people responsible for carrying out the work of risk management, mitigation, or response and recovery will trust and follow expert guidance. This runs counter to the findings of scholars such as Wynne, whose research found that the uptake of expert advice depends in large part on social relationships between what he calls experts and laypeople.
Wynne found that people construct their understandings of risk in the context of their social relationships with the people and institutions providing risk information – relationships which are often marked by distrust or mistrust. He also found that people outside of formal scientific institutions (i.e., laypeople) possess considerable expertise and reflexivitiy regarding risk information, which is often unacknowledged or dismissed by those considered to be experts. I take Wynne's findings to mean that those considered to be laypeople are in fact also experts, but whose knowledge and experience exist outside of formal institutions.
This is relevant in the context of digital preservation, because the work of preservation involves many different types of actors with different types and levels of expertise. Approaches that focus on the development of recommendations or guidance by experts fail to consider the multiple perspectives and different types of expertise represented in the community of people who are engaged in the work of digital preservation. It follows, then, that those approaches will fall short because they are based on the assumption that everyone will understand risk in the same way, regardless of their expertise or background.
For example, some of the scholarship about risk in digital preservation has focused on creating taxonomies of risk in order to guide organizations through risk assessment activities. Two prominent examples of this are the SPOT Model, and Saffady’s Taxonomy of Risk. Vermaaten, Lavoie, and Caplan’s Simple Property‐Oriented Threat Model for Risk Assessment (SPOT Model) defines a set of properties for successful digital preservation (availability, identity, persistence, renderability, understandability, and authenticity) and identifies risks within each of those properties. Saffady’s Taxonomy of Risk identifies 24 threats, grouped into five categories (creation and collection of information, loss of information, retention of information, retrieval and disclosure of information, and ownership of information), with descriptions of vulnerabilities and risk response recommendations for each. Both of these examples reflect the classical view of risk. They assume that people will understand and agree on a course of action in response to the risks that are identified. Saffady in particular includes recommendations for risk mitigation activities. However, we know that the various people who are engaged in the work of digital preservation are not likely to share the same perception of risk, and so models that rely on a shared or common understanding of risk among actors are likely to fall short of their goals.
Some risks associated with digital preservation are known and understood. For example, file format obsolescence, media deterioration, storage failures, and economic failures. These issues have been the focus of scholarship and the digital preservation community has created guidelines and criteria to help people make decisions about repository management that take these risks into account. Risks such as natural disasters and security breaches/attacks have also received a fair amount of attention.
Despite this strong relationship between risk and digital preservation, and the focus and attention that has been given to understanding different types of risk, research in this area has tended to treat risk as synonymous with vulnerability or threat. This has led to a strong focus on identifying or classifying those risks, vulnerabilities, and/or threats for the purposes of risk assessment and management. Digital preservation policies tend to focus on this approach to risk, attending to those that have the potential to cause harm, in order to demonstrate trustworthiness.
Best practices in digital preservation are established through certification systems for trustworthy digital repositories (TDRs). The idea of assessing the trustworthiness of organizations for digital preservation dates back to the 1996 report of the Task Force on Archiving of Digital Information, which called for a certification process in order to create a climate of trust for the repositories tasked with long-term preservation of digital information.
While trust is an important and foundational concept in digital preservation, it is not the focus of this chapter. Rather, of interest here is the fact that the concept of risk is foundational for the certification systems that emerged in the years following this report. Three prominent certification systems are: nestor (based on the DIN 31644 standard), CoreTrustSeal (CTS), and the Audit and Certification of Trustworthy Digital Repositories (TRAC) system (based on the ISO 16363 standard and administered by the Center for Research Libraries and by auditors accredited via the ISO 16919 standard). Each of these certification processes approaches risk probabilistically as something that can be identified and managed, and assumes that trustworthiness can be determined through a well-documented process of risk assessment. Although these certification systems have separate and distinct requirements and processes they share many elements, such as a broad view of organizational, economic, and technical risks that repository staff members should identify and mitigate. Other tools and assessments in digital preservation that also emphasize risk and rely on a classical definition include DRAMBORA, PORRO, and iRODS.
The tools and systems described above have an important characteristic in common: they are created, maintained, and applied by people and institutions that are in positions of relative privilege. For example, the systems and tools listed above are all managed by organizations in North American and Europe. The group of people who developed the ISO 16363 and ISO 16919 standards include representatives from organizations such as the National Aeronautics and Space Administration (NASA) and the National Archives and Records Administration (NARA) in the United States. The current Directors of the CoreTrustSeal Board (in 2022) represent two national European data archives, and the data archive from a large American university. DRAMBORA was created by the Digital Curation Centre and DigitalPreservationEurope. English is the primary language of these systems and tools.
We can consider those institutions that set and enforce standards for best practice in the context of those eight factors for the social construction of risk described above. Through this we can see how assumptions inherent in CTS exemplify the roles that power and privilege play in the construction of risk in trustworthy digital repository certification:
Communication
Information about the standard is communicated to the digital preservation community in English. And repositories that wish to become certified must provide their documentation and evidence of meeting the CTS requirements in English. This can be a particularly onerous task for small organizations whose primary operating language is not English, because it requires (1) staff who are proficient in English in addition to the primary language of their organization, and (2) the time and resources to translate repository documentation into English for the auditors. The word risk may not have the same exact meaning in every language, and the results of this process of risk assessment may differ depending on how people understand and interpret this term.
Complexity
CTS certification depends on how well repository staff can describe, and auditors can understand, complex structures and processes. The audit process depends on reviewers who can understand and assess repositories that are operating within complex organizational, financial, legal, and technical systems. In environments, such as digital repositories, the complexity of the system, combined with the tight coupling between the different elements of the repository, can lead to conditions in which seemingly small problems snowball into large-scale disasters. Depending on their understanding of this complexity, and/or how closely they interact with parts of a repository system, staff members and auditors may understand risk differently.
Expertise
The CTS certification system consists of many people and institutions with different types of expertise. The success of a repository’s application for certification may depend in part on whether they are assigned a reviewer with expertise in areas relevant to the repository’s situation. Also important are the social relationships among people, and between people and institutions. Whether a repository is able to become certified may also depend on how well the repository staff can reflect the type of information and knowledge that is deemed relevant and appropriate by those who are considered experts.
Organizations
The CTS reviewers are drawn from staff members of currently certified repositories. These individuals may perceive risk differently depending on the role that they play in their own organizations or depending on the position of their repository within a larger organization. Similarly, they may perceive risk differently depending on the role that they play within the CTS organization or on the relationship between their repository and the CTS organization.
Political Culture
As of 2022 the leadership team consist of people from large data repositories in North America and Europe. While there are CTS certified repositories around the world, the group of people charged with the maintenance and administration of this standard represent similar political cultures. This means that in order to become certified, repositories must make their own policies and practices understandable to people who hold particular worldviews as a result of the political culture in which they live and work.
Trust
The goal of CTS certification is to demonstrate trustworthiness with regard to long-term preservation. While repositories must be trusted by their communities and users, certification depends on whether they are trusted by auditors who may have very little in common with the stakeholders that repositories rely on for funding, infrastructure, data, etc. In previous research, I have found that it is possible to become TRAC certified without following recommended best practices, by meeting the expectations of repository users rather than TRAC auditors. While certifications, such as TRAC and CTS, are designed to facilitate demonstrations of trustworthiness for users and other potential stakeholders through the judgment of experts, in practice it can be the case that experts accept the judgment of non-expert communities as demonstrations of trustworthiness. At issue here is the question of whose judgment should be trusted and who should be trusting the judgment of others.
Uncertainty
Uncertainty about the likelihood and severity of threats to a repository can lead people to make very different judgments about risk. People who work within large, well-resourced organizations such as those represented by the CTS Board may have different experiences of uncertainty than repository staff from organizations that are smaller, less financially secure, or that work within infrastructures that regularly experience instability.
Vulnerability
People from socially vulnerable groups have different risk tolerances than those with privilege. For example, staff from under-resourced repositories may view risks or threats and something imposed upon them, while staff from well-resourced repositories may view risks as exciting opportunities that they choose to take on.
Each of these factors has the potential to influence how stakeholders in the CoreTrustSeal repository certification system construct their understanding of risk in the context of a repository audit. At each stage of the process the people carrying out the activities required for repository certification (e.g., preparation of repository documentation, communication between the repository staff members and CTS organization, review of repository documentation by CTS reviewers, etc.) people must exercise judgment about risk. Repository staff members must decide which threats to consider in their assessments and plans, and reviewers must decide whether the repository staff members have accurately and thoroughly documented risks, and whether the risk mitigation strategies described in their documentation are sufficient to ensure long-term preservation.
Discussion/Conclusion
The systems that shape the digital preservation community’s attitudes and approaches to risk rely on classical definitions that assume uniformity and rationality among digital preservation practitioners. However, we know that social factors can influence how people perceive and understand risk, and in turn how they behave in response to risk information. As a result, our current systems and tools fall short of their goals of understanding how the staff members of digital repositories will manage the risks that they encounter in order to ensure the long-term preservation of digital content.
What does it mean for digital preservation as a discipline that best practices are set and enforced by people with shared characteristics, who are nearly all located in the global north? These people are likely to construct their understanding of risk in similar ways. Particular sources of risk will be amplified and others attenuated by risk assessment processes that these organizations lead. And evaluations about whether a repository can manage risk will reflect the perspectives of those who develop and implement the standards that govern certification processes. In order to ensure the reliability of risk assessment processes in digital preservation, such as TDR certification, we must continually interrogate our assumptions about risk. Including a broad range of perspectives, from people with different backgrounds and experiences, will enable repositories to identify and prepare for a wider range of potential risks and ultimately will increase the odds of preserving valuable digital information long-term.
Understanding risk as a social construct has implications for digital preservation beyond risk assessments for repositories. For example, scholars such as Tonia Sutherland have examined the role that records play in carceral archives and discriminatory practices in the criminal justice system. In terms of climate change, research has examined the risk of climate change to organizations that preserve information, and the risks that large scale digital preservation efforts pose globally by contributing to climate change. In each of these examples, a critical perspective that takes social factors into account when considering how people construct their understanding of risk is needed in order to fully understand the intersection of digital preservation and risk.