Overview
The Digital Preservation platform takes sets of files - a digitised book, a manuscript, a born-digital collection - and stores them as versioned digital objects, in the Oxford Common File Layout (OCFL). The platform exists to produce those OCFL objects and to make them easy to build, update and read.
This documentation explains how to use the platform. It is written for:
- Developers writing API clients. Most integrations - Goobi, the iiif-builder, migration scripts, ad hoc tools - talk to the Preservation API. This forms the bulk of the documentation, and is accompanied by runnable Python samples.
- People using the Preservation UI. The UI section covers browsing the repository, creating deposits, uploading files and running pipelines through the web interface.
- Operators and contributors who need to know how the pieces fit together. The Storage API and Internals sections describe the layers underneath the Preservation API.
The preservation stack
Section titled “The preservation stack”flowchart TB
accTitle: The preservation stack
accDescr: Five layers, top to bottom. The Preservation UI, and other clients such as Goobi, iiif-builder and scripts, call the Preservation API. The Preservation API calls the Storage API; a few trusted API clients may call it directly too. The Storage API is the only thing that talks to Fedora. Fedora writes OCFL objects into S3. From the OCFL objects alone the content can be replicated, restored, or carried forward to whatever comes next, and very trusted readers may read the S3 origin directly.
UI["`**Preservation UI**
used by staff and external contributors`"]
Clients["`Goobi, iiif-builder,
ad hoc scripts`"]
PAPI["`**Preservation API**
understands METS; provides Deposits,
generates Import Jobs, runs pipelines,
publishes an activity stream`"]
Trusted["`Trusted API clients`"]
SAPI["`**Storage API**
Containers, Binaries, Archival Groups;
a METS file is just another binary`"]
Fedora["`**Fedora**
not accessed by anything
except the Storage API`"]
OCFL["`**OCFL in S3**
one versioned object per digital object`"]
Future["`Replication, restoration,
whatever comes next`"]
Readers["`Direct origin access for
(very) trusted readers`"]
UI --> PAPI
Clients --> PAPI
PAPI --> SAPI
Trusted --> SAPI
SAPI --> Fedora
Fedora --> OCFL
OCFL --> Future
OCFL --> Readers
classDef layer stroke-width:2px;
classDef edge stroke-dasharray:4 3;
class UI,PAPI,SAPI,Fedora,OCFL layer;
class Clients,Trusted,Future,Readers edge;
Reading the diagram from the bottom up:
| Layer | What it is |
|---|---|
| OCFL in S3 | The preserved content itself: one OCFL object per digital object (Archival Group), each with a full version history. |
| Fedora | The repository software that writes and versions the OCFL objects. Nothing talks to Fedora except the Storage API. |
| Storage API | The gateway to Fedora. Knows about Containers, Binaries and Archival Groups, and turns Import Jobs into new OCFL versions. Does not know what a METS file is. |
| Preservation API | The application-facing API. Understands METS, provides Deposits (working areas in S3) for assembling content, generates Import Jobs, runs tools over files, and publishes an activity stream. |
| Preservation UI, Goobi, iiif-builder, scripts | Clients of the Preservation API. |
The Components page describes each of these in more detail, and the Concepts page defines the terms used throughout the site.
Recoverable from OCFL alone
Section titled “Recoverable from OCFL alone”
OCFL logo: “hand-drive” by Patrick Hochstenbach, licensed under CC BY 2.0.
A design principle of the platform is that the running systems are not needed for the preserved content to be usable. Given a copy of the S3 bucket (or a hard drive containing the same file layout) and the OCFL specification, everything that was preserved can be recovered: every version of every file, with its checksum, and the METS file that describes how the files fit together. The databases behind Fedora, the Preservation API and the Storage API hold working state and history, but nothing that the OCFL objects do not also record.