Workflows

The Preservation API pages describe each operation on its own. These pages put them in order: complete walk-throughs of the things people actually do with the platform, each one ending in a runnable Python sample.

They all turn out to be the same five moves in different arrangements:

1. Make a Deposit — a working area in S3.
2. Get the files into it, and a METS file that describes them.
3. Say where in the repository the object belongs (archivalGroup).
4. Generate and run an Import Job.
5. Poll the Import Job result until it has finished.

What varies is where the files come from, who writes the METS, and how much of the existing object you need in front of you.

If you supply your own METS

Some callers use the Preservation API for versioned object storage, rather than for generating or managing the content of the objects. A digitisation system such as Goobi is the clearest case, but anyone in the same position reads the same way.

If that is you, then you manage and supply your own METS file. You will create Deposits, create and run Import Jobs, browse the repository, retrieve Archival Groups and export Archival Groups into new Deposits from which new versions can be created. You will not use the API to edit the METS file itself, to run analysis tools, or to have the platform describe your content for you — template None says exactly that.

Read in this order:

1. Make somewhere to put things. Objects live under a Container, and you can create one if the part of the tree you want does not exist yet. This is only for structure outside preserved objects — you cannot create Containers inside one except by importing a deposit.

2. Preserve for the first time. The whole path from an empty Deposit to version 1, with your own METS supplying the checksums. If your objects are identified in a Leeds catalogue, the same page covers starting from an identifier instead of a repository path.

3. Read what you preserved. The repository pages cover walking the tree, the view and version parameters, and fetching an object’s METS back out with ?view=mets.

4. Make a new version. With an export when you want the object’s existing files in front of you, without one when you already know what is changing — much cheaper for a small change to a large object — or with a custom Import Job when you want to state the operations yourself rather than have them worked out by comparison.

Caution

One rule catches everyone eventually: a Deposit is good for exactly one Import Job. Once a job has run from it, that Deposit is finished — generating another diff from it answers 409. Each new version starts with a new Deposit.

Bringing content in

Preserve for the first time

You have a set of files and your own METS file. The object does not exist in the repository yet; this makes version 1.
Preserve for the first time →

A deposit the platform manages

You have files but no METS. The platform scaffolds one, runs tools over your files, and writes what they found into it.
Managed METS deposit →

A BagIt bag

Your content arrives as an unpacked BagIt bag, checksums and all.
BagIt deposit →

Changing something already preserved

With a full export

You need the whole object in front of you before you can decide what to change.
Update with export →

Without an export

You know exactly what you are changing. Only the METS is copied out, and only the changed files go back.
Update without export →

A job you wrote yourself

No diff at all: you state the operations, and where each file’s bytes are to be found.
Custom import job →

Finding out what changed

Reading the Activity Stream

How a downstream service — a manifest builder, an index, a reporting job — keeps up with everything the platform preserves.
Reading the Activity Stream →

Two audiences

Some of these workflows are for consumers who manage and supply their own METS file: they create Deposits, run Import Jobs, browse the repository and export Archival Groups, and use the platform for versioned object storage rather than for generating or managing the content of the objects. Others are for consumers who hand the platform a pile of files and let it do the describing. The METS overview explains the difference as unmanaged and managed METS, and it is the single biggest fork in the road here.

Before you start

Every sample on these pages assumes you have credentials for a hosted instance of the platform and can write to the S3 bucket its Deposits live in. See Authentication, and check your connection with GET /whoami before anything else.

One more thing worth knowing early, because it shapes every workflow: a Deposit is good for one successful Import Job. It is a working area and then a record of what was done with it. To make another change, make another Deposit.