Store artifacts on a filesystem

By default, DearDiary stores every Resource artifact inline in the database. That works for kilobyte-sized configs and small pickled models, but a 500 MB checkpoint will balloon the database file and slow every metadata query. The DearDiary.FilesystemStore backend writes artifact bytes to a directory on local disk, keeping the database lean and letting you back up the bytes with the rest of your storage volume.

Configuration

Set two environment variables in your .env:

DEARDIARY_ARTIFACT_BACKEND=filesystem
DEARDIARY_ARTIFACT_FS_ROOT=/var/lib/deardiary/artifacts

The root directory is created on the first write. No separate provisioning step is needed.

Layout on disk

Each artifact is written to <root>/<aa>/<uuid>, where <aa> is a two-character shard of the UUID so no single directory grows unbounded. Two uploads of identical bytes produce distinct files: there is no content-addressed deduplication, so deleting one Resource cannot break a sibling that uploaded the same payload.

End-to-end example

Create a project, experiment, and iteration, then upload an artifact through the configured store:

julia> project_id, _ = create_project("Filesystem tutorial");
julia> experiment_id, _ = create_experiment(project_id, DearDiary.IN_PROGRESS, "FS experiment");
julia> iteration_id, _ = create_iteration(experiment_id);
julia> payload = rand(UInt8, 4096);
julia> resource_id, _ = create_resource(experiment_id, "checkpoint.bin", payload);

The resource row records the new backend and the URI that points at the bytes on disk:

julia> resource = get_resource(resource_id)DearDiary.Resource
 ├ id = "b1af5dc0-9b8b-4e97-8ac7-f12f3b3bfb83"
 ├ experiment_id = "abc869b0-4e06-42d6-9ac0-26293b18fc53"
 ├ name = "checkpoint.bin"
 ├ description = ""
 ├ data = UInt8[]
 ├ created_date = 2026-07-19T05:20:30.846
 ├ updated_date = nothing
 ├ backend = "filesystem"
 ├ uri = "file:///tmp/jl_aJBfAl/19/196b132c-6cb0-4d79-aadc-9e4376d097fe"
 ├ size_bytes = 4096
 └ content_hash = "0c7dedf80c38f41279d79a5d71be2ef9deacc16acc983b769036b03e7f7ff120"

julia> resource.backend"filesystem"

julia> resource.uri |> startswith("file://")true

The on-disk path is reachable directly for inspection or streaming from another process. DearDiary itself reads through read_resource_data:

julia> read_resource_data(resource_id) == payloadtrue