Store artifacts on a filesystem

By default, DearDiary stores every Resource artifact inline in the database. That works for kilobyte-sized configs and small pickled models, but a 500 MB checkpoint will balloon the database file and slow every metadata query. The DearDiary.FilesystemStore backend writes artifact bytes to a directory on local disk, keeping the database lean and letting you back up the bytes with the rest of your storage volume.

Configuration

Set two environment variables in your .env:

DEARDIARY_ARTIFACT_BACKEND=filesystem
DEARDIARY_ARTIFACT_FS_ROOT=/var/lib/deardiary/artifacts

The root directory is created on the first write. No separate provisioning step is needed.

Layout on disk

Each artifact is written to <root>/<aa>/<uuid>, where <aa> is a two-character shard of the UUID so no single directory grows unbounded. Two uploads of identical bytes produce distinct files: there is no content-addressed deduplication, so deleting one Resource cannot break a sibling that uploaded the same payload.

End-to-end example

Create a project, experiment, and iteration, then upload an artifact through the configured store:

julia> project_id, _ = create_project("Filesystem tutorial");
julia> experiment_id, _ = create_experiment(project_id, DearDiary.IN_PROGRESS, "FS experiment");
julia> iteration_id, _ = create_iteration(experiment_id);
julia> payload = rand(UInt8, 4096);
julia> resource_id, _ = create_resource(experiment_id, "checkpoint.bin", payload);

The resource row records the new backend and the URI that points at the bytes on disk:

julia> resource = get_resource(resource_id)DearDiary.Resource
 ├ id = 1
 ├ experiment_id = 1
 ├ name = "checkpoint.bin"
 ├ description = ""
 ├ data = UInt8[]
 ├ created_date = 2026-06-23T18:15:09.466
 ├ updated_date = nothing
 ├ backend = "filesystem"
 ├ uri = "file:///tmp/jl_bxKSa9/59/5974c385-bee8-49e0-ab28-0c0977ee82f1"
 ├ size_bytes = 4096
 └ content_hash = "27c3b89cf10e631678668dd3474a697ada1701102697cf7f71cecd7d5af842e6"

julia> resource.backend"filesystem"

julia> resource.uri |> startswith("file://")true

The on-disk path is reachable directly for inspection or streaming from another process. DearDiary itself reads through read_resource_data:

julia> read_resource_data(resource_id) == payloadtrue