Artifact Storage

AbstractArtifactStore

Marker abstract supertype for pluggable artifact backends. Concrete subtypes (SQLiteStore, FilesystemStore, S3Store) implement write_artifact, read_artifact, and delete_artifact.

Service code never instantiates a store directly. It calls current_artifact_store, which reads the active APIConfig and dispatches to the configured backend.

ArtifactWriteResult

The metadata returned by write_artifact. Carries everything a Resource row needs to record the new artifact — the canonical URI, the on-disk size, and the sha256 content hash.

Fields

• uri::String: Canonical pointer at the stored bytes. The empty string is the sentinel for the SQLite backend (the bytes live inline in resource.data).
• size_bytes::Int64: The exact number of bytes written.
• content_hash::String: Lower-case sha256 hex digest of the bytes. Always populated — every backend hashes on write.

SQLiteStore <: AbstractArtifactStore

The legacy artifact backend: bytes live inline in the resource.data column of the project SQLite database. No separate write to disk or object store — the bytes travel into the database as part of the same INSERT that creates the Resource row.

This is the default for offline tests and for installations that have not opted into the external backends. It is also the only backend that does not require any out-of-band setup (filesystem root, S3 credentials).

FilesystemStore <: AbstractArtifactStore

Filesystem-backed artifact store. Writes each artifact to a fresh file under <root>/<aa>/<uuid>, where <aa> is a two-character shard derived from the UUID (so a single directory never grows unbounded). The bytes never enter the SQLite database — the Resource row holds only metadata (size, sha256, the file:// URI).

Fields

• root::String: Absolute directory under which every artifact lives.

Selected at server startup by DEARDIARY_ARTIFACT_BACKEND=filesystem with the root supplied through DEARDIARY_ARTIFACT_FS_ROOT. The directory is created on first write.

S3Store <: AbstractArtifactStore

S3-compatible object-store backend. Speaks the S3 protocol via a minimal hand-rolled SigV4 signer, so the same struct talks to AWS S3, MinIO, Cloudflare R2, Backblaze B2, or any endpoint that implements the wire format — pick the right endpoint URL and bucket name.

Path-style addressing is used (<endpoint>/<bucket>/<key>) so this works against MinIO out of the box; AWS S3 still accepts path-style for buckets created before the virtual-hosted cut-over.

Fields

• bucket::String: Target bucket name.
• endpoint::String: Scheme + host (+ optional port). Examples: https://s3.us-east-1.amazonaws.com, http://localhost:9000 (MinIO).
• region::String: Region used in the SigV4 credential scope (e.g. us-east-1).
• access_key::String, secret_key::String: SigV4 credentials.
• http_transport::Function: (method, url, headers, body) -> response. Defaults to _default_s3_transport. Overridden by tests with a closure that captures the request rather than hitting the network.

write_artifact(store::AbstractArtifactStore, data::AbstractVector{UInt8})::ArtifactWriteResult

Persist data through the backend represented by store and return the metadata needed to register the new artifact in the Resource table.

For backends that store bytes outside the database (filesystem, S3) this performs the actual upload. For the SQLiteStore it is a no-op write — the bytes still travel into the resource.data column via the service layer's INSERT.

read_artifact(store::AbstractArtifactStore, uri::AbstractString, inline::Optional{AbstractVector{UInt8}})::Vector{UInt8}

Fetch the bytes for the artifact identified by uri, dispatching on store.

inline is the fallback the caller passes when the canonical bytes live in resource.data rather than in an external store. Backends that store bytes externally ignore inline; the SQLiteStore returns it directly.

delete_artifact(store::AbstractArtifactStore, uri::AbstractString)::Bool

Remove the artifact identified by uri from the underlying store. Returns true on success.

For the SQLiteStore this is a no-op (the bytes vanish when the parent Resource row is deleted); external backends issue the real delete.

backend_id(store::AbstractArtifactStore)::String

Return the short string identifier ("sqlite", "filesystem", "s3") used to populate resource.backend when a new artifact is written through store.

sha256_hex(data::AbstractVector{UInt8})::String

Compute the lower-case sha256 hex digest of data. Used by every backend to populate ArtifactWriteResult.content_hash. Delegates to the Julia stdlib SHA module, so the project takes on no extra dependency.

current_artifact_store()::AbstractArtifactStore

Return the AbstractArtifactStore selected by the active APIConfig.

run populates _DEARDIARY_APICONFIG at startup, so this helper is only meaningful after the server has booted (or a test harness has supplied a config). When no config is loaded the function falls back to SQLiteStore so offline code paths (@with_deardiary_test_db) behave the same as before the artifact-store refactor.

artifact_store_for(config::APIConfig)::AbstractArtifactStore

Return the concrete AbstractArtifactStore selected by config. Unknown backends fall back to SQLiteStore with a warning — the server stays bootable, but the operator gets a loud signal that the env config is wrong.

artifact_store_for(backend::AbstractString)::AbstractArtifactStore

Backend-only convenience: returns SQLiteStore for "sqlite" and warns otherwise. Useful for tests and tooling that want to dispatch on a backend label without constructing a full APIConfig. Backends that need additional config (filesystem root, S3 credentials) must go through the APIConfig overload.

migrate_artifacts!(target::AbstractArtifactStore = current_artifact_store())::MigrateArtifactsResult

Walk every Resource row whose backend == "sqlite", replay its inline bytes through target, and update the row to point at the new artifact. Designed for one-shot backfill after switching DEARDIARY_ARTIFACT_BACKEND away from "sqlite":

using DearDiary
DearDiary.run(; env_file=".env")
DearDiary.migrate_artifacts!()   # uses the now-configured target store

The migration is idempotent and restartable: rows that have already been migrated have a different backend value and are skipped on subsequent passes. A row that fails (e.g. S3 returns 503) is left untouched, so re-running the function picks up where it stopped.

Refuses to do anything when target is itself a SQLiteStore — there is nowhere to move the bytes to.

Arguments

• target::AbstractArtifactStore: Destination store. Defaults to the server's currently configured store.

Returns

A MigrateArtifactsResult summarising the pass.

MigrateArtifactsResult

Return value of migrate_artifacts!.

Fields

• migrated::Int64: Number of rows successfully moved off the SQLite backend.
• skipped::Int64: Number of rows that were already on a non-SQLite backend at scan time (typical when a previous pass was interrupted partway through).
• failed::Int64: Number of rows where the move could not be completed. Each failure is logged via @error so the operator can inspect the offending row.