File storage

A Blueprint stores files in buckets. A module declares the buckets, a FileField names the one its files go into, and the server stores the bytes through a pluggable byte store, keeps the record of every file in the product's own runtime schema, and serves downloads through short-lived signed links.

Three things are fixed by design:

• The tables belong to the product. Storage never touches the platform storage schema, and the generated SQL never mentions auth.* or storage.*, so it runs on Supabase and on plain PostgreSQL alike.
• The server decides who is calling. The tenant and the actor come from the authenticated session (serverAuthoritativeRequestContext). A client cannot name either, and a file of another tenant looks the same as a file that does not exist.
• Every file has a trail. Each transition of an upload or an object writes a row to an append-only events table.

Declare buckets and bind a field

A module declares its buckets. A bucket fixes the largest file, the MIME types it admits, who may read it, and the processors that run over its files.

const workOrderFiles = StorageBucket(
  'work_orders_files',
  title: 'Work order files',
  maxSizeMb: 20,
  mimeTypes: ['image/*', 'application/pdf'],
);

final module = Module(
  name: 'work_orders',
  version: '1.0.0',
  entities: [workOrder],
  buckets: const [workOrderFiles],
  db: const ModuleDb(schemaName: 'work_orders'),
);

A FileField names its bucket with bucket:. The field's own kind, maxSizeMb, mimeTypes and multiple are checked when the model is validated (they must fit inside the bucket's, as the table below lists), and the form can use them to guide a user. They are not enforced at upload time: the server holds an upload to the bucket's limits and type list, whichever field the file is for.

const attachments = FileField(
  'attachments',
  kind: FileKind.image,
  bucket: 'work_orders_files',
  maxSizeMb: 10,
  multiple: true,
);

StorageBucket option	Meaning
name	Lower-case identifier, ^[a-z_][a-z0-9_]*$. Unique across the whole Blueprint
title	Descriptive. Not stored in the database; a change to it alone changes no row
visibility	private (default): signed links only. public: also served from a stable public path
maxSizeMb	Largest file, in megabytes. Greater than zero
mimeTypes	Admitted types, exact or family/*. Empty admits any type
access	tenant (default): every actor of the tenant. followsRecord: only actors who may read the record that holds the file
processors	Names of the processors that run over each stored file. Each needs an ObjectProcessor in the server configuration

The value of a FileField is one JSON object, whatever multiple says. key is the stored object's id:

{"files": [{"bucket": "work_orders_files", "key": "9f1c...", "mime": "image/png", "size": 482113, "name": "site-plan.png"}]}

StorageClient returns each entry (StoredFileRef.toJson()), so a form puts {'files': [ref.toJson()]} into the record.

What the definition document carries

The definition document, the canonical form of the model, publishes both declarations. A module with no bucket has no storage key, so a document that declares none is unchanged.

{
  "name": "work_orders",
  "storage": {
    "buckets": [
      {
        "name": "work_orders_files",
        "title": "Work order files",
        "visibility": "private",
        "max_size_mb": 20,
        "mime_types": ["image/*", "application/pdf"],
        "access": "tenant",
        "processors": []
      }
    ]
  }
}

A field publishes its binding in file, next to type: jsonb; the limits it does not declare are omitted:

{"name": "attachments", "type": {"kind": "jsonb"},
 "file": {"kind": "image", "bucket": "work_orders_files", "max_size_mb": 10, "multiple": true}}

Validation

The Blueprint validator reports these findings (codes carry the prefix blueprint.validation.):

Code	When
storage_bucket_duplicate	Two buckets share a name, in one module or across modules
storage_bucket_name_invalid	A bucket or FileField.bucket name is not a lower-case identifier
storage_bucket_size_invalid	maxSizeMb is zero or negative
storage_processor_name_empty	A processor name is blank
storage_processor_name_invalid	A processor name is not a lower-case identifier (^[a-z_][a-z0-9_]*$); it becomes a path segment of every derivative key
storage_bucket_unknown	A FileField names a bucket no module declares
storage_field_size_invalid	The field's maxSizeMb is zero or negative
storage_field_size_exceeds_bucket	The field allows more than the bucket
storage_field_mime_outside_bucket	A field MIME type is outside a non-empty bucket list
storage_field_kind_outside_bucket	The field's kind has no type in a non-empty bucket list: image, video and audio need a type of that family, document an application/ or text/ type; binary fits any bucket. */* admits everything, and comparison ignores case

The runtime tables and the worker role

A Blueprint with at least one bucket gets five more tables in the runtime schema (vyuh_runtime unless BlueprintPersistenceConfig.runtimeSchema says otherwise). A Blueprint with none gets none of them.

Table	Holds
storage_buckets	One row per bucket: limits, MIME list, access, processors, and the Blueprint that owns the name. These are runtime contract rows: migrations keep them exact
storage_upload_sessions	An upload in progress: declared name, type and size, bytes received, status, creator, expiry
storage_objects	A stored file: key, size, SHA-256, creator, deleted_at, purged_at
storage_object_events	The audit trail. Append-only
storage_derivatives	The work queue and results of processors, one row per object and processor

Row-level security is on for all five, keyed on the vyuh.tenant_id the server sets from the session. The runtime role gets exact grants and no more: it may update only the columns that change over a file's life (bytes received, status, deleted_at, purged_at, derivative state), it is never granted DELETE, and the events table accepts SELECT and INSERT only. A trigger refuses to update or delete an event even for the table's owner, and another refuses to undo a deletion. An object key must start with <tenant>/, so a row cannot point at another tenant's bytes.

The worker role

The background work reads every tenant: expiring abandoned uploads, purging deleted files, claiming derivatives. The runtime role must not, so that work runs on the access profile's worker role (see Database Access Profiles):

const app = DbRole.product('shop_app', login: true, runtime: true);
const worker = DbRole.product('shop_worker', login: true, worker: true);
// BlueprintDb(access: DbAccessProfile(roles: [app, worker]))

The worker role may read sessions, objects and derivatives, and update only the claim columns of derivatives. It settles nothing: results are written back as the runtime role in the file's own tenant. The compiler reports:

• an error when a bucket declares processors and the profile has no worker role;
• a warning when there is a runtime role but no worker role, or buckets but no runtime role.

At boot the server checks that BlueprintStorageConfig.workerConnection really logs in as the worker role, and refuses to start when it does not. With no roles in the profile there is no row-level isolation to step around, and the sweeps run on the runtime connection.

Configure the server

Pass a BlueprintStorageConfig to BlueprintServerPlugin(storage: ...), or to BlueprintProtocolsConfig(storage: ...) for a feature-based server. A Blueprint with buckets and no configuration stops the server at startup, naming the buckets; a configuration with no buckets mounts nothing.

BlueprintServerPlugin(
  targets: [BlueprintTarget(blueprint: blueprint)],
  name: 'work_orders',
  storage: BlueprintStorageConfig(
    store: FilesystemObjectStore(Directory('/var/lib/work_orders/files')),
    signingSecret: signingSecretBytes, // at least 32 bytes
    connection: storagePool, // its own pooled DbAdapter
    workerConnection: workerPool, // logs in as the worker role
    urlTtl: const Duration(minutes: 5),
    processors: [ThumbnailProcessor()],
  ),
);

Setting	Default	Meaning
store	required	Where the bytes live: FilesystemObjectStore or MemoryObjectStore (tests)
signingSecret	required	At least 32 bytes, the same on every process that serves links. There is no key rotation: changing it invalidates every link in flight
connection	the server's database	The tenant-scoped runtime connection storage runs on
workerConnection	none	The connection that logs in as the worker role. Required for the sweeps and for public buckets
uploadTtl	24 hours	How long an upload session stays open
urlTtl	15 minutes	How long a signed link is valid. Keep it short (see Security)
processors	none	The ObjectProcessors buckets name; each declared name must be provided exactly once
readability	none	Decides access to followsRecord buckets. Without it those files are served to no one
derivativeLease	10 minutes	How long a processor may hold a derivative before another runner takes it over
sweeps	true	Whether this process runs the sweeps
sweepInterval	1 minute	How often the sweeps run; must be positive
purgeAfter	30 days	How long a soft-deleted file keeps its bytes

Give storage its own connection

An upload keeps one database transaction open while it writes each chunk to the byte store. On the single adapter the rest of the API shares, every request queues behind each chunk. In production hand connection (and workerConnection) their own pooled DbAdapters. When connection is not set the server logs a warning about it at boot.

One byte store for the whole deployment

FilesystemObjectStore is safe for one writer process per root: it serializes writes inside the process, not across processes. Every process of a deployment that serves requests or runs sweeps must therefore see the same store, the same root on shared storage or the same object store, and not a copy of it. A chunk that lands on a replica whose store does not hold the upload's earlier bytes finds the store shorter than the ledger, and the upload is aborted as store_inconsistent. A separate sweep or worker process needs the shared store too: without it the expiry sweep cannot remove the part bytes and the purge sweep cannot remove deleted files. Until a networked store exists, run one serving process per filesystem root.

Sweeps

While the server runs, a schedule runs these every sweepInterval, in any process whose sweeps is true (also one that serves no routes):

• expire sessions: open uploads past their expiry become expired, their part bytes are removed, and upload_expired is recorded;
• purge deleted: a soft-deleted file older than purgeAfter has its bytes and derivative bytes removed and is marked purged. The row stays;
• run processors: only when processors is set (below).

A sweep that fails is logged at error level, with its name and how many times in a row it has failed, and runs again at the next tick. A process that only serves requests sets sweeps: false and needs no worker connection.

Depend on the storage package

FilesystemObjectStore, MemoryObjectStore, ObjectProcessor, ProcessorResult, ObjectKey and the wire types live in vyuh_blueprint_storage, which the server does not re-export. A host that builds a BlueprintStorageConfig or a processor adds vyuh_blueprint_storage to its own pubspec.yaml.

Processors and derivatives

A bucket lists processor names; the host implements them. Completing an upload in such a bucket writes one pending row to storage_derivatives per processor. runProcessors claims pending rows (and rows whose lease lapsed), calls ObjectProcessor.process(object, store), and records the result as ready with the derivative's key, type and size, or failed with the error, together with a derivative_ready or derivative_failed event. A claim is a compare-and-set on the lease, so two runners never both settle a row. Write processors so they are deterministic and write their output atomically; a processor that lost its lease may still write bytes before it finds out.

final class ThumbnailProcessor implements ObjectProcessor {
  @override
  String get name => 'thumbnail';

  @override
  Future<ProcessorResult> process(StoredObject object, ObjectStore store) async {
    // Read object.key from the store, write the derivative to
    // ObjectKey.forDerivative(object.tenantId, object.bucket, object.id, name).
    return ProcessorResult.ready(key: key, mime: 'image/png', size: size);
  }
}

Blueprint ships no image or video code; the processor port is the extension point.

Routes

The routes mount under <basePath>/storage, /api/storage by default, and only when the Blueprint has a bucket. Every route needs a signed-in session except the download link and the public path. Errors have one shape: {"error": {"code": "storage.<reason>", "message": "..."}}.

Route	Purpose
POST /uploads	Start an upload session: {bucket, name, mime, size, sha256?}
GET /uploads/:id	Where the upload is: the resume point
PATCH /uploads/:id	Append bytes at Upload-Offset
POST /uploads/:id/complete	Verify and store. Repeating it returns the same object
DELETE /uploads/:id	Abort
GET /objects/:id	The stored object's metadata
POST /objects/:id/url	Issue a signed link: {disposition?: "inline" or "attachment"}
DELETE /objects/:id	Soft delete
GET /files/:token	Download by link. No session. Supports Range
GET /public/:bucket/:objectId	Download from a public bucket. No session. Only when the Blueprint has a public bucket

Status mapping

Situation	Status	Code
No tenant or actor on the request	401	storage.unauthenticated
Absent, another tenant's, another actor's upload, deleted, not readable (followsRecord), or a bad, tampered, expired or wrong-tenant token	404	storage.not_found
Unknown bucket on create	404	storage.bucket_unknown
Malformed JSON or field, bad sha256, bad Upload-Offset or disposition	400	storage.malformed_request
Invalid file name (empty, over 255 characters, control characters)	400	storage.name_invalid
The client's connection broke during a chunk	400	storage.request_interrupted, with Upload-Offset
Wrong or duplicate offset	409	storage.offset_mismatch, with Upload-Offset
Chunk on, or abort of, a completed upload	409	storage.upload_completed
Chunk or complete on an aborted or expired upload	410	storage.session_closed or storage.expired
File over the bucket limit	413	storage.too_large
Chunk runs past the declared size	413	storage.size_mismatch, with Upload-Offset
Type not in the bucket's list, or bytes contradict the declared type	415	storage.mime_not_allowed
Complete with fewer bytes than declared, or a wrong checksum	422	storage.size_mismatch, storage.checksum_mismatch
The caller's tenant id cannot be part of a storage key (for example it contains /)	422	storage.key_invalid; the client does not retry it
Range selects no byte	416	storage.range_not_satisfiable, with Content-Range: bytes */size
A failure inside the service	500	storage.internal; 503 when the database is unavailable

Cross-tenant access is always 404, never 403: the answer does not reveal whether the object exists.

Upload protocol

An upload is a session the server keeps in PostgreSQL, so it survives a restart of the server and a loss of the client's connection.

1. POST /uploads creates the session and returns {id, object_id, bucket, name, mime, size, offset, status, expires_at} with Upload-Offset: 0. The server stores the MIME essence (lower-case type/subtype, parameters dropped), never the string the client sent. The file's id, object_id, is allocated here.
2. PATCH /uploads/:id with header Upload-Offset: n and the raw bytes of [n, n + length). The offset must equal the bytes the server holds. The answer carries the new offset in the header and in the body (offset). Chunks are sequential; the size is the client's choice, and the server writes in 1 MiB pieces whatever the chunk.
3. POST /uploads/:id/complete, with an empty body or {sha256}. The server hashes what it stored, sniffs the first bytes, and checks size, checksum and type before it commits the part as the object. It answers {id, file: {bucket, key, mime, size, name}, sha256, created_at}.

A chunk that is refused writes no bytes. A client that loses an answer, or is refused with 409, asks GET /uploads/:id and continues at the offset it returns, without resending the bytes below it. The offset is the byte store's truth: bytes the store accepted but the ledger had not yet counted (a crash between the two) are adopted, not lost. After 409 and 413, the server may close the connection without reading the body; the client resumes with a GET.

Status of a session	Meaning
open	Accepting chunks
completed	An object exists. Complete again returns it
aborted	Aborted by the owner, or refused at complete (a bad upload is aborted and its part removed)
expired	Past uploadTtl; swept

A session belongs to the actor who created it. Another actor of the same tenant gets 404 for it, so an upload cannot be taken over, and there is no way yet for an administrator to abort someone else's session.

From Dart

final storage = StorageClient(
  Uri.parse('https://api.example.com/api/storage/'),
  httpClient: authenticatedClient, // the client that carries the session
);

final ref = await storage.uploadBytes(
  bucket: 'work_orders_files',
  name: 'site-plan.png',
  mime: 'image/png',
  bytes: bytes,
  chunkSize: 1024 * 1024,
  onProgress: (sent) => print('$sent of ${bytes.length}'),
);
record['attachments'] = {'files': [ref.toJson()]};

final link = await storage.downloadUrl(ref.key); // absolute Uri

HttpBlueprintProtocolClient(storageUrl: ...) exposes the same client as .storage, on the protocol client's own http.Client. Authentication is the http.Client's job; the storage client adds none and never names a tenant or actor.

uploadBytes computes the SHA-256, which the server verifies. upload takes a Stream and can resume only inside the chunk in flight, because a stream cannot be read twice; uploadResumable takes a function that reopens the source at an offset. A dropped connection, a 5xx, 408 or 429, a 409, or a storage.request_interrupted answer is retried (up to maxRetries in a row, reset by progress) after a status request. A chunk that times out is aborted before the next goes out, and its deadline grows with its size. Failures arrive as StorageClientException with a kind: notFound, conflict, tooLarge, unsupportedType, checksumMismatch, invalid, closed, unauthenticated, transport (the only retryable one) and notResumable. A call that fails for good aborts the upload.

Download links

POST /objects/:id/url returns {url, expires_at}, with url a root-relative path (/api/storage/files/<token>). StorageClient.downloadUrl resolves it against the client's own storageUrl, so a server behind a path prefix works. The link needs no session. It is a bearer credential: whoever holds it can read the file until it expires, or until the file is deleted, which revokes every link at once because each download re-checks the object. A tenant that is handed another tenant's link can read it too; links are never audited at the time they are used, only when they are issued (url_issued).

GET /files/:token supports a single Range (bytes=a-b, a-, -n) and answers 206 with Content-Range, or 416. Several ranges, another unit, or a malformed header get the whole file. There is no HEAD, ETag or If-Range, and responses are no-store.

Tenant isolation and audit

• Object keys are <tenant>/<bucket>/<objectId>; the part of an upload in progress is ….part. The key is computed from the caller's tenant, never read from the request, and the table refuses a key outside the tenant's prefix.
• Every route handler takes the tenant and actor from the session. Headers and body fields that claim otherwise are ignored.
• The service checks the caller's tenant and, for an upload, the creator as well as the database doing it, so a deployment that connects as the table owner (which bypasses row-level security) still cannot cross tenants.
• storage_object_events records, for each transition, the actor, the tenant, a correlation id (the X-Correlation-Id request header when it is a plain token, otherwise a fresh one) and a detail object.

Event	Written when
upload_started	A session is created
upload_completed	The upload became an object
upload_aborted	Aborted by the owner or refused at complete, with the reason
upload_expired	The expiry sweep closed it
url_issued	A signed link was issued
deleted	The object was soft-deleted
derivative_ready, derivative_failed	A processor settled a derivative

The sweeps write as the actor system:storage-sweep. The purge sweep writes no event, and a public read writes none either, because an event needs an actor.

Security properties

• Types. The server stores and serves the MIME essence only. A declared type must be type/subtype of token characters; a comma in a parameter, or anything that could smuggle a second type, is refused even for a bucket with no list. At complete the first bytes are sniffed (PNG, JPEG, GIF, WebP, HEIC, AVIF, PDF, MP4, WebM, ZIP and their relatives): a declared type that has a known signature must match its bytes, and application/octet-stream passes only when the sniffed type is one the bucket admits. A client cannot get past the list by lying about the type.
• Active content. Every download carries X-Content-Type-Options: nosniff, Cache-Control: private, no-store and Referrer-Policy: no-referrer. Only a fixed set of passive types (PNG, JPEG, GIF, WebP, AVIF, HEIC, PDF, common audio and video) may be served inline. Anything else, SVG and HTML included, is forced to attachment and sent with Content-Security-Policy: sandbox; default-src 'none', even when the link asks for inline and also on the public route.
• File names. The stored name is kept verbatim. In Content-Disposition it is encoded (RFC 5987) with format characters, bidirectional overrides, controls and separators removed. A UI that shows file.name from JSON must still escape it.
• Links. A token is base64url(claims).base64url(HMAC-SHA256), compared in constant time, with an expiry. The claims name the object, the tenant and the disposition; the server re-checks that the object is not deleted, still in that tenant and still in a bucket that exists. There is one secret and no rotation.
• Tokens appear in paths. A link carries its token in the URL path, so request logs, proxies and browser history see it. Keep urlTtl short, treat a request log as a store of credentials for that time, and leave the request logger off the storage mount if you can. The framework's logger cannot redact path segments today.
• Size. Each chunk is bounded by the bytes still owed: a body longer than that is refused before it is read (known length) or at the first byte past it. JSON bodies are capped at 64 KiB.

What is not included yet

• The Flutter picker and preview: nothing in vyuh_blueprint_ui draws an upload control or a file preview.
• Byte stores other than the filesystem and memory: there is no S3 or Supabase Storage store.
• Real thumbnail and transcode processors: only the ObjectProcessor port.
• Write-time verification of a FileField value: the create and update paths store the reference as given. They do not check that the object exists, belongs to the caller's tenant, or fits the field's own limits, so a record can name a file that is not there.
• A record link on the object: followsRecord buckets depend on the readability callback the host supplies, which is not wired to the entity read path.
• An administrator's abort of someone else's upload.
• Wiring processors through the outbox; they run on the sweep schedule.
• Public-bucket audit, HEAD, ETag and conditional requests.

Operational notes

• A used bucket cannot be retired. Removing a bucket from the model is refused while any object row still names it, soft-deleted and purged rows included. The runtime role never deletes object rows, so a bucket that has held a file stays until the database owner has archived and deleted those rows. An empty bucket's removal deletes its row.
• Removing the last bucket drops the tables. That is destructive: cut the migration with --allow-destructive, and apply it only when storage_object_events and storage_upload_sessions are empty. Events cannot be deleted by the runtime role, so the owner archives them, disables the storage_object_events_immutable trigger, clears the rows and then applies the migration. The plan refuses with the row counts otherwise.
• One Blueprint per bucket name in a runtime schema. storage_buckets records the Blueprint that declares a name, and a second Blueprint declaring the same name in the same runtime schema is refused (SQLSTATE 23505) at install and at migration, naming both. Rename the bucket or give the Blueprint its own runtime schema.
• Bucket rows come from the emitted contract. blueprint db emit writes them to 90_runtime_contract/500_storage_buckets.sql and migrations keep them exact; blueprint dev installs the emitted contract files on a local database. The server checks at boot that every bucket has its row and names the file to apply when one is missing. The plugin has no development shortcut: apply the emitted files. A test that opens StorageService itself can pass StorageService.open(developmentOwner: ...), a connection that owns the schema, to install the rows.
• public exposes existing files. Turning a bucket public serves every file already in it. The public path reads through the worker connection (it must read across tenants), so a process with a public bucket and no workerConnection cannot mount the routes, and says so at startup. Only a public bucket with access: tenant is served there; a public followsRecord bucket answers 404 on the public path. A public read writes no audit row.
• Any actor of a tenant can delete or link any file of a tenant bucket. Reading, linking and soft-deleting are checked against the tenant, not against the record that holds the file or the actor who uploaded it (only an upload session is bound to its creator). An application that must restrict who may delete or share a file, as a regulated one usually must, enforces that above these routes or uses a followsRecord bucket with a careful readability callback.
• followsRecord is only as good as the callback. FileField values are not verified when a record is written, so a record can name an object id that belongs to another record. A RecordReadability callback must therefore not grant access merely because some readable record references the object id. Until objects carry a link to their record, decide from facts the host owns (for example a table in which the host records which record it attached each object to) or leave the bucket tenant.
• No quota and no rate limit. The server does not limit how many upload sessions a user may open or how much a tenant may store, and it does not rate limit /files or /public. Apply those at the gateway or in the host.
• Early refusals close the connection. A 409 or 413 is sent before the request body is read, so the connection may be closed. The client reads the offset with GET and continues, which StorageClient does on its own.
• Titles are not data. A bucket's title is not stored; changing only it cuts no change to the bucket row.