Amazon S3

Amazon S3 (and S3-compatible), Google Cloud Storage and Azure Blob Storage: the s3 destination, from the object_store package. Declare the package once in dependencies.yml; DRE installs it on demand:

dependencies.yml

plugins:
  - object_store

Profile fields

Set in profiles.yml, under destinations:, in a target with type: s3.

Field	Default	Notes
bucket		Default bucket (or use s3://bucket/… paths)
region		AWS region, e.g. ap-southeast-2.
access_key_id		Access key id (leave empty to use the ambient credential chain) Secret: use env_var().
secret_access_key		Secret access key. Secret: use env_var().
session_token		Temporary session token. Secret: use env_var().
profile		A profile from the AWS config files.
endpoint		An S3-compatible store’s URL.
allow_http		Allow a plain-HTTP endpoint.
connect_timeout	30s	How long to wait for a connection (30s, 2m, or seconds) A duration (30s, 2m) or seconds.
timeout	60s	How long a read or write may make no progress before it fails. A duration (30s, 2m) or seconds.
retries	3	How many times to try again after a temporary error (0: never)

Options

Set in the report’s output.destination entry.

Option	Type	Default	Notes
if_exists	string	overwrite	When a file is already at the path: overwrite it, fail with error, or number the new one. One of overwrite, error, number.

Notes

bucket, region, and access_key_id + secret_access_key (+ session_token). Leave the keys out to use AWS’s default credential chain, the same as the AWS CLI: environment variables, the shared config and credentials files (the profile named by profile:, else AWS_PROFILE), SSO, credential_process, web identity, and container or instance roles. AWS_EC2_METADATA_DISABLED is honoured, and with no credentials anywhere the delivery fails at once, listing what it tried. The region comes from region:, else the AWS config. endpoint and allow_http point it at S3-compatible stores. Paths are s3://bucket/key, or a bare key in bucket. Takes if_exists (see A file already at the path); the S3-compatible store must support conditional writes for error and number.

Deliver a dated finance export

Use S3 for a shared archive or a downstream data pipeline. After signing in with the AWS credential chain, add a profile and a destination to the monthly finance report:

profiles.yml

destinations:
  archive:
    targets:
      dev: {deliver: false}
      prod: {type: s3, bucket: finance-reports, region: ap-southeast-2}

# within the report's output
destination:
  profile: archive
  path: "s3://finance-reports/monthly/{{ run.date.yyyymm }}.xlsx"
  if_exists: error

Preview first, then dre validate <report> --target prod to check the profile and path. s3/file-exists means the destination already holds that file; use another dated path or deliberately choose overwrite. delivery-failed with an authorization error needs access to the bucket and key prefix, not more retries. For an S3-compatible service, set its endpoint; allow_http is only needed for a plain-HTTP endpoint. See delivery behavior, destination keys, and destination.