Skip to content
sandadocs
Console Start free trial

S3 / object storage

Connect S3 / object storage to sanda: what to prepare, the fields the connection form asks for, and how its data lands in your warehouse.

View as Markdown

Connect S3 / object storage

  1. Open the catalogue. In the console, go to Data · Connections and press New connection.
  2. Choose S3 / object storage. It is listed under files and storage; you can also search for it by name.
  3. Fill in the form. Enter the fields below. Credentials go straight to sanda’s sync engine and are never stored in sanda’s database.
  4. Choose what to sync. Pick the streams you want, and a sync mode for each. See choose what to sync.
  5. Run the first sync. sanda tests the connection, then lands each stream as a table in your warehouse.

Configuration fields

What the connection form asks for. Required fields are marked; the rest are optional.

Field Type Required Description
Bucket string Yes Name of the S3 bucket where the file(s) exist.
Start Date string No UTC date and time in the format 2017-01-25T00:00:00.000000Z. Any file modified before this date will not be replicated.
Access Key ID string, secret No In order to access private Buckets stored on Amazon S3, this connector requires credentials with the proper permissions. If accessing publicly available data, this field is not necessary.
Secret Access Key string, secret No In order to access private Buckets stored on Amazon S3, this connector requires credentials with the proper permissions. If accessing publicly available data, this field is not necessary.
Endpoint string No Endpoint to an S3 compatible service. Leave empty to use Amazon S3.
Region string No Region where the S3 bucket is located. If not provided, the region will be determined automatically.
Delivery Method one of 2 options No
Role ARN string No Specifies the Amazon Resource Name (ARN) of an IAM role that you want to use to perform operations requested using this profile.
The list of streams to sync array Yes Each item defines a stream: which files belong in it, their format, and how they are parsed and validated. Each stream lands in your warehouse as its own table.

Delivery Method

Choose one of the following. Each asks for its own fields.

Replicate Records

No further fields.

Copy Raw Files

Field Type Required Description
Preserve Sub-Directories in File Paths boolean No If enabled, sends subdirectory folder structure along with source file names to the destination. Otherwise, files will be synced by their names only. This option is ignored when file-based replication is not enabled. Default true.

Something unclear or out of date? Tell us, and we will fix the page.