MongoDB
Connect MongoDB to sanda: what to prepare, the fields the connection form asks for, and how its data lands in your warehouse.
Connect MongoDB
- Open the catalogue. In the console, go to Data · Connections and press New connection.
- Choose MongoDB. It is listed under databases; you can also search for it by name.
- Fill in the form. Enter the fields below. Credentials go straight to sanda’s sync engine and are never stored in sanda’s database.
- Choose what to sync. Pick the streams you want, and a sync mode for each. See choose what to sync.
- Run the first sync. sanda tests the connection, then lands each stream as a table in your warehouse.
Configuration fields
What the connection form asks for. Required fields are marked; the rest are optional.
| Field | Type | Required | Description |
|---|---|---|---|
| Cluster Type | one of 2 options | Yes | Configures the MongoDB cluster type. |
| Initial Waiting Time in Seconds (Advanced) | integer | No | The amount of time the connector will wait when it launches to determine if there is new data to sync or not. Defaults to 300 seconds. Valid range: 120 seconds to 1200 seconds. Default 300. |
| Size of the queue (Advanced) | integer | No | The size of the internal queue. This may interfere with memory consumption and efficiency of the connector, please be careful. Default 10000. |
| Document discovery sample size (Advanced) | integer | No | The maximum number of documents to sample when attempting to discover the unique fields for a collection. Default 10000. |
| Document discovery timeout in seconds (Advanced) | integer | No | The amount of time the connector will wait when it discovers a document. Defaults to 600 seconds. Valid range: 5 seconds to 1200 seconds. Default 600. |
| Invalid CDC position behavior (Advanced) | string | No | Determines whether becca should fail or re-sync data in case of an stale/invalid cursor value into the WAL. If 'Fail sync' is chosen, syncs fail until becca support resets the connection. If 'Re-sync data' is chosen, becca will automatically trigger a refresh but could lead to higher costs and data loss. One of Fail sync, Re-sync data. Default Fail sync. |
| Capture mode (Advanced) | string | No | Determines how becca looks up the value of an updated document. If 'Lookup' is chosen, the current value of the document will be read. If 'Post Image' is chosen, then the version of the document immediately after an update will be read. Warning: Severe data loss will occur if this option is chosen and the appropriate settings are not set on your Mongo instance: https://www.mongodb.com/docs/manual/changeStreams/#change-streams-with-document-pre-and-post-images. One of Lookup, Post Image. Default Lookup. |
| Initial Load Timeout in Hours (Advanced) | integer | No | The amount of time an initial load is allowed to continue for before catching up on CDC logs. Default 8. |
Cluster Type
Configures the MongoDB cluster type.
Choose one of the following. Each asks for its own fields.
MongoDB Atlas Replica Set
| Field | Type | Required | Description |
|---|---|---|---|
| Connection String | string | Yes | The connection string of the cluster that you want to replicate. |
| Database Names | array | Yes | The names of the MongoDB databases that contain the collection(s) to replicate. |
| Username | string | Yes | The username which is used to access the database. |
| Password | string, secret | Yes | The password associated with this username. |
| Authentication Source | string | Yes | The authentication source where the user information is stored. Default admin. |
| Schema Enforced | boolean | No | When enabled, syncs will validate and structure records against the stream's schema. Default true. |
Self-Managed Replica Set
| Field | Type | Required | Description |
|---|---|---|---|
| Connection String | string | Yes | The connection string of the cluster that you want to replicate. |
| Database Names | array | Yes | The names of the MongoDB databases that contain the collection(s) to replicate. |
| Username | string | No | The username which is used to access the database. |
| Password | string, secret | No | The password associated with this username. |
| Authentication Source | string | No | The authentication source where the user information is stored. Default admin. |
| Schema Enforced | boolean | No | When enabled, syncs will validate and structure records against the stream's schema. Default true. |
Related
Something unclear or out of date? Tell us, and we will fix the page.