Outbound

Overall process

If you're syncing multiple things to an app (e.g. Salesforce Contacts and Accounts), each will have its own sync.

A sync record state is maintained for each sync. This is a table in Snowflake which is used to keep track of the state of data in the application. It stores a transformed version of the source table, which includes only the data being sent to the application.

A sync run applies new Snowflake changes to the application. These can be triggered using Omnata's own scheduler (which leverages Snowflake tasks), or via some external system like airflow or dbt.

During a sync run, the following steps occur:

1. The records in the source Snowflake table/view are passed through a transformer, and then compared with the sync record state to determine whether or not any action is required. This takes into account which sync strategy is being used, e.g. an updated Snowflake record is ignored if the sync strategy is 'create'

2. If action is required, the record is included in the sync run

3. If there are records requiring action, the sync engine asks the app's plugin to sync the records

4. The plugin syncs the records. The Omnata plugin framework provides common features like rate limiting, jinja rendering, record batching and concurrent execution

5. For each record the plugin was given, it returns a success flag, a result object (containing arbitrary information or errors), and an App identifier if applicable (an identifier for the record in the app)

6. The sync engine updates the record state, and also adds to the sync record history so that there is a historical audit trail for all records in all sync runs

7. The health status of the sync is updated, based on what happened to the records in the sync run

Outbound strategies

With outbound syncs, record state is maintained locally and different record actions are applied depending on the pattern of changes and the chosen strategy.

All Sync Strategies except for "Replace" treat records differently as they change over time.

"Replace" is a simple strategy where every current record is always included (except where manually excluded). It is intended for apps that do a bulk overwrite of records.

Strategy	Newly created records	Updated records	Deleted records	Unchanged records
Create	Create	Ignored	Ignored	Ignored
Update	Ignored	Update	Ignored	Ignored
Delete	Ignored	Ignored	Delete	Ignored
Upsert	Create	Update	Ignored	Ignored
Mirror	Create	Update	Delete	Ignored
Replace	Recreate *	Recreate *	Ignored	Recreate *

* Replacing any existing records.