Internal tables

Omnata exposes read-only views of some of its internal tables for the convenience of customers.

You can find these views in the OMNATA_SYNC_ENGINE.DATA_VIEWS database/schema.

CONNECTION

This table stores connections that have been fully configured.

It has the following columns:

CONNECTION_ID

The unique ID number for the connection, used as a stable identifier internally.

CONNECTION_NAME

The name of the connection.

CONNECTION_METHOD

The authentication method used to connect to the external application. This value is defined by the plugin and may impact which connection parameters are present.

CONNECTION_SLUG

A unique identifier for the connection, used externally as a human-readable stable identifier. Note that this value is allowed to change.

IS_PRODUCTION_ENVIRONMENT

Whether the connection is to a production environment. This is a choice made by the user during connection configuration and impacts scheduling and branching functionality.

PLUGIN_FQN

The fully qualified name of the plugin that this connection is for.

EXTERNAL_ACCESS_INTEGRATION_NAME

The name of the connection's external access integration object. This value may change if the connection is edited.

OAUTH_SECRET_NAME

The name of the Snowflake Secret object that contains the OAuth secret for this connection, if OAuth is used. This value may change if the connection is edited.

OTHER_SECRETS_NAME

The name of the Snowflake Secret object that contains other secrets for this connection. This value may change if the connection is edited.

NETWORK_RULE_NAME

The name of the Snowflake Network rule object that contains the permitted addresses for this connection (addresses are determined by the plugin code using parameter values). This value may change if the connection is edited.

OAUTH_SECURITY_INTEGRATION_NAME

The name of the Snowflake security integration object for this connection, if OAuth is used. This value may change if the connection is edited.

OUTBOUND_SYNC_RECORD_STATE

OUTBOUND_SYNC_RECORD_STATE_ID

The unique ID number for the outbound sync record state, used as a stable identifier internally.

SYNC_ID

The ID of the sync that this record belongs to.

SYNC_BRANCH_ID

The ID of the branch that this record belongs to, or null if it belongs to the main sync.

IDENTIFIER

The unique identifier for the record, which is the value from the source table's ID column.

APP_IDENTIFIER

The identifier for the record in the external application, if set during a previous sync operation.

TRANSFORMED_RECORD

The transformed record, which is the result of applying the record transformer to the source table's records.

RECORD_STATE

The last observed state of the record in the source table.

See docs page for details.

APPLY_STATE

The state of the apply operation (indicates success or otherwise of the sync action).

See docs page for details.

FAILURE_COUNT

The number of times the record has failed to apply

SYNC_ACTION

The name of the Sync Action to take in the external system, based on the sync strategy and the source record state. This can be a standard action (like 'Create', or a custom action). It is up to the plugin to know how to handle all of the possible Sync Actions it supports.

RECORD_STATE_DATETIME

The datetime when the record state was last updated.

APPLY_STATE_DATETIME

The datetime when the apply state was last updated.

LAST_RUN_ID

The ID of the last sync run that processed this record.

LAST_RUN_APPLY_REASON

The reason for the last apply operation.

See docs page for details.

LAST_RESULT

The result of the last apply operation. This is a dictionary that can contain any information about the result of the apply operation. It is intended to be understood by a human troubleshooting a sync error.

SYNC

Represents a configuration and schedule to sync between Snowflake and another application.

SYNC_ID

The unique ID number for the sync, used as a stable identifier internally

SYNC_NAME

The name of the sync.

SYNC_SLUG

A unique identifier for the sync, used externally as a human-readable stable identifier.

CONFIGURATION_MODE

The configuration mode of the sync, either 'simple' or 'advanced'. Advanced mode is presented in the UI as "branching mode".

RUN_STATE

Also known as Activity State, this describes the syncs state in terms of activity (Pending, Running, Waiting, Paused).

HEALTH_STATE

Describes the Sync's state from a success perspective (Health, Failed, Incomplete, etc).

Note that if a sync run is currently in progress, this will reflect the health state at the end of the previous run.

CONNECTION_ID

The ID of the connection to use for the sync. If null, it means the Sync was created as a branch using a non-production connection.

PLUGIN_FQN

The fully qualified name of the plugin to use for the sync.

SYNC_DIRECTION

The direction of the sync, either 'inbound' or 'outbound'.

SYNC_PARAMETERS

The parameters for the sync as defined during configuration.

SYNC_SCHEDULE

The schedule for the sync, the structure of this varies depending on the scheduler used.

OUTBOUND_SYNC_STRATEGY

The strategy to use for outbound syncs. This object describes the strategy and also the associated sync actions.

OUTBOUND_SOURCE_DATABASE

The name of the source database for the outbound sync, if the sync is outbound.

OUTBOUND_SOURCE_SCHEMA

The name of the source schema for the outbound sync, if the sync is outbound.

OUTBOUND_SOURCE_TABLE

The name of the source table for the outbound sync, if the sync is outbound.

OUTBOUND_SOURCE_ID_COLUMN

The name of the source ID column for the outbound sync, if the sync is outbound.

OUTBOUND_FIELD_MAPPINGS

Maps source columns to target fields, if the sync is a record-style outbound sync.

INBOUND_STREAMS_CONFIGURATION

The configuration for the inbound streams, if the sync is an inbound sync.

INBOUND_LATEST_STREAMS_STATE

The latest state for each stream, if the sync is an inbound sync. Used to manage incremental syncing over time.

INBOUND_LATEST_STREAMS_SCHEMA

The latest json-schema for each stream, if the sync is an inbound sync. Used to build normalized views over the raw data.

LATEST_RUN_START

The datetime when the latest run started.

LOGGING_LEVEL

The logging level for the sync, determines how much information is logged in the event table.

This column is deprecated and may be removed in a future release.

INBOUND_STORAGE_CONFIGURATION

The storage configuration for sync, if the sync is an inbound sync. This object determines where to store the inbound raw tables and normalized views.

SYNC_BRANCH

Represents a branch of a sync, if configuration mode is 'advanced'. Sync branches are used to test changes in a non-production environment or limited subset of production, before merging them into the main sync.

SYNC_BRANCH_ID

The unique ID number for the sync branch, used as a stable identifier internally.

SYNC_ID

The ID of the sync that this branch belongs to

BRANCH_NAME

The name of the branch

OUTBOUND_RECORD_STATE_BEHAVIOUR

Determines how the outbound record state is initially populated.

• START_EMPTY - All records will be re-synced. You would do this if you wanted to deliberately re-populate an empty environment.

• FORK - The sync will continue where it left off in production, including updating/deleting already sync'd records. You would do this if your environment contained an up-to-date copy of production data.

• FORK_AND_IGNORE - The sync will continue where it left off in production, but updates/deletes to existing records will be ignored. You would do this if your environment is empty or does not contain the same data as production, but you don't want to fully re-populate it. For sync strategies which only action new records (usually event-based or messaging apps), this will behave identically to 'Fork current state'.

• SHARE - The branch sync will run in concert with the main sync, so that changes applied in the branch will not be reapplied in the main sync, and vice versa. You would do this if you had no other place to send test records to, and need to test in production. If you select this option, you must use a branch record filter to limit the records synced in the branch.

OUTBOUND_BRANCH_RECORD_FILTER

If provided, means that only the records with these identifiers will be included in the sync. These records will also be excluded from the main sync while ever the branch is active

INBOUND_RECORDS_BEHAVIOUR

Determines how the inbound records raw table is initially populated.

• START_EMPTY - The inbound raw tables will start empty

• FORK - The inbound raw tables will be copied from the main sync

INBOUND_STREAM_STATE_BEHAVIOUR

Determines how the inbound stream state is initially populated

• START_EMPTY - The stream states will start empty

• FORK - The stream states will be copied from the main sync

REOPEN_BEHAVIOUR

Determines how the branch records and state are initialised when it is reopened.

• CONTINUE - The branch records/state will continue on from last time it was used

• REAPPLY - The branch records/state will be reapplied from the main sync, as per defined behaviour

RUN_STATE

Also known as Activity State, this describes the syncs state in terms of activity (Pending, Running, Waiting, Paused)

HEALTH_STATE

Describes the Sync's state from a success perspective (Health, Failed, Incomplete, etc)

CONNECTION_ID

The ID of the connection to use for the sync branch. This must be a non-production connection.

SYNC_DIRECTION

The direction of the sync, either 'inbound' or 'outbound'. This field is not used, as syncs cannot change direction after creation.

SYNC_PARAMETERS

The parameters for the sync as defined during configuration.

OUTBOUND_SYNC_STRATEGY

The strategy to use for outbound syncs. This object describes the strategy and also the associated sync actions. Currently this cannot be changed from the main sync setting, but may be supported in future

OUTBOUND_SOURCE_DATABASE

The name of the source database for the outbound sync branch, if the sync is outbound.

OUTBOUND_SOURCE_SCHEMA

The name of the source schema for the outbound sync branch, if the sync is outbound.

OUTBOUND_SOURCE_TABLE

The name of the source table for the outbound sync branch, if the sync is outbound.

OUTBOUND_SOURCE_ID_COLUMN

The name of the source ID column for the outbound sync branch, if the sync is outbound.

OUTBOUND_FIELD_MAPPINGS

Maps source columns to target fields, if the sync is a record-style outbound sync.

INBOUND_STREAMS_CONFIGURATION

The configuration for the inbound streams, if the sync is an inbound sync.

INBOUND_LATEST_STREAMS_STATE

The latest state for each stream, if the sync is an inbound sync. Used to manage incremental syncing over time, but only in the branch.

INBOUND_LATEST_STREAMS_SCHEMA

The latest json-schema for each stream, if the sync is an inbound sync. Used to build normalized views over the raw data.

BRANCH_STATE

The state of the branch, either 'draft' or 'activated', 'merged' or 'abandoned'.

LAST_OPEN_DATETIME

The datetime when the branch was last opened.

LAST_CLOSE_DATETIME

The datetime when the branch was last closed.

LATEST_RUN_START

The datetime when the latest run started.

INBOUND_STORAGE_CONFIGURATION

The storage configuration for sync, if the sync is an inbound sync. This object determines where to store the inbound raw tables and normalized views.

SYNC_RECORD_STATE_HISTORY

A long term audit table which tracks all inbound and outbound record changes over time. This table is subject to retention policies.

SYNC_RECORD_STATE_HISTORY_ID

The unique ID number for the sync record state history, used as a stable identifier internally.

SYNC_ID

The ID of the sync that this record belongs to.

SYNC_BRANCH_ID

The ID of the branch that this record belongs to, or null if it belongs to the main sync.

CONNECTION_ID

The ID of the connection that this record belongs to.

SYNC_RUN_ID

The ID of the sync run that this record belongs to.

IDENTIFIER

The unique identifier of the record.

EVENT_DATETIME

The datetime of the event.

RECORD

The record content.

OUTBOUND_SYNC_ACTION

The action taken, if this is an outbound sync.

OUTBOUND_RECORD_STATE

The state of the source record, if this is an outbound sync.

OUTBOUND_APPLY_STATE

The apply state, if this is an outbound sync.

OUTBOUND_APPLY_REASON

The reason the record was included in the sync, if this is an outbound sync.

INBOUND_STREAM_NAME

The name of the inbound stream, if this is an inbound sync.

INBOUND_IS_DELETED

Whether the record was deleted, if this is an inbound sync.

OUTBOUND_TRANSFORMED_RECORD

The transformed record, if this is an outbound sync.

OUTBOUND_RESULT

The result of the record being sync'd, if this is an outbound sync.

SYNC_RUN

Represents a Sync Run, which is a single execution of a sync triggered by some scheduler.

SYNC_RUN_ID

The unique ID number for the sync run, used as a stable identifier internally.

SYNC_ID

The ID of the sync that this run belongs to.

SYNC_BRANCH_ID

The ID of the branch that this run belongs to, or null if it belongs to the main sync.

RUN_START_DATETIME

The datetime when the sync run started.

RUN_PREPROCESSING_START_DATETIME

The datetime when the sync run preprocessing started.

RUN_PLUGIN_START_DATETIME

The datetime when the sync engine invoked the plugin in order to perform sync operations.

RUN_POSTPROCESSING_START_DATETIME

The datetime when the sync run postprocessing started.

RUN_END_DATETIME

The datetime when the sync run ended.

RUN_DEADLINE_DATETIME

The datetime when the sync run must be completed by, before cancellation occurs.

HEALTH_STATE

The health state of the sync run.

INBOUND_TOTAL_COUNT

The total number of records in the sync run, if this is an inbound sync. This is calculated after the storage behaviour is applied, so may be less than the number of records retrieved from the remote system.

INBOUND_NEW_COUNT

The number of records in the sync run which were newly seen, if this is an inbound sync.

INBOUND_CHANGED_COUNT

The number of records in the sync run which were changed from their previous value, if this is an inbound sync.

INBOUND_DELETED_COUNT

The number of records in the sync run which were deleted, if this is an inbound sync.

INBOUND_STREAM_TOTAL_COUNTS

The total number of records per stream, if this is an inbound sync. This is calculated after the storage behaviour is applied, so may be less than the number of records retrieved from the remote system.

INBOUND_STREAM_NEW_COUNTS

The number of records per stream which were newly seen, if this is an inbound sync.

INBOUND_STREAM_CHANGED_COUNTS

The number of records per stream which were changed from their previous value, if this is an inbound sync.

INBOUND_STREAM_DELETED_COUNTS

The number of records per stream which were deleted, if this is an inbound sync.

INBOUND_COMPLETED_STREAMS

The names of the streams which have completed, if this is an inbound sync.

INBOUND_SUCCESSFUL_STREAMS

The names of the streams which were successful, if this is an inbound sync.

INBOUND_ERRORED_STREAMS

The names of the streams which errored, if this is an inbound sync.

INBOUND_ABANDONED_STREAMS

The names of the streams which were abandoned, if this is an inbound sync.

INBOUND_CANCELLED_STREAMS

The names of the streams which were cancelled, if this is an inbound sync.

OUTBOUND_INCLUDED_RECORDS

The records which were included in the sync, if this is an outbound sync and a record filter applies.

OUTBOUND_EXCLUDED_RECORDS

The records which were excluded from the sync, if this is an outbound sync and a record filter applies.

OUTBOUND_TOTAL_COUNT

The total number of records in the sync run, if this is an outbound sync.

OUTBOUND_APPLY_REASON_COUNTS

The number of records in the sync run by apply reason, if this is an outbound sync.

OUTBOUND_APPLY_STATE_COUNTS

The number of records in the sync run by apply state, if this is an outbound sync.

CONNECTION_ID

The ID of the connection to use for the sync.

OUTBOUND_SOURCE_DATABASE

The name of the source database, if this is an outbound sync. Some schedulers, such as dbt, may directly set this value when they initiate a run.

OUTBOUND_SOURCE_SCHEMA

The name of the source schema, if this is an outbound sync. Some schedulers, such as dbt, may directly set this value when they initiate a run.

OUTBOUND_SOURCE_TABLE

The name of the source table, if this is an outbound sync. Some schedulers, such as dbt, may directly set this value when they initiate a run.

OUTBOUND_SOURCE_ID_COLUMN

The name of the ID column in the source table, if this is an outbound sync.

OUTBOUND_SYNC_STRATEGY

The strategy to use for outbound syncs. This object describes the strategy and also the associated sync actions.

CANCELLED_DATETIME

The datetime when the sync run was cancelled, or null if it wasn't cancelled.

GLOBAL_ERROR

The global error message, if the sync run failed in a catastrophic way and per-record or per-stream errors are not available.

INBOUND_GLOBAL_ERROR_BY_STREAM

The error message for each failed stream, if this is an inbound sync.

INBOUND_STREAMS_STATE_START

The initial state for each stream, if this is an inbound sync.

INBOUND_STREAMS_STATE_END

The final state for each stream, if this is an inbound sync.

INBOUND_STREAMS_STATE_ACTIVE

The active state for each stream, if this is an inbound sync.