📘Release History
V1.114
Created: 2024-07-25
Allowed plugin development sessions to have their sync run closed off without providing results
Outbound syncs now upload result objects as JSON, to avoid issues with the arrow serializer. This is enabled in plugin runtime 0.4.10 and newer
V1.113
Created: 2024-07-24
Fixed an issue where an error was displayed on the outbound records tab after record updates occurred
V1.112
Created: 2024-07-23
Fixed an issue where the hover text on the Syncs list did not show the name of the connection
Provided a process to choose the admin tasks warehouse again after initial onboarding
V1.111
Created: 2024-07-23
Added additional checks to ensure no ngrok API errors include API tokens
Modified the SYNC and SYNC_BRANCH data views to include more configuration information
V1.110
Created: 2024-07-21
Fixed an issue where Snowflake account names containing underscores would break the ngrok tunnel configuration process
Adding multiple ngrok connections will now reuse the same auth token to simplify agent configuration
V1.109
Created: 2024-07-19
When changing the outbound sync strategy during initial configuration, the form definition is reloaded in case there are variations between strategies
Outbound syncs will now have access to the previous state of an updated record, in case there are situations where the plugin needs to know exactly what changed between syncs
Errors during stream listing are now handled gracefully and can be retried
V1.108
Created: 2024-07-15
Syncs can no longer be deleted if other sync's schedules depend on them
Fixed a Streamlit error that can occur when rendering outbound records table
Changed the behaviour of inbound syncs, so that full refresh+append streams can have previously deleted records restored back to an undeleted state
Errors that occur while populating form data sources from plugins, are now caught and displayed within the form boundary
Fixed some Streamlit dependencies who's pinned versions had been removed from Anaconda
V1.107
Created: 2024-07-10
Improved handling of transient transaction-related errors when syncing
V1.106
Created: 2024-07-09
Changed inbound stream lister to ignore source defined primary keys if it's set to an empty list
Added a button to refresh the fields list for primary key/cursor selection
When saving inbound configuration, store any newer json schemas in case the sync is edited again before the next sync
V1.105
Created: 2024-07-08
Modified PLUGIN_CONNECTION to only return connection metadata for completed connections. This ensures that consumer UDFS do not try to use partially completed connections
V1.104
Created: 2024-07-08
Fixed an issue where quoted columns in outbound syncs caused an error during record staging
V1.103
Created: 2024-07-05
Add retries to the plugin message receiver on the Sync Engine
Modified stream configuration UI to always require a primary key, even with the Append storage behaviour
V1.102
Created: 2024-07-03
Fixed an issue where inbound syncs were able to be created without all streams having a ID Field chosen
V1.101
Created: 2024-07-03
Modified the function which fetchings connection information for consumer functions, to make it more compatible with UDTFs
For inbound syncs, reduced the amount of concurrent post-processing work to reduce transaction errors
Fixed an issue where viewing inbound sync records in the UI caused an error
V1.100
Created: 2024-07-02
Fixed an issue with the outbound field mapper where removal of a column in the source table which was already mapped to an optional column yielded an error
Trace events in Streamlit are now logged asyncronously
V1.99
Created: 2024-07-01
Fixed an issue where outbound syncs using the Create strategy, don't retry after initial failures using the latest record value
V1.98
Created: 2024-06-29
Fixed an issue where parsing a list of tables in some environments would raise an error
V1.97
Created: 2024-06-28
Fixed an issue where inbound storage locations that are already pre-quoted would be quoted again
Fixed an issue where errors streams may not display in the progress accordion of the sync run detail page
Added a "check again" button to the outound source table selector, to re-check for duplicate ID column values
Fixed an issue for the inbound sync run view, where some in-progress streams were not visible
Handled the scenario where the inbound record count exceeds the initial estimate
V1.96
Created: 2024-06-24
Fixed an issue where expired plugin trials could break the plugin selector
V1.95
Created: 2024-06-24
Plugins reporting total record estimates for streams now show a progress bar in the UI
Fixed an issue for inbound syncs, where default inbound storage locations would fail to delete tables when the sync is deleted
Using async stored procs has resulted in occasional transaction-related errors that need retrying. We now retry and eventually ignore these during the plugin execution, but raise an error if it's still occurring at the final commit
Fixed an issue where inbound streams may show as both "completed" and "successful"
V1.94
Created: 2024-06-23
Fixed issue where outbound staging duplicate detection is not isolated by sync branch
Fixed an inbound sync error affecting the most recent plugin runtime
New async results processing now only occurs on plugin runtime 0.4.2, where results are guaranteed to be uploaded before stream status updates
V1.93
Created: 2024-06-22
Inbound syncs now have their deadline and cancellation monitored by the sync engine (thanks to the new async query support)
Post-processing of completed streams now happens during the sync run, so that cancelled/abandoned syncs save progress more reliably
Plugins can now report a total count of records ahead of time
Update inline product help
Fixed an issue where editing existing syncs could result in the inbound storage location being reset to the default
Outbound record staging now detects duplicate identifiers prior to the record state being updated. This reduces the need for manual record cleanup if you accidentally introduce duplicates in your source table
V1.92
Created: 2024-06-18
Added a workaround for a streamlit bug which affects the stream lister
V1.91
Created: 2024-06-17
Removed some excess logging
Fixed an issue introduced by the previous release
V1.90
Created: 2024-06-17
Inbound storage location can now include the type of table to use: standard (with or without change tracking) or hybrid
Changing the storage location copies any existing data to the new location, and also removes the old table
V1.89
Created: 2024-06-11
Fixed an issue where ngrok connection edits did not check for pending API configuration changes before proceeding to the final connection form
V1.88
Created: 2024-06-08
Switched Streamlit record selectors from experimental_data_editor to data_editor
V1.87
Created: 2024-06-08
Added logging to the stream lister component to determine whether users have deliberately toggled a stream
Changed the way the app artifact is added to the system path to improve Streamlit cycle times
Reduced the size of the packaged codebase (both streamlit and stored libraries)
V1.86
Created: 2024-06-07
Fixed an issue where an abandoned notification recipient configuration causes a persistent error on the settings page
V1.85
Created: 2024-06-02
UI now uses Streamlit 1.26.0. Inbound stream lister now uses toggles instead of checkboxes
V1.84
Created: 2024-06-01
Updated inline documentation for inbound syncs
Plugin registrations can now be deleted from the sync engine UI following the uninstall of a plugin application
V1.83
Created: 2024-05-24
A default custom storage location for inbound syncs can now be configured globally
V1.82
Created: 2024-05-21
Changed the behaviour of dynamic-valued drop-downs so that the first option is not auto-selected. This prevents long initial waits for forms that progressively load options from a remote source
Provided ability for users to bring their own GPG/SSH keys during connection creation
When prompting the user to deploy the ngrok agent, the address and port are now prompted for, and the specific cli command or service configuration file are provided
V1.81
Created: 2024-05-17
Snowflake connector/snowpark logging level is now set to WARN by default, to reduce noise in event tables. It can be increased for individual customers via a shared table parameter, up to the level configured for the native app
Any custom snowflake logging level is applied to plugin apps using runtime 0.3.25 or newer
Fixed issue where stream configuration screen could error when editing a sync
Added monthly billing events for ngrok tunnels
V1.79
Created: 2024-05-06
Added logging for sync request payloads
V1.78
Created: 2024-05-02
For inbound streams, improved handling of default primary key fields when they are composite
V1.77
Created: 2024-05-02
Slider fields now have their step size correctly configured to match what was set by the plugin
Added support for composite id fields on inbound streams
V1.76
Created: 2024-04-27
Fixed error that can occur when selecting a cursor field for an inbound sync
Connections using ngrok now have the agent launch command available when you view the connection details
V1.75
Created: 2024-04-23
Fixed an issue where id-only deletions in run results resulted in an empy record rather retaining the previous value of the record
Added cursor field and primary key field to the sync home page summary
V1.74
Created: 2024-04-22
Disabled client side encryption on plugin results stage, so that results can be streamed
If plugin application has been uninstalled, show a warning message when creating a new connection
V1.73
Created: 2024-04-17
Plugins are now provided with an internal stage to upload run results to
V1.72
Created: 2024-04-17
Fixed an error where ngrok resources are not cleaned up when connections are deleted
Fixed error that may occur when editing connections with ngrok tunnels
Removed the requirement that plugins return a list of errored streams from the SYNC_INBOUND operation. This simplifies deferring the operation to other languages like java.
V1.71
Created: 2024-04-10
ngrok tunnel limits can now be increased centrally by Omnata
V1.70
Created: 2024-04-10
Fixed an issue where an error occurs if an inbound stream does not report state before ending
Added support for ad-hoc consumer-facing functions that can be shipped with plugins and invoked using a connection slug
Fixed an issue where an error can occur when customizing streams
Fixed an issue where ngrok-based connections may display an error during initial creation
V1.69
Created: 2024-04-03
If the native app doesn't own an inbound raw table, it will no longer error on configuration save when attempting to run grant statements
V1.68
Created: 2024-03-28
When hovering on the help text for a custom cron schedule, a human readable description is now shown instead of the cron expression
All streamlit dependencies are now pinned to specific package versions, to ensure Anaconda availability
V1.67
Created: 2024-03-27
Fixed an issue where outbound sync pre-flight checks would fail due to source table not being quoted correctly
Provided a fallback configuration option for new streams when the plugin runtime is less than 0.3.19
Provided a maintenance process which deletes Sync Record State History older than a configurable number of days, to reduce storage costs at high record volume
Fixed an issue where outbound syncs could not have record transformation applied due to table names not being quoted
Created all Sequences with the ORDER attribute set, to ensure identifiers increment
Included the application's own database in the list of outbound source databases
Reverted the UI back to Streamlit 1.22.0
V1.66
Created: 2024-03-25
Fixed an issue where results deduplication did not work correctly for inbound syncs
Fixed an issue where the append storage behaviour would create duplicate records
V1.65
Created: 2024-03-25
During post run processing, added some exception logging logic so that transaction rollback errors don't cause the original error to be lost
V1.64
Created: 2024-03-22
On the Syncs list, custom schedules now display the cron string without needing to hover on the help text
Inbound syncs no longer offer a choice of Incremental or Full Refresh unless you choose the Customize option. Instead, Auto mode will choose incremental where possible, and fall back on full refresh.
Storage behaviour is still a choice, but it is now "Merge changes" or "Keep history" in the UI and an explanatory diagram is included.
For full refreshes, the term "Replace" has been deprecated in favour of "Merge"
Upgraded the UI to Streamlit 1.26.0
Fixed streamlit warning on pre-populated checkboxes on sync configuration forms
V1.63
Created: 2024-03-21
Syncs list now has a column for Sync Schedule
Fixed issue where multi-select dropdown with required value allowed the user to continue without any values selected
Added some extra columns to the SYNC_RUN view, relating to inbound record counts
Fix issue where deletion by criteria with the append storage behaviour could append multiple delete records for the same identifier
Added a button on the inbound sync run screen to refresh the list of streams
V1.62
Created: 2024-03-19
The total number of inbound records for a run is now logged pre and post deduplication. A warning is logged if deduplication removed records.
Fixed an issue with the new deletion functionality, where the new sync engine version was not compatible with older runtime versions
Fixed an issue where non-empty inbound runs were hidden by the "Hide empty completed runs" checkbox
V1.61
Created: 2024-03-17
After clearing state for an inbound sync, the table is truncated when applying the results for the next run
V1.60
Created: 2024-03-15
The "Append" inbound storage behaviour now only stores changed records, making it much more storage efficient when used with the Full Refresh strategy.
Inbound sync tables now contain a flag which indicates when the record has been deleted at the source. When using the Full Refresh strategy, this is maintained automatically via the absence of previous records.
Inbound sync tables now contain the run id that the most recent change for each record belongs to.
Plugins can now provide individual record deletions, and also deletions by criteria for inbound incremental syncs. This requires omnata-plugin-runtime version 0.3.15 or greater.
When processing results of inbound syncs at the end of the run, the engine will commit the data and latest state of each stream individually, so that if errors are encountered then progress is not lost.
When a sync runs for longer than its configured limit, it is assumed failed and can be re-ran. The previous run will be marked as failed when this occurs.
Sync run id is now included in event span attributes, to make it easier to filter event table queries
When comparing branch stream configurations, the stream state is no longer included in the comparison.
Fixed issue with Sync History tab where incomplete outbound syncs did not render correctly
Inbound stream cursor field now has the default value applied to it at initial sync creation, so that plugins don't need to check both values
Fully deprecated the internal Sync Run Log table in favour of event tables
V1.59
Created: 2024-03-07
Added the PLUGIN_UPDATE_API_CONFIGURATION stored procedure, which can reset the integrations and secret bindings for plugin udfs/procedures
Fixed Streamlit error when switching between full and incremental inbound sync strategy for a single stream
Provided a button in the Advanced area of Settings which re-binds all plugin integration and secret objects
Ensure default cursor field is applied for incremental syncs
Fixed issue where Streamlit could not show information about an SSL certificate because the pyopenssl library was no longer available
V1.58
Created: 2024-03-06
Fixed streamlit UI error on sync home screen when production connection is not yet configured for inbound syncs in branch mode
Fixed issue when nominating a production connection for inbound syncs
When presenting dbt model configuration for streams, handled the scenario where JSON schema is not available
Inbound stream schemas are now fetched when syncs or branches are created or updated, so that the schema is visible in the UI prior to the first run
V1.57
Created: 2024-03-06
In normalized views, object fields are now supported as columns
V1.56
Created: 2024-03-05
Fixed issue where refreshing inbound stream schemas manually resulted in an error
V1.55
Created: 2024-03-03
This is the first version to contain release notes.
Changes in this version:
Reset page number when stream search changes
Killing sync runs no longer attempts to kill the Snowflake query
Added a refresh button to the Syncs page
Inbound sync history is now paginated
Added a refresh button to the Connections page
Fixed error when recreating views for inbound syncs
Removed sync engine billing from daily billing run in preparation for switch to plugin-only billing
Added consumer facing views of internal app state
Last updated