📘Release History
V1.69
Created: 2024-11-28
added the ability to configure the batch limit for bulk inserts
fixed issue in outbound sync where multiple order by clauses were not being adhered to
updated logging so all plugin messages appear with plugin. prepended to log messgaes
V1.68
Created: 2024-11-26
changed data types ntext and text to nvarchar(max) for outbound syncs
updated inbound key fields so they can be manually set when no indexes are available
updated inbound sync strategies so a full refresh is the only option when there are no date/time fields available or rowversion fields
added ability to use non clustered indexes which allows to multi thread the retrieval of data from tables with no clustered indexes or tables which have columnstore indexes and have a non clustered index on the fields to be used for multi threaded data retrieval.
consolidated how data types are determined for outbound syncs ddl creation.
V1.67
Created: 2024-11-22
bumped runtime version to v0.8.0
fixed outbound issue when there are composite primary keys
V1.66
Created: 2024-11-21
added additional connection option to disable encryption if necessary
modified outbound to take into account change from merge to append in source data
modified outbound to create varbinary columns based on the column lengths as defined by the source
downgraded some log levels from info to debug to reduce noise in the logs
V1.65
Created: 2024-11-19
fixed issue with varbinary not being used correctly by outbound sync
fixed issue around timestamps and image fields being incorrectly set for outbound syncs
added ability to prevent stage table from being truncated after a sync for debug purposes
V1.64
Created: 2024-11-18
bumped devkit version number
V1.63
Created: 2024-11-18
Added support for privatelink and direct connections via the new Connectivity Options flags Fix for ngrok connections under new runtime
modified outbound datatypes
V1.61
Created: 2024-11-12
updated plugin runtime version to 0.6.9
V1.60
Created: 2024-11-12
ensured keys are not null for outbound syncs when creating tables
V1.59
Created: 2024-11-12
made columns nullable for outbound sync schema creation
added sync date/time to outbound sync
V1.58
Created: 2024-11-11
fixed alter statements for outbound syncs
V1.57
Created: 2024-11-08
updated outbound to take into account binary fields
updated json schema to correctly handle towversions and timestamps as strings
V1.56
Created: 2024-11-07
fixed check if a condition criteria is a digit or a string
V1.55
Created: 2024-11-06
added procedure which gathers the details required for a given inbound sync
removed sys schema from watermark schema selection
added ability to specify deleted_fields inside outbound sync for soft delete capabilities
V1.54
Created: 2024-10-31
fixed outbound sync issue when alerting tables on destination server
V1.53
Created: 2024-10-31
added foundations of Private Link capabilities
fixed bug associated with view lookups
V1.52
Created: 2024-10-27
modified outbound sync so it will include multiple batches in a single run instead of each value in a cursor field.
V1.51
Created: 2024-10-23
updated outnbound sync processing of order by clauses to accept with a string or a list of strings
fixed issue when retreiving the min and max lsn
V1.49
Created: 2024-10-16
added Change Tracking as a Sync Method
modified log messages to utilize the extra data field
V1.48
Created: 2024-10-15
fixed casting issues for outbound sync
V1.47
Created: 2024-10-15
fixed minoir issue with outbound sync
V1.46
Created: 2024-10-14
tidied up connection failure messages
outbound: added in database compatability level detection for outbound sync schemas
inbound: no longer process streams where there is no data to be retreived for an incremental run
inbound: fixed issue for high water mark initial fetch of data
V1.45
Created: 2024-10-09
exposed additional function to execute commands against sql server
updated outbound sync properties for Monitorial Outbound Sync Native App
V1.44
Created: 2024-10-05
modified visibility of procedure 'run_outbound_sync'
V1.43
Created: 2024-10-02
updated cdc table look up to use non clustered indexes as well as clustered indexes
added experimental outbound sync functionality
removed Entra ID Password as an auth mechanism due to login issues around server names
V1.42
Created: 2024-09-30
fixed an issue where inbound syncs would experience intermitted transaction-related errors
added in missing microsoft-authentication-library-for-java package
updated error messages for connection tests so errors are clearer
V1.41
Created: 2024-09-25
updated docs for Azure Database Configuration
added square brackets around column names for the initial cdc full refresh sync
modified logging levels
V1.40
Created: 2024-09-06
Updated to new plugin runtime which publishes consumer UDF information to the sync engine
Reduced the size of the bundled jars
Upgraded some java dependencies to newer versions
Reduced logging from info for debug
V1.39
Created: 2024-09-03
added additional isolation level to allow for server defaults
V1.35
Created: 2024-09-03
changed devkit version to 0.5.4
V1.34
Created: 2024-09-03
fixed issue where incorrect property was being used for retries
V1.33
Created: 2024-09-02
fixed issue where the statements being issued were being concatenated together for a retry
V1.32
Created: 2024-09-02
added in the ability to set transaction isolation levels
added in additional checks for error codes to force retry due to SQL execution canceled during records being committed
V1.31
Created: 2024-08-21
modified hwm path for how the initial upfront data is returned when a view is not indexed so not to cause timeouts
added check to cdc path to ensure the min lsn is still valid at the start of the run, if not then sets the min lsn to be the earliest available
V1.30
Created: 2024-08-19
fixed issues whre hwm failed to bring back view information at startup causing initial sync to fail
V1.29
Created: 2024-08-16
removed the need for plugin to need view state permissions at the server level
fixed issue where where clause caused Ambiguous column name error
V1.28
Created: 2024-08-15
modified cdc runs so that fewer queries are issued to speed up sync process
modified hwm runs so that fewer queries are issues to speed up sync processes
V1.27
Created: 2024-08-13
fixed issue where CDC incremental runs were failing
V1.26
Created: 2024-08-13
modified how indexes are filtered out when retreiving table metadata
modified error handling around aync failures to force retries instead of erroring out
V1.25
Created: 2024-08-12
fixed issue where filter clause didn't have quotes around dates or strings
ensured an incremental cdc run doesn't return a null row when the cdc table is empty
modified async job to retry when "Unknown Error" is thrown by snowflake
V1.24
Created: 2024-08-12
Added exception handling when Async Job throws unkown error even after a successful completion
V1.23
Created: 2024-08-09
Fixed an issue where an error was thrown during configuration
V1.22
Created: 2024-08-08
bumped runtime version
modified concurrency so that the max queries is adhered to
V1.21
Created: 2024-08-05
updated initial cdc all records fetch to use seqval instead of startlsn so chuncks are more even
updated concurrency for data pulls from SQL Server to improve performance
V1.20
Created: 2024-08-02
update last-retrieve datetime to use the min sync time instead of the max sync time so no records are missed after the initial sync has been completed
V1.19
Created: 2024-08-02
ensured cdc capture instance is reused if multiple available for the same table
V1.18
Created: 2024-08-01
modified default for concurrent queries from 8 to 2
modified date translation to take into account Snowflake formatted date which originates from SQL Server
ensured CDC table exists when obtaining the current active CDC table
V1.17
Created: 2024-07-31
fixed issued with sync_timestamp returning null when setting state
V1.16
Created: 2024-07-17
bumped version number
add support for pre SQL Server 2017
V1.15
Created: 2024-07-04
added support for TimeStamp and RowVersion Cursor Field
modified retreive timestamp to be the sync_timestamp from SQL Server instead of Snowflake generated timestamp
updated high watermark to use full refresh path when the sync_strategy has been set to full refesh
added additional error information when the data returned doesn't include the required key fields
updated documentation
updated version numbers of omnata packages
V1.14
Created: 2024-06-12
fixed error when retreiving certificate
V1.13
Created: 2024-06-11
Separated out connection methods
SQL Server Authentication (on-premisis or Azure based)
Microsoft Entra Password (Azure SQL Database, Azure SQL Managed Instance)
Microsoft Entra Service Principal (Azure SQL Database, Azure SQL Managed Instance)
V1.12
Created: 2024-06-11
Used latest plugin devkit, which addresses an issue where errors occur when editing connections
Fixed an issue where the connection test did not actually execute a SQL statement
V1.11
Created: 2024-05-27
when a cdc columns change, a sync retreives the new set of columns and resets the state
cdc stream selection now correctly returns streams where net changes are requested
high watermark stream selection now displays indexed as well as heaped tables
V1.10
Created: 2024-05-26
FIltered Out Tables which don't support net changes if only net changes are required
Ensured when net changes are not supported that the all changes are retreived
Each sync queries sql server directly instead of using metadata on retreived on initial connection
When running a cdc sync, the columns for the capture instance are retreived instead of what was available on initial setup.
V1.9
Created: 2024-05-21
Update SQL DataTypes to correctly specify schema definition for datetime2, datetimeoffset, smalldatetime, money, smallmoney, and real
Modified Cursor Data when the cursor field is of type rowversion to include 0x at the start
CDC Streams can now only be incremental
If row_version field is defined, for non CDC syncs, the rowversion field is selected by default as the cusor field
V1.8
Created: 2024-05-19
Updated SQL DataTypes to correctly specify the schema definition for datetime2, datetimeoffset, smalldatetime, money, smallmoney and real
Expanded the list of column datatypes which can be used for a cursor field check to include datetime2, smalldatetime, datetimeoffset
V1.7
Created: 2024-05-18
increased plugin runtime to 0.3.26
Added billing events for ngrok tunnels
V1.6
Created: 2024-05-17
updated how indexed data is retreived for full data loads where the index is non integer based.
set the default cursor if a modified to create date is found for high watermark incremental merge based syncs
V1.5
Created: 2024-05-13
SQL Server Certificate Retreival Fix
V1.4
Created: 2024-05-13
Initial release of MSSQL plugin, supports inbound syncing using the following techniques:
Change Data Capture (CDC Tables)
Highwater mark (Tables and Views)
V1.3
Created: 2024-04-02
Switched to new plugin-oriented billing mechanism
Bump plugin runtime version
Built with omnata-plugin-devkit==0.3.17 which includes pinned stored proc dependencies
Last updated