📘Release History


V1.69

Created: 2024-11-28

  • added the ability to configure the batch limit for bulk inserts

  • fixed issue in outbound sync where multiple order by clauses were not being adhered to

  • updated logging so all plugin messages appear with plugin. prepended to log messgaes


V1.68

Created: 2024-11-26

  • changed data types ntext and text to nvarchar(max) for outbound syncs

  • updated inbound key fields so they can be manually set when no indexes are available

  • updated inbound sync strategies so a full refresh is the only option when there are no date/time fields available or rowversion fields

  • added ability to use non clustered indexes which allows to multi thread the retrieval of data from tables with no clustered indexes or tables which have columnstore indexes and have a non clustered index on the fields to be used for multi threaded data retrieval.

  • consolidated how data types are determined for outbound syncs ddl creation.


V1.67

Created: 2024-11-22

  • bumped runtime version to v0.8.0

  • fixed outbound issue when there are composite primary keys


V1.66

Created: 2024-11-21

  • added additional connection option to disable encryption if necessary

  • modified outbound to take into account change from merge to append in source data

  • modified outbound to create varbinary columns based on the column lengths as defined by the source

  • downgraded some log levels from info to debug to reduce noise in the logs


V1.65

Created: 2024-11-19

  • fixed issue with varbinary not being used correctly by outbound sync

  • fixed issue around timestamps and image fields being incorrectly set for outbound syncs

  • added ability to prevent stage table from being truncated after a sync for debug purposes


V1.64

Created: 2024-11-18

  • bumped devkit version number


V1.63

Created: 2024-11-18

  • Added support for privatelink and direct connections via the new Connectivity Options flags Fix for ngrok connections under new runtime

  • modified outbound datatypes


V1.61

Created: 2024-11-12

  • updated plugin runtime version to 0.6.9


V1.60

Created: 2024-11-12

  • ensured keys are not null for outbound syncs when creating tables


V1.59

Created: 2024-11-12

  • made columns nullable for outbound sync schema creation

  • added sync date/time to outbound sync


V1.58

Created: 2024-11-11

  • fixed alter statements for outbound syncs


V1.57

Created: 2024-11-08

  • updated outbound to take into account binary fields

  • updated json schema to correctly handle towversions and timestamps as strings


V1.56

Created: 2024-11-07

  • fixed check if a condition criteria is a digit or a string


V1.55

Created: 2024-11-06

  • added procedure which gathers the details required for a given inbound sync

  • removed sys schema from watermark schema selection

  • added ability to specify deleted_fields inside outbound sync for soft delete capabilities


V1.54

Created: 2024-10-31

  • fixed outbound sync issue when alerting tables on destination server


V1.53

Created: 2024-10-31

  • added foundations of Private Link capabilities

  • fixed bug associated with view lookups


V1.52

Created: 2024-10-27

  • modified outbound sync so it will include multiple batches in a single run instead of each value in a cursor field.


V1.51

Created: 2024-10-23

  • updated outnbound sync processing of order by clauses to accept with a string or a list of strings

  • fixed issue when retreiving the min and max lsn


V1.49

Created: 2024-10-16

  • added Change Tracking as a Sync Method

  • modified log messages to utilize the extra data field


V1.48

Created: 2024-10-15

  • fixed casting issues for outbound sync


V1.47

Created: 2024-10-15

  • fixed minoir issue with outbound sync


V1.46

Created: 2024-10-14

  • tidied up connection failure messages

  • outbound: added in database compatability level detection for outbound sync schemas

  • inbound: no longer process streams where there is no data to be retreived for an incremental run

  • inbound: fixed issue for high water mark initial fetch of data


V1.45

Created: 2024-10-09

  • exposed additional function to execute commands against sql server

  • updated outbound sync properties for Monitorial Outbound Sync Native App


V1.44

Created: 2024-10-05

  • modified visibility of procedure 'run_outbound_sync'


V1.43

Created: 2024-10-02

  • updated cdc table look up to use non clustered indexes as well as clustered indexes

  • added experimental outbound sync functionality

  • removed Entra ID Password as an auth mechanism due to login issues around server names


V1.42

Created: 2024-09-30

  • fixed an issue where inbound syncs would experience intermitted transaction-related errors

  • added in missing microsoft-authentication-library-for-java package

  • updated error messages for connection tests so errors are clearer


V1.41

Created: 2024-09-25

  • updated docs for Azure Database Configuration

  • added square brackets around column names for the initial cdc full refresh sync

  • modified logging levels


V1.40

Created: 2024-09-06

  • Updated to new plugin runtime which publishes consumer UDF information to the sync engine

  • Reduced the size of the bundled jars

  • Upgraded some java dependencies to newer versions

  • Reduced logging from info for debug


V1.39

Created: 2024-09-03

  • added additional isolation level to allow for server defaults


V1.35

Created: 2024-09-03

  • changed devkit version to 0.5.4


V1.34

Created: 2024-09-03

  • fixed issue where incorrect property was being used for retries


V1.33

Created: 2024-09-02

  • fixed issue where the statements being issued were being concatenated together for a retry


V1.32

Created: 2024-09-02

  • added in the ability to set transaction isolation levels

  • added in additional checks for error codes to force retry due to SQL execution canceled during records being committed


V1.31

Created: 2024-08-21

  • modified hwm path for how the initial upfront data is returned when a view is not indexed so not to cause timeouts

  • added check to cdc path to ensure the min lsn is still valid at the start of the run, if not then sets the min lsn to be the earliest available


V1.30

Created: 2024-08-19

  • fixed issues whre hwm failed to bring back view information at startup causing initial sync to fail


V1.29

Created: 2024-08-16

  • removed the need for plugin to need view state permissions at the server level

  • fixed issue where where clause caused Ambiguous column name error


V1.28

Created: 2024-08-15

  • modified cdc runs so that fewer queries are issued to speed up sync process

  • modified hwm runs so that fewer queries are issues to speed up sync processes


V1.27

Created: 2024-08-13

  • fixed issue where CDC incremental runs were failing


V1.26

Created: 2024-08-13

  • modified how indexes are filtered out when retreiving table metadata

  • modified error handling around aync failures to force retries instead of erroring out


V1.25

Created: 2024-08-12

  • fixed issue where filter clause didn't have quotes around dates or strings

  • ensured an incremental cdc run doesn't return a null row when the cdc table is empty

  • modified async job to retry when "Unknown Error" is thrown by snowflake


V1.24

Created: 2024-08-12

  • Added exception handling when Async Job throws unkown error even after a successful completion


V1.23

Created: 2024-08-09

  • Fixed an issue where an error was thrown during configuration


V1.22

Created: 2024-08-08

  • bumped runtime version

  • modified concurrency so that the max queries is adhered to


V1.21

Created: 2024-08-05

  • updated initial cdc all records fetch to use seqval instead of startlsn so chuncks are more even

  • updated concurrency for data pulls from SQL Server to improve performance


V1.20

Created: 2024-08-02

  • update last-retrieve datetime to use the min sync time instead of the max sync time so no records are missed after the initial sync has been completed


V1.19

Created: 2024-08-02

  • ensured cdc capture instance is reused if multiple available for the same table


V1.18

Created: 2024-08-01

  • modified default for concurrent queries from 8 to 2

  • modified date translation to take into account Snowflake formatted date which originates from SQL Server

  • ensured CDC table exists when obtaining the current active CDC table


V1.17

Created: 2024-07-31

  • fixed issued with sync_timestamp returning null when setting state


V1.16

Created: 2024-07-17

  • bumped version number

  • add support for pre SQL Server 2017


V1.15

Created: 2024-07-04

  • added support for TimeStamp and RowVersion Cursor Field

  • modified retreive timestamp to be the sync_timestamp from SQL Server instead of Snowflake generated timestamp

  • updated high watermark to use full refresh path when the sync_strategy has been set to full refesh

  • added additional error information when the data returned doesn't include the required key fields

  • updated documentation

  • updated version numbers of omnata packages


V1.14

Created: 2024-06-12

fixed error when retreiving certificate


V1.13

Created: 2024-06-11

Separated out connection methods

  • SQL Server Authentication (on-premisis or Azure based)

  • Microsoft Entra Password (Azure SQL Database, Azure SQL Managed Instance)

  • Microsoft Entra Service Principal (Azure SQL Database, Azure SQL Managed Instance)


V1.12

Created: 2024-06-11

  • Used latest plugin devkit, which addresses an issue where errors occur when editing connections

  • Fixed an issue where the connection test did not actually execute a SQL statement


V1.11

Created: 2024-05-27

  • when a cdc columns change, a sync retreives the new set of columns and resets the state

  • cdc stream selection now correctly returns streams where net changes are requested

  • high watermark stream selection now displays indexed as well as heaped tables


V1.10

Created: 2024-05-26

  • FIltered Out Tables which don't support net changes if only net changes are required

  • Ensured when net changes are not supported that the all changes are retreived

  • Each sync queries sql server directly instead of using metadata on retreived on initial connection

  • When running a cdc sync, the columns for the capture instance are retreived instead of what was available on initial setup.


V1.9

Created: 2024-05-21

  • Update SQL DataTypes to correctly specify schema definition for datetime2, datetimeoffset, smalldatetime, money, smallmoney, and real

  • Modified Cursor Data when the cursor field is of type rowversion to include 0x at the start

  • CDC Streams can now only be incremental

  • If row_version field is defined, for non CDC syncs, the rowversion field is selected by default as the cusor field


V1.8

Created: 2024-05-19

  • Updated SQL DataTypes to correctly specify the schema definition for datetime2, datetimeoffset, smalldatetime, money, smallmoney and real

  • Expanded the list of column datatypes which can be used for a cursor field check to include datetime2, smalldatetime, datetimeoffset


V1.7

Created: 2024-05-18

  • increased plugin runtime to 0.3.26

  • Added billing events for ngrok tunnels


V1.6

Created: 2024-05-17

  • updated how indexed data is retreived for full data loads where the index is non integer based.

  • set the default cursor if a modified to create date is found for high watermark incremental merge based syncs


V1.5

Created: 2024-05-13

SQL Server Certificate Retreival Fix


V1.4

Created: 2024-05-13

Initial release of MSSQL plugin, supports inbound syncing using the following techniques:

  • Change Data Capture (CDC Tables)

  • Highwater mark (Tables and Views)


V1.3

Created: 2024-04-02

Switched to new plugin-oriented billing mechanism

  • Bump plugin runtime version

  • Built with omnata-plugin-devkit==0.3.17 which includes pinned stored proc dependencies


Last updated