Connect to Batch Sources
Connect to Snowflake
Step 1: In the Data Portal, click Tables and then click Create Table.
Step 2: Select Snowflake as the Data Source.
Step 3: Create a New Connection.
Click New Connection. If you want to use an existing connection, select the connection from the list and proceed to Step 5.
Enter a Source Name for the new connection.
Step 4: Configure Connection Parameters
Use the following JSON configuration:
Property Descriptions
Property | Required | Description |
---|---|---|
sql.snowflake.account | Yes | The name of the Snowflake account. Found in your Snowflake URL (e.g., xyz.snowflakecomputing.com ). |
sql.snowflake.user | Yes | The username for authenticating with Snowflake. Ensure the user has appropriate read access. |
sql.snowflake.password | Yes | The password for authenticating with Snowflake. |
sql.snowflake.db | Yes | The name of the database where the data resides. |
sql.snowflake.schema | Yes | The name of the schema where the data resides. |
Step 5: Test the Connection and Configure Data Ingestion
After you have configured the connection properties, test the connection to ensure it is working.
When the connection is successful, use the following JSON to configure additional data settings:
Property Descriptions
Property | Required | Description |
---|---|---|
sql.queryTemplate | Yes | The SQL query template for fetching data from Snowflake. |
sql.timeColumnName | Yes | The time column used for filtering data while ingesting. |
sql.timeColumnFormat | Yes | The datetime format of the time column used for filtering. |
sql.startTime | Yes | The start time for data extraction. Dynamically used in the $START placeholder of sql.queryTemplate . It should match the mentioned time column format. |
sql.endTime | Yes | The end time for data extraction. Dynamically used in the $END placeholder of sql.queryTemplate . It should match the mentioned time column format. |
Additional Optional Parameters
Use the following JSON for optional configurations:
Optional Configuration Properties
Property | Required | Description |
---|---|---|
sql.bufferTimePeriod | No | The buffer time period before fetching data. Default value: 0d . Defines additional time to consider before fetching data. |
sql.bucketTimePeriod | No | The time period granularity for each data bucket. Default value: 1d . |
sql.numBucketsToLookBack | No | The number of time buckets to look back when fetching data. Default value: 0 . Used for incremental data fetching. |
Step 6: Preview the Data
Click Show Sample Data to preview the source data before finalizing the configuration.
Next Step
Proceed with Data Modeling.