Feature/databricks compat

.buildkite/pipeline.yml

@@ -42,3 +42,17 @@ steps:
             - "CI_REDSHIFT_DBT_USER"
     commands: |
       bash .buildkite/scripts/run_models.sh redshift
+
+  - label: ":bricks: Run Tests - Databricks"
+    key: "run_dbt_databricks"
+    plugins:
+      - docker#v3.13.0:
+          image: "python:3.8"
+          shell: [ "/bin/bash", "-e", "-c" ]
+          environment:
+            - "BASH_ENV=/tmp/.bashrc"
+            - "CI_DATABRICKS_DBT_HOST"
+            - "CI_DATABRICKS_DBT_HTTP_PATH"
+            - "CI_DATABRICKS_DBT_TOKEN"
+    commands: |
+      bash .buildkite/scripts/run_models.sh databricks

.buildkite/scripts/run_models.sh

@@ -1,5 +1,7 @@
 #!/bin/bash
 
+set -euo pipefail
+
 apt-get update
 apt-get install libsasl2-dev
 

.github/pull_request_template.md

@@ -1,4 +1,3 @@
-Pull Request
 **Are you a current Fivetran customer?** 
 <!--- Please tell us your name, title and company -->
 
@@ -29,9 +28,9 @@ Pull Request
 
 **How did you test the PR changes?** 
 <!--- Proof of testing is required in order for the PR to be approved. -->
-<!--- To check a box, remove the space and insert an x in the box (eg. [x] CircleCi). --> 
+<!--- To check a box, remove the space and insert an x in the box (eg. [x] Buildkite). --> 
 <!--- To select a checkbox you simply need to add an "x" with no spaces between the brackets (eg. [x] Yes). -->
-- [ ] CircleCi <!--- CircleCi testing is only applicable to Fivetran employees. --> 
+- [ ] Buildkite <!--- Buildkite testing is only applicable to Fivetran employees. --> 
 - [ ] Local (please provide additional testing details below)
 
 **Select which warehouse(s) were used to test the PR**

CHANGELOG.md

@@ -36,7 +36,8 @@
     - `dbt.current_timestamp_in_utc_backcompat`
 - Dependencies on `fivetran/fivetran_utils` have been upgraded, previously `[">=0.3.0", "<0.4.0"]` now `[">=0.4.0", "<0.5.0"]`.
 ## 🎉 Documentation and Feature Updates 🎉:
-- Updated README documentation for easier navigation and dbt package setup.
+- Updated README documentation for easier navigation and dbt package setup. [#35](https://github.com/fivetran/dbt_github/pull/35)
+- Added Databricks compatibility. [#38](https://github.com/fivetran/dbt_github/pull/38)
 
 # dbt_github v0.5.1
 ## Fixes

README.md

@@ -36,7 +36,15 @@ The following table provides a detailed list of all models materialized within t
 To use this dbt package, you must have the following:
 
 - At least one Fivetran Github connector syncing data into your destination.
-- A **BigQuery**, **Snowflake**, or **Redshift*** destination.
+- A **BigQuery**, **Snowflake**, **Redshift**, or **Databricks** destination.
+
+### Databricks Dispatch Configuration
+If you are using a Databricks destination with this package you will need to add the below (or a variation of the below) dispatch configuration within your `dbt_project.yml`. This is required in order for the package to accurately search for macros within the `dbt-labs/spark_utils` then the `dbt-labs/dbt_utils` packages respectively.
+```yml
+dispatch:
+  - macro_namespace: dbt_utils
+    search_order: ['spark_utils', 'dbt_utils']
+```
 
 ## Step 2: Install the package
 Include the following github package version in your `packages.yml` file.
@@ -116,6 +124,9 @@ packages:
 
     - package: fivetran/github_source
       version: [">=0.6.0", "<0.7.0"]
+
+    - package: dbt-labs/spark_utils
+      version: [">=0.3.0", "<0.4.0"]
 ```
 # 🙌 How is this package maintained and can I contribute?
 ## Package Maintenance

integration_tests/ci/sample.profiles.yml

@@ -1,5 +1,5 @@
 
-# HEY! This file is used in the dbt package integrations tests with CircleCI.
+# HEY! This file is used in the dbt package integrations tests with Buildkite.
 # You should __NEVER__ check credentials into version control. Thanks for reading :)
 
 config:

integration_tests/dbt_project.yml

@@ -55,3 +55,7 @@ seeds:
     github_requested_reviewer_history_data:
       +column_types:
         created_at: timestamp
+
+dispatch:
+  - macro_namespace: dbt_utils
+    search_order: ['spark_utils', 'dbt_utils']