Getting started

What is Refter?

refter is a hosted documentation tool for dbt projects with a heavy focus on data end-users. It lives on the edge of your data stack and aims to simplify documenting your modeled data and making it available to end-users and stakeholders. It enhances and adds on to the default dbt documentation by introducing support for:

• Entity relationship diagrams (ERDs)
• Lifecycle management:
  • support for dbt model versioning
  • enhances dbt model deprecation
  • easily communicate data changes
• Easy and straightforward search
• RBAC and access control
• Knowledge sharing and collaboration
• Data model curation

All documentation is defined as code, fully versioned controlled and lives side-by-side with your dbt models. Refter uses model properties to configure and render your documentation.

info

Refter is a SaaS product that is currently in private beta. If you're interested in trying it out, sign up here.

info

refter uses model properties .yml files to configure relations between models. This is done by adding a refter section to your model properties file. If you're looking for an easy way to automatically generate properties.yml files, check out dbt-invoke.

How does it work?

refter is an extension for dbt model properties. It uses a models properties.yml file to add additional metadata to your models that help with the effectiveness and usefulness of your data documentation. Model relations and lifecycle management are configured in your code and transparently flow through to your refter documentation site.

refter is not a dbt package, that means it doesn't require any additional dependencies or changes to your dbt project. It's a standalone tool that uses your dbt project to generate documentation.

Here's an example of a model properties file that adds additional refter configuration:

models/product/customers.yml

version: 2
models:
  - name: customers
    config:
      tags:
        - pii
      refter: # refter model properties
        disabled: false
        deprecated:
          enabled: false
        group: customers
        owner: data-team
    columns:
      - name: customer_id
        tests:
          - unique
          - not_null

  - name: orders
    config:
      meta:
        owner: sales-team # this is a dbt property
    columns:
      - name: order_id
        data_type: int # refter uses dbt data types
        tests:
          - unique
          - not_null
      - name: old_order_id
        refter: # refter column properties
          deprecated:
            enabled: true
            reason: "This column is deprecated, use `order_id` instead"
            date: "2021-01-01"
      - name: customer_id
        refter:
          relations: # define ERD relations
            - model: customers
              column: customer_id
              type: many-to-one
      - name: status
        description: The status of the order

To learn more about refter table and column properties check out our configuration documentation. If you want to learn more about how to start dbt model properties and start building out a data dictionary, check out our blog post on data dictionaries here.

Deployment

To deploy your dbt project to your refter instance, you'll need to use the refter-cli. The refter-cli is a command-line tool that helps you deploy and validate your dbt project to refter.

Install the refter-cli

pip install refter-cli

Update your dbt manifest

info

refter uses the dbt manifest to generate documentation. Make sure you have compiled your dbt project before deploying to refter.

dbt compile

or

dbt ls

Deploy your dbt project to refter

info

refter-cli will validate your dbt project configuration before it tries to deploy. If any validation errors are detected, refter-cli will report them and exit.

refter deploy --token <your-api-key>

To learn more about refter deployments check out the deployment docs. To learn more about setting up CI/CD to automatically deploy your dbt project to refter check out the CI/CD docs.