Dynamic Fields

Dynamic Fields

In Panoply, you can dynamically define the primary keys, desination tables, and incremental keys for many of your data sources. Dynamic fields provide a way to fine tune how Panoply stores and retrieves your data.

Dynamic fields are indicated by wrapping the field name in curly brackets. For example, if you have a column in a CSV file with a unique value for each row, and that column header is campaign_id, the column is an ideal candidate to set as the primary key for that data source. To do this, you would type {campaign_id} into the Primary Key field when defining that data source’s advanced options.

You can mix dynamic fields with other characters when you define primary keys, desination tables, and incremental keys. For example, the default desination for Shopify data is shopify_{__resource}. The dynamic field is {__resource}. If you wanted to keep that dynamic field but change the destination table names to all start with good-store_, you would change the shopify_ prefix and end up with a destination defined as good-store_{__resource}. The underscore at the end of the prefix is not required, strictly speaking, but it makes the final table names easier to read.

You can also use multiple dynamic fields together when you define your primary keys, desination tables, and incremental keys. For example, the default destination table definition for the Google Sheets data source is sheets_{file name}_{sheet name}. If you had a Google Sheet with a file name of sales figures and it contained three sheets called summary, 2018q3, and 2018q4, data would be stored on the following destination tables:

  • sheets_sales figures_summary
  • sheets_sales figures_2018q3
  • sheets_sales figures_2018q4

For more information on working with dynamic fields, read our docs on primary keys, incremental keys, and destination tables.