Advanced Settings

Advanced Settings

We do not recommend changing advanced settings unless you are an experienced Panoply user.

For users who have some experience working with their data in Panoply, there are a number of items that can be customized for this data source.

  1. Destination Schema: This is the name of the target schema to save the data. The default schema is Public. This cannot be changed once a source had been collected.
  2. Destination: Panoply selects a default destination. These are the tables where data is stored.
    • The default naming convention is sheets_<filename>_<sheetname>. For example if you had a spread sheet named “App Install Metrics” and it contained a sheet (tab) named “app_installs”, it would be stored in Panoply as sheets_app_install_metrics_app_installs
    • To prefix all table names with your own prefix, use this syntax: prefix_<__tablename>, where prefix is your desired prefix name and <__table_name> is a variable that represents the <filename>_<sheetname>.
  3. Primary Key: Users can define which column contains the table’s Primary Key. If this option is left blank and the sheet does not contain an ID column, Panoply will insert an id, formatted as a GUID, such as 2cd570d1-a11d-4593-9d29-9e2488f0ccc2.
  4. Exclude: The Exclude option allows you to exclude certain data, such as names, addresses, or other personally identifiable information. Enter the column names of the data to exclude.
  5. Parse String: If the data to be collected contains JSON, include the JSON text attributes to be parsed.
  6. Truncate: Truncate deletes all the current data stored in the destination tables, but not the tables themselves. Afterwards Panoply will recollect all the available data for this data source.
  7. Click Save Changes then click Collect.
    • The data source appears grayed out while the collection runs.
    • You may add additional data sources while this collection runs.
    • You can monitor this collection from the Jobs page or the Data Sources page.
    • After a successful collection, navigate to the Tables page to review the data results.

Data Dictionary

Because Google Sheets data comes from a spreadsheet file Panoply cannot provide a data dictionary. But Panoply does automate the data schema for the collected data. This is the useful information to know about the Panoply automations:

  • A column in a table uses the same data type for all values in that column. Panoply automatically chooses the data type for each column based on the available values. This is important to note for this data source. If even one value in a column has text, then the entire column is considered data type Text.
    • For example, the following combination of values in a single column will be data type Number:
      • 10000
      • 10,000
      • 10.10
    • For example, the following combination of values in a single column will be data type Text:
      • 10000
      • 10,000
      • 10.10
      • 10000x
  • Regarding data types, values using commas as a decimal place (such as “12,45”) can be imported as data type Number with some restrictions.
    • The “location” of the Google Sheet determines if “12,45” is a number or a text. See the discussion of decimal point and comma and the Google Sheets API documentation on ValueRenderOption.
    • Someone in the United States, and using the United States version of Google Sheets, enters “12,45” into a Google Sheet cell then Google will automatically format that value as a Text. Even if you manually change the cell format to Number, Google will treat it as a Text when added to Panoply.
    • Someone in the France, and using the French version of Google Sheets, enters “12,45” into a Google Sheet cell then Google will automatically format that value as a Number.
  • Dates are formatted as formatted strings.
  • For each sheet, Panoply opens the individual sheet (tab) and collects the values row by row.
  • A column with a header but without values will be ignored. This is a limitation built into the Data Engine.
  • Empty columns and empty rows are not collected.
  • The following metadata columns are added by Panoply to the destination table(s):
    • id - If the user does not enter a primary key, and no id column exists in the source, Panoply will insert an id. Formatted as a GUID, such as 2cd570d1-a11d-4593-9d29-9e2488f0ccc2
    • __updatetime - Formatted as a datetime, such as 2018-06-26T01:26:14.695Z
    • __senttime - Formatted as a datetime, such as 2018-06-26T01:26:14.695Z
    • __tablename - The name of the sheet (tab), in Google Sheets, where the data originated. Formatted as <filename>_<sheet name>, such as app_install_metrics_app_installs.
Getting started is easy! Get all your data in one place in minutes.
Try Free