This document provides step-by-step instructions for integrating File Upload data into Panoply. The following items will be covered:
Supported File Types
Panoply supports the following file types for the File Upload data source:
- Archive (
.tarand.gzip) - JSON (
.json) - Character-delimited files (
.csvand.tsv) - See our sample .csv file - Microsoft Excel (
.xlsx) - Plain text (
.txt)
Maximum size for uploading files is 100 MB. For larger files, consider using our Amazon S3 data source.
Missing column headers
The first row of your data should be a complete set of column headers. If your data includes some columns without headers, the data ingestion process may exclude some of your data. Be sure each of your columns has a header.
Archives with unsupported files
Collection will fail if you try to upload an archive file (such as .tar or .gzip) that includes unsupported file types or other archive files. To resolve this issue, review your archive, remove unsupported files, upload the modified archive file, and collect again.
File Upload Data Integration
To integrate File Upload data into Panoply using default selections, complete the following steps.
- Click Data Sources in the navigation menu.
- Click the Add Data Source button.
- In the Add Data Source section, select File Upload.
- On the File Upload page, click Choose File and select the file you want to upload.
A message appears when the file can be added to the warehouse.
- Enter a Destination, which is the table where Panoply will collect the data. Note that Panoply will create the table for you. If your file generates more than one table, the Destination value will be used a a prefix for all the table names. Learn more about Destinations.
- (Optional) To customize the collection from your data source, review the advanced options.
- After selecting a file and entering a Destination, click Collect.
The File Upload page is grayed out while the data collects into the warehouse. A progress bar appears at the top.
You can safely navigate to other data sources. More data sources can be added without impacting the collection of the File Upload data source.
From the Data Sources main menu you can monitor the data ingestion status. After the data collection is complete, navigate to the Tables page to view the data stored in the warehouse.
Data Schema
Your source file determines the schema for the File Upload data source. If your source file contains nested data, Panoply creates subtables based on the data’s structure. For more, see Nested Data.