MySQL

MySQL

This document provides instructions for integrating MySQL data into Panoply. The following items will be covered:

MySQL Data Integration

To integrate MySQL data into Panoply using default selections, complete the following steps. For more advanced options, complete the following and refer to the subsequent sections for detailed information.

  1. Click Data Sources in the navigation menu.
  2. Click the Add Data Source button.
  3. In the Data Sources – Choose Source Type window, select MySQL. MySQL is listed under Most Popular and Databases.
  4. In the Data Sources – MySQL screen, type the host address of the database you are connecting to, along with a username and password.
  5. Click Next.
  6. Select the Database you want to connect to. This loads a list of tables.
  7. Select the tables you want to import.
  8. (Optional) To customize the ingestion from your data source, review the advanced options.
  9. Click Collect.

The Data Sources – MySQL window will appear grayed out while the data integration is pending. A small green progress bar appears below MySQL.

You will be prompted to set up the integration of another data source. You can set up multiple data integrations without impacting the ingestion of the already scheduled or pending data integrations.

From the Data Sources main menu, you can monitor the data ingestion status of the scheduled and pending data integrations. After the data ingestion is complete, you can clean or transform your data in the Tables menu.

Advanced Options

Clicking Show next to Advanced will expand the Data Sources - MySQL window to include Destination, Primary Key, Incremental Key, Exclude, Parse string, SSL Certificates, and Truncate table.

  • Destination - Default is {__tablename}, where __tablename is the source table name.

  • Primary Key - Default is id. The primary key here determine which field(s) to use as the deduplication key when ingesting data.

  • Incremental Key - By default, Panoply fetches all of your MySQL data on each run. If you only want to collect some of your data, enter a column name to use as your incremental key. The column must be logically incremental. Panoply will keep track of the maximum value reached during the previous run and will start there on the next run. If you set an incremental key, you should only select one table. Otherwise the collection will use a single set of incremental key & value for all the tables.

  • SSL Certificates - You can optionally add SSL certificates for the certificate authority, client certificate, and client key. If you want to have an SSH connection, please request configuration help from Panoply Support.

Data Schema

The data schema in Panoply will reflect the data schema from the MySQL source. Additionally, Panoply includes these metadata columns in each table: __tablename, __senttime, and __updatetime.