Greenplum reduces data silos by providing you with a single, scale-out environment for converging analytic and operational workloads, like streaming ingestion.
Hightouch lets you pull data stored in your Greenplum database and push it to downstream destinations. Most of the setup occurs in the Hightouch UI, but you need access to your Greenplum instance for information like your host, port, database name, and credentials.
You may need to allowlist Hightouch's IP addresses to let our systems connect to your warehouse. Reference our networking docs to determine which IP addresses you need to allowlist.
Hightouch can connect directly to Greenplum over the public internet or via an SSH tunnel. Since data is encrypted in transit via TLS, a direct connection is suitable for most use cases. You may need to set up a tunnel if your Greenplum instance is on a private network or virtual private cloud (VPC).
Hightouch supports both standard and reverse SSH tunnels. To learn more about SSH tunneling, refer to Hightouch's tunneling documentation.
For optimal performance, Hightouch tracks incremental changes in your data model—such as added, changed, or removed rows—and only syncs those records. You can choose between two different sync engines for this work.
The Basic engine requires read-only access to Greenplum. Hightouch executes a query in your database, reads all query results, and then determines incremental changes using Hightouch's infrastructure. This engine is easier to set up since it requires read—not write—access to Greenplum.
The Lightning engine requires read and write access to Greenplum. The engine stores previously synced data in a separate schema in Greenplum managed by Hightouch. In other words, the engine uses Greenplum to track incremental changes to your data rather than performing these calculations in Hightouch. Therefore, these computations are completed more quickly.
Hightouch recommends using the Lightning sync engine when syncing more than 100 thousand rows of data.
If you select the Basic engine, you can switch to the Lightning engine later. Once you've configured the Lightning engine, you can't move back to the Basic engine without recreating Greenplum as a source.
The Lightning sync engine requires granting write access to your data warehouse, which makes its setup more involved than the Basic sync engine. However, it is more performant and reliable than the Basic engine. This makes it the ideal choice to guarantee faster syncs, especially with large data models. It also supports more features, such as Warehouse Sync Logs, Match Booster, and Identity Resolution.
Criteria
Basic sync engine
Lightning sync engine
Performance
Slower
Quicker
Ideal for large data models (over 100 thousand rows)
No
Yes
Reliability
Normal
High
Resilience to sync interruptions
Normal
High
Extra features
None
Warehouse Sync Logs, Match Booster, Identity Resolution
Ease of setup
Simpler
More involved
Location of change data capture
Hightouch infrastructure
Greenplum schemas managed by Hightouch
Required permissions in Greenplum
Read-only
Read and write
Ability to switch
You can move to the Lightning engine at any time
You can't move to the Basic engine once Lightning is configured
To set up the Lightning engine, you need to grant Hightouch write access to Greenplum. You can do so by running the following SQL snippet.
CREATEUSER hightouch_user WITH PASSWORD '********';
CREATE SCHEMA IF NOTEXISTS hightouch_audit;
CREATE SCHEMA IF NOTEXISTS hightouch_planner;
GRANTCREATE, USAGE ON SCHEMA hightouch_audit TO hightouch_user;
GRANTCREATE, USAGE ON SCHEMA hightouch_planner TO hightouch_user;
The snippet creates a dedicated Greenplum user for Hightouch. It also provisions two schemas (hightouch_planner and hightouch_audit) for storing logs of previously synced data.
User: This can be your personal Greenplum login or a dedicated user for Hightouch. At minimum, this user must have read access to the data you wish to sync. If using the Lightning sync engine, you must also grant this user additional permissions as described above.
Password: The password for the user specified above.
When setting up a source for the first time, Hightouch validates the following:
Network connectivity
Greenplum credentials
Permission to list schemas and tables
Permission to write to hightouch_planner schema
Permission to write to hightouch_audit schema
All configurations must pass the first three, while those with the Lightning engine must pass all of them.
Some sources may initially fail connection tests due to timeouts. Once a connection is established, subsequent API requests should happen more quickly, so it's best to retry tests if they first fail. You can do this by clicking Test again.
If you've retried the tests and verified your credentials are correct but the tests are still failing, don't hesitate to .
Once your source configuration has passed the necessary validation, your source setup is complete. Next, you can set up models to define which data you want to pull from Greenplum.
The Greenplum source supports these modeling methods:
To date, our customers haven't experienced any errors while using this source. If you run into any issues, please don't hesitate to . We're here to help.
Ready to get started?
Jump right in or a book a demo. Your first destination is always free.