Hightouch supports streaming events from your existing Kafka cluster in realtime.
Simply configure your Kafka connection in the Hightouch UI, and Hightouch will automatically extract, transform, and load your data into Hightouch Events. Your data can then be synced to your warehouse or streamed to our catalog of realtime destinations.
Hightouch can connect directly to Kafka over the public internet or via an SSH tunnel. Since data is encrypted in transit via TLS, a direct connection is suitable for most use cases. You may need to set up a tunnel if your Kafka instance is on a private network or virtual private cloud (VPC).
Hightouch supports both standard and reverse SSH tunnels. To learn more about SSH tunneling, refer to Hightouch's tunneling documentation.
All events are ingested into Hightouch as track events. The Kafka message key, value, and headers are exposed as custom properties on the track event. Metadata such as topic, partition, and offset are also exposed in the context.kafka object.
You can configure how Hightouch transforms messages from your Kafka topic to Hightouch track events. Mappings can be configured for the following fields:
event
userId
anonymousId
messageId
timestamp
You can also transform your events using Functions in order to standardize the schema or enrich its contents.