Schedule syncs with Sequences

Overview

Sync Sequences allow you to execute a series of related syncs in a predefined order. Sequences are helpful when you have dependencies between syncs. For example, you may have two syncs you need to run in order:

1. Sync A creates a set of new contacts in your CRM
2. Sync B adds the new contacts to their associated accounts

Without Sequences, you would have to schedule the two syncs individually to give the first enough time to finish before the second starts. It would be challenging to guarantee the correct behavior since sometimes sync duration can vary.

Sequences make orchestration a first-class feature in Hightouch so you can control how and when to sync data to your destinations. You can schedule sequences to run on various schedules, such as triggered by dbt Cloud runs or cron schedules, just as you would with syncs. You can also initiate sequences manually.

Sequence example

Suppose you manage your staff members, their permissions, customer information, and staff customer assignments using Hightouch. To do so, you use four syncs that need to run in a specific order:

1. The table of staff members needs to be updated first.
2. Then the staff permissions can be updated.
3. Separately, a table of customer prospects needs to be updated.
4. The final sync updates staff member assignments to customer prospects.

To represent this sequence of syncs, you can create a sequence that executes these four syncs in order with the following error handling:

• The staff member sync terminates the sequence if it fatally errors or has too many rejected rows.
• The second and third syncs only terminate the sequence if they fatally error.
• The final staff member assignment sync doesn't check for errors, so the sequence completes successfully even if this sync fails.

Sequence setup

Before you create a sequence, ensure that all syncs you plan to use in it have their schedule type set to Manual. You can configure a sync to have a set schedule and use it in a sequence simultaneously, but we recommend against it unless you have a particular reason to do so.

Then, to create a new sequence, go to the Sequences page and click the Add a sequence.

Add syncs to the sequence

Click Add syncs to select syncs for the new sequence. You can include as many syncs as you want and add more later as needed.

Configuring sync order and error handling

Once you've added syncs to the sequence, you can re-order them using the up and down arrows.

You can configure each sync's error-handling behavior using the Terminate drop-down menu.

Set your sequence schedule

To finalize your sequence, give it a name and define its schedule type. You have all the same options for sequence scheduling as for sync scheduling.

Once you save your new sequence, it's visible on the main Sequences page.

Sequence management

You can click a sequence from the Sequences page to go to its details page. There you can:

1. Disable or enable it
2. Run it manually
3. See the sequence's run history
4. Or update its configuration or schedule

Clicking on a particular run displays its details in a pop-up, including the execution time and final state of each sync run belonging to the sequence run.