Skip to content

Enable history sync for apps built with XMTP

Enable the history sync feature to give your users a way to sync an archive (decrypted historical data) from an existing app installation to a new app installation.

This historical data includes:

  • Conversations
  • Conversation messages
  • Consent state
  • HMAC keys (for push notifications)

History sync enables your users pick up conversations where they left off, regardless of the app installation they use. All they need is a pre-existing and online app installation to provide the archive.

🎥 walkthrough: History sync

This video provides a walkthrough of history sync, covering the key ideas discussed in this doc. After watching, feel free to continue reading for more details.

Enable history sync

History sync is enabled by default. When creating a client, the historySyncUrl client option is dynamically set to a default history sync server URL based on your env client option setting.

  • When env is set to dev, the historySyncUrl is set to https://message-history.dev.ephemera.network/
  • When env is set to production, the historySyncUrl is set to https://message-history.production.ephemera.network

These default servers are managed and operated by XMTP Labs, a steward of the development and adoption of XMTP.

You can choose to run your own server and set the historySyncUrl to your server's URL.

If needed, you can turn off history sync by setting the historySyncUrl client option to an empty string.

How history sync works

When a sync request is sent (via sendSyncRequest()) and a historySyncUrl client option is present, the receiving installation creates an encrypted archive.

Diagram showing step 1 of history sync: client initialization triggers sync request and creates encrypted archive

History sync then uploads the archive, sends a sync reply, and pulls all conversation state history into the new app installation, merging it with the existing app installations in the sync group.

Diagram showing step 2 of history sync: archive upload, sync reply, and merging conversation state history across app installations

Ongoing updates to history are streamed automatically. Updates, whether for user consent preferences or messages, are sent across the sync group, ensuring all app installations have up-to-date information.

History syncs are accomplished using these components:

  • Sync group
  • Sync worker
  • History server

Sync group

A sync group is a special Messaging Layer Security group that includes all of the user's devices. The sync group is used to send serialized sync messages across the user's devices. The sync group is filtered out of most queries by default so they don't appear in the user's inbox.

Sync worker

A sync worker is a spawned background worker that listens for sync events, and processes them accordingly. This worker:

  • Emits and updates consent
  • Creates and consumes archives for and from other devices
  • Keeps preferences synced across devices

History server

A history server acts as a bucket that holds encrypted sync archives. The URL location of these archives and the password (cipher encryption key) to decrypt these archives is sent over the sync group for the recipient to decrypt.

Control history sync

Use these methods to manage the sync flow.

Send a sync request

Send a request to other installations to sync conversations and messages to the current installation. You should call this after creating a client on a new installation to pull in historical data.

See Send a sync request for code examples.

Send a sync archive

Send an archive to the sync group without waiting for a request from another installation.

Sync device sync groups

Sync all device sync groups from the network.

List available archives

Get a list of archives available for download from other installations. This method reads from the local database only and does not sync from the network. If you believe your data may be out of date, call syncAllDeviceSyncGroups() first to pull the latest state.

Process a sync archive

Manually process an available archive from the sync group.

FAQ

A user logged into a new app installation and doesn't see their conversations. What's going on?

Ensure that you've called sendSyncRequest() after creating the client on the new installation, and that a pre-existing app installation is online to receive and respond to the request.

A user logged into a new app installation and sees their conversations, but no messages. What's going on?

Ensure that you've initiated a call to sync messages and that the pre-existing app installation is online to receive the sync request, process and encrypt the archive, upload it to the history server, and send a sync reply message to the new app installation.

I called a sync method (messages, consent state, or conversations), but nothing is happening. What's going on?

After calling sendSyncRequest(), ensure that the pre-existing app installation is online to receive the sync request, process and encrypt the archive, and send a sync reply.