Data Sync Using Watermarking and Batch

Illustrates the concept of data synchronization using watermarking. The concept of a watermark refers to a flood after-match in which you look at the water stains in a wall to figure how high the water rose. This example determines what the last item was that was updated.

Example

In this example, watermarking periodically makes pull requests to the data store to retrieve only records that were modified since the last request. Specifically, Salesforce CRM is queried for contacts that were recently modified and just prints them in the console.

Set Up and Run the Example

Complete the following procedure to create, then run this example in your own instance of Anypoint Studio. Skip ahead to the next section if you prefer to simply examine this example.

1. Create the example application in Anypoint Studio. Do not run the application.
2. Log in to your Salesforce account. From your account menu. Your account is labeled with your name. Select Setup.
3. In the left navigation bar, under the Personal Setup heading, click to expand the My Personal Information folder.
4. Click Reset My Security Token. Salesforce resets the token and emails you the new one.
5. Access the email that Salesforce sent and copy the new token onto your local clipboard.
6. In your application in Studio, click the Global Elements tab.
7. Double-click the Salesforce global element to open its Global Element Properties panel. In the Security Token field, paste the new Salesforce token you copied from the email. Alternatively, configure the global element in the XML Editor.
8. Change the contents of the Username and Password fields to your account-specific values, then click OK to save your changes.
9. In the Package Explorer, right-click the connect-with-salesforce project name, and select Run As > Mule Application. Studio runs the application on the embedded server.
10. In your browser, access your Salesforce account, then navigate to the Contacts tab.
11. Use the drop-down menu to display All Contacts, then select one contact and edit to change the first name.

12. Go back to Anypoint Studio and click the console view and search for a log similar to this one:

     INFO  2017-07-04 08:17:01,502 [batch-job-watermarkingBatch1-work-manager.05] org.mule.api.processor.LoggerMessageProcessor: contact recently modified: {Name=Avi123 Green, LastModifiedDate=2017-07-04T06:16:54.000Z, Id=0032000001DpkrEAAR, type=Contact}

13. Stop the Mule application by clicking the square, red terminate button in the Console.

Documentation

Read full documentation in GitHub