Salesforce Bulk v2.0 Connector Example¶ The Salesforce Connector provides seamless integration with the Salesforce Bulk v2.0 REST API, enabling easy and efficient handling of large volumes of data. The SalesforceBulk API operates on a RESTful architecture, offering a fast and reliable method to load or delete vast amounts of data from your organization's Salesforce account. With SalesforceBulk, you can perform asynchronous operations like querying, inserting, updating, upserting, or deleting a considerable number of records by submitting them in batches. These batches can be processed by Salesforce in the background, ensuring minimal disruption to your workflow. What you'll build¶ The following example demonstrates how to use the Salesforce Bulk v2.0 Connector for performing various operations on your Salesforce data: Insert account records from a file to the salesforce. Get the created bulk job information. Get the successfully processed records. You can use the following resources to achieve your requirements. /createJobAndUploadFile : Create a new bulk ingest job for insert operation. Read a CSV file using File Connector. Upload the CSV content read from the file. Close the job to denote that the upload is completed. /getJobInfo : Get the bulkJob info identified by the jobId passed through the request body. /getSuccessfulResults : Retrieve the successful results of the bulk job identified by the jobId. Set up the environment¶ By default, the text/csv message formatter and message builder are not configured in the WSO2 Integrator: MI settings. To enable this connector to function correctly with text/csv data, you will need to follow these steps to add the necessary message formatter and message builder configurations. Consider the root of the WSO2 Integrator: MI/ Enterprise Integrator as <PRODUCT_HOME>. If you are using the WSO2 Integrator: MI 4.4.0 or above, you need to add the following message builder to <PRODUCT_HOME>/conf/deployment.toml file. For more information, refer to the Working with Message Builders and Formatters and Product Configurations documentation. [[custom_message_formatters]] class = "org.apache.axis2.format.PlainTextFormatter" content_type = "text/csv" [[custom_message_builders]] class="org.apache.axis2.format.PlainTextBuilder" content_type = "text/csv" Save the files and restart the WSO2 Integrator: MI. Set up the integration project¶ Follow the steps in create integration project guide to set up the integration project. Add integration logic¶ First, create a REST API called Salesforce in your project. Name Context Salesforce /salesforce Create the following resources in 'Salesforce' REST API uri-template method /createJobAndUploadFile GET, POST /getJobInfo POST /getSuccessfulResults POST /getUnprocessedResults POST /deleteJob POST /createQuery GET, POST /getQueryJobInfo POST /getSuccessfulQueryResults POST Let's add the operations to the resources in the Salesforce API - /createJobAndUploadFile¶ Users can utilize this resource to send CSV content for upload via a CSV file. The API will utilize a File Connector to store the CSV content in a csvContent property. The 'UploadJobData' operation will then upload the csvContent. After uploading the content, the CloseJob operation will be used to change the job status to UploadComplete. Select the createJob operation from Salesforce_Connector. In the 'Salesforce Configuration' section of the properties, select the Salesforce connection configuration you created. In the properties section, under Basic, select INSERT in the Operation dropdown. Input Account in the Object text box Select COMMA in the Column Delimiter dropbox Select LF or CRLF in the Line Ending dropbox based on your operating system. IF Windows : CRLF, for Unix-based systems : LF Select the read operation from File_Connector. Prior to this step, you must configure the File Connector. For setup instructions, please refer to the File Connector Documentation. Create a File Connection and select it. In the Basic section, enter the file path. Select the uploadJobData operation from Salesforce_Connector. In the 'Salesforce Configuration' section of the properties, select the Salesforce connection configuration you created. For Job ID text box select the expression of the variable value for id returned from createJob operation. For Input Data text box select the expression of the variable value for payload returned from file read operation. Select the closeJob operation from Salesforce_Connector. In the 'Salesforce Configuration' section of the properties, select the Salesforce connection configuration you created. In the 'Job ID' text box select the expression of the variable value for id returned from createJob operation. Select the 'Respond' mediator. - /getJobInfo¶ Using this resource, users can get the job information. Select the getJobInfo operation from Salesforce_Connector. In the 'Salesforce Configuration' section of the properties, select the Salesforce connection configuration you created. In the 'Job ID' text box, enter the expression ${payload.jobId}. Select the 'Respond' mediator. - /getSuccessfulResults¶ Using this resource, users can retrieve the successfully processed records of a particular bulk job. Select the getSuccessfulResults operation from Salesforce_Connector. In the 'Salesforce Configuration' section of the properties, select the Salesforce connection configuration you created. In the 'Job ID' text box, enter the expression ${payload.jobId}. In the 'Output type' dropdown, select JSON or CSV. Select the 'Respond' mediator. Test the resources¶ Let's test the API. Start the MI and deploy the API. Let's create a bulk ingest job using our /createJobAndUploadFile resource. To invoke the resource, use the following curl command: curl --location 'http://localhost:8290/salesforce/createJobAndUploadFile' \ --header 'Content-Type: text/plain' \ --header 'Cookie: CookieConsentPolicy=0:1; LSKey-c$CookieConsentPolicy=0:1' You will receive a response similar to the following: { "id": "XXXXXXXXXXXXXXXXX", "operation": "insert", "object": "Account", "createdById": "XXXXXXXXXXXXXXXXX", "createdDate": "2023-03-16T06:43:09.000+0000", "systemModstamp": "2023-03-16T06:43:09.000+0000", "state": "UploadComplete", "concurrencyMode": "Parallel", "contentType": "CSV", "apiVersion": 57.0 } Note down the id from the response. Let's get the job information of the bulk job using our /getJobInfo resource. To invoke the resource, please use the following curl command: curl --location 'http://localhost:8290/salesforce/getJobInfo' \ --header 'Content-Type: application/json' \ --header 'Cookie: CookieConsentPolicy=0:1; LSKey-c$CookieConsentPolicy=0:1' \ --data '{ "id" : "XXXXXXXXXXXXXXXXX" }' Make sure you replace the id value. You will receive a response similar to the following: { "id": "XXXXXXXXXXXXXXXXX", "operation": "insert", "object": "Account", "createdById": "XXXXXXXXXXXXXXXXX", "createdDate": "2023-03-16T06:43:09.000+0000", "systemModstamp": "2023-03-16T06:43:13.000+0000", "state": "JobComplete", "concurrencyMode": "Parallel", "contentType": "CSV", "apiVersion": 57.0, "jobType": "V2Ingest", "lineEnding": "LF", "columnDelimiter": "COMMA", "numberRecordsProcessed": 1, "numberRecordsFailed": 0, "retries": 0, "totalProcessingTime": 139, "apiActiveProcessingTime": 81, "apexProcessingTime": 0 } Let's get the successfully processed records using our /getSuccessfulResults resource. To invoke the resource, please use the following curl command: curl --location 'http://localhost:8290/salesforce/getSuccessfulResults' \ --header 'Content-Type: application/json' \ --header 'Cookie: CookieConsentPolicy=0:1; LSKey-c$CookieConsentPolicy=0:1' \ --data '{ "id" : "XXXXXXXXXXXXXXXXX" }' Make sure you replace the id value. You will receive a response similar to the following: [ { "sf__Id": "XXXXXXXXXXXXXXXXX", "sf__Created": "true", "Name": "Lorem Ipsum", "ShippingCity": "Milano", "NumberOfEmployees": "2676", "AnnualRevenue": "9.12260031E8", "Website": "https://ft.com/lacus/at.jsp", "Description": "Lorem ipsum dolor sit amet" } ] Let's get the successfully processed records using our /getUnprocessedResults resource. To invoke the resource, please use the following curl command: curl --location 'http://localhost:8290/salesforce/getUnprocessedResults' \ --header 'Content-Type: application/json' \ --header 'Cookie: CookieConsentPolicy=0:1; LSKey-c$CookieConsentPolicy=0:1' \ --data '{ "id" : "XXXXXXXXXXXXXXXXX" }' Make sure you replace the id value. Upon successful execution, you will receive a 200 OK response, and the output will be written to the designated file. { "result": "success" } What's next¶ To customize this example for your own scenario, see Salesforce bulk V2 Connector Configuration documentation for all operation details of the connector.