Skip to content

Google Pub Sub Connector Example

The Google Pub/Sub connector allows you to access the Google Cloud Pub/Sub API Version v1 from an integration sequence.

What you'll build

Given below is a sample scenario that demonstrates how to work with the WSO2 Google Pub Sub Connector to:

  1. Create a Topic to store company update notifications.
  2. Insert company update notifications to the created topic.
  3. Retrieve company updates from the created topic.

To work with the Google Pub/Sub connector, you need to have a Google Cloud Platform account. Please refer the Setting up the Google Pub Sub Environment documentation to setup an account.

In this scenario, the user needs to create a Topic in Google Cloud Platform account. This topic is used to store notifications related to company update notifications. Once the user invokes the createTopic resource, the subscribing operation also gets triggered simultaneously. Then the user can insert company update notifications to the created topic. Finally, the user can retrieve the company updates from the subscribed topic while invoking the API.

All three operations are exposed via an API. The API with the context /resources has three resources.

  • /createTopic : Used to create a Topic for store company notifications and subscribe to the topic.
  • /insertCompanyNotifications : Used to insert company update notifications to the subscribed topic.
  • /getCompanyNotifications : Used to retrieve information about company updates.

The following diagram shows the overall solution. The user creates a topic, stores some company update notifications, and then receives them back. To invoke each operation, the user uses the same API.

pub-sub connector example

If you do not want to configure this yourself, you can simply get the project and run it.

Set up the integration project

Follow the steps in create integration project guide to set up the integration project.

Add integration logic

First create an API, which will be where we configure the integration logic. Specify the API name as pubsubApi and API context as /resources.

Adding a Rest API

Configure the API

Configure a resource for the /createTopic operation

  1. Initialize the connector.

    1. Create the /createTopic resource. We will use a URL template called /createTopic and the method will be Post.

    2. Click Connection and select Google Pub Sub to initialize the connector.

    3. Then fill the clientId, clientSecret, and refreshToken fields with the values that you obtained while setting up the Google Pub/Sub environment.

      • clientId : The client ID of the Google Cloud project.
      • clientSecret : The client secret of the Google Cloud project.
      • refreshToken : The refresh token that you obtained while setting up the Google Pub/Sub environment.

      Initialize connector

  2. Set up the Create Topic operation.

    Navigate into the Connectors pane and select the createTopic operation listed under Googlepubsub Connector section. The createTopic operation creates a new topic with the name that you specify.

    Add values to the createTopic operation

    • projectId : The unique ID of the project within which you want to create a topic.
    • topicName : The name that you want to give the topic that you are creating.

    While invoking the API, topicName values is populated as an input value for the operation.

    Therefore, we can use ${$payload.topicName} to capture the topicName value from the request payload.

    Add values to the createTopic operation

  3. Set up the Subscribe Topic operation.

    Navigate into the Connectors pane and select the Googlepubsub Connector section. Then select the Subscribe Topic operation.

    • projectId : The unique ID of the project within which the topic is created.
    • subscriptionName : The name of the subscription.
    • topicName : The name of the topic for which you want to create a subscription.

    Add values to the createTopicSubscription operation

  4. Forward the backend response to the API caller.

    When you are invoke the created resource, the request for the message is going through the /createTopic resource.Then it will invoke the Create Topic operation and then the Subscribe Topic operation. The response from these operations is captured in variables and then a payload mediator is used to format the response. Finally, it is passed to the Respond mediator. The Respond Mediator stops the processing of the current message and sends the message back to the client as a response.

    1. Add a payload mediator to capture the response from the Create Topic operation and Subscribe Topic operation.

      Add Payload mediator

      (You can click on the fx icon and go to variables to select the response payload from the Create Topic operation and Subscribe Topic operation.)

    2. Add the respond mediator to the Design view.

      Add Respond mediator

    3. Once you have setup the sequences and API, you can see the /createTopic resource as shown below.

      Overall createTopic resource

Configure a resource for the /insertCompanyNotifications operation

  1. Create the /insertCompanyNotifications resource. We will use a URL template called /insertCompanyNotifications and the method will be Post.

    Add insertCompanyNotifications resource

  2. Set up the publishMessage operation. Navigate into the Connectors pane and select the Google Pub/Sub Connector section. Then select the publishMessage operation.

    • projectId : The unique ID of the project within which the topic is created.
    • topicName : The name of the topic for which you want to create a subscription.
    • data : The message payload.

    Add values to the publishMessage operation

  3. Add the Respond mediator to return the response to the API caller. The response will contain the message ID of the published message. Overall, the /insertCompanyNotifications resource will look like the following.

    Overall insertCompanyNotifications resource

Configure a resource for the /getCompanyNotifications operation

  1. Create the /getCompanyNotifications resource. We will use a URL template called /getCompanyNotifications and the method will be Post.

    Add getCompanyNotifications resource

  2. Set up the pullMessage operation. Navigate into the Connectors pane and select the Google Pub/Sub Connector section. Then select the pullMessage operation.

    • projectId : The unique ID of the project within which the topic is created.
    • subscriptionName : The name of the subscription from which you want to pull messages.
    • maxMessages : The maximum number of messages to return.
    • returnImmediately : If set to true, the server will return immediately even if there are no messages available to return.

    Add values to the pullMessage operation

  3. Add the Respond mediator to return the response to the API caller. The response will contain the received messages. Overall, the /getCompanyNotifications resource will look like the following.

    Overall getCompanyNotifications resource

Now you can switch into the Source view and check the XML configuration files of the created API and sequences.

Full Synapse Code : pubsubApi.xml
<?xml version="1.0" encoding="UTF-8"?>
<api context="/resources" name="pubsubApi" xmlns="http://ws.apache.org/ns/synapse">
    <resource methods="POST" uri-template="/createTopic">
        <inSequence>
            <googlepubsub.createTopic configKey="GooglePubSub">
                <projectId>mirelated-455805</projectId>
                <topicName>{${payload.topicName}}</topicName>
                <responseVariable>googlepubsub_createTopic_1</responseVariable>
                <overwriteBody>false</overwriteBody>
            </googlepubsub.createTopic>
        <googlepubsub.subscribeTopic configKey="GooglePubSub">
    <projectId >mirelated-455805</projectId>
    <subscriptionName >{${payload.subscriptionName}}</subscriptionName>
    <topicName >{${payload.topicName}}</topicName>
    <ackDeadlineSeconds ></ackDeadlineSeconds>
    <pushEndpoint ></pushEndpoint>
    <attributes ></attributes>
    <responseVariable >googlepubsub_subscribeTopic_1</responseVariable>
    <overwriteBody >false</overwriteBody>
            </googlepubsub.subscribeTopic>
            <payloadFactory media-type="json" template-type="default">
                <format>{"CreateTopicResponse":${vars.googlepubsub_createTopic_1.payload},"SubscribeTopicResponce":${vars.googlepubsub_subscribeTopic_1.payload}}</format>
            </payloadFactory>
            <respond/>
        </inSequence>
        <faultSequence>
        </faultSequence>
    </resource>
    <resource methods="POST" uri-template="/insertCompanyNotifications">
        <inSequence>
        <googlepubsub.publishMessage configKey="GooglePubSub">
    <projectId >mirelated-455805</projectId>
    <topicName >{${payload.topicName}}</topicName>
    <data >{${payload.notification}}</data>
    <attributes ></attributes>
    <responseVariable >googlepubsub_publishMessage_1</responseVariable>
    <overwriteBody >true</overwriteBody>
            </googlepubsub.publishMessage>
            <respond/>
        </inSequence>
        <faultSequence>
        </faultSequence>
    </resource>
    <resource methods="POST" uri-template="/getCompanyNotifications">
        <inSequence>
            <googlepubsub.pullMessage configKey="GooglePubSub">
                <projectId>mirelated-455805</projectId>
                <subscriptionName>{${payload.subscriptionName}}</subscriptionName>
                <maxMessages>10</maxMessages>
                <returnImmediately>true</returnImmediately>
                <responseVariable>googlepubsub_pullMessage_1</responseVariable>
                <overwriteBody>true</overwriteBody>
            </googlepubsub.pullMessage>
            <respond/>
        </inSequence>
        <faultSequence>
        </faultSequence>
    </resource>
</api>

Get the project

You can download the ZIP file and extract the contents to get the project code.

Download ZIP

Tip

You may need to update the connection configurations and make other such changes before deploying and running this project.

Deployment

In order to deploy and run the project, refer the build and run guide.

Test

Invoke the API as shown below using the inbuilt API tester or using a tool like Postman or curl.

  1. Create a Topic for store company update notifications.

    Sample request

    Create Topic request

    Expected response

    Create Topic response

    You will see the results from G-Cloud console

    • Created Topic.

      pubsub-gcloudTopic

    • Created subscription for the Topic that you specify in the G-Cloud.

      pubsub-gcloudSubscription

  2. Insert company update notifications to the created topic.

    Sample request

    Insert company notifications request

    Expected response

    Insert company notifications response

  3. Retrieve company updates from the created topic.

    Sample request

    Get company notifications request

    Expected response

    Get company notifications response