Skip to content

Intelligent Document Processing (IDP) Connector Reference

The following operation allows you to work with the Intelligent Document Processing Connector. Click the operation name to see parameter details and samples on how to use it.


Connection configurations

The IDP connector allows you to work with any OpenAI-compatible endpoints. Some tested LLMs are listed below. If you want to use a different LLM endpoint than those listed, choose the custom LLM connection option.

  • OpenAI: Standard OpenAI endpoints.
  • Ollama: Local LLMs served via Ollama server.
  • Custom LLM: Any other OpenAI-compatible endpoint.

types of llm connections

Regardless of the endpoint, all endpoints use the same set of common configurations as below.

Common configs to all connection types

Parameter Name Element Type Description Default Value Required
Connection Name name String A unique name to identify the connection. - Yes
API Key apiKey String The API key used to authenticate with the LLM endpoint. - Yes
Model model String The multimodal LLM model that supports structured responses.
Examples:
- OpenAI: gpt-4o-mini or higher
- Ollama: qwen2.5vl:3b
OpenAI: gpt-4.1-mini
Ollama: qwen2.5vl:3b
No
Endpoint URL endpointUrl String The endpoint URL of any service that is compatible with OpenAI's v1/chat/completions API.
Examples:
- OpenAI: https://api.openai.com/v1/chat/completions
- Ollama: http://localhost:11434/v1/chat/completions
OpenAI: https://api.openai.com/v1/chat/completions
Ollama: http://localhost:11434/v1/chat/completions
No

Note

  • The model should be a multimodal model (especially for Ollama) that can process both images and text. Refer to Ollama multimodal models.
  • The model should be capable of producing structured outputs (for example, OpenAI GPT-4.0 and above). See the OpenAI models documentation for more details.

Sample configuration

<?xml version="1.0" encoding="UTF-8"?>
<localEntry key="ollama_connection_1" xmlns="http://ws.apache.org/ns/synapse">
    <idp.init>
        <connectionType>OLLAMA</connectionType>
        <apiKey>ollama</apiKey>
        <model>qwen2.5vl:3b</model>
        <endpointUrl>http://localhost:11434/v1/chat/completions</endpointUrl>
        <name>ollama_connection_1</name>
    </idp.init>
</localEntry>

Tip

If you experience a socket timeout error (especially when using Ollama server), increase the HTTP socket timeout in the <PRODUCT_HOME>/conf/deployment.toml file as shown below:

[transport.http]
socket_timeout = 1200000   # timeout in milliseconds

Document Processing

processDocuments

The processDocuments operation submits a document to the configured backend service for data extraction based on a defined schema.

Parameter Name Element Type Description Default Value Required
Connection configKey String The LLM connection to use for document processing. - Yes
Content Format contentFormat String The format of the input file content.
Allowed values: Base64, Data URI.
Note: In MI server, multipart form data will be available as Base64. If you are sending an image as form data, use Base64.
Base64 No
MIME Type mimeType String The MIME type of the file being processed.
Allowed values (when using Base64): image/png, image/jpeg, image/gif, image/webp, application/pdf
- No
Content fileContent String The content of the image or PDF to be processed, based on the selected content format. - Yes
Schema idpSchema String The predefined schema for data extraction.
If you do not have a schema, click +Add new schema. Refer to the Schema Generation Reference for more information.
- Yes
Max Tokens maxTokens Integer The maximum number of tokens to generate in the completion. 4096 No
Response Variable responseVariable String Name of the variable to which the output of the operation should be assigned. - No
Overwrite Body overwriteBody Boolean Replace the Message Body in the Message Context with the response of the operation. false No

Sample configuration

<idp.processDocuments configKey="ollama_connection_1">
    <maxTokens>4096</maxTokens>
    <fileContent>{${payload.file}}</fileContent>
    <idpSchema>schema_2</idpSchema>
    <responseVariable>idp_processDocuments_1</responseVariable>
    <overwriteBody>true</overwriteBody>
    <contentFormat>Base64</contentFormat>
    <mimeType>image/jpeg</mimeType>
</idp.processDocuments>

Sample request

Following is a sample REST/JSON request that can be handled by the processDocuments operation.

{
    "file": "<Base64 string>"
}

Response

The response may vary based on the defined schema.