Intelligent Document Processing (IDP) Connector Reference¶ The following operation allows you to work with the Intelligent Document Processing Connector. Click the operation name to see parameter details and samples on how to use it. Connection configurations¶ The IDP connector allows you to work with any OpenAI-compatible endpoints. Some tested LLMs are listed below. If you want to use a different LLM endpoint than those listed, choose the custom LLM connection option. OpenAI: Standard OpenAI endpoints. Ollama: Local LLMs served via Ollama server. Custom LLM: Any other OpenAI-compatible endpoint. Regardless of the endpoint, all endpoints use the same set of common configurations as below. Common configs to all connection types Parameter Name Element Type Description Default Value Required Connection Name name String A unique name to identify the connection. - Yes API Key apiKey String The API key used to authenticate with the LLM endpoint. - Yes Model model String The multimodal LLM model that supports structured responses. Examples: - OpenAI: gpt-4o-mini or higher - Ollama: qwen2.5vl:3b OpenAI: gpt-4.1-mini Ollama: qwen2.5vl:3b No Endpoint URL endpointUrl String The endpoint URL of any service that is compatible with OpenAI's v1/chat/completions API. Examples: - OpenAI: https://api.openai.com/v1/chat/completions - Ollama: http://localhost:11434/v1/chat/completions OpenAI: https://api.openai.com/v1/chat/completions Ollama: http://localhost:11434/v1/chat/completions No Note The model should be a multimodal model (especially for Ollama) that can process both images and text. Refer to Ollama multimodal models. The model should be capable of producing structured outputs (for example, OpenAI GPT-4.0 and above). See the OpenAI models documentation for more details. Sample configuration <?xml version="1.0" encoding="UTF-8"?> <localEntry key="ollama_connection_1" xmlns="http://ws.apache.org/ns/synapse"> <idp.init> <connectionType>OLLAMA</connectionType> <apiKey>ollama</apiKey> <model>qwen2.5vl:3b</model> <endpointUrl>http://localhost:11434/v1/chat/completions</endpointUrl> <name>ollama_connection_1</name> </idp.init> </localEntry> Tip If you experience a socket timeout error (especially when using Ollama server), increase the HTTP socket timeout in the <PRODUCT_HOME>/conf/deployment.toml file as shown below: [transport.http] socket_timeout = 1200000 # timeout in milliseconds Document Processing¶ processDocumentsThe processDocuments operation submits a document to the configured backend service for data extraction based on a defined schema. Parameter Name Element Type Description Default Value Required Connection configKey String The LLM connection to use for document processing. - Yes Content Format contentFormat String The format of the input file content. Allowed values: Base64, Data URI. Note: In MI server, multipart form data will be available as Base64. If you are sending an image as form data, use Base64. Base64 No MIME Type mimeType String The MIME type of the file being processed. Allowed values (when using Base64): image/png, image/jpeg, image/gif, image/webp, application/pdf - No Content fileContent String The content of the image or PDF to be processed, based on the selected content format. - Yes Schema idpSchema String The predefined schema for data extraction. If you do not have a schema, click +Add new schema. Refer to the Schema Generation Reference for more information. - Yes Max Tokens maxTokens Integer The maximum number of tokens to generate in the completion. 4096 No Response Variable responseVariable String Name of the variable to which the output of the operation should be assigned. - No Overwrite Body overwriteBody Boolean Replace the Message Body in the Message Context with the response of the operation. false No Sample configuration <idp.processDocuments configKey="ollama_connection_1"> <maxTokens>4096</maxTokens> <fileContent>{${payload.file}}</fileContent> <idpSchema>schema_2</idpSchema> <responseVariable>idp_processDocuments_1</responseVariable> <overwriteBody>true</overwriteBody> <contentFormat>Base64</contentFormat> <mimeType>image/jpeg</mimeType> </idp.processDocuments> Sample request Following is a sample REST/JSON request that can be handled by the processDocuments operation. { "file": "<Base64 string>" } Response The response may vary based on the defined schema.