Skip to content

Intelligent Document Processing (IDP) Connector Schema Generation

The IDP Schema Generation tool enables you to create, edit, and test JSON schemas for document extraction. You can build schemas manually using the UI, extract them automatically from documents using AI, or edit them directly as JSON. Once finalized, schemas are stored in the Governance Registry.


Components of IDP Schema Generation

1. Schema Editor

The Schema Editor lets you visually design the output schema for your document extraction. You can add fields, nested fields, tables (arrays), and object tables.

Supported Field Parameters

Parameter Name Element Name Description
name - The key used at runtime for this parameter.
type type The output type. Supported types: string, number, integer, boolean.
description description Prompt for AI to extract data for this field.
format format Predefined formats for string fields:
FormatDescription
date-timeISO 8601 date and time (e.g., 2024-06-01T12:00:00Z)
timeTime only (e.g., 12:00:00)
dateDate only (e.g., 2024-06-01)
pattern pattern Regex pattern for validation.
Note: Local Ollama models do not support regex validation. Avoid using pattern if you are using local Ollama models.
allowed values enum Allowed values for the field, entered as comma-separated list.

Creating a Schema Using the UI

Create a Field

Field creation UI placeholder

  • Select the field parameters (name, type, etc.) and click Add.
Create Nested Field

To create a nested field, use the dot operator to specify the hierarchy. For example, if you want to create a field named city inside a location object, which itself is inside a shop object, enter shop.location.city as the field name.

  • Enter shop.location.city as the field name.
  • This will create the structure:
    shop → location → city

Nested field UI placeholder

Create Array (Table of Primitive Data Types)
  • Go to Tables View.
  • Add a table, select type as string, integer, etc.
  • Fill in all necessary fields and click Add.

Array/table UI placeholder

Create Table of Objects (Array of Objects)
  • Go to Tables View.
  • Select type as object.
  • Under Table Items, add all fields for the object.
  • Click Save.

Object table UI placeholder


Extract Schema Using AI

You can extract a schema from a PDF or image using AI:

  • Upload your document (PDF/image).
  • Click Extract Schema.
  • The schema is generated based on the document.

AI extraction UI placeholder


Edit Existing Schema Using AI

You can use the Copilot chat interface below the Schema Editor to make changes to your schema using natural language instructions.

  • Chat with Copilot to make changes to the schema.

Chat UI placeholder

Direct JSON Editing

You can directly view and edit the raw JSON schema source.

  • Click the Show Source icon (top-right) to view and edit the raw JSON file.

Source code UI placeholder


2. TryOut Panel

Once your schema is ready:

  • Click Go to TryOut.

Go to TryOut UI placeholder

  • Upload another document to test extraction against your schema.
  • Select your connection (built-in Connection given by wso2 or custom).

TryOut connections UI placeholder

  • The extraction results for your uploaded document will be displayed as below.

TryOut result UI placeholder


Managing and Editing Saved Schemas

All changes are autosaved. You can close the window at any time.

To edit a saved schema later:

  • Go to WSO2 Integrator: MI Project Explorer.
  • Under idp-schemas, find your schema.
  • Click to edit again.

Project explorer UI placeholder