Exporting Logs with OpenTelemetry¶ WSO2 MI can export logs over the OpenTelemetry Protocol (OTLP), which lets you send structured log data to any compatible observability backend. The setup involves three steps: updating the MI log format, enabling OpenTelemetry in MI, and configuring an OpenTelemetry Collector to forward the logs. Step 1 - Update the MI log configuration¶ Add trace and span IDs to each log line so that logs can be correlated with traces. Open <MI_HOME>/conf/log4j2.properties and update the CARBON_LOGFILE appender's layout pattern: appender.CARBON_LOGFILE.layout.pattern = [%d] %5p {%c} - %m%ex %notEmpty{trace_id=%X{trace_id}} %notEmpty{ span_id=%X{span_id}}%n After this change, log lines will include trace and span IDs when a request is processed. For example: [2023-10-25 10:00:00,000] INFO {org.wso2.carbon.core.internal.CarbonCoreDataHolder} - Message execution started trace_id=5b8aa5a2d2c872e8321acf1234567890 span_id=1234567890abcdef Note If you also want trace and span IDs to appear in the console log, apply the same pattern update to the CARBON_CONSOLE appender in log4j2.properties: appender.CARBON_CONSOLE.layout.pattern = [%d] %5p {%c} - %m%ex %notEmpty{trace_id=%X{trace_id}} %notEmpty{ span_id=%X{span_id}}%n Step 2 - Enable OpenTelemetry in WSO2 MI¶ Enable the OpenTelemetry publisher and point it to the gRPC endpoint of your OpenTelemetry Collector. Add the following to your deployment.toml file: FormatExample [opentelemetry] enable = true type = "otlp" host = "<hostname-of-otel-collector>" port = "<port-of-otel-collector>" # instead of 'host' and 'port', 'url' can be used directly in the following way. url = "<url-of-otel-collector>" [opentelemetry] enable = true type = "otlp" host = "localhost" port = 4317 # or url = "http://localhost:4317" Step 3 - Configure the OpenTelemetry Collector¶ The OpenTelemetry Collector reads the MI log file and forwards the logs to your observability backend. Create a file named otel-collector-config.yaml with the following content: receivers: filelog: include: [ /var/log/wso2mi/wso2carbon.log ] start_at: end storage: file_storage processors: resource: attributes: - key: service.name value: "WSO2-SYNAPSE" action: upsert exporters: otlp: endpoint: "<backend-host>:4317" tls: insecure: true extensions: file_storage: directory: /etc/otelcol-contrib/.data create_directory: true service: extensions: [ file_storage ] pipelines: logs: receivers: [ filelog ] processors: [ resource ] exporters: [ otlp ] Replace <backend-host> with the hostname of your observability backend's OTLP receiver. If your backend requires TLS or authentication, update the exporters.otlp section accordingly - refer to your backend's documentation for the exact configuration. The filelog receiver tails the wso2carbon.log file. The resource processor tags each log record with the service name WSO2-SYNAPSE. The otlp exporter sends the logs to your backend over gRPC. Once the configuration file is ready, start the OpenTelemetry Collector Contrib by pointing it to the otel-collector-config.yaml file you created. docker run --rm \ -v $(pwd)/otel-collector-config.yaml:/etc/otelcol-contrib/config.yaml \ -v <MI_HOME>/repository/logs:/var/log/wso2mi \ -v $(pwd)/.data:/etc/otelcol-contrib/.data \ otel/opentelemetry-collector-contrib:latest \ --config /etc/otelcol-contrib/config.yaml Note Make sure the Collector has read access to the wso2carbon.log file specified in the filelog receiver. The file_storage extension persists the read offset so that the Collector resumes from where it left off if restarted. Once both the WSO2 MI server and the OpenTelemetry Collector are running, the Collector will begin tailing the log file and forwarding structured log records to your backend automatically. See also For a complete end-to-end observability setup using OpenTelemetry - including traces, metrics, and logs - see OpenTelemetry-Based Observability.