File Connector Reference¶ The following configurations allow you to work with the File Connector version 4. Connection configurations¶ The File connector can be used to deal with two types of file systems: Local File System: A file system of the server where the WSO2 integration runtime is deployed. Remote File System: A file system outside the server where the WSO2 integration runtime is deployed. There are few industry standard protocols established to expose a file system over TCP. Following protocols are supported by the File connector. FTP FTPS SFTP SMB2 There are different connection configurations that can be used for the above protocols. They contain a common set of configurations and some additional configurations specific to the protocol. Note The File connector internally uses the Apache VFS Library. According to the selected connection type, the following VFS connection URLs will be generated. Local FileFTPFTPSSFTPSMB2 [file://] absolute-path file:///home/someuser/somedir file:///C:/Documents and Settings ftp://[ username[: password]@] hostname[: port][ relative-path] ftp://myusername:mypassword@somehost/pub/downloads/somefile.tgz ftps://[ username[: password]@] hostname[: port][ absolute-path] ftps://myusername:mypassword@somehost/pub/downloads/somefile.tgz sftp://[ username[: password]@] hostname[: port][ relative-path] sftp://myusername:mypassword@somehost/pub/downloads/somefile.tgz smb2://[ username[: password]@] hostname[: port][ relative-path] smb2://myusername:mypassword@somehost:445/SMBTesting Tip There are instances where errors occur when using .csv files and the output is encoded. To overcome this, add the following configuration to the <PRODUCT_HOME>/repository/conf/deployment.toml file. [[custom_message_formatters]] content_type = "text/csv" class = "org.apache.axis2.format.PlainTextFormatter" [[custom_message_builders]] content_type = "text/csv" class = "org.apache.axis2.format.PlainTextBuilder" Also, you need to modify your proxy service as indicated below. <?xml version="1.0" encoding="UTF-8"?> <sequence name="proxyDeployingSeq" trace="disable" xmlns="http://ws.apache.org/ns/synapse"> <property name="slotNumber" value="1"/> <property expression="get-property("SYSTEM_DATE", "mm")" name="currentMin" scope="default" type="STRING"/> <file.read configKey="slotFileConnection"> <path>/csv/slot.csv</path> <readMode>Specific Line</readMode> <startLineNum>0</startLineNum> <endLineNum>0</endLineNum> <lineNum>{get-property('currentMin')}</lineNum> <contentType>text/csv</contentType> <includeResultTo>Message Property</includeResultTo> <resultPropertyName>slotNumber</resultPropertyName> <enableStreaming>false</enableStreaming> <enableLock>false</enableLock> </file.read> <log level="custom"> <property name="slott" expression="get-property('slotNumber')"/> </log> </sequence> Common configs to all connection types¶ Parameter Name Element Type Description Default Value Required Connection Name name String A unique name to identify the connection. - Yes Connection Type connectionType String The protocol used for communicating with the file system. Possible values: Local: Provides access to the files on the local physical file system. FTP: Provides access to the files on an FTP server. FTPS: Provides access to the files on an FTP server over SSL. SFTP: Provides access to the files on an SFTP server (that is, an SSH or SCP server). SMB2: Provides access to the files on a Samba server. - Yes Working Directory workingDir String This is the working directory. The file paths in operations, which are associated with the connection, should be provided relative to this folder. Note: As per VFS documentation, for windows, the working directory of local connections should be as follows: /C:/Documents. Defaults to file system root. No File Locking Behaviour fileLockScheme String Specify whether to acquire node-specific lock (Local) or cluster-wide lock (Cluster) when locks are acquired in read and write operations. Local When a lock is acquired, it is acquired within the context of file operations performed by that server node only. Local lock acquired by some file operation on a particular server node is not visible to the other server nodes that may access the same file system. Cluster When multiple server nodes access the same file system performing read and write operations, you may use this behaviour. Here, when a file lock is acquired, it is visible to all file connector operations across the nodes. This is acquired by creating a .lock file in the same file system (for the file that is being accessed). The behaviour depends on the OS and the file system. Therefore, this feature may not work as intended in high-concurrent scenarios. Note: File locking is available for read and write operations. When enabled, a file specific lock is acquired before the operation and released after the operation. Parallel read/write operations are restricted when locking is enabled by a file operation. Local Yes Common remote connection configs¶ Parameter Name Element Type Description Default Value Required Host host String Host name of the file server. - Yes Port port Number The port number of the file server - Yes Username username String The username used to connect with the file server. If the username contains special characters, you will need to use the URL encoded value. - No Password password String The password to connect with the file server. If the password contains special characters, you will need to use the URL encoded value. - No User Directory Is Root userDirIsRoot Boolean If set to false (default), VFS will choose the file system's root as the VFS's root. If you want to have the user's home as the VFS root, then set this to 'true'. false No FTP/FTPS-specific configs¶ Parameter Name Element Type Description Default Value Required Is Passive isPassive Boolean If passive mode is enabled, set this to 'true'. Note the following about 'Active/Passive' mode: Active Mode: The client starts listening on a random port for incoming data connections from the server (the client sends the FTP command PORT to inform the server on which port it is listening). Nowadays, the client is typically behind a firewall (e.g. built-in Windows firewall) or NAT router (e.g. ADSL modem), unable to accept incoming TCP connections. The passive mode was introduced and is heavily used for this reason. Passive Mode: In the passive mode, the client uses the control connection to send a PASV command to the server and then receives a server IP address and server port number from the server, which the client then uses to open a data connection to the server IP address and server port number received. true No FTP Connection Timeout ftpConnectionTimeout Number Specify the timeout in milliseconds for the initial control connection. 100000 No FTP Socket Timeout ftpSocketTimeout Number Specify the socket timeout in milliseconds for the FTP client. 150000 No FTPS-specific configs¶ Parameter Name Element Type Description Default Value Required KeyStore Path keyStorePath String The keystore path. - No KeyStore Password keyStorePassword String The password to the keystore. - No TrustStore Path trustStorePath String The truststore path. - Yes TrustStore Password trustStorePassword String The password to the truststore. - Yes Implicit Mode Enabled implicitModeEnabled Boolean Set this to 'true' if implicit mode is enabled. Implicit: The TLS ClientHello message should be initiated by client. Explicit: The client must "explicitly request" security from an FTPS server. false No Channel Protection Level channelProtectionLevel String The FTP Data Channel protection level. Possible values: C,S,E,P. Example: Sends a “PROT P” command when implicit SSL is enabled. SFTP connection configs¶ Parameter Name Element Type Description Default Value Required SFTP Connection Timeout sftpConnectionTimeout Number The Jsch connection timeout in milli seconds. - No SFTP Session Timeout sftpSessionTimeout Number The Jsch session timeout in milli seconds. 100000 No Strict Host Key Check strictHostKeyChecking Boolean Specifies whether the Host key should be checked. If set to 'true', the connector (JSch) will always verify the public key (fingerprint) of the SSH/SFTP server. false No Private Key File privateKeyFilePath String Path to the private key file. Note: You can only use a key generated in a classic manner (ssh-keygen -m PEM). false No Private Key Passphrase privateKeyPassword String The passphrase of the private key. The security of a key (even if encrypted) is retained because it is not available to anyone else. You can specify the passphrase when generating keys. false No File System Permission Check setAvoidPermission Boolean Set to true if you want to skip the file permission check. Available in file-connector v4.0.9 and above. false No Cache File Stats cacheFileStats Boolean Enable this to cache file stats for better performance. Since the stats(metadata) are cached, if another process modifies the metadata while the file is being read, those changes may not be detected by the File Connector operation. Available in file-connector v4.0.43 and above. false No Info The following SFTP connection parameters are available in File Connector version 4.0.21 and later. Note The WSO2 File Connector uses connection pooling for enhanced performance. It uses the Apache Commons Pool as the base framework for connection pooling. For detailed information on the pooling mechanism, refer to the Apache Commons Pool documentation. The following parameters can be used to fine-tune the connection pool according to your requirements. Parameter Name Element Type Description Default Value Required Connection Pool Aged Timeout sftpPoolConnectionAgedTimeout Integer Interval to close connections in the connection pool in seconds. Never No Max Active Connections maxActiveConnections Integer The maximum number of connections (including both idle and active/borrowed) that can exist within the pool at a given time. 8 No Max Idle Connections maxIdleConnections Integer The maximum number of connections that can remain idle in the pool at any time, awaiting to be borrowed. Excess idle objects may be removed. 8 No Max Wait Time maxWaitTime Integer This parameter determines how long the connector is willing to wait in the queue for a connection to become available. If the wait time exceeds the configured maximum wait time, the pool may throw an exception when it is exhausted and no connections are available. indefinite (wait forever) No Eviction Check Interval evictionCheckInterval Integer This parameter specifies how frequently the evictor thread scans the pool for idle connections eligible for eviction. By configuring this interval, developers can control the frequency of resource checks, optimizing performance without unnecessary overhead. Eviction doesn't run if this isn't defined. No Min Eviction Time minEvictionTime Integer Connections in the pool must remain idle for at least this specified duration before the evictor considers them for removal. This ensures that only connections inactive beyond a defined threshold are evicted, preventing premature eviction of frequently used resources. Eviction doesn't run if this and evictionCheckInterval aren't defined. No Exhausted Action exhaustedAction String Determines the action to take when the borrowObject() method is called, but the pool is exhausted. WHEN_EXHAUSTED_BLOCK No SMB connection configs¶ Info The following SMB connection parameters are available in File Connector version 4.0.26 and later. Note The WSO2 File Connector uses connection pooling for enhanced performance. It uses the Apache Commons Pool as the base framework for connection pooling. For detailed information on the pooling mechanism, refer to the Apache Commons Pool documentation. The following parameters can be used to fine-tune the connection pool according to your requirements. SMB2 servers will close idle connections forcefully based on their configurations. In such cases, the WSO2 Integrator: MI may throw connection errors. To avoid these errors, you can fine-tune the connection eviction configurations to remove idle connections from the pool. Parameter Name Element Type Description Default Value Required Connection Pool Aged Timeout sftpPoolConnectionAgedTimeout Integer Interval to close connections in the connection pool in seconds. Never No Max Active Connections maxActiveConnections Integer The maximum number of connections (including both idle and active/borrowed) that can exist within the pool at a given time. 8 No Max Idle Connections maxIdleConnections Integer The maximum number of connections that can remain idle in the pool at any time, awaiting to be borrowed. Excess idle objects may be removed. 8 No Max Wait Time maxWaitTime Integer This parameter determines how long the connector is willing to wait in the queue for a connection to become available. If the wait time exceeds the configured maximum wait time, the pool may throw an exception when it is exhausted and no connections are available. indefinite (wait forever) No Eviction Check Interval evictionCheckInterval Integer This parameter specifies how frequently the evictor thread scans the pool for idle connections eligible for eviction. By configuring this interval, developers can control the frequency of resource checks, optimizing performance without unnecessary overhead. Eviction doesn't run if this isn't defined. No Min Eviction Time minEvictionTime Integer Connections in the pool must remain idle for at least this specified duration before the evictor considers them for removal. This ensures that only connections inactive beyond a defined threshold are evicted, preventing premature eviction of frequently used resources. Eviction doesn't run if this and evictionCheckInterval aren't defined. No Exhausted Action exhaustedAction String Determines the action to take when the borrowObject() method is called, but the pool is exhausted. WHEN_EXHAUSTED_BLOCK No Enable Encryption enableEncryption Boolean Determines whether to enable data encryption. This must be enabled when connecting to an SMB server that enforces encryption. Available in file-connector version v4.0.34 and above. false No Encode Password encodePassword Boolean Determines whether special characters in the password should be URL-encoded. Available in file-connector version v4.0.30 and above. false No Operations¶ The following operations allow you to work with the File Connector version 4. Click an operation name to see parameter details and samples on how to use it. createDirectoryCreates a new folder in a provided directory path. Parameter Name Element Type Description Default Value Required File Connection name String The name of the file connection configuration to use. - Yes Directory Path directoryPath String The new directory path. - Yes Sample configuration <file.createDirectory configKey="CONNECTION_NAME"> <directoryPath>{$ctx:directoryPath}</directoryPath> </file.createDirectory> Sample request Following is a sample REST/JSON request that can be handled by the append operation. { "directoryPath":"/home/vive/Desktop/file", } Response <createDirectoryResult> <success>true</success> </createDirectoryResult> checkExistCheck if a given file or folder exists. Parameter Name Element Type Description Default Value Required File Connection name String The name of the file connection configuration to use. - Yes File/Folder Path path String The new directory path that should be scanned. - Yes Disk Share Access Mask diskShareAccessMask String The diskShareAccessMask configuration applies when using the SMB2 or SMB3 protocol for file operations through the file connector. This setting defines the access permissions (ACCESS_MASK) for the disk share as a comma separated string. Refer to the SMBJ documentation for the available access masks. Note Requires WSO2 Integrator: MI version 4.4.0.7 (U2) or later and file connector version 4.0.39 or later to use this parameter. MAXIMUM_ALLOWED No Sample configuration <file.checkExist configKey="CONNECTION_NAME"> <path>{$ctx:path}</path> </file.checkExist> Sample request Following is a sample REST/JSON request that can be handled by the append operation. { "path":"/home/vive/Desktop/file/append.txt", } Response <checkExistResult> <success>true</success> <fileExists>true</fileExists> </checkExistResult> compressArchives a file or a directory. Parameter Name Element Type Description Default Value Required File Connection name String The name of the file connection configuration to use. - Yes Folder/File To Compress sourceDirectoryPath String The path to the folder that should be compressed. - Yes Targer File Path targetFilePath String The path to the compressed file that will be created. If the file already exists, it is overwritten. - Yes Include Sub Directories includeSubDirectories Boolean Specifies whether the sub folders in the original folder should be included in the compressed file. true No Sample configuration <file.compress configKey="CONNECTION_NAME"> <sourceDirectoryPath>{$ctx:sourceDirectoryPath}</sourceDirectoryPath> <targetFilePath>{$ctx:targetFilePath}</targetFilePath> <includeSubDirectories>{$ctx:includeSubDirectories}</includeSubDirectories> </file.compress> Sample request Following is a sample REST/JSON request that can be handled by the append operation. { "sourceDirectoryPath":"/home/vive/Desktop/file", "targetFilePath":"/home/user/test/file.zip" } Response <compressResult> <success>true</success> <NumberOfFilesAdded>16</NumberOfFilesAdded> </compressResult> Error <compressResult> <success>false</success> <code>700102</code> <detail>File or directory to compress does not exist</detail> </compressResult> copyCopies the file or folder specified by a source path to a target path. The source can be a file or a folder. If it is a folder, the copying is recursive. Parameter Name Element Type Description Default Value Required File Connection name String The name of the file connection configuration to use. - Yes Source Path sourcePath String The path to the file that should be copied. - Yes Targer Path targetPath String The location (folder) to which the file should be copied. If the target folder does not exist at the time of copy, a new folder is created. - Yes Source File Pattern sourceFilePattern String The file name pattern of the source file. Example: [a-zA-Z][a-zA-Z].(txt|xml|jar) - No Copy Including Source Parent includeParent Boolean Specify whether the parent folder should be copied from the file source along with the content. By default, only the content inside the folder will get copied. false No Overwrite Existing Files overwrite Boolean Specifies whether or not to overwrite the file if the same file already exists in the target destination. false No Rename To renameTo String The new name of the copied file. Original file name. No Max Retries maxRetries Integer The maximum number of retries to be done in case of a failure. 0 No Retry Interval retryDelay Integer The time interval between retries in milliseconds. 0 No Disk Share Access Mask diskShareAccessMask String The diskShareAccessMask configuration applies when using the **SMB2* or SMB3 protocol for file operations through the file connector. This setting defines the access permissions (ACCESS_MASK) for the disk share as a comma separated string. Refer to the SMBJ documentation for the available access masks. Note Requires WSO2 Integrator: MI version 4.4.0.7 (U2) or later and file connector version 4.0.39 or later to use this parameter. MAXIMUM_ALLOWED No Sample configuration <file.copy configKey="CONNECTION_NAME"> <sourcePath>{$ctx:sourcePath}</sourcePath> <targetPath>{$ctx:targetPath}</targetPath> <sourceFilePattern>{$ctx:sourceFilePattern}</sourceFilePattern> <includeParent>{$ctx:includeParent}</includeParent> <overwrite>{$ctx:overwrite}</overwrite> <renameTo>{$ctx:renameTo}</renameTo> <maxRetries>{$ctx:maxRetries}</maxRetries> <retryDelay>{$ctx:retryDelay}</retryDelay> </file.copy> Sample request Following is a sample REST/JSON request that can be handled by the append operation. { "sourcePath":"/home/vive/Desktop/file/append.txt", "targetPath":"/home/user/test/file" } Response <copyFilesResult> <success>true</success> </copyFilesResult> Error <copyFilesResult> <success>false</success> <code>700103</code> <detail>Destination file already exists and overwrite not allowed</detail> </copyFilesResult> moveMoves the file or folder specified by the source path to the target directory. The source can be a file or a folder. If it is a folder, the moving is recursive. The move operation can only move a file/folder within the same server. For example, you can move a file/folder from one local location to another local location, or from one remote location to another remote location on the same server. You cannot use the move operation to move a file/folder between different servers. Parameter Name Element Type Description Default Value Required File Connection name String The name of the file connection configuration to use. - Yes Source Path sourcePath String The path to the file that should be copied. - Yes Targer Path targetPath String The location to which the file should be copied. - Yes Create Parent Directories createParentDirectories Boolean Specifies whether the parent directory should be created if it doesn't already exist in the target folder. - No Include Parent includeParent Boolean Specify whether the parent folder should be copied from the file source along with the content. false No Overwrite Existing Files overwrite Boolean Specifies whether or not to overwrite the file if the same file already exists in the target destination. false No Rename To renameTo String The new name of the moved files. Original file name. No File Pattern filePattern String The pattern (regex) of the files to be moved. Example: [a-zA-Z][a-zA-Z].(txt|xml|jar). Available in file-connector v4.0.5 and above - No Is Source Mounted isSourceMounted Boolean Whether the source path is a mounted path or not. false No Is Target Mounted isTargetMounted Boolean Whether the target path is a mounted path or not. false No Disk Share Access Mask diskShareAccessMask String The diskShareAccessMask configuration applies when using the **SMB2* or SMB3 protocol for file operations through the file connector. This setting defines the access permissions (ACCESS_MASK) for the disk share as a comma separated string. Refer to the SMBJ documentation for the available access masks. Note Requires WSO2 Integrator: MI version 4.4.0.7 (U2) or later and file connector version 4.0.39 or later to use this parameter. MAXIMUM_ALLOWED No Sample configuration <file.move configKey="CONNECTION_NAME"> <sourcePath>{$ctx:sourcePath}</sourcePath> <targetPath>{$ctx:targetPath}</targetPath> <createParentDirectories>{$ctx:createParentDirectories}</createParentDirectories> <includeParent>{$ctx:includeParent}</includeParent> <overwrite>{$ctx:overwrite}</overwrite> <renameTo>{$ctx:renameTo}</renameTo> <filePattern>{$ctx:filePattern}</filePattern> <isSourceMounted>{$ctx:isSourceMounted}</isSourceMounted> <isTargetMounted>{$ctx:isTargetMounted}</isTargetMounted> </file.move> Sample request Following is a sample REST/JSON request that can be handled by the append operation. { "sourcePath":"/home/vive/Desktop/file/append.txt", "targetPath":"/home/user/test/file" } Response <moveFilesResult> <success>true</success> </moveFilesResult> Error <moveFilesResult> <success>false</success> <code>700103</code> <detail>Destination file already exists and overwrite not allowed</detail> </moveFilesResult> readReads the content and metadata of a file at a given path. Metadata of the file is added as properties while content is set to the message body (or optionally to a message context property). Known message properties representing file properties: Parameter Name Type Description Default Value Required FILE_LAST_MODIFIED_TIME DateTime The time at which the file was last modified. - Yes FILE_SIZE Number The file size (in bytes). - Yes FILE_IS_DIR Boolean Specifies whether a folder directory is represented as the file. false Yes FILE_PATH String The file path. - Yes FILE_URL String The VFS URL of the file. - Yes FILE_NAME String The file name or folder name. - Yes FILE_NAME_WITHOUT_EXTENSION String The file name without the extension. - Yes Important: When reading a folder, the first file that matches the pattern will be read first. Note that sub directories are not scanned. If you need to move or delete the file before reading the folder again, use the FILE_NAME context variable. The MIME type (content-type) of the message is determined by the file extension (i.e an XML file will be read as a message with the application/xml MIME type). However, users can force the MIME type by the ContentType parameter. Similarly, the Encoding parameter can be used to force the encoding. You can set EnableLock to true to enable file system lock until the reading is completed and the stream is closed. When large files are read, use streaming=true. Note that you need to first make necessary changes in the deployment.toml. The ContentType parameter also needs to be application/binary. Note that file reading modes are not applicable when streaming is set to true. The complete file is always streamed. Parameter Name Element Type Description Default Value Required File Connection name String The name of the file connection configuration to use. - Yes File Path path String The path to the file that should be read. - Yes File Pattern filePattern String Regex The file name pattern that should be matched when reading the file. All text files (..txt) No Add Result To includeResultTo String Specify where to add the result of the file that is read. Message Body Message Property: The payload that was in the message body before applying the file read operation will remain intact. - No Property Name resultPropertyName String If Add Result To==Message Property, you need to specify this value. Result of the file read operation will be added as a default scope property by the specified name. This can now be accessed later in the flow. - No Read Mode readMode String Available file reading modes: Complete File, Between Lines, Starting From Line, Up To Line, and Specific Line. Complete File. Yes Start Line Num startLineNum Number Starts reading the file from the specified line. 1 No End Line Num endLineNum Number Reads the file upto the specified line. Last line of file. No Specific Line number lineNum Number Specific line to read. When the reading mode is Specific Line. No MIMEType contentType String Content type of the message set to the payload by this operation Determined by the file extension. No Encoding encoding String Encoding of the message set to the payload by this operation. UTF-8 No Enable Streaming enableStreaming Boolean Specifies whether or not streaming is used to read the file without any interpretation of the content. false No Enable Locking enableLock Boolean Specifies whether or not to lock the file. false No Max Retries maxRetries Integer The maximum number of retries to be done in case of a failure. 0 No Retry Interval retryDelay Integer The time interval between retries in milliseconds. 0 No Disk Share Access Mask diskShareAccessMask String The diskShareAccessMask configuration applies when using the **SMB2* or SMB3 protocol for file operations through the file connector. This setting defines the access permissions (ACCESS_MASK) for the disk share as a comma separated string. Refer to the SMBJ documentation for the available access masks. Note Requires WSO2 Integrator: MI version 4.4.0.7 (U2) or later and file connector version 4.0.39 or later to use this parameter. MAXIMUM_ALLOWED No Sample configuration <file.read configKey="CONNECTION_NAME"> <path>{$ctx:path}</path> <filePattern>{$ctx:filePattern}</filePattern> <includeResultTo>{$ctx:includeResultTo}</includeResultTo> <resultPropertyName>{$ctx:resultPropertyName}</resultPropertyName> <readMode>{$ctx:readMode}</readMode> <startLineNum>{$ctx:startLineNum}</startLineNum> <endLineNum>{$ctx:endLineNum}</endLineNum> <lineNum>{$ctx:lineNum}</lineNum> <contentType>{$ctx:contentType}</contentType> <encoding>{$ctx:encoding}</encoding> <enableStreaming>{$ctx:enableStreaming}</enableStreaming> <enableLock>{$ctx:enableLock}</enableLock> <maxRetries>{$ctx:maxRetries}</maxRetries> <retryDelay>{$ctx:retryDelay}</retryDelay> </file.read> Sample request Following is a sample REST/JSON request that can be handled by the append operation. { "path":"/home/vive/Desktop/file/read.txt", "readMode":"Complete File" } Response This is line one. This is line two. This is line three. This is line four. This is line five. This is line six. This is line seven. This is line eight. This lis line nine. This is line ten. Full Log [2020-10-06 06:01:44,083] INFO {LogMediator} - {api:TestAPI} To: /filetest, MessageID: urn:uuid:7ab557c0-f9cb-4cf6-9c7b-f06a4640522a, Direction: request, message = After Read, FILE_LAST_MODIFIED_TIME = 10/06/2020 05:46:39, FILE_SIZE = 30, FILE_IS_DIR = false, FILE_NAME = test1.txt, FILE_PATH = /wso2/test, FILE_URL = file:///Users/hasitha/temp/file-connector-test/wso2/test/test1.txt, FILE_NAME_WITHOUT_EXTENSION = test1, Envelope: <?xml version='1.0' encoding='utf-8'?><soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"><soapenv:Body><text xmlns="http://ws.apache.org/commons/ns/payload">This is test1.txt file content</text></soapenv:Body></soapenv:Envelope> Error <readResult> <success>false</success> <code>700102</code> <detail>File or folder not found: file:///Users/hasitha/temp/file-connector-test/wso2/test/abcd.txt</detail> </readResult> renameRename a file in a specified path. The new name cannot contain path separators. Parameter Name Element Type Description Default Value Required File Connection name String The name of the file connection configuration to use. - Yes Path path String The path to the file that should be renamed. - Yes Rename To renameTo String The file's new name. - Yes Overwrite Existing Files overwrite Boolean Specifies whether or not to overwrite the file in the target directory (if the same file exists). false No Disk Share Access Mask diskShareAccessMask String The diskShareAccessMask configuration applies when using the SMB2 or SMB3 protocol for file operations through the file connector. This setting defines the access permissions (ACCESS_MASK) for the disk share as a comma separated string. Refer to the SMBJ documentation for the available access masks. Note Requires WSO2 Integrator: MI version 4.4.0.7 (U2) or later and file connector version 4.0.39 or later to use this parameter. MAXIMUM_ALLOWED No Sample configuration <file.rename configKey="CONNECTION_NAME"> <path>{$ctx:path}</path> <renameTo>{$ctx:renameTo}</renameTo> <overwrite>{$ctx:overwrite}</overwrite> </file.rename> Sample request Following is a sample REST/JSON request that can be handled by the append operation. { "path":"/home/vive/Desktop/file/read.txt", "renameTo":"test" } Response <renameFileResult> <success>true</success> </renameFileResult> Error <renameFileResult> <success>false</success> <code>700103</code> <detail>Destination file already exists and overwrite not allowed</detail> </renameFileResult> deleteDeletes the files matching in a given directory. Parameter Name Element Type Description Default Value Required File Connection name String The name of the file connection configuration to use. - Yes File/Directory Path path String The path to the file/folder that should be deleted. - Yes Pattern to Match Files matchingPattern String The pattern that should be matched when listing files. This does not operate recursively on sub folders. All files. No Max Retries maxRetries Integer The maximum number of retries to be done in case of a failure. 0 No Retry Interval retryDelay Integer The time interval between retries in milliseconds. 0 No Disk Share Access Mask diskShareAccessMask String The diskShareAccessMask configuration applies when using the SMB2 or SMB3 protocol for file operations through the file connector. This setting defines the access permissions (ACCESS_MASK) for the disk share as a comma separated string. Refer to the SMBJ documentation for the available access masks. Note Requires WSO2 Integrator: MI version 4.4.0.7 (U2) or later and file connector version 4.0.39 or later to use this parameter. MAXIMUM_ALLOWED No Sample configuration <file.delete configKey="CONNECTION_NAME"> <path>{$ctx:path}</path> <matchingPattern>{$ctx:matchingPattern}</matchingPattern> <maxRetries>{$ctx:maxRetries}</maxRetries> <retryDelay>{$ctx:retryDelay}</retryDelay> </file.delete> Sample request Following is a sample REST/JSON request that can be handled by the append operation. { "path":"/home/vive/Desktop/file/read.txt" } Response For a single file: <deleteFileResult> <success>true</success> </deleteFileResult> For a folder: <deleteFileResult> <success>true</success> <numOfDeletedFiles>5</numOfDeletedFiles> </deleteFileResult> unzipUnzip a specified file to a given location. If a folder with the same name exists, it is overwritten. Parameter Name Element Type Description Default Value Required File Connection name String The name of the file connection configuration to use. - Yes Zip File Path sourceFilePath String The path to the ZIP file that should be unzipped. - Yes Target Directory targetDirectory String The location (folder) to which the ZIP file should be unzipped. - Yes File Name Encoding fileNameEncoding String The character encoding to interpret the file names inside the ZIP archive. UTF-8 No Include File Names includeFileNames Boolean If set to true, the names of the extracted files are returned in the response under zipFileContent. Available from File Connector v4.0.45 and above. false No NOTE: The latest File connector (v4.0.7 onwards) supports decompressing the .gz files. Sample configuration <file.unzip configKey="CONNECTION_NAME"> <sourceFilePath>{$ctx:sourceFilePath}</sourceFilePath> <targetDirectory>{$ctx:targetDirectory}</targetDirectory> </file.unzip> Sample request Following is a sample REST/JSON request that can be handled by the append operation. { "sourceFilePath":"/home/vive/Desktop/file/test.zip", "targetDirectory":"/home/user/test/file" } Response By default (when includeFileNames is not set or is false): <unzipFileResult> <success>true</success> </unzipFileResult> When includeFileNames is set to true: <unzipFileResult> <success>true</success> <zipFileContent> <file>test1.txt</file> <file>test2.txt</file> <file>docs/a1.txt</file> <file>docs/archive/c1.txt</file> </zipFileContent> </unzipFileResult> On Error <unzipFileResult> <success>false</success> <code>700102</code> <detail>File not found: file:///Users/hasitha/temp/file-connector-test/wso2/archievenew.zip</detail> </unzipFileResult> JSON equivalent: { "unzipFileResult": { "success": false, "code": 700102, "detail": "File not found: file:///Users/hasitha/temp/file-connector-test/wso2/archievenew.zip" } } splitFileSplits a file into multiple smaller files. If the folder does not exist, it will be created. If the folder has files, they will be overwritten. Parameter Name Element Type Description Default Value Required File Connection name String The name of the file connection configuration to use. - Yes Path to the file to split sourceFilePath String The path to the file that should be split. - Yes Target Directory targetDirectory String The path to the target folder where the new files should be created. - Yes Split Mode splitMode String The split mode to use. The available options are as follows: ChunkSize Linecount XPATH Expression - Yes Chunk Size chunkSize Number If the Split Mode is 'Chunk Size', specify the chunk size (in bytes) into which the file should be split. - - Line Count lineCount Number If the Split Mode is 'Line Count', specify the number of lines by which the original file should be split. - - XPATH Expression xpathExpression Number If the Split Mode is 'XPATH Expression', specify the expression by which the file should be split. Only applies when splitting XML files. Chunk Size Yes Disk Share Access Mask diskShareAccessMask String The diskShareAccessMask configuration applies when using the SMB2 or SMB3 protocol for file operations through the file connector. This setting defines the access permissions (ACCESS_MASK) for the disk share as a comma separated string. Refer to the SMBJ documentation for the available access masks. Note Requires WSO2 Integrator: MI version 4.4.0.7 (U2) or later and file connector version 4.0.39 or later to use this parameter. MAXIMUM_ALLOWED No Sample configuration <file.splitFile configKey="CONNECTION_NAME"> <sourceFilePath>{$ctx:sourceFilePath}</sourceFilePath> <targetDirectory>{$ctx:targetDirectory}</targetDirectory> <splitMode>{$ctx:splitMode}</splitMode> <xpathExpression>{$ctx:xpathExpression}</xpathExpression> </file.splitFile> Sample request Following is a sample REST/JSON request that can be handled by the append operation. { "sourceFilePath":"/home/vive/Desktop/file/test.txt", "targetDirectory":"/home/user/test/file", "splitMode":"Linecount", "lineCount":"10" } Response <splitFileResult> <success>true</success> <numberOfSplits>6</numberOfSplits> </splitFileResult> On Error <splitFileResult> <success>false</success> <code>700107</code> <detail>Parameter 'xpathExpression' is not provided</detail> </splitFileResult> listFilesLists all the files (that match the specified pattern) in the directory path. Parameter Name Element Type Description Default Value Required File Connection name String The name of the file connection configuration to use. - Yes Directory Path directoryPath String The path to the directory from which files should be listed. - Yes Matching Pattern matchingPattern String The file pattern that should be used to select files for listing. - No List Files in Sub Directories recursive Boolean List files from sub directories recursively. false No File Sort Attribute sortingAttribute String Files will get sorted and listed according to one of the follow: Name, Size, LastModifiedTime. Name No Sort Order sortingOrder String The sorting order applicable to the File Sort attribute. Possible Values: Ascending, Descending. Ascending No Response Format responseFormat String Format to list the files in response. Possible Values: Hierarchical, Flat. Hierarchical No Max Retries maxRetries Integer The maximum number of retries to be done in case of a failure. 0 No Retry Interval retryDelay Integer The time interval between retries in milliseconds. 0 No Disk Share Access Mask diskShareAccessMask String The diskShareAccessMask configuration applies when using the SMB2 or SMB3 protocol for file operations through the file connector. This setting defines the access permissions (ACCESS_MASK) for the disk share as a comma separated string. Refer to the SMBJ documentation for the available access masks. Note Requires WSO2 Integrator: MI version 4.4.0.7 (U2) or later and file connector version 4.0.39 or later to use this parameter. MAXIMUM_ALLOWED No Sample configuration <file.listFiles configKey="CONNECTION_NAME"> <directoryPath>{$ctx:directoryPath}</directoryPath> <matchingPattern>{$ctx:matchingPattern}</matchingPattern> <recursive>{$ctx:recursive}</recursive> <sortingAttribute>{$ctx:sortingAttribute}</sortingAttribute> <sortingOrder>{$ctx:sortingOrder}</sortingOrder> <responseFormat>{$ctx:responseFormat}</responseFormat> <maxRetries>{$ctx:maxRetries}</maxRetries> <retryDelay>{$ctx:retryDelay}</retryDelay> </file.listFiles> Sample request Following is a sample REST/JSON request that can be handled by the append operation. { "directoryPath":"/home/vive/Desktop/file" } Response <listFilesResult> <success>true</success> <directory name="test"> <file>.DS_Store</file> <directory name="aa"/> <file>abc.txt</file> <directory name="hasitha"> <file>a1.txt</file> <file>a2.txt</file> </directory> <file>input.xml</file> <file>output.csv</file> </directory> </listFilesResult> exploreZipFileExplore the contents of a ZIP file in a specific location. Parameter Name Element Type Description Default Value Required File Connection name String The name of the file connection configuration to use. - Yes Zip File Path zipFilePath String The path to the ZIP file that should be explored. - Yes File Name Encoding fileNameEncoding String The character encoding to interpret the file names inside the ZIP archive. Available from File Connector v4.0.45 and above. UTF-8 No Sample configuration <file.exploreZipFile configKey="CONNECTION_NAME"> <zipFilePath>{$ctx:directoryPath}</zipFilePath> </file.exploreZipFile> Sample request Following is a sample REST/JSON request that can be handled by the append operation. { "zipFilePath":"/home/vive/Desktop/file/test.zip" } Response <exploreZipFileResult> <success>true</success> <zipFileContent> <file>test1.txt</file> <file>test2.txt</file> <file>docs/a1.txt</file> <file>docs/archive/c1.txt</file> </zipFileContent> </exploreZipFileResult> On Error <exploreZipFileResult> <success>false</success> <code>700102</code> <detail>Zip file not found at path file:///Users/hasitha/temp/file-connector-test/wso2/test/archieve.zip</detail> </exploreZipFileResult> mergeFilesMerge the contents of multiple files in a folder to a single file. The source files are merged in the order of their file names (case-sensitive string order). For example, to merge a CSV header file followed by data chunks, name them header.csv, part-001.csv, part-002.csv, and so on. Use zero-padded numbers, as part-10.csv is ordered before part-2.csv. To merge the files in a different order, use the sortingAttribute and sortingOrder parameters. Note Ordering source files by name is available in file connector version 4.0.46 or later. Earlier versions merge the files in the order returned by the file system, which is not guaranteed. The sortingAttribute and sortingOrder parameters are available in file connector version 4.0.47 or later. Parameter Name Element Type Description Default Value Required File Connection name String The name of the file connection configuration to use. - Yes Source Directory Path sourceDirectoryPath String The path to the source folder containing the files that should be merged. The files are merged in the order defined by the sortingAttribute and sortingOrder parameters. - Yes Target File Path targetFilePath String The path to the merged file. If the file does not exist, it is created. - Yes File Pattern filePattern String The regular expression that the source file names should match to be merged. Example: [a-zA-Z][a-zA-Z].(txt|xml|jar). - No Write Mode writeMode String If the target file already exists, this parameter determines whether the existing file should be overwritten or appended during the merge. Possible values are Overwrite or Append. Overwrite: The existing target file is replaced with the merged content. Append: The merged content is added after the existing content of the target file. Overwrite Yes Sorting Attribute sortingAttribute String The attribute used to order the source files before they are merged. Files with the same value are ordered by name. Possible values are Name, Size, or LastModifiedTime. Name: The case-sensitive string order of the file names. Use this when the file names define the order, such as a header file followed by numbered data chunks. Size: The size of the files. LastModifiedTime: The last modified time of the files. Name No Sorting Order sortingOrder String The order in which the source files are merged, based on the sortingAttribute. Possible values are Ascending or Descending. Ascending No Disk Share Access Mask diskShareAccessMask String The diskShareAccessMask configuration applies when using the **SMB2* or SMB3 protocol for file operations through the file connector. This setting defines the access permissions (ACCESS_MASK) for the disk share as a comma separated string. Refer to the SMBJ documentation for the available access masks. Note Requires WSO2 Integrator: MI version 4.4.0.7 (U2) or later and file connector version 4.0.39 or later to use this parameter. MAXIMUM_ALLOWED No Sample configuration <file.mergeFiles configKey="CONNECTION_NAME"> <sourceDirectoryPath>{$ctx:sourceDirectoryPath}</sourceDirectoryPath> <targetFilePath>{$ctx:targetFilePath}</targetFilePath> <filePattern>{$ctx:filePattern}</filePattern> <sortingAttribute>Name</sortingAttribute> <sortingOrder>Ascending</sortingOrder> <writeMode>{$ctx:writeMode}</writeMode> </file.mergeFiles> Sample request Following is a sample REST/JSON request that can be handled by the mergeFiles operation. { "sourceDirectoryPath":"/home/vive/Desktop/file", "targetFilePath":"/home/user/test/file/merged.csv", "filePattern":"(header|part-[0-9]{3})\\.csv", "writeMode":"Overwrite" } Response <mergeFilesResult> <success>true</success> <detail> <numberOfMergedFiles>5</numberOfMergedFiles> <totalWrittenBytes>992</totalWrittenBytes> </detail> </mergeFilesResult> On Error <mergeFilesResult> <success>false</success> <code>700102</code> <detail>Directory not found: file:///Users/hasitha/temp/file-connector-test/wso2/toMergesnsdfb</detail> </mergeFilesResult> writeWrites content to a specified file. Parameter Name Element Type Description Default Value Required File Connection name String The name of the file connection configuration to use. - Yes File Path filePath String The path to the file that should be written (include file name and extension). - Yes Content/Expression contentOrExpression String Static content or expression to evaluate content. The content will be fetched from the body ("$Body") of the incoming message. Yes MIME Type mimeType String The MIME type that will be applied in order to format the outgoing message. Possible values: "Automatic","text/plain", "application/xml", "application/binary", "application/json", "text/xml". If you don't want to change the MIME type of the message that has been mediated before this operation, use the default "Automatic" value. If the value is set to "application/binary", a binary file will get created with base-64 decoded content. Automatic Yes Write Mode writeMode String If the file already exists, this parameter will determine whether the existing file should be overwritten or appended. You can also specify if a new file should be created. Possible values: Ovewrite, Append, Create New. Overwrite Yes Append New Line appendNewLine Boolean Specifies whether a new line should be added to the end of the file after the content is written. false Yes Encoding encoding String Applied only when some static content or evaluated content is written. Possible Values: US-ASCII, UTF-8, or UTF-16. UTF-8 No Compress compress Boolean Specifies whether the content should be compressed after the content is written. Only available when the Write Mode is ‘Create New ‘or ‘OverWrite’. false No Enable Streaming enableStreaming Boolean Write file using the stream set to the message context. false No Enable Locking enableLock Boolean Specifies whether or not to lock the file during file write. Note: If the connector is processing a file named 'xyz.xml', a file called 'xyz.xml.lock' is created to represent the lock (with the CREATE_NEW mode). Once the file connector operation is completed, the file is deleted. When you create the lock, you can set an expiry time as well. If the connector operation fails to create the file because it already exists, that means that another process is working on it. Then connector operation will fail and the application will have to retry. Information such as the servername and PID is written to the lock file, which may be important for debugging. false No Add Result To includeResultTo String Specify where to add the result after writing the file. Message Body: The result will be added to the message property. Message Property: The payload that was in the message body before applying the file write operation will remain intact. Message Body No Property Name resultPropertyName String If the Add Result To attribute is set to "Message Property", specify a property name. The result of the file write operation will be added as a default scope property by the specified name. This property can be accessed later in the message flow. - Yes (If Add Result To is "Message Property") Update Last Modified Timestamp. updateLastModified Boolean Specify whether to update the last modified timestamp of the file. This is available from version 4.0.4. true No Max Retries maxRetries Integer The maximum number of retries to be done in case of a failure. 0 No Retry Interval retryDelay Integer The time interval between retries in milliseconds. 0 No Disk Share Access Mask diskShareAccessMask String The diskShareAccessMask configuration applies when using the SMB2 or SMB3 protocol for file operations through the file connector. This setting defines the access permissions (ACCESS_MASK) for the disk share as a comma separated string. Refer to the SMBJ documentation for the available access masks. Note Requires WSO2 Integrator: MI version 4.4.0.7 (U2) or later and file connector version 4.0.39 or later to use this parameter. MAXIMUM_ALLOWED No Sample configuration <file.write configKey="CONNECTION_NAME"> <filePath>{$ctx:filePath}</filePath> <contentOrExpression>{$ctx:contentOrExpression}</contentOrExpression> <mimeType>{$ctx:mimeType}</mimeType> <writeMode>{$ctx:writeMode}</writeMode> <appendNewLine>{$ctx:appendNewLine}</appendNewLine> <encoding>{$ctx:encoding}</encoding> <compress>{$ctx:compress}</compress> <enableStreaming>{$ctx:enableStreaming}</enableStreaming> <enableLock>{$ctx:enableLock}</enableLock> <includeResultTo>{$ctx:includeResultTo}</includeResultTo> <resultPropertyName>{$ctx:resultPropertyName}</resultPropertyName> <updateLastModified>{$ctx:updateLastModified}</updateLastModified> <maxRetries>{$ctx:maxRetries}</maxRetries> <retryDelay>{$ctx:retryDelay}</retryDelay> </file.write> Sample request Following is a sample REST/JSON request that can be handled by the append operation. { "filePath":"/home/vive/Desktop/file.test.txt" } Response <writeResult> <success>true</success> <writtenBytes>16</writtenBytes> </writeResult> Error <writeResult> <success>false</success> <code>700108</code> <detail>Target file already exists. Path = file:///Users/hasitha/temp/file-connector-test/copy/kandy/hasitha.txt</detail> </writeResult> fetchDirectoryContentRead the content of files in a given folder to a base64 encoded files stream list. Available in file-connector v4.0.22 and above. Parameter Name Element Type Description Default Value Required File Connection name String The name of the file connection configuration to use. - Yes Directory Path directoryPath String The path to the directory to list files. - Yes Matching Pattern matchingPattern String Pattern to match when listing files. All Files No Recursive recursive Boolean Whether to list files in sub-directories. false No File Sort Attribute sortingAttribute String Sort files when listing. Name Yes Sort Order sortingOrder String File sorting order. Ascending No Property Name resultPropertyName String Name of property to add the list of base64 encoded files operation result. - Yes Sample configuration <file.fetchDirectoryContent configKey="CONNECTION_NAME"> <directoryPath>{$ctx:directoryPath}</directoryPath> <matchingPattern>{$ctx:matchingPattern}</matchingPattern> <recursive>{$ctx:recursive}</recursive> <sortingAttribute>{$ctx:sortingAttribute}</sortingAttribute> <sortingOrder>{$ctx:sortingOrder}</sortingOrder> <resultPropertyName>{$ctx:resultPropertyName}</resultPropertyName> </file.fetchDirectoryContent> Sample request Following is a sample REST/JSON request that can be handled by the append operation. { "directoryPath":"/home/vive/Desktop", "sortingAttribute":"Name", "resultPropertyName":"fetchDirectoryContent" }