Appending JSON content to a JSON file in WSO2

Question

I have few JSON Payloads that have to be appended to a JSON file. This is how my JSON Payload will look:

{"variant": {"940894": {"attributes": {"Size": "XL" } } } }

I will be getting multiple json Payloads of this type. At the end I need to form a file that looks like this:

[
{"variant": {"940894": {"attributes": {"Size": "XL" } } } },
{"variant": {"940895": {"attributes": {"Size": "Med"    } } } },
{"variant": {"940895": {"attributes": {"Size": "Small"  } } } }
]

These are the properties I have set for the requirement:

<property expression="fn:concat('Test',get-property('File_Name'),'.txt')" name="FILE_NAME" scope="default"
<propertyGroup description="File and OutputPayload properties">
        <property name="messageType" scope="axis2" type="STRING" value="text/plain"/>
        <property name="ContentType" scope="axis2" type="STRING" value="text/plain"/>
        <property expression="$ctx:FILE_NAME" name="transport.vfs.ReplyFileName" scope="transport" type="STRING"/>
        <property name="OUT_ONLY" scope="default" type="STRING" value="true"/>
    </propertyGroup>       
            <call>
        <endpoint>
            <address uri="vfs:sftp://username:password/home/user/test/out/json?transport.vfs.Append=true">
                <suspendOnFailure>
                    <initialDuration>-1</initialDuration>
                    <progressionFactor>1</progressionFactor>
                </suspendOnFailure>
                <markForSuspension>
                    <retriesBeforeSuspension>0</retriesBeforeSuspension>
                </markForSuspension>
            </address>
        </endpoint>
    </call>

But I get an error saying "The file type does not support append mode". What else can I try here.

This means the file system doesn't support appending files. May be read the file append the content and replace the file? — ycr, Feb 08 '23 at 14:27
But it gives the same error for every kind of file. I've tried csv, xml, txt, json. — Ruby, Feb 09 '23 at 07:11

ycr · Accepted Answer · 2023-02-09T12:51:33.430

I had a look at the code, and as you already noticed SFTP protocol doesn't allow you to do file appending. The supported operations for SFTP are here in the code.

Capability.CREATE, Capability.DELETE, Capability.RENAME, Capability.GET_TYPE,
Capability.LIST_CHILDREN, Capability.READ_CONTENT, Capability.URI, Capability.WRITE_CONTENT,
Capability.GET_LAST_MODIFIED, Capability.SET_LAST_MODIFIED_FILE, Capability.RANDOM_ACCESS_READ

But if you use FTP protocol instead you should be able to do File Appending as it's supported here.

Capability.CREATE, Capability.DELETE, Capability.RENAME,
Capability.GET_TYPE, Capability.LIST_CHILDREN,
Capability.READ_CONTENT, Capability.GET_LAST_MODIFIED,
Capability.URI, Capability.WRITE_CONTENT,
Capability.APPEND_CONTENT, Capability.RANDOM_ACCESS_READ

Appending works with the Local file system too.

If you want to append the file and still use SFTP you will have to READ -> APPEND -> WRITE the file. Or can consider writing a Custom Class mediator.

Appending JSON content to a JSON file in WSO2

1 Answers1