How can I chunk xml to multiple parts in java?

Question

I have a code that creates xml.

public void createXML(InputStream in, String fileName) throws IOException {
    baos = new ByteArrayOutputStream();
    byte[] buf = new byte[BUF_SIZE];
    int readNum = 0;

    Writer writer = new BufferedWriter(new OutputStreamWriter(FileUtil.getOutputStream(fileName, FileUtil.HDD)));
    writer.write("<?xml version=\"1.0\" encoding=\"UTF-8\"?>\r\n");
    writer.write("\t<" + imageFileName + ">\r\n");
    writer.write("\t\t");

    try {
        while ((readNum = in.read(buf)) >= 0) {
            baos.write(buf, 0, readNum);
            writer.write(baos.toString());
            baos.reset();
        }
    }
    finally {
        if (in != null) {
            in.close();
            baos.close();
        }
    }
    writer.write("\r\n\t<" + imageFileName + ">");
    writer.close();
    baos = null;
    buf = null;
}

I want to create this xml into multiple parts (maximum of 500kb each). How can I do this? Is there any way for me to determine that the created file is already 500kb and write the remaining data to a different file?

I used this but the image after decoding the base64 string, the image produced is corrupted on the portion where it was cut.

for(int i = 1; i <= numberOfFiles; i++){
            baos = new ByteArrayOutputStream();
            String filePartName = fileName + ".part" + i + ".xml";
            Writer writer = new BufferedWriter(new OutputStreamWriter(FileUtil.getOutputStream(filePartName, FileUtil.HDD)));
            writer.write("<?xml version=\"1.0\" encoding=\"UTF-8\"?>\r\n");
            writer.write("\t<" + filePartName + ">\r\n");
            writer.write("\t\t");

            int size = 0;
            while ((readNum = in.read(buf)) >= 0 && size < 512000) {
                baos.write(buf, 0, readNum);
                size = size + readNum;
                writer.write(baos.toString());
                baos.reset();
            }

            writer.write("\r\n\t<" + filePartName + ">");
            writer.close();
            baos.close();
        }
    }
    in.close();

Try to swap these two conditions: size < 512000 && (readNum = in.read(buf)) >= 0. In your code, I see the potential for data loss. — DmitryKanunnikoff, Mar 10 '14 at 09:20

score 1 · Accepted Answer · answered Mar 07 '14 at 08:31

1

If you keep a sum total of readNum that is in

while ((readNum = in.read(buf)) >= 0)

then you can test for its value and create new files when ever you want.

answered Mar 07 '14 at 08:31

Scary Wombat

44,617
6
35
64

thanks for this I was able to trim the xml to 500kb but how will I know the number of xmls that I have to make? Is there something that will tell me the size of the input stream so I can divide it to 500kb and get the number of files I should create? – pmark019 Mar 07 '14 at 09:57
I doubt that you would be able to know the length of a stream until it has been fully read. Unless the sending process sends this data first. – Scary Wombat Mar 10 '14 at 00:19
I just got the size of the file and used it instead of reading the whole stream. My new problem is that the image is corrupted in a portion where it was combined. I edited my post and added the code. – pmark019 Mar 10 '14 at 07:51
writer.write(baos.toString()); `toString` conversion will corrupt your byte stream. – Scary Wombat Mar 11 '14 at 04:38
it doesn't corrupt the data. The reason why the image was corrupted was because I read the next 1024 bytes but didn't write it so the stream was missing 1024 bytes thus making the image corrupted. – pmark019 Mar 11 '14 at 06:18
ah, thats right you are only dealing with text. I read the word `image` above and got confused - sorry. – Scary Wombat Mar 11 '14 at 06:20

score 1 · Answer 2 · answered Mar 07 '14 at 08:31

1

500kb = 512000 bytes. Use another ByteArrayOutputStream instanse to control this limit.

answered Mar 07 '14 at 08:31

DmitryKanunnikoff

2,226
2
22
35

How can I chunk xml to multiple parts in java?

2 Answers2