Uploading large documents to FileNet Content Engine

Question

Using FileNet Java API version 5.1.0.3

I am trying to upload big document to P8. I am reading file using FileInputStream and setting the input stream to contentTransfer.setCaptureSource(file); I am getting OOM exception (complete stacktrace below). Looks like it trying to read whole document in heap memory but that not what I want, I want to stream the document to P8.

Code

//Get CE Connetcion
//Create Subject
//Push Subject

//Get Domain (domain)
ObjectStore os = null;
objectStoreName = "COS" 
os = Factory.ObjectStore.fetchInstance(domain, objectStoreName, null); 


 //Get Folder
 Folder folder=null;
 folderName = ''/Sample";
folder=Factory.Folder.fetchInstance(os, folderName, null); 


 //Get the File details
InputStream file = ""; 
String fileName = "";
int fileSize = "";

// Create Document

String docClass = "dcumnet class name";
Document doc = Factory.Document.createInstance(os, docClass); 
if (file != null && fileSize > 0) {
                        ContentTransfer contentTransfer = Factory.ContentTransfer.createInstance();
                        contentTransfer.setCaptureSource(file);
                        contentElementList.add(contentTransfer);
                        doc.set_ContentElements(contentElementList);
                        contentTransfer.set_RetrievalName(fileName);                        
                        doc.set_MimeType(getMimetype(fileName));
                    }


//Check-in the doc
doc.checkin(AutoClassify.DO_NOT_AUTO_CLASSIFY,CheckinType.MAJOR_VERSION);                   
//Get and put the doc properties
String documentName =""
Properties p = doc.getProperties();
p.putValue("DocumentTitle","abcd");
p.putValue("Name","Rakesh");
p.putValue("Number","01234"); 


doc.save(RefreshMode.REFRESH)

Can you please help, how to send bigger document as stream to P8?

Caused by: java.lang.OutOfMemoryError: Java heap space
    at weblogic.utils.io.UnsyncByteArrayOutputStream.resizeBuffer(UnsyncByteArrayOutputStream.java:59) ~[com.bea.core.utils_1.10.0.0.jar:1.10.0.0]
    at weblogic.utils.io.UnsyncByteArrayOutputStream.write(UnsyncByteArrayOutputStream.java:89) ~[com.bea.core.utils_1.10.0.0.jar:1.10.0.0]
    at com.filenet.apiimpl.wsi.MtomOutputStream.write(MtomOutputStream.java:39) ~[Jace.jar:dap501.003.019]
    at com.filenet.apiimpl.wsi.AttachmentHelperNst.spillAndClearAttachments(AttachmentHelperNst.java:137) ~[Jace.jar:dap501.003.019]
    at com.filenet.apiimpl.wsi.ServiceSessionNst.bCloseSoapEnvelopeAndWriteAttachments(ServiceSessionNst.java:262) ~[Jace.jar:dap501.003.019]
    at com.filenet.apiimpl.wsi.ServiceSessionNst.cReqRespPath(ServiceSessionNst.java:159) ~[Jace.jar:dap501.003.019]
    at com.filenet.apiimpl.wsi.ServiceSessionNst.executeChanges(ServiceSessionNst.java:71) ~[Jace.jar:dap501.003.019]
    at com.filenet.apiimpl.util.SessionHandle.executeChanges(SessionHandle.java:130) ~[Jace.jar:dap501.003.019]
    at com.filenet.apiimpl.core.Session.callExecuteChanges(Session.java:142) ~[Jace.jar:dap501.003.019]
    at com.filenet.apiimpl.core.Session.executeChanges(Session.java:525) ~[Jace.jar:dap501.003.019]
    at com.filenet.apiimpl.core.Session.executeChange(Session.java:816) ~[Jace.jar:dap501.003.019]
    at com.filenet.apiimpl.core.IndependentlyPersistableObjectImpl.save(IndependentlyPersistableObjectImpl.java:83) ~[Jace.jar:dap501.003.019]

Are you wrapping your input stream in a buffered input stream? — Christopher Powell, Nov 29 '17 at 03:11
@ChristopherPowell The error originates in the output stream designated to write data to the server. The specifics of how file is being read is not related to this. Buffering (which quite likely is already used internally, by the way) will only increase memory consumption. — ﾤﾡ, Nov 29 '17 at 13:08
@fnt, yep you are correct. I missed that part. So this isn't so much a problem with his code, rather a problem with either the buggy version (as you suggest) or the WebLogic setup. Might help to define what size of file we are talking about here. — Christopher Powell, Nov 29 '17 at 19:00
The program runs fine independently in eclipse but above issue happens when I run it within weblogic (to be specific weblogic version 10.3.6) After debugging and scratching my head I figured that weblogic is using its default Weblogic HTTP handler which was actually reading whole file in heap instead of streaming. Switching to Sun HTTP handler via start up JVM/Java Option `-DUseSunHttpHandler=true` solves the OOM issue. This makes me solve the problem but setting up JVM option is not so good option when there are multiple application deployed on same JVM/server instance. — Rakesh Prajapati, Dec 01 '17 at 13:24
@fnt, As you suggested, I moved my answer to comment. I agree this may be a weblogic bug. I will see if there is any fix provided by weblogic for this.. — Rakesh Prajapati, Dec 01 '17 at 13:26

score 2 · Answer 1 · answered Nov 29 '17 at 00:33

2

WSI transport that you are using is capable of transferring content of arbitrary size. It utilizes MTOM attachments which are streamed over HTTP. It is not clear why there is an attempt to allocate a huge buffer in your case. Streaming requires some buffering as well, but there is no need for large buffers.

Assuming you are using java.io.FileInputStream and nothing fancy, it looks like a bug in the Content Engine client library. You should note that the version 5.1.0.3 is more than 4 years old and there were several fix packs since then (most recent one is 5.1.0.7).

answered Nov 29 '17 at 00:33

ﾤﾡ

5,669
6
43
57

Though the problem doesn’t look like related to API version but I take your suggestion, it is worth upgrading to latest API.. – Rakesh Prajapati Dec 01 '17 at 13:33
@RakeshPrajapati I suspected CE client in the first place since there were similar issues with it in the past – although not in this particular version. Once you pin down the cause and find a solution, it would be great to have it as answer. Actually, setting `UseSunHttpHandler` is a sort of solution. I just meant not to have further questions posted in the answer. – ﾤﾡ Dec 01 '17 at 17:36
1

The program runs fine independently in eclipse but above issue happens when I run it within weblogic (version 10.3.6) After debugging I figured that by default weblogic is use Weblogic HTTP handler which was actually reading whole file in heap instead of streaming. Switching to Sun HTTP handler via start up JVM/Java Option `-DUseSunHttpHandler=true` solves the OOM issue. This solves the problem but setting up JVM option is not so good option when there are multiple application deployed on same JVM/server instance. – Rakesh Prajapati Apr 04 '18 at 15:42
@RakeshPrajapati As I have already noted, this info deserves to be posted as an answer IMO. – ﾤﾡ Apr 04 '18 at 16:42

Uploading large documents to FileNet Content Engine

1 Answers1