We have a use case where we have to transfer Large data files from environment A to environment B over http. What we want to achieve is that the sender sends the data in chunks and receiver starts writing it in file in chunks. Thus we decided to use MTOM.
Web Service Code :
@MTOM(enabled = true, threshold=1024)
@WebService(portName = "fileUploadPort", endpointInterface = "com.cloud.receiver.FileUploadService", serviceName = "FileUploadService")
@BindingType(value = SOAPBinding.SOAP12HTTP_MTOM_BINDING)
public class FileUploadServiceImpl implements FileUploadService {
@Override
public void uploadFile(FileUploader Dfile) {
DataHandler handler = Dfile.getFile();
try {
InputStream is = handler.getInputStream();
OutputStream os = new FileOutputStream(new File(absolutePath));
byte[] b = new byte[10000000];
int bytesRead = 0;
while ((bytesRead = is.read(b)) != -1) {
os.write(b, 0, bytesRead);
}
os.flush();
os.close();
is.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
Client Code:
public static void main(String args[]) throws Exception {
URL url = new URL("http://localhost:8080/CloudReceiver/FileUploadService?wsdl");
QName qname = new QName("http://receiver.cloud.com/", "FileUploadService");
Service service = Service.create(url, qname);
FileUploadService port = service.getPort(FileUploadService.class);
// enable MTOM in client
BindingProvider bp = (BindingProvider) port;
SOAPBinding binding = (SOAPBinding) bp.getBinding();
binding.setMTOMEnabled(true);
FileUploader f = new FileUploader();
DataSource source = new FileDataSource(new File("G:\\Data\\Sender\\temp.csv"));
DataHandler dh = new DataHandler(source);
Map<String, Object> ctxt = ((BindingProvider) port).getRequestContext();
// Marking Chunk size at Client.
ctxt.put(JAXWSProperties.HTTP_CLIENT_STREAMING_CHUNK_SIZE, 100);
f.setFile(dh);
port.uploadFile(f);
}
Every things works fine when we transfer data less than 100MB. Data Files beyond (100MB+) App server (JBoss 8.2) throws below exception at the receivers end.
java.io.IOException: UT000020: Connection terminated as request was larger than 104857600
I under stand that this error is because of the property in standalone.xml
<http-listener name="default" socket-binding="http" max-post-size="104857600"/>
This means that Data is not written to file in Chunks instead it is kept in memory and than written to file after transmission completes.
How do we achieve writing of data to file in Chunks? We don't want the post memory to increase. The File size can go up to the scale of 1 TB.
Environment: JBoss 8.2 Wild Fly, Java 8
