File Image to Servlet Fails

Question

I am trying to upload an image to a servlet, but every once and a while during automated testing, it silently fails.

Do you guys know what would cause this?

Here is the code on the server:

    @ResponseBody
    @RequestMapping(method = RequestMethod.POST)
    public String upload(HttpServletRequest request) throws Exception {
     BufferedImage image = null;

     @SuppressWarnings("unchecked")
     List<FileItem> items = new ServletFileUpload(
           new DiskFileItemFactory()).parseRequest(request);

    Logger.log(LogLevel.INFO, "Upload contains " + items.size()
            + " items.");
    int i = 0;
    for (FileItem item : items) {
        Logger.log(LogLevel.INFO, "\tItem " + (i++) + ". Name:\t"
                + item.getName() + ", Type:\t" + item.getContentType());

        // File is of type "file"
        if (!item.isFormField()) {
            InputStream inputStream = null;
            try {
                inputStream = item.getInputStream();
                if (inputStream.available() == 0) {
                    Logger.log(LogLevel.WARN,
                            "Item shows file type, but no bytes are available");
                }
                image = ImageIO.read(inputStream);
                if (image != null) {
                    break;
                }
            } catch (Exception e) {
                Logger.log(LogLevel.ERROR,
                        "There was an error reading the image. "
                                + ExceptionUtils.getFullStackTrace(e));
                throw new Exception("image provided is not a valid image");
            } finally {
                if (inputStream != null) {
                    IOUtils.closeQuietly(inputStream);
                }
            }
        }
    }

     if (image == null) {
        Logger.log(LogLevel.ERROR, "Image was supposedly read correctly, but was null afterwards");
        throw new Exception("Image provided could not be read");
     }

     //do stuff with image
     ...
    }

Here is the test:

 public void testImageUpload throws Exception {
    HttpPost httppost = new HttpPost("path/to/endpoint");
    File file=new File(imgLoc);
    FileBody bin = new FileBody(file);
    StringBody comment = new StringBody("Filename: " + file);

    MultipartEntity reqEntity = new MultipartEntity();
    reqEntity.addPart("upload-file", bin);
    reqEntity.addPart("comment", comment);
    httppost.setHeader("Accept", "application/json");
    httppost.setHeader("Connection","Keep-Alive");
    httppost.setEntity(reqEntity);
    HttpResponse response =testClient.getClient().execute(httppost);
    imgResponse=response.getStatusLine().toString();
    System.out.println(imgResponse);
    BufferedReader reader = new BufferedReader(
           new InputStreamReader(response.getEntity().getContent()));
    String line;
    while ((line = reader.readLine()) != null){
       output = output + " " +line;}
    System.out.println("Image Response: "+output);
}

Here is the output from the server when it fails:

2013/10/02 05-53-32,287::LOG:INFO[com.example#upload:L130 -- Upload contains 2 items.]
2013/10/02 05-53-32,288::LOG:INFO[com.example#upload:L133 --        Item 0. Name:   Dog.jpg, Type:  application/octet-stream]
2013/10/02 05-53-32,288::LOG:WARN[com.example#upload:L140 -- Item shows file type, but no bytes are available]
2013/10/02 05-53-32,289::LOG:INFO[com.example#upload:L133 --        Item 1. Name:   null, Type:     text/plain; charset=ISO-8859-1]
2013/10/02 05-53-32,290::LOG:ERROR[com.example#upload:L159 -- Image was supposedly read correctly, but was null afterwards]

We catch the exception from the image upload and send back a response code of 422 back to the client, so on the test, we get imgResponse==422 which is a failure case.

Note: this only happens sometimes you run the test.

Could you provide compilable and runnable code and the example upload you are using? — xwoker, Oct 15 '13 at 08:39
@Jason, can you explain the `if (image != null) { break; }` part ? — Andrea Ligios, Oct 16 '13 at 15:06
I'm not sure which file element is the image, so we iterate through the files until we can parse one. We are only expecting one image per upload. — Jason, Oct 16 '13 at 15:34
@AndreaLigios Yes, Good old-school SSCCE. It's good to know it's still around. Looking at most "what's wrong with my code" questions you would think that it was lost in space somehow. perhaps it should be linked from the "how to ask questions" SO page... — xwoker, Oct 16 '13 at 17:13
When you run the test and it succeeds, what does the output look like? — jharig23, Oct 17 '13 at 15:00

Prakash Bhagat · Answer 1 · 2013-10-16T06:17:46.043

Here is step by step configuration for file uploading by using Apache Commons FileUpload:

1. Add dependency jars for the following component. Here is the maven dependencies:

pom.xml

 <dependencies>
     <!-- Spring 3 MVC  -->
     <dependency>
         <groupId>org.springframework</groupId>
         <artifactId>spring-webmvc</artifactId>
         <version>3.2.4.RELEASE</version>
     </dependency>
     <!-- Apache Commons file upload  -->
     <dependency>
         <groupId>commons-fileupload</groupId>
         <artifactId>commons-fileupload</artifactId>
         <version>1.2.2</version>
     </dependency>
     <!-- Apache Commons IO -->
     <dependency>
         <groupId>org.apache.commons</groupId>
         <artifactId>commons-io</artifactId>
         <version>1.3.2</version>
     </dependency>
     <!-- JSTL for c: tag -->
     <dependency>
         <groupId>jstl</groupId>
         <artifactId>jstl</artifactId>
         <version>1.2</version>
     </dependency>
 </dependencies>

If you are not using maven then download respective jar from the maven repository online.

2. Create a FileUploadForm model

FileUploadForm.java

import java.util.List;
import org.springframework.web.multipart.MultipartFile;

public class FileUploadForm {

    private List<MultipartFile> files;

    //Getter and setter methods
}

3. Add resolver to MVC config file

<bean id="multipartResolver"
    class="org.springframework.web.multipart.commons.CommonsMultipartResolver">

    <!-- one of the properties available; the maximum file size in bytes -->
    <property name="maxUploadSize" value="100000"/>
</bean>

4. Write FileUploadController

FileUploadController.java

@Controller
public class FileUploadController {
     
    @RequestMapping(value = "/show", method = RequestMethod.GET)
    public String displayForm() {
        return "file_upload_form";
    }
     
    @RequestMapping(value = "/save", method = RequestMethod.POST)
    public String save(
            @ModelAttribute("uploadForm") FileUploadForm uploadForm,
                    Model map) {
         
        List<MultipartFile> files = uploadForm.getFiles();
 
        List<String> fileNames = new ArrayList<String>();
         
        if(null != files && files.size() > 0) {
            for (MultipartFile multipartFile : files) {
 
                String fileName = multipartFile.getOriginalFilename();
                fileNames.add(fileName);
                //Handle file content - multipartFile.getInputStream()
 
            }
        }
         
        map.addAttribute("files", fileNames);
        return "file_upload_success";
    }
}

5. Write jsp views

file_upload_form.jsp

<html>
<head>
    <title>Spring MVC Multiple File Upload</title>
<script
src="http://ajax.googleapis.com/ajax/libs/jquery/1.8.2/jquery.min.js"></script>
<script>
$(document).ready(function() {
    //add more file components if Add is clicked
    $('#addFile').click(function() {
        var fileIndex = $('#fileTable tr').children().length - 1;
        $('#fileTable').append(
                '<tr><td>'+
                '   <input type="file" name="files['+ fileIndex +']" />'+
                '</td></tr>');
    });

});
</script>
</head>
<body>
<h1>Spring Multiple File Upload example</h1>

<form method="post" action="save.html"
        **enctype="multipart/form-data"**>

    <p>Select files to upload. Press Add button to add more file inputs.</p>

    <input id="addFile" type="button" value="Add File" />
    <table id="fileTable">
        <tr>
            <td><input name="files[0]" type="file" /></td>
        </tr>
        <tr>
            <td><input name="files[1]" type="file" /></td>
        </tr>
    </table>
    <br/><input type="submit" value="Upload" />
</form>
</body>
</html>

Reference: http://docs.spring.io/spring/docs/3.2.4.RELEASE/spring-framework-reference/html/mvc.html#mvc-multipart

I appreciate the long response, but file upload works fine through the ui. it's just that sometimes, during automated testing, that the image upload fails. — Jason, Oct 16 '13 at 14:47

score 0 · Answer 2 · answered Oct 02 '13 at 15:14

0

It seems your content type is application/octet-stream. Please add the below Header in your request and give a try

("Content-Type", "multipart/form-data");

answered Oct 02 '13 at 15:14

Prabhakaran Ramaswamy

25,706
10
57
64

Will try that and let you know. It works sometimes, so that is strange if it's the header. – Jason Oct 02 '13 at 15:33

score 0 · Answer 3 · answered Oct 15 '13 at 12:25

0

You are using InputStream#available. As the documentation states this is the number of bytes that can be read from the stream without blocking. Now, how many bytes are available from the TCP input stream depends on the size of the the packets and how your request is sliced amongst them (and a lot more other factors).

If your intention is to always read the stream in full, forget the available() method, just read it out until the end of stream and you should be fine.

answered Oct 15 '13 at 12:25

boky

815
5
10

I was using `available()` as a check, but I'm using `image = ImageIO.read(inputStream);` to read the image. – Jason Oct 15 '13 at 14:31
For this specific test's sake, I would try reading the input stream into a ByteArrayOutputStream and checking if it's really zero-length before passing the byte buffer to ImageIO. This should at least help you find where the problem is. – boky Oct 16 '13 at 14:04

score 0 · Answer 4 · answered Oct 16 '13 at 14:46

0

I've come across this before under two conditions. Once was when I ran low on disk space and the other was when I was doing a bit of load test.

If you take a look at the How it works page, you can make the tool dump items to disk or keep them in memory. Under one case I filled up the drive during testing and the other I was keeping items in memory but the load blew my memory limit.

How do you have it set up? How big is the image you are using to test? How many times do yo upload it during your tests? With this info, I should be able to help a bit more.

answered Oct 16 '13 at 14:46

Jeff Richley

1
2
8

Throughout the test suite, we upload a total of four images between 10kb-1Mb (none over 1 mb). On the server, we never write the image to disk, as the image is processed in memory. During the tests, the images are packaged with our test suite. – Jason Oct 16 '13 at 14:54
I think I have the issue then. The [javadoc](http://commons.apache.org/proper/commons-fileupload/apidocs/org/apache/commons/fileupload/disk/DiskFileItemFactory.html) states that by default anything over 10K is written to disk. If you actually want to always process items in memory, you can call setSizeThreshold(Integer.MAX_VALUE) on your DiskFileItemFactory. That really bit me in the backside a few times. Try that and let me know if it works better for you. If not, I can play around and see if I can get it reproduced on my box. – Jeff Richley Oct 16 '13 at 17:19

score 0 · Answer 5 · answered Oct 16 '13 at 21:51

This code is used on my site currently, works like a charm:

package com.example;

import java.awt.image.BufferedImage;
import java.io.IOException;

import javax.imageio.ImageIO;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.stereotype.Controller;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.multipart.MultipartFile;

@Controller
@RequestMapping("/api/media")
public class ImageRestService {

    private static final Logger LOG = LoggerFactory.getLogger(ImageRestService.class);

    @RequestMapping(value = "/uploadtemp", method = RequestMethod.POST)
    public String upload(@RequestParam(value = "image") MultipartFile image) {
        try {
            BufferedImage bufferedImage = ImageIO.read(image.getInputStream());
            // process image here
        } catch (IOException e) {
            LOG.error("failed to process image", e);
            return "failure/view/name";
        }
        return "success/view/name";
    }

}

score -1 · Answer 6 · answered Oct 17 '13 at 12:49

-1

Maybe the order of the items list is not fixed (timing dependent?). Your code

if (image != null) {
    break;
}

quits the loop instead of trying the next parts. In the comments you state we iterate through the files until we can parse one, which should read

if (image != null) {
    continue;
}

then.

answered Oct 17 '13 at 12:49

Waldheinz

10,399
3
31
61

Would the downvoter please explain what's wrong with this answer? – Waldheinz Oct 17 '13 at 14:06

File Image to Servlet Fails

6 Answers6