0

I have a cataloging application in Android for engineering industry. Over 300 thousand design images have been uploaded into the Android device. Over 200 new images are being synchronized daily to the device.

There is a table called CatalogDetail wherein the details of the designs are stored. It has around 20 columns for storing different attributes and two image paths, one for thumbnail and other for large image. The average size of thumbnail image is around 20KB and that of large image is around 40KB. And all these images are stored in the same folder (not in the database). The overall size of 300 thousand images is around 6GB. As per various search conditions, the thumbnail images and short descriptions are shown in the grid in the application.

If I’ve 5000 images in the folder, the application works properly and as the size of the data increases, the application became slower and slower. After loading 300 thousand images now, when I load 250 images (end user want at least 250 images to be loaded at a stretch) into the grid, loading to grid is comparatively slower, but when I try to scroll, it is taking quite a long time (around 10 – 15 seconds), which is not desirable.

I’ve profiled my SQLs and have found out that SQLs are NOT taking considerable time (less than 1 second to get data). Finally I’ve figured out that, the problem is with loading images from the folder. The images are stored in the internal memory. The application is working on Samsung Galaxy Tab 750 10.1 inch with Android 3.1 (Honeycomb) OS, RAM 1GB and 16GB internal memory. The application is developed using SDK 10. Also one of the basic requirements from the end user is that the design images are very proprietary and they should be copied from the device. So the image folder is hidden and is not accessible outside the application.

Kindly let me know how the performance can be improved.
1. If images are index can the performance be improved? Is it possible to index (Is there any software, app, etc, available?
2. If I save 1000 images in each of the folders and if I create 300 folders and access from the images from them, can the performance be improved? But this should be the last option, since I’ve to re-load all images and modify the image location in my table.
3. Can threading help? (Sincerely I don’t know threading!)

Kindly suggest me how I can improve the performance.
Thanks for your help.

The search functionality will execute this method: 

private void ShowData() {
    try {
        Cursor countc = dal.getCursor("Select count(*) From Product " + whereCondition);
        SQL = "Select ID as _id, Name, Code, ProductWeight, '" + fileLoc + "' || ImageLoc as ImageLoc, ReleaseDate From CatalogDetail " + whereCondition;
        Cursor c = dal.getCursor(SQL + " Order By " + orderByField + " " + OrderSeq);

        ProductGridView.setAdapter(new CatalogueListAdapter(this, c));
    } catch (Exception e) {
        e.printStackTrace();
    }
}

This is how the catalogs are loaded to the grid:

public class CatalogueListAdapter extends SimpleCursorAdapter {
    private Context context;
    private Cursor c;

    public CatalogueListAdapter(Context context, Cursor c) {
        super(context, R.layout.thumbnail, c, new String[] {}, new int[] {});
        this.context = context;
        this.c = c;
    }
    @Override
    public View getView(int position, View inView, ViewGroup parent) {
        View v = inView;
        if (v == null) {
            LayoutInflater inflater = (LayoutInflater) context.getSystemService(Context.LAYOUT_INFLATER_SERVICE);
            v = inflater.inflate(R.layout.thumbnail, null);
        }
        c.moveToPosition(position);

        ImageView ivThumb = (ImageView) v.findViewById(R.id.thumbnail);
        String imageLoc = this.c.getString(this.c.getColumnIndex("ImageLoc"));
        if (imageLoc != null) 
            ivThumb.setImageURI(Uri.parse(imageLoc));
        return v;
    }
}
Nagesh
  • 1,288
  • 3
  • 22
  • 46

1 Answers1

0

One thing that comes to my mind spontaneously would be to simply distribute the pictures into subfolders based on their names, or, if that is not possible, giving them names suitable for that.

Usually, an open() call requires to scan through a special file called a directory. If there are many entries, this linear seach can become very slow.

If you will, my suggestion is a bit like "indexing", although I like the "hybrid" approach here. Just because Bubble Sort is always fastest for only a few items because of less overhead, similar rules apply for directories, too.

So if the images can be distributed by their file names or hash values of their file names, then use a subdirectory structure to place them. An evenly distribution must, of course, be the primary goal here, so choose the names or hash function wisely.

To suimmarize:

  1. Yes and no; see text above. ;)
  2. Yes, and I'd prefer it, because an additional index alone won't help if my assumption is correct that the problem is really with open().
  3. Again, if my assumption is right, then threading won't help.
class stacker
  • 5,357
  • 2
  • 32
  • 65
  • Thanks @Class Stacker for your suggestions. I'm finding it to test in the emulator :) – Nagesh Feb 01 '13 at 11:45
  • @Nagesh Maybe you can first find out whether it's really `open()` that kills your performance, but I'm almost willing to bet that it causes your slowdown. – class stacker Feb 01 '13 at 11:57
  • I'm not using the open(), I've edited and added code to my post. Please check it. – Nagesh Feb 01 '13 at 12:23
  • 1
    Well how do you expect `ImageView.setImageURI()` to access data from the file system? At least on operating system level, it will need to open the file. This means, that a special file has to be openend, which keeps the directory entries. And this file is organized in a linear fashion, so in the worst case (if the file is the last one in the list), the system has to scan all entries in that file and do a string compare. -> This takes especially long if all file names start with the same prefix. Conclusion: Reduce number of directory entries via a directory hierarchy of suitable depth. – class stacker Feb 01 '13 at 12:36
  • Thanks for your inputs. Kindly give the code snippet of using open() method and setting the image to ImageView. Your help is appreciated. – Nagesh Feb 01 '13 at 12:50
  • That's a misunderstanding. In my answer, I made a suggestion which only makes sense if `open()` is the source of your performance problem. Then, you were saying that you don't use open. So I clarified that `setImageURI()` uses `open()` internally to make the point that I would still try to find out whether `open()` is the source of the problem. -- You can simply write a small test app which measures the time to `open()` all files in a folder and see whether the time for a single `open()` increases while you fill the folder with more and more files. – class stacker Feb 01 '13 at 12:56
  • In Android documentation it is documented like this: setImageURI does Bitmap reading and decoding on the UI thread, which can cause a latency hiccup. If that's a concern, consider using setImageDrawable(android.graphics.drawable.Drawable) or setImageBitmap(android.graphics.Bitmap) and BitmapFactory instead. Do you think it is better to find an alternative? – Nagesh Feb 01 '13 at 13:08
  • I would _never_ do this on the UI thread. Sorry, didn't think of this aspect. But _all_ these methods will imply using `open()`, so what do you really want to avoid? A lagging UI? Then you can just open the files concurrently. I thought you wanted to speed up the processing also. Then you'll need the directory structure. – class stacker Feb 01 '13 at 13:19
  • Definitely I want to avoid lagging UI and speed up processing. As you suggest, it is imminent now to use directory structure since speed is a great concern. There are over 100 tablets where 300 thousand images are loaded already. With new directory structure again I've to reload all the images & save the image URL with directory path in the database table. I appreciate if you can provide a sample snippet or suggest a link. – Nagesh Feb 01 '13 at 14:03
  • I see. I just haven't understood yet for what exactly you want a snippet or link. Could you describe again? Thanks. – class stacker Feb 01 '13 at 14:10
  • Just needed code snippet for opening files concurrently and processing them (loading to grid). – Nagesh Feb 01 '13 at 14:15
  • Heard of Google? ;) http://www.google.com/search?q=android+load+drawable+from+file – class stacker Feb 01 '13 at 14:31
  • By the way, if I were you, I wouldn't load 300.000 images on 100 devices _again_. I'd just let my app analyze the starage structure and rearrange the images automatically on startup if it's not yet optimized. – class stacker Feb 01 '13 at 15:20
  • @Nagesh If you found my comments and/or answer helpful, I'd appreciate it if you marked them accordingly. I like to support people, but I also like to be rewarded for it. ;) – class stacker Feb 05 '13 at 07:55
  • @Class Stacker & Nagesh I am also facing the same problem. Nagesh, finally what approach you provide to resolve the problem. I have more than 1000 images in a folder. I do LazyLoading using ImageLoader class. Can you help me ? – Hardik Trivedi May 23 '13 at 11:58
  • @HardikTrivedi As I said, opening a file involves a linear search for a text in a special file on operating system level, so arranging for a folder hierarchy based on a file name hash (unless renaming is a frequent option, maybe) will solve the issue. The optimal width/depth ratio is operating system dependent so make same tests and share the results! ;) – class stacker May 26 '13 at 09:12