3

I'm developing a web application in which users are allowed to upload pictures, the system will then generate thumbs for them.

My problem relies on the fact that EJBs can be distributed on several servers and thus are not allowed to handle files directly. I could store the images in the databases but I was hoping to store them as files in one of the servers. How can I do this? Is there any way to centralize the storage of files? Or any approach to deal with files in Java EE with EJBs?

Currently, I'm storing my files in a database. So I have centralized access and I don't need a dedicated file server. I'm doing this because I don't know how to integrate ftp servers and EJBs. Is this however a good alternative?

What I want is: Using Stateless EJBs, store the uploaded images as files and the path to them in the database. So I can display them using 

<h:graphicImage ... />
arg20
  • 4,893
  • 1
  • 50
  • 74
  • what will be the capacity / number of uploads per year? And number of thumbnails generated per upload? – ThomasRS Mar 29 '11 at 10:00
  • @Thomas big numbers, what's the impact of that in my solution? – arg20 Mar 29 '11 at 13:50
  • Well first structure your files (directories) according to the limits in your file system - ours EXT3 went full. And then think about maintainance and performance, if you go with a soft launch, at least plan for SAN. Do the math, I guess, and no binary images in the database of course.. – ThomasRS Mar 29 '11 at 16:00

6 Answers6

6

You actually have four aspects here,

  1. Receiving and sending files
  2. Creating thumbnails
  3. Storing the files somewhere every node can access
  4. Storing the original + thumbnail entities in a common database

Since you already have Java EE server, you probably also already have a (HTTP) servlet server, in which there is numerious ways of doing load balancing and caching, not to mention the obious potential for web-based interaction. If anything, support FTP transfer with a directory watcher as a bonus.

You should not create the thumbnails using stateless session beans, this means your servers will be crap at peak time - the server will give priority to buisness logic over making new connections. Rather, first receieve and store the file + original entity in the database, and then use a service bean to queue up thumbnail creation (maybe with n worker threads or message queues if you want). You can also use native tools in some cases, we do in linux.

You should use a shared file system, SAN, which is the right tool for sharing files across several machines. And structure your files according to your file system's limits - like number of files per directory and read/write capacity.

And a single database will be good enough for at least a small cluster, as long as you are not killing it with big binary blobs.

If in doubt, buy more ram ;) Especially the thumbnails are very cachable and will give good performance also in Tomcat - if you are not familiar with multi-threading, find a cache at google. Also cache the entities naturally, not only the files.

ThomasRS
  • 8,215
  • 5
  • 33
  • 48
  • The problem is how do I actually store the files, since EJBs are forbidden to access files or persist them. I also don't understand why storing the image as a file and as database entity. – arg20 Mar 30 '11 at 04:08
  • JAVA EE 6 has a beautiful support for ORM with JPA specification, yet it seems like I need third party libraries for distributed storage of files. I'm looking into WebDAV like Shamit suggested, but I thought such an important feature should be included in an app server – arg20 Mar 30 '11 at 04:13
  • so why cannot your EJB access the file system? It is not forbidden as far as our JBoss goes. Store your file in the filesystem + store an entity REFERENCING the file (path, filename, etc) in the database :P – ThomasRS Mar 30 '11 at 11:43
  • EJBs specification forbids access to the filesystem, but I guess that heavily depends on the application server. – arg20 Mar 30 '11 at 15:01
  • 1
    yeah well, just make the file location configurable and you'll be good. if someone requires you to follow the spec 100%, have a look at for example http://sourceforge.net/projects/filera/, but I bet it will do just the same with different wrapping. – ThomasRS Mar 30 '11 at 17:30
1

You might want to use a (private) FTP server for this. Your EJB beans can contact this server for storing and retrieving files.

There are various libraries in Java for accessing FTP servers. Specifically well suited for use in an EJB environment would be JCA based FTP connectors, but 'normal' ones will usually work fine too.

Arjan Tijms
  • 37,782
  • 12
  • 108
  • 140
  • Can you suggest me any JCA based FTP connectors? I don't really understand much on connectors. Is there any tutorial on this? – arg20 Mar 13 '11 at 20:31
1

You can also look into using a clustered file system. RedHat Global File System and Symantec's Veritas Clustered File System are two I have experience with. These products allow you to mount the same file system across several servers for read/write access. All your application sees is just another directory. If your environment already has the requisite components (SAN and a good Sys Admin), this might be the best performing solution in a lot of use cases.

However, there are drawbacks to this approach:

  1. You shift complexity from your app to the OS. These products aren't trivial to set up.
  2. Scalability might become an issue if you have a large server farm. And when scaling problems arise finding the bottleneck is not as straight forward as arjan's ftp solution.
  3. You need a SAN.
che2cbs
  • 369
  • 1
  • 4
1

If you can make reasonable assumptions about "where" your EJB instance is, direct handling of a file is no problem. In your case (since you want to have files) I would read the image into a local temp folder and upload it to a remote destination.

A possible way to do that is http://txconnect.sourceforge.net/ a JCA Transaction Adapter that handes (among others) ftp connections. Configure the factory in xml and inject the connection into your bean and you have ready to go.

Depending on your Application server there might be a special connector available (f.e.: Oracle or IBM systems)

Laures
  • 5,389
  • 11
  • 50
  • 76
1

I'd suggest you to stick to your current solution. Ftp access (if needed for purposes other than just keeping files together) can be build on top of your ejb layer. Displaying images stored in DB is not a problem, simple servlet will do the trick.

andbi
  • 4,426
  • 5
  • 45
  • 70
1

You can :

  1. Create a WebDAV based file share. This can be done by using many libraries available for Java or other languages. One such library is : http://milton.ettrema.com/index.html
  2. All EJB instances can read /write images from this file share. They would need to use WebDav client libraries
  3. DO setup backups of directories behind this file share
Shamit Verma
  • 3,839
  • 23
  • 22