Cannot Create Directory On Server. Permission Issue?

Question

I am using lucene to store an index of some data. The following code checks for the existance of a directory in /home/username and if its not found, builds the index from scratch creating the directories etc.

public static final String INDEX_PATH = "/home/username/appname/lucene/index";
private void buildCompleteIndex(int organizationId) {
        synchronized(mutex) {
        File path = new File(INDEX_PATH + "/" + String.valueOf(organizationId));
        if(!path.exists()) {            
            try {
                Utils.deleteDirectory(path);
            } catch (IOException e) {
                throw new LuceneIndexException("Error rebuilding index directory.", e);
            }
            path.mkdirs();
        }

        List<Contact> contactList = contactDAO.findAll(organizationId, true);
        if(contactList != null) {
            for(Contact contact : contactList) {
                add(contact);
            }
        }
    }
}

//Getters
private IndexReader getIndexReader(boolean readOnly, int organizationId) {
    try {
        if(directory == null) {
            File path = getFile(organizationId);
            directory = FSDirectory.open(path);
            if(!IndexReader.indexExists(directory)) {
                buildCompleteIndex(organizationId);
            }
        }
        return IndexReader.open(directory, readOnly);
    } catch (CorruptIndexException e) {
        buildCompleteIndex(organizationId);
    } catch (IOException e) {
        buildCompleteIndex(organizationId);
    }
    return null;
}

This all works great in development when i'm deploying from a virtual tomcat instance inside of eclipse, but fails on the production server.

Why would I be able to write to the directory in dev mode, but not when the app is deployed on the server? Do I not have permission to create directories? I'm using Ubuntu Server 12.10 and Tomcat7.

How can I get this creating the proper folders and files on the server?

Is there a specific folder I am supposed to allow my applications to write to on the server? It always worked with the home/user folder on my dev box, but maybe thats different on the server because the user isn't actually logged in when the application is running?

UPDATE: I checked the permissions of the folder which are currently set to 700. Could that be the problem? Is it safe to set this folder to 666 or 777 on a production server? Is this folder writable even if the username of /home/username is not logged in? I know that 700 means owner has full access, but does that include the tomcat application?

UPDATE: I attempted to change the /home/username's permissions to 755 and the same issue persisted.

The stacktrace showing the error generated when trying to create the folder.

java.io.IOException: Cannot create directory: /home/ryandlf/thinkbooked.com/lucene/contacts/1
    at org.apache.lucene.store.NativeFSLock.obtain(NativeFSLockFactory.java:171)
    at org.apache.lucene.store.Lock.obtain(Lock.java:72)
    at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:1108)
    at com.thinkbooked.search.LuceneContactSearchEngine.getIndexWriter(LuceneContactSearchEngine.java:321)
    at com.thinkbooked.search.LuceneContactSearchEngine.add(LuceneContactSearchEngine.java:68)
    at com.thinkbooked.search.LuceneContactSearchEngine.buildCompleteIndex(LuceneContactSearchEngine.java:285)
    at com.thinkbooked.search.LuceneContactSearchEngine.getIndexReader(LuceneContactSearchEngine.java:303)
    at com.thinkbooked.search.LuceneContactSearchEngine.find(LuceneContactSearchEngine.java:150)
    at com.thinkbooked.search.LuceneContactSearchEngine.find(LuceneContactSearchEngine.java:145)
    at com.thinkbooked.handlers.ClientListSearchHandler.init(ClientListSearchHandler.java:49)
    at com.thinkbooked.event.EventListener.doPost(EventListener.java:59)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:641)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:722)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:168)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
    at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:929)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
    at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1002)
    at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:585)
    at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:312)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:722)

You should first try adding a printStackTrace in the catch block and see what's causing the Exception. Then review the logic that leads to an infinite recursion. (Maybe show us the code for `add()` too) — Dek Dekku, May 26 '13 at 06:59
The stackoverflow is happening because the code tries to find the folder, and if it can't its supposed to create it and move on. In this situation the folder is never able to be created so it just keeps going back to trying to create it. I think this is more an issue with not being able to create a folder on the server for some reason. — ryandlf, May 26 '13 at 07:01
But this loop is almost certainly triggered by some Exception. I see a catch statement in `getIndexReader` that calls `buildCompleteIndex`, and the latter calls `add` and from the stack trace it's easy to infer that add closes the loop. To get at the root of the issue you should understand the root cause. — Dek Dekku, May 26 '13 at 07:19
I'm thinking it might be a permission issue. I know the stackoverflow is just a design flaw I need to fix but not causing the "unable to create folders" problem because this works just fine on my dev machine. Could it be a permission issue with the folder? — ryandlf, May 26 '13 at 07:25

score 1 · Answer 1 · answered May 26 '13 at 07:29

1

The loop is almost certainly triggered by some Exception. I see a catch statement in getIndexReader that calls buildCompleteIndex, and the latter calls add and from the stack trace it's easy to infer that add closes the loop. To get at the root of the issue you should understand the root cause.

Which I can't possibly know. First try would be: replace all the code in the catch blocks with printStackTrace() and maybe System.exit() to see if it yields more useful info about the first Exception that triggers everything else.

answered May 26 '13 at 07:29

Dek Dekku

1,441
11
28

I'll fix the stackoverflow within the catch blocks. But I still think the root of the problem has something to do with the fact that the folders aren't being created. As I mentioned, the code works fine in my dev environment. I assume the server is a little more strict and maybe i'm trying to write to a folder that I can't actually write to? – ryandlf May 26 '13 at 07:43
I'm asking you for the stack trace of the original exception because I'm confident that Lucene added an error message that could point out to what the problem is (but even without that, it would still be useful to know if you have a `CorruptIndexException` rather than an `IOException`, right?); without that there's just past experience with the same issue or wild guessing. – Dek Dekku May 26 '13 at 07:53
No worries. It seems a server issue indeed and I don't have the know-how to help you anymore, but perhaps you're one step closer to find the answer. Maybe try serverfault.com too. (If I had to guess, I'd say that even after giving 755 to the root dir, subdirs are created with restricted access, so the full path to `/ryandlf/thinkbooked.com/lucene/contacts/1` cannot be created. Something to setup in Tomcat maybe? But again, just a guess.) – Dek Dekku May 26 '13 at 08:24

score 0 · Accepted Answer · answered May 26 '13 at 15:30

According to an answer I received on the question posted in the original question, on Ubuntu's standard packaged version of tomcat, tomcat runs under a different username. To find this you can use this command from the command line:

ps aux | grep catalina

In my case it told me the username tomcat was using is tomcat7.

To edit the default tomcat username, edit the config file in:

/etc/default/tomcat7

The first line should list the default tomcat username and next, the group ID which in my case is the same.

To change ownership of a folder to tomcat so it can write to the directory, use the chmod command.

sudo chown -R username:group directory

The -R means all subfolders and files will also have new ownership. Omit this if you don't want that.

Cannot Create Directory On Server. Permission Issue?

2 Answers2

Linked