6

We have a number of lookup databases that we use in multiple applications and I'm trying to figure out the best and most efficient way to make these lookup databases available via a java function or bean in an OSGi Plugin library.

What I'd like to achieve is some way of creating a function that I can pass in a lookup key and a field name and the function will return the correct value and possibly object type ( to deal with datatime values ). It would also need to cache the value for about an hour at the application level as these lookup documents do not change at all.

Typically I'd want to use them for display purposes so that I only need to store the key in my notes document and then use something like the following to display on screen what I need

<xp:text escape="true" id="computedField1">
    <xp:this.value><![CDATA[#{javascript:com.mycompany.lookup.GetDoc("docID","fieldName")}]]></xp:this.value>
</xp:text>
Declan Lynch
  • 3,345
  • 17
  • 36
  • To be sure, does it need to be a osgi plugin? Or could it also be done with a userbean defined in the database? Where is the information about the lookup databases stored? – jjtbsomhorst Mar 06 '12 at 15:28
  • It could be a bean defined in an OSGi plugin. My thinking is that having it in the plugin will make it global to the server so any app that needs to do a lookup can just call the function to return a value. The lookup databases are in a fixed location on the server. – Declan Lynch Mar 06 '12 at 15:32

2 Answers2

8

You can pretty well do this now with scoped beans, with the one caveat that a bean is NSF specific. Although the XSP Starter kit I believe includes an example of how to do a Server scoped bean (which is really a singleton, meaning there is only one instance of the class for the entire JVM).

First create a simple serializable POJO called CachedData that has two member fields, the first is a field that holds a date time value that indicates when you last read the data from the disk, and the second is some sort of list object, like a vector, that holds your values.

Then create another POJO called ServerMap that has a map<string, map<string, map<string, map<Object, map<object, CachedData>>>> as a member, and a function called doCachedLookup() or something like that. The parameters to that function can be almost the same as an @DbLookup, server, database, view, key, etc. Then in the doCachedLookup, check your ServerMap for the existence of the specified server as the key. If it doesn't exist, then create a new map and insert it into the ServerMap with the key being the server name. If it does exist, then look up the database name in that map, then the view in the next map, then finally the value in the last map. Once you get the CachedData object, you can check the date time field and see if it is expired, if it isn't, return the vector, and if it is, discard it, and then do a fresh lookup, and re-cache the data, and then return the vector.

Here's the code samples, i was a bit lazy in my overloaded methods for getting a column versus getting a field name, and i use some deprecated java date methods, but it will give you a good base to start with. All code is tested:

CachedData Class:

package com.ZetaOne.example;

import java.io.Serializable;
import java.util.Date;
import java.util.Vector;

public class CachedData implements Serializable {

    private static final long serialVersionUID = 1L;
    private Date updateTime;
    private Vector<Object> values;

    public Date getUpdateTime() {
        return this.updateTime;
    }

    public void setUpdateTime(Date UpdateTime) {
        updateTime = UpdateTime;
    }

    public Vector<Object> getValues() {
        return this.values;
    }

    public void setValues(Vector<Object> values) {
        this.values = values;
    }
}

CachedLookup class that is implemented as a singleton so that it can be used server-wide:

package com.ZetaOne.example;

import java.io.Serializable;
import java.util.Date;
import java.util.Vector;
import com.ZetaOne.example.CachedData;
import java.util.HashMap;
import java.util.Collections;
import java.util.Map;

import lotus.domino.Session;
import lotus.domino.Database;
import lotus.domino.View;
import lotus.domino.NotesException;
import lotus.domino.ViewEntryCollection;
import lotus.domino.ViewEntry;
import lotus.domino.Document;

import javax.faces.context.FacesContext;

public class CachedLookup implements Serializable {

    private static CachedLookup _instance;

    private static final long serialVersionUID = 1L;
    private Map<String, HashMap<String, HashMap<String, HashMap<Object, HashMap<Object, CachedData>>>>> cachedLookup;

    public static CachedLookup getCurrentInstance() {
        if (_instance == null) {
            _instance = new CachedLookup();
        }
        return _instance;
    }       

    private CachedLookup() {
        HashMap<String, HashMap<String, HashMap<String, HashMap<Object, HashMap<Object, CachedData>>>>> cachedLookupMap =
            new HashMap<String, HashMap<String, HashMap<String, HashMap<Object, HashMap<Object, CachedData>>>>>();
        this.cachedLookup = Collections.synchronizedMap(cachedLookupMap);
    }

    @SuppressWarnings("deprecation")
    public Vector<Object> doCachedLookup(String serverName, String filePath, String viewName, Object keyValues, int columnNumber, boolean exactMatch) {

        if (cachedLookup.containsKey(serverName)) {
            if (cachedLookup.get(serverName).containsKey(filePath)) {
                if (cachedLookup.get(serverName).get(filePath).containsKey(viewName)) {
                    if (cachedLookup.get(serverName).get(filePath).get(viewName).containsKey(keyValues)) {
                        if (cachedLookup.get(serverName).get(filePath).get(viewName).get(keyValues).containsKey(columnNumber)) {
                            CachedData cache = cachedLookup.get(serverName).get(filePath).get(viewName).get(keyValues).get(columnNumber);
                            if (cache.getUpdateTime().compareTo(new Date()) > 0) {
                                System.out.println("Cache Hit");
                                return cache.getValues();
                            }
                        }
                    }
                }
            }
        }

        System.out.println("Cache Miss");
        // if we drop to here, cache is either expired or not present, do the lookup.

        try {
            Session session = (Session)resolveVariable("session");
            Database db = session.getDatabase(serverName, filePath);
            View view = db.getView(viewName);
            ViewEntryCollection vc = view.getAllEntriesByKey(keyValues, exactMatch);
            ViewEntry ve, vn;
            ve = vc.getFirstEntry();
            Vector<Object> results = new Vector<Object>();
            while (ve != null) {
                results.add(ve.getColumnValues().elementAt(columnNumber));

                vn = vc.getNextEntry();
                ve.recycle();
                ve = vn;
            }

            vc.recycle();

            if (!cachedLookup.containsKey(serverName)) {
                cachedLookup.put(serverName, new HashMap<String, HashMap<String, HashMap<Object, HashMap<Object, CachedData>>>>());
            }

            if (!cachedLookup.get(serverName).containsKey(filePath)) {
                cachedLookup.get(serverName).put(filePath, new HashMap<String, HashMap<Object, HashMap<Object, CachedData>>>());
            }

            if (!cachedLookup.get(serverName).get(filePath).containsKey(viewName)) {
                cachedLookup.get(serverName).get(filePath).put(viewName, new HashMap<Object, HashMap<Object, CachedData>>());
            }

            if (!cachedLookup.get(serverName).get(filePath).get(viewName).containsKey(keyValues)) {
                cachedLookup.get(serverName).get(filePath).get(viewName).put(keyValues, new HashMap<Object, CachedData>());
            }

            CachedData cache;
            if (cachedLookup.get(serverName).get(filePath).get(viewName).get(keyValues).containsKey(columnNumber)) {
                cache = cachedLookup.get(serverName).get(filePath).get(viewName).get(keyValues).get(columnNumber);  
            } else {
                cache = new CachedData();
            }

            Date dt = new Date();
            dt.setHours(dt.getHours() + 1);
            cache.setUpdateTime(dt);
            cache.setValues(results);           

            cachedLookup.get(serverName).get(filePath).get(viewName).get(keyValues).put(columnNumber, cache);

            view.recycle();
            db.recycle();

            return results;

        } catch (NotesException e) {
            // debug here, im lazy
            return null;
        }
    }

    public Vector<Object> doCachedLookup(String serverName, String filePath, String viewName, Object keyValues, String fieldName, boolean exactMatch) {

        if (cachedLookup.containsKey(serverName)) {
            if (cachedLookup.get(serverName).containsKey(filePath)) {
                if (cachedLookup.get(serverName).get(filePath).containsKey(viewName)) {
                    if (cachedLookup.get(serverName).get(filePath).get(viewName).containsKey(keyValues)) {
                        if (cachedLookup.get(serverName).get(filePath).get(viewName).get(keyValues).containsKey(fieldName)) {
                            CachedData cache = cachedLookup.get(serverName).get(filePath).get(viewName).get(keyValues).get(fieldName);
                            if (cache.getUpdateTime().compareTo(new Date()) > 0) {
                                System.out.println("Cache Hit");                                
                                return cache.getValues();
                            }
                        }
                    }
                }
            }
        }

        System.out.println("Cache Miss");           
        // if we drop to here, cache is either expired or not present, do the lookup.

        try {
            Session session = (Session)resolveVariable("session");
            Database db = session.getDatabase(serverName, filePath);
            View view = db.getView(viewName);
            ViewEntryCollection vc = view.getAllEntriesByKey(keyValues, exactMatch);
            ViewEntry ve, vn;
            ve = vc.getFirstEntry();
            Vector<Object> results = new Vector<Object>();
            while (ve != null) {
                Document doc = ve.getDocument();
                results.add(doc.getItemValue(fieldName));
                doc.recycle();

                vn = vc.getNextEntry();
                ve.recycle();
                ve = vn;
            }

            vc.recycle();

            if (!cachedLookup.containsKey(serverName)) {
                cachedLookup.put(serverName, new HashMap<String, HashMap<String, HashMap<Object, HashMap<Object, CachedData>>>>());
            }

            if (!cachedLookup.get(serverName).containsKey(filePath)) {
                cachedLookup.get(serverName).put(filePath, new HashMap<String, HashMap<Object, HashMap<Object, CachedData>>>());
            }

            if (!cachedLookup.get(serverName).get(filePath).containsKey(viewName)) {
                cachedLookup.get(serverName).get(filePath).put(viewName, new HashMap<Object, HashMap<Object, CachedData>>());
            }

            if (!cachedLookup.get(serverName).get(filePath).get(viewName).containsKey(keyValues)) {
                cachedLookup.get(serverName).get(filePath).get(viewName).put(keyValues, new HashMap<Object, CachedData>());
            }

            CachedData cache;
            if (cachedLookup.get(serverName).get(filePath).get(viewName).get(keyValues).containsKey(fieldName)) {
                cache = cachedLookup.get(serverName).get(filePath).get(viewName).get(keyValues).get(fieldName); 
            } else {
                cache = new CachedData();
            }

            Date dt = new Date();
            dt.setHours(dt.getHours() + 1);
            cache.setUpdateTime(dt);
            cache.setValues(results);           

            cachedLookup.get(serverName).get(filePath).get(viewName).get(keyValues).put(fieldName, cache);

            view.recycle();
            db.recycle();

            return results;

        } catch (NotesException e) {
            // debug here, im lazy
            return null;
        }
    }   

    private static Object resolveVariable(String variable) {
        return FacesContext.getCurrentInstance().getApplication()
                .getVariableResolver().resolveVariable(
                        FacesContext.getCurrentInstance(), variable);
    }   

}

Example on how to use in an XPage:

<xp:text id="text1">
    <xp:this.value><![CDATA[#{javascript:
        com.ZetaOne.example.CachedLookup.getCurrentInstance().doCachedLookup(
            database.getServer(),
            database.getFilePath(),
            "lookup",
            "Test Category",
            "Value",
            true
        )
    }]]></xp:this.value>
</xp:text>

Jeremy Hodge
  • 1,655
  • 9
  • 8
  • 1
    This has been really helpful. As my lookup databases are fixed I reduced the number if variables needed to pass into the function to just the key and field and the rest needed for the lookup are set within the function. Thanks for this. – Declan Lynch Mar 06 '12 at 19:28
  • I dont have any experience with the server scoped beans but how long would such a bean take to invalidate? At nsf level you can specify the timeout before the application gets cleaned from the memory. – jjtbsomhorst Mar 07 '12 at 05:29
  • 1
    This isn't really a server scoped bean in the same sense as a view scoped bean. Its whats called a singleton, meaning only one instance is created and used for the entire JVM and does not die until the JVM is restarted. Since that is the case, thats why the CachedData has the updateTime member, which controls the valid lifetime of the cached data. This example doesn't cull any of the cache so it still sits in memory even if its no longer valid, and a more robust implementation might work to cull stale cache records at specified periods. – Jeremy Hodge Mar 07 '12 at 14:50
  • Jeremy if you were using this technique in an XAgent type approach what would you set scope to be in the faces context? I assume Application as its the only shared scope? – markbarton Jul 19 '12 at 09:42
  • I'm assuming you mean setting it up in faces-config.xml ? In any case, a scope really isn't necessary since it uses the singleton pattern. The map is a static map so it belongs to the class definition and not an instance, so it is shared across the entire JVM. If you added it to the faces-config to make referencing it a bit easier set the scope to none so it just discards the instance, as it is useless since everything on it would be static anyway. – Jeremy Hodge Jul 19 '12 at 15:40
  • Thanks Jeremy - might explain why our server keeps running out of JVM memory. – markbarton Jul 23 '12 at 08:33
  • Hi Jeremy - Java newbie so excuse the innane comments. Just want to get this right as we want to use this caching pattern in several different places (mainly for multi db type aheads which are cached). You mention static map - but I don't see one referenced - is it static because its a singleton and the ServerMap class you mentioned I assume its just keep the class you listed above cleaner? – markbarton Jul 23 '12 at 09:21
  • Sorry for the delay getting back to you. Static refers to the static keyword used to declare _instance in the class (private static CachedLookup _instance;). The static keyword signifies that the declaration belongs to the CLASS not the instance. This is something a bit different to grasp, but if you think of a class as just an object that provides the definition of other objects, and there is only one instance of a definition for the entire JVM, you can start to see how members can belong to the class. Statics are created only on the class and are shared across all instances-cont'd below – Jeremy Hodge Jul 31 '12 at 19:07
  • That means that no matter what you do, you can't create multiple instances of the cached map. "There can be only one." If you are running out of memory, my guess would be you are caching a lot of data, and your available memory is limited. (Note you are limited to 2 or 3GB total for the entire JVM, even if you re running 64bit if my memory serves me right. Like any type of memory based cache, you should use this solution with care, as it can lead to increased memory consumption and impact scalability. – Jeremy Hodge Jul 31 '12 at 19:12
2

When you do beans you can use EL to get the content which is faster than SSJS. And YES - use the server scope from the XSP Starter kit. For the caching: don't reinvent the wheel! There is a VERY complete implementation of all sorts of fancy caching available: http://commons.apache.org/jcs/ I used that in a Tomcat app before and it worked very reliable. I don't see why it wouldn't be a good choice for your task.

stwissel
  • 20,110
  • 6
  • 54
  • 101