Efficient hash table access with constructed keys

Question

Common Lisp allows any lisp object to serve as a hash table key. But what if you only want to use part of an object as the key. For example, in

(defstruct action
  (name nil :type symbol)
  (parameters nil :type list)
  (time 0 :type fixnum)
  (database nil :type hash-table))

the time slot is inappropriate for equalp hashing situations. What is a good strategy for accessing a hash table using a (partial) lisp object as key? One approach might use a key like (list (action-name act1) (action-parameters act1) (action-database act1)), but this seems rather inefficient. Another approach might create a substructure to the action defstruct with just the three appropriate slots, and use that substructure as a key, but this seems somewhat ad-hoc just for the purpose of hash table access. Are there other methods that could work better?

See: http://stackoverflow.com/q/33828408/124319. As for being inefficient, you can't say that without proper timing. It seems like this is a very tiny allocation that gets garbage-collected right away. — coredump, Dec 05 '16 at 07:15
I think `(database nil :type hash-table)` should be `(database (make-hash-table) :type hash-table)` instead. Otherwise an error is thrown when an `action` is created. — anonymous, Dec 05 '16 at 15:33
@coredump: You're right, a quick test shows timing difference negligible using the structure as key vs. a list of slots. Also, the interning approach in your reference, although not applicable in my app, could be useful in certain circumstances. Another idea is to temporarily set the inappropriate slots to null values just for hash table access, and keep the whole structure as key. — davypough, Dec 05 '16 at 18:17
@tsikov: Thanks for revising. For simple prototyping, I've been using Allegro CL Express, which doesn't complain about a lot of similar things. — davypough, Dec 05 '16 at 18:18

anquegi · Accepted Answer · 2016-12-14T10:28:50.633

I will use Common Lisp library from here, as for example

cl-custom-hash-table

Then going to your code, first when you create an action like this:

CL-USER>  (setq action-1 (make-action 
   :parameters '(1 2 3) 
   :time 45))

produce this error:

The value NIL is not of the expected type HASH-TABLE.
   [Condition of type TYPE-ERROR]

so you need to change your definition to something like this:

CL-USER> (defstruct action
  (name nil :type symbol)
  (parameters nil :type list)
  (time 0 :type fixnum)
  (database (make-hash-table) :type hash-table))
ACTION
CL-USER>  (setq action-1 (make-action 
   :parameters '(1 2 3) 
   :time 45))

#S(ACTION :NAME NIL :PARAMETERS (1 2 3) :TIME 45 :DATABASE #<HASH-TABLE :TEST EQL size 0/60 #x3020012E708D>)

Then you should define a function for equal time or what ever you need as follow: accesing to the data

CL-USER> (action-time action-1)
45

create another action

CL-USER>  (setq action-2 (make-action 
   :parameters '(1 2 3) 
   :time 102))

#S(ACTION :NAME NIL :PARAMETERS (1 2 3) :TIME 102 :DATABASE #<HASH-TABLE :TEST EQL size 0/60 #x3020012FE58D>)

create the function for testing

CL-USER> (defun equal-actions-by-time (a1 a2) (= (action-time a1) (action-time a2)))
EQUAL-ACTIONS-BY-TIME

define the hash-function:

CL-USER> (defun hash-action (a) (action-time a))
HASH-ACTION

create your hash

CL-USER> (define-custom-hash-table-constructor make-action-hash :test equal-actions-by-time :hash-function hash-action)
MAKE-ACTION-HASH
CL-USER> (defparameter *foo-hash* (make-action-hash) "variable for stackoverflow")
*FOO-HASH*

try it:

CL-USER>  (setf (gethash action-1 *foo-hash*) 1
          (gethash action-2 *foo-hash*) 10)
10

CL-USER>  (gethash action-1 *foo-hash*)
1
T

You can avoid using a library if the distribution will work in implementations that support custom TEST/HASH functions natively, if not you can use with-custom-hash-table

In the optimus case you can work as follow:

CL-USER> (defparameter *foo-hash* (make-hash-table :test 'equal-actions-by-time :hash-function 'hash-action))
*FOO-HASH*
CL-USER>  (setf (gethash action-1 *foo-hash*) 1
          (gethash action-2 *foo-hash*) 10)
10
CL-USER>  (gethash action-1 *foo-hash*)
1
T

Thanks for your insights re custom hash tables. But I'm still not clear on the advantage of custom vs standard for my original problem. For that 3-slot effective key, I assume the custom hash-table test function would use (and (eq (action-name a1) (action-name a2)) (equal (action-parameters a1) (action-parameters a2)) (equalp (action-database a1) (action-database a2))), whereas the standard equalp hash-table would simply use (list (action-name a1) (action-parameters a1) (action-database a1)) as key. Is the custom table significantly more efficient than the standard table in this case? — davypough, Dec 15 '16 at 04:04
Well using a hash table is more efficient than a list, take a look here https://www.cs.cmu.edu/Groups/AI/html/cltl/clm/node154.html — anquegi, Dec 15 '16 at 08:57
@davypough you could manually "convert" ACTION instances into a list before using them as key for the hash table. An advantage of using a custom hash table is readability: you don't have to mention this conversion for all hash table calls, but only once when defining the hash table. A performance advantage is that you avoid these temporary lists. — zut, Jan 25 '17 at 12:27

Efficient hash table access with constructed keys

1 Answers1