Smallest sub-list that contains all numbers

Question

I am trying to write a program in sml that takes in the length of a list, the max number that will appear on the list and the list of course. It then calculates the length of the smallest "sub-list" that contains all numbers.

I have tried to use the sliding window approach , with two indexes , front and tail. The front scans first and when it finds a number it writes into a map how many times it has already seen this number. If the program finds all numbers then it calls the tail. The tail scans the list and if it finds that a number has been seen more times than 1 it takes it off.

The code I have tried so far is the following:

structure Key=
 struct
  type ord_key=int
  val compare=Int.compare
 end


fun min x y = if x>y then y else x;


structure mymap = BinaryMapFn ( Key  );

fun smallest_sub(n,t,listall,map)=
let
 val k=0
 val front=0
 val tail=0

 val minimum= n; 

 val list1=listall;
 val list2=listall;

 fun increase(list1,front,k,ourmap)=
  let 
   val number= hd list1
   val elem=mymap.find(ourmap,number)
   val per=getOpt(elem,0)+1

   fun decrease(list2,tail,k,ourmap,minimum)=
    let 
     val number=hd list2
     val elem=mymap.find(ourmap,number)
     val per=getOpt(elem,0)-1
     val per1=getOpt(elem,0)
    in
     if k>t then
      if (per1=1) then decrease(tl list2,tail+1,k-1,mymap.insert(ourmap,number,per),min minimum (front-tail))
      else decrease(tl list2,tail+1,k,mymap.insert(ourmap,number,per),min minimum (front-tail))
     else increase (list1, front,k,ourmap)
    end

  in
   if t>k then
    if (elem<>NONE) then increase (tl list1,front+1,k,mymap.insert(ourmap,number,per))
    else increase(tl list1,front+1,k+1,mymap.insert(ourmap,number,per))
   else (if (n>front) then decrease(list2,tail,k,ourmap,minimum) else minimum)
  end


in
  increase(list1,front,k,map)
end


fun solve (n,t,acc)= smallest_sub(n,t,acc,mymap.empty)

But when I call it with this smallest_sub(10,3,[1,3,1,3,1,3,3,2,2,1]); it does not work. What have I done wrong??

Example: if input is 1,3,1,3,1,3,3,2,2,1 the program should recognize that the parto of the list that contains all numbers and is the smallest is 1,3,3,2 and 3,2,2,1 so the output should be 4

Answer 1

This problem of "smallest sub-list that contains all values" seems to recur in new questions without a successful answer. This is because it's not a minimal, complete, and verifiable example.

Because you use a "sliding window" approach, indexing the front and the back of your input, a list taking O(n) time to index elements is not ideal. You really do want to use arrays here. If your input function must have a list, you can convert it to an array for the purpose of the algorithm.

I'd like to perform a cleanup of the code before answering, because running your current code by hand is a bit hard because it's so condensed. Here's an example of how you could abstract out the book-keeping of whether a given sub-list contains at least one copy of each element in the original list:

Edit: I changed the code below after originally posting it.

structure CountMap = struct
    structure IntMap = BinaryMapFn(struct
        type ord_key = int
        val compare = Int.compare
    end)

    fun count (m, x) =
        Option.getOpt (IntMap.find (m, x), 0)

    fun increment (m, x) =
        IntMap.insert (m, x, count (m, x) + 1)

    fun decrement (m, x) =
        let val c' = count (m, x)
        in if c' <= 1
           then NONE
           else SOME (IntMap.insert (m, x, c' - 1))
        end

    fun flip f (x, y) = f (y, x)
    val fromList = List.foldl (flip increment) IntMap.empty
end

That is, a CountMap is an int IntMap.map where the Int represents the fixed key type of the map, being int, and the int parameter in front of it represents the value type of the map, being a count of how many times this value occurred.

When building the initialCountMap below, you use CountMap.increment, and when you use the "sliding window" approach, you use CountMap.decrement to produce a new countMap that you can test on recursively.

If you decrement the occurrence below 1, you're looking at a sub-list that doesn't contain every element at least once; we rule out any solution by letting CountMap.decrement return NONE.

With all of this machinery abstracted out, the algorithm itself becomes much easier to express. First, I'd like to convert the list to an array so that indexing becomes O(1), because we'll be doing a lot of indexing.

fun smallest_sublist_length [] = 0
  | smallest_sublist_length (xs : int list) =
    let val arr = Array.fromList xs
        val initialCountMap = CountMap.fromList xs
        fun go countMap i j =
            let val xi = Array.sub (arr, i)
                val xj = Array.sub (arr, j)
                val decrementLeft = CountMap.decrement (countMap, xi)
                val decrementRight = CountMap.decrement (countMap, xj)
            in
                case (decrementLeft, decrementRight) of
                   (SOME leftCountMap, SOME rightCountMap) =>
                     Int.min (
                       go leftCountMap (i+1) j,
                       go rightCountMap i (j-1)
                     )
                 | (SOME leftCountMap, NONE) => go leftCountMap (i+1) j
                 | (NONE, SOME rightCountMap) => go rightCountMap i (j-1)
                 | (NONE, NONE) => j - i + 1
            end
    in
      go initialCountMap 0 (Array.length arr - 1)
    end

This appears to work, but...

Doing Int.min (go left..., go right...) incurs a cost of O(n^2) stack memory (in the case where you cannot rule out either being optimal). This is a good use-case for dynamic programming because your recursive sub-problems have a common sub-structure, i.e.

go initialCountMap 0 10
 |- go leftCountMap 1 10
 |   |- ...
 |   `- go rightCountMap 1 9  <-.
 `- go rightCountMap 0 9        | possibly same sub-problem!
     |- go leftCountMap 1 9   <-'
     `- ...

So maybe there's a way to store the recursive sub-problem in a memory array and not perform a recursive lookup if you know the result to this sub-problem. How to do memoization in SML is a good question in and of itself. How to do purely functional memoization in a non-lazy language is an even better one.

Another optimization you could make is that if you ever find a sub-list the size of the number of unique elements, you need to look no further. This number is incidentally the number of elements in initialCountMap, and IntMap probably has a function for finding it.