3

It appears that the two options currently available (6.3.0.Final) are for a value range to be defined as a Collection or as a ValueRange (defined by bounds). My domain has a large number of potential values for each variable that are represented by Java Objects (not counting or number-line values).

As our problems scale up, it becomes impractical for all of the possible values for all of the different planning variables to be in memory together.

Is there any way to provide values to the engine as needed? In batches or with an iterator?

If so, I could imagine the bulk of the values being stored in a database or other disk structure and only brought into memory in small batches for consideration.

Jon L
  • 41
  • 2

1 Answers1

3

For clarity to others reading this post: do these need to fit into memory?

  • All planning entities? Yes. For example when scheduling 30k processes to 10k computers, all 30k processes must fit into memory, twice (once for the working solution and once for the best solution found along the way) but not squared (like in simplex, which doesn't scale without partitioning).
  • All planning values? (this is your question) No. That's why planning the ValueRange interface was invented. However, when you return a simple Collection from your @ValueRangeProvider, of course it does. Also note that there's a much bigger memory usage if you're using value range from entity instead of the normal value range from solution.
  • All possible solutions? No, if that were the case we couldn't even solve small problems like 300 processes on 100 computers.

To answer your question:

Because your planning values don't fit into memory, your @ValueRangeProvider can't return a Collection. Instead return a CountableValueRange (or ValueRange if the former is impossible). If the value type is number (int, long, double, BigDecimal) use a method on ValueRangeFactory. If not - and you don't want or can't refactor (which I would recommend) - then go down the rabbit hole and use deep implementation classes to implement your own CountableValueRange... Just start reading from ValueRangeFactory.

Out of interest: what type are your planning value classes? Why can't it be a number? I 'd like to understand your use case so I can evaluate if we need to add out-of-the-box support in OptaPlanner for such value ranges (or a more generic variant).

Geoffrey De Smet
  • 26,223
  • 11
  • 73
  • 120
  • 1
    We simply have complex structures with nested content as our planning variables. Each planning value has a number of unique characteristics that are used during score evaluation. It seems difficult to refactor this down to a single number other than an index value into some other in-memory collection (which doesn't really address the problem). – Jon L Oct 23 '15 at 15:12
  • Could you give a concrete example? (doesn't need to be your actual case if you can't share that, just an example that allows me to reason on it). Implementing `ValueRange` should fix your problem. – Geoffrey De Smet Oct 23 '15 at 15:36
  • @GeoffreyDeSmet I have a similar use case: an example planning entity is a `Person` object that has attributes like `age`, `gender`, and `location`. All of these individual attributes are used in constraints, but it is the overall `Person` that is a planning variable in the planning entity. I don't see any easy way to reduce that down to something `ValueRangeFactory` can produce out of the box, but `ValueRange` seems like a good approach. Unless I'm approaching the problem incorrectly. – Raman May 18 '22 at 16:25
  • I note that `ValueRange` doesn't offer the ability to load data in batches though, which limits the possibilities in terms of optimizing any related I/O. – Raman May 18 '22 at 16:29
  • If `age` is a planning variable, you can do @ValueRangeProvider on a method that returns `ValueRangeFactory.createIntValueRange(0,120)` – Geoffrey De Smet May 18 '22 at 23:26
  • Well an instance of `person` is the planning variable in the planning entity -- age is just an attribute of each person. – Raman May 19 '22 at 12:41