-1

I have been working on a simulation model for battery swapping in Anylogic. So far I have developed the simulation model, optimization experiment and parameters variation experiment.

There are no errors in the model but the output values are unsatisfactory. Small changes such as changing the step size of the decision variables results in a drastic change in the best value obtained after every experiment. Though the objective does not change much but I am concerned about the other variables that are changing with each run. Even with multiple optimization runs it is difficult to come to a conclusion.

For reference I am posting an output of parameters variation experiment here. I ran the experiment with an optimized value but I was getting feasible results (percentile > 95%) far off the expected input values. Although, the overall result is correct (decreasing percentile with increasing charging time) but it is difficult to understand the variability.

Can anyone help?enter image description here

Abhinav saini
  • 1
  • 1
  • Impossible to solve "I cannot get the outputs I want" questions – Benjamin May 26 '21 at 15:11
  • Battery swapping sounds like the output data are a time series, so they are *not* independent observations. This affects analysis in two ways. 1) There can be a "warm-up" period - known as initialization bias - during which the starting conditions of the model influence the observed behaviors. 2) Serial correlation of the outputs can drastically affect the variance, so using s^2 to estimate variance can be wildly off (as in orders of magnitude). See Section 6 of [this tutorial](https://www.informs-sim.org/wsc07papers/007.pdf) for an overview of these issues. – pjs May 27 '21 at 18:52
  • You might want to have a look at this tutorial paper which is specifically about [analysis of simulation output data](https://www.informs-sim.org/wsc16papers/013.pdf). – pjs May 27 '21 at 19:01
  • Last, but not least, if you want to see how your results are related to multiple inputs, you should be applying [design of experiments](https://informs-sim.org/wsc20papers/135.pdf) to maximize the information obtained relative to the work expended. – pjs May 27 '21 at 19:09

1 Answers1

0

When building a model, this is a common problem you will have when looking at high level overall outputs. You could have a model bug, but it is just as likely (if not more likely) that there is some dynamic to your system that was not clear in simple Excel spreadsheets or mental models. The DES may be telling us something truly interesting about the system behavior, but without additional outputs, there is no way to understand what that is.

A few suggestions:

  1. Run this as a simple single scenario, where you manually update inputs. When you run this with the low range of input values and then the high range of input values, what do you see on the animation or additional outputs that is different than you expected or could explain the overall output trend? Try running several intermediate points.
  2. Add additional output metrics. If you look at queue sizes, resource utilizations, turn-around-times, etc; do you see anything at that level that is different than expected?
  3. Add a "replication" log. When you run a set of inputs for multiple scenarios, does any single replication stand out as an outlier? If so, re-run the scenario with that set of inputs and that random seed.

There is no substitute for understanding underlying system behavior, and without understanding those dynamics, looking at overall correlation with optimization or parameter variation experiments will often lead companies to make the wrong policies decisions.

Amy Brown Greer
  • 706
  • 4
  • 7
  • The arrival pattern of the agents is in the form of a rate schedule. From what I have observed after several runs is that: 1. the rate schedule changes for every run (to some extent) 2. I have no control over at what exact times the agents will arrive in the system. (I can control the rate/hour but not exact arrival time). So far, I believe that this might be the reason for the high variability. – Abhinav saini May 27 '21 at 07:40
  • here is a short example: 2 cars arrive at the station in two hours (1 per hour). There is only one battery available initially, charging time is 60 mins. the car that comes in the 1st hour will take that battery and leave the other for charging. Now when the 2nd car arrives, it is difficult to say whether it will have to wait or not. It depends upon the arrival time of the car before it. If the difference in the arrival time is more than 60 mins, then no waiting otherwise the 2nd car will have to wait for the battery to be charged. – Abhinav saini May 27 '21 at 07:40
  • Sounds like a pretty realistic model considering randomness. – Amy Brown Greer May 28 '21 at 15:52