0

I am considering trying to optimize money generation ("farming") in the tower defense game Bloons TD 6. For simplicity, only consider a small set of farm types, which cost a certain amount of money to buy and produce a set amount of money per round (actually the farms produce money during the round, but ignore this). It is also possible to sell farms for a fixed percentage of their purchase cost (say 75%), and the ultimate goal is to accumulate as much money as possible after a fixed number of rounds (say 40).

If it helps, here are some example numbers:

  • Merchantman: costs $3000, produces $200 each round
  • Favored Trades: costs $5500 to upgrade from Merchantman, produces $500 per round
  • Trade Empire: limit 1, costs $23000 to upgrade from Favored Trades, produces $800 each round, has a complex interaction which increases income of existing Merchantman and Favored Trades depending on how many there are

My first thought was implementing as dynamic programming, but there is no easy way to break this down into subproblems. The state can be represented as (round, money, farms), but these may take on exponentially many (integer) values, and it's also not clear what to maximize: greedily maximizing only money per round would never result in any investment.

A natural way to view this problem is as a tree search, where each node represents game state per round and each edge represents possible actions taken between rounds (multiple actions like buys/sells are possible). Another formulation is to have each edge be one action such as "buy farm" or "advance round".

A tree search better than exhaustive search must have some kind of heuristic guiding it, so I came up with the heuristic of money + total sell value + total farm income from now to the end. This represents the ending money if you did nothing further until the end, then sold all farms.

Further, I came up with potential search strategies:

  1. Greedy search with limited depth: Search exhaustively as deep as feasible, say 5 rounds, move to the solution with best heuristic value, then repeat. I'm not sure if this algorithm has a name, but it is like chess engines who can only see 10 moves deep and then have to make a move (however, chess engines employ alpha-beta pruning to prune many branches)

  2. Branch-and-bound: Maintain a queue of candidate solutions, and branch (advance one round and try all actions) if the heuristic value is at least a fraction of the best known heuristic and is considered good enough. The exact implementation details are unclear to me.

Are these realistic approaches? What are algorithms used to solve these kinds of optimization over discrete steps problems?

qwr
  • 9,525
  • 5
  • 58
  • 102
  • Dear close voter: nothing is opinion based about asking for an algorithm. – qwr Sep 14 '22 at 18:43
  • All the possible states and the transitions between them are known from the start? – ravenspoint Sep 14 '22 at 18:54
  • @ravenspoint yes. but there may be exponentially many. each farm type count, round, and money take integer values. – qwr Sep 14 '22 at 18:57
  • I suggest calculating the NPV ( net present value ) of each farm that is available. Select the edge from each state that gives the greatest NPV. – ravenspoint Sep 14 '22 at 18:59
  • @ravenspoint how is NPV defined? Is it sell value + total earning potential? – qwr Sep 14 '22 at 19:04
  • 1
    Usually NPV is discounted by an rate representing the opportunity cost of committing cash. In your case, you could start with a zero discount rate and experiment for there. – ravenspoint Sep 14 '22 at 19:10
  • Being too lazy to play a game so you're thinking about an algorithm to automatically play it in an optimized way deserves my upvote, you damn geek :) – Cid Sep 15 '22 at 15:59
  • @Cid yes, well I know the general meta strategies (build up 10 merchantmen, save up for trade empire, finish 20 merchantmen, upgrade those to favored trades) but it's actually quite time consuming to test in game versus calculating it out. – qwr Sep 15 '22 at 17:14

1 Answers1

0

I suggest calculating the NPV ( net present value ) of each farm that is available. Select the edge from each state that gives the greatest NPV. Start with a zero discount rate and experiment from there

e.g. ( based on your example )

NPV( Merchantman ) = 200 * RR - 750

( RR is the number of rounds remaining. )

If RR = 40 then NPV is 7250, if RR = 5, NPV = 250

ravenspoint
  • 19,093
  • 6
  • 57
  • 103