While researching the architecture of the AlphaStar neural network, I came across a description that I am not fully clear on. Despite understanding the majority of the blocks, this specific part have left me with some questions. Knowledge about the context of AI and the game Starcraft 2 are not required for understanding.
From https://github.com/chengyu2/learning_alpha_star/blob/master/detailed-architecture.txt
Core - Deep LSTM
Core: Inputs: prev_state, embedded_entity, embedded_spatial, embedded_scalar
Outputs: next_state - The LSTM state for the next step, lstm_output - The output of the LSTM
The Core concatenates
embedded_entity,embedded_spatial, andembedded_scalarinto a single 1D tensor, and feeds that tensor along withprev_stateinto an LSTM with 3 hidden layers each of size 384. No projection is used. We apply layer norm to the gates. The outputs of the LSTM are the outputs of this module.
When the term "3 hidden layers" is used, is it referring to 3 separate LSTM cells or 3 layers within a single cell? If there is only one input, how would multiple LSTM cells be implemented? Are there any modifications that need to be made to the LSTM cell in this scenario?
I could not find material on the topic.
