Memory is not the place to store the time, as memory looses it's contents when power is off. To keep track of time you need some kind if persistend storage (like EEPROM or flash) in combination with an actual clock.
Servers and end-user devices (pc, laptop, tablet, smartphone, ...) usually have a built-in hardware clock and some means to keep that clock going when the machine is switched off.
In de pc/server world this is usually the RTC (real-time clock) in combination with a battery.
In some other cases it's possible to get an add-on providing RTC+battery (see image below).
Embedded/simple devices (e.g. Arduino) may not have this and so they will loose track of time when shut down.

For ESP32 it seems there are possibilities to have RTC, see for example: http://www.esp32learning.com/code/esp32-and-ds3231-rtc-example.php