Yes. You need a NTP server on the local LAN. There's two things here - accurate time (e.g. the time is actually correct) and time sync (they all have the same time, whether correct or not).
If you just need the systems to simply be in sync, to agree on what the time is even if they're not using the correct time, you can just install a cheap box with your favourite Linux distro, configure it to act as a time server (example) and tell both these embedded systems to use that as their time source.
If you need accurate time then you need a GPS or radio time clock, which will set time to an external source without needing an Ethernet connection. This is the one I use, for example, but there are others out there.