This answer points out that the MP4 file header size really depends on a number of factors, like:
- number of tracks
- the H.264 encoding properties
- some muxers put their names into remaining free space ("atoms" - see link above)
As stated in the linked answer, you can expect the header to be from 1 to 4 kBytes large.
Since one megabyte is a comparably small amount of data nowadays (considering only a small amount of files - it may be different for you), you are very safe if you fetch the first megabyte only (1 MByte = 1024 KByte).
If you can stream the data you are getting and constantly check the bytes, you can listen for the character sequence mdat
, as stated in the linked answer (again). It should denote that the file header is probably over.