If all your data has the same spatial reference system (SRS), then you can plot them all in the same space (comparing apples and apples). No transform is done with SRID=-1. If you want to ignore the SRS and you are the only user (i.e., you aren't sharing the data to others), go ahead and make the SRID '-1'.
However, if you want to mix data with different SRSes (for example, data in WGS84 Long/Lat and data in UTM Eastings/Northings), then the GIS needs to know what to transform to reproject the data so it can be overlain beside each other. There are thousands of different SRSes out there.
Furthermore, if you are sharing the data to someone else, the data has diminished value. (I've spent hours trying to brute-force guess someone else's projection when they didn't specify it). For example, there is often a couple hundred metres of error between identical Long/Lat degree coordinates between NAD27 and NAD83 datums.
Mind you, that you don't get any performance gains by specifying a '-1' SRID. If you really don't know the SRID, then better keep it '-1' than incorrectly guessing the wrong one.