The collection view will adapt well to the safe area provided you constrain it to said safe area.
If you have gaps on the top and bottom this is the intended result from the safe area, with it having the gaps it needs on iPhone X and no gaps on other devices.
You can get around this by constraining to the safe area top with a negative constant, however this is not recommended and will result in bad effects on other devices i.e the bottom element being off the screen when you hit scroll bottom.
The best way to handle this if you have no way around it is to constrain to the edges of the parent view as opposed to the safe area, but then you have to consider how you will be obscuring content under the notch.