Path analysis invariance

Question

I am running a path analysis in order to examine its invariance between two different cultures. When I examine the fit and the parameters of the path model seperately for each culture, I find that some paths are significant in both cultures, whereas some are not significant in one culture and some are not significant in the other culture. So should I first respecify the path models, by deleting the non significant paths from both samples and then continue to examine the configural or metric invariance or is it ok to continue the analysis with no change in the path models provided that the fit indices are satisfactory?