First let's cut to the chase:
That password has been compromised. There is no way for you to trace where it might have gone after you pushed it to the remote. You can't put that genie back in the bottle; the only secure thing to do is to change the password.
That makes the rest of the discussion mostly academic. Either the password won't be changed and removing it from the history is an insufficient measure, or it will be changed and removing it from the history is cosmetic.
Nonetheless, to answer your direct question:
If I merge the feature branch to the source branch with squash merge and delete the feature branch, will that effectively remove the password from all git history?
Not really. To fully remove the data from your local repo, you would have to make sure that the commit(s) containing it are unreachable from all refs as well as the reflog, and then you would have to force garbage collection. To truly remove it from the remote requires similar procedures, but how that sort of housekeeping is handled varies depending on how the repo is hosted.
And again, all of this would assume that nobody had already fetched a ref whose history contains the password. If they did, nothing you can do will force them to discard it.