how to deploy with git without including sensitive data in history

Question

there is a way to manage my deployments and development workflow with git but without including sensitive data in the history (example database, secret tokens and so on..)? I need to have more branch (ex master, staging and production?) I ask this because one day a project could become open source and it will be very useful exclude sensitive data from git's history or from the deployment to a production server, of course I can always deleting the .git folder.. but could be very useful have the original history of the project.. so how I can avoid this problems? thanks to all.

Answer 1

Get rid of the sensitive data

First and foremost, get rid of your sensitive data. It has no place in your code repository. Transfer your passwords, hostnames etc. to a configuration file which lives outside the repository (you will have an example/template inside, of course).

Get rid of the the secret history, the insane way

It won't happen easily. You could in theory do a batch operation which basically goes through your whole commit history and creates "mirror commits", each and everyone of which having the secret removed by some script. For any non-trivial size (especially if you also want to keep branches and not just the main timeline), this will be "interesting" to say the least.

The general outline of this strategy is this:

1. Checkout the very first commit on your old repository repA.
2. Create a new, empty repository repB.
3. Copy (using cp) the whole content of your repA working directory (excluding .git) to the empty repB working directory.
4. In repB, do a git add -A ; git commit -m XXX where XXX is the original commit message from repA. If there are any refs (tags or branches) in repA which point to the current commit, create then in repB.
5. Find all child commits of the current commit in repA.
6. For every child commit, checkout, rinse and repeat (from step 3).

If it is enough for you to have a single time line (i.e., only one child per commit, all on the master branch), then this is not hard and pretty feasible.

If you want to get your branches, you can do that too, same logic, just recurse into all children at each point. Skip merge commits for any branch except master; and for master, commit your merge commits as above (do not try to make them a "real" merge).

If you want to have merges as well, then it gets insane since you will have to do any and all conflict resolutions again. Forget about that.

I have done similar things for single branches; it works, but as soon as you get into conflict resolution, it will not be worth the trouble. It is kind of a giant, automated git rebase -i spanning the whole repository.

Cut off history to get rid of secrets

Instead, create a new repsitory, copy the first state of your old repository which does not include your secrets anymore in it, and commit. Done. You lose all the old history.

You probably don't want this either, because of that.

Make your sensitive data insensitive

So you have secrets in your repository. So what, passwords can be changed. Change your passwords and be done with it. Still, introduce configuration files, but don't bother about history.

Answer 2

You can create a text file called .gitignore and save that into your project directory. In there you would have to write out all the files/extensions that you do not want Git to track. You can even use regular expressions such as the *, which will represent all files. i.e.:

projectfilename/.gitignore
somefile.txt
*.php --> will ignore all files with the extension of PHP

For more info on .gitignore, you can checkout https://git-scm.com/docs/gitignore

Answer 3

You can write a .gitignore file including files you wouldn't like git to track. For example, say I wanted to make sure git doesn't track log files

I can create a .gitignore file and write something like

logs/*.log

Then add and commit the .gitignore file and git would stop tracking folder logs with * all files with .log extension.

Here are some exclude patterns

video.mp4 # git will not track this particular file
*.mp4     # git will not track files with *.mp4 extension
folder/   # git will not track this file

Hope you get the idea

Answer 4

Your best bet would be to add any sensitive files to your .gitignore file.

You would need to devise a way to add those files back in, when moving between environments (if it's only 1 or 2 files - FTP might suffice).

Alternatively, build an exception into your application work out that it has files missing - and auto-generate them based on user input.

If you've ever used WordPress with Git, you'll have experienced this already. WordPress has a configuration file called wp-config.php - which contains all of the database connection information. Obviously this is something that you don't want to share around (and it's also something that will likely change between development environments).

As a result - wp-config.php is ignored from version control. When deploying WordPress to a new environment, it detects that wp-config.php doesn't exist, and generates a new version; after asking for database information.

Hope this helps.

Answer 5

One approach would be to keep all your sensitive data in a sub-folder of you project. This sub-folder could then either be managed with git submodule or git subtree. If your project is not compatible with keeping sensitive data in one folder, you could store it in a single folder and use a Makefile/script/etc to copy the sensitive data to the needed location and .gitignore to prevent it from being checked into git.