Why does 'Git add *' leave some files unstaged?

Question

I initialized a new git repo in my Django project. All files( including the files in subdirectories and pipfile) are staged when I run git add * except the .__atomic-writegxd2tirf file. Why is this one excluded?

Answer 1

The git add * stages only the whose names do not begin with a "." hence the problem. An alternate way could be running git add . for all files OR running git add for the file name beginning with a "."(dot) exclusively.

Answer 2

Now that you have experienced this with several different commands (git add and echo in particular), it's time to generalize.

On Unix-like systems (including Linux, macOS Terminal windows, and so on), or when using bash as a command interpreter on Windows, you are dealing with a programming language that is called a shell. There are different shell programming languages (bash, dash, sh, zsh, fish, and so on—most tend to have sh in their names for historical reasons) but they all tend to share certain attributes. One of these is how they expand command line arguments to be passed to commands.

$ mkdir t && cd t && touch .a .b c d e
$ ls -A
.a      .b      c       d       e
$ echo *
c d e

There are five files¹ in this temporary directory (besides the standard two named . and .. that ls -A omits). But echo * only expands to three of them. That's because * expansion deliberately omits any files whose name² starts with ..

Some shells have an option to turn this on or off: e.g., in bash, you can set the dotglob option. (Users probably shouldn't fiddle with these options too much, lest they get surprises when using someone else's shell or some other shell, but it's good to know that they exist.)

Shell programming languages are mostly constructed around the idea of running other programs. So when the shell produces a prompt, like $ or > or whatever the shell's standard prompt is, and you type in a series of white-space-separated words, the shell breaks up those words and uses one of them as a command name and the rest as arguments to the command, and runs that command. Hence, entering:

git add *

has the shell run the git command with arguments add and *. But * itself is treated specially by the shell, expanded out to file names, excluding those that start with dot. So the git command sees only the arguments:

add
c
d
e

(one argument per line). The first argument to git is a sub-command, in this case add, and the remaining are interpreted by the sub-command.³

Since * got expanded by the shell, Git never sees the * and never has its own chance to do its own thing with the *. You can, however, protect the * from the shell:

*

Here, the shell treated the double quotes as a sort of protection. This kept the shell from interpreting the *. The shell did interpret the double quotes themselves though, so echo did not see the double quotes. It saw only the asterisk *.

$ echo '"*"'
"*"

This time I used single quotes as the outer layer. These also tell the shell to protect stuff, i.e., not to interpret either the double quotes or the asterisk. So this time echo saw both the double quotes and the asterisk.

The shells' rules for quoting vary from one shell to another, and this can all get very complicated. None of this has anything to do with Git itself: the shell is doing the same kind of interpretation regardless of which command is to be run.

If you do manage to pass a literal asterisk on to Git, Git will do its own interpretation of the asterisk, and its interpretation differs from the shell's. So git add '*' behaves differently.

If you use Windows and CMD.EXE, note that CMD.EXE itself is not nearly as programmable as a shell and has much simpler command-line rules. In some ways this makes its use easier and more predictable. This is its advantage, but also its disadvantage: the programmability of the typical Unix shells makes it possible to write entire systems using them.

---

¹In the world of Unix, even a directory (or folder, if you prefer that term) is a kind of file. Sometimes people use the word file to mean both files and directories, but since I've excluded . and .., which are directory-files, I bypassed the need to mention it—except for this footnote!

²In a hierarchical file system like those found on Unix-like systems, files don't necessarily have just one name. Here, the word name really refers to a component name as found in some containing directory. The file's name in Git might be path/to/.file for instance. At the Unix-file-system level, however, this is a directory named path holding a directory named to holding a file whose component name is .file. To the shell there's a dot in the part of the file name at the end, so echo path/to/* won't match it.

³This pattern of having a top level command with sub-commands has become more common recently, but only parts of the Unix system work this way. In Python in particular you can use argparse to emulate this same kind of behavior.