Nectarines are actually peaches with an active recessive gene (e.g. similar genetics to red hair in humans). Commercial nectarines are cultivars with this gene.
Although it is relatively rare, in the case of peaches this means a peach stone can produce a nectarine tree, and peach trees can produce nectarine sports (I assume this is from a graft but I don't know the details).
Is it possible to identify a nectarine tree (or branch sport) before it produces fruit?
Mike is correct here. From whatever I've been able to find on the subject, the two are different only in the texture of the skin of the fruit (and nutritional values). Nectarines might be slightly more susceptible to brown rot disease, but that's not something you can identify a plant with.
From this site:
A peach and a nectarine are very similar. Genetically, there is not much difference between the two. The main difference is that a peach has fuzz on its skin while a nectarine does not.
Nectarines are a bit more likely to be affected by diseases such as brown rot and bacterial spot. Many nectarine varieties have a spicy "zing" to their taste. The nectarine is thought to have originated as a mutant of the peach.
Thanks to Mike's twitter efforts, here's an expert opinion from @ReadsNursery, which confirms the info from other articles:
No chance , the only technical bits to go on are the glands at the base of the leaves which can be specific to certain cv's [cultivars]
