Let's assume a graph with Euclidean distances. The source is s and the destination is t.
As you know, Dijkstra's algorithm visits vertices x in order of nondecreasing distance(s, x).
The A* algorithm visits vertices x in order of nondecreasing distance(s, x) + h(x). The function h must be an admissible heuristic, which in this setting means that h(x) ≤ distance(x, t). Consider various choices of h.
h(x) = 0. This makes A* behave like Dijkstra.
h(x) = distance(x, t). This is the best possible admissible heuristic. A* will visit only those vertices with distance(s, x) + distance(x, t) = distance(s, t), i.e., those vertices on a shortest path from s to t. Unfortunately, this choice of h expensive to compute.
h(x) = ||x - t||. The straight-line distance is computable in time O(1) and is always a lower bound on the distance.
The last heuristic works well when there's a reasonably straight shot from s to t, but as detours pile up, A* will visit many vertices that are "out of the way".
Sedgewick–Vitter turns it up to 11 by using h(x) = a ||x - t|| for a > 1. This heuristic is not admissible, so we may not find the shortest path, but by punishing early detours, it hopefully trims the search space without decreasing the solution quality too much.