All the colour tells you is how fast it is moving away. The colour of a star doesn't tell you the distance.
They use the star's color, its intensity, in addition to its size in the sky to guess how far away it is.
Basically they're operating under the assumption that a majority of stars in the universe are "average stars" with very similar properties. Hence if one is bigger than another, one is more intense than another, one is colored slightly differently than another, it can give the astronomer a loose guess where it is.
How Doppler Shift plays into it has mostly to do with the current "expanding universe" hypothesis, where we exist in an expanding balloon, the stars shifted blue being moving away from the observer and the stars shifted red moving towards the observer. The smallest blue stars being hypothesized to be the furthest stars and the bigger red ones being hypothesized to be the closest stars.
Star distances are generally one hypothesis built upon the next.
IE. "We know that the sun is 93 million miles away and takes up 5 degrees of the sky. We also think that it's an average sized star. Therefore these little dots in the night sky with an arc-minute size of 0.000...1 degrees must be xxxx million miles away."