Common in many threads both here and in the General Discussion forums are posts of the form:
"Theory-X can't be right because it doesn't explain observation-Y".Almost every post defending theory-X (where both sides agree that observation-Y is a valid observation) typically looks like "just because theory-x doesn't explain it (yet) doesn't invalidate theory X". (Wardogg's post on methane invalidating the evolutionary timeline)
In essence, observation Y may or may not be explainable in theory X.
Yet there are some observations that DO invalidate previous theories. So what's so special about those observations?
The clearest examples are where the theory actually predicts something different from the observation. The galactic rotation curves, for example that lead to dark matter. Things move slower the further they get from the center of mass and yet stars rotational velocities are essentially constant wrt distance from the center of the galaxy. "Normal" "gravity" doesn't work this way.
To summarize these 2 cases are sort of the extreme.
- First, observation-Y may still be able to be explained in the theory, we just don't know yet. (A type-1 observation)
- Second, observation-Y is contrary to what Theory-X would predict, so we gotta revise our theories. (a type-2 observation)
However, there also are lots of a 3rd type of observation and that one may be worth discussing.
Specifically there does seem to be a slow accumulation of observations of the first type that at some point seem to tip the balance in favor of a new theory. The shift from steady-state universe to big-bang may be in this category and the "discovery" of
glacial lake missoula may also fit.
I think that what happens here is that a slow accumulation of observations of type-1 (above) build up. These observations are not "random" but instead begin to paint a coherent picture of a different theory, then someone discovers or invents that theory and furthermore predicts a few more observations that will support it. When those observations are made the old theory dies and the new one is confirmed.
Glacial Lake Missoula is interesting. There are some geological features in Northern Idaho and eastern Washington State that are hard to explain through a gradual erosion process. Some of these features look like they could have been caused by a flood but there was no evidence of a body of water large enough in the area to have caused such a flood. Turns out they were looking for the wrong type of water body. People now believe that at the end of the last ice age there was a big ice dam that held back several cubic miles (IIRC) of melt water and when this dam cracked it flooded the entire region causing these landscape features.
I think that a lot of "crack pot" theorists think they have found observations of type-2 but instead have merely found observations of type-1 instead. Furthermore, even if they have multiple observations of type-1 they fail to be in "type-3" because these multiple type-1 observations do not paint a coherent alternative theory, they're just different random problems with the current theory.