When dealing with a noninertial reference frame, one must find a way to convert it to an inertial reference frame so we can use Newton's laws, because Newton's laws are only valid in an inertial frame. There are two methods of accomplishing this. One is by using pseudoforces. The other is searching for another reference frame for which the motion is inertial.
As we have seen, including the existence of pseudoforces in a noninertial frame can convert it entirely into an inertial one. We are now able to use the laws of physics to make accurate predictions about our formerly noninertial reference frame.
The second method, finding another reference frame for which motion is inertial, is the method I believe you are trying to advocate. I think you are aware how it works. For circular motion, you simply shift to a non-rotating frame and you can explain the motions such as a ball flying off in a straight line quite easily. Gravitation is a little more tricky. You shift to a reference frame in non-Euclidian space such that the motion we observe, such as orbits of planets, is inertial. This is just general relativity
Both of these methods of transforming a noninertial frame into an inertial one are equally valid. One method may be simpler and more enlightening than the other, such as the case with gravitation, but both methods are equally capable of accurate predictions. So which method is superior? Which method has more right to "existence"? A pseudoforce no more exists than the space-time manifold of general relativity. They are both just constructs of our minds to explain the motions we observe.