What's wrong with gravity is that no one can fully explain where it comes from or what it really is. You say that it's a force that's caused by an object's mass? Why does mass do that? OK, mass somehow bends space and time and we feel the effect as a force? What does that happen?
Never mind that the motions of the planets, moon, stars, etc. can all be explained if you allow for a force that is proportional to the mass and inversely proportional to the square of the distance. Even though that "works", and also happens to explain why planets are round and things fall on Earth, you can't "see" gravity, so therefore it is made up.
Of course you can do the same thing with magnetism, electricity, anything really. At some point you have to realize this is the way things behave, and these are the laws of that behaviour.
The fact that we can use these theories of gravity, electromagnetism, etc. to completely explain and accurately predict phenomena make them very robust. You can deny they have any basis in reality, but only if you continually ask "why?" until you fail to get an answer, and then conclude that because you can't get an answer at that point that the whole thing is "made up".
So what's "wrong" with gravity? Only that at the bottom of it, it's utterly mysterious, like everything else at that level. Oh well. Some of us like mystery, and some of us only like easy answers.