The Bill of Rights addressed many of the concerns that the anti federalists had, but not all. In reality the signing of the Constitution should be known as "The Great Compromise".
The Federalists (which is a misnomer, they were actually nationalists) supported a strong national government, with all government powers originating at the national level, then the national government would give some of that power to the state governments, who would give some to the local governments, etc.
The Anti-Federalists (also a misnomer, since they actually were federalists) supported a federal government, with all power originating with the people, who give up some power to the local governments, who in turn give some to the state governments, who in turn give some to the national government.
What we ended up getting in the end was mostly a federalist document, where powers are left with the people, and the states, and the federal government is limited. There were several nationalist compromises though, the federal government is solely responsible for regulating, printing, and coining money, for example. States are also not allowed to make treaties with foreign nations, that can only be done at the Federal level.
Madison was against enumerated rights at first, but not for the reason that they wanted rights restricted. His reasoning was that because of the doctrine of enumeration (that is, if the specific power is not enumerated in the Constitution, then the Federal Government does not have it) there was no specific power given to the Federal government to limit speech, religion, property, etc. in article 1 section 8. His fear was if they enumerated rights in the Constitution, it would imply that the government was free to limit other rights not enumerated, and to expand on the powers enumerated. For this he proposed the 9th amendment.