We are all born with no knowledge of gods and therefore we are all atheist by default - we have no belief in gods when we are born.
I've always found this particular fallacy irritating. We are not atheists by default any more than we're mentally challenged by default.

When we are born we do not believe in god. Do you disagree?
As I alluded, no more than we are retarded when we are born. At that point we are a clean slate, we don't not believe in God anymore than we do believe in God. So when we are born, as Parsifal indicated, we are purely agnostic.
But I think this is a selective definition of the word "default". Evidence seems to support the assertion that early humans, once they started questioning the world around them, attributed existence to a supernatural agent. Such an assumption comes before the scientific understanding that leads some to become atheists. So I think it's more appropriate to say that we are deists by default, once we start questioning the world around us.
I'm really tired so I may not have expressed that as well as I intended. But yeah, I think that since the human mind, lacking the appropriate evidence, defaults to a supernatural explanation for existence, it's more appropriate to say that we are deists by default than that we are atheists by default.