This is the Kalām Cosmological Argument, revived in recent times by Daniel Lane Craig. It goes like this:
1. Everything that has a beginning to existence has a cause of existence.
2. The Universe has a beginning to its existence.
3. Therefore, the Universe has a cause of its existence.
Note that this presents no problem for the atheist. 1) The argument does not prove that the cause is in fact God, only, if you accept the premises, that there was indeed some cause. The exact nature of the cause is left to the imagination. 2) The premises don't have to be accepted at all. The whole argument relies upon cause and effect, dictated by objects and actions moving through time. If time did not exist before the Universe existed, how can there be cause and effect relationships?
Further, unless God exists indefinitely and infinitely, he is bound by the same logic. It would seem, anyway, that God is affected by cause and effect relationships. After all, people do things, and God gets angry. Cause and effect. So it would seem that God is infinite. So if God creates a finitely existing Universe while he exists infinitely, does that mean God waited forever until he decided to create the Universe? That seems contradictory. Indeed, some of Craig's arguments are based upon true infinities being impossible. Does this same logic apply to God as well?
In conclusion, the Cosmological Argument doesn't prove God, and further its premises are speculative at best.