That's no proof, I don't remember what I did last week.. But seriously, we would only have memories of one of those time lines, the one we lived in.
Your assuming other time lines exist, or order to not remember them.
Occam's razor suggests it is more likely one time line exists, because the effect is the same, and one time line is simpler.
When it comes to Occam's Razor, don't take shaving lessons from Tom Bishop.
The universal top speed of the speed of light, time dilation, gravitational bending of space, relative frames - these are very counter intuitive concepts but the math supports them
and they have lead to successful predictions of astronomical phenomena.
Then we add quantum mechanics into the mix, wave/particle duality - the double slot experiment defies all logic. As of right now, the concept of parallel timelines is actually taken seriously in physics. It won't have a huge impact on your life in all likelihood as you cannot interact with them in any way, but they may very well exist.
Also, your memories are based on patterns of data stored in your brain that have been built up in this timeline - so not remembering them is of course
expected, not a
complicating factor.