This is a good point, one which I can answer. In the first year when I was in the 6th form (nobody drove a car at this point, or had racing experience) there was a massive spread of lap times and racecraft abilities. Myself and one of my mates were consistently 3 seconds a lap faster than even the quickest of the rest, even though we all had the same experience prior to it. When we got out of the car, the head marshal said to us "You boys have got raw talent, you should do this more often."
Talent is clearly important, but you can get there through lots of practice and dedication, even if `the gift` isn't there. Just takes a little longer. That's why I think Lewis Hamilton is clearly one of the gifted ones, as he made such an impact in his first year in F1
It is so tricky to rule out other factors though. As an F1 fan you will probably have a better idea about racing lines etc. and probably have taken more of an interest in cars in general than the others you went with. Also, perhaps you were more confident about driving a go-kart for the first time and so willing to push harder.
It's just so difficult to separate some sort of "racing gene" from life experiences, even if they aren't obviously applicable to driving.