I managed to find a link with the attendance figures from the 2009 British GP.
The diehard fans number around 85,000 as they are the ones who come for every session. An extra 20,000 turn up for qualifying with a further 15,000 who come for the race only. Compare that to the race day figures and if Saturday was practise as well the number of fans attending would be around 35,000 lower.
35,000 people can buy a lot of merchandise and food and drink at a GP, not to mention the loss in ticket prices. As most circuits are running at a loss already an extra 30% drop in revenue will make the difference between wanting to host a GP or not.
I chose Silverstone as it has one of the highest numbers of weekend attendance for a GP, a less popular GP would have an even greater number between those who come just for the race and those who attend the whole weekend.
Your figures are flawed. How do you know that Friday's crowd comes back on Saturday and Sunday? In the past I've had tickets for qual but not the race or practice and in the past i've had tickets for the race but not practice or qual. I've never been to practice. I try and watch practice on telly if work allows, i try and watch qual but generally recorded as the wife and daughter reckon shopping to be more important, if I can't watch any race live I'm grumpy, if i ever miss a race completely then i need to have an exclusion zone. Do you consider me a casual viewer? it would seem your 85,000 doesn't include me so i suppose I am. Maybe if i wore a starched collar, frock coat and knickerbockers i would appear less casual?