A comparison between 3 mathematical models

[Spinodontosaurus]

Everybody has their own subjective ranking list of who they believe the greatest F1 driver in history was. In recent years a few mathematical models have been constructed to try and answer the question in an objective manner; Eichenberger and Stadelmann (2009), Phillips (2014) and most recently Bell et al. (2016).
Phillips runs the F1Metrics blog and has posted several statistical analyses on it over the last few years, and he has recently posted an article comparing the 3 mathematical models to each other, and to trends seen in subjective lists.

Experts versus models: How do we rank drivers?

Included in the article is a sort of 'consensus' list between the 3 models. The top 5 drivers from the consensus list are posted below, but I think the article is worth a read regardless as this isn't even the main focus of it.

1. Fangio
2. Clark
3. Schumacher
4. Stewart
5. Alonso

Note: I didn't know whether to post this in F1 Discussion or Statistics and Analysis. I settled on the former because when Bell et al. (2016) was first published earlier this year the thread about it was posted in this section, but if this thread needs to be moved then my bad.

 

[Greenlantern101]

I think the 3 models demonstrate perfectly how totally pointless such comparisons are. If we take just one example Lewis Vs Nico. This year not many would argue that Nico has been second best to Lewis, yet due Lewis's reliability issues Nico is ahead on points. Which by the systems demonstrated makes Nico the better man. Stats never tell the full picture.

Nico rankings with the 2014 method was 7th. Does anyone really believe he is the 7th best driver of all time? He is a decent driver, solid and fast but nowhere near 7th best. The 2016 stats have him as 49th. That's some difference.

Assumption based purely on stats will never give the whole picture. There are simply to many variables between teams, crashes (to blame), crashes (not at fault), breakdowns, pit crew cock ups etc etc the list goes on. Stats ultimately never take into account these things, certainly the 3 methods in that article never take them all into account.

The whole idea is flawed. It's like trying to bring order to any hugely complex chaotic subject, the result is ultimately determined by the parameters. One slight change and you get a whole different result, be it driver rankings, most dangerous animal, or most insightful forum member.

Pointless.

 

[cider_and_toast]

The one thing these models could ever capture is the skew created by an opinion of how the driver made someone feel. That's as important as metrics and data.

Senna doesn't make that top five (which I find absurd by the way) but back in the day he was my hero. In that black and gold Lotus he was superb. I'd take just Senna's big toe over a thousand Alonsos. A driver who leaves me cold and completely unable to see what apparently every expert can.

 

[Speshal]

There are 2 types of people in this World.

1. Those that can extrapolate from incomplete data.

 

[TR]

Assumption based purely on stats will never give the whole picture. There are simply to many variables between teams, crashes (to blame), crashes (not at fault), breakdowns, pit crew cock ups etc etc the list goes on. Stats ultimately never take into account these things, certainly the 3 methods in that article never take them all into account.

On the other hand, any ranking based on "expert" opinions will always be colored by bias. It is therefore interesting to compare the lists produced by experts with lists based purely on objective data. Both are imperfect, and comparing highlights where each has short comings. For example, one aspect highlighted by this comparison, is that drivers that died during the peak of their career (like Senna or G Villeneuve) or probably somewhat overrated by experts due to the resulting mythology about their person.

The overrating of Rosberg in the "objective" metrics, on the other hand highlights one of the main shortcomings of all three models: not accounting for the change in driver skill over time. (Rosberg is highly rate in some of the models, because he did well against Schumacher at the end of his career.)

The whole idea is flawed. It's like trying to bring order to any hugely complex chaotic subject, the result is ultimately determined by the parameters. One slight change and you get a whole different result, be it driver rankings, most dangerous animal, or most insightful forum member.

One thing coming from the comparison is that result of the objective models is actually remarkably robust against varying the parameters.

Pointless.

Yes, ultimately producing any sort of TOP list is pointless. But I think that can safely said of anything posted on this forum .

 

[Greenlantern101]

The only thing that can truly be compared is team mates and even then you have to take the DNF's and Penalty races out of the data.

That's why it's interesting when drivers move teams as it's the only time you see a comparison against non-team mates. Ricciardo, Vettel, Kimi for example.

 

[TR]

The only thing that can truly be compared is team mates and even then you have to take the DNF's and Penalty races out of the data.

That's why it's interesting when drivers move teams as it's the only time you see a comparison against non-team mates. Ricciardo, Vettel, Kimi for example.

You do you realize that that is exactly what some of these models do? Compare team mates against each other in each season the get a set of pairwise comparisons between drivers. This data is then used to build a model ranking of all drivers. The Philips 2014 even goes as far as to remove all mechanical DNFs from the data first.

 

[Rutherford]

I haven't read the article yet and only had a look at the rankings. Niki Lauda as a three time world champion is missing on two of the three lists.

 

[Greenlantern101]

So if a driver is in the lead and has a DNF the data is removed. Thats the point. How do these things get considered in the stats.

There is no perfect way to do it. It's so big a data field, which is very much incomplete. More so the further back you go, it is effectively a chaos system. Trying to bring order to it is pointless.

 

[TR]

That appears to be a side effect of those two models not distinguishing between mechanical DNFs and driver DNFs (counting all as non-scoring finishes). Consequently, the 1985 season makes Lauda look really bad.

 

[TR]

There is no perfect way to do it. It's so big a data field, which is very much incomplete. More so the further back you go, it is effectively a chaos system. Trying to bring order to it is pointless.

The system is not nearly as chaotic as you make it out to be. (As is demonstrated by different models producing fairly similar results.)

 

[Greenlantern101]

I just don't see the point in trying through 'models' to compare the likes of Fangio who started F1 racing aged 39, retired age 47 with todays todays young guns. If you brought him back in a time machine they would blow him out the water. This is despite Fangio being a mile clear in stats of wins per race and championships per year.

Compairing radically different eras is pointless, the similarity between the 1950's and 2010's are almost zero, they are effectively different sports now. Yet these stat models jam them together in an idiotic attempt at the impossible. You might as well be jaming Boxing and Tennis together for a who's the greatest stat.

Can't we just have an appreciation for the history and watch the current sport and enjoy todays racing heros for what they are.

 

[Olivier]

I'm going to reserve my opinion until I have read the whole article, but the picture at the top of the article is great. I'd really like to hear what Mansell has just said to De Angelis. The Italian seems to be bothered by the topic (or the conversation) ...

 

[cider_and_toast]

We had a similar conversation about an ex Bristol City player on our football forum.

Dariusz "Jacki" Dziekanowski played 43 games for Bristol City in 18 months scoring 7 goals. Any statistical analysis would never place him in the top 10 greatest players to have played for us.

However, you ask anyone who saw him play and he would easily make the top 2 or 3, if not number 1 as the greatest player to ever pull on a City shirt. Legendary commentator John Motson talking about the level of Jacki's skill compared to the rest of the players on the pitch in an FA Cup match exclaimed "he's not just from another country, the Pole, he's from another planet"

Jacki could be lazy, go missing in games when the tean were a goal down a needed a lift but he could do stuff with a football that most players could only dream of. He was cheeky, cocky, handsome and an absolute fan favourite.

To switch this back to F1, no metric will ever capture that. Some may argue that why should it? I would suggest its an essential part of what makes a great driver. Prost was arguably a better driver than Senna but did he make people's hearts race as much as Senna? Of course not but then Prost knew how to drive within himself and would have never taken the risks Senna did at Monaco. That's why Prost had more success.

It's an interesting exercise but it should help inform the debate not be seen as the definitive answer.

 

[Spinodontosaurus]

I think the 3 models demonstrate perfectly how totally pointless such comparisons are. If we take just one example Lewis Vs Nico. This year not many would argue that Nico has been second best to Lewis, yet due Lewis's reliability issues Nico is ahead on points. Which by the systems demonstrated makes Nico the better man. Stats never tell the full picture.

When there are what appears to be glaring problems in the models' placement of a particular driver we can examine the model and understand why the rankings are how they are. They are objective. This is not possible with any subjective list.

For instance, we know exactly why Nico Rosberg is rated so highly by the Phillips model. He faced Schumacher when the latter was miles past his peak, and since the model doesn't account for changes in performance over a career Rosberg benefits hugely from this pairing. On the other hand, we have no way of knowing why Mansell is consistently rated much more highly than de Angelis by experts. Or why Heidfeld is consistently missing from expert lists despite beating every team mate he had besides Frentzen, including numerous very strong drivers (Alesi, Raikkonen, Massa, Webber, Villeneueve, Kubica).

In the example you gave yes, Rosberg would be ranked ahead of Hamilton so far this year. But we know why that is. Besides, that might not even be the case at the end of the season. For example if Rosberg scores 2 wins and 3 second places in the final 5 races, and Hamilton the opposite (3 wins, 2 seconds) then Phillip's model will rate Hamilton as the stronger driver, since he will be fractionally ahead on a points per race basis despite Rosberg leading the actual points tally by 16 points.

 

[cider_and_toast]

I don't know which experts fail to rate Elio above Mansell but whoever those experts are I want them sacked.



As I keep saying, emotion and attitude play a huge part of a drivers make up. It won't show up on your statistics but it is important when talking about great drivers.

Up until 1985 Elio had comfortably beaten Mansell in 2 out their 3 seasons together and beat Mario in both of their seasons. In 85 Senna, who arrived with joint number 1 status comfortably outscored Elio. The Team manager of Lotus at the time, Peter Warr, said that it wasn't long before Elio's motivation went. That doesn't mean he was any less of a driver just that he didn't display it.

That will never be picked up by a metric.

 

[The Pits]

There is also a strong confirmation bias on these kind of things. things don't look right if you believe differently to the outcome of the numbers. I am not saying that any model is perfect, but that the reason that we do not think a model is correct is largely because it does not align with our own views, so we look to find why.

 

[KekeTheKing]

Mr Phillips sure has an interesting model

6. Hunt
17. Frentzen
25. Heidfeld

 

[cider_and_toast]

And that tops out the bovine excrement chart for me. I'm sorry, there is not a universe in which James Hunt ranks above Niki Lauda. Not now, not then, not ever.

 

[teabagyokel]

cider_and_toast - Unless Lauda withdraws, of course.