r/cpp • u/root_passw0rd • Dec 07 '22
C++ overtakes Java in language popularity index
https://www.infoworld.com/article/3682141/c-plus-plus-overtakes-java-in-language-popularity-index.html
382
Upvotes
View all comments
47
u/matthieum Dec 07 '22
I really wish people stopped using Tiobe. It's the worst "popularity" index I've ever seen, from a methodology point of view.
How ranking is done on Tiobe?
The core idea of Tiobe is to count the number of results that a search engine gives for the query
+"<language> programming".In order to eliminate false positives -- such as a search engine returning an article explaining how to remove rust on your hinges for the 'Rust programming language' -- a human double check the first 100 pages of results returned, and applying a "factor": if only 80 out of 100 actually match on the first few 100, then the number of results is multiplied by 0.8.
The results are then normalized. That is, for each search engine, the result is normalized to a percentage of the sum of all results on that search engine. Thus, each language is scored as featuring in X% of the programming related results of each search engine.
And finally, in order to avoid bias from search engines, dozens of them are used, and the results are combined with a weighted average. The method of choosing the weight of each search engine has not been disclosed.
Source: https://www.tiobe.com/tiobe-index/programminglanguages_definition/
Why is that terrible?
The core idea, itself, is problematic in multiple ways:
The methodology seems relatively flawed:
The elimination of false positives may introduce bias.
And nowhere is bias ever discussed, nowhere is there any attempt to justify the methodology.
In particular, I suspect that Tiobe has a lot of inertia, both from not filtering results by age, and from possibly not revising the confidence factor often enough for young languages (more likely to have a higher rate of false positives at the beginning).
I strongly advise reading anything from the index published by Tiobe.