It's hard for humans to evaluate data from two different perspectives at once. For example, let's take another look at my old reliable data source, men's 100 meter sprint records.
Poking around at the Wikipedia page on National Records in Athletics, I come up with these estimates of racial records (I am no doubt missing some):
West African: 9.58 (Jamaica)
Southwest African: 9.86 (Namibia)
South East Africa: 9.89 (Zimbabwe)
European: 9.92 (France)
Australia Aborigine-White Mix: 9.93
East Asian: 10.00 (Japan)
East African: 10.26 (Kenya)
Pacific Islander: 10.26 (Fiji)
South Asian: 10.30 (India)
Papuan: 10.40 (Papua New Guinea)
As of ten days ago (it's currently prime time for track and field, so this probably has changed), 81 men had run under 10.00 seconds in the 100m dash, and all but two were black. So, that's a big difference.
On the other hand, consider the difference between the best time of Usain Bolt of Jamaica (the world record of 9.58 seconds) and the two Indians who have run India's national record time. Now, India is not a very sports-oriented society, but the difference is still only 0.72 seconds, or well under 10%. So, compared to, say, tortoises and hares, human racial groups are pretty similar at sprinting.
On the other other hand, these racial differences really do make a difference in the real world.
It's hard to keep all of that straight in one's head simultaneously, so most people don't. It's easier just to assume that a person who understands things you don't must be evil.