Member-only story
Higher Education
Universities Reporting on ‘Complaints’ are Largely Bullshit. There’s a Much Deeper Issue.
Finding what is ‘not there’ compared against ‘what is there’ — is a key problem in statistics — and many other social issues and opportunities. It’s also why university reporting on Complaints are often: Bullshit.
The story of statistician Abraham Wald is told in Jordan Ellenberg’s 2014 book How not to be wrong: The power of mathematical thinking. With an excerpt here on Medium as well: Abraham Wald and the Missing Bullet Holes.
Wald was a Jewish man living in Austria as the Germans took power in the 1930s. He left to the United States and became part of the classifed program — The Statistical Research Group (SRG).
The SRG was a collection of American statisticians working on the war effort.
The story goes, that the military came to the SRG with a problem related to putting armour on planes. Armour is heavy, and so it must be strategically placed and thought out.
The data the military provided focussed on the location of bullet holes in bombers and planes returning from runs over Europe and specifically Germany.
Damage on the planes was not uniformly distributed. There were more bullet holes per sq. foot on the fuselage; the main body of the planes. Not as many on the engines and wings.
The military officers believed that if more armour was concentrated on the locations where the planes were getting hit the most, that would provide better protection.
Wald thought about this, and said the opposite. Don’t put more armour on the body of the planes, put more on the wings.
Why?
Because of what was missing, not what was showing.
The thing that was missing was bullet holes.
Not the missing bullet holes on the planes that returned, but the bullet holes on the missing planes that did not return.
Based on the data that Wald had — bullet holes on returning planes — less on the engines — meant that most likely the planes that did not return had more hits on the wings and engines. And, that’s why they went down and did not return.
