Meeting the Tests

We have to deal with people as they are, not as we would like them to be or think they should be. Many of our failures, as individuals and collectively, result from applying normative principles to real individuals. We make tests for people of median intelligence & ability even though we know that half of all Americans are below median intelligence or ability. Consider something as simple as a driving test. 27% of American female drivers and 13.6% of men could not pass the driving exam if they had to take it today.

This made me think of tests & standards in general. There are two kinds of useless standards; those that everybody can pass and those that nobody can pass. The essence of a test is to differentiate among people. If it doesn’t do that, why have it at all? Olympic swimmer Mark Phelps is faster in the pool than I am. Yet if we have a test that involves only swimming across a small pool in less than a minute, we are equally good; and if we have a test that requires us to swim from California to Hawaii, we are equally bad. Those tests do not make distinctions. Unfortunately, tests that actually differentiate (or discriminate) among people are almost universally unpopular, since they usually reveal something as it is, and not as we might think it should be. Even the simple driver’s test, for example, is very un-PC and you would probably be in violation of some Federal rules if you applied the results.

Of course, most tests are proxies for something else that we cannot test directly. We stipulate that a passing score on the test above means you are a better driver, but that is not always true. People can have abilities and still not use them right. We have to guard against an over-reliance on tests and the credentials they produce. Whenever possible, the test of something should be doing that thing or something very much like it.

I call this the “Gold’s Gym Test.” Lots of people talk about how much they go to the gym and they have all sorts of complicated workout programs. If you judge by time spent at the gym or time talking about time spent, however, you would not find the strongest guys. The simple test is asking somebody to pick up the weights. If Mr. A can pick up more than Mr. B, he is stronger. Time spent at the gym doesn’t matter for the measurement. Lots of people hang around the gym w/o doing much and some people are just naturally weaker than others. In either case, ostensible effort doesn’t really count. This is another reason why some people dislike effective tests, BTW. The test of actual ability will produce results different from those from social or political processes. In other words, the geek or the jerk might be a lot better at something than the popular people who look the part.

Generally speaking, there are “threshold tests” and those that provide additional differentiation. The threshold test is a pass-fail affair. You are either qualified or not. Additional points above the qualifying point do not matter and being close to passing doesn’t matter either. It is summed up in an old joke about the guy who thought he could jump over the chasm … in two hops. The differentiating test will provide a ranking. When we think of tests, we are actually usually thinking of a combination, with some cut off. This also creates confusion.

Lots of lawsuits are filed by “qualified applicants” who didn’t get the job. There may be many qualified applicants, but only one best qualified and if there are ten qualified applicants, usually nine of them feel unfairly treated. It is easy to think of reasons for bias these days. Most Americans are “protected” for some reason, such as age, gender, race, disability etc. Still, there is no way around this judgment. You probably cannot devise a system that would produce one and only one qualified person. If you did that, you certainly would be accused of bias or favoritism and you probably would be guilty of it. Making choices is hard and to some extent arbitrary.

I don’t have a solution and I do not believe a sustainable solution is possible. Judgments must be made using a combination of supposedly objective criteria and subjective ones. We almost always have to use a mix of threshold criteria and differentiation. Holding people “accountable” goes only so far. Clever people can think of ways to manipulate any system. So is there anything that can be done?

Yes, actually. It is an old and very human solution that has been producing good, if not perfect results since the dawn of history. You have to align the interests of the decision maker with the needs of the task. When people are making decisions that will affect them directly and that they will have to pay for in terms of money, time, trouble or some combination, they try very hard to make good decisions based on multiple useful factors. There is no incentive to cheat if you are going to be stuck with the result. The consequences must flow logically from the decision. This is hard in a big business or governmental bureaucracy, where bad performers can hide among procedures and rules, knowing that if they follow the procedures, the practical result doesn’t matter … to them or their career.

That is why I advocate flatter structures, where those making decisions are near the things, physically or otherwise, they are deciding about. Rules should set goals and parameters, but not means to achieve the desired results, since conditions will constantly change and nobody can make rules that will effectively address all the possible situations. It is why I hate to make a decision until I have actually put my eyes on the situation, if possible touched it or at least talk to the people who have. It is also how I know there I many things I - or anybody not closely involved - just cannot understand. Let people make their own choices & enjoy or suffer the consequences.

The farther you get from actually having to do something yourself, the more the situation seems amenable to central planning or fine sounding theories, the more it seems that what you think SHOULD be done can be done. Unfortunately, reality never cooperates. People are complex and imperfect. We have to deal with them as they are, not as we would like them to be or think they should be. If we recognize the truth, we might even be able to make things better some times, although never perfect.

Posted by Christine & John at May 27, 2011 8:47 PM
Comment #323629

Good article. Flatter organizational structures make a lot of sense. They provide better accountability, more effective “hands on” decision making and reduce the costs of redundant multi-layered bureaucracies. There is one problem. The political actors love a hierarchical structure where they can ply their trade: acquisition of power for powers sake.

Posted by: Rich at May 27, 2011 9:57 PM
Comment #323630


It is political with a small “p”. All organizations tend towards rule making and hierarchy. It must be constantly resisted.

As Ronald Reagan said, “No government ever voluntarily reduces itself in size. Government programs, once launched, never disappear. Actually, a government bureau is the nearest thing to eternal life we’ll ever see on this earth!”

Posted by: C&J at May 27, 2011 10:18 PM
Comment #323632

“It is political with a small “p”.”

C&J, I don’t disagree. However, I wouldn’t restrict the problem to government. All large organizations suffer from the same problem. The art of politics trumps competence every time.

Posted by: Rich at May 27, 2011 10:38 PM
Comment #323634

Flatter organizational structures? It’s not that easy. Flat organizations overwhelm managers with excessive span of control. In very general terms, a ratio of 12:1 is ideal; for some strange reason, people function best in groups of that size. A ratio of 16:1 is about as large as practical.

Time and again, we see organizations fail due to excessively flat structures. It’s especially tempting and attractive for a publicly owned company to flatten- after all, when the only concern is profit for the next few quarters, anything beyond that horizon is the next manager’s problem. The problems inherent in flattening- the lack of direction, interaction between organizational layers, and communication between tiers- result from flattening because the manager simply cannot deal with the wide span of control.

Classroom student/teacher ratios provide a classic example of the perils of flattening.

In a political context, we see an excessively flat national structure due to the sheer size of the electorate. The House is large, yet the span of control for each representative v district is so unwieldy that even the supposedly most representative wing of our government fails to adequately represent ‘we the people.’ Instead, like the earlier mentioned example of a publicly owned company, the House is forced by the excessively flat structure to sell to the highest bidder. What else can they do? The existence of our corporate driven government is no coincidence- it is a direct result of the alientation of ‘we the people’ from our representatives.

Posted by: phx8 at May 27, 2011 11:03 PM
Comment #323636


I agree. That is what I meant. All large organizations develop politics. Small ones do too, but it cannot thrive.


Re flatter organizations - We need to allow more autonomy to the doers. I think that one of the threats not only to our liberty but also to our prosperity is the bureaucratization of society. We try to make rules to create fairness and stability, but our rules work against both these things.

It is possible to be very disciplined and still have a working organization. I admired the USMC in Iraq. Their organization was both hierarchical and flat. The 19-year-old corporal has significant autonomy in his area of responsibility, within a broad strategy.

Re government - it is better if it stays flatter by staying out of the people’s stuff as much as possible. Government should not be in the business of overall management. It should merely create conditions under which people can make decisions and prosper.

Posted by: C&J at May 27, 2011 11:56 PM
Comment #323638


I don’t disagree with your point that organizations can get too flat with the day to day operational span of control overwhelming. That issue, however, needs to be separated from the issue of decision making and authority at the operational level. C&J provide an excellent example of the approach taken by the Marines in providing its frontline officers and its non-coms sufficient authority and latitude to react to the conditions on the ground while maintaining discipline within the overall strategy.

Your example of the classroom is a good one to illustrate both points. Too large a teacher/student ratio and chaos will result. But even an ideal ratio can fail if the teacher’s authority and autonomy in the classroom is compromised by rigid lesson plans and required adherence to a teaching philosophy dictated by a bureaucracy well removed from the reality of the classroom.

Posted by: Rich at May 28, 2011 6:45 AM
Comment #323641

We can look at this problem in terms of our philosophical hopes, or we can deal with it in practical terms.

The hopes of some in education is to reduce education down to a simple set of numbers by which you can objectively measure performance. But neither the process of learning, nor many of the subjects it’s necessary to learn can be totally reduced in that manner, or even reduced at all in certain cases. It seems like an advanced way to run an education system, until you realize that in most advanced arts and sciences, the most recent discoveries have called into question the old way of thinking about things.

Take the weather and climate for example. In the old days, the belief was that if you got the calculations advanced enough, if you got the Scientific principle down pat enough, you could predict it with arbitrary precision. Many contrarians on Global warming use that image of arbitrary precision being the gold standard to try and debunk current climate science.

Only, current climatology and meteorology tells us that uncertainty is inherent in the system, and in our ability to predict things. Atmospheric physics shows us that while we can make short term or ranged solutions far in advanced, and have them work, the minute we try to predict long term weather, or produce exact figures for climate forecasts, the adherence of those forecasts to reality becomes ever weaker, the farther you go along, the more you try to nail it down. These are systems built on feedbacks, feedbackst that produce dramatically different results, depending on different inputs.

If you were to model the system with any one input alone, you couldn’t see this effect. It’s only when all these different factors add together, that you see the behavior emerge from the system.

Similarly, if you try to reduce learning to just rote memorization and test preparation, you might get better performance on the tests, but better performance on tests isn’t necessarily the only factor in whether students learn, much less succeed.

As far as hierarchies go? An organization, in my opinion, is no stronger than it’s ability to coordinate information, and it’s response to information. Let me make the radical suggestion that rather than just assume that one kind of organizational pattern is right for all, that there might be different models that suit different industries and sectors better than others.

Posted by: Stephen Daugherty at May 28, 2011 11:36 AM
Comment #323642


You are right about making the response appropriate to the conditions and the organization too.

A machine bureaucracy works extraordinarily well when conditions are stable and predictable. Hierarchies work in similar conditions.

Large organizations, including government, must work with bureaucracies. That is why they are not appropriate for all situations, i.e. those not amenable to bureaucratic solutions.

Posted by: C&J at May 28, 2011 1:05 PM
Comment #323643

I was taught that school gave you tools to teach you how to learn. It was up to the individual to learn the specifics of whatever trade or career he or she desired, be it thru college, a trade school or life experences.

Posted by: tdobson at May 28, 2011 1:12 PM
Post a comment