Saturday, May 21, 2016

Judgment Aggregation Theory & Why Education Forces Multi-level Alignment

The Alberta Teacher's Association is having its annual convention this weekend.  A post by the always insightful Phil McRae caught my eye

which ties in the union/profession's official comments,

with a bit more context by the union,

So what I'm going to do today is explain why these types of dynamics happen.  I'll do this from judgment aggregation theory (List & Pettit's formulation).   I'll then reference this back to Argyris' old theory-in-action work (which I'm sad never caught on as well as it would now - the theoretical support tools just weren't there when he wrote it).

The basic tenet behind judgment aggregation theory is that individual judgments when aggregated on a multi-step issue can create paradoxical results.

For example, in a court case you may need to evaluate two separate premises before reaching a conclusion. Each position may be based on "majority wins".  In Kornhauser & Sager's (1993) classic example below, both premises are true, yet the aggregate decision is false.  This is the paradox.  Do you evaluate down each premise then aggregate across (a true/guilty conclusion)? Or do you evaluate across each individual conclusion then aggregate down (a false/innocent conclusion)?

See Pettit's paper for a deeper technical presentation using formal logic conventions.


List & Pettit (2011) have taken judgment aggregation further by endogenizing group agency.  This is REALLY good work.  Here's a quote:
Individuals may form a group agent in virtue of evolutionary selection or cleverly designed incentives to act as required for group agency, perhaps within independent cells.  here the individuals contribute to the group agent's performance, but do not explicitly authorize the group agent, the need not even be aware of its existence. (p. 36)
One of the challenges is how to make a Turing machine aggregator "rational".  They take a fairly generous approach here simply saying that the aggregate needs to :
  1. respond to its surroundings (be rational & responsive), and
  2. rationalize its positions (beliefs & desires)
They also add in the need for binary attitudes (but that is more to tie up details associated with a rigorous formal logical approach).

On a whole, this boils down to something like the Turing test.  The aggregator is rational if people ascribe intentionality to it.

While their work has tonnes of depth and fascinating detail,  I'll resist tangent-temptations and pull another quote.
The supervenient relation between the members' attitudes and actions and those of the group can be so complex that the group agent may sometimes think or do something that few, if any, members individually support.  And secondly, the supervenient of the group's attitudes and actions on those of its members is entirely consistent with some individuals being systematically overruled on issues that matter to them a lot.  Accordingly, the kind of control that individual members must be able to exercise in order to enjoy effective protection has to be stronger.  It is not enough for individuals to be able to contribute to what the group thinks or does on the issues that particularly matter to them.  They must be individually decisive on those issues.  Roughly speaking, we say that an individual is 'decisive' on a particular issue if he or she is able to determine fully -  and not merely in conjunction with other individuals - how this issue is to be settled. (p. 130)

This leads to a very interesting (and rigorously argued) conclusion: group agents 'control' individual actors via attitude biasing.  This is in stark contrast to usual definitions of control which assume the application or threat of application of formal power.  Here's another quote on this:
Since our definition of control has taken the group's attitudes, rather than its actions themselves, to be the targets of the individuals' control - the reason being that the group's actions are usually mediated by its attitudes - we can focus, once more on the part of the organizational structure that is easiest to model theoretically: its underlying aggregation function. (p. 136)
As mentioned the aggregation function is based in attitude biasing.  More specially it is a type of espoused, and complexly aggregated, morality.  Here are a few more quotes to take us to the end.
To be a person is to have the capacity to perform as a person.  An to perform as a person is to be party to a system of accepted conventions, such as a system of law, under which one contracts obligations to others... In particular it is to be a knowledgeable and competent party to such a system of obligations.  One knows what is owed to one, and what one owes to others, and one is able and willing to pay one's debts or to recognize that censure and sanction are reasonable in cases of failure... But non-persons cannot be moved by being made aware of obligations they owe to others. (p. 173)
To be sure, group agents are not flesh-and-blood persons.  They are pachydermic and inflexible in various ways, and lack the perceptions and emotions of human individuals.  But they nonetheless have the basic prerequisites of personhood.  Not only do they form and enact a single mind, displaying beliefs and acting on their basis.  They can speak for that mind in a way that enables them to function within the space of mutual recognized obligations. (p.176)

Viewing institutionalized education as a group agent is fairly easy.  Education operates on multiple organizational levels just as judgment aggregation and judgment aggregator agents do.  What we want to find out is how this informs the conflict coming to a head in some parts of the Alberta educational system?

The conflict is largely between different philosophical approaches and the relative benefit of secondary feed-back effects on students, teachers, and the system itself.  The actors are:
  • the superintendent / government class who increasingly appear as managerial achievement optimizers, and 
  • the teacher / union class who are perennial individualizing optimizers.
  • Students are the units of analysis (and the ultimate beneficiaries or casualties of the fight).
  • Influenceable teachers who are the unit of leverage (and the source of capital).

Judgment aggregation theory suggests the conflict might be framed as fight for different aggregation methods: do you aggregate down each premise or across the whole?  I suspect technocrats favour the premise based approach.  I suspect social scientists favour conclusion based aggregation.

However, this simplistic view isn't what is of interest (after all, how do you really falsify such sociology?).  Rather what's of real interest in what List & Pettit found in the process of judgment aggregation....

Remember that List & Pettit found that the group agent exerted control by biasing the actions of actors and their judgment aggregation processes via a single (ill constrained) "morality" or "intentionality".  In simpler terms, what the "group" was seen as wanting tended to bias individual actors.

The history of former decisions and actions was essential in anthropomorphizing this intention (see also Atran's Big Gods for some good work on anthropomorphizing Big Brother agents).  The history of individuals' asquiesences was also informative.

The conclusions are quite stark.

Individual premises, no matter how wise they may be, don't control judgment aggregation.  Any single level within education which deviates from past histories and intentionalities, is almost certain to get clawed back.  The larger the system, the less disruptive any single level or agent can be.  The longer the system's history, the less disruptive any single level or agent can be.  The more morally-imbued the system is, the less disruptive any single level or agent can be.

The system doesn't "average" each level's conclusions.

The conclusions suggest that institutionalized education is a behemoth.  Railing against it just burns up capital.  Nonetheless, the system as a whole moves as a net result of endless waves of such expenditures (see Tyack & Cuban's surface wave analogy to ed reform).  However any single level acting on its own is unlikely to be productive.  Rather, the degree of any level's variance to aggregrate intentionality correlates with the degree of pushback it is likely to get.

Incremental change is the purview of morally-infused organizations.  Radical change is not.

Radical change can, however, occur when one part of the system breaks off from the rest.  The result is between-group competition (or multi-level selection type 2 reproduction).

All-in-all, dust ups between unions and senior managers is never productive for students.  Nor is it productive for teachers as a whole, even though individual loci may show gains.  In education, thinking you're right may be noble.  Forcing other levels in the system to act on your nobility is... naive.

Next post - Argyris' espoused vs. theory-in-use perspective.


  1. Nice, Chris - fascinating thoughts and application of List & Pettit.

  2. Thanks Robert. This is basically an extension to network theory approaches to educational change

    I love List & Pettit's work because its one of the few things I've seen which offers a rigorous, falsifiable, interpretation of network structure aggregation. Most of the things I've seen (which admittedly may be limited), tends to make hypotheses about the emergent structures and then look for evidence of plausibility. While informative, it also tend to be an echo chamber for ex post facto justifications.