Integrating the dynamics of multi-level economic agency

Don Ross

dross@commerce.uct.ac.za

1. Introduction

Three recent book-length studies in the philosophy of economics (Mirowski 2002, Davis 2003, Ross 2005) have drawn attention to the fact that mainstream economic theory has consistently avoided commitment to any particular model of the person. This is the most significant respect in which economics has kept aloof from part of psychology. The widespread belief, on the other hand, that economists’ attentiveness to the psychology of choice and decision had to wait for the Allais challenge and then for Kahneman and Tversky is a myth. It is true that for a brief period after World War II economists led by Samuelson tried to operationalize choice as analytically derivative from observed consumer demand. This was a minor episode in the history of theory. Ross (2005) argues that, if anything, mainstream microeconomics has been more sensitive to theoretical fashions in the psychology of choice than has been good for it.

This sensitivity is an aspect of economists’ abiding preoccupation with agency. To the extent that agency is grounded in the psychology of decision-making – as the subjective experience of weighing alternatives – then the basis of agency might be taken to be clear. Hence the attraction for Lionel Robbins (1935) of resting economic theory’s foundations on putatively clear and universal knowledge of the experience of ranking subjective preferences. Alternative strategies for demarcating agents from non-agents, which one is forced to consider if one adopts a behavioristic viewpoint, or indeed just a sensible degree of Wittgenstein-style verificationism about the contents of introspection, lead into choppy philosophical waters.

Suppose one holds that agents are just those entities who turn out to be truly (or merely usefully, if one is a pragmatist) describable as consistently maximizing a specifiable function or class of functions over possible consumption bundles. The last four words in this formulation do no independent work, since what counts as a possible consumption bundle for an agent is determined by the function its behavior is taken to maximize. Then the claim that an agent is something that acts to maximize a function is not sufficient for fixing the scope of microeconomic theory, since everything that retains system integrity over detectable stretches of time – planets, rocks in gravitational fields, atomic nuclei – can be modeled in some sort of equilibrium dynamics. To be an economic agent, an entity to which a utility function can be imputed, a system must perform work to track environmental changes in order to try to maintain itself in equilibrium, and there must be non-trivial possibility of failure lest the ‘tracking’ metaphor be empty. The metaphysical ideas underlying this conception are quite vague. Is a system that always loses equilibrating capacity after a time – e.g., every organism – a system that, by eventually failing to track its sources of utility (i.e., dying), thereby reveals its agency? This doesn’t seem helpful, since in a dynamic and finite universe this criterion would again qualify everything as an agent. The most promising way to obtain restrictions is by reference to the core idea of post-Darwinian biology: require the possibility of local failure against a backdrop of system stability by identifying agency with units produced and shaped by some selection dynamics or other. Then organisms and species and firms and clubs are agents but rocks and planets aren’t – or, if this still doesn’t quite work because some sort of pan-selectionism holds in the limit (as in cosmological models taken seriously by physicists such as Smolin 1997), then one can rule (e.g.) evolving galaxies off the table as economic agents by allowing that agency might be scale-relative and then adopting a pragmatic policy of restricting attention to local scales.

Philosophers’ eyes will roll at such slapdash ontology, but they should hardly be surprised that economists aren’t really interested in metaphysics. Almost no scientists are. In general, a science will resort to metaphysical or strictly logical individuation of types when and only when the relevant mathematics can’t be induced to pull all the necessary weight for practical purposes. In their high-neoclassical phase economists thought that mathematical relations borrowed from classical thermodynamics were adequate to the job; these days dynamic-systems theory is widely thought to provide foundations for those who need them – in particular who need them for an economics built on informational asymmetries, market inefficiencies and increasing returns (Anderson et al 1988, Arthur et al 1997, Albin 1998, Blume and Durlauf 2005). The career of the critical historian of economics Philip Mirowski (see his 1989, 2002, this volume) can be summarized as a running campaign for the charge that in no period has economists’ math done as much foundational work for them as they’ve imagined. Scarcely any economists accept Mirowski’s allegation, though what they often offer as grounds for rejecting it is just indignant huffing and puffing (see Binmore 2004, Durlauf 200x).[1] Much of this is a battle over calling the glass half-full or half-empty. Since economists are constantly enlarging their mathematical toolkit there is some sense in which their mathematics at a given time is obviously never sufficient to obviate tacit appeal to metaphysical assumptions in the uncleared patches of the conceptual jungle – we know from induction there’s always some new patch about to be cut. But since it is also true that economists never decide that some mathematical resources they once found significant are in fact irrelevant – the reader is invited to try to nominate even one important case of this – they can argue that, for any given area where concepts are not yet explicit, induction advises patience while we wait for technique to advance into it, rather than dissipation of energy on frontal philosophical assault.

In this chapter I will observe this pragmatic economist’s attitude where the philosophy of agency is concerned. That is, I will assume cavalierly that if we get non-redundant explanation and prediction of some system’s trajectory through successions of states by regarding it as maximizing or meliorating a utility function,[2] then that system is an agent. My interest here is in general questions that persist after we agree to be cheerfully unphilosophical about agency. These questions arise precisely because aphilosophical pragmatism does have a price, even if it’s a bargain. That price with respect to the concept of agency in economics is as follows. According to the pragmatic attitude, there is a wide variety of kinds of economic agents. Most people seem to be agents, or at least successions of them (as their utility functions change). Groups of people seem to be agents. For reasons I have explored in detail elsewhere (Ross 2005), both functional aspects and some neurophysiological parts of people seem to be agents. These various kinds of agents (among others) don’t inhabit separate worlds, so a unified view of natural economic history must depict them as interacting. We can’t rely on another science – say, psychology – to show us how to model their interaction, for in adopting cheerful pragmatism we decide to prefer a notion of agency tailored to specifically economic uses; thus we must not expect psychology to provide foundations for us. This is not a declaration of strict independence: of course we must pay attention to constraints arising from what neighboring sciences tell us is true about the world. (For example: which parts of people are agents, and what motivates these agents, is being discovered by neuroscience; see sections 4 and 5 below.) But the problem of how to apply our metaphysically sloppy but mathematically good-enough idea of agency to empirical processes that interest us as economists is all ours to deal with.

In this chapter I aim to make some progress on a few aspects of this problem, namely: what is the concept of a person in economic theory (if there is any role at all for such a concept)? Asked another way, the question is: what sort of economic agent is a person? And then: what, in general, is the relationship between personal agents and suprapersonal and subpersonal agents, the dynamics of which, I contend, explain why there are personal agents in the first place?

The chapter will be structured as follows. In Part 2 I will set the context for discussion a bit more precisely in current issues from the philosophy of economics. In Part 3 I will briefly give a high-level sketch of an approach to modeling people as production outputs of social dynamics, intended to explain why there are people. I confine myself to a sketch because I have defended this approach in detail and at substantial length elsewhere, and there is only room for so much in the chapter. Following a somewhat more detailed review of current foundational issues in sub-personal economics (picoeconomics and neuroeconomics), I will, in the most substantial part of the chapter, provide an extended example of modeling a behavioral phenomenon from this domain. The example in question will be the prevailing neuroeconomic model of addiction. In showing why it takes a person – and not just a brain – to overcome addiction, we will be provided with an illustration of why there is a role for the concept of the person in economic reasoning, and what sort of role this is. The final section will offer synthetic reflections on the earlier parts.

2. Philosophy and pragmatism

Let us contrast two conceptions of economics which, I would argue, have contested for the soul of the discipline[3] throughout its history. On the first conception, economics is any body of theory or application of a body of theory that generalizes over maximizing, optimizing, or meliorating relationships among (i) utility functions, (ii) scarce production inputs, and (iii) reallocations of (ii) with reference to (i). This is equivalent in spirit to what Alex Rosenberg (this volume) calls a “nightmare” view of economics as consisting in any model or application of linear or dynamic programming – I say ‘in spirit’ because we clearly must add game theory to the technical set, but Rosenberg leaves it out.[4] I think it is clear from Mirowski (2002) that he shares Rosenberg’s attitude to this conception, which we might refer to a bit irreverently as ‘Debreuvian’. What contrasting pleasant dream might people who think Debreuvian economics is a nightmare have on mind? Here is the alternative conception: economics is any body or application of theory that generalizes over the behavior of some specified class of people or their aggregates taking actions to optimize or improve their well being with respect to availability for use of scarce assets. This seems to be the conception embraced by Rosenberg in this volume.

I do not share Rosenberg’s and Mirowski’s attitude to nightmares and dreams. In my opinion, a great deal – the overwhelming majority – of misguided philosophy of economics over the years[5] has consisted precisely in criticism of approaches developed under conception #1 on grounds that they are not motivated by conception #2. By contrast, I maintain, the many useful exercises we find in economics that fit conception #2 are always best understood as special applications of models developed under conception #1.

My basis for this opinion is that part of what crucially makes an inquiry or enterprise scientific – in addition to devoted subservience to the authority of empirical measurement – is concern for systematicity under a formalizable framework that abstracts away from the parochial and mutually inconsistent folk ontologies inherited from human natural and cultural selection and built into the structures of natural languages. Scientific inquirers care about systematicity to the extent that they are care about maximizing theoretical power for explanation and prediction. In economics, this concern is clearly manifest in the strong pressure to organize all historical data into either a linear programming model, a dynamic programming model, a game, or a complex combination of such models.

This point by no means applies only to data about people, but equally to data about institutional agents such as firms and countries. There are also interesting lessons to be learned from recent debates over how best to model putatively economic interactions among non-human animals. There has been a tendency to focus exclusively on game-theoretic approaches to such interactions, for reasons summarized by Bowles and Hammerstein (2003): animal markets (for, e.g., mates) generally aren’t anonymous and aren’t set in institutional contexts with exogenous enforcement that could facilitate complete contracts. On the other hand, as Hammerstein (2003) points out, empirical data on inter-specific mutualism show few cases of the sort of reciprocal altruism that abstract models based on evolutionary game theory often emphasize. The problem here is likely that reciprocity places strong demands on cognitive book-keeping; but production of capacities for this is rather a boutique industry for natural selection, however salient it might be to economists who are members of H. sapiens. Furthermore, as Hammerstein also emphasizes, game-theoretic models often implicitly restrict players’ choices of interaction partners in ways that don’t bind natural interactions. For both these reasons, nature is much stingier than game theorists in generating evolution of sophisticated strategic ploys for meta-exploitation and meta-meta-exploitation (and so on). Now: do economists respond to this situation by relying less on abstract models and more on natural history? Not at all. Bshary and Noë (2003), for example, show how lack of restriction on interaction-partner switching among cleaner fish and their clients on reefs promotes a preponderance of competition over non-parametric cunning. Members of species that cannot travel significant distances between reefs, and thus have reduced choice of cleaner services compared with more pelagic species, are more likely to have their scales nipped by cleaners and are made to wait in longer queues. The markets among these fish appear to be highly efficient, and are elegantly captured by linear programming models.

The point I want to make about these applications is not that anyone can know a priori that a given behavioral phenomenon among groups of animals must either yield to a game-theoretic model or to an aggregated optimization model. My point, instead is that it is the discovery that one sort of model or the other applies usefully to a phenomenon that identifies the phenomenon in question as one partly governed by economics. If neither sort of model often yielded superior predictions and explanations of non-human interactions, then economists would be dis-invited from the councils of ethology. Their domain of study is coextensive with the reach of their distinctive analytical technologies, not with an a priori conception of what constitutes an economic phenomenon. This is of course a sociological point, not a metaphysical one – but so, obviously, is Rosenberg’s rival claim. The advantage of the first conception of economics is that it correctly describes the sociology. And then we can add the point that this sociology is maintained because it works: game-theoretic models of interaction among animals with big brains yield powerful and successful generalizations, as do competitive models of mutualisms and other interactions among less cognitively well-armed organisms. Evolutionary game theory generates systematically useful models of longer-run interactions among lineages.

Defenders of conception #2 might try to reply by arguing that economic modeling of non-human agents is merely an extension of economic methods into another proper field (ethology). But for this reply to be convincing, it would have to be the case that economics provided a superior basis for understanding acquisitive behavior of people than of non-human agents. Otherwise, why would people be the base case and non-human animals secondary extension cases? However, it seems to me that critics such as Sen are partly right about one important claim, namely, that there aren’t in fact many powerful generalizations to be had specifically about people as economic agents (qua economic agents). I say ‘partly’ here because it turns out, as I will explain, that we can derive interesting ideas about the processes that give rise to people from applications of economics under conception #1. But with respect to the negative claim, the most directly relevant literature is that which records the history of devoted experimental efforts to determine whether people in microeconomic settings make choices according to expected utility theory, prospect theory, a host of specialized derivations from the matching law, or a grab-bag of situation-specific heuristics. The best justified generalization to be drawn from this literature at present is: people are highly plastic and they make choices in all of these ways under subtly different circumstances. This is not the kind of conclusion that theorists who take for granted that people, whatever they’re actually like, must be the paradigmatic economic agents want to admit. For them, it is the very point of behavioral economics to determine what properties distinguish these paradigmatic economic agents – and, then, by extension, all possible non-paradigmatic economic agents to at least some extent. For a leading exemple of this assumption at work, see the influential advanced text of Camerer (2003). We may state the assumption as follows: normal adult H. sapiens organisms, regarded as identical to people, are the basic, naturally occurring economic agents.

Elsewhere (Ross 2005, 2006a) I defend an opposed view. According to this perspective, and to similar ones discussed by Clark (1997), Rovane (1998) and Dennett (2003), people are not identical to H. sapiens organisms. The latter are products of genetic evolution. People, on the other hand, are normatively regulated virtual constructs that arise out of complex dynamics operating at multiple interacting levels. These levels include genetic-evolutionary dynamics, but then also cultural-evolutionary dynamics, and information-processing dynamics at micro timescales both within the complex H. sapiens brain and at the level of social signaling. There are powerful, systematic generalizations to be derived from models at each of these levels, some of which can be synthesized using the technologies of economics, viz., linear and dynamic programming and game theory (classical and evolutionary). Economics contributes to our understanding of people by designing, applying and testing such models. From a very abstract philosophical perspective, we can summarize what the synthesis shows us as follows: people are stabilization devices in social-evolutionary dynamics and, simultaneously, in picoeconomic and neuroeconomic dynamics, who are recursively produced as outputs of games.

I cannot try to recapitulate the full argument for or explication of this claim, provided in the work cited above, in this chapter. The next section will summarize its implications. For a critic who finds it novel to the point of unacceptable rashness, let me note that it has precedents of respectable vintage. It is, I suggest, substantially anticipated in the following remark by the early Darwinian psychologist James Mark Baldwin (1913): “The society into which the child is born is … not … merely a loose aggregate, made up of a number of biological individuals. It is rather a body of mental products, an established network of psychical relationships. By this the new person is moulded and shaped … He enters into this network as a new cell in the social tissue … He does not enter it as an individual; on the contrary, he is only an individual when he comes out of it … In the personal self the social is individualized.” It almost surely did not cross Baldwin’s mind that economic modeling tools might help to make this claim more rigorous; before dynamic game theory these tools indeed couldn’t do so, and under the strongly individualist spin attached to them by neoclassical spokesmen such as Jevons and Robbins they in fact encouraged its denial.

A principal conclusion of both Davis (2003) and Ross (2005) is that this was spin, not something built into the literal foundations of neoclassical theory. The very thin concept of agency in mainstream economics, from Jevons and Walras through the present, identifies agents with the gravitational centres of consistent preference fields. The theory incorporates no thesis about which empirical entities in particular implement such roles. Nor does it entail anything about how long their embodiment typically persists. Agents may be as transient as a modeler likes; so although agents may not change their preferences and remain the same agents, people may do so and can simply be modeled as successions of agents. (This requires no denial that these successive agents have some special relationships to one another, including – possibly – economic relationships.) Davis, assuming that people should be the paradigmatic economic agents, puts these points to work in mounting a complaint against mainstream (neoclassical) economic theory. We can all agree with this up to a point: we would have grounds for disquiet if economic theory turned out to have no useful applications to human behavior. However, as Ross (2005) argues, it is a non sequitur to jump to such disquiet from the weaker claim (endorsed in my book) that insects are better exemplars of basic economic agency than people (while people approximate such agency from time to time in something like the way that countries do), or from the claim that economic theory attributes no distinctive properties to people. The main point for now is that the relationship between individual humans and economic agents remains open so far as the modeling technology is concerned.

I have been referring to ‘mainstream’ economic theory. In Ross (2005) I follow most commentators in identifying this with ‘neoclassical’ theory. This terminology is distracting to the extent that it encourages us to focus narrowly on a specific moment in the history of economics (from, typically, the first marginalists up to Hicks’s and Samuelson’s conversion to Keynesianism). In an effort to give substantive distinctions pride of place over primarily historical ones, I will here appeal instead to a comparative matrix offered by Leijonhufvud (2004). He distinguishes what he calls ‘classical’ and ‘modern’ economics as follows:

	Classical	Modern
Objective of theory	Laws of motion of the system	Principles of efficient allocation
Individual motivation	Maximize utility or profit (intent)	Maximize utility or profit (performance)
Individual behavior	Adaptive “Procedural rationality” (often gradient climbing)	Optimizing choice “Substantive rationality”
Behavior and time	Backward-looking, causal	Forward-looking, teleological
Cognitive competence	Capable of learning Well-adapted “locally”	“Unbounded”
Role of institutions	Essential in guiding behavior, making behavior of others predictable	Problematic: Why use money? Why do firms exist?
Equilibrium concept	Constancy (point attractor)	Mutual consistency of plans

The idea behind this sorting is best illustrated in the way Leijonhufvud does, by reference to historical figures who instantiate each type. His examples of ‘classicals’ are Ricardo, Marx, Marshall and Keynes. It is clear, I suggest, that to this we could add Hayek, Schumpeter, most contemporary behavioral and institutional economists, and Sen. The examples of ‘moderns’ are Arrow and Debreu, Lucas, Sargent, and Prescott.

Hicks and Samuelson are cited by Leijonhufvud as “moving back and forth between the two [kinds of economics] as the problems they dealt with would dictate.” Should we read this as meaning that Hicks and Samuelson were opportunists, or merely transition figures? Let us take this not as a question about Hicks and Samuelson per se but about our way of organizing conceptions of economics. Leijonhufvud’s matrix is not intended as an intervention in the history of thought but as an exercise in pedagogy for graduate economics students who don’t know much history or methodology, and it is frankly driven by practicality and hindsight. Its interest to me in the present context derives from the fact that all the standard neoclassicals except Marshall are missing from the list of exemplars. Were nearly all the neoclassicals ‘proto-opportunists’, or was all of neoclassicism a transitional phase between classical and modern economics? This is a false dilemma, since both horns make valid points. Almost all economists in every generation, except those who work mainly to promote philosophical positions (for example, Hayek after the 1930s) have been and are ‘opportunists’ in the sense attributed to Hicks and Samuelson. There are forty economists on full-time faculty in my department at the University of Cape Town. Exactly one consistently fits the profile of a ‘modern’ in all of his work; but there are none who never make use of modern instruments or techniques. So these colleagues of mine join the early neoclassicals in not having clear columns on Leijonhufvud’s matrix. In addition, the early neoclassicals don’t fit because, indeed, they were transitional figures. But they were not transitional in the sense that classical assumptions were perishing in their era – they have not perished to this day. They were merely transitional in the sense that they anticipated important aspects of contemporary economics but didn’t have the analytical tools to be fully ‘modern’.

The point I am getting at here is simply that philosophers and historians of thought, in trying to identify relationships among theoretical assumptions and preferences in economists, readily encourage themselves and their audiences to think of economists as more like philosophers – consistent developers and promoters of intellectual doctrine – then they really are. Economics, like all healthy sciences, is first and foremost a box of tools. Excellence in economics consists mainly of well-trained and disciplined judgment in seeing how to fit tools to problems in ways that will generate explanatory insight.

In this spirit, I will in the remainder of this chapter drop all talk of methodological ‘isms’ (though some philosophical isms not specific to economics will feature here and there). The aim is simply to use items from the economist’s toolbox to shed some light on the nature of people, in a manner different from that pursued by those who think that people manifest, in general, one or another distinctive kind of rationality and quarrel over which kind it is.

3. People as products of cultural macrodynamics

I said above that neighboring sciences will not tell economists how to model people as economic agents. However, it is to other disciplines – psychology, sociology, anthropology, evolutionary ethology and neuroscience – that we must turn to find out what people are in the first place. In Ross (2005) I argue that a crucial insight from the social, cognitive and behavioral sciences is that a person is not identical to a biological organism with the DNA characteristic of H. sapiens. In particular, as Baldwin stresses in the quote from him cited above, such organisms (or most of them) develop into people through processes of socialization and enculturation. The basis of a person’s distinctiveness and coherence is narrative: a person is, fundamentally, an entity that can narrate a history of its dispositions, actions, tastes and motivations that other people will reliably interpret and respond to as a biography. (For similar ideas see Bruner 1992, Hutto 2007). A biography is (roughly) a teleologically structured history of an entity whose behavior and manifest emotional expressions can be non-redundantly predicted and explained using the intentional stance (Dennett 1987) – the perspective that organizes behavioral data against a backdrop of attributed beliefs, desires and similar representational states,[6] the so-called ‘propositional attitudes’. Psychologists study the process by which unenculturated individuals of the human species turn themselves into people under the rubric of ‘the social construction of the self’. I therefore use the concepts of ‘person’ and ‘self’ interchangeably. In my terminology, then, ‘biological individual’ = `pre-enculturated human’ ≠ ‘self’ = ‘person’. The former pair are theoretical constructs for purposes of modeling, rather than kinds of entities we find running around, because human babies begin to be drawn into cultural personhood from the moment they can respond interactively to their parents.

Economic concepts and models can contribute to our understanding of both why and how humans are motivated to narrate selves and to pressure others to do so. As with any social species, humans are disposed by nature to seek gains from trade, including trade in labor and skill-capital contributions to projects that require joint investment in order to be feasible. Achieving efficient equilibria in such games often requires strategic commitment by one or more parties, and stakes in reputations are the most basic commitment devices available among non-kin in typical ecological circumstances. In this context, narrative selves fuse two functions: (1) they provide structure that allows specific reputations to be encoded and remembered using pneumonic devices (natural story plots, as it were) biologically natural to humans; and (2) the coherence requirements that govern them embody strategic commitments – agents know that if they re-narrate themselves too freely in order to allow more strategic flexibility, they risk being regarded as unpredictable (in contemporary English, labelled ‘flighty’ or ‘two-faced’) and to be excluded from potentially profitable relationships and projects. Supplementing these considerations is the fact that most games that are not strictly competitive have multiple equilibria. Relatively fine constraints on stability of selves (e.g., “She’s the sort of person who puts more value in an exciting new experience than in security”) facilitate joint equilibrium selection. Put another way: selves are sources of focal points in coordination games.

I noted in the previous section that the concept of agency incorporated into the formalism of mainstream economic theory encourages us to map agents onto transient stages of personal lives rather than onto whole personal biographies. Far from being the basis for an objection to mainstream theory (as in Davis 2003), this fact provides a natural way of factoring two (typical) standing empirical conditions in social life into models. The conditions in question are: (i) normal ecological cues for humans underdetermine which strategic moves in which games with which players are represented by any given human action; and (ii) people are expected by others (and hence by themselves) to dynamically revise their narratives, though within limits that leave them on equilibrium paths (point 2 above), as they move through their life cycles. The fact that we are not forced to treat whole personal biographies as careers of single economic agents encourages us to make narrative revision endogenous, opening space for a new game-theoretic modeling domain I call ‘game determination’. I will very briefly explain this; readers who are puzzled by the concise account should consult Ross (2006a).

In consequence of fact (i) above, people engage in meta-games over which games to play. These would be intractable in the framework of non-cooperative game theory were it not for the constraints on narrative flexibility emphasized by point (2); bargaining among agents who have valuable reputations for being the sorts of people who keep their word allows non-cooperative players to make binding promises in meta-games that thereby simulate cooperative first stages relative to the games they determine – that is, that they generate as equilibrium outcomes. Game determination models the process by which narratives are revised as follows. Players of meta-games bargain not just over immanent games in which they themselves, with their present utility functions, will be involved, but also with respect to expectations about the ranges of possible future agents, with new utility functions, who will face the consequences of present games. Present agents, that is, act with the likely circumstances of their descendent agents in consideration. Of course, sequences of agents that map onto the same person have many important properties in common (including a great deal of shared preference structure). These properties are no doubt crucial for grounding partiality of agents towards future stages of the person they share. However, we should bear in mind that many people are in fact deeply neglectful of their future personal utility. In addition, most people are also relatively ignorant about the utility functions of the future agents they will be, which is manifest in the fact that the matching law more accurately describes human behavior than do models based on expected utility maximization by a single persisting agent (Herrnstein 1997, Ainslie 2001). The main factor that compensates for neglect and ignorance of the future in preserving the biographical unity we observe in people is the fact that their later stages inherit narrative constraints established by earlier stages, as a result of social expectations. (Institutions strongly reinforce this basic incentive; future agents associated with my body and social reputation will spend the money I earn and enjoy the security of the tenure I obtained. This gives me a special reason to be interested in these agents, and people tend, all else being equal, to favor the welfare of agents that interest them.) This explains my preference for making agent re-narration endogenous in game-theoretic models of interaction, rather than for applying game-theoretic models only after psychological ones have been used to describe agent formation and stabilization.

Individual determining games are modeled using classical game theory, but these static games should be understood as abstractions from (or snapshots of) continuing evolutionary games among lineages of agents. In fully testing the applicability of a given game determination model, one would check to ensure that all equilibria of particular games corresponded to points in basins of attraction in plausible underlying evolutionary dynamics (see Cressman 2003 for techniques). The most important parameter in a determining game that must be chosen on the basis of empirical evidence (as opposed to modeling convenience or rigour etc.) is the solution concept to be applied. The more foresight and concern for descendent agents is displayed by players of determining games, the stronger the restriction we can motivate on the relationship between a determining game and those it determines. In the limit, where agents have perfect foresight and are perfectly altruistic with respect to their descendents, determining games will simply be stages in chains of extensive form games to be solved by finding subgame-perfect equilibria. Institutional devices that strongly reward and reinforce time-consistent preferences, such as modern financial institutions, may push agents towards this limit. The economics appropriate to analyzing markets involving such agents will be Leijonhufvud’s modern variant. Where players of determining games are altruistic about their descendents but information is incomplete, the appropriate solution concept might be sequential equilibrium. At the other extreme, where players of determining games have little interest in and/or foresight about their descendents, outcomes of 2-stage games with determining games as first stages might not even induce Nash equilibria in the determining stages. At this limit, construction of classical (game theoretic) abstractions from the underlying evolutionary games would be pointless, and the economics we would usefully apply would be, in Leijonhufvud’s terms, more classical in spirit than modern. It is doubtful that any actual people play determining games at either limit.[7] Where between the limits the games of a given actual set of people in a given circumstance fall must be discovered empirically by seeing which solution concepts yield outcomes that predict observations.

It might be regarded as an objection to the proposed modeling framework that it leaves so much open. However, I take it as simply a fact about people that their degree of intertemporal preference consistency is variable and highly sensitive to institutional and cultural contexts. This is a (massive) complication that any way of modeling them as economic agents must accommodate. I have discussed the implications of this variation in human economic behavior as a result of ‘top-down’ influences in some detail elsewhere (Ross 2005, 2006a). In the present paper, I will therefore leave this topic and concentrate instead on the source of variation from ‘below’. Not only are people not unified economic agents over time. They also only approximate unified agency at a time, in essentially the way that a country does. We can often usefully identify what a country wants (to reduce poverty, to win the war, etc.), but we recognize that such desires are complex functions of the interacting goals and beliefs of its citizens. Similarly, when we represent a person as an agent with consistent preferences, we perform an abstraction, suppressing attention to the fact that molar behavioral dispositions are products of competitive and coalitional dynamics among sub-personal interests at the molecular level (Schelling 1978, 1980, 1984; Ainslie 1992, 2001). In previous work on this theme (Ross 2005, 2006b), I claimed that this fact has its origins in the information-processing complexity of the brain, arguing that were all valuation performed at a single bottleneck point people would resemble mini Soviet Unions that would fail for the same reason the big one did. However, I provided no evidence for or elaboration of this claim based on neuroscience. In the part 5 of the chapter I aim to repair this deficit by means of an extended example of the conflict among brain regions for control of behavior that, in an unfortunate subset of people, leads to addiction. To set this up, however, we first need background on the best developed general approaches to modeling sub-personal economic processes.

4. People as products of neuroeconomic and picoeconomic dynamics

The most sophisticated account of human molar behavior as a product of interaction among sub-personal agents is that of Ainslie (1992, 2001). Ainslie’s theory, which he calls ‘picoeconomics’, explains a range of common behavioral phenomena, including among others procrastination and neurosis. The theory’s most direct application is to difficulties people experience with impulse control, which it explains by appeal to the equilibria of games played amongst hypothesized sub-personal interests. The identities of such interests are directly inferred from common-sense personal goals; thus someone might now be experiencing conflict between an interest in cleaning the wash basin and an interest in watching the rugby test. Picoeconomics is not integrated with the more recent ‘neuroeconomics’ that uses economic theory to model the activity of functionally specialized groups of neurons (Montague and Berns 2002; Glimcher 2003). It is thus an open question as to what relationship picoeconomic interests might have to functional parts of the brain.

The virtual nature of Ainslie’s interests leads many people’s intuitions, in my experience, to judge interests as not plausibly real forces or processes, but as at best fictions constructed for modeling purposes. People with these intuitions generally take a different view of the reality of interests if they are thought instead to be based on independently discriminable brain regions. This of course reflects the intuitive reductionism of most scientists. However, in Ross (2006b) I argue that such reduction between picoeconomics and neuroeconomics is not in the cards, and Ainslie (2005) agrees. Picoeconomic interests only do the explanatory work required of them in Ainslie’s model to the extent that they can be fleeting, persisting only for as long as the behavior they motivate. For example, as I write I have an interest in watching a baseball game, which has to this point been unsuccessful in competition with another interest in making progress on this chapter. The first interest will not outlast the baseball game, and the second interest will hopefully have nothing further to do in a few days. Of course, I have a number of less fleeting, related interests such as in preserving a reputation for meeting deadlines; but this is precisely not subject to overthrow by baseball games in the way that the interest in working right now is. Another of Ainslie’s favorite examples is of an annoying interest in scratching an itch, which will fade entirely if even briefly ignored; unless the itch is caused by a foreign irritant, the interest in scratching is the itch. These examples should make clear that picoeconomic interests aren’t sub-personal in the same sense as groups of neurons with specialist functions. The former are sub-personal in the sense that they have sharply limited projects that may not be endorsed by the whole person, but it is molar phenomena – behavior of a whole agent at a time – with which they are associated. The agents of neuroeconomics, by contrast, are sub-personal in the sense of being molecular components of organisms. People who think picoeconomics is valuable science must defend it not be plumping for reductive vindication of the reality of sub-personal interests, but by resisting the widespread intuition that virtual entities are necessarily not fully real – a task for which good philosophical assistance is available (e.g. Dennett 1991).

What follows from this is not that neuroeconomics isn’t important to the economics of the sub-personal, but that the relationship among neuroeconomics, picoeconomics and microeconomics will be more complicated than a chain of intertheoretic reductions that model a parallel chain of ontological reductions. I will say a bit more about the positive relationship between neuroeconomics and picoeconomics later in the paper. First, however, I want to set picoeconomics aside for awhile and focus on the brain.

If neuroeconomics were to have a bumper sticker, its best message might be the following quote from Greg Berns (2003, p. 156): “The interaction of different pools of neurons in the brain may result in phenotypic behavior that appears to be irrational, but it is possible that the rational agents are the neurons, not the person.” This should remind us of the earlier analogy between people and countries. We all know that countries often behave irrationally- erecting self-harming barriers to imports, for example – due to the interactions of rational citizens acting in pursuit of their parochial interests. I will illustrate expression of this same pattern in the case of the relationship between people and their neurons by focusing on some properties of a particular brain system, the dopaminergic reward system, and its relations with other brain systems and with molar behavior.

Neuroscientists individuate ‘systems’ in the brain by identifying generic functional responses with relatively encapsulated neurotransmitter pathways. The reward system is distinguished as a pathway that transmits signals using the neurotransmitter dopamine. Activity in midbrain areas that people share with other vertebrates, the ventral tegmental area (VTA) and pars compacta of substantia nigra (SNpc), release dopamine in response to surprising magnitudes of learned contingencies. These signals project most directly to the ventral striatum (VS) and especially to the nucleus accumbens (Nacc). For reasons to be explained later, persistently high concentrations of dopamine in Nacc are a basic neural signature of addiction. The reward system’s dopamine signal also projects to pre-frontal cortex (PFC), where it appears to produce, at least in non-addicts, a serotonin signal that acts as an opponent process. I will have more to say about this opposition later.

First, however, more detail is needed on the nature and function of the dopamine signal itself. fMRI evidence (McClure, Daw annd Montague 2003) strongly suggests that the reward system implements what is called temporal difference learning (Sutton and Bartow 1998). This denotes a family of functions that relate a situation at a particular time s_t to a time-discounted sum of expected rewards (idealized as numeric measures r of received utility) that can be earned into the future. Suppose, following McClure, Daw and Montague (2003), that t, t+1, t+2 etc. represent times on some arbitrary measurement scale. Then the TD equation is

V^*(s_t) = E[r_t + gr_t+1 + g²r_t+2 + g³r_t+3 + …]

which we close by writing

V^*(s_t) = E[r_t + g V^*(s_{t + 1})]

This describes the procedure by which the reward-system learning algorithm continuously inputs new information to keep refining its estimate of V^* to get a particular stream of actual temporal valuations V. From this we can define a measure d of the extent to which the value estimates of two successive states and a reward experienced by the system are consistent with one another:

d(t) = r_t + g V(s_{t + 1}) – V(s_t),

where d is an error signal that pushes V(s) towards better estimates as it gets more data. If V(s_{t + 1}) turns out to be better than expected, then d(t) will be positive, thus indicating that V(s_t) needs to be adjusted upward. If V(s_{t + 1}) turns out to be worse than expected, d(t) will be negative and V(s_t) will be adjusted downwards. If d(t) = 0 then of course no learning occurs.

That the dopamine system implements TD learning is not presently controversial. However, there are different views among researchers as to the wider role of this learning. The reward system appears to integrate all of the following functions: (i) learning environmental cues that predict reward, (ii) learning comparative values of rewards, (iii) focusing attention on cues that predict rewards, and (iv) motivating the system to act on the basis of these cues (thanks to projection from the dopamine system to motor neurons). However, it is not necessarily the case that TD learning is the source of the integration. In its basic form – and the form as presented above – TD learning does not predict a discovery due to Gallistel (1990), that conditioning outcomes in animal learning are timescale invariant. This means that responses from a given animal will be the same in two otherwise identical learning conditions if the interval durations between cues and rewards in one condition are a constant multiple of the durations of the other. In general, TD learning has generally been framed as a model of classical association conditioning, after Rescorla and Wagner (1972). By contrast with association models, Scalar Expectancy Theory (SET) (Gibbons 1977) and Rate Estimation Theory (RET) (Gallistel 1990), as unified in Gallistel and Gibbon (2000), are timing models of conditioning phenomena. According to such models, animals represent the durations of intervals and the rates of relevant events, and conditioned responding occurs as a function of the comparison of rates of reinforcement. Animals are drawn to environments with higher such rates by gradient climbing, rather than forming explicit associations between stimuli and conditioned responses. For ease of reference, I will call the unification of SET and RET ‘G-learning’.

G-learning has attractive properties from the point of view of prospective unification of molecular accounts of learning with picoeconomics. In a recent reply to critics, Ainslie (2005) says at one point “Sanabria & Killeen were alone among the commentators in mentioning the opportunity [raised by picoeconomics] to revise ‘the hoary study of CRs [conditioned responses]’. I judge such a revision to be among [my theory’s] most important implications … [F]or the selection of responses, the potential of brief temporary preferences to lure organisms into responses that are aversive overall, could let us do without the deux ex machina of a second, ‘conditioned’ selective principle that is so often invoked to explain aversive, involuntary or maladaptive processes … The model of aversion as a rapidly cycling addiction comprising reward and inhibition of reward lets us add conditioned processes to the marketplace of rewarded processes.” What Sanabria and Killeen (2005) had said that prompted this response of Ainslie's was: “A key property of signals of reinforcement is that they become both conditioned reinforcers, or CRs, and conditioned stimuli, eliciting approach … Is this the behavioral substrate of desire, of appetite? If so, then Ainslie's hyperbolic interests, Skinner's CRs, and Pavolv's CSs are the same entity. A theory of one is a theory of all” (p. x).

In this perspective, G-learning, to the extent that it applies directly to the reward system, would suggest a natural way of unifying the molar and the molecular accounts of entrapment by addictive targets. Ainslie's “rapidly cycling addictions” would simply be conceptualized as high-reward-rate ‘environments’ that lure organisms’ attention and approach, unless the dopamine system is opposed – as it appears to be, successfully in non-addicts, by seratonergic and (probably) GABAnergic signals from PFC.[8] Gallistel likewise argues that classical and operant conditioning amount to the same thing. In addiction studies there has been a long-running and unresolved debate over the relationship between supposedly classically conditioned cravings and apparently instrumentally conditioned preparations for consumption of addictive targets. Perhaps the debate has been inconclusive because these are one and the same process. But if G-learning is sufficient to provide neurocomputational foundations for Ainslie’s molar account, then what is the role in impulsive choice, if any, of TD learning in the dopamine system?

A recent computational model of the reward system by Daw (2003) makes the two kinds of learning complementary. In this model G-learning is taken to precede, and indeed to enable, TD learning. Suppose an animal has learned a function that predicts a reward at t, where the function in question decomposes into models of two stages: one applying to the interval between the conditioned and the unconditioned stimulus, and one applying to the interval between the unconditioned stimulus and the next conditioned stimulus. Then imagine that a case occurs in which at t nothing happens. Should the animal infer that its model of the world needs revision, perhaps to a one-stage model, or should it retain the model and regard the omission as noise or error? In Daw's account the animal uses G-learning to select a world-model: whichever such model matches behavior that yields the higher reward rate will be preferred to alternatives. Given this model as a constraint, TD learning can then predict the temporal placement of rewards (‘when’-learning). This hybrid approach allows Daw to drop two unbiological features of the original Montague, Dayan and Sejnowski (1996) model of TD learning by the dopamine system: tapped-line delay timing and exogenously fixed trial boundaries (as can be justified only in the sparse and controlled setup of the laboratory). This is surely progress, as it is doubtful that anyone ever took these two properties for anything other than modeling conveniences. But at this point it can no longer be said to be clear what, if anything, is the ‘canonical’ model of the role of TD learning in the dopamine system.

Fortunately for present purposes, at a slightly higher level of abstraction this uncertainty about molecular mechanisms becomes less important. As noted above, there is increasing consensus that the reward system integrates reward prediction, valuation, salience and approach, whether it does so by bolting together two learning algorithms, as in Daw’s model, or by implementing a single more complex one, as in the Predictor-Valuation (PV) model of Montague and Berns (2002). Given the possibilities left open by present empirical knowledge, we can treat the latter either as a direct molecular account of one learning process, or as an account one level up in the molar direction of a function implemented by G-learning and TD learning together. The PV model is characterized as follows. Suppose R(x,n) estimates the value of a reward distributed at various possible times x, y, z, …, n in the future, scaled according to the uncertainty attending to the intervals between the estimation point and each time, as in:

R(x,n; D) = ò_-¥^+¥ dy G(x – y, (x – n)D)r(y)

where G(z,b) = (2pb)^-1/2 exp{-z²/2b} and D is a constant. Then the value F(n) the brain attaches to getting a particular predictor signal at perceptual time n is given by:

F(n) = ò_n^+¥ dx e^-q(x-n) ò_-¥^{+¥ dy} G(x – y₁(x – n)D)r(y) = ò_n^+¥ dx {e^-q(x-n)} X {R(x,n; D) = ò_n^+¥ dx {discount future time x relative to perceptual time n} X {diffused version of reward estimate r(x) for some x and n}

Interestingly, as Montague and Berns point out, this functional form corresponds to the Black-Scholes model of portfolio option pricing.

The label ‘neuroeconomics’ has been attached to modeling of this sort[9] on grounds that seem entirely appropriate. PV is essentially a model of the reward system’s estimate of the expected opportunity costs of attending to one stimulus rather than another and of preparing one motor response rather than another. (It seems that, in light of the brain’s architecture, these opportunity costs can’t be factored out separately.) Where there is an opportunity cost there must be an underlying utility function. This in turn invites us to ask which natural entity we should associate as an agent with this utility function. Three possibilities might occur from a naïve perspective: that the utility function should be assigned to (i) the reward system, (ii) the brain as a whole, or (iii) the person.

It seems implausible that this question might reasonably be settled on any general ontological grounds of the sort that might be recommended by a metaphysical argument. The question asks for a pragmatic decision, though not one independent of the scientific facts. Different decisions will facilitate varying ways of modeling the relevant empirical phenomena. Deciding, for example, that it isn’t helpful to model the reward system as an agent would be equivalent to concluding that it is a neurochemical system rather than a neuroeconomic one. This wouldn’t amount to rejection of PV; it would be compatible with treating PV as a molar-level description (from one of two levels: molar neuroscience if the whole brain is assigned a utility function, or behavioral if one assigns opportunity costs at no finer a scale than that of the person).

If we expected each of the three possible options under consideration[10] to be equally empirically adequate to the full range of phenomena up for prediction and explanation, then we would rightly regard the decision among them as having no objective basis; in making it, we would be in the domain of pure pragmatics. This situation is of course logically possible, but if it turned out to be actual this would be an astonishing coincidence. More seriously, if we don’t think there is any hope for achieving an integrated scientific picture of the world – if, that is, we agree with Dupré (1992) and Cartwright (1999) that the best we can get from science is a disordered plurality of models of phenomena that can neither be unified nor pared down to a mutually consistent set on the basis of unambiguous criteria of preference – then the decision again leaves us in the domain of pure pragmatics. On Cartwright’s picture, for example, we might write down both a neurochemical model and a neuroeconomic model of the reward system, and if each yielded non-redundant predictions good enough for some purposes, then we would have no reason to ask whether we should try to explain one in terms of the other (or, perhaps, both in terms of some third account). The reason we would have no reason is that, according to Cartwright, belief that there is a unified reality underlying the undisputed cacophony of models we find in science (Cartwright being right, in general about the existence of such cacophonies) is ‘fundamentalism’, and fundamentalism is just a metaphysical hankering.

However, on inductive grounds I venture the following prediction about the future of neuroscience. Attempts will be (vigorously) pursued to provide a neurochemical explanation of the phenomena described by the PV model. If such attempts are completely successful, in the sense that a purely neurochemical model eventually explains everything the neuroeconomic model does (and, presumably, more besides), then we shall conclude that we’ve experienced an episode of intertheoretic (Nagelian) reduction and that attributing a utility function – that is to say, economic agency – to the reward system has been (as Dennett would put it) discharged. Alternatively, it may turn out that neurochemical facts constrain (in enlightening ways) the way PV is implemented in the reward system, but other aspects of PV can only be explained by reference to functional properties of neural learning more generally and perhaps by reference to ecological generalizations about the reward environments in which brains evolved and develop. In that case the neuroeconomic model of the reward system will remain an irreducible part of our best scientific account of the world. And in that case, for a naturalist there is no reason to hedge saying that the reward system has a utility function – is an agent – by using scare-quotes or finger wiggles. The basic commitment of the naturalist, after all, is that the best scientific account of the world is the best account of the world, period. If that account assigns a utility function to the reward system, then it is true – in the one and only plain old sense of ‘true’ there is – that the reward system is an agent.

What seems to me very unlikely indeed is an outcome in which we get a decent neurochemical account of the reward system and a decent neuroeconomic account (i.e., answer presently open questions such as whether PV describes integration of TD learning and G-learning, whether we need separate accounts of trace conditioning and delay conditioning within the PV or G-learning frameworks, etc.), and then nobody is interested in trying to tie them together. This expectation, of a kind common across the sciences, is why I think there is still some use for philosophy of science and why I think that Cartwright, despite her repeated powerful insights into the way science is carried out and models are put to use, is wrong about the importance of unification.

While we wait to find out what happens in the complex integration of neurochemistry and neuroeconomics, we can and should ask about the extent to which we get present explanatory payoffs from modeling the reward system as an agent. I contend that we do. In particular, I contend that the neuroeconomic account of reward prediction and valuation supports an elegant account of a widespread and important phenomenon, addiction, as th