Abstract
Learning and evolution are adaptive or “backward-looking” models of social and biological systems. Learning changes the probability distribution of traits within an individual through direct and vicarious reinforcement, while evolution changes the probability distribution of traits within a population through reproduction and selection. Compared to forward-looking models of rational calculation that identify equilibrium outcomes, adaptive models pose fewer cognitive requirements and reveal both equilibrium and out-of-equilibrium dynamics. However, they are also less general than analytical models and require relatively stable environments. In this chapter, we review the conceptual and practical foundations of several approaches to models of learning that offer powerful tools for modeling social processes. These include the Bush-Mosteller stochastic learning model, the Roth-Erev matching model, feed-forward and attractor neural networks, and belief learning. Evolutionary approaches include replicator dynamics and genetic algorithms. A unifying theme is showing how complex patterns can arise from relatively simple adaptive rules.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
- 1.
- 2.
A multilayer neural net requires a nonlinear activation function (such as a sigmoid). If the functions are linear, the multilayer net reduces to a single-layer I-O network.
- 3.
However, if an input node is wired to hidden nodes as well as output nodes, the error for this node cannot be updated until the errors for all hidden nodes that it influenced have been updated.
- 4.
The Cournot rule may be considered as a third degenerate model of belief learning. According to the Cournot rule, players assume that the behavior of the opponent in the previous round will always occur again in the present round.
References
Allison, P. (1992). The cultural evolution of beneficent norms. Social Forces, 71, 279–301.
Bandura, A. (1977). Social learning theory. Englewood Cliffs, NJ: Prentice Hall.
Börgers, T., & Sarin, R. (1997). Learning through reinforcement and replicator dynamics. Journal of Economic Theory, 77, 1–14.
Boyd, R., & Richerson, P. J. (1985). Culture and the evolutionary process. Chicago, IL: University of Chicago Press.
Bush, R. R., & Mosteller, F. (1950). Stochastic models for learning. New York: Wiley.
Byrne, D. E. (1971). The attraction paradigm. New York: Academic.
Byrne, D. E., & Griffitt, D. (1966). A development investigation of the law of attraction. Journal of Personality and Social Psychology, 4, 699–702.
Cartwright, D., & Harary, F. (1956). Structural balance: A generalization of Heider’s theory. Psychological Review, 63, 277–293.
Catania, A. C. (1992). Learning (3rd ed.). Englewood Cliffs, NJ: Prentice Hall.
Churchland, P. S., & Sejnowski, T. J. (1994). The computational brain. Cambridge, MA: MIT Press.
Cohen, M. D., Riolo, R., & Axelrod, R. (2001). The role of social structure in the maintenance of cooperative regimes. Rationality and Society, 13(1), 5–32.
Davis, J. A. (1967). Clustering and structural balance in graphs. Human Relations, 20, 181–187.
Dawkins, R. (1976). The selfish gene. Oxford: Oxford University Press.
Durham, W. H. (1992). Coevolution: genes, culture and human diversity. Stanford, CA: Stanford University Press.
Erev, I., & Rapoport, A. (1998). Coordination, “magic”, and reinforcement learning in a market entry game. Games and Economic Behavior, 23, 146–175.
Erev, I., & Roth, A. E. (1998). Predicting how people play games: Reinforcement learning in experimental games with unique, mixed strategy equilibria. American Economic Review, 88(4), 848–879.
Erev, I., Bereby-Meyer, Y., & Roth, A. E. (1999). The effect of adding a constant to all payoffs: Experimental investigation, and implications for reinforcement learning models. Journal of Economic Behavior and Organizations, 39(1), 111–128.
Flache, A., & Macy, M. W. (2002). Stochastic collusion and the power law of learning: A general reinforcement learning model of cooperation. Journal of Conflict Resolution, 46(5), 629–653.
Frank, R. (1988). Passions within reason: The strategic role of the emotions. New York: Norton.
Fudenberg, D., & Levine, D. (1998). The theory of learning in games. Boston: MIT Press.
Goldberg, D. (1989). Genetic algorithms in search, optimization, and machine learning. New York: Addison-Wesley.
Hamilton, W. (1964). The genetic evolution of social behaviour. Journal of Theoretical Biology, 17, 1–54.
Harsanyi, J. (1973). Games with randomly disturbed payoffs. International Journal of Game Theory, 2, 1–23.
Heider, F. (1958). The psychology of interpersonal relations. New York: Wiley.
Herrnstein, R. J., & Drazin, P. (1991). Meliorization: A theory of distributed choice. Journal of Economic Perspectives, 5(3), 137–156.
Holland, J. (1975). Adaptation in natural and artificial systems. Ann Arbor: University of Michigan Press.
Holland, J. H., Holyoak, K. J., Nisbett, R. E., & Thagard, P. R. (1986). Induction: Processes of inference, learning, and discovery. Cambridge, MA: MIT Press.
Homans, G. C. (1951). The human group. New York: Harcourt Brace.
Hopfield, J. J. (1982). Neural networks and physical systems with emergent collective computational abilities. Proceedings of the National Academy of Sciences, 79, 2554–2558.
Hopfield, J. J., & Tank, D. W. (1985). “Neural” computation of decisions in optimization problems. Biological Cybernetics, 52, 141–152.
James, W. (1981). Principles of psychology. Cambridge, MA: Harvard University Press.
Lopreato, J. (1990). From social evolutionism to biocultural evolutionism. Sociological Forum, 5, 187–212.
Macy, M. (1989). Walking out of social traps: A stochastic learning model for the prisoner’s dilemma. Rationality and Society, 2, 197–219.
Macy, M. (1991). Learning to cooperate: Stochastic and tacit collusion in social exchange. American Journal of Sociology, 97, 808–843.
Macy, M. (1996). Natural selection and social learning in prisoner’s dilemma: Co-adaptation with genetic algorithms and artificial neural networks. Sociological Methods and Research, 25, 103–137.
Macy, M. (1997). Identity, interest, and emergent rationality. Rationality and Society, 9, 427–448.
Macy, M. (1998). Social order in an artificial world. Journal of Artificial Societies and Social Simulation, 1(1). http://jasss.soc.surrey.ac.uk/1/1/4.html
Macy, M. (2004). Learning theory. In G. Ritzer (Ed.), Encyclopedia of social theory. Thousand Oaks, CA: Sage.
Macy, M., & Flache, A. (2002). Learning dynamics in social dilemmas. Proceedings of the National Academy of Sciences, 99, 7229–7236.
Macy, M., & Skvoretz, J. (1998). The evolution of trust and cooperation between strangers: A computational model. American Sociological Review, 63, 638–660.
Macy, M., Kitts, J., Flache, A., & Benard, S. (2003). Polarization in dynamic networks: A Hopfield model of emergent structure. In R. Breiger & K. Carley (Eds.), Dynamic social network modeling and analysis: Workshop summary and papers (pp. 162–173). Washington, DC: National Academy Press.
March, J. G., & Simon, H. A. (1958). Organizations. New York: Wiley.
McPherson, J. M., & Smith-Lovin, L. (1987). Homophily in voluntary organizations: status distance and the composition of face to face groups. American Sociological Review, 52, 370–379.
Nowak, A., & Vallacher, R. R. (1998). Toward computational social psychology: Cellular automata and neural network models of interpersonal dynamics. In S. J. Read & L. C. Miller (Eds.), Connectionist models of social reasoning and social behavior (pp. 277–311). Mahwah, NJ: Lawrence Erlbaum.
Offerman, T. (1997). Beliefs and decision rules in public good games. Dordrecht: Kluwer.
Quinlan, P. T. (1991). Connectionism and psychology: A psychological perspective on new connectionist research. Chicago, IL: University of Chicago Press.
Roth, A. E., & Erev, I. (1995). Learning in extensive-form games: Experimental data and simple dynamic models in intermediate term. Games and Economic Behavior, 8, 164–212.
Rumelhart, D. E., & McClelland, J. L. (1988). Parallel distributed processing: Explorations in the microstructure of cognition. Cambridge, MA: MIT Press.
Schwartz, S. H., & Ames, R. E. (1977). Positive and negative referent others as sources of influence: A case of helping. Sociometry, 40, 12–21.
Sun, R. (Ed.). (2008). The Cambridge handbook of computational psychology. Cambridge: Cambridge University Press.
Taylor, P. D., & Jonker, L. (1978). Evolutionary stable strategies and game dynamics. Mathematical Biosciences, 40, 145–156.
Thorndike, E. L. (1898). Animal intelligence: An experimental study of the associative processes in animals, Psychological Review, Monograph Supplements, No. 8. New York: Macmillan.
Vanberg, V. (1994). Rules and choice in economics. London: Routledge.
Williams, G. C. (1966). Adaptation and natural selection. Princeton, NJ: Princeton University Press.
Wilson, E. O. (1988). On human nature. Cambridge, MA: Harvard University Press.
Wynne-Edwards, V. C. (1962). Animal dispersion in relation to social behaviour. Edinburgh: Oliver & Boyd.
Wynne-Edwards, V. C. (1986). Evolution through group selection. Oxford: Blackwell Scientific.
Young, H. P. (1998). Individual strategy and social structure. An evolutionary theory of institutions. Princeton, NJ: Princeton University Press.
Zeggelink, E. (1994). Dynamics of structure: An individual oriented approach. Social Networks, 16, 295–333.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Further Reading
Further Reading
We refer readers interested in particular learning models and their application in agent-based simulation to Macy and Flache (2002), which gives a brief introduction into principles of reinforcement learning and discusses by means of simulation models how reinforcement learning affects behavior in social dilemma situations, whereas Macy (1996) compares two different approaches of modeling learning behavior by means of computer simulations. Fudenberg and Levine (1998) give a very good overview on how various learning rules relate to game-theoretic rationality and equilibrium concepts.
For some wider background reading, we recommend Macy (2004), which introduces the basic principles of learning theory applied to social behavior; Holland et al. (1986), which presents a framework in terms of rule-based mental models for understanding inductive reasoning and learning; and Sun (2008), which is a handbook of computational cognitive modeling.
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this chapter
Cite this chapter
Macy, M.W., Benard, S., Flache, A. (2017). Learning. In: Edmonds, B., Meyer, R. (eds) Simulating Social Complexity. Understanding Complex Systems. Springer, Cham. https://doi.org/10.1007/978-3-319-66948-9_20
Download citation
DOI: https://doi.org/10.1007/978-3-319-66948-9_20
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-66947-2
Online ISBN: 978-3-319-66948-9
eBook Packages: Computer ScienceComputer Science (R0)