Recent Development of Social Simulation as Reflected in JASSS Between 2008 and 2014: A Citation and Co-Citation Analysis

Introduction

Social simulation is considered to be a multidisciplinary and rapidly developing field (Meyer, Lorscheid & Troitzsch 2009; Squazzoni, Jager & Edmonds 2014). This is mirrored by increasing citations in different ISI- and Scopus-indexed sources (Squazzoni 2010) and by the fact that simulation methods have more recently gained a foothold in different social science publication outlets (Fioretti 2013; Leitner & Wall 2015; Secchi & Seri 2016). Nevertheless, the Journal of Artificial Societies and Social Simulation (JASSS) remains one of the major publication outlets for research in social simulation (Secchi & Seri 2016; Squazzoni 2010; Squazzoni & Casnici 2013).

In 2014, JASSS experienced a significant change that led to intensive discussions about the journal. After 17 years, Nigel Gilbert, the founder of the journal, handed over his responsibilities as editor, which resulted in a debate in the social simulation community about the future direction of JASSS (see, e.g. SIMSOC 2015). This discussion addressed the scope of the journal and its possible future direction. Some perceive JASSS as an interdisciplinary journal at the intersection of various fields such as the social, behavioral, and computational sciences, whereas others suggest that the journal’s scope should be extended to include more fundamental questions of science that support simulation research in general. Another group points out that JASSS frequently publishes technical, epistemological, and methodological papers. This discussion shows the different perspectives on the journal and its role in the community.

This diversity might also be due to the multidisciplinary and dynamic character of JASSS, which makes it even more challenging to get an overview of the journal, the related fields of social simulation, and the recent development of both. Such an overview, however, would be beneficial for several reasons. First, it would allow for an empirical basis for the above-described discussions about the past and future direction of JASSS. Second, given the increasing interest from other disciplines, it would make access to the field much easier, as newcomers could inform themselves about the current state of the literature. Of particular relevance for this would be a summary of the most influential articles and main foci of research. Finally, such an overview would complement a previous study about the development of the field in the early years, from its beginning in 1998 until 2007 (Meyer et al. 2009). In combination, the two may provide an overview of the whole time span of the first 17 years of JASSS under the editorship of Nigel Gilbert and the field during that time.

The objective of this paper is to provide such an overview. We map the recent developments in the field of social simulation as reflected in JASSS publications from 2008 to 2014. This study focuses on developments regarding the most cited sources and on networks of frequently co-cited publications. Bibliometric methods such as citation and co-citation analysis are suitable to uncover hidden patterns in publication outlets. These patterns delineate historical developments, depict the current situation, and provide a foundation to discuss future developments (van Raan 2014).

In terms of the methods used, we closely link our work to a previous study of the development of JASSS in its first 10 years (Meyer et al. 2009), which allows us to identify continuities as well as changes. We particularly want to investigate the following issues: (1) What are the recent developments in view of cited publications, types of citation sources, or influences of certain fields? Does more recent development differ from that in the first 10 years and in what respect? (2) Which co-citation networks emerged, how strong are their relations, and to what research topics are these related? (3) Are trends observable, in the past seven years or the overall time span of 17 years? Are there indicators for the future direction of the journal and social simulation as a discipline?

The paper is organized as follows. In the next section, we describe our method and data set. Afterwards, we present the most influential publications in JASSS and highlight some specific source characteristics regarding publication age, source type, and cited journals classified by discipline. Subsequently, we present our results of the co-citation analysis to identify research clusters in JASSS and investigate the relationships between these clusters. Further, we present a longitudinal analysis of the research topics in social simulation. Finally, we draw some conclusions and make suggestions for future research.

Method

This paper investigates the development of the intellectual structure of JASSS from 2008 to 2014. To this end, we apply the bibliometric methods of citation and co-citation analysis. Both methods are established for the analysis of scientific fields (Osareh 1996a, 1996b) and have been successfully applied to the analysis of different journals (Meyer et al. 2009; Meyer, Zaggl & Carley 2011; Mustafee, Katsaliaki & Fishwick 2014a, 2014b; Squazzoni & Casnici 2013). To maximize comparability with the previous study, we mainly follow the methods applied in Meyer et al. (2009), but extend them to explore additional questions.

Citation analysis investigates the occurrences of referenced publications. Via citations, an author shows the relation between the own work and the work of other scholars (Osareh 1996a). Whilst the number of citations is generally considered to be an indicator of the degree of a study’s perceived relevance and influence (Bornmann & Daniel 2008; Radicchi & Castellano 2012), citation counts also have weaknesses. Studies of citation behavior show that scientists not only cite other work to acknowledge the intellectual or cognitive influence of scientific peers, but also for other, probably less scientific reasons, which are individual and different (Bornmann & Daniel 2008). Furthermore, so-called “sleeping beauties,” which are publications whose importance is not recognized for several years after publication, may remain undiscovered (Ke et al. 2015). Still, citations are an indicator to determine the influence of publications and thus are commonly used evaluation measures.

Co-citation analysis examines the relationships between cited publications. A co-citation means that two publications are cited in the same document. For example, citation A and citation B are co-cited if both publications are listed in the same reference list of article C. The number of co-citations among publications is regarded as an indicator of their proximity (Gipp & Beel 2009; Small 1973). The identified relationships between cited publications allow us to draw conclusions about the internal structure of research, based on the resulting clusters of publications.

Using absolute citation values is not suitable to generate clearly defined clusters of publications. Sources with a high number of citations tend to appear more frequently in clusters than less cited sources due to their wide dissemination. To address this problem, several scaling methods have been developed. Gmür (2003) evaluated established methods and suggested a measure called a CoCit score, which sets the squared co-citation count in relation to the minimum and mean counts of two individual citations A and B (Gmür 2003).

$$CoCit_{AB}=\frac{(co-citation_{AB})^2}{minimum(citation_A;\,citation_B) \ast mean(citation_A;\,citation_B)}$$

In this paper, we apply this well-established measure to calculate the strength of co-citations in JASSS (see also Allmayer & Winkler 2013; Backhaus, Lügger & Koch 2011; García-Lillo, Úbeda-García & Marco-Lajara 2016; Greene et al. 2008; Moqri, Bandyopadhyay & Kitchens 2014). To reduce the complexity of analysis, we focus on the most cited publications, only including those with at least three co-citations. Further, we set a minimum CoCit score value of 0.25 as the threshold (both in line with Meyer et al. 2009). As a result, groups with distinguishable network topologies emerge such as isolated pairs, trees, mesh, and fully connected clusters. In this paper, we refer to a cluster if a network contains at least three sources linked by at least three co-citation relationships, with CoCit scores greater than or equal to 0.25 (as in Meyer et al. 2009).

For the data set generation, we used the online index of JASSS articles. This index provides an open access database to all articles and their references^[1]. We gathered the data by parsing all journal articles published in JASSS between 2008 and 2014, excluding book review articles^[2]. The parser retrieved the associated lists of references in a CSV file and assigned each source an ID. Identical sources were assigned the same ID. We corrected parser bugs and data inconsistencies manually^[3]. Afterwards, we were able unambiguously to verify citations as duplicates. If necessary, we corrected the source IDs and frequency of citations by hand. We used the resulting data set for the citation analysis and proceeded with the calculation of symmetric reference-reference matrices for the co-citation analysis, in line with the description of Zhang, Chen, and Li (2009).

Data Set

The resulting data set forms the basis for the citation and co-citation analysis. First, we want to provide some descriptive statistics. This comprises the number of JASSS articles included as well as the publications referenced in these articles. Table 1 shows the data set for 1998-2014. The whole time span of 17 years is included in the table to compare the results of our study with the results from the previous study (Meyer et al. 2009).

Table 1: Overview of the data sets of both studies: articles and citations (1998-2014)

	Meyer et al. (2009)		This study		Both
	1998-2002	2003-2007	2008-07/2011	08/2011-2014	1998-2014
Number of articles	110	184	133	165	592
Number of citations	2873	5375	4583	6191	19022
Average articles per issue	5.58	9.15	9.47	12.00	8.76
Average citations per article	26.12	29.21	34.46	37.52	32.13
Average source age (years)	10.85	10.91	12.20	12.61	11.77

The first study investigated two time periods, 1998-2002 and 2003-2007. In this study, we address the subsequent years from 2008 to 2014 and divide these seven years into two periods of 3.5 years^[4]. This is driven by the motivation to create comparable time periods – in terms of the number of analyzed articles and citations – to the previous study. The number of JASSS articles increased over time per issue and year from 133 articles in the first period of this study to 165 articles in the second period. The overall increase in the number of articles and citations is also reflected in the average number of articles per issue. JASSS issues of later periods include more articles than issues in the early years (see, for example, an average number of 11.8 articles per issue in the last period compared with 9.5 articles per issue in the period before). Even though the recent periods are shorter, their number of citations is comparable with the two periods of the previous study.

To deal with the rise in scientific publications, we determine the growth rate for the domain of social simulation and incorporate that with regard to the split of the data sets. First, we consider the general rise in scientific publications. This trend was identified as corresponding to a doubling of the global number of references in scientific publications within 12 years (Pan et al. 2016). Some scholars come up with effective indices to discount exponential growth for selected domains (e.g., Parolo et al. 2015). Nevertheless, deflation indices are dependent on research domains and average indices neglect the heterogeneity within and across disciplines. Following our assumption that social simulation is at least partially reflected in JASSS, we determined its growth rate by analyzing the yearly number of citations in JASSS (see Figure 1). Given our data set, we assumed an exponential function and identified an 8.5% growth rate of references per year (R²=0.76). This growth rate is above average in relation to the growth rate of 5.8% for all domains, and remarkably higher than 4.8% for the social sciences (Pan et al. 2016).

To understand this result, we consider the two parameters that determine the growth rate: the increasing number of publications and the increasing average number of references per publication. As previously mentioned, the social simulation growth rate is also driven by both parameters. We identify an increasing number of articles in JASSS as well as higher numbers of references. Overall, the comparable high growth rate indicates fast growth.

JASSS for 17 years (1998-2014) in comparison to other domains">

The growth rate of references influences the comparability of citation metrics in different time spans. Average growth and deflation or inflation rates are metrics to reflect trends. Definite rates are per se not retrievable considering that references are commonly made in discrete time intervals of years. Given the calculated social simulation growth rate of 8.5%, a six-year time span (2008-2013) would be the most statistically comparable time span to the previous study (1998-2007) to come to comparable data set sizes. Following our aim to cover the full time span of the editorship of Nigel Gilbert, we cover a seven-year time span in this study (2008-2014) and accept a slight distortion. A division of the seven-year time span by growth rate approximately results in a four-year (2008-2012) and a three-year (2012-2014) period. Nevertheless, we slightly deviate from this and use equal periods of 3.5 years, on the grounds that an uneven split of the recent time span would dilute the tangibility of comparison. Finally, we account for the growth rate at a higher aggregation level. The previous study covers ten and our study a shorter time span of seven years.

Looking at the citation characteristics, we identify that the average source age increased from 1998 to 2014. This effect can be ascribed to the fact that some fundamental work is still cited in more recent publications. To visualize this, we plotted the publication years of the cited publications in JASSS per time period (see Figure 2). The distribution graph shows a typical shape found in other fields as well, and could be approximated by a left-skewed distribution (see e.g. Schäffer et al. 2011).

JASSS from 1998 to 2014">

Next, we investigate the frequency of citations. The repeated occurrence of citations in JASSS articles is the basis for our citation and co-citation analysis. The overview in Table 2 shows the number of citations that occur once, twice, or at least three times. Most publications are only cited once, while only about 10% of the references occur multiple times. Compared with the previous study, the relative frequency of single citations increased from about 88% to more than 91% in the most recent period (Meyer et al. 2009). This indicates slightly increasing diversification in terms of sources.

Table 2: Frequency of citations in JASSS from 2008 to 2014

	Previous study (Meyer et al. 2009)						This study
	1998-2001			2002-2007			2008-07/2011			08/2011-2014
Frequency of occurrence of citations	Absolute frequency	Relative frequency (%)	Cumulative relative frequency (%)	Absolute frequency	Relative frequency (%)	Cumulative relative frequency (%)	Absolute frequency	Relative frequency (%)	Cumulative relative frequency (%)	Absolute frequency	Relative frequency (%)	Cumulative relative frequency (%)
1	2078	87.94	87.94	3765	87.33	87.33	3441	89.82	89.82	4921	91.64	91,64
2	190	8.04	95.98	336	7.79	95.13	254	6.63	96.45	308	5.74	97.37
≥3	95	4.02	100.00	210	4.87	100.00	136	3.55	100.00	141	2.63	100.00
Number of different citation sources			2363			4311			3831			5370

Citations that occur at least three times represent the data set for the co-citation analysis. We identify on average 145.5 citations that occur at least three times per classified time period in JASSS. Regarding this measure, the two time spans (10 years vs. 7 years) of the studies are comparable. Within these time spans, however, only the two recent periods are similar in view of the number of citations that occur more than three times (136 and 141). We see an imbalanced distribution of 95 citations vs. 210 citations within the first time span of the previous study. This also results in a denser co-citation network for the second period of the first study (Meyer et al. 2009).

Results of the Citation Analysis

The citation analysis identifies the most cited sources and their characteristics, such as the external publication sources acknowledged in JASSS and corresponding disciplines. We extract the most common sources from the data set for both periods. Table 3 ranks the most cited sources in descending order^[5]. In addition, the relative citation value is calculated as the number of citations divided by the number of JASSS articles published in the corresponding time period. Moreover, we classify the types of sources into books, journal articles, web pages, and proceeding papers.

Table 3: Most frequently cited sources
[Sources represented in both periods are highlighted in bold. Sources in grey boxes were among the most influential sources in Meyer et al. (2009)]

	2008-07/2011				08/2011-2014
Rank	Cited source	Type	Citations	Relative citation value	Cited source	Type	Citations	Relative citation value
1	EPSTEIN, J. & Axtell, R. (1996). Growing Artificial Societies: Social Science from the Bottom Up. Cambridge, MA: MIT Press	Book	16	12.0%	WILENSKY, U. (1999). NetLogo. Center for Connected Learning and Computer-Based Modeling, Northwestern University, Evanston, IL	Web Page	25	15.2%
2	GILBERT, N. & Troitzsch, K. G. (2005). Simulation for the Social Scientist (second ed.). New York, NY: McGraw-Hill International	Book	14	10.5%	GILBERT, N. & Troitzsch, K. G. (2005). Simulation for the Social Scientist (second ed.). New York, NY: McGraw-Hill International	Book	14	8.5%
3	AXELROD, R. (1984). The Evolution of Cooperation. New York, NY: Basic Books	Book	13	9.8%	GILBERT, N. (2008). Agent-Based Models: Quantitative Applications in the Social Sciences, 153. Los Angeles, London: Sage Publications	Book	11	6.7%
4	AXELROD, R. (1986). An evolutionary approach to norms. American Political Science Review, 80(4), 1095-1111	Journal	10	7.5%	GRIMM, V. et al. (2006). A standard protocol for describing individual and agent-based models. Ecological Modelling, 198(1-2), 115-126	Journal	11	6.7%
5	WILENSKY, U. (1999). NetLogo. Center for Connected Learning and Computer-Based Modeling, Northwestern University, Evanston, IL	Web Page	10	7.5%	GRIMM, V., Berger, U., DeAngelis, D. L., Polhill, J. G., Giske, J. & Railsback, S. F. (2010). The ODD protocol: A review and first update. Ecological Modelling, 221(23), 2760-2768	Journal	11	6.7%
6	BARRETEAU, O. et al. (2003). Our companion modelling approach. Journal of Artificial Societies and Social Simulation, 6(2), 1	Journal	9	6.7%	EPSTEIN, J. & Axtell, R. (1996). Growing Artificial Societies: Social Science from the Bottom Up. Cambridge, MA: MIT Press	Book	10	6.1%
7	BONABEAU, E. (2002). Agent-based modeling: Methods and techniques for simulating human systems. PNAS, 99(10), 7280-7287	Journal	9	6.7%	EPSTEIN, J. M. (2008). Why model? Journal of Artificial Societies and Social Simulation, 11(4), 12	Journal	10	6.1%
8	EDMONDS, B. & Hales D. (2003) Replication, replication, and replication: Some hard lessons from model alignment. Journal of Artificial Societies and Social Simulation, 6(4), 11	Journal	9	6.7%	MCPHERSON, M., Smith-Lovin, L. & Cook, J. M. (2001). Birds of a feather: Homophily in social networks. Annual Review of Sociology, 27, 415-444	Journal	10	6.1%
9	GALAN, J. M. & Izquierdo, L. R. (2005). Appearances can be deceiving: Lessons learned re-implementing Axelrod's 'evolutionary approach to norms.' Journal of Artificial Societies and Social Simulation, 8(3), 2	Journal	9	6.7%	SCHELLING, T. C. (1971). Dynamic models of segregation. The Journal of Mathematical Sociology, 1(2), 143-186	Journal	10	6.1%
10	HALES, D., Rouchier, J. & Edmonds, B. (2003). Model-to-model analysis. Journal of Artificial Societies and Social Simulation, 6(4), 5	Journal	9	6.7%	WATTS, D. J., and Strogatz, S. (1998). Collective dynamics of 'small-world' networks. Nature, 393, 440-442	Journal	10	6.1%
11	MACY, M. W. & Flache, A. (2002). Learning dynamics in social dilemmas. PNAS, 99(3), 7229-7236	Journal	9	6.7%	AXELROD, R. (1984). The Evolution of Cooperation. New York, NY: Basic Books	Journal	9	5.5%
12	MOSS, S. & Edmonds, B. (2005). Sociology and simulation: Statistical and qualitative cross-validation. American Journal of Sociology, 110(4), 1095-1131	Journal	9	6.7%	DEFFUANT, G., Amblard, F., Weisbuch, G. & Faure, T. (2002). How can extremism prevail? A study based on the relative agreement model. Journal of Artificial Societies and Social Simulation, 5(4), 1	Journal	9	5.5%
12	SCHELLING, T. C. (1971). Dynamic models of segregation. The Journal of Mathematical Sociology, 1(2), 143-186	Journal	9	6.7%	SCHELLING, T. C. (1978). Micromotives and Macrobehavior. New York, NY: Norton	Book	9	5.5%

The results show three standard books that are cited frequently in both periods: Axelrod’s The Evolution of Cooperation (1984), Epstein and Axtell’s Growing Artificial Societies (1996), and Gilbert and Troitzsch’s Simulation for the Social Scientist (2005)^[6]. In the second period, Nigel Gilbert’s (2008) Agent-Based Models appeared. This book became the third most cited source with 11 citations in the second period, and thereby superseded the highly cited books of Axelrod (1984) and Epstein and Axtell (1996). With the exception of the top three most cited sources, the majority of cited sources are journal articles. It is remarkable that in both periods, six articles were published in JASSS itself.

The NetLogo website became the most referenced source in JASSS with 25 citations. In comparison to the first period, the relative citation value of NetLogo doubled to 15.2%. To understand the reason for citing this source better, we investigated the context in which these 25 NetLogo citations occurred. Twenty articles use NetLogo as a simulation platform for their simulation studies, and the remaining five articles have a clear methodological focus on agent-based modeling and related tools (Bersini 2012; Le Page et al. 2012; Schwarz et al. 2012; Thiele, Kurth & Grimm 2012, 2014). Thus, one can conclude that the simulation tool NetLogo is used in at least 12% of the studies recently published in JASSS^[7].

In addition, there are indicators of another standard that may emerge for agent-based models. In particular, the Overview Design Details (ODD) protocol (Grimm et al. 2006) as well as its review and first update (Grimm et al. 2010) has 6.67% relative citations in the second period. This protocol provides a framework with which to specify and communicate agent-based models (Railsback & Grimm 2011).

We further investigate the types of sources cited in the two periods (see Figure 3). To this end, we classify the citations into nine categories: journal, book, book chapter, proceeding paper, lecture notes, working paper, thesis, web page, and miscellaneous^[8]. The trend reported in the previous study towards more journal article citations continues. In the recent period, 59.8% of the total citations are journal publications, which reflects a continuous increase compared with 36.9% for the first period. Correspondingly, the number of book and book chapter citations declined over time. This also indicates that more relevant journal publications exist that specialize in topics around social simulation, which again hints at a maturation of the field.

JASSS from 1998-2014">

In the previous study, web pages were part of the category “miscellaneous,” but in the subsequent years they have become more cited. For this reason, we introduced “web page” as a new distinct category. Regarding the most cited sources, the increasing percentage of web citations basically results from the acknowledgement of simulation tools such as NetLogo, Repast, and Swarm, which are provided as open-source software via web pages. Articles consequently cite web pages that use these tools. In addition, a number of methodological papers compare simulation methods and tools by referencing the corresponding web pages.

Finally, we investigate whether the multidisciplinary nature of social simulation observed by Meyer et al. (2009) is supported by our citation data. To this end, we identify the 15 most frequently cited journals and their relative citation values based on the total number of 6,197 journal citations between 2008 and 2014 (see Table 4). The ranking shows ten journals, which are represented in both time spans, which may indicates certain stability. Nevertheless, the impact of many journals changed.

JASSS itself is still by far the most cited journal (11.2%). This result is in line with other self-citation rates of journals, which are about 12% (Thomson Reuters 2015). The second and third most cited journals are Nature (2.6%) and Science (1.9%). The list again provides evidence of the multidisciplinary nature of JASSS. The referenced journals cover a broad field of research disciplines among natural scientific journals (e.g., Physica A, Physical Review E, Journal of Theoretical Biology, and Ecological Modeling), social science and economic journals (e.g., American Journal of Sociology and American Economic Review), and a journal related to psychology (Journal of Personality and Social Psychology).

Table 4: The top 15 most frequently cited journals in JASSS from 1998 to 2014
[Journals represented in both time spans are highlighted in grey]

	Journal Rank 1998-2007 (Meyer et al. 2009)		Journal Rank 2008-2014
1	Journal of Artificial Societies and Social Simulation (9.1%)	1	Journal of Artificial Societies and Social Simulation (11.2%)
2	Nature (2.4%)	2	Nature (2.6%)
3	American Economic Review (2.0%)	3	Science (1.9%)
4	Science (1.9%)	4	Ecological Modelling (1.3%)
5	American Journal of Sociology (1.8%)	5	Physica A (1.2%)
6	Physical Review E (1.6%)	6	Physical Review E (1.2%)
7	Physica A (1.5%)	7	American Journal of Sociology (1.1%)
8	Artificial Intelligence (1.4%)	8	Journal of Theoretical Biology (1.0%)
9	American Sociological Review (1.1%)	9	American Economic Review (1.0%)
10	Journal of Personality and Social Psychology (1.1%)	10	Management Science (0.9%)
11	Complexity (1.0%)	11	Computational and Mathematical Organization Theory (0.9%)
12	Computational and Mathematical Organization Theory (1.0%)	12	Journal of Personality and Social Psychology (0.9%)
13	Journal of Economic Dynamics and Control (1.0%)	13	Ecology and Society (0.8%)
14	Computational and Mathematical Organization Theory (1.0%)	14	American Sociological Review (0.7%)
15	The Quarterly Journal of Economics (0.8%)	15	Journal of Conflict Resolution (0.7%)

Against the background that the number of journal citations increased, we expect that journal citations are diverse. We applied the Herfindahl-Hirschman Index (HHI) to provide a concentration measure for journal citations (Schmalensee 1977), which has already been applied in other bibliometric studies (e.g., Chi 2016). Considering the number of journals (N), the share of a journal based on citations (x_i), and the arithmetic average of the shares X, the HHI is calculated as follows:

$$HHI=\sum_{i=1}^N \left(\frac{x_i}{N\ast X}\right)^2$$

The result of the index is proportional to the average market share, and ranges from 1/N to 1. A higher index indicates a concentration of citations. Including the journal citations in the first time span (1998-2007: N=3560) and the journal citations in the second time span (2008-2014: N=6093), the HHIs result in 1.45% and 1.59%, respectively, which indicates a concentration of journal publications. This result is contractionary to our expectation of diversification as well as to the fact that N increases remarkably. The HHI is not invariant to N, as a greater N usually decreases the index. To examine this phenomenon further, we excluded the self-citations of JASSS and recalculated the index. This result shows diversification, with an HHI of 0.63% for the first time span and 0.44% for the second time span^[9]. Thus, the external environment of JASSS diversifies in terms of cited publications outside JASSS, while the high number of self-citations in JASSS points to concentration.

To investigate the multidisciplinary character of JASSS further, we classify all cited journals into subject fields. We conduct the classification by using the list “Essential Science Indicators Subject Areas” provided by Thomson Reuters (2016)^[10]. The list classifies 27,208 journals into a subject field, but some journals cited in JASSS are not covered. No category is assigned to 15.4% of the journals cited in the first time period and 15.2% in the second time period. JASSS is recorded in the category “social science, general.” However, self-references of JASSS are excluded from the analysis in order to focus on the outgoing citations that unambiguously reflect the journal environment.

Based on journal citations, the impact of different subject fields on JASSS is shown in Figure 4. Recently, the category “social science, general” occurred at the top of the list. In both time spans, most journals can be classified into the categories “social science, general” and “economics & business.” The category “psychiatry/psychology” remains the third most influential category. The decreasing influence of computer science-related journals is noticeable, while journals classified as “environment/ecology” gained more impact^[11].

JASSS based on journal citations normalized to 1998-2007">

The diversity of cited journals and subject fields supports JASSS’s self-description as an “interdisciplinary journal for the exploration and understanding of social processes by means of computer simulation” (JASSS 2015). Given its orientation to the method of computer simulation, however, one may expect a stronger acknowledgement of journals in the discipline of computer science (Wellman 2014). However, this assumption is not reflected in the results. The methods of computer simulation are rather briefly referenced by authors. This can be ascribed to the fact that simulation concepts are also partially devTITLEeloped within social sciences (Davidsson 2002). This is different for the externally addressed research topics, mainly related to social science and economics and business.

Overall, the results of the citation analysis empirically support the dynamic and interdisciplinary character of social simulation. Moreover, the shift in publication outlets towards journals continues, which was considered to be an indicator for maturation in the previous study (Meyer et al. 2009). This indication is further supported in this study by the more frequent use of certain tools and standards.

Results of the Co-Citation Analysis

While the citation analysis gives a first impression of the development in the academic community, the co-citation analysis uncovers the structure and interrelations within a discipline. By using a co-citation analysis, we identified clusters and distinct groups within them. All nodes and links that belong to a distinct group are numbered and colored, and these groups typically represent subfields and specific research streams in the discipline^[12]. Figure 5 depicts the results of the co-citation analysis for the period from 2008 to 07/2011. The network has a density of 0.044 and is composed of 368 links and 92 nodes. It consists of two clusters: cluster (1) that comprises seven groups and a separated cluster (2) that consists of a single group.

JASSS co-citation network from 2008 to 07/2011 with CoCit scores ≥ 0.25">

At the center of the first cluster is the group (1.1) Learning in Social Dilemmas^[13]. The hub of this cluster is the publication by Izquierdo, Izquierdo, and Gotts (2008) entitled “Reinforcement Learning Dynamics in Social Dilemmas,” with nine links. The group is composed of 15 nodes, 84 links, and has a density of 0.4^[14]. It is the most connected group within the surrounding groups and is centrally located in the first cluster.

Next, we find two basic classes of topics in this cluster: Social science-related topics and methodological-oriented topics. The groups (1.2) Norms and (1.7) Evolution of Cooperation represent social science topics such as group (1.1) Learning in Social Dilemmas. Topic (1.5) Environmental Aspects is rather separated. The other groups are related to methodological aspects in social simulation, namely (1.3) Modeling, (1.4) Validation, and (1.6) Replication.

The second cluster represents the topic (2) Opinion Dynamics, which can also be classified as related to social sciences. Five topics (Opinion Dynamics, Learning in Social Dilemmas, Norms, Modeling, and Environmental Aspects) have already been identified in the previous study (Meyer et al. 2009). We identify many connected groups in this period. General topics such as Learning, Validation, Replication, and Modeling are relevant for all simulation studies, and thus are acknowledged from many perspectives.

In the most recent period (08/2011 to 2014), we observe a process of differentiation (see Figure 6), and identify six clearly separated clusters and eight groups. This network has a slightly higher density of 0.047, including 297 links and 80 nodes in comparison with the earlier network (2008 to 07/2011). Nevertheless, the groups in the most recent network are more separated.

JASSS co-citation network from 08/2011 to 2014 with CoCit scores ≥ 0.25">

First, we identify some recurrent non-methodological topics in the network. Cluster (1) consists of two content-related groups about the research topics (1.1) Reciprocity and (1.2) Evolution of Cooperation. The central publications in this cluster are “Five Rules for the Evolution of Cooperation” (Nowak 2006) and “The Evolution of Cooperation” (Axelrod & Hamilton 1981), both of which have nine links to other publications in the cluster. The topic Evolution of Cooperation was already identified in the network from 2008-07/2011. In addition, we again found a group related to (4) Opinion Dynamics in the recent network.

Further, new non-methodological topics emerged in the most recent period. The group (3) Marriage Models shows a fully connected network pattern with a density of 0.64, indicating a strong integration of publications. In addition, the new group (2) Simulation of Science fulfills the minimum criteria of three linked nodes to be depicted in the network.

Given the co-citation results, social simulation scientists still vividly discuss methodological topics, as three out of eight identified groups deal with methodological aspects. Methodological topics are represented in the most recent period by the groups (5) Standards, (6.1) Methodology, and (6.2) Tools and Platforms. The scientific discussion about tools and platforms is in line with our result from the citation analysis that NetLogo has become an established tool for social simulation researchers (see Data Set).

After the identification of groups and their structure within the networks, we investigate how the identified groups are linked with each other. Due to the chosen CoCit score of 0.25, not all links between groups are depicted. Thus, a further analysis investigates the aggregated strength of connections between groups. To analyze the strength of the connection between group X and Y with n possible links, we calculated the GroupCoCit score as follows (Meyer, Schaffer & Gmür 2008):

$$GroupCoCit_{XY}=\frac{\sum_1^n CoCitScore}{number\,of\,nodes_X\ast number\,of\,nodes_Y}\ast 100$$

Subsequently, we use the same technique as before to visualize the resulting networks. The result for the period from 2008 to 07/2011 is shown in Figure 7. The average GroupCoCit score is 1.98 and the median is 0.99. Again, we depict only the value of the strongest GroupCoCit scores between the groups with a threshold of 1.0 (all scores are listed in Appendix B).

In line with the strongly interconnected co-citation network, we find strong relations among the methodological groups. The weakest link between the methodological groups (1.4) Validation and (1.6) Replication has a GroupCoCit score of 1.4. The other links have relatively high scores, such as a score of 2.8 between (1.4) Validation and (1.3) Modeling and a score of 3.7 between (1.3) Modeling and (1.6) Replication.

The group (1.1) Learning in Social Dilemmas is centrally located in the network and has six distinct connections to others. Looking more closely at this group, we can identify many publications around general topics such as reinforcement learning (Izquierdo et al. 2008), agent-based models (Epstein 1999), and the prominent publication “Why agents?” by Axtell (2000). Thus, many articles in this group address general issues of agent-based modeling, which is relevant for many research perspectives in social simulation. The centrally located group (1.1) Learning in Social Dilemmas shows no link to the group (2) Opinion Dynamics. This supports our observation that (2) Opinion Dynamics is a rather separated group, while the others are connected.

The network and GroupCoCit scores for the period from 08/2011 to 2014 are depicted in Figure 8. For the most recent period, we identify considerably low values of the GroupCoCit scores with a mean of 0.52 and a median of 0.26. This supports our observation concerning the process of differentiation in the most recent period.

First, we see a triangle among (5) Standards, (6.1) Methodology, and (6.2) Tools and Platforms. As in the previous period, we find strong connections among the methodological topics. The group (6.2) Tools and Platforms is linked with a score of 2.1 to (6.1) Methodology. Similarly strong is the connection between (6.1) Methodology and (5) Standards. A weaker connection exists between (5) Standards and (6.2) Tools and Platforms.

The social science-related group (2) Simulation of Science has no links to the methodological groups. In this group, we identify epistemological articles about the structure and process of academic sciences, which can be considered to be an independent branch in social simulation science. The highest link score of 2.8 has the connection between the topics (1.1) Reciprocity and (1.2) Evolution of Cooperation, which indicates a differentiation process of the topics (1.2) Evolution of Cooperation and (1.1) Reciprocity into two distinct groups.

The two groups (3) Marriage Models and (4) Opinion Dynamics form separate groups that are only connected to each other with a link score of 1.1. (4) Opinion Dynamics was already observed as a rather separated group in the former period. The connection to (3) Marriage Models is mainly driven by the publication “Birds of a feather: Homophily in social networks” (McPherson, Smith-Lovin & Cook 2001) about the homophily concept. This publication is located in the group (4) Opinion Dynamics, but is also co-cited with all eight publications in the marriage group^[15]. This connection between (4) Opinion Dynamics and (3) Marriage Models exemplifies the interdisciplinary use of common methods and concepts in the community, beyond the variety of topics. Thus, social simulation has interdisciplinary characteristics, represented here, for example, by the links between the groups, as well as multidisciplinary characteristics, as shown by distinguishable groups in the co-citation networks.

Longitudinal Analysis of Social Simulation

The co-citation analysis allows us to identify developments in recent years as well as in comparison with the earlier years investigated in Meyer et al. (2009). Given our results of the two analyzed periods, a certain level of stability is observable, while at the same time some issues have been dropped and new topics have emerged. Methodological topics are strongly represented in both periods, as are the topics Opinion Dynamics and Evolution of Cooperation. On the other hand, some topics only emerged recently such as Marriage Models, Simulation of Science, and Tools and Platforms. In contrast, other topics disappeared such as environmental aspects and learning. This illustrates the dynamic processes in JASSS.

Overall, the results of this final analysis suggest that the research topics in the most recent period are more distinguishable and less strongly cross-linked. Methodological issues form their own clusters and groups, while several social science-related issues have evolved over time. The coexistence of methodological and social science-related subjects can be seen as a major pattern emerging from our co-citation analysis.

To foster the interpretation of the longitudinal development, we provide a rank flow chart. Figure 9 highlights the issues discussed in JASSS, as identified by co-citation analyses, along the 17-year editorship of Nigel Gilbert. The topics are ranked according to the number of publications per topic-related group^[16]. For an overview of network metrics within the co-citation networks for all periods, see Appendix C.

JASSS from 1998 to 2014">

Clearly visible is the dominance of Opinion Dynamics as the biggest cluster in the recent three periods. This shows that Opinion Dynamics has developed into a long-term topic for social simulation researchers. Similar is the development of Evolution of Cooperation, which emerged in the last two periods. Alongside this, the development of the topic Reciprocity is closely connected to Evolution of Cooperation.

Overall, three topics were prominent in three periods: Methodology, Opinion Dynamics, and Norms. These topics may constitute social simulation as a discipline. However, the fact that several non-methodological topics dropped, changed, and emerged over time indicates the multidisciplinary character of JASSS and the domain of social simulation. Noticeably, no topic is represented in all four periods.

Methodological issues have occurred in each analyzed period since the foundation of JASSS. Within this, processes of differentiation and consolidation are observable. In the very first years from 1998 to 2002, just a single group about the methodology of social simulation was identified. From 2003 to 2007, two distinct methodological topics emerged: Modeling and Methodology. In the third period from 2008 to 07/2011, we identified the topics replication, validation, and modeling. Recently, in the fourth period from 08/2011 to 2014, three topics cover methodological aspects, which are Methodology, Standards, and Tools and Platforms.

One can speculate whether this reflects the development of the research method simulation in the social sciences. First, general methodological aspects were discussed, informed initially by books such as Gilbert and Troitzsch’s text (2005). Next, issues such as the ghost in the model (Polhill, Izquierdo & Gotts 2004), simulation model alignment (Epstein & Axtell 1996), and the value of replication (Edmonds & Hales 2003) became relevant for the first applications, discussed in the groups Modeling and Methodology in the second period. In the next period, Replication and Validation became such prominent topics that they emerged as groups of their own in the co-citation network. In recent years, the maturity of the method can be recognized by the development of tools, platforms, and standards as prominent topics and thus individual groups.

The topic Environmental Aspects was a big group in the co-citation network of the second period, yet it became a smaller group in the third period and disappeared in the last. There are many simulation studies of environmental aspects such as socio-ecological systems. Possibly, the discussions of environmental aspects shifted to other journals outside JASSS. Here, we identified more referenced journals in environment and ecology in recent years.

The theme Network and Diffusion only emerged in the second period but disappeared thereafter. Still, the citation analysis shows that the publication “Collective dynamics of 'small-world' networks” by Watts and Strogatz (1998) remains one of the most cited publications in JASSS. Thus, the topic Network and Diffusion seems to merge with other fields of social simulation.

Conclusion

This paper investigates the recent development of social simulation as reflected in articles published in JASSS from 2008 to 2014 by means of bibliometric methods. Therby, it offers several theoretical and practical contributions. First, it provides an empirical basis from which to discuss the intellectual structure of JASSS and the related community. The results from the citation and co-citation analysis confirm the continuing multidisciplinary nature of JASSS, which is in line with its self-characterization. Further, the interdisciplinary exchange of knowledge extends beyond the boundaries of single disciplines and can be considered to be a distinctive characteristic of both the journal and the discipline. Furthermore, the citation analysis identifies NetLogo (Wilensky 1999) as the most cited source, which indicates that it has become an important modeling environment in the field of social simulation. The most cited sources as well as the co-citation networks indicate that general methodological issues are vividly discussed by the researchers publishing in JASSS. This result underpins the perceived role of JASSS as an interface for many researchers in the social sciences, who are linked with each other due to their common research interests in method simulation.

Second, in combination with the previous study, changes and continuities can be identified over the longer time span of 17 years. Looking at the most cited sources, several publications persist with high citation values, particularly Axelrod (1984), Epstein and Axtell (1996), and Gilbert and Troitzsch (2005). This result highlights some of the well-established publications in the social simulation community. The trend to acknowledge more journal articles continues, as already identified in the previous study (Meyer et al. 2009). At the same time, some frequently cited book publications in the first years of JASSS lost their relevance. The citation results show that certain standards such as the ODD protocol (Grimm et al. 2006; Grimm et al. 2010) and tools such as NetLogo (Wilensky 1999) are now more frequently cited and thus are becoming increasingly established among researchers in the field. This observation indicates that the field might be evolving into a discipline with shared tools and standards. Along this line, the co-citation analysis depicts two long-term research topics in the field, which are Opinion Dynamics and Evolution. We conclude that these research fields are well-established in JASSS. On the other hand, new topics such as marriage formation and the simulation of science indicate a certain dynamic and openness concerning the main topics discussed in JASSS.

Third, this study may provide an orientation for newcomers to the field. The list of most cited publications gives an overview of the basic literature and illuminates the standards and tools currently in use in the field, such as the ODD protocol and NetLogo. The co-citation networks display the current (and past) topics and their main publications. In combination with the publications citing them, these offer good starting points for individuals seeking to become familiar with the specific topics or the field in general.

As with any study, this paper has limitations. First, we focused on a single journal, while also drawing some more general conclusions regarding social simulation. For a more comprehensive overview of the development of social simulation, more journals and proceedings should be included in the analysis. However, since JASSS is the main journal in the field, it is thus a good indicator for analyzing the general development. Moreover, citation studies suffer from a certain time lag, as it takes some time for publications to appear and to be referenced by other authors. Further, we subdivided the seven-year time span into two equal 3.5 year periods to have a comparable picture about overall development with the previous study. Another subdivision may lead to slightly different results. The analysis also needs to be restricted to the most cited publications for reasons of complexity reduction. Still, we conducted robustness tests with varying CoCit scores, which overall showed similar qualitative results.

Future research could, besides addressing these limitations, use different bibliometric methods such as author co-citation analysis or bibliographic coupling. Further, the set of investigated articles could be extended by using a key word search and related methods to identify relevant papers. Finally, additional insights could be gained from repeating this study in some years to map the next steps in the development of JASSS and social simulation. Still, we hope that this study currently fosters the understanding and acknowledgement of the development of the intellectual structure of JASSS and its related community of social simulation.

References

ALLMAYER, S. & Winkler, H. (2013). Interface management research in supplier–customer relationships: Findings from a citation analysis of international literature. Journal of Business Economics, 83(9), 1015-1061. [doi:10.1007/s11573-013-0685-1]

AXELROD, R. (1984). The Evolution of Cooperation. New York, NY: Basic Books.

AXELROD, R. & Hamilton, W. D. (1981). The evolution of cooperation. Science, 211(4489), 1390-1396. [doi:10.1126/science.7466396]

AXTELL, R. (2000). Why agents? On the varied motivations for agent computing in the social sciences. Center on Social and Economic Dynamics, Retrieved from: http://www.brook.edu/es/dynamics.

BACKHAUS, K., Lügger, K. & Koch, M. (2011). The structure and evolution of business-to-business marketing: A citation and co-citation analysis. Industrial Marketing Management, 40(6), 940-951. [doi:10.1016/j.indmarman.2011.06.024]

BENSMAN, S. J. & Leydesdorff, L. (2009). Definition and identification of journals as bibliographic and subject entities: Librarianship versus ISI Journal Citation Reports methods and their effect on citation measures. Journal of the American Society for Information Science and Technology, 60(6), 1097-1117. [doi:10.1002/asi.21020]

BERSINI, H. (2012). UML for ABM. Journal of Artificial Societies and Social Simulation, 15(1), 9: https://www.jasss.org/15/1/19.html [doi:10.18564/jasss.1897]

BORNMANN, L. & Daniel, H.-D. (2008). What do citation counts measure? A review of studies on citing behavior. Journal of Documentation, 64(1), 45-80. [doi:10.1108/00220410810844150]

CARLEY, K. M., Reminga, J., Storrick, J. & Columbus, D. (2010). ORA user’s guide. Carnegie Mellon: Technical Report CMU-ISR-11.

CHI, P.-S. (2016). Differing disciplinary citation concentration patterns of book and journal literature? Journal of Informetrics, 10(3), 814-829. [doi:10.1016/j.joi.2016.05.005]

CLAUSET, A., Newman, M. E. & Moore, C. (2004). Finding community structure in very large networks. Physical Review E, 70(6), 066111. [doi:10.1103/PhysRevE.70.066111]

DAVIDSSON, P. (2002). Agent based social simulation: A computer science view. Journal of Artificial Societies and Social Simulation, 5(1), 7: https://www.jasss.org/5/1/7.html

EDMONDS, B. & Hales, D. (2003). Replication, replication and replication: Some hard lessons from model alignment. Journal of Artificial Societies and Social Simulation, 6(4), 11: https://www.jasss.org/16/1/11.html

EPSTEIN, J. M. (1999). Agent-based computational models and generative social science. Complexity, 4(5), 41-60. 3.0.CO;2-F" target="_blank">[doi:10.1002/(SICI)1099-0526(199905/06)4:5<41::AID-CPLX9>3.0.CO;2-F]

EPSTEIN, J. M. & Axtell, R. (1996). Growing Artificial Societies: Social Science From the Bottom Up. Washington, DC: Brookings Institution Press.

FIORETTI, G. (2013). Agent-based simulation models in organization science. Organizational Research Methods, 16(2), 227-242. [doi:10.1177/1094428112470006]

GARCÍA-LILLO, F., Úbeda-García, M. & Marco-Lajara, B. (2016). The intellectual structure of research in hospitality management: A literature review using bibliometric methods of the journal international journal of hospitality management. International Journal of Hospitality Management, 52, 121-130. [doi:10.1016/j.ijhm.2015.10.007]

GILBERT, N. (2008). Agent-Based Models. Los Angeles, London, New Delhi, Singapore: Sage. [doi:10.4135/9781412983259]

GILBERT, N. & Troitzsch, K. G. (2005). Simulation for the Social Scientist (second ed.). New York, NY: McGraw-Hill International.

GIPP, B. & Beel, J. (2009). Citation Proximity Analysis (CPA): A new approach for identifying related work based on Co-Citation Analysis. Paper presented at the Proceedings of the 12th International Conference on Scientometrics and Informetrics (ISSI’09).

GMÜR, M. (2003). Co-citation analysis and the search for invisible colleges: A methodological evaluation. Scientometrics, 57(1), 27-57. [doi:10.1023/A:1023619503005]

GREENE, D., Freyne, J., Smyth, B. & Cunningham, P. (2008). An Analysis of Research Themes in the CBR Conference Literature Advances in Case-Based Reasoning (pp. 18-43). Berlin Heidelberg: Springer.

GRIMM, V., Berger U, Bastiansen F, Eliassen S, Ginot V, Giske J, Goss-Custard J, Grand T, Heinz S K, Huse G, Huth A, Jepsen J U, Jørgensen C, Mooij W M, Müller B, Pe'er G, Piou C, Railsback S F, Robbins A M, Robbins M M, Rossmanith E, Rüger N, Strand E, Souissi S, Stillman R A, Vabø R, Visser U and DeAngelis D L (2006). A standard protocol for describing individual-based and agent-based models. Ecological Modelling, 198(1), 115-126. [doi:10.1016/j.ecolmodel.2006.04.023]

GRIMM, V., Berger, U., Deangelis, D. L., Polhill, J. G., Giske, J. & Railsback, S. F. (2010). The ODD protocol: A review and first update. Ecological Modelling, 221(23), 2760-2768. [doi:10.1016/j.ecolmodel.2010.08.019]

IACOBUCCI, D. (1994). Graphs and Matrices. In S. Wasserman & K. Faust (Eds.), Social Network Analysis: Methods and Applications (pp. 92-166). Cambridge: Cambridge University Press. [doi:10.1017/CBO9780511815478.005]

IZQUIERDO, S. S., Izquierdo, L. R. & Gotts, N. M. (2008). Reinforcement learning dynamics in social dilemmas. Journal of Artificial Societies and Social Simulation, 11(2), 1: https://www.jasss.org/11/2/1.html

JASSS. (2015). Journal of Artificial Societies and Social Simulation. Retrieved from https://www.jasss.org/jasss.html

KE, Q., Ferrara, E., Radicchi, F. & Flammini, A. (2015). Defining and identifying sleeping beauties in science. Proceedings of the National Academy of Sciences, 112, 7426–7431. [doi:10.1073/pnas.1424329112]

LE PAGE, C., Becu, N., Bommel, P. & Bousquet, F. (2012). Participatory agent-based simulation for renewable resource management: The role of the cormas simulation platform to nurture a community of practice. Journal of Artificial Societies and Social Simulation, 15(1), 10: https://www.jasss.org/15/1/10.html [doi:10.18564/jasss.1928]

LEITNER, S. & Wall, F. (2015). Simulation-based research in management accounting and control: An illustrative overview. Journal of Management Control, 26(2), 105-129. [doi:10.1007/s00187-015-0209-y]

MCPHERSON, M., Smith-Lovin, L. & Cook, J. M. (2001). Birds of a feather: Homophily in social networks. Annual Review of Sociology, 27, 415-444.

MEYER, M., Lorscheid, I. & Troitzsch, K. G. (2009). The development of social simulation as reflected in the first ten years of jasss: A citation and co-citation analysis. Journal of Artificial Societies and Social Simulation, 12(4), 12: https://www.jasss.org/12/4/12.html

MEYER, M., Schäffer, U. & Gmür, M. (2008). Transfer und Austausch von Wissen in der Accounting-Forschung: Eine Zitations-und Kozitationsanalyse englischsprachiger Accounting-Journals 1990-2004. Schmalenbachs Zeitschrift für Betriebswirtschaftliche Forschung, 60(2), 153. [doi:10.1007/BF03371732]

MEYER, M., Zaggl, M. A. & Carley, K. M. (2011). Measuring CMOT’s intellectual structure and its development. Computational and Mathematical Organization Theory, 17(1), 1-34. [doi:10.1007/s10588-010-9076-0]

MOQRI, M., Bandyopadhyay, S. & Kitchens, B. (2014). Knowledgemap, an Automated classification of Research Areas in a Scientific Discipline. Available at SSRN. [doi:10.2139/ssrn.2523735]

MUSTAFEE, N., Katsaliaki, K. & Fishwick, P. (2014a). Exploring the modelling and simulation knowledge base through journal co-citation analysis. Scientometrics, 98(3), 2145-2159. [doi:10.1007/s11192-013-1136-z]

MUSTAFEE, N., Katsaliaki, K. & Fishwick, P. (2014b). A journal profiling and co-citation study of SIMULATION: the transactions of the society for modeling and simulation international. Paper presented at the Proceedings of the 2014 Annual Simulation Symposium, Society for Computer Simulation International.

NOWAK, M. A. (2006). Five rules for the evolution of cooperation. Science, 314(5805), 1560-1563. [doi:10.1126/science.1133755]

OSAREH, F. (1996a). Bibliometrics, citation analysis and co-citation analysis: A review of literature I. Libri, 46(3), 149-158. [doi:10.1515/libr.1996.46.3.149]

OSAREH, F. (1996b). Bibliometrics, citation analysis and co-citation analysis: A review of literature II. Libri, 46(4), 217-225. [doi:10.1515/libr.1996.46.4.217]

PAN, R. K., Petersen, A. M., Pammolli, F. & Fortunato, S. (2016). The Memory of Science: Inflation, Myopia, and the Knowledge Network. arXiv preprint arXiv:1607.05606.

PAROLO, P. D. B., Pan, R. K., Ghosh, R., Huberman, B. A., Kaski, K. & Fortunato, S. (2015). Attention decay in science. Journal of Informetrics, 9(4), 734-745. [doi:10.1016/j.joi.2015.07.006]

POLHILL, G., Izquierdo, L. & Gotts, N. (2004). The ghost in the model (and other effects of floating point arithmetic). Journal of Artificial Societies and Social Simulation, 8(1), 5: https://www.jasss.org/8/1/5.html

RADICCHI, F. & Castellano, C. (2012). Testing the fairness of citation indicators for comparison across scientific domains: The case of fractional citation counts. Journal of Informetrics, 6(1), 121-130. [doi:10.1016/j.joi.2011.09.002]

RAILSBACK, S. F. & Grimm, V. (2011). Agent-Based and Individual-Based Modeling: A Practical Introduction. New Jersey: Princeton University Press.

SCHÄFFER, U., Nevries, P., Fikus, C. & Meyer, M. (2011). Is finance research a 'normal science'? A bibliometric study of the structure and development of finance research from 1988 to 2007. Schmalenbach Business Review, 63, 189-225.

SCHMALENSEE, R. (1977). Using the H-index of concentration with published data. The Review of Economics and Statistics, 1977, 186-193. [doi:10.2307/1928815]

SCHWARZ, N., Kahlenberg, D., Haase, D. & Seppelt, R. (2012). ABMland-a tool for agent-based model development on urban land use change. Journal of Artificial Societies and Social Simulation, 15(2), 8: https://www.jasss.org/15/2/8.html. [doi:10.18564/jasss.1875]

SECCHI, D. & Seri, R. (2016). Controlling for false negatives in agent-based models: A review of power analysis in organizational research. Computational and Mathematical Organization Theory, 22, 1-28. [doi:10.1007/s10588-016-9218-0]

SIMSOC. (2015). SimSoc mailing list managed by JISCM@il. Retrieved from https://www.jiscmail.ac.uk/cgi-bin/webadmin?A0=SIMSOC.

SMALL, H. (1973). Co‐citation in the scientific literature: A new measure of the relationship between two documents. Journal of the American Society for Information Science, 24(4), 265-269. [doi:10.1002/asi.4630240406]

SQUAZZONI, F. (2010). The impact of agent-based models in the social sciences after 15 years of incursion. History of Economic Ideas, 18(2), 197-233.

SQUAZZONI, F. & Casnici, N. (2013). Is social simulation a social science outstation? A bibliometric analysis of the impact of jasss. Journal of Artificial Societies and Social Simulation, 16(1), 10: https://www.jasss.org/16/1/10.html. [doi:10.18564/jasss.2192]

SQUAZZONI, F., Jager, W. & Edmonds, B. (2014). Social simulation in the social sciences: A brief overview. Social Science Computer Review, 32(3), 279-294. [doi:10.1177/0894439313512975]

THIELE, J. C., Kurth, W. & Grimm, V. (2012). Agent-based modelling: Tools for linking NetLogo and R. Journal of Artificial Societies and Social Simulation, 15(3), 8. [doi:10.18564/jasss.2018]

THIELE, J. C., Kurth, W. & Grimm, V. (2014). Facilitating parameter estimation and sensitivity analysis of agent-based models: A cookbook using NetLogo and R. Journal of Artificial Societies and Social Simulation, 17(3) 11: https://www.jasss.org/17/3/11.html [doi:10.18564/jasss.2503]

THOMSON REUTERS. (2015). Journal self-citation in the journal citation reports - science edition (2002). Retrieved 03-02-2015 http://wokinfo.com/essays/journal-self-citation-jcr/.

THOMSON REUTERS. (2016). ESI Category - Journal Mapping. Retrieved from ipscience-help.thomsonreuters.com/incitesLive/7622-TRS.html.

VAN RAAN, A. F. (2014). Advances in bibliometric analysis: Research performance assessment and science mapping. Bibliometrics. Use and Abuse in the Review of Research Performance, 17-28.

WATTS, D. J. & Strogatz, S. H. (1998). Collective dynamics of ‘small-world’ networks. Nature, 393(6684), 440-442. [doi:10.1038/30918]

WELLMAN, M. P. (2014). Putting the agent in agent-based modeling. Invited talk at AAMAS, Paris.

WILENSKY, U. (1999). NetLogo. Retrieved from http://ccl.northwestern.edu/netlogo/.

ZHANG, J., Chen, C. & Li, J. (2009). Visualizing the intellectual structure with paper-reference matrices. IEEE Transactions on Visualization and Computer Graphics, 15(6), 1153-1160. [doi:10.1109/TVCG.2009.202]