From anecdote to evidence: the relationship between personality and need for cognition of developers

“You have to simulate in your mind how the program’s going to work, and you have to have a complete grasp of how the various pieces of the program work together.” —Bill Gates (Lammers 1986, p. 73)”

There is a considerable body of research that has studied software engineers as “problem solvers,” that seeks to understand how developers make sense of problems and how they solve design puzzles (Petre and Blackwell 1997). The sense of accomplishment from solving problems and programming activities is often seen as a critical characteristic of developers (Almstrum 2003). Similarly, a review article on motivation factors of software engineers concludes that:

“learning, exploring new techniques and problem solving appear to be the motivating aspects of SE. However little work has focused on the specific nature of Software Engineering itself [...] neither did any of the models focus on the nature of the SE’s job itself such as [...] the logical nature of problem solving, use of creativity, complex problem solving, and so on.” Beecham et al. (2008, p. 874)

Beecham et al. suggest that a software engineer’s job involves problem solving and use of creativity—in other words, cognitive tasks. The role of cognition and, in particular, the ability to solve problems in programming has been acknowledged as an important factor in developer productivity since the early days of computing (Curtis 1984). One key psychological attribute for software engineers is the joy of effortful cognitive activities, or what has been termed need for cognition (NfC) (Cacioppo and Petty 1982). The level of need for cognition indicates the extent to which a developer enjoys cognitive effort, which is directly related to problem solving skills (Heppner et al. 1983). Understanding what drives developers’ need for cognition can be useful to software development organizations, for example, in hiring decisions and onboarding new staff (Sharma and Stol 2020). Despite decades of research and an increasing level of attention for human aspects in software engineering (Cruz et al. 2015; Curtis 1984; Lenberg et al. 2015; Storey et al. 2020), need for cognition of software engineers has, as of yet, not been studied (cf. Capretz 2003).

Given the importance of the role that need for cognition plays in software engineers’ problem solving skills, it is surprising that there is a paucity of research that has focused on this construct. Prior studies that focused on software development tasks and developer preferences have sought explanations in developers’ personality traits. For example, studies have linked personality traits to developers’ ability in code review (Da Cunha and Greathead 2007), exploratory testing (Shoaib et al. 2009), object-oriented programming (Cegielski and Hall 2006), and programming proficiency (Evans and Simkin 1989). Similarly, other studies highlight the relation between developer behavior and attitudes, and personality traits. For example, decision-making (Feldt et al. 2010), working attitude (Smith 1989), or preferences for using offline batch versus online time-sharing equipment (Lee and Shneiderman 1978). More recently, Mellblom et al. (2019) found a strong link between neuroticism (one of the personality traits in the Five Factor Model that is also used in this study) and developer burnout. These studies suggest that personality traits theory can be an insightful theoretical perspective to study software developers. Hence, in this study we adopt this perspective as we seek to find empirical evidence, rather than relying on anecdote, to establish developers’ need for cognition. Specifically, we address the following research question:

Research Question: What is the relationship between personality traits of software engineers and their need for cognition?

To answer this question, we conducted a large-scale sample study of almost 500 software professionals. We analyzed this rich data set using Bayesian analysis procedures. Based on the findings, we offer recommendations for software organizations for attracting and retaining software professionals.

This article is structured as follows. Section 2 provides an overview of prior studies of personality traits of software developers. Section 3 presents the research design. Afterward, in Section 4, we present the results of our analysis. Findings with recommendations are discussed in Section 5. Finally, Section 6 concludes the paper and offers suggestions for future research.

Software engineering is a socio-technical field of research (Perry et al. 1994), which seeks to advance the state of practice of software development. While historically the software engineering field has had a strong focus on the technical aspects, there has been a growing interest in studying human factors in software engineering (Boehm and Papaccio 1988; Glass 2002; Lenberg et al. 2015; Feldt et al. 2010; Storey et al. 2020).

To explain developer preferences and behavior, several studies have considered personality traits (Weinberg 1971; Lee and Shneiderman 1978; Shneiderman 1980). For example, Li et al. (2020) provides “18 attributes pertained to engineers’ personalities” by asking developers to rank 54 attributes. Cruz et al. (2015) identified around 90 articles on personality research in software engineering, published between 1970 and 2010. However, the interest for personality studies has surged dramatically in the last 20 years or so. Over 70% of studies reviewed by (Cruz et al. 2015) were published after 2002.

Need for cognition (Cacioppo and Petty 1982) is a critical psychological attribute for software professionals, and related to problem-solving skills (Heppner et al. 1983), which are essential for productive software engineers (Graziotin et al. 2014). We identified only three studies that discussed the need for cognition related to computer applications, though none of them studied software professionals specifically. The most recent study concludes through a small-scale laboratory experiment involving 18 undergraduate students that using a web development learning platform improves students’ need for cognition (Bergande et al. 2020). However, since several previous studies in social psychology through investigations involving a large number of subjects suggest that need for cognition is a stable trait (Furnham and Thorne 2013; Roberts and Mroczek 2008; Bruinsma and Crutzen 2018), we suggest treating the findings of that laboratory experiment with care. Another study by Amichai-Hamburger et al. (2007) tested several hypotheses related to the level of need for cognition and internet use. Findings from a laboratory experiment with 182 participants suggest that people use the same number of hyperlinks regardless of their level of need for cognition; that individuals scoring high in NfC are less influenced by interactive or aesthetic components; and that the time spent on the internet is the same for both groups. Finally, van Nimwegen et al. (2006) conducted a laboratory experiment involving 43 participants and demonstrated that the usability of Graphical User Interfaces does not depend on the users’ NfC. We identified only one study that linked NfC to personality traits (Sadowski and Cogburn 1997), but like the studies discussed above, this study also relied on a sample of participants from the general population rather than software engineers. In this study, we seek to investigate the relationship between personality traits and need for cognition that specifically focuses on software developers. Studying software developers will allow us to draw comparisons with studies using samples of a general population, and to identify specific implications for a software engineering context.

2.1 Personality Traits

Personality can be considered as a set of patterns of thinking, feeling, and behaving, which are based on a set of traits that are predictors of a person’s behavior and action (Corr and Matthews 2009). Karl Gustav Jung first introduced personality traits-based theories in the early 1930s (Jung 1931). These theories were later operationalized employing measurement instruments, such as the Meyers-Briggs type indicators (MBTI) (Myers 1976). At least until 2010, the MBTI was the most widely used personality test within software engineering research (Cruz et al. 2015).

However, the Myers-Briggs classification has been criticized for its lack of validity and utility, such as unstable test-retest reliability and inaccurate predictive validity (Boyle 1995). Moreover, the MBTI is not correlated with other personality scales (Furnham 1996), and has not been validated by empirical research (Druckman and Bjork 1991). Psychology researchers observed that the Meyers-Briggs type indicators do not measure categorically distinct types, generating quasi-random traits assignments (McCrae and Costa 1989). The reason is that the MBTI is based on a dichotomous preference score and not a continuous one, which limits the extent to which personality traits can be measured (Devito 1985).

To address these limitations, McCrae and colleagues developed the Five-Factor Model (FFM), which is grounded in a series of empirical evaluations (Mccrae and Costa 1989, 1992). The FFM is also known as the OCEAN model, an acronym for the five personality traits it measures: Openness (O), Conscientiousness (C), Extraversion (E), Agreeableness (A), and Neuroticism^{Footnote 1} (N). Ashton et al. (2004) have extended the OCEAN model by including an additional trait: Honesty-Humility. This trait emerged from large-scale studies that identified it as a distinct personality trait (Saucier 2009). The HEXACO model has become the new standard in social psychology (Ashton et al. 2014). Replication studies of the OCEAN model (which used more advanced computation techniques that were not available when the OCEAN model was initially proposed) confirmed the validity of this new trait (Ashton 2013).

The characteristics (or facets) of the six personality traits of the HEXACO model are:

1. 1.

   Honesty-Humility (H-H): sincerity, fairness, greed avoidance, and modesty.

2. 2.

   Emotionality (Emo): fearfulness, anxiety, dependence, and sentimentality.

3. 3.

   Extraversion (Ext): social self-esteem, social boldness, sociability, and liveliness.

4. 4.

   Agreeableness (Agr): forgivingness, gentleness, flexibility, and patience.

5. 5.

   Conscientiousness (Con): organization, diligence, perfectionism, and prudence.

6. 6.

   Openness to Experience (OtE): aesthetic appreciation, inquisitiveness, creativity, and unconventionality.

Personality traits can be classified as either ‘bright’ or ‘dark’ (Judge et al. 2009). The HEXACO (as also the OCEAN) model focuses on bright traits, i.e. those traits that are considered “desirable.” High scores (low scores only for Emotional stability, since it measures the tendency to experience danger) indicate bright personality traits.

There are also dark personality traits; that is, high scores on these traits suggest less desirable characteristics. Commonly used in personality studies is the so-called “Dark Triad” comprising Narcissism, Psychopathy, and Machiavellianism, which are defined as follows Paulhus and Williams (2002):

1. 1.

   Narcissism (Nar): is related to the individual gratification through self-centered, selfish, and egoistic behaviors.

2. 2.

   Psychopathy (Psy): anti-social behaviors such as lack of empathy or remorse characterize this trait.

3. 3.

   Machiavellianism (Mac): is named after Niccolo Machiavelli (author of The Prince (Machiavelli 2008)), focusing on the manipulation to gain self-interested goals and readiness to exploit other people to achieve them.

2.2 Need for Cognition

Need for cognition is a psychological attribute advanced in the early 1980s by Cacioppo and Petty (1982), and refers to an individual’s tendency to engage and enjoy effortful thinking. This construct was developed to assess differences among individuals of cognitive processing (Sadowski and Cogburn 1997). Individuals with high scores on this construct have positive attitudes toward problem-solving, reasoning, and abstraction (Cacioppo et al. 1996). People scoring low rely on cognitive heuristics and stereotypes to interpret reality (Leary and Hoyle 2009).

Individual psychological attributes are measurable proxies of behavioral characteristics (Salzberger 2013), such as intelligence and satisfaction with life (Kosinski et al. 2013). Personality traits are also defined as a set of psychological attributes, which are able to predict other attributes, for example cognitive ability (Roberts et al. 2007). In this study we assess the link between personality traits and need for cognition.

It is important to emphasize that need for cognition does not represent a person’s intelligence, although some studies found a positive correlation between these two constructs (Hill et al. 2013). A key difference is that need for cognition only reflects information processing motivations, and not cognitive abilities (Cacioppo et al. 1996). Notwithstanding, need for cognition is related to other attributes, such as cognitive and sensory innovativeness, which refers to the joy to engage in new experiences to “stimulate the mind” (Venkatraman and Price 1990), attributional complexity (i.e., the level of interest or motivation about reality, a preference for complex over simple explanations, meta-cognition capabilities, abstraction skills, and a tendency to infer abstract or complex attributions) (Fletcher et al. 1986), and a high level of tolerance for ambiguity and cognitive flexibility (Petty and Jarvis 1996).

Tuten and Bosnjak (2001) noted that need for cognition is usually correlated with certain personality traits, in particular with openness to experience (defined above). However, scholars have pointed out that need for cognition is a theoretically distinct construct: an individual might be open to new experiences without reflecting on the consequences of such new experiences (Kearney et al. 2009). On the other hand, introverts who are reluctant to new experiences might enjoy thinking in a reassuring environment, such as reading an interesting book in the comfort of their home. In fact, extraversion (the tendency to enjoy human interactions) is not related to need for cognition (Sadowski and Cogburn 1997). Therefore, regardless of the fact whether personality traits are correlated or not with need of cognition, all those personality attributes have unique meanings and are thus, considered separately (McCrae 1996).

Need for cognition is also a relevant predictor of positive behaviors in working environments. Research suggests that a high level of need for cognition is a robust negative moderator of social loafing (the tendency to exert less effort when working in groups), showing significantly higher team efforts (Smith et al. 2001). Further, people with high need for cognition values tend to exhibit higher job satisfaction if they face complex engineering tasks, and are generally more satisfied with their jobs than employees scoring low (Park et al. 2008). Finally, the need for cognition reduces depressed moods at work (Gallagher 2012).

Anecdotal evidence suggests that software engineers would score high on need for cognition, but there are no studies that have empirically investigated this. Given the socio-technical nature of software development and the importance of human aspects in software engineering, we set out a large-scale study to investigate need for cognition among software engineers.

To investigate the need for cognition attribute of software developers, we conducted a large-scale sample study. Sample survey research is highly suitable to seek answers that can be generalized to a larger population (Stol and Fitzgerald 2018). In this study we seek to understand the relationship between personality traits and need for cognition of software engineers, considering both bright and dark personality traits.

3.1 Measurement Instrument

All constructs in the study were defined as latent variables which we measured using existing instruments that we adopted from the psychology literature. To measure the bright personality traits, we adopted the HEXACO model of personality (Ashton et al. 2004). While different survey instruments exist for measuring traits, we adopted the Brief HEXACO Inventory (BHI), which consists of 24 questions (De Vries 2013). This instrument has shown a high level of stability and accuracy; moreover, it correlates well with other similar and longer instruments (e.g., the HEXACO-PI-R) (De Vries 2013). To measure the dark traits, we adopted the so-called “Dirty Dozen” (Jonason and Webster 2010), which is also a widely used survey instrument in psychology research (Rauthmann and Kolar 2012). Need for cognition was measured using the NCS-6 instrument, which is a parsimonious instrument with six items (Lins de Holanda Coelho et al. 2018). We followed the instrument guidelines (which indicate which items are reverse-coded). We calculated factor scores (which represent a respondent’s score on a particular trait) using the mean function; which is a standard procedure when using frequently used instruments such as this (Weiner and Greene 2017).

3.2 Data Collection and Screening

To collect data we used the Prolific data collection platform, which is a dedicated platform for academic research (www.prolific.co) with over 88,000 active members worldwide at the time of the survey. The use of Prolific for data collection has several advantages over other data collection procedures (for example, invitations through mailing lists), such as reliability, replicability, and data quality (Peer et al. 2017; Palan and Schitter 2018). Prolific is widely used in other research fields such as economics (Marreiros et al. 2017), psychology (Callan et al. 2017), and food science (Simmonds et al. 2018). More recently, studies within computer science and software engineering have adopted the Prolific as a data collection platform (Hosio et al. 2020; van Berkel et al. 2021; Russo et al. 2021a, 2021b, 2021c; Russo 2021, Cucolas and Russo 2021). To implement the survey questionnaire, we used Qualtrics (www.qualtrics.com). To minimize response bias, we randomized questions within their blocks (Paulhus 1991; Gravetter and Forzano 2018).

The choice of sampling strategy is key for sample research. Baltes and Ralph (2020) recently suggested that most studies in software engineering research use non-probability sampling, such as purposive or convenience sampling, and less than 10% use random sampling. We selected a cluster-randomized sampling strategy (Gravetter and Forzano 2018): a number of potential respondents were selected at random from the Prolific population. In other words, we did not selected the entire world population, but just a cluster of it (i.e., the Prolific community). From our cluster we proceeded to identify software professionals through a multi-stage selection process described below. Compared to random sampling, this is less precise (since only a portion of the population is considered, in this case, that portion which has a presence on Prolific), but it is very cost-effective compared to the other strategies. To be sure to include only software developers in our sample, we ran a three-step screening process as detailed below.

Step 1. Pre-Screening

The first step was a pre-screening of the active members on Prolific based on some general criteria related to software development, such as knowledge of software development techniques, doing computer programming for a living, and the use technology at work. Further, we required a Prolific platform approval rate of 100%, suggesting that all included respondents successfully completed prior surveys (and did not fail any screening criteria in previous surveys). After pre-screening, our sample included 2,897 people.

Step 2. Competence Screening

In the second step we invited candidates who self-identified as software engineers to take part in a competence screening to ensure that they were sufficiently knowledgeable in the subject matter. We asked three multiple-choice questions,^{Footnote 2} one about software design and two about programming, and also time-boxed the responses within three minutes to avoid suspicious behavior and cheating. At the end of this stage, we had a sample of 514 participants.

Step 3. Quality Screening

At this point we administered the survey to the remaining participants, but included a quality screening to ensure high-quality data. The quality screening comprised three attention checks; participants who did not pass these checks were excluded from the sample. As a final result of the entire screening process, we received 483 valid and complete responses that could be used for analysis.

Ethical issues were considered according to the recommendations in the Declaration of Helsinki (General Assembly of the World Medical Association 2014). Participants were identified only through a user ID provided by the Prolific data collection platform. We did not collect any potentially sensitive information of participants. Further, participants were free to withdraw their participation at any time up to the point of submitting, and some (n= 306) decided not to participate to this investigation although invited. Finally, several of the authors have had research ethics and integrity training, which was specifically tailored to behavioral sciences.

3.3 Sample Description

The final sample included 483 complete responses, representing the largest sample for a personality research study in software engineering (Cruz et al. 2015). In addition to data on personality traits, we also collected demographic data. Respondents were between 18 and 68 years of age, with an average of 33 years (see Fig. 1). Table 1 shows that almost 19% of respondents were women. Most of the respondents (ca. 93%) were born in Western countries. Table 2 lists the 15 most frequent countries of origin (representing approximately 90% of all subjects). Almost three-quarters of the selected professionals have at least one university degree (Table 3).

Distribution of participants’ age

Full size image

We conducted the analyses using several software packages. After collecting the data, we performed several sanity checks using IBM SPSS (version 26).

The primary analyses were conducted using the JASP (version 0.11.1) statistical package (JASP Team 2018). JASP is a relatively new open source package that is actively being developed, and seeks to offer a user-friendly interface to conduct Bayesian analyses. We modeled personality traits as exogenous variables (or predictors) and need for cognition as the endogenous (or criterion) variable (Table 4).

The interest in Bayesian statistics is rising in the empirical software engineering research community (Furia et al. 2017, 2019, 2021). One reason for this is an increasing awareness of the limitations of ‘traditional’ frequentist statistics, which relies on p-values and significance testing of null-hypotheses (Cohen 1994; Baker 2016). By convention, p-values are considered as a measure of evidence in favor (or against) of a null hypothesis. Small p-values indicate strong evidence against the null hypothesis, and p-values lower than 0.05 are widely considered as the critical threshold that determines when we have sufficiently strong evidence against the null hypothesis. Nevertheless, p-values cannot be interpreted as the probability that the null hypothesis is true. What they provide is the probability of having data at least as extreme as the one observed, if the null hypothesis was indeed true.^{Footnote 3} Consequently, the use of p-values as a measure of strength of the evidence to support any claims must be done with caution (Nuzzo 2014). In recent years, scholars of statistics have stressed these arguments and advocated for the use of Bayesian approaches instead of frequentist ones, since the former can offer a more comprehensible, nuanced understanding of the statistical data analysis (Wagenmakers et al. 2018; Dienes 2014; Lee and Vanpaemel 2018; Furia et al. 2019; van Doorn et al. 2021).

Bayesian statistics are not based on a dichotomous evaluation mode (i.e., hypotheses are not accepted or rejected). Instead, the Bayesian paradigm accepts that different explanations may be possible and that evidence favoring one or another hypothesis can be properly quantified. Bayesian methods allow the computing of the probability that a given hypothesis is true.^{Footnote 4} This probability is the so-called Bayesian posterior probability, which is denoted as P(H—data). This probability quantifies the strength of our evidence in favor of H. Bayesian posterior probabilities are computed by applying the Bayes rule over the following two quantities: the observed data’s prior probability and the likelihood. Prior probabilities allow us to integrate any prior evidence that we have in favor of a given hypothesis. However, non-informative (usually uniform) probabilities can also be used when little or no prior information is available. The likelihood of the data can be easily understood in terms of Bayes factors. A Bayes factor, defined for two competing hypotheses and noted as BF₁₀, represents the observed evidence in favor of H₁ over H₀.^{Footnote 5} Table 5 presents heuristics to interpret Bayes factors. Rather than delineating explicit thresholds or cut-off values, beyond which a conclusion is supported, as is typical in frequentist statistics, Bayes factors should be interpreted more qualitatively. For example, an analysis resulting in a Bayes factor BF₁₀ of, say, 5, suggests moderate evidence for H₁ over H₀. If the Bayes factor is 50, on the other hand, the heuristics suggest that there is robust evidence for H₁ over H₀. Consequently, the ‘thresholds’ in Table 5 should not be interpreted strictly but rather heuristically, as a continuum representing strength of evidence. Finally, the following equation described how Bayesian posterior probabilities, prior probabilities, and Bayes factors relate:

For example, if H₁ and H₀ are, a priori, equally probable (i.e., \(\frac {P(H_{1})}{P(H_{0})}=1\)), but the observed data suggests strong evidence for H₁ with a BF₁₀ equal to 20, then we have that \(\frac {P(H_{1}|D)}{P(H_{0}|D)} = 20\). And, solving the equation, we have that P(H₁—D)=\(\frac {20}{1+20}\) ≈ 0.95 and P(H₀—D)=\(\frac {1}{1+20}\) ≈ 0.05.

One of the critical advantages of Bayesian statistics is that the above reasoning can be naturally extended to multiple hypotheses or to more complex settings like multi-model linear regression (as we do in the next section). Because, in Bayesian statistics, all models and observations are equally treated as random variables. So, all computations are always done following standard probability operations, like the ones described above.

4.1 Bayesian multi-model linear regression

In this study we use a Bayesian multi-model linear regression. Under this model, we assume that the nine personality traits studied here (a.k.a. the independent variables) have a linear relationship with need for cognition (a.k.a. the dependent variable). But, crucially, this analysis assumes that not necessarily all the personality traits are needed to predict developers’ need for cognition. In consequence, we explore all regression models defined by all possible combinations of the different personality traits. As we are considering nine traits, this analysis explores 2⁹= 512 alternative regressions models. Each model is treated as a different hypothesis.

We followed the standard guidelines reported by van Doorn et al. (2021) and van den Bergh et al. (2021) for conducting the Bayesian data analysis. We first performed several checks to verify that the use of a linear regression model to study the relationship between the considered personality traits and need for cognition is a reasonable choice. Following van den Bergh et al. (2021), we assessed the assumptions of normality, linearity, homoscedasticity, and the absence of multicollinearity. We discuss the procedures for assessing these next.

Data Normality

We first evaluated data normality. Residuals can be seen as the differences between the observed value of need for cognition (i.e., the endogenous variable) and the personality trait values (i.e., the exogenous variables). Figure 2 presents a normal predicted probability plot (P-P plot), which shows that the residuals are normally distributed since they are reasonably aligned to the diagonal normality line.

Normal P-P plot of regression standardized residuals for need for cognition

Full size image

Homoscedasticity

Second, we evaluated homoscedasticity by checking whether the residuals are equally distributed. Heteroscedasticity means that residuals are unevenly spread, leading to clustering on some values. This is undesirable as it would mean that the variance of the observations is not equal. The scatterplot shown in Fig. 3 shows that the residuals are randomly distributed.

Scatterplot of the residuals to assess homoscedasticity for need for cognition

Full size image

Since our residuals are normally distributed and homoscedastic, we can also assume that the exogenous variables (i.e., personality traits) in the regression have a straight-line relationship with the endogenous variable (i.e., need for cognition), concluding that the data have a linear relationship. It is linear because the residuals appear to be linear since we do not see any curved pattern in Fig. 3.

Multicollinearity

Finally, we checked for multicollinearity; that is, we evaluated whether the exogenous variables are correlated with each other. This step is essential because our primary goal is to isolate the relationship between the endogenous variable and each of the exogenous variables. If the exogenous variables are correlated, changes in one of them will also influence another, leading to unreliable results. A standard measure for assessing multicollinearity is the variance inflation factor (VIF). A commonly accepted cut-off for the VIF is 5, but a more conservative cut-off is 3 (Miles 2014; O’Brien 2007). Table 6 shows that all VIF values are below two, well below the cut-off value. Hence, we are confident that our regression model will associate the variance of the endogenous variable with the exogenous ones.

Another key step in a Bayesian analysis is the specification of a prior (van Doorn et al. 2021; van den Bergh et al. 2021; Furia et al. 2021). In our case, the prior of a Bayesian multi-model regression is composed of two parts. A first part refers to the prior over the 512 different linear regression models explored in this analysis. In our case, we decided to use a uniform prior (i.e. all models have the same prior probability). Eventually, a non-uniform prior could be employed encoding some previously available information about the relevance of some of our personality traits for predicting need for cognition. However, we did not find a sensible way to define a precise probability value for all (or some of) the personality traits reflecting the information present in the literature.

The second part of the prior in a Bayesian multi-model regression refers to the prior over the linear coefficients associated to each of the independent variables (i.e. the personality traits). For example, a negative linear coefficient for a given personality trait indicates an negative correlation between this personality trait and need for cognition. Finding a prior for these continuous values is not as easy as in the case of the different models. Following the guidelines by van Doorn et al. (2021) and van den Bergh et al. (2021), we considered four different priors which are commonly used in the Bayesian literature for the coefficients of a linear regression model (Liang et al. 2008; Rouder and Morey 2012). By using different priors, we can assess the sensitivity of our analysis to the choice of the prior, which is a standard robustness check in Bayesian data analysis (van Doorn et al. 2021; van den Bergh et al. 2021). In the following section, we report the average plus/minus the standard deviation of the different computations, such as the Bayes Factors and the posterior probabilities, obtained by repeating the whole data analysis using a different prior at a time. The standard deviations will help us to identify the sensitivity of the Bayesian analysis to the choice of the prior. Small standard deviation values, in comparison to the mean value, suggests that the results do not vary much across different priors, and thus that the prior of choice does not have an impact on the results that would raise any concerns as to the sensitivity of the prior of choice. On the other hand, large standard deviation values, in comparison to the mean value, suggest that the choice of prior does have an impact on the results, and this should be reason for concern, because the results depend on a specific prior.

The results of the various computations are included in the Supplementary Material at the end of this article. In Appendix A, we provide full details about these priors used and how to reproduced the analysis shown in this paper using JASP.

4.2 Results

We now present the results of our analysis. We establish (i) which personality traits best predict developers’ need for cognition; (ii) how much developers’ personality traits contribute in explaining their need for cognition; and (iii) whether or not software engineers have a higher need for cognition compared to a general population.

The result of the ten best regression models out the 512 models considered in this Bayesian analysis are summarized in Table 7. The table shows P(H—data) which is the posterior probability of each model/hypothesis after observing the data, the Bayes factor (BF₁₀), and the explained variance R². As noted, Table 5 offers heuristics to interpret Bayes factors. We note that JASP fixes as the reference model (H₀ following previous notation) the model on top of the list (this is because BF₁₀ is equal to one in this case). The other models have Bayes factors smaller than one, showing that they are all less supported by the data compared to the reference model. Nevertheless, as can be seen, no model has an overwhelming advantage over the others. This suggests that any of these subsets of personality traits explains NfC similarly well. Based on the R² values, we conclude that personality traits explain approximately one-third of the variation in need for cognition of software engineers in our sample.

As discussed in the previous section, Table 7 reports the average value plus/minus the standard deviation of the different quantities over four analysis, each of them performed with a different prior. As previously discussed, the standard deviations help us to identify the sensitivity of the Bayesian analysis to the choice of the prior. As can be seen, standard deviations values are very small and, in consequence, the main conclusions of this analysis are not affected by the particular choice of the prior.

If we jointly consider the different models shown in Table 7, several traits appear in most of them. Table 8 lists the individual probabilities of each trait to be relevant for predicting need for cognition. The inclusion probability of a given trait is computed by summing up (i.e., marginalizing) the posterior probabilities of all of the models where this trait is included (all the 512 evaluated models are considered here, not only the ten ones shown in Table 7). Whereas Table 7 considers traits only in combinations, Table 8 presents information on the likelihood of individual traits to be needed to predict the developer’s need for cognition. Note that, as in the previous table, we report average values plus/minus the standard deviation in order to evaluate the sensitivity of the choice of the prior.

As a result of this Bayesian analysis, four traits in particular have an extremely high chance of almost 100% to be needed to predict need for cognition, namely: openness to experience (OtE), conscientiousness (Con), honesty-humility (H-H), and emotionality (Emo). This is reflected in the probability of inclusion in the regression model after observing the data (i.e., P(incl—data)), with values of 100% (with the only exception being emotionality which has a probability of 96.3%). Note that in this analysis, we assumed that a priori (i.e., before considering the data from the survey), all traits had the same chance of 50% to be relevant for predicting need for cognition (i.e., P(incl)= 0.5).

The next question is whether these personality traits are positively, negatively, or not correlated with the need for cognition. Table 9 presents the Pearson coefficients that represent the relations between need for cognition and personality traits. Based on the correlation using Pearson’s r, we can see the sign (negative or positive) of the individual relation but not the strength of the evidence supporting this relationship. To address this issue, we also compute the Bayes factor for a model where no relationship is established between the trait and need for cognition. In this case, the specification of the prior is different from the case of the Bayesian multi-model regression because we now use a simple correlation model. Again, we consider four different priors and report average plus/minus standard deviations for the quantities depending on the prior. In this case, the standard deviations of some of the Bayes factors are, at first sight, relatively high, but we should note that in these cases, Bayes factors always vary within a range of extremely supportive evidence according to the ranges provided in Table 5. This effect can be appreciated when looking at the associated posterior probabilities, which can be computed from the Bayes factors, because, in those cases, the associated standard deviations become negligible. Again, we can see that the main conclusions of this analysis are not affected by the particular choice of the prior. In Appendix A, we provide more details about these priors and about how to reproduce this analysis using JASP.

In particular, we can see how dark traits (i.e., psychopathy, machiavellianism, and narcissism) are negatively associated with need for cognition, although those associations are barely significant (i.e., they have negative Pearson’s r coefficients but very small Bayes factors). Honesty-humility, extraversion, conscientiousness, and openness to experience are significantly positively associated with need for cognition (i.e., they have positive Pearson’s r coefficients and large Bayes factors). Also, emotionality is strongly negatively associated to NfC. However, according to this analysis, we can not infer whether agreeableness is associated to NfC in a significant way.

Finally, we assess the level of need for cognition for software engineers through an analysis of variance (ANOVA). To do so, we consider the descriptive statistics in Table 10 and the boxplot in Fig. 4. To interpret these values, we compare these to findings by Popoviciu et al. (2011), who observed a mean value of 3.97 (on a scale of 1-5), suggesting that software professionals are “very likely to engage in and enjoy effortful cognitive activities.”

Distribution of software professionals’ need for cognition

Full size image

We now compare those values with previous studies reporting scores of need for cognition, summarized in Table 11. Based on the reported mean and standard deviation (SD) values of need for cognition, as reported by Kearney et al. (2009) and Madrid and Patterson (2016), we replicate the prior research data using two normal distributions (we could not replicate (Popoviciu et al. 2011), since the standard deviation was not reported). We subsequently use these distributions to compare prior work to our results. First, we ran a one-way ANOVA comparing need for cognition between these two studies and our results. This shows a significant effect of the study sample on need for cognition (F (2) = 38.16, p < 0.01). The ANOVA confirms that there is an overall significant difference among groups; however, it does not provide any information about which group scores significantly higher in need for cognition. Therefore, we performed a two-way Tukey posthoc comparison between the studies. This test is a widespread posthoc analysis based on the assumption of normally distributed data.

This analysis shows that need for cognition in our study is significantly higher, compared to Kearney et al. (2009) (p < 0.001) and Madrid and Patterson (2016) (p = 0.001). Figure 5 shows the boxplot distribution for need for cognition scores. We conclude, thus, that software engineers report a higher need for cognition compared to other professionals.

Boxplot distribution of need for cognition studies

Full size image

Upon closer inspection of need for cognition, Table 10 shows that the mean value is high, close to four (3.97) on a scale of one to five. Also, the descriptive statistics of the variables suggest that need for cognition is relatively skewed towards high values. The boxplot shown in Fig. 4 confirms this, where both upper and lower quartiles are close to the median value of four, with very few outliers below the lower whisker. Accordingly, we conclude that software engineers have a significantly higher need for cognition than other knowledge workers.

5.1 Discussion of Findings

Based on the results of our analysis presented in the previous section, we can now address our central research question: What is the relationship between the personality traits of software engineers and their need for cognition?

Following increasing attention to human factors in general and personality traits in particular, this paper presents an investigation that links personality traits and the need for cognition among software professionals. Psychology literature suggests that personality is stable over time; i.e., it does not overall change along the maturation process of people (Cobb-Clark and Schurer 2012; Gustavsson et al. 1997; Blonigen et al. 2008). This stability of personality has also been confirmed for software developers (Calefato et al. 2019). Need for cognition has also been identified as a stable psychological attribute (Furnham and Thorne 2013; Roberts and Mroczek 2008). A longitudinal study confirmed this stability over time^{Footnote 6} (Bruinsma and Crutzen 2018).

This evidence allows us to make reliable predictions because all traits included in this study are stable and not time-dependent. Therefore, we can conclude that software engineers have a higher need for cognition for the general population, which is predicted by their defining characteristics, i.e., personality traits. Although we did not investigate causality, we found that personality traits are good predictors of developers’ high need for cognition. Moreover, since both bright and dark personality traits and the need for cognition are stable over time, it suggests that the predictive relationship is also stable over time.

Anecdotal evidence has suggested that software engineers must enjoy effortful thinking to be good developers (Petre and Blackwell 1997). This investigation aims to move from anecdote to evidence through a large-scale empirical study. Operationalizing the joy of thinking with the need for cognition (as shown by Strobel et al. 2018) brings several advantages to the community’s body of knowledge because it allows us to infer previous evidence on the need for cognition with tailored recommendations for software professionals.

We now focus our discussion on the regression model and the role of each trait. From the posteriors of Table 8, we can see that four traits have a probability of inclusion that is close to 100%, namely openness to experience, conscientiousness, honesty-humility, and emotionality. These are the only four traits that have a significant BF_inclusion (i.e., higher than ten (Lee and Wagenmakers 2014)), and are present in all ten of the most likely regression models listed in Table 7. To recall, we reported only the ten best models of the Bayesian regression that showed the greater likelihood to predict our dependent variable (i.e., Need for Cognition). In particular, based on the correlation signs of Table 9, we conclude that high openness to experience, high conscientiousness, high honesty-humility, and low emotionality predict a high level of need for cognition. Similar patterns have also been observed in a previous study (Sadowski and Cogburn 1997).

Prior studies of personality traits in software engineering are limited to only bright personality traits (Cruz et al. 2015). This study is among the first to include dark personality traits.^{Footnote 7} The inclusion of dark traits offers a more comprehensive and holistic understanding of personality traits of software engineers and, in this study, in particular, how these relate to developers’ need for cognition. However, Table 9 shows that none of the three dark traits, machiavellianism, psychopathy, and narcissism, are significantly correlated to the need for cognition (with a BF₁₀ close to 0). We therefore conclude that bright traits play a much greater role in predicting the need for cognition than dark traits. Furthermore, the BF_inclusion values of these dark traits are also relatively low. This suggests that dark traits do not play a significant role in the need for the cognition of software professionals.

Additionally, our ANOVA analysis confirmed that software professionals score significantly higher in their need for cognition compared to the general population. Need for cognition seems to be one unique characteristic of developers.

Finally, we also highlight that personality traits predict one-third of the need for cognition software engineers.

This evidence has important implications for both practice and research discussed next.

5.2 Implications for Research and Practice

Table 12 summarizes the key findings of our study and provides recommendations for software organizations. First, our study finds that software engineers score, on average, high on the construct need for cognition. Second, in seeking to predict need for cognition with personality traits, we find that the latter account for ca. 33% of the variation in need for cognition. This is a crucial finding of this study and has several implications for research and practice.

Recruitment strategies can be better tailored for software engineers. A study showed that affective (i.e., feeling-based) recruitment messages on company websites are the most attractive for a general population (Kraichy and Chapman 2014). However, for an individual with a high need for cognition, such messages are rather uninteresting. The same research showed that software professionals are more receptive to cognitive (i.e., thought-based) recruiting messages. From a practical perspective, software organizations should not devote resources to create an emotional link between themselves and candidates. Instead, it should focus on communicating fairly upon reasons why they are attractive to future employees. One example is to describe in recruitment advertisements work-life benefits since they positively influence applicant perceptions of the software company (Firfiray and Mayo 2017).

Moreover, regarding software teaming, the need for cognition predicts team performance (Kearney et al. 2009). According to Kearney et al., a team with a high need for cognition can better process task-relevant information. For example, such a team will better understand the customer’s needs of the technology stack they are supposed to work on. Also, the collaborative team identification will be higher since the team cohesion will not depend on emotions but cognitive motivations. These two points are significant predictors of how a team can perform, i.e., delivering quality software.

Further, a high need for cognition of software engineers likely influences also working behaviors. The working ethic of an individual with a high need for cognition is above-average (Singer et al. 1998). Research by Singer et al. highlighted employees’ high level of empathy with a high need for cognition with potential victims (i.e., colleagues who have been unjustly treated) of their organizations, which predicted the likelihood of whistleblowing behavior and harmful consequences for the company. Similarly, Pohling et al. (2019) suggests that individuals’ high need for cognition increases active cooperation and pro-social behaviors within organizations. Thus, if developers’ working environment is perceived as unfair, unwelcome, or toxic by developers, they will likely start looking for another workplace and increase their whistleblowing intention (Valentine and Godkin 2019). Consequently, an organizational culture based on fairness, transparency, and accountability is considered very important for subjects high in need of cognition, such as software professionals. Accordingly, software companies that embrace such organizational behaviors will more likely experience a higher sense of belonging and community of their employees.

Likewise, although individuals such as software engineers are open to change, they do not follow it without questioning it (Haugtvedt and Petty 1992). They tend to have a critical approach, especially towards contradictory or ill-defined problems. Therefore, whatever changes a software organization might be willing to introduce, it must ensure that such changes have been communicated sufficiently precisely so that software engineers understand the need and benefits of such change; otherwise, they will be reluctant to change stick to old habits.

Another appealing implication to bear in mind is a high degree of creativity at work (Madrid and Patterson 2016). Cognitive-laden individuals are very likely to generate new ideas. Interestingly, as Madrid and Patterson (2016) found, a fair working environment leads to novel idea implementation. These findings are essential in creative jobs, where employees have to develop new solutions, such as software development. A fair and transparent organizational culture plays an essential role in delivering customers with novel solutions. Similarly, Wu et al. reported that the need for cognition is an antecedent of individual innovation behaviors. In other words, the authors suggest that the high need for cognition explains innovation behaviors. This results in a higher degree of job autonomy and the ability to better handle time pressure (Wu et al. 2014). It is also worth mentioning that people scoring high on need for cognition are less prone to the so-called primacy effect: they recall presented information comprehensively and uniformly and are not biased by the first items of a list, showing a higher mental flexibility (Ahlering and Parker 1989).

5.3 Threats to Validity

We now discuss threats to validity following Gren (2018)’s validity concerns for behavioral software engineering studies.

Reliability. We only used stable and reliable measurement instruments that have been validated by social psychology scholars and used in a large number of prior studies. For example, HEXACO has been used in almost 1,000 studies.^{Footnote 8}

Construct validity

Our three measurement instruments reflect the constructs for which they have been developed. Constructs are well-established in the scholarly debate. Instruments have been refined and improved over time. We acknowledge having used short versions of the measurement instruments, which were slightly less accurate. Nonetheless, all three instruments can estimate with high accuracy more extended inventories. This permitted us to be parsimonious and to use multiple psychometric models (i.e., bright, dark traits, and the need for the cognition of software developers), with a high engagement rate and a low drop-out ratio in the study.

Conclusion validity

Bayesian statistics help us to gather a more comprehensive understanding of the subject matter, overcoming the typical shortcomings of p-values-based findings of frequentist null-hypothesis significance testing. Conclusions are drawn based on the odds related to the best predictive model and not just by answering yes or no to a specific hypothesis. We also stress the fact that we did not observe the discussed implications directly. Instead, we inferred our findings with relevant literature on the subject matter. Accordingly, we developed four propositions that need further investigation. We artificially generated normal distributions of previous studies to compute the ANOVA since we did not have access to such studies’ data for replication.

External validity

External validity is a crucial concern when conducting sample studies, which seek to maximize the potential for generalizability (Stol and Fitzgerald 2018). In order to establish a sample size that is representative of the software developer population, we relied on Yamane (1973)’s formula. While there are several estimates of the size of the global software developer pool (Russo and Stol 2020), we used a conservative (i.e., the highest) estimate, which was approximately 36.5 million at the time of our study. Following Yamane’s formula, a sample of ca. 400 was suggested. Our sample is well above that minimum.

We put the highest care in our multi-stage data collection process using a cluster-randomized sampling strategy, through which we gathered 483 validated responses. Such a multi-stage process aimed to identify software professionals in our cluster. As a cluster, we used the academic crowdsourcing platform Prolific. Although cluster-randomized sampling is considered less efficient than standard-randomized sampling, it provides a fair representation of a population and is very cost-effective (Gravetter and Forzano 2018). This approach also follows the most recent recommendations for sampling strategies in software engineering research (Baltes and Ralph 2020).

The individual motivation to pursue a software engineering career is a puzzling issue to address since there are so many confounding factors that are very hard to control, e.g., the socio-economic background, individual experiences, and aspirations. Nevertheless, some specific psychological characteristics are intuitively linked to any software professional, such as the joy for problem-solving and, more generally, thinking. Therefore, we used in this study need for cognition (i.e., a tendency to engage and enjoy effortful thinking Cacioppo and Petty 1982) as a regressor of individual defining or innate characteristics, namely personality traits. Our analysis shows that the personality traits of software engineers have a considerable explanatory power with a coefficient of determination (R²) of ca. 0.33 of the need for cognition. Thus, individuals who have a high need for cognition are more likely to start a career in software engineering than those who do not. This finding also has several managerial propositions for software organizations since software professionals are analytical subjects, likely prefer cognitive communication with a transparent, accountable, and fair organizational culture.

Most of the insights we have reported might be seen as self-evident or already apparent, at least to some practitioners. Nevertheless, we wish to stress that so far, those insights had no scholarly grounding. Therefore, we put considerable effort into recruiting a large sample of almost 500 software professionals to claim generalization of results, who have been screened throughout a multi-stage selection process, using a cluster-randomized sampling strategy. Data have been analyzed through Bayesian linear regression to provide a solid understanding of the studied phenomenon.

This is regression analysis, and it does not assess any causal relationship between independent variables and dependent opens. Nevertheless, our research design implicitly assumes a causal-predictive assumption of our theoretical hypothesis that has been validated. Also, we observe that the general population scores significantly lower than software engineers in need of cognition and that the personality traits of developers can predict the individual need for cognition. Accordingly, we speculate that people with a high need for cognition are more likely to start a career in computer science than those who score low. In other words, there are some individual characteristic that will lead someone more naturally to a computer science career compared to others. This insight is, for now, an educated guess and needs to be carefully validated with a proper research design to assess causation.

Although we grounded our recommendations in well-established social psychology and management literature, we have not reported them directly with this study. Therefore, further research should investigate the suggested relations in a software engineering population.

Raw data, survey questions, and additional material are openly available under a CC-BY 4.0 license at the following URL: https://osf.io/n57as/.