Open AccessReview

Factors Influencing Solvers’ Behaviors in Knowledge-Intensive Crowdsourcing: A Systematic Literature Review

Xuefeng Zhang

^1,*,

Enjun Xia

²,

Chao Shen

¹ and

Jiafu Su

School of Economics and Management, Anhui Polytechnic University, Wuhu 241000, China

School of Management and Economics, Beijing Institute of Technology, Beijing 100811, China

International College, Krirk University, Bangkok 10220, Thailand

Author to whom correspondence should be addressed.

J. Theor. Appl. Electron. Commer. Res. 2022, 17(4), 1297-1319; https://doi.org/10.3390/jtaer17040066

Submission received: 30 August 2022 / Revised: 8 October 2022 / Accepted: 10 October 2022 / Published: 12 October 2022

(This article belongs to the Section Digital Marketing and the Connected Consumer)

Download

Browse Figure

Versions Notes

Abstract

Solver participation plays a critical role in the sustained development of knowledge-intensive crowdsourcing (KI-C) systems. Extant theory has highlighted numerous factors that influence solvers’ participation behaviors in KI-C. However, a structured investigation and integration of significant influential factors is still lacking. This study consolidated the state of academic research on factors that affect solver behaviors in KI-C. Based on a systematic review of the literature published from 2006 to 2021, this study identified five major solver behaviors in KI-C. Subsequently, eight solver motives and seventeen factors under four categories, i.e., task attributes, solver characteristics, requester behaviors, and platform designs, were identified to affect each of the solver behaviors. Moreover, the roles of solver motives and the identified factors in affecting solver behaviors were demonstrated as well. We also suggested a number of areas meriting future research in this study.

Keywords:

knowledge-intensive crowdsourcing; solver behaviors; solver motivation; the influential factors; literature review

1. Introduction

Knowledge-intensive crowdsourcing (KI-C) is becoming a domain of innovation and an increasingly diffused phenomenon in today’s knowledge-intensive economy [1,2,3]. KI-C refers to a type of participative online activity in which an individual or an organization proposes a defined knowledge-intensive task on an intermediary platform to a group of individuals of varying knowledge, expertise, skills, or experience via a flexible open call in order to acquire their ideas, suggestions, or solutions for handling the task. Unlike simple tasks [4], routine tasks, and content collection tasks [5,6], knowledge-intensive tasks are always much more complicated, not easy to be decomposed in a clear way, do not have one (or more) correct answer(s), and are proposed through a flexible open call [1,3,7]. Moreover, their completion heavily relies on the knowledge, expertise, skills, and experience of engaged solvers. Common knowledge-intensive tasks include product design, software development, graphic design, ideation, and content creation.

Generally, a KI-C system consists of four major components, which are task, solver, requester, and online intermediary platform [8]. Solvers are the essential actors in KI-C systems whose wide and active participation heavily determines the sustained development of systems [9,10,11]. To understand and stimulate solvers’ participation in KI-C, prior studies have made great effort to identify the factors that influence solvers’ participation. These factors include solver motives [12,13,14,15], solver expertise [7,16,17], participation experience [18,19], cultural background [20,21], fairness of the requester and platform [22,23], platform designs [24,25], requester feedback [26,27], and task attributes [28,29,30]. Moreover, the relationships among some of these factors and their integrated effects on solvers’ participation have been examined as well. For example, Zheng, Li and Hou [29] found that contest autonomy, variety, and analyzability were associated with solvers’ intrinsic motivation and further impacted their participation intention. Franke, Keinz and Klausberger [22] indicated that the identity of the requester and some of the designs of the organized crowdsourcing system acted as the antecedents of solvers’ perceived fairness, which resulted in the consequences of solver participation willingness and ex post identification with the requester and crowdsourcing system. Tang et al. [31] proposed that feedback information type and valence affected solvers’ intrinsic motivation and, thus, influenced their knowledge contribution performance.

It can be indicated from these prior studies that there are multiple factors that have influences on solver participation in crowdsourcing. Nevertheless, to the best of our knowledge, few studies have been dedicated to summarizing and integrating these factors, which have mainly been investigated separately and partially in the existing studies to form a relatively comprehensive list of factors. In addition, as mentioned in the prior studies, solvers’ participation can be specialized as different behaviors, e.g., participating in a crowdsourcing platform [22,32] or engaging and contributing in a task [7,13,33], which are affected by different factors. Therefore, another essential work that is needed is to link the identified factors with specific solver participation behaviors.

This study offers a “state-of-the-art” overview of the KI-C body of literature to address the following issues: (1) What behaviors do solvers exhibit in KI-C? (2) Why (motives) do solvers exhibit these behaviors? (3) What factors influence the solvers’ behaviors? (4) What roles do those factors play in the solvers’ behaviors?

With respect to crowdsourcing, some reviews have been conducted. These reviews have focused on clarifying crowdsourcing definition [6,8], categorizing crowdsourcing [34], and further positioning it in relation to other concepts (such as open innovation [35], open-source software [36], and organizational forms [37]), broadly describing and exploring the development of the crowdsourcing literature [38,39,40,41], revealing the applications of crowdsourcing in different areas (such as the medical and health sectors [42], data management [43], and information systems [40]), defining the components of crowdsourcing systems and their relationships [44,45], and investigating a specific component or issue of crowdsourcing (such as privacy challenges [46], social mechanisms [47], gamification designs [48], task affordance [49], and factors influencing decisions to crowdsource [50]). The present review concentrates on solver behaviors in KI-C and is dedicated to synthesizing the factors that influence solver behaviors, which is different from past reviews on crowdsourcing. Our review, on one hand, can aid us in having a comprehensive understanding of solver behaviors in KI-C and their influential factors by summarizing and integrating the most significant factors investigated in the KI-C body of literature and categorizing them into multiple facets. On the other hand, our review supports academics and practitioners in developing theoretical models and managerial means that can be used to attract and retain solvers’ active participation and contribution in KI-C.

We begin by illustrating the methods used for conducting our review. Thereafter, the results and analysis are presented in Section 3. Section 4 discusses the results from the review and identifies future research needs. Section 5 concludes the study.

2. Methods Used

This study employed a systematic review method, a structured and comprehensive method aiming to make the literature selection and review process replicable and transparent [51], to conduct the review. We followed a three-stage procedure, which was planning, execution, and reporting.

2.1. Planning the Review

At this stage, we confined the research scope, defined the research purposes, and identified key data sources. This work focused on KI-C and aimed to investigate solver behaviors and their influential factors. We limited our data sources to peer-reviewed journals since these can be considered validated knowledge and are likely to have good quality and impact in the field [52]. Two data sources from the ISI Web of Science, including Science Citation Index Expanded (SCI-EXPANDED) and the Social Science Citation Index (SSCI), were chosen because they are two of the most comprehensive databases of peer-reviewed journals in the science and social science fields. Further, some search results from different dimensions, such as research type, published time, and citation counts, were presented, which allowed us to analyze and depict the search results more conveniently and effectively.

2.2. Conducting the Review

Four steps, i.e., identification of keywords and search terms, assessment and selection of studies, data extraction, and data synthesis, were executed in this stage. The first two steps pertained to the collection, filtering, and organization of the data, and the last two steps involved data processing and analysis.

2.2.1. Identification of Keywords and Search Terms

This study chose the term “crowdsourcing” as the search phrase because it fit well with our research focus and is a well-representative term that has been employed in other relevant reviews as well. Additionally, other similar keywords, including “crowdsource” and “crowdsourced”, were used to conduct the search in order to ensure we did not overlook any important literature. Even though the notion of “crowd” has generated streams, such as the major research streams of crowdfunding and crowdsensing, we excluded these from this review a priori. Consequently, we used the terms crowdsourcing, crowdsource, and crowdsourced (i.e., TS = (crowdsourcing or crowdsource or crowdsourced)) as a selection criterion for the topic (title, keywords, or abstract) to conduct our initial search.

The other search terms were: language “English”; document type “article” or “review”; published time “from 2006 to 2021”; and subject area “computer science information systems” or “management” or “information science library science” or “behavioral sciences” or “social science interdisciplinary” or “business” or “public environmental occupational health” or “operations research management science” or “urban studies” or “psychology multidisciplinary” or “ergonomics” or “public administration”.

We conducted the search on 29 January 2022, and captured an initial sample of 2549 papers, including 2487 articles, 96 review articles, 96 early-access, 1 data paper, and 1 retracted publication.

2.2.2. Assessment and Selection of Studies

At this step, we examined titles, abstracts, and even the full text of the papers and retained the articles that met our refined criteria: those belonging to the crowdsourcing of knowledge-intensive tasks; focusing on crowdsourcing with estimated requesters, solvers, and online platforms or websites; and investigating solver behaviors, solver motivations, or factors affecting solver behaviors. Meanwhile, we excluded the papers that concentrated on: simple tasks, indoor localization tasks, spatial tasks; specific crowdsourcing systems that were out of our research scope, such as crowdfunding, crowdsensing, and citizen sensing; developing techniques for building crowdsourcing system infrastructures or algorithms of matching tasks and solvers; and user-generated content voluntarily online, such as Wikipedia and Openstreetmap.

This step resulted in a significant reduction in the number of papers in the initial pool and the reservation of 119 article papers.

2.2.3. Data Extraction

Regarding the selected papers, this step examined them by employing a data extraction form used in systematic reviews and aimed at reducing human error and bias when looking into and assessing the papers [51]. It also acted as the data repository form through which the analysis emerged, a historical record of decisions made during the review process, and a visual representation of features of the selected papers. Based on the research questions and aims of this paper, the data extraction form contained the following information: titles, authors, institution of the first author, published years, publications, citation counts, article types, research approaches, task types, crowdsourcing taxonomy, research questions, employed theories, solver behaviors, solver motivations, and factors that influenced solver behaviors.

3. Results and Analysis

3.1. Descriptive Analysis

3.1.1. Publications by Year

The research on exploring the effects of various factors on solver participation in KI-C has gained prominence over the last several years. Table 1 shows the distribution of all the studies from 2006 to 2021.

Overall, there was a recent increase in published articles focusing on the topic of this study, especially from 2017 to 2021. This recent increase is consistent with the development of the crowdsourcing field. Meanwhile, it, to some degree, could reflect that there is a growing awareness of the importance of attracting and sustaining solvers’ participation in KI-C.

3.1.2. Publications by Journal

The selected 119 articles were published in 64 journals covering information systems and information management, management and business, social science, psychology and ergonomics, and public and urban studies. The top journals publishing no less than three articles included Information Systems Research (5), Production and Operations Management (5), International Journal of Information Management (5), Computers in Human Behaviors (5), Research Policy (5), Journal of Management Information Systems (4), Online Information Review (4), Behaviour & Information technology (4), Internet Research (3), Journal of Product Innovation Management (3), R&D Management (3), and Technological Forecasting and Social Change (3). These journals have high ranks in their corresponding fields, which indicated that the topic of this paper acts as an important field in crowdsourcing and has been studied deeply. In terms of the audience, these publishers from various disciplines demonstrated that the topic has a broad audience base.

3.1.3. Types of Knowledge-Intensive Task

Based on the descriptions in the selected papers directly, the KI-C tasks could be simply classified into the following groups: complex tasks (e.g., IT-relevant service, data modeling, translation and programming, technical problems, logo design), idea generation, innovative or creative tasks, citizen ideation and innovation tasks (e.g., open government project, civic problems, public transportation), knowledge sharing or exchange, open-source software projects, and cultural crowdsourcing projects (e.g., correcting errors in digitized newspapers, manuscript translation).

3.1.4. Research Methods Used

At a high level, the research methods used in the sample could be divided into empirical and non-empirical methods. The empirical methods included case studies, surveys, interviews, field experiments, and primary data. Non-empirical methods included conceptual or theoretical studies, optimization modeling, and literature reviews. Among the 119 studies, empirical studies had a prominent role. The top used empirical methods were surveys (45), primary data (i.e., analysis using data directly collected from crowdsourcing platforms) (35), interviews (18), field experiments (19), and case studies (11). The non-empirical methods consisted of conceptual or theoretical analyses (3), optimization modeling (2), and lab experiments (simulation experiment) (1). Additionally, 25 (21.01%) studies used a combination of methods.

3.1.5. Theoretical Foundations

In the sample, 69 papers offered some theoretical bases in their studies, whereas a few studies did not invoke a strong underlying theory. The most-used theories were motivation theory (9), self-determination theory (8), expectancy–value theory (4), social exchange theory (3), justice theory (3), and identity theory (2).

3.2. RQ1: What Behaviors Do Solvers Exhibit in KI-C?

This study categorized the solver behaviors examined in the prior studies into five types. They were participating in a KI-C platform, choosing a task to participate in, making effort in a task, contributing high-quality solutions, and continuous participation and contribution [53].

3.2.1. Participating in a KI-C Platform

Participating in a KI-C platform was the first and most essential activity that solvers had to implement before further accessing published tasks and interacting with others [54,55]. This behavior was mainly indicated by solvers’ self-expressed willingness to participate in a platform [22,32,56].

3.2.2. Choosing a Task to Participate in

Choosing a task from the task pool in the preferred platform to participate in and contribute to was a following behavior that solvers could exhibit. This behavior was directly related to the subsequent solution development. Solvers’ self-expressed willingness to participate in a specific task [29,30], the number of engaged solvers in a task [57,58,59], and task choice [33] were always employed as indicators for measuring this behavior.

3.2.3. Making Effort in a Task

Making effort referred to the extent to which a solver dedicated his or her time, expertise, and energy to complete the selected tasks. Two types of indicators, i.e., status-based and outcome-based, were proposed to measure the levels of effort that solvers made in a task. The status-based indicators included intensive and persistent engagement of energies [13]; physical, cognitive, and emotional engagement of energies [28,60]; and the committed resources [61] of a solver exerted. The outcome-based indicators were the number of solutions [62,63] and comments [64] that solvers submitted in a task. Sometimes, solvers could make a certain effort in a task but, finally, abandon it, which was named task abandonment [65]. This study incorporated this behavior into the category of making effort in a task.

3.2.4. Contributing High-Quality Solutions

The quality of a solution could be evaluated based on requesters’ scores [19,66], team rank in the contest [67], the likelihood of a solution implemented [18], solution innovativeness [7], and solution appropriateness [68]. This behavior drew the most significant attention from requesters who aimed to find the best solutions for their tasks. There is no doubt that it has become an extremely important focus in the field of solver behavior in KI-C, as indicated in Table 2.

3.2.5. Continuous Participation and Contribution

After a solver participated in and contributed to a platform initially, he or she would confront the decision of whether to participate and contribute continuously or not. Generally, scholars have paid most attention on solvers’ continuous intention of participating in and contributing to their engaged KI-C platforms. A major indicator of solvers’ self-expressed further willingness to participate and contribute was proposed to measure this behavior [69,70,71]. Solvers’ positive participation experience and satisfaction are critical for keeping solver continuous participation and contribution. Moreover, prior studies have exerted a lot of concern on the antecedents of solver past participation satisfaction [71,72].

To sum up, the solver behaviors and their major indicators are shown in Table 2.

As shown in Table 2, the quality of solutions submitted by solvers in a task drew the most attention in the prior studies, which was followed by the behaviors of task choice and effort made. The number of articles focused on solver continuous participation and contribution in KI-C received less concern than the other four behaviors. In addition, according to our analysis, most of the studies focused on the investigation of a single behavior, and studies that concentrated on the relationships between different behaviors were limited. However, solvers’ different behaviors were not independent. For example, the higher level of effort that solvers exerted in a task, the higher the likelihood of developing high-quality solutions [73]. Solvers’ prior experience imposed an effect on their future participation [11,74].

3.3. RQ2: What (Motives) Motivates Solvers to Exhibit the Behaviors?

3.3.1. The Solver Motives

Many specific solver motives were mentioned in the prior studies, including almost twenty motives in [12], fourteen motives in [13], and ten motives in [14]. However, some of these motives were repetitive and had similar meanings. This study mainly demonstrated eight significant solver motives that were mostly emphasized.

(1) Monetary reward was an important motivation that stimulated solver participation and contribution in crowdsourcing. For a specific task, large monetary rewards were helpful for increasing solvers’ anticipation and compensation for the cost and, thus, had great attractiveness [9,10,75]. However, the quality of solvers’ submissions may not continuously increase with the rise in monetary reward size [10,76]. Consequently, some scholars have tried to explore efficient reward schemes [77,78] or to offer non-monetary rewards, such as free final products, free service, and training [12,76,79].

(2) Learning or developing competence was another important extrinsic motive that was emphasized in prior studies [80]. It indicated that solvers tend to acquire new and valuable knowledge and practice skills by participating in crowdsourcing. However, the studies that focused on examining the effect of this motive on solvers’ behaviors were not too many. Solvers’ learning motivation was positively associated with their participation intention in a crowdsourcing platform [9] but was negatively associated with solution appropriateness [68].

(3) Crowdsourcing assisted solvers to be freelancers because a lot of work opportunities were provided to them. To access new work and to improve career prospects became one of solvers’ major motives. This motivation has been examined to impact solvers’ participation in internal or firm-hosted crowdsourcing systems, especially crowdsourcing for employment [79,81], and online collaboration communities, such as question-and-answer websites [82,83].

(4) Gaining reputation or recognition was a relevant motivation when solvers were offered opportunities to share their know-how, enhance ego gratification, and be visible to and recognized by others [14]. To build an interactive social community and an online reputation system are becoming two main approaches adopted by crowdsourcing platforms to respond to this motivation [68,81,84].

(5) Being challenged and having fun was the utmost pure intrinsic motivation. It could be reflected by some other terms with similar meanings, such as curiosity [30], playfulness [56], enjoyment and addiction [80], and pastime [14]. This motivation has been investigated in different crowdsourcing contexts, especially contests for problem solving that would allow solvers to demonstrate themselves and enjoy the process of completing tasks [13,68,85].

(6) Self-efficacy could stimulate solvers to participate and contribute because they believed that their specific knowledge, skills, and experience could meet the requester and task requirements. Solvers’ self-efficacy enabled them to make more effort in competition conditions but less in non-competition conditions. In turn, solvers’ good performance aided in increasing their self-efficacy [86]. In addition, the influence of this motivation could be moderated by task complexity. Specifically, when task complexity was high (low), there was a convex (concave) relationship between solvers’ self-efficacy and effort [87].

(7) Altruism or reciprocity referred to one of the goals of solver engagement in crowdsourcing, which was helping others and enjoying the feeling of appreciation and respect from others [14,88]. This motive was positively associated with solvers’ willingness to share ideas [88] and willingness to participate continuously [80].

(8) The last, but not least, was having a sense of belonging to a community and was an important solver intrinsic motive. Solvers could identify themselves as a member of the community and align their benefits and goals with other members [81,89]. It helped to keep solvers’ participation in the crowdsourcing system [84,90]. As Fedorenko et al. [91] indicated, solvers would change their identity from “I” to “me” and ”we” with sustained involvement in a community.

3.3.2. The Interactive Effect of Solvers’ Motives

In practice, solvers’ different motives always integrate together to stimulate them to conduct activities in KI-C [14,76]. However, most of the current studies that have examined the influence of solvers’ motives have mainly considered these motives separately. Only several studies of our sample paid attention to investigating the relationships of motives and further examining their combination impacts. For example, Pee, Koh and Goh [33] clarified the effect of solvers’ motives on their task choice in terms of the distal–proximal perspective and indicated that solvers who were motivated by payment tended to focus on demonstrating a higher competence relative to others. Frey et al. [92] noted that the interaction of a strong desire for monetary rewards and higher intrinsic enjoyment of contributing decreased the number of non-substantial contributions to open projects. Sun, Fang and Lim [10] presented that solvers’ self-efficacy had a positive moderating effect on the relationship between solvers’ intrinsic motives and continuance intention. Liang, Wang, Wang and Xue [60] checked the interaction effect of solvers’ intrinsic and extrinsic motives on their effort in crowdsourcing contests and found that the positive association between intrinsic motivation and effort was reduced when a high level of extrinsic incentives was presented. In addition, extrinsic incentives were found to have a negative moderating effect on the relationship between intrinsic motivation and solvers’ engagement.

3.3.3. The Transformation of Solvers’ Motives

Solvers’ motives to exhibit behaviors in KI-C were not fixed but changed with development of involvement degree, participated tasks, and innovation stages.

Firstly, solvers with different roles had different motives. For instance, Frey, Lüthje and Haag [92] found that most valuable contributors were those who were motivated by high levels of intrinsic enjoyment of contributing in open projects. Battistella and Nonino [12] classified solvers in web-based crowdsourcing platforms into four categories in terms of their roles, (champion and expert roles; relationship roles; process roles; and expert, power, and process roles) and presented different motives for these four groups of solvers. Mack and Landau [93] compared the differences in motives of winners, losers, and deniers in innovation crowdsourcing contests. Moreover, extrinsic motivation was found to be distinguished successfully from unsuccessful participants more than intrinsic motivation.

Secondly, solvers’ motives evolved in different participation stages. For example, Soliman and Tuunainen [74] suggested that there was a transformation of solvers’ motives in their initial and continuous participation in a photography crowdsourcing platform. The solvers’ initial participation seemed to be inspired by selfish motives, but sustained participation seemed to be driven by both selfish and social motivations. Alam and Campbell [80] indicated that volunteers in a newspaper digitization crowdsourcing project showed intrinsic motives initially, though both intrinsic and extrinsic motivations played a critical role in their continued participation. Heo and Toomey [26] also demonstrated that participants’ motivations to share knowledge in crowdsourcing changed over time.

Thirdly, the motives of solvers who engaged in different tasks were different. Battistella and Nonino [76] found that the motives in successful web-based crowdsourcing platforms were a function of the phase of the innovation phase. Specifically, the more the phase became “concrete”, the more extrinsic motivations were used. Alam and Campbell [80] noted that volunteers’ intrinsic or extrinsic motives changed with different kinds of contributions, i.e., data shaping to knowledge shaping, in a newspaper digitization crowdsourcing project. Wijnhoven, Ehrenhard and Kuhn [14] found that citizens’ motivations to participate in open government projects varied from projects of citizen innovation and collaborative democracy to citizen sourcing.

3.4. RQ3: What Factors Influence the Solvers’ Behaviors?

According to the components of the KI-C system [8], this study categorized the factors that influenced the solver behaviors into four groups. They were task attributes, solver characteristics, requester behaviors, and platform designs. Under each group, the specific factors were identified from the selected articles. Table 3 presents the specific factors that corresponded to each of the solvers’ behaviors.

The number at the intersection of a factor and a behavior represented the number of articles that investigated their relationship. The symbol “-” was used if there were no articles focusing on the relationship between a factor and a behavior. Two conditions need to be noted during building of this table. First, a factor remained only when it was suggested in multiple studies. A factor proposed in a single study was not presented, as it could be derived from an author bias or be applied to a particular context. Second, we assigned a factor to its influential behaviors according to the explicit findings in the articles that mentioned their relation and did not deduce excessively.

Moreover, several cues can be draw from Table 3. First of all, in terms of the number of supporting articles, “task attributes” was the factor category that was investigated the most, and it was followed by “platform designs”, “solver characteristics”, and “requester behaviors”. However, if we viewed solver motivation as one of the solver characteristics, then this factor category, i.e., solver characteristics, was considered to affect solver behaviors in KI-C the most. In addition, under these four categories, seventeen specific factors were identified to influence the solvers’ behaviors.

Second, each of the solver behaviors was influenced by multiple factors. For example, there were twelve specific factors covering four factor categories that affected solvers’ effort made in the selected tasks. More specifically, from the perspective of the number of related factors, the behaviors of task selection, effort made, and solution contribution were the top three behaviors that were studied the most. This is in line with the findings presented in Section 3.2.

Third, the major factors that influenced each of the solver behaviors were different. In particular, the factors for “task attributes” and “solver characteristics” were investigated to influence the solver behaviors, including task selection and effort making. Nevertheless, the factors in the “platform designs” mainly affected solvers’ participation in and contribution to a crowdsourcing platform initially and continuously.

Last but not least, the concerns for different specific factors under each factor category were different. For example, under the category of “task attributes”, task rewards was found to be one of utmost important attributes that affected solvers’ behaviors. Solvers’ domain expertise was found to mostly influence their behaviors compared to other factors in the category of “solver characteristics”. In addition, giving feedback to solvers was highlighted and required in order to promote solvers to contribute more and long-term. Incentive design was one of the priorities for crowdsourcing platforms to improve their solver services and further keep solver engagement.

3.4.1. Task Attributes

A task’s attributes referred to the information that reflected its context, problem, requirements, and other facets. The attributes were defined by requesters and were presented to solvers in order to facilitate them to make sense of the task and, thus, had an impact on their behaviors in the task as well. The major task attributes mentioned in our sample are illustrated below.

(1): Monetary reward

Monetary reward was one of the important task attributes that was emphasized in academic studies but also was concerned with solvers. It acted as an informational cue indicating fairness and the appreciation of participation and effort that were internalized by solvers [54,94]. A certain level of monetary reward was indispensable for attracting solvers’ participation in a knowledge-intensive task, which was especially performed by crowdsourcing contests [54,57]. However, increasing the amount of monetary reward for a task did not always raise solvers’ willingness to participate [95]. For example, Liu, Hatton, Kull, Dooley and Oke [59] found that reward size and crowd size were correlated to one another in the form of an inverted U-shape, namely that moderate rewards maximized crowd size.

For the solvers who chose a task, a monetary reward generally positively affected their effort exerted in the task [87,96,97], but did not contribute greatly to the quality or novelty of submitted solutions [95]. In order to improve the effect of monetary reward on solvers’ efforts and contributions, award structure (e.g., performance contingent award) [3] and prize guarantee (ensuring that a winner will be picked and paid) [98] were suggested to be considered while setting a monetary reward for a task.

(2): Task complexity

Task complexity referred to the difficulty level of a task [59,99] and could be demonstrated by two dimensions, i.e., analyzability and variability [29]. The analyzability highlighted the availability of concrete knowledge on a task, and the variability indicated the frequency of unexpected events and contingencies that could happen while performing the task. It was an important inherent characteristic that influenced solvers’ selection of a task and further effort. Task complexity was positively associated with solvers’ intrinsic motivation [29,96]. Tasks should preferably be less complex in order to attract solvers’ participation [29,99]. In addition, task complexity had a mediating role in the relationship between task reward [59], solvers’ self-efficacy [87], and solvers’ effort in a task.

(3): Task autonomy and variety

Since these two attributes have been considered simultaneously in prior related studies and have similar roles in affecting solvers’ behaviors, we combined and explained them together. Specifically, task autonomy referred to the degree of freedom of solvers execute the task. Task variety referred to the degree to which the task required solvers to apply a wide range of skills and perform a variety of activities in order to complete it [29,30]. The autonomy and variety of a task, on the one hand, were positively related with solvers’ intrinsic motivations [29,30,96]; on the other hand, they played an important role in creating favorable crowd working conditions [54]. As a result, scholars have verified that these two task attributes are positively associated with solvers’ participation in a task and further have a positive effect on solvers’ effort and contribution quality in a task as well [73,96].

(4): Task instruction

Task instructions often constitutes the first contact point between requesters and potential solvers [28]. Generally, clarity, accuracy, and targeting for potential solvers were the major indexes indicating the quality of task instruction. A task that was not described accurately or with high tacit and vagueness in its definition could indicate that requesters did not have clarity and specific criteria to evaluate outcomes. Solvers may doubt the objectiveness of evaluation process and perceive the risk that their outcomes may not be legitimately evaluated. In contrast, a well-articulated task statement reduced the uncertainties of potential solvers and increased their willingness to participate and put further effort into the task [100,101]. Specifically, clearly estimating a solver’s responsibility in contributing ideas [102], offering sufficient instructions and exemplars of expected outcomes [28,103], and illustrating intellectual property right arrangement [104] were suggested to requesters in presenting their task statement. However, good delineation did not mean to bound a task strictly, and lower task autonomy was greatly unfavorable to increasing of novelty and originality of solvers’ solutions [102,105].

(5): Task in-process status

This attribute reflected the execution progress of a task, which could be indicated by the number of participants, submitted solutions, requesters’ feedback, etc. Out of these statuses, the quantity and quality of submitted solutions were emphasized mostly by scholars. In a crowdsourcing contest, high-quality submitted solutions decreased the likelihood of high-quality solvers entering it, resulting in lower overall quality in subsequent submissions [57,58,75]. In addition, it made solvers entering later exert less effort than early entrants due to a lower probability of winning the contest [3,63]. However, Körpeoğlu and Cho [106] noted that high-expertise solvers may raise their effort in response to increased competition. This was because more entrants raised the expected best performance among other solvers, creating positive incentives for solvers to exert higher effort to win the contest. In addition, Dissanayake, Mehta, Palvia, Taras and Amoako-Gyampah [86] also found that solvers’ self-efficacy showed a positive effect on their effort under competitive conditions.

3.4.2. Solver Characteristics

This study mainly presents the following solver characteristics that have been discussed extensively in prior studies. Characteristics such as age, gender, educational background, and employment status [14,66,107] were not included because there were few studies investigating their impacts directly and specifically.

(1): Domain expertise

A solver’s domain expertise, to some extent, demonstrated his or her knowledge, skills, experience, and even creativity [3,67]. In KI-C, solvers’ domain expertise, especially that related to task domain, was an utmost important characteristic that determined the capacity of solvers to capture a task statement and further affected their participation possibility [28,93]. In addition, solvers with high expertise may have more privileges to take ownership of their behaviors and were more likely to develop solutions with high quality. For example, Dissanayake, Zhang, Yasar and Nerur [62] indicated that solvers, especially those who had high skill, tended to increase their effort toward the end of tournaments or when they grew closer to winning positions. Zhang et al. [108] suggested that participants’ domain knowledge had a significant effect on their manuscript transcription quality. Boons and Stam [66] argued that individuals having a related perspective associated with their educational background positively influenced their idea performance. In addition to the depth of domain expertise, expertise diversity also played a key role in affecting solvers’ substantial contributions [16,92].

(2): Participation experience

Solvers’ participation experience was often indicated from their participation history, past submissions, and past interaction with others in KI-C. In general, solvers’ participation experience had a positive influence on their performance [19]. However, rich participation experience, especially past success experience, may result in solvers used to proposing ideas similar to their previous ideas [18]. Koh [103] also found that solvers’ domain experience negatively moderated the relationship between solvers’ adoption of exemplars provided by requesters and their advertised design quality. Additionally, solvers with positive and rich participation experience would inspire them to participate and contribute continuously [11,72].

(3): Cultural background

Practically, a lot of crowdsourcing platforms are structured to perform foreign and local tasks with support from solvers all over the world. In these platforms, different solvers from different countries may have various cultures and wealth, which affects their task choice, effort, and contribution quality [20,21]. Specifically, Chua, Roth and Lemoine [20] found that solvers from countries with strict cultures were more likely to participate and succeed at foreign creative tasks than counterparts from loose cultures. Bockstedt, Druehl and Mishra [21] investigated the influence of contestants’ national cultural background, i.e., performance orientation and uncertainty avoidance, on their problem-solving efforts. Furthermore, these two studies both confirmed the positive effect of small culture distance between solvers and requesters on solvers’ performance.

(4): Personality traits

Solver personality trait in this study mainly referred to solver value orientation during participation in KI-C. More specifically, Steils and Hanine [28] found that individuals who were monetary-reward-oriented were less likely to choose tasks with constraining instructions. Zhang, Chen, Zhao Yuxiang, Song and Zhu [108] suggested that solvers who were social-value-oriented, i.e., pro-social and pro-self, produced influence on their manuscript transcription quality by joining with solvers’ domain expertise. Schäper, Foege, Nüesch and Schäfer [88] argued that individuals’ psychological ownership of ideas would weaken their willingness to share ideas in a specific crowdsourcing platform. Shi et al. [109] examined the effects of solver personal traits, including conscientiousness, extraversion, and neuroticism, on solver engagement in crowdsourcing communities that were mediated by solver self-efficacy and task complexity.

(5): Interaction with peers

A solver’s interaction with peers, i.e., other solvers, mainly is presented as peer feedback, which can be likes, votes, and comments on his or her contributions. The feedback brings solvers cognitive, integrative, and affective benefits. These benefits may promote and strengthen solvers’ credibility, confidence, and pleasure experience, thus strengthening their willingness to develop ideas and communicate with peers [64,110]. With specific feedback, its characteristics, including response speed [64], diversity, constructiveness, and integrativeness [111], brought different values for stimulating solvers. In addition, Chan et al. [112] demonstrated a significant impact of peer-to-peer interaction measured by interaction size, direction, and strength on participants’ likelihood of subsequent idea generation in crowdsourcing community. In order to increase the opportunities to be recognized by peers, sharing online profiles is an option for solvers [113].

3.4.3. Requester Behaviors

Requesters are the people seeking solutions from solvers and communicating with solvers during task completion, their characteristics, and things they do to impact solvers’ behaviors. In this study, we concentrated on two major requester behaviors, which were fairness and giving feedback.

(1): Fairness

Solvers in a crowdsourcing platform or a task not only want a good deal, but also want a fair deal [22]. Fairness embodied three specific dimensions, which were distribution fairness (the fairness of resource allocation among solvers), procedural fairness (the fairness of requesters evaluating and selecting submissions), and interactional fairness (the fairness of requesters treating solvers with respect and politeness and offering information related to the process and the results of task completion) [22,23,71]. Fairness showed by requesters influenced solvers’ perceptions of trustworthiness of requesters and crowdsourcing platforms [71,87]. It was concerned with solvers and had an impact on solvers’ selection of tasks [22], effort making [87], solution development [23], and future participation in and contribution to a crowdsourcing platform [71]. However, the influences of these three fairness dimensions were asymmetric. As Faullant et al. [114] indicated, distribution fairness acted as a basic factor that must be fulfilled in order to avoid negative behavioral consequences, while procedural fairness could be considered as an excitement factor that caused truly positive behavioral consequences. In addition, Liu and Liu [71] revealed that distribution and interactional fairness was perceived by solvers to have an impact on their trust in requesters, while solvers’ perceived procedural and informational fairness significantly affected their trust in crowdsourcing platform management.

(2): Give feedback

Feedback given by requesters could make solvers perceive that their contributions were important. Furthermore, solvers did think of feedback on their contributions as a genuine sign of appreciation and, consequently, stimulated their efforts exerted in a task [73,115], promoted their solution quality [27,116], and increased the likelihood of continuous participation [72]. Therefore, requesters were encouraged to give feedback to solvers, even negative feedback [115]. In addition, the positive effect of feedback was not always created casually but was influenced by its form and content. Particularly, forms of feedback could be no feedback, random feedback, indirect feedback, and direct feedback [73,115]. The content of feedback could be rejection with explanation or with similar linguistic style with the submitted ideas [70] or message framings, such as alignment with cooperation, individualism, and competition in the contents proposed by a solver [116]. Furthermore, the timing of feedback, such as early-stage and late-stage, in a contest [115], in-process feedback, its volume [98], and the direction and strength of feedback [112,117] needed to be considered when presenting feedback to solvers.

3.4.4. Platform Designs

Crowdsourcing platforms link requesters and solvers and facilitate them to conduct activities effectively. Many specific pieces of design advice have been developed by scholars for crowdsourcing platforms to improve their solver service and further attract and keep solvers’ participation and contribution. Out of these suggested designs, this study mainly emphasized four designs, i.e., present system-generated visual feedback, be trustworthy, have an incentive system, and foster a community, that were discussed in prior studies.

(1): System-generated visual feedback

In general, the visual feedback presented by a crowdsourcing platform is specialized as status, rank contrast, reputation score, reward points, average, and top assimilation [26,69]. Sometimes, these types of feedback can be called gamification artifacts as well [56]. Further, these feedbacks offer solvers individual-level status information, providing them a sense of being fairly treated and respected [118]. Consequently, system-generated visual feedback could affect solvers’ participation in a crowdsourcing platform, both initially and continuously, by affecting solvers’ intrinsic motivation. For example, Heo and Toomey [69] indicated that simple, system-generated visual feedback on social comparisons could manipulate solvers’ motivation to continuously share knowledge in crowdsourcing. Feng, Jonathan Ye, Yu, Yang and Cui [56] also found that solvers’ intrinsic motives, e.g., self-presentation, self-efficacy, and playfulness, positively mediated the relationship between gamification artifacts, e.g., point rewarding and feedback giving, and their willingness to participate in a crowdsourcing platform.

(2): Be trustworthy

Under a virtual environment, solvers always have a need of psychological expectation since interaction on a virtual platform is vulnerable to a few risks and uncertainties [119]. Commonly, a high-level trust of a platform can meet solvers’ psychological requirements and, thus, is likely to alleviate solvers’ hesitations. Being trustworthy was critical for a crowdsourcing platform to motivate solvers to participate and contribute [119,120]. A crowdsourcing platform aimed at showing its trustworthiness could formulate many effective mechanisms and regulations. For instance, Hanine and Steils [120] suggested avoiding negative feelings of solvers by dealing with their three emerging sources, which were information about the use of contributions and overpromising, protection of intellectual property rights, and prize allocation and selection criteria. At the same time, Bauer, Franke and Tuertscher [119] emphasized the importance of well-established norms that regulated the use of intellectual property within a crowdsourcing community. In contrast, there is no doubt that enhancing solvers’ feelings of pride and respect playe a central role in driving their ongoing activity in a crowdsourcing platform [118].

(3): Incentive design

Incentives design is essential for crowdsourcing platforms because it plays role of appealing to or matching solvers’ motives [121]. Therefore, the development of incentives on the basis of motivations of potential solvers has become an alternative approach for crowdsourcing platforms. Consequently, a comprehensive package of incentives should be equipped in order to be responsive to the various motives indicated in Section 3.3 [89,122]. Of these incentives, prize and reputation were two highlighted incentives. For example, Kohler [123] suggested that prize, reputation system, socialization, feedback, and framing were the important components of an incentive system of a crowdsourcing platform. With respect to prize incentive, reward structure had a lot of attention in order to maximize its value of activating solvers. For instance, Tang, Li, Cao, Zhang and Yu [77] proposed a novel payment scheme with ex-ante and ex-post schemes to improve solvers’ malicious behaviors in macrotasking crowdsourcing systems. Malhotra and Majchrzak [124] suggested dual reward incentives, one for contributing best solutions and the other for undertaking actions in the knowledge integration process. On the other hand, the provision of opportunities for solvers to demonstrate themselves was critical for reputation incentive design [84]. These opportunities included featured solver profiles [125], multiple sources for solvers to be recognized [123], and the aforementioned system-generated visual feedback [56,69]. Finally, regarding innovation contests, a mixture of cooperative and competitive design features positively affected the creative process and solvers’ effort in developing solutions [105].

(4): Foster a community

Having a sense of belonging to a community was one of the important motivations of solvers to participate and contribute in crowdsourcing [81,89]. It is, therefore, hardly surprising that fostering a community for solvers and requesters has become a concern of crowdsourcing platforms. Further, having a good community was helpful for sustaining solvers’ participation [84,126]. A lot of functions have been suggested to be included in a community, such as publishing news, offering tips and tricks for solvers to develop solutions, supporting interaction among solvers and requesters, and supporting self-demonstration [123,127]. With these functions, crowdsourcing platforms mainly look forward to facilitating identity creation and fostering social interaction for solvers [91,127]. As Fedorenko, Berthon and Rabinovich [91] suggested, identity construction offers powerful tools for adding value to solvers, developing sustainable relationships, and generating enhanced-value co-creation.

3.5. RQ4: What Roles Do the Factors Play in the Solvers’ Behaviors?

We demonstrated the ways that the identified factors and solvers’ motives affected the solvers’ behaviors, drawing upon the model of motive-incentive-activation-behavior (MIAB) [121,128]. The MIAB model suggests that individuals undertake a particular action when their motivations are activated. Moreover, the right mix of incentives that appeal to an individuals’ motives must be distributed to stimulate individuals. Consequently, a framework for demonstrating the roles of the factors and solvers’ motives in affecting the solvers’ behaviors was developed and is presented in Figure 1. There is no doubt that the framework could not cover all specific situations, but could provide a perspective to understand the functions of these factors.

The five solver behaviors associated with each other sequentially. There was an increase in engagement level with the evolvement of participation behaviors. Additionally, a latter behavior could be implemented by a solver only after he or she completed its former behaviors. Meanwhile, the results of a former behavior had influence on its following behaviors. For example, the more effort that solvers expended on a task, the higher performance they may present [86,97]. With respect to the solvers’ continuous participation and contribution, solvers’ satisfaction and earned financial and non-financial benefits in the former four behaviors possibly imposed impacts separately or comprehensively [69,93].

Regarding each of the solver behaviors, the influential factors that were related to solver motives and characteristics, task attributes, requester behaviors, and platform designs may have an effect. As mentioned in Table 3, it can be found that there were differences in the influential factors between behaviors. Moreover, there were differences in the roles of the identified factors in affecting the solvers’ behaviors, which is illustrated below.

(1): The solver motives

Motivational factors are the stable, psychological drivers within solvers that stimulate goal-directed actions [129]. Eight major motives of solvers were summarized from prior studies. They were monetary rewards, learning, career prospects, reputation or recognition, fun, self-efficacy, altruism or reciprocity, and belonging to a community. These motives drove solvers to undertake different actions in KI-C. Additionally, four conditions needed to be concerned in understanding the role of solvers’ motivations. First, multiple motives generally worked together to affect each of the solver behaviors. For example, Ebner, Leimeister and Krcmar [79] found that solvers who decided to participate in SAPiens had five different motives. Chris Zhao and Zhu [13] argued that there were eight major motives that affected solvers’ effort levels exerted in a task. Second, solvers’ motives to participate in KI-C varied from initiative to initiative. Third, solvers with different roles and characteristics had different motives. Finally, as indicated in Section 3.3.3, a solver’s motives to participate in crowdsourcing were not fixed, but evolved with the development of his or her participation experience and times.

(2): Task attributes

As presented in Table 3, this factor category was particularly important for solvers’ selection of tasks and further efforts and contributions in the tasks. Regarding these behaviors, task attributes mainly acted as the incentives that appealed to or matched solvers’ motives [54,96]. Specifically, task awards and recognition acted as extrinsic motivation, while task autonomy, complexity, and variety were associated with intrinsic motivations [29,30,96]. In addition, solvers’ motivations could be influenced by how a task was designed for innovative solution search [68], and solvers could be motived by the linguistic cues used in the task instructions. Consequently, task instructions were encouraged to be simple and without many strict restrictions in order to stimulate solvers to develop solutions with high originality and value [101,102].

(3): Requester behaviors

Being fair and giving feedback were two important behaviors of requesters that had an impact on solvers’ efforts and contributions in a task and afterward continuous participation and contribution. These two requester behaviors acted as the incentives that appealed to solvers’ motives. More specifically, being fair was associated with solvers’ trust in requesters and solvers’ expectations to receive something in return for what they gave [71,114,117]. The expectations could be financial rewards, as well as non-financial benefits, according to solvers’ extrinsic and intrinsic motives to participate in crowdsourcing. Violation of distribution fairness resulted in violation of the rule of reciprocity in tangible materials, while violations of procedural and interactional fairness concerned the solvers’ anticipation of ideals and intangible values [22,94].

With respect to requester feedback, it could help to enhance solvers’ concentration on task accomplishment, goal pursuit, and social values [72,116]. Moreover, providing feedback was a way of social interaction with requesters for solvers and eliminating solvers’ worries about requester behavioral uncertainty [98,116]. In addition, it could be a very effective motivator and could facilitate tapping into the altruistic, benevolent, self-efficacy, and learning motivations of solvers [27,72,116].

(4): Solver characteristics

Solver characteristics produced an impact on solvers’ behaviors mainly through three ways. Firstly, solvers with different roles and characteristics had different motives and, thus, undertook different actions in crowdsourcing. For instance, Battistella and Nonino [12] showed that users with different roles were attracted by various motives. Wijnhoven, Ehrenhard and Kuhn [14] verified that participants with different citizens’ characteristics had different motives of involve in open government projects.

Second, solvers’ performances in a task were directly related to their characteristics, especially expertise, experience, and cultural background. For example, Bayus [18] found that an individual’s past success experience in generating implemented ideas negatively related to the likelihood of developing an implemented idea again. Bockstedt, Druehl and Mishra [21] and Chua, Roth and Lemoine [20] indicated that the national cultural background of solvers was associated with the number of ideas that they submitted in a given contest. Mack and Landau [7] confirmed that there was a form of an inverted U-shaped relationship between individuals’ domain-relevant skills and the innovative level of their proposed ideas. Khasraghi and Aghaie [19] demonstrated that most competitor participation history factors, including participation frequency, winning frequency, and last performance, had a significant effect on their performance in a contest.

Finally, solvers’ characteristics influenced their perception of task attributes, requester behaviors, and platform designs. In other words, understanding of a same object may vary from solver to solver. For instance, with respect to task attributes, solvers with different expertise and experience had different ideas against the instructions, autonomy, and even requester exemplars of a task [54,103]. Liu, Yang, Adamic and Chen [75], and it was found that experienced users were more likely to select tasks with high rewards than inexperienced users. In addition, regarding requester feedback, female users were more effectively motivated to generate more content than male users by cooperatively framed feedback, whereas competitively framed feedback was more effective at motivating male users [116]. Additionally, Chan, Li, Ni and Zhu [72] found that experienced ideators were less demotivated to improve their subsequent ideation performance when they received negative firm feedback, while inexperienced ideators were more motivated by positive firm feedback.

It was noted that the accumulated participation actions of a solver updated his or her individual characteristics, including skills, interests, participation, and past success experience. In the meantime, they were likely to change a solver’s motives and perception of various incentives [80,92,93]. These transformations could, in turn, exert positive or negative influences on solvers’ future participation behaviors.

(5): Platform designs

In this study, we mainly paid attention on four important platform designs, which were system-generated visual feedback, being trustworthy, incentive design, and fostering a community. However, other designs, such as norms of IPR allocation [119], explicit instructions [124], and user interface [84] have also been considered by solvers and have influence on solvers’ behaviors. Generally, platform designs are activation-supporting IT artifacts for influencing solvers’ behaviors [82,83,121,122,130]. These IT artifacts are directly responsive to the incentives that match solvers’ motives. For instance, a diversity of prizes, e.g., requester prize and platform prize, and fees, e.g., membership fee, service fee, and possible taxes, were offered to match the motive of earning monetary rewards. With respect to the motive of learning, the platforms offered solvers three approaches to practice their skills, which were feedback, disclosed solutions, and suggestions including techniques, tips, and tricks for developing solutions. As another example, virtual prizes and solver show systems were two kinds of designs for matching the motive of gaining reputation and self-demonstration.

Second, platform designs provided solvers with a good environment to participate in and contribute to. They not only included the provided work opportunities, such as various tasks and real problems, but also included a space, such as a community or forum, for solvers to interact with peers and requesters. Last but not least, platform designs supported solvers in performing effectively in a platform. The support covered the full process of solver participation, including pre-participation, task selection, submission development, post evaluation, winner selection, completion of payment, IPR transfer and use, and further handling of possible conflicts and disputes.

4. Discussion and Identification of Future Research Needs

Based on the aforementioned results, this section aims to suggest potential ways to move research on solvers’ behaviors and their influential factors forward.

4.1. Linking the Factors Together and Investigating Their Relations

The selected articles suggested a wide range of factors and further explored their relations with solvers’ behaviors. However, there was a lack of in-depth investigation on how these identified factors were related and how they might affect each other.

Regarding the relationships among the factors, a focus is required on the interactions of solvers’ motives. As indicated in Section 3.3.2, only several of the studies looked into the relations of solvers’ motives and their interactive impact on solvers’ behaviors [10,33,92]. Most of our reviewed studies considered the influences of solvers’ motives separately.

Solvers’ expectations and motives exerted direct effect on their decision to participate and contribute in KI-C [33,121]. As indicated in Section 3.5, the other identified factors influenced solvers’ behaviors mainly through affecting their perception of the likelihood of their motives or expectations being satisfied [32,56,71,121]. Therefore, another focus required is to explore the factors and their priorities of being responsive to solvers’ major motives. In addition, the appropriate forms, content, and strength or level of some factors are likely to be concerned when presenting them to activate solvers’ motives [70,102].

The last attention that is required is examining the relations of factors, including solver characteristics, task attributes, requester behaviors, and platform designs. Some studies have started to examine this aspect, such as similar feedback from requesters or a platform having different meanings for different solvers and regulating task design and requester behaviors by improving platform designs [22,55,79,98,119].

Consequently, it is important to understand these factors from a whole perspective rather than a solo perspective that views them independently. Therefore, an important work that might be needed in the future is to examine and map the relationships among these identified factors. Knowing this need, a framework is required initially to cover these factors and link them together. To achieve this, a few solid theories should be involved or developed as the foundation.

4.2. Exploring the Transformation of Solvers’ Behaviors and Their Influential Factors

In this study, we indicated that the behaviors exhibited by a solver interacted. Prior studies have made great efforts to investigate factors that influence solvers’ single behavior, while knowledge and practices about how to connect their interrelated behaviors and transfer one behavior to another one are limited. This results in difficulty for fully making use of the wisdom of solvers. For example, one of practical facts is that a large proportion of solvers do not participate in or contribute to a task after signing up. Therefore, for a crowdsourcing platform, it is imperative to focus on solvers’ initial participation and adopt effective incentives to promote them to participate deeply. In a word, if a crowdsourcing platform expects solvers to participate continuously, a lot of efforts should be made to ensure that solvers have good satisfaction in their initial participation behaviors. As a result, an important future research theme is investigating the transformation of solvers’ participation behaviors and the drivers, barriers, and factors that promote these transformations.

4.3. Improving Solvers’ Behaviors from a Global Perspective and a Local Perspective

This suggestion mainly concentrates on improving solvers’ behaviors from a practical perspective. Based on previous findings, solvers with different roles exhibited various behaviors that were stimulated by various motives. Further, the same factor had different meanings for different solvers. These conditions create some implications for crowdsourcing platforms and requesters to improve solvers’ behaviors. Specifically, on behalf of the platform benefits, managers of platforms would like to have more and more qualified solvers to register and contribute. If managers adopt a series of governance mechanisms or policies for a specific group of solvers with similar backgrounds and motives but neglect other solvers, it may be very helpful for improving the participation intention and effort of members in this group but result in the loss of other types of solvers with different backgrounds and motives. Therefore, we suggest that crowdsourcing platforms implement designs to improve solver behaviors from a local perspective (improving behaviors of a specific group of solvers) or a global perspective (improving behaviors of solvers with different roles and motives). Under this view, future studies may focus on investigating the roles that solvers play in KI-C, the differences in motives and behaviors of solvers with different roles, and the effects of factors identified in this study on the behaviors of solvers with different roles.

On behalf of requester benefits, it may be simpler since requesters may just handle a task in a platform at a time in general. From Table 3, we can find that task attributes, including prizes, complexity, instructions, and autonomy, requester fairness, and feedback had an effect on solvers’ behaviors. In addition, some scholars have started to highlight the importance for requesters to keep in contact with solvers who contribute to their tasks, especially with great performances, and view them as high-quality external assets [91,123,124,131]. These conditions could drive requesters to design their tasks and interact with solvers properly and positively in order to facilitate the achievement of the current task goals and long-term benefits as well.

5. Conclusions

This study aimed to build a comprehensive view of what the literature has reported on the factors that affect solver behaviors in KI-C. With a thorough selection process, a total of 119 articles were selected from the data sources of SCI-EXPANDED and SSCI. Our literature review indicated that there was an increase in studies on investigating solver behaviors in KI-C and their influential factors. Furthermore, empirical methods, especially surveys, case studies, field experiments, and primary data, and various theories were introduced to support research on this topic.

Furthermore, the review divided solvers’ participation in KI-C into five specific interactive behaviors, which were participating in a KI-C platform, choosing a task to participate in, making an effort in a task, contributing high-quality solutions, and continuous participation and contribution. With respect to each of these behaviors, eight major solver motives, i.e., monetary rewards, learning, career prospects, reputation or recognition, fun, self-efficacy, altruism, and sense of belonging to a community, were summarized. In addition, we identified seventeen specific factors under four groups, which were task attributes, solver characteristics, requester behaviors, and platform designs. It was noted that there were differences between the factors that influenced different behaviors. To understand these factors and solver motives more, their roles in affecting solver behaviors were demonstrated as well. With these results and the aim of moving the literature forward, we developed some research needs concentrating on three themes. They were exploring the inner relationships among the identified factors, examining the transformation in solvers’ behaviors and their influential factors, and suggesting global and local perspectives for improving solvers’ participation behaviors in KI-C.

This study still also suffers from a couple of limitations. First of all, we conducted a thorough review of journal articles from 2006 to 2021, but we excluded conference papers, which accounted for a proportion of the related studies, and could not ensure the capture of all the material related to the topic with the increasing number of studies. Second, this study extracted the factors presented in the literature that influenced solvers’ behaviors but paid less attention to investigating and summarizing how each factor influenced each of the solvers’ behaviors specifically. Another limitation was the limit of generalizing our results in KI-C for other crowdsourcing systems, such as crowdsourcing of simple tasks.

Author Contributions

X.Z.: conceptualization, methodology, formal analysis, investigation, data curation, validation, visualization, writing—original draft, review, and editing. E.X.: visualization, supervision, validation, writing—review and editing. C.S. and J.S.: visualization, validation, writing—review and editing. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by the National Natural Science Foundation of China (71802002), the Anhui Education Department (gxyqZD2022045), and Excellent Young Scholars funded by Anhui Polytechnic University and the Anhui Education Department (2008085QG336).

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Not applicable.

Acknowledgments

We sincerely appreciate Mats Magnusson, a distinguished professor of KTH Royal Institute of Technology, for presenting constructive and very helpful suggestions especially about review methodology and writing.

Conflicts of Interest

The authors declare no conflict of interest.

References

Basu Roy, S.; Lykourentzou, I.; Thirumuruganathan, S.; Amer-Yahia, S.; Das, G. Task assignment optimization in knowledge-intensive crowdsourcing. VLDB J. 2015, 24, 467–491. [Google Scholar] [CrossRef]
Kittur, A.; Nickerson, J.V.; Bernstein, M.; Gerber, E.; Shaw, A.; Zimmerman, J.; Lease, M.; Horton, J. The future of crowd work. In Proceedings of the 2013 Conference on Computer Supported Cooperative Work, San Antonio, TX, USA, 23–27 February 2013; pp. 1301–1318. [Google Scholar]
Terwiesch, C.; Xu, Y. Innovation Contests, Open Innovation, and Multiagent Problem Solving. Manag. Sci. 2008, 54, 1529–1543. [Google Scholar] [CrossRef] [Green Version]
Zhao, Y.; Zhu, Q. Evaluation on crowdsourcing research: Current status and future direction. Inf. Syst. Front. 2014, 16, 417–434. [Google Scholar] [CrossRef]
Cappa, F.; Oriani, R.; Pinelli, M.; De Massis, A. When does crowdsourcing benefit firm stock market performance? Res. Policy 2019, 48, 103825. [Google Scholar] [CrossRef]
Penin, J.; Burger-Helmchen, T. Crowdsourcing of inventive activities: Definition and limits. Int. J. Innov. Sustain. Dev. 2011, 5, 246–263. [Google Scholar] [CrossRef]
Mack, T.; Landau, C. Submission quality in open innovation contests—An analysis of individual-level determinants of idea innovativeness. RD Manag. 2020, 50, 47–62. [Google Scholar] [CrossRef]
Estellés-Arolas, E. Towards an integrated crowdsourcing definition. J. Inf. Sci. 2012, 38, 189–200. [Google Scholar] [CrossRef] [Green Version]
Ye, H.; Kankanhalli, A. Solvers’ participation in crowdsourcing platforms: Examining the impacts of trust, and benefit and cost factors. J. Strateg. Inf. Syst. 2017, 26, 101–117. [Google Scholar] [CrossRef]
Sun, Y.; Fang, Y.; Lim, K.H. Understanding sustained participation in transactional virtual communities. Decis. Support Syst. 2012, 53, 12–22. [Google Scholar] [CrossRef]
Wang, X.; Khasraghi, H.J.; Schneider, H. Towards an Understanding of Participants’ Sustained Participation in Crowdsourcing Contests. Inf. Syst. Manag. 2020, 37, 213–226. [Google Scholar] [CrossRef]
Battistella, C.; Nonino, F. Exploring the impact of motivations on the attraction of innovation roles in open innovation web-based platforms. Prod. Plan. Control. 2013, 24, 226–245. [Google Scholar] [CrossRef]
Chris Zhao, Y.; Zhu, Q. Effects of extrinsic and intrinsic motivation on participation in crowdsourcing contest: A perspective of self-determination theory. Online Inf. Rev. 2014, 38, 896–917. [Google Scholar] [CrossRef]
Wijnhoven, F.; Ehrenhard, M.; Kuhn, J. Open government objectives and participation motivations. Gov. Inf. Q. 2015, 32, 30–42. [Google Scholar] [CrossRef]
Cappa, F.; Rosso, F.; Hayes, D. Monetary and Social Rewards for Crowdsourcing. Sustainability 2019, 11, 2834. [Google Scholar] [CrossRef] [Green Version]
Zhu, J.J.; Li, S.Y.; Andrews, M. Ideator Expertise and Cocreator Inputs in Crowdsourcing-Based New Product Development. J. Prod. Innov. Manag. 2017, 34, 598–616. [Google Scholar] [CrossRef]
Piazza, M.; Mazzola, E.; Perrone, G. How can I signal my quality to emerge from the crowd? A study in the crowdsourcing context. Technol. Forecast. Soc. Chang. 2022, 176, 121473. [Google Scholar] [CrossRef]
Bayus, B.L. Crowdsourcing New Product Ideas over Time: An Analysis of the Dell IdeaStorm Community. Manag. Sci. 2013, 59, 226–244. [Google Scholar] [CrossRef]
Khasraghi, H.J.; Aghaie, A. Crowdsourcing contests: Understanding the effect of competitors’ participation history on their performance. Behav. Inf. Technol. 2014, 33, 1383–1395. [Google Scholar] [CrossRef]
Chua, R.Y.J.; Roth, Y.; Lemoine, J.-F. The Impact of Culture on Creativity: How Cultural Tightness and Cultural Distance Affect Global Innovation Crowdsourcing Work. Adm. Sci. Q. 2014, 60, 189–227. [Google Scholar] [CrossRef]
Bockstedt, J.; Druehl, C.; Mishra, A. Problem-solving effort and success in innovation contests: The role of national wealth and national culture. J. Oper. Manag. 2015, 36, 187–200. [Google Scholar] [CrossRef]
Franke, N.; Keinz, P.; Klausberger, K. “Does This Sound Like a Fair Deal?”: Antecedents and Consequences of Fairness Expectations in the Individual’s Decision to Participate in Firm Innovation. Organ. Sci. 2012, 24, 1495–1516. [Google Scholar] [CrossRef] [Green Version]
Zou, L.; Zhang, J.; Liu, W. Perceived justice and creativity in crowdsourcing communities: Empirical evidence from China. Soc. Sci. Inf. 2015, 54, 253–279. [Google Scholar] [CrossRef]
Kohler, T.; Chesbrough, H. From collaborative community to competitive market: The quest to build a crowdsourcing platform for social innovation. RD Manag. 2019, 49, 356–368. [Google Scholar] [CrossRef]
Randhawa, K.; Wilden, R.; West, J. Crowdsourcing without profit: The role of the seeker in open social innovation. RD Manag. 2019, 49, 298–317. [Google Scholar] [CrossRef]
Heo, M.; Toomey, N. Motivating continued knowledge sharing in crowdsourcing: The impact of different types of visual feedback. Online Inf. Rev. 2015, 39, 795–811. [Google Scholar] [CrossRef]
Wooten, J.O.; Ulrich, K.T. Idea Generation and the Role of Feedback: Evidence from Field Experiments with Innovation Tournaments. Prod. Oper. Manag. 2017, 26, 80–99. [Google Scholar] [CrossRef]
Steils, N.; Hanine, S. Recruiting valuable participants in online IDEA generation: The role of brief instructions. J. Bus. Res. 2019, 96, 14–25. [Google Scholar] [CrossRef]
Zheng, H.; Li, D.; Hou, W. Task Design, Motivation, and Participation in Crowdsourcing Contests. Int. J. Electron. Commer. 2011, 15, 57–88. [Google Scholar] [CrossRef] [Green Version]
Garcia Martinez, M. Inspiring crowdsourcing communities to create novel solutions: Competition design and the mediating role of trust. Technol. Forecast. Soc. Chang. 2017, 117, 296–304. [Google Scholar] [CrossRef] [Green Version]
Tang, J.; Zhou, X.; Zhao, Y.; Wang, T. How the type and valence of feedback information influence volunteers’ knowledge contribution in citizen science projects. Inf. Process. Manag. 2021, 58, 102633. [Google Scholar] [CrossRef]
Taylor, J.; Joshi, K.D. Joining the crowd: The career anchors of information technology workers participating in crowdsourcing. Inf. Syst. J. 2019, 29, 641–673. [Google Scholar] [CrossRef]
Pee, L.G.; Koh, E.; Goh, M. Trait motivations of crowdsourcing and task choice: A distal-proximal perspective. Int. J. Inf. Manag. 2018, 40, 28–41. [Google Scholar] [CrossRef]
Nakatsu, R.T.; Grossman, E.B.; Iacovou, C.L. A taxonomy of crowdsourcing based on task complexity. J. Inf. Sci. 2014, 40, 823–834. [Google Scholar] [CrossRef]
Cricelli, L.; Grimaldi, M.; Vermicelli, S. Crowdsourcing and open innovation: A systematic literature review, an integrated framework and a research agenda. Rev. Manag. Sci. 2021, 16, 1269–1310. [Google Scholar] [CrossRef]
Olson, D.L.; Rosacker, K. Crowdsourcing and open source software participation. Serv. Bus. 2013, 7, 499–511. [Google Scholar] [CrossRef]
Martinez-Corral, A.; Grijalvo, M.; Palacios, M. An organisational framework for analysis of crowdsourcing initiatives. Int. J. Entrep. Behav. Res. 2019, 25, 1652–1670. [Google Scholar] [CrossRef]
Modaresnezhad, M.; Iyer, L.; Palvia, P.; Taras, V. Information Technology (IT) enabled crowdsourcing: A conceptual framework. Inf. Process. Manag. 2020, 57, 102135. [Google Scholar] [CrossRef]
Hossain, M.; Kauranen, I. Crowdsourcing: A comprehensive literature review. Strateg. Outsourcing: Int. J. 2015, 8, 2–22. [Google Scholar] [CrossRef]
Nevo, D.; Kotlarsky, J. Crowdsourcing as a strategic IS sourcing phenomenon: Critical review and insights for future research. J. Strateg. Inf. Syst. 2020, 29, 101593. [Google Scholar] [CrossRef]
Assis Neto, F.R.; Santos, C.A.S. Understanding crowdsourcing projects: A systematic review of tendencies, workflow, and quality management. Inf. Process. Manag. 2018, 54, 490–506. [Google Scholar] [CrossRef]
Wang, C.; Han, L.; Stein, G.; Day, S.; Bien-Gund, C.; Mathews, A.; Ong, J.J.; Zhao, P.-Z.; Wei, S.-F.; Walker, J.; et al. Crowdsourcing in health and medical research: A systematic review. Infect. Dis. Poverty 2020, 9, 8. [Google Scholar] [CrossRef] [Green Version]
Crescenzi, V.; Fernandes, A.A.A.; Merialdo, P.; Paton, N.W. Crowdsourcing for data management. Knowl. Inf. Syst. 2017, 53, 1–41. [Google Scholar] [CrossRef]
Ghezzi, A.; Gabelloni, D.; Martini, A.; Natalicchio, A. Crowdsourcing: A Review and Suggestions for Future Research. Int. J. Manag. Rev. 2018, 20, 343–363. [Google Scholar] [CrossRef]
Jiang, J.; An, B.; Jiang, Y.; Lin, D.; Bu, Z.; Cao, J.; Hao, Z. Understanding Crowdsourcing Systems from a Multiagent Perspective and Approach. ACM Trans. Auton. Adapt. Syst. 2018, 13, 8. [Google Scholar] [CrossRef]
Xia, H.; McKernan, B. Privacy in Crowdsourcing: A Review of the Threats and Challenges. Comput. Support. Coop. Work (CSCW) 2020, 29, 263–301. [Google Scholar] [CrossRef]
Jain, S.; Deodhar, S.J. Social mechanisms in crowdsourcing contests: A literature review. Behav. Inf. Technol. 2021, 41, 1080–1114. [Google Scholar] [CrossRef]
Morschheuser, B.; Hamari, J.; Koivisto, J.; Maedche, A. Gamified crowdsourcing: Conceptualization, literature review, and future agenda. Int. J. Hum. -Comput. Stud. 2017, 106, 26–43. [Google Scholar] [CrossRef] [Green Version]
Zhao, Y.; Zhu, Q. Conceptualizing task affordance in online crowdsourcing context. Online Inf. Rev. 2016, 40, 938–958. [Google Scholar] [CrossRef]
Thuan, N.H.; Antunes, P.; Johnstone, D. Factors influencing the decision to crowdsource: A systematic literature review. Inf. Syst. Front. 2016, 18, 47–68. [Google Scholar] [CrossRef]
Tranfield, D.; Denyer, D.; Smart, P. Towards a Methodology for Developing Evidence-Informed Management Knowledge by Means of Systematic Review. Br. J. Manag. 2003, 14, 207–222. [Google Scholar] [CrossRef]
Crossan, M.M.; Apaydin, M. A Multi-Dimensional Framework of Organizational Innovation: A Systematic Review of the Literature. J. Manag. Stud. 2010, 47, 1154–1191. [Google Scholar] [CrossRef]
Zhang, X.; Chen, Q. Towards an understanding of the decision process of solvers’ participation in crowdsourcing contests for problem solving. Behav. Inf. Technol. 2021, 1–19. [Google Scholar] [CrossRef]
Durward, D.; Blohm, I.; Leimeister, J.M. The Nature of Crowd Work and its Effects on Individuals’ Work Perception. J. Manag. Inf. Syst. 2020, 37, 66–95. [Google Scholar] [CrossRef] [Green Version]
Blohm, I.; Zogaj, S.; Bretschneider, U.; Leimeister, J.M. How to Manage Crowdsourcing Platforms Effectively? Calif. Manag. Rev. 2018, 60, 122–149. [Google Scholar] [CrossRef]
Feng, Y.; Jonathan Ye, H.; Yu, Y.; Yang, C.; Cui, T. Gamification artifacts and crowdsourcing participation: Examining the mediating role of intrinsic motivations. Comput. Hum. Behav. 2018, 81, 124–136. [Google Scholar] [CrossRef]
Li, D.; Hu, L. Exploring the effects of reward and competition intensity on participation in crowdsourcing contests. Electron. Mark. 2017, 27, 199–210. [Google Scholar] [CrossRef]
Deodhar, S.J. Interplay between Constraints and Rewards in Innovation Tournaments: Implications for Participation. Int. J. Coop. Inf. Syst. 2020, 29, 2040004. [Google Scholar] [CrossRef]
Liu, Z.; Hatton, M.R.; Kull, T.; Dooley, K.; Oke, A. Is a large award truly attractive to solvers? The impact of award size on crowd size in innovation contests. J. Oper. Manag. 2020, 67, 420–449. [Google Scholar] [CrossRef]
Liang, H.; Wang, M.-M.; Wang, J.-J.; Xue, Y. How intrinsic motivation and extrinsic incentives affect task effort in crowdsourcing contests: A mediated moderation model. Comput. Hum. Behav. 2018, 81, 168–176. [Google Scholar] [CrossRef]
Li, J.; Wang, Y.; Yu, D.; Liu, C. Solvers’ committed resources in crowdsourcing marketplace: Do task design characteristics matter? Behav. Inf. Technol. 2021, 41, 1689–1708. [Google Scholar] [CrossRef]
Dissanayake, I.; Zhang, J.; Yasar, M.; Nerur, S.P. Strategic effort allocation in online innovation tournaments. Inf. Manag. 2018, 55, 396–406. [Google Scholar] [CrossRef]
Deodhar, S. Different eyes on the same prize: Implications of entry timing heterogeneity and incentives for contestant effort in innovation tournament. Inf. Technol. People 2020, 34, 526–556. [Google Scholar] [CrossRef]
Ogink, T.; Dong, J.Q. Stimulating innovation by user feedback on social media: The case of an online user innovation community. Technol. Forecast. Soc. Chang. 2019, 144, 295–302. [Google Scholar] [CrossRef]
Han, L.; Roitero, K.; Gadiraju, U.; Sarasua, C.; Checco, A.; Maddalena, E.; Demartini, G. The Impact of Task Abandonment in Crowdsourcing. IEEE Trans. Knowl. Data Eng. 2021, 33, 2266–2279. [Google Scholar] [CrossRef]
Boons, M.; Stam, D. Crowdsourcing for innovation: How related and unrelated perspectives interact to increase creative performance. Res. Policy 2019, 48, 1758–1770. [Google Scholar] [CrossRef]
Dissanayake, I.; Zhang, J.; Gu, B. Task Division for Team Success in Crowdsourcing Contests: Resource Allocation and Alignment Effects. J. Manag. Inf. Syst. 2015, 32, 8–39. [Google Scholar] [CrossRef]
Acar, O.A. Motivations and solution appropriateness in crowdsourcing challenges for innovation. Res. Policy 2019, 48, 103716. [Google Scholar] [CrossRef]
Heo, M.; Toomey, N. Supporting sustained willingness to share knowledge with visual feedback. Comput. Hum. Behav. 2016, 54, 388–396. [Google Scholar] [CrossRef]
Piezunka, H.; Dahlander, L. Idea Rejected, Tie Formed: Organizations’ Feedback on Crowdsourced Ideas. Acad. Manag. J. 2018, 62, 503–530. [Google Scholar] [CrossRef] [Green Version]
Liu, Y.; Liu, Y.M. The effect of workers’ justice perception on continuance participation intention in the crowdsourcing market. Internet Res. 2019, 29, 1485–1508. [Google Scholar] [CrossRef]
Chan, K.W.; Li, S.Y.; Ni, J.; Zhu, J.J. What Feedback Matters? The Role of Experience in Motivating Crowdsourcing Innovation. Prod. Oper. Manag. 2021, 30, 103–126. [Google Scholar] [CrossRef]
Garcia Martinez, M. Solver engagement in knowledge sharing in crowdsourcing communities: Exploring the link to creativity. Res. Policy 2015, 44, 1419–1430. [Google Scholar] [CrossRef]
Soliman, W.; Tuunainen, V.K. Understanding Continued Use of Crowdsourcing Systems: An Interpretive Study. J. Theor. Appl. Electron. Commer. Res. 2015, 10, 1–18. [Google Scholar] [CrossRef] [Green Version]
Liu, T.X.; Yang, J.; Adamic, L.A.; Chen, Y. Crowdsourcing with All-Pay Auctions: A Field Experiment on Taskcn. Manag. Sci. 2014, 60, 2020–2037. [Google Scholar] [CrossRef] [Green Version]
Battistella, C.; Nonino, F. Open innovation web-based platforms: The impact of different forms of motivation on collaboration. Innovation 2012, 14, 557–575. [Google Scholar] [CrossRef]
Tang, C.; Li, X.; Cao, M.; Zhang, Z.; Yu, X. Incentive Mechanism for Macrotasking Crowdsourcing: A Zero-Determinant Strategy Approach. IEEE Internet Things J. 2019, 6, 8589–8601. [Google Scholar] [CrossRef]
Xie, H.; Lui, J.C.S. Incentive Mechanism and Rating System Design for Crowdsourcing Systems: Analysis, Tradeoffs and Inference. IEEE Trans. Serv. Comput. 2018, 11, 90–102. [Google Scholar] [CrossRef]
Ebner, W.; Leimeister, J.M.; Krcmar, H. Community engineering for innovations: The ideas competition as a method to nurture a virtual community for innovations. RD Manag. 2009, 39, 342–356. [Google Scholar] [CrossRef] [Green Version]
Alam, S.L.; Campbell, J. Temporal Motivations of Volunteers to Participate in Cultural Crowdsourcing Work. Inf. Syst. Res. 2017, 28, 744–759. [Google Scholar] [CrossRef]
Brabham, D.C. Moving the crowd at threadless. Inf. Commun. Soc. 2010, 13, 1122–1145. [Google Scholar] [CrossRef]
Xu, L.; Nian, T.; Cabral, L. What Makes Geeks Tick? A Study of Stack Overflow Careers. Manag. Sci. 2019, 66, 587–604. [Google Scholar] [CrossRef] [Green Version]
Lee, J.; Park, H.; Zaggl, M. When to Signal? Contingencies for Career-Motivated Contributions in Online Collaboration Communities. J. Assoc. Inf. Syst. 2022. [Google Scholar] [CrossRef]
Johnson, E.; Liew Chern, L. Engagement-oriented design: A study of New Zealand public cultural heritage institutions crowdsourcing platforms. Online Inf. Rev. 2020, 44, 887–912. [Google Scholar] [CrossRef]
Bakici, T. Comparison of crowdsourcing platforms from social-psychological and motivational perspectives. Int. J. Inf. Manag. 2020, 54, 102121. [Google Scholar] [CrossRef]
Dissanayake, I.; Mehta, N.; Palvia, P.; Taras, V.; Amoako-Gyampah, K. Competition matters! Self-efficacy, effort, and performance in crowdsourcing teams. Inf. Manag. 2019, 56, 103158. [Google Scholar] [CrossRef]
Sun, Y.; Wang, N.; Yin, C.; Zhang, J.X. Understanding the relationships between motivators and effort in crowdsourcing marketplaces: A nonlinear analysis. Int. J. Inf. Manag. 2015, 35, 267–276. [Google Scholar] [CrossRef]
Schäper, T.; Foege, J.N.; Nüesch, S.; Schäfer, S. Determinants of idea sharing in crowdsourcing: Evidence from the automotive industry. RD Manag. 2021, 51, 101–113. [Google Scholar] [CrossRef]
Tinati, R.; Luczak-Roesch, M.; Simperl, E.; Hall, W. An investigation of player motivations in Eyewire, a gamified citizen science project. Comput. Hum. Behav. 2017, 73, 527–540. [Google Scholar] [CrossRef]
Wu, W.; Gong, X. Motivation and sustained participation in the online crowdsourcing community: The moderating role of community commitment. Internet Res. 2020, 31, 287–314. [Google Scholar] [CrossRef]
Fedorenko, I.; Berthon, P.; Rabinovich, T. Crowded identity: Managing crowdsourcing initiatives to maximize value for participants through identity creation. Bus. Horiz. 2017, 60, 155–165. [Google Scholar] [CrossRef]
Frey, K.; Lüthje, C.; Haag, S. Whom Should Firms Attract to Open Innovation Platforms? The Role of Knowledge Diversity and Motivation. Long Range Plan. 2011, 44, 397–420. [Google Scholar] [CrossRef]
Mack, T.; Landau, C. Winners, losers, and deniers: Self-selection in crowd innovation contests and the roles of motivation, creativity, and skills. J. Eng. Technol. Manag. 2015, 37, 52–64. [Google Scholar] [CrossRef]
Mazzola, E.; Piazza, M.; Acur, N.; Perrone, G. Treating the crowd fairly: Increasing the solvers’ self-selection in idea innovation contests. Ind. Mark. Manag. 2020, 91, 16–29. [Google Scholar] [CrossRef]
Acar, O.A. Harnessing the creative potential of consumers: Money, participation, and creativity in idea crowdsourcing. Mark. Lett. 2018, 29, 177–188. [Google Scholar] [CrossRef] [Green Version]
Lee, C.K.M.; Chan, C.; Ho, S.; Choy, K.; Ip, W. Explore the feasibility of adopting crowdsourcing for innovative problem solving. Ind. Manag. Data Syst. 2015, 115, 803–832. [Google Scholar] [CrossRef]
Zheng, H.; Xie, Z.; Hou, W.; Li, D. Antecedents of solution quality in crowdsourcing: The sponsor’s perspective. J. Electron. Commer. Res. 2014, 15, 212–224. [Google Scholar]
Jian, L.; Yang, S.; Ba, S.L.; Lu, L.; Jiang, L.C. Managing the Crowds: The Effect of Prize Guarantees and In-Process Feedback on Participation in Crowdsourcing Contests. Mis Q. 2019, 43, 97–112. [Google Scholar] [CrossRef] [Green Version]
Shao, B.; Shi, L.; Xu, B.; Liu, L. Factors affecting participation of solvers in crowdsourcing: An empirical study from China. Electron. Mark. 2012, 22, 73–82. [Google Scholar] [CrossRef]
Pollok, P.; Lüttgens, D.; Piller, F.T. Attracting solutions in crowdsourcing contests: The role of knowledge distance, identity disclosure, and seeker status. Res. Policy 2019, 48, 98–114. [Google Scholar] [CrossRef]
Vrgović, P.; Jošanov-Vrgović, I. Crowdsourcing user solutions: Which questions should companies ask to elicit the most ideas from its users? Innovation 2017, 19, 452–462. [Google Scholar] [CrossRef]
Gillier, T.; Chaffois, C.; Belkhouja, M.; Roth, Y.; Bayus, B.L. The effects of task instructions in crowdsourcing innovative ideas. Technol. Forecast. Soc. Chang. 2018, 134, 35–44. [Google Scholar] [CrossRef]
Koh, T.K. Adopting Seekers’ Solution Exemplars in Crowdsourcing Ideation Contests: Antecedents and Consequences. Inf. Syst. Res. 2019, 30, 486–506. [Google Scholar] [CrossRef]
Mazzola, E.; Acur, N.; Piazza, M.; Perrone, G. “To Own or Not to Own?” A Study on the Determinants and Consequences of Alternative Intellectual Property Rights Arrangements in Crowdsourcing for Innovation Contests. J. Prod. Innov. Manag. 2018, 35, 908–929. [Google Scholar] [CrossRef]
Natalicchio, A.; Messeni Petruzzelli, A.; Garavelli, A.C. Innovation problems and search for solutions in crowdsourcing platforms—A simulation approach. Technovation 2017, 64–65, 28–42. [Google Scholar] [CrossRef]
Körpeoğlu, E.; Cho, S.-H. Incentives in Contests with Heterogeneous Solvers. Manag. Sci. 2017, 64, 2709–2715. [Google Scholar] [CrossRef] [Green Version]
Zhang, X.; Gong, B.; Cao, Y.; Ding, Y.; Su, J. Investigating participants’ attributes for participant estimation in knowledge-intensive crowdsourcing: A fuzzy DEMATEL based approach. Electron. Commer. Res. 2022, 22, 811–842. [Google Scholar] [CrossRef]
Zhang, X.; Chen, S.; Zhao Yuxiang, C.; Song, S.; Zhu, Q. The influences of social value orientation and domain knowledge on crowdsourcing manuscript transcription: An empirical investigation of the Transcribe-Sheng project. Aslib J. Inf. Manag. 2019, 72, 219–242. [Google Scholar] [CrossRef]
Shi, X.; Pan, W.; Shan, W. Understanding the effects of personality traits on solver engagement in crowdsourcing communities: A moderated mediation investigation. Inf. Technol. People 2022, 35, 750–780. [Google Scholar] [CrossRef]
Majchrzak, A.; Malhotra, A.; Zaggl, M.A. How Open Crowds Self-Organize. Acad. Manag. Discov. 2020, 7, 104–129. [Google Scholar] [CrossRef]
Zhu, H.; Kock, A.; Wentker, M.; Leker, J. How Does Online Interaction Affect Idea Quality? The Effect of Feedback in Firm-Internal Idea Competitions. J. Prod. Innov. Manag. 2019, 36, 24–40. [Google Scholar] [CrossRef] [Green Version]
Chan, K.W.; Li, S.Y.; Zhu, J.J. Fostering Customer Ideation in Crowdsourcing Community: The Role of Peer-to-peer and Peer-to-firm Interactions. J. Interact. Mark. 2015, 31, 42–62. [Google Scholar] [CrossRef]
Ren, R.; Yan, B.; Jian, L. Show me your expertise before teaming up. Internet Res. 2020, 30, 845–868. [Google Scholar] [CrossRef]
Faullant, R.; Fueller, J.; Hutter, K. Fair play: Perceived fairness in crowdsourcing competitions and the customer relationship-related consequences. Manag. Decis. 2017, 55, 1924–1941. [Google Scholar] [CrossRef] [Green Version]
Camacho, N.; Nam, H.; Kannan, P.K.; Stremersch, S. Tournaments to Crowdsource Innovation: The Role of Moderator Feedback and Participation Intensity. J. Mark. 2019, 83, 138–157. [Google Scholar] [CrossRef] [Green Version]
Huang, N.; Burtch, G.; Gu, B.; Hong, Y.; Liang, C.; Wang, K.; Fu, D.; Yang, B. Motivating User-Generated Content with Performance Feedback: Evidence from Randomized Field Experiments. Manag. Sci. 2019, 65, 327–345. [Google Scholar] [CrossRef]
Wang, M.-M.; Wang, J.-J. Understanding Solvers’ Continuance Intention in Crowdsourcing Contest Platform: An Extension of Expectation-Confirmation Model. J. Theor. Appl. Electron. Commer. Res. 2019, 14, 17–33. [Google Scholar] [CrossRef] [Green Version]
Boons, M.; Stam, D.; Barkema, H.G. Feelings of Pride and Respect as Drivers of Ongoing Member Activity on Crowdsourcing Platforms. J. Manag. Stud. 2015, 52, 717–741. [Google Scholar] [CrossRef]
Bauer, J.; Franke, N.; Tuertscher, P. Intellectual Property Norms in Online Communities: How User-Organized Intellectual Property Regulation Supports Innovation. Inf. Syst. Res. 2016, 27, 724–750. [Google Scholar] [CrossRef] [Green Version]
Hanine, S.; Steils, N. Ideation contests: Crowd management and valorization to avoid negative feelings of participants. Creat. Innov. Manag. 2019, 28, 425–435. [Google Scholar] [CrossRef]
Leimeister, J.M.; Huber, M.; Bretschneider, U.; Krcmar, H. Leveraging Crowdsourcing: Activation-Supporting Components for IT-Based Ideas Competition. J. Manag. Inf. Syst. 2009, 26, 197–224. [Google Scholar] [CrossRef] [Green Version]
Ren, J.; Ozturk, P.; Yeoh, W. Online Crowdsourcing Campaigns: Bottom-Up versus Top-Down Process Model. J. Comput. Inf. Syst. 2019, 59, 266–276. [Google Scholar] [CrossRef]
Kohler, T. How to Scale Crowdsourcing Platforms. Calif. Manag. Rev. 2017, 60, 98–121. [Google Scholar] [CrossRef]
Malhotra, A.; Majchrzak, A. Managing Crowds in Innovation Challenges. Calif. Manag. Rev. 2014, 56, 103–123. [Google Scholar] [CrossRef]
Schörpf, P.; Flecker, J.; Schönauer, A.; Eichmann, H. Triangular love–hate: Management and control in creative crowdworking. New Technol. Work Employ. 2017, 32, 43–58. [Google Scholar] [CrossRef]
Schäfer, S.; Antons, D.; Lüttgens, D.; Piller, F.; Salge, T.O. Talk to Your Crowd. Res. -Technol. Manag. 2017, 60, 33–42. [Google Scholar] [CrossRef]
Pacauskas, D.; Rajala, R.; Westerlund, M.; Mäntymäki, M. Harnessing user innovation for social media marketing: Case study of a crowdsourced hamburger. Int. J. Inf. Manag. 2018, 43, 319–327. [Google Scholar] [CrossRef]
Rosestiel Von, L. Grundlagen der Organisationspsychologie: Basiswissen und Anwendungshinweise [Basics of Organizational Psychology]; Schäffer-Poeschel: Stuttgart, Germany, 2007. [Google Scholar]
Ryan, R.M.; Deci, E.L. Intrinsic and Extrinsic Motivations: Classic Definitions and New Directions. Contemp. Educ. Psychol. 2000, 25, 54–67. [Google Scholar] [CrossRef]
Zhang, X.; Su, J.; Herrera-Viedma, E. A decision support model for estimating participation-oriented designs of crowdsourcing platforms based on quality function deployment. Expert Syst. Appl. 2022, 202, 117308. [Google Scholar] [CrossRef]
May, A.; Ross, T. The design of civic technology: Factors that influence public participation and impact. Ergonomics 2018, 61, 214–225. [Google Scholar] [CrossRef]

Figure 1. A framework for demonstrating the roles of the identified factors.

Table 1. Publications by year from 2006 to 2021.

Year	2009	2010	2011	2012	2013	2014	2015	2016	2017	2018	2019	2020	2021
No. of articles	2	0	2	3	3	3	13	6	15	13	25	21	13

Table 2. Solvers’ participation behaviors and their indicators.

Solvers’ Behaviors	Major Indicators	No. of Relevant Articles
Participating in a KI-C platform	solvers’ self-expressed willingness to participate in a platform	21
Choosing a task to participate in	solvers’ self-expressed willingness to participate in a task, the number of engaged solvers in a task, and task choice	31
Making effort in a task	physical, cognitive, and emotional engagement of energies exerted in a task; number of solutions and comments that solvers submit in a task	30
Contributing high-quality solutions	the quality of a solution evaluated by requesters, team rank in the contest, the likelihood of solution implementation, solution innovativeness, and solution appropriateness	36
Continuous participation and contribution	solvers’ self-expressed willingness to participate and contribute further	19

Table 3. The factors influencing the solvers’ behaviors in KI-C.

Categories	Specific Factors	Participating in a KI-C Platform	Choosing a Task to Participate in	Making Effort in a Task	Contributing High-Quality Solutions	Continuous Participation and Contribution
Task attributes (37)	Monetary reward (21)	-	10	11	5	-
	Task complexity (7)	-	4	3	-	-
	Task autonomy (6)	-	3	3	2	-
	Task variety (6)	-	3	3	2	-
	Task instruction (8)	-	4	3	4	-
	Task in-process status (8)	-	4	4	-	-
Solver characteristics (22)	Domain expertise (10)	-	2	2	7	-
	Participation experience (5)	-	-	-	3	2
	Cultural background (2)	-	1	1	1	-
	Personality traits (6)	-	3	-	3	-
	Interaction with peers (4)	-	-	2	2	-
Requester behaviors (19)	Fairness (7)	-	1	2	1	4
Requester behaviors (19)	Give feedback (13)	-	-	6	5	4
Platform designs (31)	System-generated visual feedback (5)	4	-	-	-	2
	Trustworthy (7)	1	-	-	-	6
	Incentive design (17)	9	-	2	5	2
	Foster a community (6)	6	-	-	-	5

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

Share and Cite

MDPI and ACS Style

Zhang, X.; Xia, E.; Shen, C.; Su, J. Factors Influencing Solvers’ Behaviors in Knowledge-Intensive Crowdsourcing: A Systematic Literature Review. J. Theor. Appl. Electron. Commer. Res. 2022, 17, 1297-1319. https://doi.org/10.3390/jtaer17040066

AMA Style

Zhang X, Xia E, Shen C, Su J. Factors Influencing Solvers’ Behaviors in Knowledge-Intensive Crowdsourcing: A Systematic Literature Review. Journal of Theoretical and Applied Electronic Commerce Research. 2022; 17(4):1297-1319. https://doi.org/10.3390/jtaer17040066

Chicago/Turabian Style

Zhang, Xuefeng, Enjun Xia, Chao Shen, and Jiafu Su. 2022. "Factors Influencing Solvers’ Behaviors in Knowledge-Intensive Crowdsourcing: A Systematic Literature Review" Journal of Theoretical and Applied Electronic Commerce Research 17, no. 4: 1297-1319. https://doi.org/10.3390/jtaer17040066

APA Style

Zhang, X., Xia, E., Shen, C., & Su, J. (2022). Factors Influencing Solvers’ Behaviors in Knowledge-Intensive Crowdsourcing: A Systematic Literature Review. Journal of Theoretical and Applied Electronic Commerce Research, 17(4), 1297-1319. https://doi.org/10.3390/jtaer17040066

Article Menu

Factors Influencing Solvers’ Behaviors in Knowledge-Intensive Crowdsourcing: A Systematic Literature Review

Abstract

1. Introduction

2. Methods Used

2.1. Planning the Review

2.2. Conducting the Review

2.2.1. Identification of Keywords and Search Terms

2.2.2. Assessment and Selection of Studies

2.2.3. Data Extraction

3. Results and Analysis

3.1. Descriptive Analysis

3.1.1. Publications by Year

3.1.2. Publications by Journal

3.1.3. Types of Knowledge-Intensive Task

3.1.4. Research Methods Used

3.1.5. Theoretical Foundations

3.2. RQ1: What Behaviors Do Solvers Exhibit in KI-C?

3.2.1. Participating in a KI-C Platform

3.2.2. Choosing a Task to Participate in

3.2.3. Making Effort in a Task

3.2.4. Contributing High-Quality Solutions

3.2.5. Continuous Participation and Contribution

3.3. RQ2: What (Motives) Motivates Solvers to Exhibit the Behaviors?

3.3.1. The Solver Motives

3.3.2. The Interactive Effect of Solvers’ Motives

3.3.3. The Transformation of Solvers’ Motives

3.4. RQ3: What Factors Influence the Solvers’ Behaviors?

3.4.1. Task Attributes

3.4.2. Solver Characteristics

3.4.3. Requester Behaviors

3.4.4. Platform Designs

3.5. RQ4: What Roles Do the Factors Play in the Solvers’ Behaviors?

4. Discussion and Identification of Future Research Needs

4.1. Linking the Factors Together and Investigating Their Relations

4.2. Exploring the Transformation of Solvers’ Behaviors and Their Influential Factors

4.3. Improving Solvers’ Behaviors from a Global Perspective and a Local Perspective

5. Conclusions

Author Contributions

Funding

Institutional Review Board Statement

Informed Consent Statement

Data Availability Statement

Acknowledgments

Conflicts of Interest

References

Share and Cite

Article Metrics

Article Access Statistics

Further Information

Guidelines

MDPI Initiatives

Follow MDPI