Measuring Users’ Psychophysiological Experience in Non-linear Omnichannel Environment

As the use of technological products in the complex user journeys has increased, both the physical and digital worlds need to be considered in measuring the customer’s experience. Enabled by technology, users expect to interact with companies interchangeably across channels (Carroll and Guzmán 2013). Consequently, consumers’ path can be more or less direct, but the use of different channels within the same journey favor non-linearity. Their journey can encompass multiple and much less predictable exits, even as consumers can circle back to previous choices and steps until their final purchase (Carroll and Guzmán 2013). Hence, measuring customer experience requires to embody multiple sets of non-linear interactions. In a physical context of interactions, the uniqueness of the user’s experience and the ecological validity issues have to be taken into account. Various methods already exist to measure customer journeys, but those methods rely mostly on retrospective measures, like questionnaires, interviews, or focus groups. For example, Customer Experience Modelling has been used in the service sector to gain an accurate picture of the whole customer journey, by exploring touchpoints sequence with customer-centric soft goals (Verma et al. 2012). Indeed, user journeys have been observed more frequently in qualitative and retrospective assessments.

The field of user experience (UX) research is defined by the International Organization for Standardization (ISO) as the set of user perceptions and responses resulting from the use or anticipation of the use of a system, a product, or a service (ISO 2018). User experience encompasses all consumer-firm touchpoints and interactions encountered within the user journey. Generally, to measure this experience, user testing has been mostly associated with online interactions. Indeed, UX professionals create tasks to replicate real interactions that are logically organized and monitor the user in this artificially rational process (Nielsen 2012). In the assessment of this experience, most UX research focuses on explicit and self-reported and retrospective measurement tools, such as questionnaires. However, prior research also shows that there is an important discrepancy between what users feel during the experience and how they recall it afterward (Cockburn et al. 2017; Eich and Schooler 2000).

In order to mirror the actual complexity of the omnichannel customer interactions, a quantitative analysis must be combined with qualitative, judgment driven evaluations (Rawson et al. 2013). As such, implicit psychological measures are favorable to overcome the potential self-reported biases (Léger et al. 2014). Indeed, cognition and affect are psychological constructs that have been proved to influence customer behavior and customer experience (Bagozzi et al. 1999; Frow and Payne 2007; Tynan and McKechnie 2009). Recent research also showed the effectiveness of implicit psychological pain points in insights during peak emotional responses in a user’s experience (Giroux-Huppé et al. 2019). Implicit pain points are defined here as a moment, in reaction to an event during the interaction, during which the user experiences an automatic physiological activation characterized by a high level of emotional arousal and negative emotional valence (Giroux-Huppé et al. 2019). But to use psychological measures and tools, ecological validity is necessary and easier to ensure in an online experience, where the user does not have to move or to manipulate an object. In omnichannel shopping, however, as more frequent business-consumer interactions with technology take place within a physical context, challenges arise in measuring the emotional and cognitive user experience.

In this article, we propose a methodology to collect and analyze insightful psychophysiological data in a non-linear physical interaction. We will use the case study of a telecommunication provider’s equipment set up using implicit psychophysiological measures and explicit qualitative assessment. The task consisted of completing the un-installation and installation of technological equipment for cable and television services, with the sole support of the instruction manual to replicate as close as possible the real-life context. We aim to measure the cognitive and emotional experience of a user in their interaction sequence from uninstalling existing equipment to installing a new technological experience, using novel types of equipment in the telecommunication industry.

The proposed methodology aims to overcome the different challenges raised by measuring user experience in physical interactions. Physiological measures, such as electrocardiography (ECG), respiration rate, skin-based measures (EDA), and psychological measures (EEG) allow a deeper understanding of the emotional and cognitive experience of a user without interfering with the interaction (Dufresne et al. 2010). Moreover, the use of non-intrusive psychophysiological measures provides the opportunity to assess multiple aspects of an experience that cannot be accurately reported by the users at a precise moment in time. Many of those measures are related to the user experience field, such as emotional valence, arousal, and cognitive load (de Guinea et al. 2009). However, the main disadvantage when using physiological tools is that their use requires great execution precision and minimum external noise, which are complicated to overcome in the case of physical interactions with technology.

3.1 Experimental Procedure

The objective of this case study is to demonstrate the feasibility of the proposed methodology. The context of this illustrative case is an auto-installation of a new entertainment platform using novel types of technological equipment. In order to acquire valid data, it is important to replicate the setting of real-life usage. In a laboratory setting, the participant completed the tasks in a room arranged to resemble as close as possible the living room environment of the installation. The laboratory setting allowed for a mirror room for researchers to observe and follow the experiment closely.

We anticipated an average of two and a half hours for each participant, including tool installation. It is important to be mindful of the length of the experiment to minimize the exhaustion of the participant. Indeed, participants’ fatigue caused by the context of an experiment can skew the data and lead to biased conclusions. Thus, it is crucial to pretest the protocol of the experiment to ensure the quality of the data recorded but also that the tools used, and the tasks’ steps order run fluently and consistently from one participant to another. For running experiences in physical contexts, we suggest pretesting the protocol with at least three participants, as improvements from the first one to the last will make this more complex experiment easier to run when collecting the real data. Then, once the protocol has proved to run smoothly, each experiment should be closely observed by researchers. Detailed notetaking on any predetermined events of interest or on participants’ actions should be recorded.

In the end, our final sample consisted of 24 participants (10 women, 14 men). All participants provided signed consent. Participants recruited from research panels each received money as compensation. This experiment was approved by the research committee of our institution.

3.2 Experimental Tasks

The study was divided into five sub-tasks to successfully complete the installation, as shown in the instruction booklet support given to participants. In general, determining sub-tasks allows for more control over the potential non-linearity factor of any physical interaction. Indeed, dividing the ideal path to success into smaller steps will help to maintain the same user journey language throughout participants, and also facilitates the elaboration of a user’s path to completion. Finally, sub-tasks can also be used as event markers during or after the test, to dissect the overall task into several pieces. In our case, participants performed all five sub-tasks with the support of an instruction booklet:

• Unboxing: This task is the first step of the experiment and the first contact the customer has with the new product. Participants needed to take the equipment out of the delivery box.

• Uninstalling: This task is crucial to the process flow for a successful installation. Since the equipment already in place can be unique to each user, participants needed to follow the instructions carefully in order to take out the right cables and equipment.

• Installation: This task includes the installation of the two different pieces of equipment (Gateway and Terminal) needed to assess the experience fully.

• Remote control configurations: To finalize the installation, the participant has to activate the smart remote control equipped with a voice command. A pairing of the remote control and the TV is the last step to fully complete the installation.

The proper order of those tasks was as stated above in the instruction booklet. However, the participants were free to complete the auto-installation as they would at home. To keep the results as close as possible to reality, only a general verbal explanation of the test was given to participants (Fig. 1).

The figure above provides an overview of the experimental procedure.

Full size image

Before the experiment, all events of interest identified with the help of the experimental procedure were coded into the behavioral and analysis software used, in our case we opted for Observer XT (Noldus, Wageningen, Netherlands). An event can be defined as an action or situation of interest expected to happen or that has happened during the interaction measured. Those events are the results of variations in constructs such as emotional valence, arousal, and cognitive load. The value of analyzing events resides in the interpretation of concrete observations of the emotional and cognitive experience within a certain time frame in the overall interaction. For example, in a relatively linear website interaction, one can predict, prior to the test, that in order to complete a purchase (the final objective), the user will have to create an account first. Hence, the account creation step will be considered as an event and can be interpreted as a moment of interest in the evaluation of the emotional and cognitive user experience. The early identification of those events—i.e., defining tasks or subtasks—lightens the post-processing of the data and facilitates the understanding of the context. Hence, those events should be aligned with the objectives of the research. It is highly possible that some of the defined events will require modifications or adjustments post-experience, but at least this pre-experience preparation gives a general guideline as to what should be closely observed and noted. We use those events to average values of psychophysiological data in between each of them. During the experiment, the execution of those events can be done by pressing on the existing marker pre-coded when the participant reaches this event. However, when the interaction measured is exposed to non-linear steps in the journey, a manual codification of those events of interest can be done posterior to avoid errors.

3.3 Instruments and Apparatus

A total of five cameras were placed around the experiment room to record movements from different angles. Those cameras not only facilitate the observation of the participant’s reactions at every step of the experience but also allow us to review all the actions that could be reported in synchronization with the other tools. Indeed, recordings grant a posteriori identification of the specific moments in time coupled with varying levels of arousal and cognitive load. Hence, the cameras ensure a safe recall option of all the micro-actions performed by the participants to complete the notes taken via an observation grid during the experiment. If needed, this gives researchers a more accurate view of the participant’s actions even during the analysis phase.

While determining cameras’ placement, it is crucial to choose adequately room locations where a sync markers’ light would be visible to the researcher, to ensure the cameras are working properly throughout the experience. Indeed, the use of a sync light issued by all five cameras secures that all cameras start recording seamlessly and that they function in sync during the whole experiment. In fact, the cameras should be synchronized using a sync box to facilitate the analysis. We followed Léger et al. (2014) guidelines for synchronization. The automatic data stream will allow additional precision when placing markers in Noldus Observer XT (Wageningen, Netherlands) during the analysis phase.

Finally, the cameras were placed in strategic places to cover all the room’s angles to the maximum extent (see Fig. 2):

• One camera on the ceiling over the unboxing table (initial place of the box)

• One camera on the ceiling, on the top of the coaxial wire (to see if fixated correctly)

• One camera on the wall to see the side of the television (to see the connection—the disconnection of wires)

• One camera below the TV furniture (to see from the TV the participants during the remote-control configuration)

• One camera on the wall in front of the TV (to see what was displayed).

One view of the laboratory room where cameras’ placement is highlighted.

Full size image

Depending on the objects of interest to observe, the Go-Pro camera can simply be strapped around the participant’s chest, since the EEG helmet prevents the support of any other devices on the participant’s head. Indeed, the use of a Go-Pro camera (San Mateo, United States) is optional but allows for more precise visuals on participants’ manipulations. The Go-Pro camera (San Mateo, United States) can act as a zoom on an event of interest that can be too subtle for long-range camera recording.

4.1 State Factors

This illustrative case focuses on three physiological state factors: emotional arousal, valence, and cognitive load. Users’ emotions are considered an important factor in their experience since the emotional evaluation of that experience allows them to compare possibilities (Russell 2003) and substantiate future behavior (Hassenzahl 2013).

Valence.

Emotional valence can be defined as “the value associated with a stimulus as expressed on a continuum from pleasant to unpleasant or from attractive to aversive” according to the APA Dictionary of Psychology (Online APA Dictionary of Psychology, 2020). Emotions are of interest since they are an omnipresent part of consumers’ decision-making and behavior (Schiffman et al. 2010). Measuring the emotional experience of a user allows for the identification of unwanted negative emotional states.

Arousal.

Arousal refers to the user’s emotional state indicating physiological activity (Deng and Poole 2010; Russell 2003). A measure of a user’s arousal will allow to nuance its affective state, since different level combinations of valence and arousal lead to diverse states, for example, a positive valence with high arousal (happy) will be different than a positive valence with low arousal (pleasant).

Cognitive Load.

Cognitive load refers to the mental effort required to carry out certain tasks (Fredricks et al. 2004). In this experiment, participants are to achieve goals, such as the installation. Fredericks et al. (2004) showed that cognitive and metacognitive strategies are crucial to achieving the aforementioned goals. Namely, metacognitive strategies refer to the setting and planning of goals when performing a task. Hence, cognitive load can influence how the user will employ the best strategies to install new electronic devices.

4.2 Attitudinal Factors

Attitudinal factors can be examined in relation to variations in emotional experiences. For example, Maunier et al. (2018) explored the level effect of valence, arousal, and cognitive load on the impact of success rates during a task. Attitudinal factors can be used as discriminant elements that can explain variation in a user’s emotional journey. This illustrative case focuses on two attitudinal factors that could be of interest in measuring the emotional and cognitive experience of a user: self-efficacy and task success.

Self-efficacy.

Perceived self-efficacy is concerned with people’s beliefs in their capabilities to produce given outcomes (Bandura 1997). Indeed, self-efficacy plays a key role in the likeliness of task accomplishment. Previous research suggests that individuals with a high level of self-efficacy think they possess the capacity to succeed in specific tasks (Walker et al. 2006). Consequently, individuals who believe they are self-efficient at completing a task will in fact generate the necessary efforts to succeed (Bandura 1993).

By using a measurement scale to assess self-efficacy, the performance results of the study can be separated into groups based on the high and low levels to examine significant differences between groups, if any. Given the usefulness of a theoretical construct, it is relevant to find previously established constructs and measurement scales, appropriate for the research objectives. Indeed, using additional self-reported behavioral measures will nuance the physiological data collected. Especially within studies involving a spatial context, attitudinal measures can help understanding and explaining the sequence of actions undertaken by the participant. Moreover, researchers suggest that combining complementary methods of assessment offers a deeper understanding of user experience, while adding implicit measurement, such as physiological tools, allows for a more rigorous measure of the participant’s emotional journey (Bigras et al. 2018). Self-reporting measures of emotions, like the Self-Assessment Manikin (Bradley and Lang 1994), can also be administered to be able to see the personal differences across participants.

Success Variable.

To evaluate the success of each user’s path, which could be different from one another, we developed thresholds of success for each identified task. Installation task success was measured using sub-tasks success thresholds (such as the completion of the task, the appropriate installation of the wire connections, the firmness of the cable fastening, etc.). Thus, overall success was achieved when the participant completed the installation with few or no mistakes.

5.1 Psychophysiological Measures

In order to accurately measure users’ reactions and behaviors, it is crucial to choose measurement tools that are adequate for a moving subject. Aligned with the objectives of the experiment, psychophysiological measures were used to capture participants’ emotional valence, arousal, and cognitive load (Riedl and Léger 2016).

A precise measure of arousal is needed to assess a quantitative activation level of emotion, from not aroused to excited. For this study, arousal was measured using the electrodermal activity with the Acqknowledge software (Biopac, Goleta, USA) as pictured in the figure below. Sensors (BIOPAC, Goleta, USA) were applied in the palm of the non-dominant hand of participants to measure skin conductance during the experience (Fig. 3).

A measure of electro-dermal activity using the Acqknowledge software (Biopac, Goleta, USA), with sensors in the palm of the hand.

Full size image

Valence is typically measured through facial expressions (Ekman 1993), as micro-movement on the user’s face can be detected using a video webcam. However, the constant movement of participants does not allow for precise facial detection, hence valence and cognitive load needed to be measured with tools that can easily move with the participant. As such, a wireless EEG cap (Brainvision, Morrisville, NC) was used, which allows to detect “inner” emotions and cognitive load. The mobile EEG cap contains 32 Ag-AgCl electrodes and an amplifier (Brainvision, Morrisville, NC), and was used to measure variations in brainwave activity in the θ (4–8 Hz), α (8–12 Hz) and β (12–30 Hz) bands, isolated by a bank of filters. Previous research has also used EEG signals to detect emotions (Maunier et al. 2018; Chanel et al. 2007; Brown et al. 2011) and to collect cognitive load data (Teplan 2002; Anderson et al. 2011; Park et al. 2014; Maunier et al. 2018).

5.2 Psychophysiological Pain Points

Among the different models of emotion classification, Russel (1979) proposed the use of a two-dimensional Arousal-Valence model, where pleasure-displeasure and level of arousal are sufficient to represent a wide range of emotional states (Russel 1979). As such, the implicit pain point identification method used in this article allows arousal, cognitive load, and valence data to be triangulated into specific points in time that we can identify as implicit psychophysiological pain points (PPPs) (see Fig. 4). An implicit psychophysiological pain point can be defined as a precise moment in time, when the user both feels a high level of emotional arousal and negative emotional valence, compared with his baseline state (Giroux-Huppé et al. 2019). However, PPPs are easier to diagnose in an online interaction context since recordings of the interaction are clearer to interpret via a 2D screen recording. In fact, identification of precise points in a user’s journey using mouse movement is more straightforward than with physical actions performed by moving subjects.

Consequently, the objective of PPPs in this experiment would relate to portray users’ emotional frictions throughout the tasks of the physical experience to avoid relying solely on emotional memory recall (Cockburn et al. 2017; Eich and Schooler 2000), as shown in Fig. 4. The main actionable insight of PPPs analysis is the possibility to rectify and optimize the experience to promote users’ autonomy during any interaction. Frustrations along the user journey can serve as optimization’s starting points in the physical interaction stage with a new technology, where the quality of the first experience can be a crucial determinant of future usage and adoption. Indeed, an unpleasant first experience can have negative consequences for the user’s experience and perceptions of the brand (Brakus et al. 2009). Therefore, PPPs can be used as a preventive tool to ensure satisfaction and future usage.

A detailed schema shows how to mobilize physiological data into pain points.

Full size image

5.3 Psychometric Measures

Perceived self-efficacy was measured via questionnaire before and after the experiment, to evaluate any discrepancies that can be linked to the effects of the tasks performed. A 6-item measure was used to assess the perceived self-efficacy construct (Sherer et al. 1982).

Once the data collection is completed, the extraction of the data is necessary to proceed to the analysis phase. Depending on the software used to collect the data, this step will be different; see the software provider’s specific instructions to ensure optimal extraction. Once all the recordings and the psychophysiological data are exported into separate files under the participants’ number (e.g., P01), the first step consists of viewing the video recordings in a behavioral coding and analysis software (e.g., Noldus Observer XT) in order to place the proper markers, which correspond to the event of interests previously established. To avoid multiple rounds of review, it is essential to examine the notes taken during each experiment to see if any new events should be added to the existing list entered before the beginning of the experiment. If this is the case, new event markers should be added to Noldus Observer before the viewing process.

Then, once all the markers are in place in each participant’s recording file, it is easier to code each of them with the different views offered by the five angles of the cameras. The markers always need to have a “start” and an “end,” in order for the analysis to be performed accurately. Precision is key in this process since coding errors in the timeline could lead to false results and interpretation. It is also crucial that every marker is properly coded into each participant’s file (if the case may be) before extracting the data for further analysis. Indeed, all events of interest, including the tasks, coded into the software will allow for an interpretation of the physiological data in time. The next step in the analysis process is to transform the physiological data to allow their use in the interpretation of the participants’ emotional and cognitive experiences.

6.1 EEG-Based Valence and Cognitive Load

In order to extract the valence and cognitive load of the raw data files, cleaning of the files is needed. We used the NeuroRT software (Mensia, Rennes) to analyze the EEG data. The acquisition rate was 500 Hz. The following steps were performed: Decrease the acquisition rate up to 256 Hz, filters 1–50 Hz, cleaning the ocular artifacts with the help of source separation, re-referencing to the mean reference, and artifact detection by calculating the Riemannian distance between the covariance matrix and the real-time mean. Then, we apply a MATLAB transformation to clean the data using ASR. It is important to verify after the cleaning that no critical channel was lost. After a cleanse of the raw files, we export the clean data into NeuroRT Studio to pass it into a pre-existing general EEG pipeline. Cognitive load was calculated as a ratio using (power) β/(α + θ) from F3, F4, O1, O2 on the international 10–20 system, following procedure from Pope (1995). Valence, collected by EEG technology, will be detected as follows: “Valence: positive, happy emotions result in a higher frontal coherence in alpha, and higher right parietal beta power, compared to negative emotion” (Bos 2006). Once the process stops, a “.csv” file should be ready to import into the next step, a triangulation software.

6.2 Identifying Pain Points

Once each physiological data file is properly transformed and the recordings are properly set with the relevant markers to speed up the visualization and interpretation steps, we have to prepare the necessary data files to allow for a triangulation of the separated data files into a valuable and understandable output. We previously developed a methodology to be able to easily find the implicit frustrations in a user journey (Giroux-Huppé et al. 2019). We will use the same method here to be able to identify more accurately the difficulties faced by users in physical interaction.

To facilitate PPPs triangulation and identification, CubeHX software was used (Courtemanche et al. 2019; Patent US10,368,741 B2 2019), which is cloud-based lab management and analytics software for triangulated human-centered research (Léger et al. 2019). This software used to triangulate all the data accumulated during the experiment generates outputs of UX attentional, emotional, and cognitive heat maps, and can also export to statistical packages from one or multiple projects (e.g., cross-project analyses, compatibility with third-party visualization software, i.e. Tableau software) (Léger et al. 2019).

Concretely, calculations of pain points are performed using a specific threshold, built on previous research, with the statistical software SAS 9.4. In this context, to be qualified as a pain point, the data point needed to be both in the ninetieth percentiles of EDA (i.e., high arousal) and in the tenth percentile of valence (i.e., large negative valence) (Giroux-Huppé et al. 2019). Once a list of all the PPPs experienced throughout the experiment’s relevant time frame is generated, the researcher has to manually identify and interpret the users’ actions during the window of time when the pain point occurs. To ensure a precise interpretation, unique micro-moments pain points can be regrouped into pain points moments if they are consecutive seconds apart. Using the same software for video recordings’ viewing (Noldus Observer XT), one must put himself in the shoes of the participant by reviewing, using the different angles of the multiple cameras, the moments when each of friction points are experienced (±10 s before and after the precise time listed to gain context). Having the same researcher interpreting all the pain points moments will limit labeling errors. To compensate for the researcher’s potential bias, another researcher can then perform the same exercise separately and a combination of the two labeling lists can allow for more reliable classification. Whenever there is a discrepancy of interpretation, a third researcher should do the same exercise, as to serve as a decisive interpretation. Once the micro-moment regrouped into pain points moments and interpreted, they can be classified and arranged into similar categories across participants, as to improve the actionable insights from a high-level overview instead of per individual participant. Precise interpretation is central to benefit from decoding the root of the frustrations, and the regrouped categories of pain points are an easy and understandable way to present the problems to the rest of the research team and to high-level management.

7.1 Visualization of Users’ Emotional Journey

Once all the psychophysiological data is treated, we may be able to build user experience journey maps and identify the different pain points in a more visual way. In addition to the interpretation of each moment of frustration, another mobilization of users’ intense frustration is the visualization of PPPs across the tasks in the form of journey maps (see Fig. 5). Timeline visualization of PPPs allows the researchers to gain a deeper understanding of the participants’ emotional journey. Using Tableau software (California, USA), individuals’ journey maps coded with the sub-tasks simplify the recognition of the most problematic tasks or subtasks but also the most critical implicit pain points in time (see figure below).

Visualization of the psychophysiological user journey by task with clear pain points moments identification (red dots). (Color figure online)

Full size image

As Fig. 6 below shows, the colored circles on the following cards illustrate all the different psychophysiological points in time through tasks undertaken by participants. Second, the user’s moments of frustration are illustrated by the red colored points. The size of the circles represents, from the smallest, a lower intensity PPP to the largest, most intense PPP. Finally, the scale on the y-axis represents the user’s activation and cognitive effort from the lowest to the highest. The x-axis represents the timeline of each task performed per participant. The most intense or most frequent moments of frustration can then be visually prompted by a legend similar to Fig. 6. This way, one quick look at the journey map can reveal the unique emotional journey of each participant, as well as how the moments of frustration unfold through the task completion.

Visualization of an individual psychophysiological user journey by task with pain points interpretation. (Color figure online)

Full size image

This visualization of an emotional journey has multiple advantages granted by its comparison elements. First, it is easy to compare users’ journeys with each other for the same task (see Fig. 7), thus being able to tell a real story for each identified user. Common difficulties can be identified and classified with users’ same frustrations, therefore emphasizing the need for a change in the experience process. Second, users’ journeys can be compared with competitors,’ in order to benchmark the emotional experience across different approaches or paths of a similar product or service. Within the same test, participants can be asked to perform a similar task with two different products or websites, then the intra-subject data can be used to assess opportunities originated by the comparison of a competitor’s experience. Third, the users’ journeys can also grant an overview of the sub-tasks’ effects on a user’s emotions. Certain sub-tasks can create more accumulation of frustrations than others, hinting at the source of problematic instances instead of re-evaluating the whole journey process. Finally, the visualization of the users’ physical journeys can serve as a common tool to evaluate user experience across all touchpoints. Ultimately, the emotional and behavioral journey of users can be compared equally across each channel, no matter their nature.

An example of the complete visualization of the psychophysiological user journey for different participants.

Full size image

7.2 Complementarity of Quantitative and Qualitative Data

The use of physiological data and retrospective measures allow a more complete understanding of the users’ experience with the technology. Self-reported measures are an interesting addition to physiological measures because they allow us to see if there are any differences between groups of different attributes. For this study, self-efficacy was used in the elaboration of two different groups, low and high self-efficacy, to test for any significant differences in overall success rates between the two groups. As portrayed in Table 1, no statistically significant difference was found between the high and low self-efficacy groups. However, results lean towards an overall higher success rate for the low self-efficacy group.

Furthermore, as self-efficacy was measured before and after the experiment, it can be useful to uncover any significant differences. As shown in Table 2, participants significantly felt more confident about their own capacities after they accomplished the task. Such findings could imply that the task has some empowerment implications and could encourage users to do it again.

Moreover, the success variable was used as an overall presentation of the journey flow undertaken by all the participants. This variable can support the identification of problems in the subtasks by comparing success rates across the journey. This refining of the overall task is useful for constructing a step-by-step success map. Compared with the emotional journey, similarities between the two journeys are raised to strengthen the insights uncovered.

All in all, each additional retrospective measure included in the study must have relevant insight potential, even if the hypothesis is not sustained after the analysis. The combination of psychophysiological, behavioral, and psychometric measures provides, beyond insights about task successes, a complete picture of emotional and cognitive influences and impacts on the user experience.

The simultaneous use of physical technology (e.g., mobile phones) and online resources (e.g., websites) in various day-to-day activities call for a closer exploration of each user interaction with a brand. As user experience is a consolidation of all consumers’ touchpoints throughout its journey, tangible interactions with technology are to be of interest. As the omnichannel approach suggests, a unified experience has to be maintained across all touchpoints (Verhoef et al. 2015). Despite the need for consistency across channels, there was still no established methodology, to our knowledge, to measure and compare user experience across both on—and offline interactions. Indeed, frequent switches between offline and online touchpoints result in higher complexity of user journeys. The combination of interactions can almost be unique to each consumer, with various exit points in their journey as they want to compare other options, retrieve more information, and then as they circle back to previous choices (Carroll and Guzmán 2013).

This article presented a methodology better suited to measure users’ experience in the omnichannel environment. Through psychophysiological variables and self-reported metrics, this article serves as a comprehensive methodological approach for experts to have a precise overview of the emotional journey of consumers. Overall, we succeeded in demonstrating the interest of this methodological approach through an illustrative case study. This methodology can help businesses evaluate with precision in-store experiences as well as face-to-face interactions. Consequently, our case study demonstrated that the measure of user experience has to include a fair assessment of emotional and behavioral influences, impacts, or consequences across all touchpoints, regardless of the online-offline element. Despite the cost of resources to run an experiment with moving subjects, the methodology presented shows the potential for faster and more accurate identification of dissatisfaction and intense frustrations, which is a tremendous competitive advantage for companies.

Although the usefulness of the proposed methodology is clear, it is still the first exploration into the preservation of ecological validation in a physical interaction study. A few recommendations are stated for replication intention or future research. First, we recommend pretesting the whole experiment until the protocol runs seamlessly and the quality of physiological data recorded is high. By doing so, we make sure the synchronization of the data is adequate, and the noise is reduced to the minimum. Since the methodology proposed requires the mobilization of EEG data, we suggest having individuals with cognitive science expertise or previous experience with EEG technology within the team. This will facilitate the execution of the tests but also ensure the quality of the data during the data collection and analysis. Also, it is important to consider that although it is a systematic approach, there is still a human factor embedded in the analysis process, in order to identify and interpret psychophysiological data. Hence, it is critical to be mindful of potential interpretation biases. We suggest relying on consistency throughout the analysis by ensuring that the same researcher performs all analysis interpretation first (event coding, pain points analysis), and then repeating the process with another researcher to compare interpretation results. In addition, it is recommended to keep the analysis period short in order to avoid bias or memory loss that can happen when there are long periods of inactivity.

As businesses tend to be competitive on all channels, modern consumer behavior urges companies to approach commerce in an omnichannel way. The user experience does not end when the purchase is completed on the website and is rather an ongoing process of interactions with the physical product or service, before and after. It is usually difficult to measure the interaction with ecological validity in a non-linear physical experience because of the noise created by the subject’s movements. This article proposed a complex yet simple methodology that allows for researchers to grasp the users’ experience using a combination of psychophysiological metrics and qualitative data, despite barriers such as movements, bias, and noise. The proposed methodology allows companies to optimize and cultivate a relationship with the user throughout the channels, identify opportunities within the interactions, and correct service failures even outside the Internet world. As usability testing prevents missteps that could be fatal to the user-business relationship, it is of equal importance to consider the physical interactions with the technology, pre-and post-purchase. Indeed, all interactions, no matter their nature, can have an impact on users’ future usage and satisfaction. This new approach consolidates user-centric data, as well as a multi-method evaluation of user experience to be applied in each step of a consumer journey, from their search of information to their unboxing at home. Not to mention that conducting valuable user testing, even in physical contexts, can assure a constant experience quality in an increasingly omnichannel world, without compromising the ecological validity of the experiments.