A Set of Usability and User eXperience Heuristics for Social Networks

We can say that today digital social networks are already part of a person’s daily routine. Everyone who owns an electronic device, whether it’s a computer or a smartphone, uses a social network to keep in touch with friends or relatives. The social media has achieved great versatility today as not even age is an impediment to use them. However, it is important to note that the use of social networks should not be confused with their correct use. When a user is using some medium to communicate with someone or keep in touch with people it does not imply that they are alienated from frustrations by how the information or content is presented to them on the website.

The User eXperience (UX) is responsible for evaluating the emotions and sensations generated in users by the use of a system, product or service [1]. These perceptions can be negative or positive because they can generate frustration when the way the information is delivered confuses users; or a certain sense of success and achievement when what is shown by the device is as expected.

Our research aimed to define a set of heuristics to evaluate UX in social networks. The set of heuristics was validated through expert’s judgement and experiments. Our study focused on two of the most popular social networks in Chile in recent months, based on the intelligent tool SimilarWeb [2]: Twitter and Facebook.

The paper is organized as follows. Section 2 presents some theoretical background that is important for understanding concepts approached by the research. Section 3 shows the set of usability/UX heuristics for Social Networks, documenting their development process. Section 4 highlights conclusions and future work.

2.1 User eXperience (UX)

Directly or indirectly every website causes sensations, and these can be positive or negative. The first impressions in the human are important, since in a short period of time we decide if something pleases us or not [3]. Therefore, the first time a user accesses an application or website is paramount. This generates in us a judgment of whether we will give that website a chance to perform the same action in the future, or we will choose to find other alternatives that could be more pleasant and comfortable for the same purpose.

There are many accepted definitions of what is known as UX, according to the ISO 9241-11:2018 standard is “user’s perceptions and responses that result from the use and/or anticipated use of a system, product or service” [1]. Several authors propose models of UX. Among others, Morville explains each of the attributes of the UX through his ‘honeycomb’ model [4]:

• Usable: The site should be easy to use, without users having to think too much about how an activity or action is performed.

• Useful: Facilitate the user to reach his goals and satisfy some need.

• Desirable: It must create content in order to evoke emotion and appreciation from users to achieve their desire to interact with the product.

• Findable: It must be impossible for users not to find your content.

• Accessible: The content must be designed for everyone, including people with specific needs.

• Credible: The user must have confidence in the product being offered and therefore in the person offering it.

• Valuable: The product, system or service must be valuable.

2.2 Usability

According to the ISO9241-11:2018 [1], this concept is understood as “extent to which a system, product or service can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use”, presenting three main features:

• Effectiveness: accuracy and completeness with which users achieve specified goals.

• Efficiency: resources used in relation to the results achieved.

• Satisfaction: extent to which the user’s physical, cognitive and emotional responses that result from the use of a system, product or service meet the user’s needs and expectations.

2.3 Social Networks

Social networks have been developing for decades. Currently, computer systems have served as a tool to complement and promote this term that has always existed. Today we can learn more about the social networks that surround us, and these can constantly change in order to satisfy the needs of its users. These changes may or may not go hand in hand with the experience they manage to deliver to users. There are social networks that give a lot of emphasis to the emotions that their users could experience with the use of their platforms, as well as there are others that prefer to give preference to their usability.

Identifying the beginnings of social networks is complex, since they were not born in a digital way, but this medium complemented them later. Similarly, we can say that a social network is what we know as a group of friends, a family, or co-workers. They have always existed but not with the help of computer systems. Highlighting the words of Ponce [5], who indicates that a social network is a social structure formed by people or entities connected and joined together by some kind of relationship or common interest.

There are many social networks throughout the world, in some countries some are used more than others and the popularity of some comes at different times. But one thing that all social networks should have in common is the UX they apply at the time of their concession. A social network is the means by which people communicate, inform, keep in touch and express through the Internet. For this reason, they require a platform that supports them in the most comfortable, pleasant and safe way possible with a correct organization of the delivery of their information and its functionalities, among other factors.

Therefore, social networks are sites that are accessed through the Internet, which allow users to connect to each other virtually, share experiences and content, and even create new ties or friendships, but it should always be borne in mind that users who interact through these media also share their personal information with the confidence that it will not be disclosed or exposed to others, so achieving credibility is a primary factor for the responsible for the system.

As a result of the observation of different social networks, it was possible to identify six characteristics that any system of this type, current or future, should satisfy:

• Security: users must be able to trust the system, refers to everything related to login and logout, registration of a new account and or close any user account.

• Connectivity: users can manage their contacts within the social network, refers to everything related to follow, add, delete, block and report users.

• Interaction: users can interact with the system and the rest of the users of it, refers to everything related to the activity between contacts within the network, either send and delete messages, create instant messaging groups, and share, view, evaluate, comment and report content.

• Customization: users can adjust various settings of the social network, refers to the customization of profile and configure notifications that are received, the permissions granted and language, among others.

• Management of content: users add, create and delete content within their space in the network, regardless of whether other users of the network will interact with it or not.

• Help Center: users will be given a help service to resolve frequent or concerns related to the system.

2.4 Heuristic Evaluation

According to the website Allaboutux, there are currently 86 known methods for evaluating UX [6], including inspections, heuristic evaluations and experiments. This research relied on the heuristic evaluation method to validate the set proposed.

A heuristic evaluation is a type of usability inspection in which a small number of evaluators look for problems within an individual system, and then compile the results along with the rest of the evaluators (ideally). The recommended number of evaluators ranges from three to five and the experience of each will influence the number of problems encountered.

The evaluators are provided with a set of heuristics to evaluate the usability/UX of a website; it is important to emphasize what [7] mentions about the existence of generic heuristics and specific heuristics. General heuristics are applicable to a wide range of applications, usually detecting general problems; they are known by most evaluators. Specific heuristics detect both general and specific problems, but are domain-oriented, applicable only to specific applications.

Heuristic evaluation as probably the best-known evaluation; usually applied when evaluating usability, but Quiñones et al. [7] suggest it could be used to evaluate UX attributes others than usability.

In this research we used the methodology to develop usability/UX heuristics for specific domains proposed by Quiñones et al. [7], its 8 stages are explained below:

• Exploratory stage: performing a literature review.

• Experimental stage: analyzing data obtained through experiments.

• Descriptive stage: selecting and prioritizing the most important topics of all information that was collected in the previous stages.

• Correlational stage: matching the features of the specific application domain with the usability/UX attributes and existing heuristics (and/or other relevant elements).

• Selection stage: keeping, adapting and/or discarding the existing sets of usability/UX heuristics that were selected during the descriptive stage.

• Specification stage: formally specifying the new set of usability/UX heuristics.

• Validation stage: validating the set of heuristics.

• Refinement stage: refining and improving the new set of heuristics based on the feedback that was obtained in the validation stage.

3.1 How the Set of Heuristics Was Developed

Our research included three iterations; none of these were performed the 8 stages of the methodology. Figure 1 shows how we worked in each iteration.

Iterative process used in the research.

Full size image

With this screenshot to the social network ‘LinkedIn’, we can see an example of heuristic compliance.

Full size image

With this screenshot to the social network ‘Facebook’, we can observe another example of heuristic compliance.

Full size image

In the first iteration, all stages of the methodology were performed except the experimental stage; In the second iteration, only the last three stages of the methodology were completed, since details of the specifications had to be improved and the feedback was not critical with respect to the set proposed. Finally, some initial definitions were corrected, and most of the initial stages of the methodology were revisited.

Evolution of Heuristics.

Thanks to the first stage of the methodology, which included a review of previous literature in order to find any research related articles that might be useful for creating or adapting a set of heuristics focused on UX in social networks. There were three investigations directly related to the domain of the problem. As the main base of this work was the set of heuristics proposed by Arancibia and Gonzalez [8] listed below (Table 1):

Although their work delivers a set of sixteen heuristics oriented to usability/UX in social networks and after analyzing each of them, we concluded that there are certain characteristics of social networks that might not be addressed by his work, in addition to not involving mobile platforms.

Document Changes.

The proposed set went through a series of modifications along its conception, which are documented below. In the first instance, the base set was adapted to work on this research; creating, modifying and eliminating specific heuristics from the Arancibia and Gonzalez set [8].

As shown in Table 2, only ten heuristics were used to develop the new set. The justification for the action that was carried out with each heuristic is presented below.

• HRS1 - It is important that the user always has a notion of what happens on the site or application given their actions.

• HRS2 - It is not considered relevant to the wide range of social networks available on the market today.

• HRS3 - It is relevant to satisfy the features: customization, connectivity and interaction.

• HRS4 - It is considered that each social network has its own symbology with which it tries to set trends. Therefore, coincidence with the real world is not considered relevant.

• HRS5 - It is not understood what is sought to evaluate with this heuristic, if the reference were between its mobile versions and its web versions would make more sense.

• HRS6 - It is considered that each social network has its own symbology with which it tries to set trends. Therefore, coincidence with the real world is not considered relevant.

• HRS7 - It is relevant to satisfy the features: management of content and interaction.

• HRS8 - It is relevant for satisfy the customization’s feature.

• HRS9 - It is relevant to satisfy the features: management of contents, customization and interaction.

• HRS10 - The purpose of heuristics is not clear enough, and its purpose is not understood.

• HRS11 - The design of a social network will vary depending on its purpose, therefore, it is not considered necessary to evaluate this topic.

• HRS12 - It is relevant to satisfy the features: help center and customization.

• HRS13 - Relevant for satisfy the help center’s feature.

• HRS14 - It is relevant to satisfy the features: management of content, customization and interaction.

• HRS15 - It is important to satisfy the feature “management of content”.

• HRS16 - It is important to satisfy the safety feature.

  Full size table

After this preliminary version of the set (Table 3), a second version of the set was obtained (Table 4); which aimed to complete the template proposed by the methodology of Quiñones et al. [7] for the specification stage along with all its attributes.

Finally, and after applying inspections to the set, it was considered that the heuristics of the set still did not have a definition that included the UX as expected from the research. So, in the third iteration we modify all definitions and explanations of heuristics to give them a focus on UX, and not just usability.

3.2 A Set of Usability and User eXperience Heuristics for Social Networks (SNUXH)

• (SNUXH1) Visual feedback and system status: The social network must inform the user of the status of the system after any action taken by the him or her.

• (SNUXH2) User control and freedom: The social network should allow the user undo and redo actions; user should always feel in control.

• (SNUXH3) Consistency and standards in multiplatform: There should be no visual or functional differences between the various platforms delivered by the same social network, to the extent that user interaction is influenced.

• (SNUXH4) Prevention and recovery from errors: The social network must prevent and avoid errors in use of the system through warning messages that deliver the right information, without too much technicality that may confuse the user.

• (SNUXH5) Minimize user memory load: User should not have to remember information that he/she already provided.

• (SNUXH6) Aesthetic and minimalist design: The social network must show an aesthetic design that includes only the elements relevant to the user in a certain context of use.

• (SNUXH7) Help center: The social network must have a space where users can resolve their doubts about the system; the help information must be brief, accurate and user-centered.

• (SNUXH8) User perception and status: The system must allow user to configure, at any time, whether he/she is available (or not) to communicate; user must easily perceive other users’ availability.

• (SNUXH9) Control of published content: The social network must control the content that publishes so as not to affect the sensitivity of users, through filters and regulations; the user must be able to denounce/report content published by other users on the network, indicating the reason (Table 5).

• (SNUXH10) Customization and configuration settings: The user must be able to adjust the different settings provided by the social network and customize the space it provides.

• (SNUXH11) Security and user account recovery: The social network must include security measures, protection of the user’s account and personal data; it must also provide an account recovery option.

  Full size table

3.3 Preliminary Validation

The set of heuristics was preliminary validated using three evaluation methods: expert judgement, heuristic evaluation and user test.

The expert judgement involved eight experts, all of them had participated in at least one heuristic evaluation. The method consisted of subjecting the experts to a survey that aimed to obtain feedback from them. The heuristics were evaluated in three dimensions (clarity, usefulness and ease of use). The results were mostly favorable, reflecting minimal modifications to the set.

The usability tests aimed to evaluate the set through the interactions that real users had using the system. Although social networks are a product that is used alone, this time it was decided to perform the co-discovery method as a test, in which users express aloud their opinion, feelings and thoughts while surfing the site. Finally, the experiment demonstrated that important heuristics were left aside within the set, and confirmed the usefulness of certain heuristics.

The last method was a heuristic evaluation, which was previously explained. There were six evaluators with similar experience, three people qualified and looked for problems based on the control set [8], while the other three used the SNUXH set. Each collaborator evaluated the study case “Twitter” in its web and mobile versions individually. In this opportunity the results were quite even, the evaluators of the sets found a similar number of problems. In addition, it was considered, based on the experience of the evaluators, that there could have been an overestimation on the part of the control group. Finally, this last validation reflects that the proposed set does not exceed the control set in all aspects evaluated.

Social networks have always existed and will continue to exist even when computer systems become extinct. Because people are what make them up. Although with the simple use of the Internet these have broken down borders that people could not cross so freely before, it should also be kept in mind that this tool is only that, and it depends on the users themselves the use they are given.

This research sought to provide a set of heuristics to evaluate the UX when using this type of systems, attempting to present a set that encompasses other aspects beside usability. After having established the attributes of usability and UX relevant to this research, accessibility was discarded, since at this moment there is not enough time to achieve a complete and correct validation of this aspect that freezes more than a simple heuristic can deliver. Finally, after applying the methodology for the development of domain-specific heuristics, it was possible to establish eleven heuristics. A preliminary validation of the set of heuristics was made.

As future work we expect to further validate the set of heuristics and check how useful it is for the evaluation of UX in social networks. We also pretend to refine the set, if necessary.