survey

Open access

Computational Resource Allocation in Fog Computing: A Comprehensive Survey

Authors:

Joao Bachiega, Jr.,

Breno Costa,

Leonardo R. Carvalho,

Michel J. F. Rosa,

Aleteia AraujoAuthors Info & Claims

ACM Computing Surveys, Volume 55, Issue 14s

Article No.: 336, Pages 1 - 31

https://doi.org/10.1145/3586181

Published: 17 July 2023 Publication History

PDF eReader

Abstract

Fog computing is a paradigm that allows the provisioning of computational resources and services at the edge of the network, closer to the end devices and users, complementing cloud computing. The heterogeneity and large number of devices are challenges to obtaining optimized resource allocation in this environment. Over time, some surveys have been presented on resource management in fog computing. However, they now lack a broader and deeper view about this subject, considering the recent publications. This article presents a systematic literature review with a focus on resource allocation for fog computing, and in a more comprehensive way than the existing works. The survey is based on 108 selected publications from 2012 to 2022. The analysis has exposed their main techniques, metrics used, evaluation tools, virtualization methods, architecture, and domains where the proposed solutions were applied. The results show an updated and comprehensive view about resource allocation in fog computing. The main challenges and open research questions are discussed, and a new fog computing resource management cycle is proposed.

1 Introduction

Fog computing has emerged as a promising solution to meet the growing demand for expansion of the processing, network, and storage capacity closer to end users, thus complementing the fragility of cloud computing [33]. However, as this is an emerging paradigm, there are several open research questions, with many challenges to be overcome [178]. Among these challenges is computational resource allocation, which aims to provide, in an appropriate manner, the computational resources for the service or application, so it can achieve the defined performance and Quality of Service (QoS) metrics [94]. Therefore, an analysis of the current state-of-the-art is relevant to assist both academia and industry in progressing work on this topic.

In a fog computing environment, the use of dynamic contexts such as the Internet of Things (IoT) is predominant, and adding the competition for allocated computational resources, this can lead the environment to have unpredictable events, such as service unavailability, high response time, and decreased reliability [46]. There are barriers to the use of resource allocation concepts applied in other computational paradigms, such as cloud computing. This requires the development of new proposals. Understanding the proposals already presented and the challenges that are still to be overcome is fundamental at this time.

Resource allocation is a step in the resource management process, which comprises several other steps in addition to allocation such as estimating, discovering and monitoring, for example. Over the years, some research has been published on resource management for fog computing [65, 75, 81]. However, as they take a more general approach, these surveys are not so specific in any of the stages, presenting a relevant but superficial conceptualization of each theme without a consensus on what are the steps that make up resource management. As fog computing has been evolving, a more specific work on each of the resource management steps, among them the allocation of resources, is already possible and is needed.

This work presents a systematic and comprehensive review of the literature on the computational resource allocation for fog computing from 2012 to 2022. To achieve this goal, six research questions were listed to guide the analysis and present the relevant results on this topic. In addition, once the important aspects of resource allocation are highlighted, the challenges of this approach are presented. Thus, the main contributions of this work are:

•

Analyze and compare the most relevant surveys on resource management for fog computing;

•

Define and delimit the scope and steps of resource management for fog computing;

•

Present the state-of-the-art regarding resource allocation for fog computing, since this process proved to be the most relevant in the evaluated articles;

•

Present the main metrics and techniques for resource allocation in a fog computing environment;

•

Discuss some of the relevant challenges to be overcome in the context of resource allocation in fog computing.

This work is organized as follows. Section 2 presents the main characteristics of fog computing, comparing it with some other similar distributed paradigms. In Section 3 the surveys on resource management are analyzed and the delimitation of the scope of this term for the development of this work is proposed. The methodology used for the elaboration of the systematic review, as well as the research questions that guided the development of this work, are presented in Section 4. The analysis of the selected papers is presented in Section 5. Related works on resource allocation are presented and compared with this work in Section 6. The challenges inherent to the theme are discussed in Section 7. Finally, Section 8 contains the conclusion and opportunities for future work.

2 Fog Computing

Fog computing was introduced in 2012 [33] with the aim of providing computing, storage, and network services between end-devices and cloud providers, complementing the resources when requirements cannot to be met with the traditional cloud services. As in nature where fog is a cloud closer to the ground, fog computing is intended to complement cloud computing, bringing computational improvements closer to end-users.

In the last years, the fog computing concept has been improved both by academia [46, 137, 181, 191] and industry [84, 145]. However, until now there is still no consensus about its definition, as well as some requirements related to it, such as scope, devices, architecture, service models, and so on. In our work [20], we state that fog computing is a distributed architecture that uses the computational resources of devices located between end-users and the cloud to optimize the processing and to reduce applications’ response times, meeting demands that until now were not possible. Besides fog computing, there are some other similar distributed paradigms, such as:

•

Edge computing: composed of devices in a fixed location that produce and consume data, participating in the processing, focusing on the end-device level, while fog computing focuses on the infrastructure level [46];

•

Multi-access Edge Computing: integrated with Radio Access Network (RAN) it is an implementation of edge computing to bring computational and storage capacities to the edge of the network [50];

•

Mobile Cloud Computing: a combination of cloud computing, mobile computing, and wireless communication where data storage and data processing occur outside of the mobile device [49];

•

Mobile Ad hoc Cloud Computing: a pool of devices that are closer to the user deployed over dynamic network addressing situations in Mobile Cloud Computing for which connectivity to cloud environments is not possible [22];

•

Mist Computing: defined as a lightweight form of fog computing, located on the edge of the network, using microcomputers and micro-controllers [153];

•

Dew Computing: the on-premises computational resources provide functionality that works regardless of internet connection, fully realizing the potential of on-premises computational resources and cloud services [154].

A comparative analysis between fog computing and these other related paradigms is presented in our work [20]. Therefore, the fog computing concept is broader and complete and can be considered as an umbrella which encompasses all these other similar paradigms [41]. Fog computing can base its communication infrastructure on Software-Defined Network (SDN) [134], on Radio Access Networks (RAN) [46], on Fog Radio Access Networks (F-RAN) [147], or on a composition of these technologies [137].

The most common architecture used to represent a fog computing environment is composed of three layers, namely IoT Layer, Fog Layer, and Cloud Layer, as presented in Figure 1. However, some authors presented hierarchical architectures with four [176], five [136], or even six [60] layers, varying only in the structuring of the Fog Layer and keeping the IoT Layer and the Cloud Layer. A comprehensive review of fog computing architectures can be found at [75]. Just as in our last work [44] but unlike other proposals that used hierarchical architectures with hard borders to delimit each layer, this work considers it is not possible to define where exactly the Fog Layer is delimited, since fog nodes can be found near the edge (e.g., IoT devices), and also near cloud computing, as in telecom devices, as demonstrated in Figure 1.

Fig. 1.

The IoT Layer represents the IoT devices connected at the edge of the network where the end-users can request services to be processed in the upper layers. The Fog Layer is positioned between the IoT devices and Cloud Computing, and provides the functionalities for processing applications, such as filtering and aggregation, before transferring the data to the cloud [11]. This layer is composed of nodes, commonly called Fog Nodes, i.e., any hardware device that has software and hardware resources with high communication capability. A comprehensive analysis of the computational perspective of a fog node can be found in the work [19].

Finally, the Cloud Layer is composed of cloud providers’ services, with more robust computational resources to process all requests made by the IoT Layer and which have not been fully met by the Fog Layer. The existence of the cloud is fundamental in a fog computing environment [11], since fog only complements and does not replaces cloud computing.

The most common characteristics to consider in fog computing environments are [84]: low latency, geographic distribution, heterogeneity, interoperability, real-time interactions, and scalability. Thus, fog computing is suitable for use when the cloud alone does not meet some requirements, such as low latency and runtime, required by applications [67].

3 Resource Management in Fog Computing

Resource management is relevant in several areas of research because it aims at the optimized use of available resources. This process is composed of steps that together intend to use a reasonable amount of computational resources and also to do it in an easy and efficient way [95]. The resource management process is widely discussed in fog computing since computational resources are often limited and must be well used [19], unlike cloud computing that gives the perception of having infinite resources, in the point of view of a single user.

Therefore, it is possible to find some literature review publications on resource management specifically for fog computing and related distributed paradigms. Table 1 shows a summary of the analyzed publications on this topic. As can be seen in Table 1, there is a divergence in the steps that compose the resource management process in fog computing available in literature, and each author considers different approaches and steps to contextualize it.

Table 1.

Paper	Year	Steps
Paper	Year	Estimation	Discovery	Allocation	Placement	Migration	Scheduling	Sharing	Optimization	Provisioning	Caching	Offloading	Preprocessing	Coordination	Load Balancing	Benchmarking	Modeling	Monitoring	Composition	Data Management	Detection	Selection	Mapping	Distribution	Assignment
[119]	2017	\(\checkmark\)		\(\checkmark\)										\(\checkmark\)
[178]	2018	\(\checkmark\)	\(\checkmark\)	\(\checkmark\)	\(\checkmark\)	\(\checkmark\)	\(\checkmark\)	\(\checkmark\)	\(\checkmark\)
[139]	2018	\(\checkmark\)		\(\checkmark\)
[108]	2018				\(\checkmark\)	\(\checkmark\)		\(\checkmark\)		\(\checkmark\)	\(\checkmark\)	\(\checkmark\)	\(\checkmark\)	\(\checkmark\)
[65]	2019			\(\checkmark\)	\(\checkmark\)		\(\checkmark\)			\(\checkmark\)		\(\checkmark\)			\(\checkmark\)
[81]	2019		\(\checkmark\)		\(\checkmark\)										\(\checkmark\)	\(\checkmark\)
[89]	2020		\(\checkmark\)
[171]	2020		\(\checkmark\)		\(\checkmark\)										\(\checkmark\)	\(\checkmark\)
[31]	2020	\(\checkmark\)	\(\checkmark\)	\(\checkmark\)													\(\checkmark\)	\(\checkmark\)
[150]	2020			\(\checkmark\)			\(\checkmark\)			\(\checkmark\)					\(\checkmark\)
[121]	2020						\(\checkmark\)			\(\checkmark\)	\(\checkmark\)	\(\checkmark\)			\(\checkmark\)
[14]	2020			\(\checkmark\)	\(\checkmark\)		\(\checkmark\)			\(\checkmark\)		\(\checkmark\)			\(\checkmark\)
[76]	2020			\(\checkmark\)			\(\checkmark\)			\(\checkmark\)		\(\checkmark\)							\(\checkmark\)	\(\checkmark\)
[29]	2020			\(\checkmark\)			\(\checkmark\)			\(\checkmark\)					\(\checkmark\)			\(\checkmark\)			\(\checkmark\)	\(\checkmark\)	\(\checkmark\)
[132]	2020			\(\checkmark\)
[75]	2020			\(\checkmark\)							\(\checkmark\)	\(\checkmark\)												\(\checkmark\)	\(\checkmark\)
[120]	2020						\(\checkmark\)			\(\checkmark\)					\(\checkmark\)			\(\checkmark\)
[28]	2020			\(\checkmark\)					\(\checkmark\)
[144]	2021			\(\checkmark\)	\(\checkmark\)		\(\checkmark\)		\(\checkmark\)	\(\checkmark\)
[128]	2021		\(\checkmark\)	\(\checkmark\)	\(\checkmark\)							\(\checkmark\)			\(\checkmark\)
[85]	2021		\(\checkmark\)				\(\checkmark\)
[127]	2021			\(\checkmark\)			\(\checkmark\)
[116]	2021			\(\checkmark\)			\(\checkmark\)			\(\checkmark\)		\(\checkmark\)			\(\checkmark\)
[164]	2021			\(\checkmark\)					\(\checkmark\)
[105]	2021						\(\checkmark\)

Table 1. Related Surveys

By detailing the definitions for all steps of each publication, it is possible to find the intersections among them, and consolidate them in a smaller but representative new group. For example: for Nath et al. [139] the resource allocation is when “different resources should be properly allocated to different devices”, but Mijuskovic et al. [128] consider this as the Placement step. This occurs with most of the definitions. To solve this confusion between the concepts and the different steps that each author assigned to the resource management process, a detailed analysis was carried out between the articles presented in Table 1. After this analysis, it was possible to group all 24 steps presented in Table 1 in just five steps, namely Discovery, Estimation, Allocation, Monitoring, and Orchestration and the whole scope of resource management in the fog computing environment can be represented by them, as presented in Figure 2.

Fig. 2.

Based on this proposal, the resource management process can be briefly explained. The Discovery step aims to find available resources in the fog environment. The Resource Estimation step defines the number of resources that will be needed for workload execution [120]. The Allocation step selects the resources that meet the requirements defined in the Estimation step, reserving, and delivering them to perform the tasks, meeting the Quality of Service (QoS) criteria defined previously [171]. Once the resource Allocation step is complete, the Monitoring process starts. It considers aspects such as elasticity, load-balancing, fault tolerance, health-check, and the like [43]. An Orchestration process must therefore be used in this cycle to provide complementary functions such as security, request management, communication management, and so on [44]. Each of the five steps is detailed in the next subsections.

3.1 Resource Estimation

One of the main requirements in managing computing resources is the ability to estimate how many resources will be needed to perform a task [178]. For Manvi and Shyam [123] this process is “a rough estimate of the real resources needed to run an application, usually with some thought or calculation involved”. Going a little deeper into this definition, Mahmud et al. [119] indicate that estimation is a process that assists in the allocation of appropriate computational resources, according to some policies and/or criteria, aiming to reach the determined level of QoS. Thus, given the restriction of computational capabilities of fog nodes, the estimation process plays a fundamental role in the allocation and optimal use of resources in fog computing [194].

The estimation for fog computing can involve perspectives other than computational resources [194], such as pricing [3, 27] and energy consumption [110]. In addition, resource estimation also depends on type of device, mobility, energy, types of data to be generated or processed, communication method, security, and also user behavior [31]. The estimation must be able to handle fluctuations in resource demand on both the provider and end-user sides. This is because resources can be mobile and therefore quickly become inaccessible, which makes them less reliable than cloud computing resources, for instance. Linked to this, the requester’s mobility must also be considered, which implies a sudden turnover of users, resulting in dynamic requests [178]. Considering this scenario, the resource estimation step should be performed with minimal overhead and high precision [139].

After analyzing all papers presented in Table 1 and considering all similar terms used that can be related to the resource estimation, as summarized in Figure 2, our proposed definition for this step is: “Resource estimation plays an essential role in the resource management process and refers to the calculation of the amount of computing resources and the time needed to perform tasks in fog computing environments”.

3.2 Resource Discovery

Manvi and Shyam [123] define the discovery process as identifying the list of authenticated computational resources that are available for submitting jobs, and choosing the best among them. For Singh and Chana [166], resource discovery is a process of identifying available resources and generating a list for later selection. Thus, resource discovery encompasses the actions of locating and disseminating resource information, this being essential to fully explore all resources distributed in the environment [197].

Considering the characteristics of fog computing, such as mobility, high geographic distribution, and also heterogeneity, the resource discovery process is considered a hard challenge and fundamental for the environment [47]. In Masip et al. [126] the resource discovery problem in fog computing is stated as a way of designing a solution to find resources belonging to components willing to join a fog environment. Such a solution must consider the different characteristics inherent to fog computing, such as mobility or collaborative models.

By analyzing the papers presented in Table 1, the Discovery step was grouped with the Detection step and both refer to the task of providing a collection of resources [128]. According to Javadpour et al. [89] there are five main classes of resource discovery mechanisms in fog computing: centralized, decentralized, peer-to-peer, hierarchical, and agent-based.

Based on these references, our proposed definition of the Resource Discovery step is the following: “Resource Discovery refers to the task that aims to find the available resources in the fog computing environment, while updating the resource catalog”.

3.3 Resource Monitoring

Monitoring is a relevant functionality and it is considered crucial to properly orchestrating fog services, and collecting updated status information about fog nodes and communication links [62]. A fog monitoring service can be composed of three essential functions [7]:

•

Observation: the acquisition of updated statuses of resource usage (e.g., CPU load and latency) or service performance (e.g., response time);

•

Data processing: is related to the necessary adjustments and transformation required on data and notifications derived from pre-configured rules and thresholds;

•

Data exposition: where the generated data is stored and how it can be accessed by a management system.

By analyzing the Table 1, the migration and the optimization steps have a similar approach to monitoring and that is why they were grouped in the proposal presented in Figure 2. A literature review and a taxonomy to resource monitoring in fog computing are presented in the work [43]. With this, our proposed definition of the resource monitoring step is: “Monitoring collects updated status information about fog nodes and communication links and sends it to the orchestrator, which can take proper action to guarantee the SLAs”.

3.4 Resource Orchestration

Fog orchestration is a management function responsible for the service life cycle. To provide requested services to the user and assure the Service Level Agreement (SLA), it must monitor the underlying infrastructure, react timeously to its changes, and comply with privacy and security rules [44].

However no paper analyzed in Table 1 used the term “orchestration” to define the steps in the resource management process. It is proposed in this paper, in Figure 2, as it would best meet the other terms used. For example: although Li et al. [108] call this the coordination step, they consider that this step “aims to coordinate various edge nodes”, which is the same purpose of orchestration.

Considering the analyzed papers, our definition for the fog orchestration step is: “Fog orchestration is a management function responsible for service life cycle. To provide requested services to the user and assure the SLAs, it must monitor the underlying infrastructure, react timeously to its changes and comply with privacy and security rules”.

3.5 Resource Allocation

Considering the steps presented in the Table 1 and grouped in Figure 2, the Allocation step plays an essential role in the resource management process, encompassing the largest number of terms and being discussed by most of the authors.

In Nath et al. [139] and in Luo et al. [116] resource allocation must deliver the most suitable resource considering the knowledge obtained from the previous steps, such as resource estimation or offloading. Both Mahmud et al. [119] and Mijuskovic et al. [128] state that resource allocation is a technique used to optimize resource utilization.

In fog computing, the allocation of resources aims to meet a set of tasks \(T = \lbrace T_1,T_2, \ldots ,T_n\rbrace\) that have different QoS requirements (such as cost and execution time), in a set of resources \(R = \lbrace R_1, R_2, \ldots ,R_m\rbrace\). These resources have different computational capacities, using an objective function as a criterion (such as minimizing cost, maximizing usage, etc.) [65]. The Resource Allocation step is represented in Figure 3. The resource allocation can be seen as the step that receives estimation information and makes the effective reservation of computational resources so that the task can be executed in the fog computing environment, according to the QoS requirements.

Fig. 3.

As with any process, resource allocation has inputs and outputs. Inputs come from the previous steps, which in this resource management proposal is the Estimation step. These entries indicate QoS metrics as well as other requirements (e.g., computational). The output of resource allocation is made up of the allocation technique, virtualization method, and the layers of architecture to be covered. An overview of the resource allocation is presented in Figure 4.

Fig. 4.

Considering both Figure 3 and Figure 4 it is possible to propose our definition of the Resource Allocation step: “Resource allocation refers to the step in the resource management process that aims to select, reserve, and use the best available resources to execute a workload in the fog environment, while respecting QoS parameters”. Based on this definition and understanding that it is necessary to be more familiar with proposals regarding the allocation of resources existing in the literature, it was possible to determine the Research Questions (RQs) to be answered by this work. This is important to help to determine what is being researched, what results should be achieved, and guide the systematic literature review. To achieve this goal, in this article, six RQs were formulated, as follows:

•

RQ1: What metrics are used? - this question aims to find the most used metrics and requirements passed as input in the resource allocation flow and help to ascertain their meaning;

•

RQ2: What techniques are used? - the answer to this question will show the most common techniques used in literature until now, indicate possible trends, and gaps to be studied;

•

RQ3: Which architecture layers are considered? - considering fog computing architecture presented in Figure 1, the answers to this question will show the trends in fog computing and the relevance of each one of them in the resource allocation approaches;

•

RQ4: What virtualization techniques are used? - this research question will help to categorize the proposals according to virtualization techniques most used;

•

RQ5: How are the proposed approaches evaluated? - this question aims to present the ways in which the proposals for resource allocation are currently being evaluated by the papers in the literature;

•

RQ6: Which are the most used use cases? - this question will help to identify the most common domains where fog computing is being applied nowadays, and highlight new areas to be studied.

The first research question (RQ1) refers to the metrics and requirements passed as input. The techniques (RQ2), the covered layers in architecture (RQ3), and the virtualization model (RQ4) form the core of the Resource Allocation step and are essential to deliver the allocated resource as output. Finally, RQ5 and RQ6 were chosen to present the main evaluation tools used to validate the resource allocation proposals and the most commonly applied use cases, respectively.

4 Research Methodology

The methodology of Systematic Literature Review (SLR) adopted in this work was based on the works [100] and [149]. The steps for the development of this research are summarized in Figure 5.

Fig. 5.

In the first stage, Research Questions (RQ) were defined (Section 3.5), aiming to determine what is being researched, as well as to define which results should be achieved with the systematic review. In addition, research questions also serve to guide the research [100]. These questions were the starting point for search string elaboration, as well as assisting in assessment of the publications. The results are expected to answer these questions.

After the research questions were formulated, the search process included the selection of the online search databases that were used, the period and the search string elaboration. For this article, Scopus,¹ Web of Science,² ACM Digital Library³ and IEEE XploreLibrary⁴ databases were used as research sources. These databases were chosen because they bring publications from different publishers, such as Elsevier, IEEE, ACM, MDPI, and the like. The period between 2012 (year of the first publication on fog computing [33]) and August 2022 was also specified. The following search string was used “TITLE-ABS-KEY((fog) AND (“resource management” OR “resource allocation” OR “resource placement” OR “resource scheduling” OR “resource sharing” OR “resource caching” OR “resource offloading” OR “resource selection” OR “resource load balancing” OR “resource provisioning”))”.

After using the search string in the related online databases, it was necessary to apply the inclusion and exclusion criteria. Therefore, the inclusion criteria defined for this research were: publications written in English; peer-reviewed publications; and, that present architectural models, techniques or methods applied to computational resource allocation, specifically, in fog computing. Likewise, exclusion criteria were those that do not meet any of the inclusion criteria listed above, as well as duplicate articles in the databases, articles that are not related to fog computing, or publications that are not oriented to computational resource allocation processes. Initially, the research carried out in the databases returned a total of 1,182 publications. Applying the inclusion and exclusion criteria and considering the contents found in the titles and abstracts of these publications, this number was reduced to 203 publications. A second filter was applied considering the article full text adherence, resulting in 108 publications to be analyzed.

Considering the 108 analyzed publications, a relevant fact is that 33 of them (about 31%) were presented in the last 3 years (2020, 2021, and 2022). This reaffirms the hypothesis that fog computing is a computational paradigm still in development in academia [180]. Another relevant point is that considering only the article title, the term “resource allocation” was the most found when compared with all other terms searched in the search string, representing about 60% of them. Finally, a full text reading and analysis of each one of the 108 selected papers was performed to help to answer the six research questions. The results are presented in the next section.

5 Results

This section presents the results obtained from the 108 selected publications. Figure 6 presents an overview of the research questions to be discussed in this section.

Fig. 6.

5.1 Resource Allocation Metrics

The first Research Question (RQ1) to be answered refers to the most used metrics for resource allocation in a fog computing environment. A metric is almost always linked to a criterion, giving rise to the allocation objective, which in turn is indicated in terms of QoS [144]. For example, the metric “cost” is linked to the criterion of “reduction”, generating the objective of “minimizing the cost”.

Analyzing the selected publications, the following resource allocation metrics were observed: resource utilization, cost, latency, energy, user experience, and execution time. They are detailed in Table 2, which also gives the percentage of the analyzed papers that each metric represents.

Table 2.

Metric	Papers	Description	%
Resource Utilization	[10, 12, 17, 23, 25, 26, 56, 69, 78, 96, 101, 104, 112, 129, 131, 161, 168, 185, 186, 188, 193]	Maximize the use and availability of resources, considering the adequate mapping of these resources and the correct distribution and allocation [135].	19.4%
Cost	[15, 16, 27, 38, 48, 56, 61, 68, 70, 80, 82, 86, 90, 91, 93, 98, 102, 107, 113, 114, 117, 138, 140, 142, 167, 168, 172, 174, 190, 198, 200]	Minimize the cost of allocating resources considering the infrastructure implementation, the operation costs, and cost of allocating instances [120]. .	28.7%
Latency	[8, 45, 54, 56, 64, 66, 72, 79, 98, 106, 107, 109, 115, 133, 141, 155, 158, 160, 169, 170, 175, 177, 182, 188, 201, 203]	It refers to the communication delay to transfer data between resources added to the time taken for the task to be processed [87].	24.1%
Energy	[40, 45, 63, 66, 110, 111, 138, 155, 156, 175, 187, 195, 204]	Fog computing devices can use both renewable and non-renewable energy. For this reason, the techniques for resource allocation with focus on energy efficiency aim to minimize energy consumption. Therefore, energy efficiency is one of the main metrics in this context and much research has been done to achieve this metric [120].	12.0%
User Experience	[1, 2, 3, 4, 5, 32, 51, 92, 99, 161, 179, 189, 202]	Consider that demands can change during the execution of tasks and that service quality levels must be guaranteed throughout the entire service life cycle [120]. The QoE is impacted by both quantifiable and non-quantifiable attributes.	12.0%
Execution Time	[13, 39, 48, 58, 59, 71, 73, 78, 80, 83, 88, 90, 102, 109, 122, 130, 148, 152, 174, 183, 184, 187, 192, 199]	The execution time metric is linked to the objective of reducing the delivery time and to meet the specified deadline for workload execution [120]. It can also be referenced as makespan, runtime, throughput, or deadline [87]. In fog computing many of the approaches that aim at reducing execution time are linked to reducing the latency or the energy consumption.	22.2%

Table 2. Resource Allocation Metrics

From the analysis of the publications presented in Table 2 and answering RQ1, it was possible to identify that the most addressed metric in resource allocation is cost, in 31 of the 108 publications (28.7%). In fact, reducing cost is an objective that can intersect with several other metrics, such as reducing energy consumption, reducing latency, or even reducing execution time. In this sense, it was noted that some papers appear in two different classifications [45, 48, 66, 78, 80, 90, 98, 102, 107, 109, 138, 155, 161, 168, 174, 175, 187, 188] which is possible when the authors of these papers combine different metrics in their proposals. Etemadi et al. [56], for example, proposed a resource allocation model that combined three metrics - cost, latency, and resource utilization - aiming to reduce the total cost and delay violation, and increase the fog node utilization. When grouped, these metrics become even more relevant for the resource allocation field in fog computing, as they optimize resource utilization, thus avoiding the waste of computational power in fog computing since the fog nodes usually offer limited resources [135].

Reduce latency is also a very significant metric in fog computing when compared to other computing paradigms, such as cloud computing [33], and this was confirmed by the results when they presented this metric as one of the most used in the analyzed articles (24.1% of them). The demand for fog computing is usually linked to the need to reduce application response time and latency has a direct impact on this. In this way, reducing latency is a metric that can be considered inherent to the resource allocation process in fog computing, and must be achieved by all resource allocation proposals, since this is an essential feature of this paradigm [67].

In addition to the metrics listed in this section, some other metrics could be proposed. The allocation time, for instance, is considered a relevant metric. This is because this metric refers to the time needed to receive the workload information, estimate, and verify the necessary resources and make the allocation effective. Considering the mobility and dynamics of fog applications, an optimized allocation time plays a fundamental role in ensuring a high user experience and in achieving the required service quality [75].

Other metrics that could be used are related to the time and number of migrations between the fog and cloud nodes, as an efficient allocation should keep the quantity of migrations as low as possible. The use of predictive algorithms is proposed, as they can assess how stable the allocated resources are, aiming to ensure that the resources will be available until the end of workload execution.

5.2 Resource Allocation Techniques

RQ2 asks which are the most used techniques in the analyzed papers for resource allocation in fog computing. Table 3 shows the techniques used in the analyzed papers, grouped in Integer Linear Programming (ILP) / Nonlinear Programming (NLP), Heuristics, Meta-heuristics, Fit-based approaches, Multiple Criteria Decision Making (MCDM), Game-based approaches, and Machine Learning.

Table 3.

Technique	Papers	Description	%
ILP / NLP	[15, 17, 40, 58, 59, 61, 70, 104, 110, 111, 133, 158, 160, 161, 167, 170, 172, 179, 183, 190, 192, 195, 201]	ILP solves optimization problems that have restrictions, with objective function linear in relation to the control variables. NLP solves an optimization problem where some of the constraints or the objective function are non-linear [30].	21.3%
Heuristic	[1, 2, 3, 4, 5, 8, 27, 32, 38, 48, 63, 70, 73, 79, 82, 86, 88, 92, 102, 109, 113, 114, 115, 117, 122, 140, 142, 156, 177, 185, 187, 193, 202, 203]	Heuristic algorithms are those that do not guarantee to find the optimal solution to a problem, but are able to return a quality solution in a time suitable for the application needs.	31.5%
Meta-Heuristic	[10, 16, 66, 72, 111, 138, 148, 175, 186, 204]	Aims to find optimal or near-optimal solutions for the resource allocation problem in fog computing within a reasonable duration of time.	9.2%
Fit	[90, 141, 152, 169, 190]	Fit-based algorithms are those that direct solutions to the parameter defined in its programming, such as best allocation, best fit, shortest job, or first fit [21].	4.6%
MCDM	[12, 13, 23, 25, 26, 51, 54, 78, 83, 107, 112, 129, 130, 182]	Is composed by decision support methods for several criteria, indicating adaptive models for tasks of scheduling, selection, and allocation of resources, assigning weights to their criteria as a way of choosing the resources that were more coherent to the QoS required.	12.9%
Game	[71, 80, 91, 93, 96, 99, 101, 184, 189, 199, 200]	Uses mathematical models to make optimal decisions under conflicting conditions. Each player has interest or preferences for each situation in the game [6].	10.2%
Machine Learning	[17, 39, 45, 56, 58, 64, 68, 69, 98, 106, 131, 155, 168, 174, 188, 198]	Aims to explore, analyze, and find meaning in complex data sets. They encompass other techniques such as Deep Learning and Reinforcement Learning.	14.8%

Table 3. Resource Allocation Techniques

Linear Programming consists of methods to solve optimization problems that have some restrictions (injunctions), the objective function being linear in relation to the control variables and the domain of these variables is composed of a system of linear inequalities [205]. The main advantage of Linear Programming is the flexibility to analyze complex problems [159]. A Linear Programming approach was used in the papers: [15, 40, 59, 61, 104, 111, 133, 158, 160, 167, 170, 172, 179, 183, 192, 195, 201], that represent 15.7% of the total analyzed papers. It is important to highlight that most of these proposals aim to meet only one metric (i.e., reduce latency in [158]), since this is the result of the linear objective function. In three other papers (2.7% of analyzed papers) [17, 110, 161] a Mixed Integer Linear Programming (MILP), which is an extension of ILP for when some decision variables are not discrete, was used to solve the resource allocation problem.

Nonlinear Programming, on the other hand, is the process of solving an optimization problem defined by a system of equalities and inequalities, called restrictions, over a set of real variables whose values are unknown, with an objective function to be maximized or minimized, and where some of the constraints or the objective function are non-linear [30]. This type of approach was used to address the resource allocation problem in fog computing in the seven publications: [15, 58, 70, 104, 111, 183, 201]. Among these works, Fan et al. [58] used, in addition to Nonlinear Programming, a Markov Decision Process technique to optimize the results for the resource allocation process in their proposal.

Heuristics were the most used techniques in the analyzed papers. In this type of solution, decisions are based only on the information available, without worrying about the future effects of such decisions, thus making the locally optimal choice at each stage of execution. The goal is to find a good, and not necessarily optimal, global solution. This type of approach suits the fog computing model as it supports the dynamics of the environment well, resulting from the essential characteristics of high geographic distribution, heterogeneity, and interoperability. Therefore, they are considered easy to implement and efficient [34]. In the analyzed papers on resource allocation in fog computing, some authors (12%) proposed new heuristic algorithms, as in [27, 63, 73, 79, 88, 102, 115, 122, 156, 177, 187, 193, 202]. Also some well-known heuristics methods were used, such as the Price-based approaches (10.1% of total) [1, 2, 3, 4, 5, 86, 92, 117, 140, 142, 203], Greedy algorithms (2.7%) [32, 82, 114], the Lyapunov optimization approach (2.7%) [8, 38, 109], and the Hungarian algorithm (0.92%) [185].

Similarly, meta-heuristic techniques combine basic heuristics at a higher structure level, but also aim to find an optimal or near optimal solution in a limited execution time, which is very relevant in fog computing environments given its dynamism and mobility characteristics. Within this category are Evolutionary Algorithms, which are based on the principles of natural evolution, maintaining a population of candidates for the solution throughout the research [53]. After initialization, new solutions are generated iteratively, selecting good solutions from the population, crossing, and mutating. New individuals are also evaluated and inserted into the population, usually replacing the worst solutions. The algorithm is normally interrupted after a certain number of iterations, returning the best solution found in that period [53]. Evolutionary algorithms for resource allocation in fog computing environments were used in seven papers, using the following algorithms: Elitist Selection Strategy [138] and [148], Pigeon Inspired Optimization [16], Weighted Sum Genetic Algorithm [72], Hungarian Algorithm [10], Directed Acyclic Graph [175], and Estimation of Distribution Algorithm [186]. Besides this, other meta-heuristic techniques were found in the reviewed papers, such as the Particle Swarm Optimization algorithm used in [66] and Ant Colony Optimization adopted in [204].

Considering the Fit-based approaches, the First-Fit algorithm was used in three papers [141, 152, 169]. In this technique, the allocation problem is solved by providing the first resource that delivers the requested parameters, regardless of whether there were better options. In this sense, the Shortest Job First algorithm used in [90] prioritizes the allocation of the smallest requests. Only the Best-fit algorithm presented in [190] looks for the best allocation considering the inputs and the available resources. Although these proposals are valid for some fog computing scenarios, they may be ineffective in environments that have high demand, that is, a large number of requests from IoT devices, for example.

With regard to MCDM techniques, in [129] the authors used the PROMETHEE method (Preference Ranking Organization Method for Enrichment Evaluation), while in [107] and [112] the authors used the ELECTRE method (Elimination Et Choice Translating Reality). The method called AHP was used in [51, 54, 130, 182]. Although they were able to meet some established QoS criteria, the methods were not able to guarantee that the minimum requirements were met. The same occurs with the method known as TOPSIS, which was used in [23, 25, 26, 83], and is also limited in its ability to achieve refined delivery quality.

The game-based approach uses mathematical models to make optimal decisions under conflicting conditions. With that, a basic element is the set of players who participate in it, and each player has strategies. The choice of a strategy reflects a situation among all possible situations. Each player has an interest or preferences for each situation in the game [6]. Between the analyzed publications, the Stackelberg game theory [143], in which the leader moves first, and the other players move in sequence was the most used (2.7% of total papers) [80, 93, 199]. Unlike proposals based on MCDM, these techniques are able to better find a way around the variations and restrictions of the fog computing environment, and therefore achieve interesting results in the resource allocation process for this paradigm.

Finally, Machine Learning techniques involve algorithms that aim to learn the fog environment, the users, and the resource allocation behaviors in order to predict new requests. In this context, there are Deep Learning techniques [188, 198], Deep Reinforcement Learning [68, 69, 98, 106], Bayesian learning [56, 168], and Fuzzy Logic [64, 155], that goes beyond the limits of Machine Learning and enters the Artificial Intelligence field [196]. Analysis of the results obtained in this survey shows that there has been an increase in the use of these techniques in recent years for proposals for resource allocation in fog computing, since 70% of them were proposed in the last 3 years. Although they may require greater computational power for their execution, given the need to process historical series and large volumes of data, they are more assertive in choosing the best resource to be allocated, even when considering different input parameters.

5.3 Covered Architecture Layers

Analysis of which layers of architecture are considered in resource allocation approaches in fog computing is necessary to answer research question RQ3. Most of the analyzed papers considered the fog architecture divided into three layers (IoT, Fog, and Cloud) in their proposals, as presented in Section 2. Therefore, the resource allocation in fog computing can be considered as a double correspondence problem [65]. Approaches to solving this problem may involve only the Fog Layer, or the communication between two (IoT x Fog or Fog x Cloud), or even three layers (IoT x Fog x Cloud) of the fog architecture. The publications are classified in these scenarios in Table 4.

Table 4.

Covered Layers	Papers	Description	%
Fog	[8, 54, 58, 66, 73, 98, 107, 112, 113, 114, 129, 148, 160, 161, 167, 179, 184, 186, 192, 193, 198, 203]	Uses only the resources available in the Fog Layer	20.4%
Cloud - Fog	[17, 45, 48, 71, 72, 90, 92, 93, 102, 158, 174]	Uses the resources allocated in the two higher layers in the three-tier architecture	10.2%
IoT - Fog	[38, 59, 61, 80, 82, 86, 106, 189, 201, 204]	Uses the resources allocated in the two lower layers in the three-tier architecture	9.3%
3-Layers	[1, 3, 4, 5, 10, 12, 15, 16, 23, 25, 26, 51, 56, 63, 64, 68, 69, 78, 79, 83, 88, 101, 104, 109, 117, 122, 130, 133, 138, 141, 152, 155, 156, 168, 169, 170, 175, 177, 182, 183, 187]	Uses the resources available in all layers in the three-tier architecture	49.1%

Table 4. Covered Architecture Layers

There are approaches that consider only the link formed between end-user devices, located in the IoT Layer, and devices in the Fog Layer. Once this link is established, they will connect to the Cloud Layer as a single resource group. Although these approaches do not disregard the existence and the relationship of the Fog Layer with the Cloud Layer, they seek alternatives to address all resource allocation problems in the Fog Layer, avoiding forwarding requests to the Cloud Layer.

Some analyzed publications focused on the connection formed between the Fog Layer devices and the Cloud Layer ones. The resources made available through this relationship allowed end users to run their workloads on them. The papers in this context consider that the services required by end-users, who are in the IoT Layer, have already been distributed in the Fog Layer and based on this premise, need to be provisioned using the resources available in that layer and in the Cloud Layer.

There are also some papers that apply their proposals considering only the Fog Layer. In this scenario, all requests must to be solved by the resources available in the fog environment. However, this is not usual since fog computing is complementary to the cloud and not a paradigm to replace it.

Finally, most works analyzed in this article had their proposals addressed using all three layers of fog computing architecture. This is the most common approach because it considers that the Fog Layer is only an intermediate layer to reach the objectives defined (for example, improving QoS). The requests are generated in the IoT Layer and are fully met using not only the Fog Layer, but also Cloud Layer. This makes sense since, as indicated in the definitions of fog computing presented in Section 2, this paradigm is intended to complement and not to substitute the cloud computing. In total, 53 out of the 108 analyzed publications (49%) considered all three fog layers to develop their proposals.

5.4 Virtualization Models

RQ4 aims to identify the virtualization models used in resource allocation approaches in fog computing. It is important to emphasize that in fog computing the availability of resources (such as processing, memory, storage, and network) is essential. Unlike cloud computing, where resources are always available, in the fog there is a strong constraint on resources, as they are often devices with low computational capacity. A switch, for example, has the main function of managing network connections, but it is used by the fog layer to provide its unused computing resources for processing, storage, and so on. Given the above, the virtualization models used in the analyzed studies can be grouped into two categories, as presented in Table 5 and discussed below.

Table 5.

Virtualization	Papers	Description	%
VM	[1, 2, 3, 4, 5, 10, 13, 15, 16, 26, 45, 56, 61, 66, 70, 78, 88, 90, 91, 102, 109, 111, 115, 117, 122, 133, 155, 156, 168, 172, 175, 187, 190, 193]	Exploits virtualization at the hardware level so multiple operating systems can run independently on a single physical resource.	31.5%
Container	[8, 12, 69, 92, 96, 115, 138, 160, 161, 188, 193]	Isolates the processes with just the necessary application packages.	10.2%

Table 5. Virtualization Models

The Virtual Machine (VM) concept is widely used, as it exploits virtualization at the hardware level so that multiple operating systems can run independently on a single physical resource. VM instances are executed on an abstraction layer called hypervisor, which allows the sharing of hardware between different instances [120]. The container is a type of virtualization which is lighter when compared to virtual machines and offers virtualization at the operational level [120]. Containers isolate processes with just the necessary application packages and are highly portable across multiple nodes of fog computing. In [193] the authors present some advantages in the use of containers when compared to virtual machines, as follows: containers start faster than VMs because hypervisors are not required, containers are better than VMs in terms of performance, and the greater the number of VMs deployed on a server, the higher the performance degradation of the server.

The use of an adequate virtualization model is fundamental for the performance of the application and the achievement of the objectives indicated in the QoS [19]. Undoubtedly, the use of VMs can be the best option in some use cases, such as those that require greater isolation of the application or service. However, according to the analyzed articles, the use of containers is more adherent to the resource allocation field in fog computing, since it is lighter and more dynamic compared to virtual machines, favoring mobility, and better adapting to resource constraints, which are characteristics of this computational model [125].

In two of the analyzed papers [115] and [193] both virtualization models - virtual machine and container - were used to better apply the resource allocation method proposed by them. Finally, none of the analyzed papers uses unikernel, which is a virtualization model that is even more lightweight than containers [118], which is a gap to be exploited in new proposals to achieve an efficient resource allocation.

5.5 Fog Computing Proposals Evaluation

Simulation tools and models are used to evaluate the work to bring the system closer to the real environment. A model is a representation of an actual or planned system [173]. Simulators are used to study the behavior of the system and understand the factors that affect its performance as it evolves over time [124]. Simulation frameworks provide solutions in cases where mathematical modeling techniques are difficult or impossible to apply due to the scale, complexity, and heterogeneity of a fog computing system [173]. Simulation is a way to imitate the operation of real systems, with the freedom to modify the inputs and model a series of characteristics, analyze existing systems, or support the design of new ones. It helps identify and balance the cost [24].

Some simulators that were already used to validate studies in cloud computing have been adapted for fog computing. In addition, new simulators have been specifically designed to meet the demand of fog computing. A detailed analysis of several simulators for fog computing was presented in [124]. This section aims to answer RQ5, which addresses the way that the proposed approaches were evaluated. An analysis of the simulators that were used in the selected publications is presented in Table 6.

Table 6.

Environment	Papers	Description	%
CloudSim	[1, 2, 3, 4, 5, 10, 13, 27, 102, 109, 122, 156, 204]	Proposed to simulate cloud computing services [36]	12.0%
iFogSim	[45, 56, 66, 69, 88, 101, 130, 141, 167, 168, 172, 175]	An extension of the CloudSim especially to Fog Computing [74]	11.1%
GridSim	[131]	Allows the modeling and simulation of application models for grid computing [35]	0.9%
Numerical (Matlab)	[15, 16, 23, 25, 26, 38, 51, 54, 58, 59, 61, 64, 71, 73, 78, 80, 82, 83, 86, 90, 91, 99, 104, 106, 107, 111, 112, 113, 117, 133, 140, 155, 177, 179, 182, 183, 184, 185, 186, 187, 189, 190, 192, 195, 201]	Used to study the behavior of systems whose mathematical models are too complex to provide analytical solutions	41.7%
Stand-alone	[8, 17, 32, 40, 48, 63, 70, 72, 92, 93, 110, 114, 115, 129, 138, 142, 148, 152, 160, 161, 169, 170, 174, 188, 193, 198, 199, 202, 203]	Hardware environment and synthetic datasets designed just for the execution of tests.	26.9%

Table 6. Simulation Frameworks

The majority of analyzed papers used numerical simulators to validate their proposals. This type of simulation is used to study the behavior of systems whose mathematical models are too complex to provide analytical solutions, as in many non-linear systems. This is the situation found in many proposals that addressed allocation resources in fog computing. The most common simulator was Matlab [163], used by 45 of the total of 108 publications, that is, about 42%.

CloudSim [36] was proposed to simulate cloud computing services. It is a library for cloud computing simulation developed in Java language, where each entity is represented as a class. Therefore, most of the works that used CloudSim presented a solution with integration with computing cloud environments, justifying the use of this simulator. An extension of the CloudSim simulator is iFogSim [74]. This simulator allows one to model IoT and fog environments to measure the impact of the proposed techniques for resource management in terms of latency, network congestion, energy consumption, and cost. Considering that it was only presented in 2017 [74], and considering the time required for its maturation and greater use, this simulator has come to be used more recently by academics.

In a less representative way, some analyzed studies used other simulators to evaluate their proposals. The GridSim simulator [35], which allows the modeling and simulation of application models for grid computing, was used in [131]. Finally, about 27% of the publications were evaluated in test environments built specifically to validate the paper’s proposal. In this type of test, all the software and the hardware are configured in a stand-alone way, using synthetic data sets.

The predominance of the use of simulators and mathematical models can be seen as a weakness in the evaluation of the proposals for the resource allocation process, since these mathematical models can hide unexpected behaviors of fog computing, mainly when considering the heterogeneity and mobility characteristics of these environments.

5.6 Fog Computing Domains

From 108 analyzed papers, just 20 of them (18%) indicated a specific domain their proposals are applied to. Thus, to address RQ6, these domains are detailed in Table 7.

Table 7.

Domain	Papers	Description	%
Health	[64, 70]	Medical Cyber-Physical Systems (MCPS) allows continuous and intelligent interaction between computational elements and medical devices needing low latency approaches	1.9%
Smart Buildings	[15, 66, 90, 158, 160, 161, 185, 193]	As fog computing is closer to IoT devices, it is widely used in smart cities, buildings, and industry projects [194] processing data and providing fast response	7.4%
Vehicular	[26, 39, 106, 148, 177, 188, 201, 203]	Vehicles must be able to calculate, store, and communicate to other vehicles or devices, improve safety, convenience, and driving satisfaction [103]	7.4%
Virtual Reality	[12, 79]	It is an advanced interface technology between a user and the computer creating an environment closer to the person’s reality with visual, sound, and tactile effects	1.9%

Table 7. Fog Computing Domains

In recent years, there has been growing attention to systems that support the development of vehicular networks. This is because vehicles have been increasingly equipped with powerful on-board computers, large capacity data storage units, and more advanced communication modules to improve safety, convenience, and driving satisfaction [103]. These vehicles must be able to calculate, store, and communicate with other vehicles or devices. The features and benefits of fog computing are totally adherent to this type of service, as vehicles have high mobility and some services, such as autonomous vehicles, and require a very low response time to be effective and safe. For this reason, resource allocation proposals for this type of domain must be prioritized by the execution time.

A trend in the health area is the use of Medical Cyber-Physical Systems (MCPS), which allow a continuous and intelligent interaction between computational elements and medical devices (e.g., heart-rate monitors) [70]. However, considering the complexity and high quality of required services, these devices need low latency and other criteria for communicating with the cloud computing platform. Therefore, fog computing is a promising approach for the use of these resources, because the proposals for resource allocation that used this domain have focused on the latency reduction metric.

As fog computing is closer to IoT devices, it is widely used in smart cities, buildings, and industry projects [194]. A smart building is one that is responsive to the requirements of occupants, organisations, and society. It also needs to be sustainable (energy and water consumption), healthy (well-being of the people living and working within it), and functional (user needs) [42]. A smart building use case was utilized in the papers [26], [66], [161], and [90] to address the resource allocation proposal. Like smart building, smart manufacturing is also a use-case that can take advantage of fog computing benefits, relying on the characteristics of high geographic distribution and heterogeneity of fog devices to seek an optimized resource allocation for the execution of applications and services.

Finally, Virtual Reality was used in two analyzed papers to explain the resource allocation use. As the need for these types of applications that require low latency is increasing, it is expected that new use-cases for fog computing will appear in the next years. The papers analyzed in this survey that used this domain have focused their proposals for resource allocation on latency reduction and on resource utilization metrics.

6 Related Work

As fog computing develops, the number of publications on this subject grows. In addition to surveys with a large scope in resource management, such as those presented in Table 8, it is already possible to find works on specific steps of the resource management process, such as discovery [97], monitoring [43], and orchestration [44].

Table 8.

Paper	Year	Analyzed Articles	Period	Number of RQ	Methodology	Metrics	Techniques	Evaluation	Virtualization	Architecture	Domains	Challenges
[165]	2019	5	2017 - 2019	-	-	\(\checkmark\)	\(\checkmark\)	-	-	-	-	-
[146]	2020	17	2015 - 2018	-	-	\(\checkmark\)	\(\checkmark\)	-	-	-	-	-
[127]	2020	28	2016 - 2019	-	-	\(\checkmark\)	\(\checkmark\)	\(\checkmark\)	-	-	\(\checkmark\)	-
[9]	2021	25	2017 - 2020	3	\(\checkmark\)	\(\checkmark\)	\(\checkmark\)	-	-	\(\checkmark\)	-	-
[52]	2021	20	-	-	-	-	-	-	-	\(\checkmark\)	\(\checkmark\)	-
[151]	2021	10	2016 - 2019	-	-	\(\checkmark\)	\(\checkmark\)	\(\checkmark\)	-	-	-	-
[87]	2022	49	-	-	-	\(\checkmark\)	\(\checkmark\)	\(\checkmark\)	-	-	\(\checkmark\)	\(\checkmark\)
[57]	2022	34	-	6	-	\(\checkmark\)	\(\checkmark\)	\(\checkmark\)	-	-	-	-
This Work	2021	108	2012 - 2022	6	\(\checkmark\)	\(\checkmark\)	\(\checkmark\)	\(\checkmark\)	\(\checkmark\)	\(\checkmark\)	\(\checkmark\)	\(\checkmark\)

Table 8. Related Works

Following this trend, considering our resource management proposal is composed of only five steps, and aiming to contribute to more in-depth research on a specific step, this article will focus on the analysis of publications on the Resource Allocation step. This step is the one that appeared most frequently in the surveys examined in Table 1, and summarized in Figure 2.

The works analyzed in this section are summarized in Table 8. They were compared considering the number and the period of the analyzed papers, the number of research questions, and the description of the methodology applied in the survey. It was also analyzed if they brought information about the six RQs defined in this current work, that is: metrics (RQ1), techniques (RQ2), architecture layers used (RQ3), virtualization applied (RQ4), evaluation methods (RQ5), and domains (RQ6). Finally, there was an evaluation of whether the work considered the challenges inherent to the Resource Allocation step.

Sindhu and Prakash [165] presented a survey of both resource allocation and task scheduling in fog computing based on IoT Applications. However, only five methods (or articles) were analyzed, and with this, little information has been added about the resource allocation step. In Patil-Karpe et al. [146] a review of resource allocation in fog computing is done by analyzing 17 articles, presenting the objectives and scope of each method. No information about the review methodology used and the challenges of the resource allocation problem was given.

A survey about scheduling is presented by Matrouk and Alatoun [127], classifying the scheduling problems into five categories: task scheduling, resource scheduling, resource allocation, job scheduling, and workflow scheduling. A comprehensive analysis of 28 selected papers, including information about the algorithm and the metric used in each one was presented. The survey partially reviews the scheduling approaches and does not include the recent studies as well.

Ahmed and Zeebaree [9] provided a systematic analysis of fog computing focusing on system models and resource allocation. The article is supported by three research questions that aimed to answer the relevancy, the metrics, and the goals of the resource allocation step, by analyzing 25 selected papers. However, the survey did not review the virtualization methods, nor the challenges related to resource allocation in fog computing.

In the paper [52], Dumitru et al. presented a review on resource allocation in both fog and edge computing focusing on the design and deployment of enterprise systems, such as medicine, automotive, and smart homes. However, no description of the research method was provided, nor details about the number of articles analyzed. Rahul and Aron [151] presented a review of 10 publications about the fog computing resource allocation problem, highlighting the performance metrics, goals, and the simulation environment of each one. Details about the methodology used to perform the review were not presented.

The authors Jamil et al. [87] provided a survey considering the Scheduling and the Resource Allocation steps in fog computing with a focus on learning-based dynamic algorithms. However, no description of the research method was provided, and information about the number and the period of the analyzed papers was missing. Finally, the work presented by Fahimullah et al. [57] brings a literature review about the use of machine learning techniques in fog computing considering six resource management steps: provisioning, placement, scheduling, allocation, offloading, and load balancing. Some metrics and simulations tools were also presented. The survey partially reviews the resource management approaches since only the machine learning techniques are considered.

By analyzing the papers presented in this section, it is apparent that all the existing surveys specifically about resource allocation (and related terms) to fog computing environments, are less comprehensive than this article. This can be explained by the fact that the period covered (10 years) by this paper is longer than in the others. Also, the number of analyzed articles is more than double those analyzed in the other papers. Finally, it is the only one that covers all topics related to the six research questions.

7 Challenges

This work analyzed 108 papers related to resource allocation in fog computing. The analysis enabled the six research questions presented to be answered, bringing to light valuable information about this research area. However, there are challenges that need more attention from the research community in the context of this work. The main identified challenges and possible future directions are detailed in this section.

•

Dynamicity: in a real fog infrastructure, users tend to change their computing resource requirements over time [164]. However, the user’s requests are considered static in most resource allocation proposals, meaning that an on-the-fly change in the requirements would not be possible. Updating of requirements after request fulfilment is a research gap declared by RQ1. A possible strategy is to verify if the new requirements can be fulfilled by the allocated resources. If they can not, the resource allocation process can be executed again, verifying if it is better to offload/migrate the service (e.g., to a powerful node or to the cloud) or to deploy it in the new place, depending on the predicted deployment time. Another possibility is the use of load balancing to permit a parallel processing and a better user experience even when there is not a powerful node available [55]. Predictive solutions, using machine learning techniques added to artificial intelligence methods seem to be a viable solution;

•

Mobility: Most of the analyzed papers assume that the fog nodes are fixed, but it is not always true. Mobility must be considered not only in IoT devices, but also in fog nodes, hindering the discovery, allocation, monitoring, and orchestration steps;

•

Privacy and security: In the context of resource allocation for fog computing, these two interrelated issues are challenging and need more research [18, 37, 164]. When accepting a user’s request, the management service in place must be in a position to validate whether the request is genuine and comes from a trusted user, before passing it to resource management [37]. However, there is a lack of efficient techniques and tools for assuring privacy and preventing attacks in resource-restricted devices [18];

•

Absence of use of unikernel: In response to RQ4, it was noted that the main model of virtualization used remains virtual machines, mainly due to the wide adherence and ease with which this model integrates with cloud computing services. However, more recent articles have given preference to the use of containers. No publication used the unikernel, which is also a virtualization method adherent to fog computing environments [19];

•

High heterogeneity of devices: Unlike cloud computing, which has more homogeneous and controlled equipment, fog computing can have equipment with different architectures (e.g., Raspberry Pi, Beagle board, several kinds of sensors, etc.). There is still a gap in the literature regarding proposals for resource allocation that is independent of the type of resource to be allocated, which would expand the scope of the environment in addition to meeting an essential characteristic of this paradigm;

•

High diversity in data: Considering the heterogeneity of the equipment, the data generated by it are also in different formats and are not standardized, making manipulation and use in applications difficult. The proposals presented do not indicate a way of dealing with these diverse data, which is a great challenge to be overcome;

•

Limited computational power: Regarding RQ3, which refers to the covered architecture layers, in the analyzed papers, there are few who consider that the fog nodes that will execute their proposed methods have low computational power. In this sense, many of the proposals can, in fact, only be executed on powerful servers allocated in the cloud layer, which generates a higher overhead than if this were allocated in the fog layer;

•

Limited connection: In the same sense, most proposals assume the existence of a high-speed connection, which may not be a reality for some equipment in the fog network and, mainly, for IoT devices that may be located in remote places, such as in cases of use in agriculture, for example;

•

Consider both QoS and QoE parameters: In response to RQ1, which considers the resource allocation metrics, the fog node is composed by objective attributes, such as CPU, storage, memory, and other computational capacities. These attributes are considered QoS parameters. Fog nodes are also composed of some subjective attributes, such as availability and reliability [162] and they are measured by QoE parameters. To achieve a proper fog computing management, consideration of both objective and subjective attributes is desirable, but this is still a gap in the analyzed papers;

•

Provider’s and User’s perspective: To perform fog computing management, two perspectives must be considered: (i) the end-user perspective; and (ii) the service provider perspective. On the one hand, end-users always want to obtain the best resources available, those with greater computational power and higher values for subjective attributes. On the other hand, service providers are interested in delivering the minimal set of resources to avoid unnecessary costs or situations involving resource unavailability for other users. There are publications that consider only the user’s perspective [77] [182] or other ones that consider only the provider’s perspective [54] [130]. Considering both perspectives, i.e., balancing the needs of end-users and providers, is still a matter that is open for investigation [194];

•

Multiple metrics: Although some analyzed articles considered two or even three metrics in their proposals, in a real-world environment the number of objectives to be met in the resource allocation can be greater, when considering, for example, the integration with the database, the connection with cloud providers, network and security protocols, relationship with legacy applications, and so on;

•

Absence of use in real scenarios: None of analyzed works has implemented the proposed resource allocation technique in a real-world use-case, as described in Section 5.5 and answered both RQ5 and RQ6, even though there are already a few experimental testbeds [157]. Nevertheless, the use of the approximation simulators, when trying to model reality, is not always sufficient to understand and predict the challenges and complexities of a real implementation [81]. In any case, the absence of fog computing services offered by public providers can still be considered as a weakness to be overcome to favor the implementation of new proposals for resource allocation.

8 Conclusion

This article provided, through a systematic literature review, an overview of the state-of-the-art in the process of computational resource allocation in fog computing, in addition to presenting the characteristics of this computational paradigm. To achieve this goal, it was necessary to first define the scope of the term resource allocation in fog computing, since there is no consensus on this among academics. An analysis of both existing surveys of resource management and resource allocation was performed. The mapping process continued with the definition of six research questions that guided the search, the selection criteria, and the evaluation of publications found.

Evaluating the 108 selected publications, it was possible to answer all the research questions. Firstly, regarding the resource allocation metrics, it was possible to identify that the cost is the most used, followed by the latency, and the execution time. It was also possible to observe the joint use of two or more metrics (i.e., reduce latency and optimize the resource utilization) and this can be seen as a good guide to achieving efficient resource allocation in new proposals.

Heuristic approaches are the most used techniques to address the resource allocation problem. However, it was also possible to observe from the research carried out, a recent growth in the use of machine learning techniques, which are effective in the resource allocation process because they rely on historical series and achieve more assertive predictions.

Most works analyzed in this article had their proposals addressed using all three layers of fog computing architecture, which is fitting when considering fog’s characteristic of being complementary to cloud. Regarding the virtualization model, virtual machines are the most preferred in the analyzed proposals, although the use of containers has grown in recent years, considering the analyzed publications. This can be seen as a good research opportunity for new resource allocation proposals.

The majority of analyzed papers used numerical simulators to validate their proposals and the absence of proposals validated in a real-world use-case is a gap to be overcome. Finally, although the adoption of use cases has little adherence among the articles analyzed, smart buildings and vehicular domains are the most used.

Thus, it was concluded that there are still many questions that need to be investigated in the academic sphere once its implementation is a reality and everyone can take advantage of the benefits of fog computing.

By presenting a systematic review of the literature specifically on resource allocation for fog computing, the present work contributes significantly to academia, providing support for researchers to direct their future works into the existing gaps yet to be explored.

Footnotes

⁴

References

[1]

Mohammad Aazam, Khaled A. Harras, and Sherali Zeadally. 2019. Fog computing for 5G tactile industrial internet of things: QoE-aware resource allocation model. IEEE Transactions on Industrial Informatics 15, 5 (2019), 3085–3092.

Abstract

1 Introduction

2 Fog Computing

3 Resource Management in Fog Computing

3.1 Resource Estimation

3.2 Resource Discovery

3.3 Resource Monitoring

3.4 Resource Orchestration

3.5 Resource Allocation

4 Research Methodology

5 Results

5.1 Resource Allocation Metrics

5.2 Resource Allocation Techniques

5.3 Covered Architecture Layers

5.4 Virtualization Models

5.5 Fog Computing Proposals Evaluation

5.6 Fog Computing Domains

6 Related Work

7 Challenges

8 Conclusion

Footnotes

References

Cited By

Index Terms

Recommendations

Orchestration in Fog Computing: A Comprehensive Survey

Recent advancements in resource allocation techniques for cloud computing environment: a systematic review

Fog-Marketing: auction-based multi-tier decentralized markets for fog resource provisioning

Comments

Information

Published In

Publisher

Publication History

Permissions

Check for updates

Author Tags

Qualifiers

Funding Sources

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

View options

PDF

eReader

Login options

Full Access

Figures

Other

Share

Share this Publication link

Share on social media

Affiliations