Open AccessCommunication

A Novel Artificial General Intelligence Security Evaluation Scheme Based on an Analytic Hierarchy Process Model with a Generic Algorithm

Guangyong Chen

^1,*

Yiqun Zhang

² and

Rui Jiang

The Third Research Institute, Ministry of Public Security, Shanghai 200031, China

School of Cyber Science and Engineering, Southeast University, Nanjing 210096, China

Author to whom correspondence should be addressed.

Appl. Sci. 2024, 14(20), 9609; https://doi.org/10.3390/app14209609

Submission received: 7 August 2024 / Revised: 28 September 2024 / Accepted: 2 October 2024 / Published: 21 October 2024

(This article belongs to the Special Issue Privacy and Security in Machine Learning and Artificial Intelligence)

Download

Browse Figure

Versions Notes

Abstract

The rapid development of Artificial General Intelligence (AGI) in recent years has provided many new opportunities and challenges for human social production. However, recent evaluation methods have some problems with regard to consistency, subjectivity and comprehensiveness. In order to solve the above problems, in this paper, we propose an Artificial General Intelligence Security Evaluation scheme (AGISE), which is based on analytic hierarchy process (AHP) technology with a genetic algorithm, to comprehensively evaluate the AGI security based on multiple security risk styles and complex indicators. Firstly, our AGISE combines AHP technology with a genetic algorithm to realize reliable, consistent and objective evaluation for AGI security. Secondly, in our AGISE, we propose implementing more effective AGI security evaluation classification and indicator settings. Finally, we demonstrate the effectiveness of our AGISE through experiments.

Keywords:

artificial general intelligence security; analytic hierarchy process; genetic algorithm

1. Introduction

In recent years, with the rapid development of the application of Large Language Models (LLMs), artificial general intelligence (AGI) technology has made many breakthroughs; however, it also faces many security issues. In order to make the AGI model more effective, it is necessary to effectively evaluate its security. Hence, in this paper, we have conducted research on three aspects. The first one is AGI security classification based on refined indicators from the perspective of AGI models. The second aspect is dynamic quantitative weight calculation, while the third aspect is an objective computable comprehensive evaluation model. In order to address the shortcomings of the existing methods, we propose a comprehensive AGI security evaluation scheme (AGISE) based on the analytic hierarchy process (AHP) with a genetic algorithm, and we conduct experiments to demonstrate the effectiveness of our method.

1.1. Related Work

The means and methods of AGI technology are developing rapidly; as a result of these developments, many researchers have proposed various AGI security evaluation models. One area of research focuses on the classification of security risk and the determination of an index system, while the other kinds focus on the design and construction of evaluation model.

In order to deal with the network attack classification and index system establishment, Nong et al. [1] proposed a system-fault-risk framework to subdivide the actions of network attacks and provide a threat risk analysis. However, in their scheme, the authors did not explicitly propose an attack classification framework. Alcaraz et al. [2] proposed an attack classification method for availability, integrity and confidentiality. According to the attack classification, the authors analyzed security threats to resources, information and users, and evaluated the network attack’s impact on critical infrastructure. Cazorla et al. [3] improved on the basis of [2] and proposed an AICAn (availability, integrity, confidentiality and anomalies) classification, which included the anomalies of the infrastructure in addition to availability, integrity and confidentiality. According to the AICAn classification, the authors [3] broadened the evaluation scope of the impact of a network attack. Gunduz et al. [4] proposed a classification method for network attacks based on confidentiality, integrity and availability (CIA) for smart grid security and divided the attacks into 17 types. However, the authors of [2,3,4] only provided a relatively effective classification of network attacks, but did not refine the quantifiable indicators of each type of attacks. Jia et al. [5] constructed a relatively complete network attack classification and index system via an analytic hierarchy process. However, the authors did not consider the consistency test and correction of the pairwise comparison matrix. Al-Zewairi et al. [6] classified unknown attacks by conducting experimental evaluations with modern shallow and deep ANN models, as well as two benchmark datasets commonly used in IDS research. However, their classification lacked a theoretical basis and could not prove the rationality of the classification. Bashaiwth et al. [7] utilized the long short-term memory (LSTM) model to classify DDoS attacks. However, their model struggled to distinguish between some types of DDoS attacks, which resulted in a poor classification performance. Ahmed et al. [8] proposed a model for detecting botnets using deep learning to identify zero-day botnet attacks in real time. However, the above schemes [7,8] had significant limitations and could only classify specific types of attacks, which made it difficult to apply them to the classification of other attacks. Aldhaheri et al. [9] developed a novel hybrid deep learning and Dendritic Cell Algorithm (DeepDCA) in the context of an Intrusion Detection System (IDS) to classify IoT intrusion and minimize false-alarm generation. However, their scheme only applied one dataset and extracted too few beneficial features, which made the results unconvincing. Kim et al. [10] proposed a TTP classification method to improve classification accuracy of cyber-threat intelligence. However, they applied an overfitting strategy to achieve their goals, which was only effective for specific versions of the dataset. Once the dataset was changed or new data were added, their results were likely to be invalid. Ahmed et al. [11] designed a 5G-enabled system, which consisted of a deep learning-based architecture that aimed to classify malware attacks on the Industrial Internet of Things (IIOT). However, all the above schemes [6,7,8,9,10,11] lacked classification indicators for attacks and were difficult to quantify.

In order to evaluate the security model, Huang et al. [12] proposed a novel federated Execution and Evaluation dual network framework (EEFED), which allowed multiple federal participants to personalize their local detection models. However, their model required additional global detection models to assist in achieving good performance. Meira et al. [13] presented an experimental study on the detection of unknown attacks with unsupervised learning techniques. However, the results showed that all their algorithms performed poorly in terms of precision. Li et al. [14] proposed an evaluation model based on the variable weight theory and technique for order preference with an ideal solution (TOPSIS) method and introduced the temporal and spatial correlation attributes into the network attack effect evaluation index system. Qi et al. [15] conducted cyber-attack analysis by constructing a knowledge graph. Kumar et al. [16] calculated the impact of attacks on critical infrastructure by designing a regression equation. However, their models [15,16] had limitations in specific scenarios. In the field of web application security, Kumar et al. [17] proposed a security durability evaluation scheme based on a hesitant fuzzy set, AHP and TOPSIS. In the field of engineering, Jiskani et al. [18] used the grey clustering method to evaluate mining engineering security effectively. In the evaluation of indicators and uncertainty, the analytic hierarchy process had a wide range of applications [19,20,21,22,23]. However, as mentioned in [12,13,14,15,16,17,18,19,20,21,22,23], there are still many problems with this process, such as the consistency of the pairwise comparison matrix and the determination of the stratification index.

1.2. Contributions

In order to achieve a more comprehensive and effective evaluation analysis of AGI security evaluation, in this paper, we propose an AGI security evaluation scheme (AGISE) based on AHP technology with a genetic algorithm. The main contributions of this paper are as follows.

Firstly, our AGISE can realize reliable, consistent and objective evaluation for AGI security. We apply the AHP (analytic hierarchy process) technology and combine the consistency correction and dynamic feedback to determine the indicators’ weight.

Secondly, in our AGISE, we propose a more effective AGI security classification and indicator settings. According to the type, frequency and threat of security, we provide the effective AGI security classification and determine the corresponding indicator settings. Hence, we can better evaluate the security risk of a certain AGI model.

Finally, we demonstrate the effectiveness of our AGISE through experiments. According to the experiment results, we can effectively evaluate the security of the AGI model.

1.3. Organization

The rest of this paper is structured as follows. In Section 2, we introduce some preliminary knowledge, including the analytic hierarchy process, priority theory, measure indices and criteria, and genetic algorithm. In Section 3, we describe our AGISE in detail. In Section 4, we conduct the experiment to evaluate an example with our AGISE. Finally, the conclusions are presented in Section 5.

2. Preliminary

2.1. Analytic Hierarchy Process

Definition 1.

[24]. A positive reciprocal matrix is defined as

A = {(a_{i j})}_{n \times n}

, if

a_{i j} > 0

a_{i j} = 1 / a_{j i}

and

a_{i i} = 1

for all

i, j \in \{1,2, \dots, n\} .

The matrix A is a pairwise comparison matrix (PCM), which is generated according to the results of a pairwise comparison. The value of

a_{i j}

is decided according to Table 1.

2.2. Measure Indices and Criteria

In this section, we introduce the result of measure indices and criteria.

Definition 2.

[24]. (Consistency index (CI)). The CI of a comparison matrix is given by

C I = \frac{λ - n}{n - 1}

where

λ

is the principal eigenvalue of the PCM and

n

is the order of the PCM.

Definition 3.

[25]. (Random consistency index (RI)). The RI is an average random consistency index, and the value is given in Table 2.

Definition 4.

[25]. (Consistency ratio (CR)). The CR is obtained via a comparison with the appropriate value of RI.

C R = \frac{λ - n}{R I \times (n - 1)}

(1)

Inference 1.

[25]. For a pairwise comparison matrix, if

C R = 0

, the PCM is completely consistent. If

0 < C R < 0.1

, the PCM is acceptably consistent. Otherwise, the PCM should be revised until

C R < 0.1

2.3. Priority Theory

In this section, we introduce the result of the logarithmic least squares method (LLSM).

Theorem 1.

[26]. Let

A = [a_{i j}]

be an

n \times n

judgement matrix and

G = [g_{i j}]

be the consistent matrix given by

g_{i j} = \frac{v_{i}}{v_{j}} .

The desired priority vector

v = (v_{1}, v_{2}, \dots, v_{n})'

can be expressed as an optimization constraint problem

M i n \sum_{i = 1}^{n} \sum_{j = 1}^{n} {(\ln a_{i j} - \ln v_{i} + \ln v_{j})}^{2} .

v_{i} = {(\prod_{j = 1}^{n} a_{i j})}^{1 / n}, i = 1,2, \dots, n .

2.4. Genetic Algorithm

In this section, we introduce the definitions of the genetic algorithm.

Definition 5.

[27]. (Fitness Function). The fitness function is mapped to the optimal solution considering two classes of constraints and denoted as

f i t n e s s = (g \circ f)

where

g

is the conditional function and

f

is the constraint function.

3. Our AGISE

3.1. Notations

The symbols in our AGISE are shown in Table 3.

3.2. Details of Our AGISE

3.2.1. AGI Security Classification

In this section, we propose the AGI security evaluation classification in Figure 1. In Figure 1, we design a hierarchical structure model which contains two layers. The first layer contains four different AGI security risks in the main AGI models, which are network security, model reliability, service security and data security and privacy protection. The second layer contains different evaluation indexes that are involved in each type of AGI security risk.

3.2.2. Construct AGI Security Risk Type PCM

In this section, according to the AGI security evaluation classification designed in Figure 1, we construct a pairwise comparison matrix

A = {(a_{i j})}_{n \times n}

through the results of expert evaluation. The value of

a_{i j}

is the importance of indicator

i

compared with indicator

j

, and its value is taken from Table 1.

3.2.3. Calculate AGI Security Risk Weight

In this section, according to the PCM of each AGI security risk type, we calculate the

ω_{t y p e} = (ω_{1}, ω_{2} \dots \dots ω_{m})

to express the weight for different security risks in a specific period.

3.2.4. Construct AGI Security Indicator PCM

In this section, according to the AGI security indicator system designed in Figure 1, we construct a series pairwise comparison matrix

B_{m} = {(b_{i j})}_{m}

through the results of expert evaluation. The value of

b_{i j}

is the importance of indicator

i

compared with indicator

j

, and its value is taken from Table 1.

3.2.5. Indicator Weight Calculation

According to the PCM

A = {(a_{i j})}_{n \times n}

and Theorem 1, the weight value of the

i

th indicator

ω_{i} (i = 1,2, \dots n)

ω_{i} = {(\prod_{j = 1}^{n} a_{i j})}^{1 / n}, i = 1,2, \dots, n o r ω_{i} = {(\prod_{j = 1}^{n} {\tilde{a}}_{i j})}^{1 / n}, i = 1,2, \dots, n, i f A i s n o t c o n s i s t e n c y

(2)

3.2.6. Consistency Correction with Genetic Algorithm

According to Definition 5, we construct the constraint function

f

and the conditional function

g

in fitness function

\{\begin{matrix} f (C R) = C R \\ g (x) = P_{1} \times P_{2} \frac{1}{x} \end{matrix} \Rightarrow f i t n e s s = (g \circ f) = P_{1} \times P_{2} \frac{1}{x}

(3)

In Equation (3), we construct

P_{1}

and

P_{2}

as follows

\{\begin{matrix} P_{1} = \{\begin{matrix} 10 \sum ω_{1} i f (\sum ω_{1} < 1.01) \\ - 10 e l s e \end{matrix} \\ P_{2} = \sum_{i = 1}^{n} (φ (i - 1) + λ) (ω_{i + 1} - ω_{i}) \end{matrix}

(4)

The part of the upper triangular matrix with the diagonal removed is taken as the initial genetic element, and the value is taken out for sequence coding to realize the selection, crossover and mutation operations.

3.2.7. Construct the Whitening Weight Function

In this section, according to different attributes of each indicator, the corresponding exponential whitening weight functions is designed as follows.

In first grey class, whitening weight function is expressed as

f_{i}^{1} (x) = \{\begin{matrix} e^{\frac{x - a_{s - 1}^{i}}{x}}, x \in [0, a_{s - 1}^{i}] \\ 1, x \in [a_{s - 1}^{i}, a_{s}^{i}] \\ 0, o t h e r s \end{matrix}

In the last grey class, the whitening weight function is expressed as

f_{i}^{s} (x) = \{\begin{matrix} 1, x \in [0, a_{1}^{i}] \\ e^{\frac{x - a_{1}^{i}}{x - a_{s}^{i}}}, x \in [a_{1}^{i}, a_{s}^{i}] \\ 0, o t h e r s \end{matrix}

In the remaining grey class, the whitening weight function is expressed as

f_{i}^{p} (x) = \{\begin{matrix} e^{\frac{x - a_{s - p}^{i}}{x}}, x \in [0, a_{s - p}^{i}] \\ e^{\frac{x - a_{s - p}^{i}}{x - a_{s}^{i}}}, x \in [a_{s - p}^{i}, a_{s}^{i}] \\ 0, o t h e r s \end{matrix}

where

a_{s - p}^{i}

is the boundary value of

p

grey class in

i

th indicator.

3.2.8. Clustering Coefficient Calculation

According to the weight value of each indicator obtained in Section 3.2.5 and the whitening weight function constructed in Section 3.2.7, the grey clustering factor is obtained as follows.

σ^{p} = \sum_{i = 1}^{n} f_{i}^{p} (x_{i}) ω_{i}

Then, the normalized clustering factor

i s

obtained as follows.

δ^{p} = \frac{σ^{p}}{\sum_{1}^{s} σ^{p}}

(5)

3.2.9. Comprehensive Result Calculation

According to Equation (5) in Section 3.2.8, under each grey class, the comprehensive clustering coefficient in one AGI security risk type is

γ_{t} = (s - \sum_{p = 1}^{s} p \cdot δ^{p}) \cdot \frac{10}{(s - 1)}

For the AGI security evaluation, the final comprehensive result is

γ_{c o m} = [\sum_{t} γ_{t} ω_{t y p e}]

According to the value of

γ

, the level of the attack behaviour can be determined. The final experimental results from 1 to 10 represent the threat level from low to high.

4. Experiments

4.1. Calculate the Weight of AGI Security Risk Types

Firstly, we construct PCM

A

of each type of AGI security risk with regard to the frequency and the influence of each indicator and the method described in Section 3.2.3.

A = (\begin{matrix} \begin{matrix} 1 & 7 \\ 1 / 7 & 1 \end{matrix} & \begin{matrix} 3 & 1 \\ 1 / 5 & 1 / 6 \end{matrix} \\ \begin{matrix} 1 / 3 & 5 \\ 1 & 6 \end{matrix} & \begin{matrix} 1 & 1 / 4 \\ 4 & 1 \end{matrix} \end{matrix})

Secondly, we calculate the CR of the PCM

A

according to Equation (1).

{C R}_{A} = 0.0512

, which satisfies the consistency requirement.

Finally, the weight

ω

of PCM

A

is calculated according to Equation (2) in Section 3.2.5. The results are shown in Table 4.

In Table 4, A denotes network security, B denotes Model Reliability, C denotes Service Security, and D denotes Data Security and Privacy Protection.

4.2. Calculate the Weight of AGI Security Indicator

Firstly, we construct PCM

B

of each indicator of AGI security according to the method described in Section 3.2.3.

B_{a} = (\begin{matrix} \begin{matrix} 1 & 1 / 5 \\ 5 & 1 \end{matrix} & \begin{matrix} 1 / 3 & 1 / 7 \\ 3 & 1 / 2 \end{matrix} & \begin{matrix} 1 / 3 \\ 1 \end{matrix} \\ \begin{matrix} 3 & 1 / 3 \\ 7 & 2 \end{matrix} & \begin{matrix} 1 & 1 / 4 \\ 4 & 1 \end{matrix} & \begin{matrix} 1 / 2 \\ 5 \end{matrix} \\ \begin{matrix} 3 & 1 \end{matrix} & \begin{matrix} 2 & 1 / 5 \end{matrix} & 1 \end{matrix}), B_{b} = (\begin{matrix} \begin{matrix} 1 & 1 / 7 & 2 \\ 7 & 1 & 5 \\ 1 / 2 & 1 / 5 & 1 \end{matrix} & \begin{matrix} 1 / 4 & 1 / 3 & 1 / 6 \\ 2 & 1 / 3 & 1 \\ 1 / 3 & 1 / 4 & 1 / 5 \end{matrix} \\ \begin{matrix} 4 & 1 / 2 & 3 \\ 3 & 3 & 4 \\ 6 & 1 & 5 \end{matrix} & \begin{matrix} 1 & 2 & 1 \\ 1 / 2 & 1 & 1 / 3 \\ 1 & 3 & 1 \end{matrix} \end{matrix}),

B_{c} = (\begin{matrix} 1 & 1 \\ 1 & 1 \end{matrix}), B_{d} = (\begin{matrix} \begin{matrix} 1 & 7 \\ 1 / 7 & 1 \end{matrix} & \begin{matrix} 3 & 2 \\ 1 / 3 & 1 / 5 \end{matrix} \\ \begin{matrix} 1 / 3 & 3 \\ 1 / 2 & 5 \end{matrix} & \begin{matrix} 1 & 1 / 3 \\ 3 & 1 \end{matrix} \end{matrix})

Secondly, we calculate the CR of each PCM according to Equation (1). The results are shown in Table 5.

Thirdly, if the CR of some matrix is greater than 0.1, we need to correct the PCM to be consistent according to Inference 1. In our experiment,

{C R}_{B_{b}} = 0.11329

, we correct

B_{b}

according to the method of Section 3.2.6.

{C R}_{B_{b}}^{*}

is satisfied as 0.03131.

{C R}_{B_{b}}^{*} = 0.03131

Finally, the weight

ω

of each PCM is calculated according to Equation (2) in Section 3.2.5. The results are shown in Table 6, Table 7, Table 8 and Table 9.

In Table 6, A1 denotes a DDoS Attack, A2 denotes Data Poisoning, A3 denotes Bot Attack, A4 denotes Jail-breaking, and A5 denotes Prompt Leaking.

In Table 7, B1 denotes Model Interpretability, B2 denotes Model Usability, B3 denotes Model Property Protection, B4 denotes Robustness, B5 denotes Failure Resolution Rate, and B6 denotes Access Control.

In Table 8, C1 denotes Content Response Management, C2 denotes Content Abuse Management.

In Table 9, D1 denotes Storage Security, D2 denotes Quality Review, D3 denotes Privacy Collection Compliance, D4 denotes Privacy Protection.

4.3. Evaluation Result

We extracted two datasets to evaluate AGI security evaluation through our model, and both of the datasets were generated based on measured data from two types of AGI models. The data are shown in Table 10.

According to Section 3.2.9, we calculate the experimental results based on these two different data and obtain the comprehensive clustering coefficient

γ_{t}

in one AGI security risk type and the final score

γ_{c o m}

of comprehensive AGI security evaluation.

The evaluation result of two dataset are shown in Table 11.

In dataset 1, the comprehensive clustering coefficient

γ

of Network security is 7.313,

γ

of Model Reliability is 8.944,

γ

of Service Security is 9.138,

γ

of Data Security and Privacy Protection is 4.761, and the final comprehensive score

γ_{c o m}

is 5.

In dataset 2, the comprehensive clustering coefficient

γ

of network security is 9.026,

γ

of Model Reliability is 5.346,

γ

of Service Security is 8.752,

γ

of Data Security and Privacy Protection is 9.470, and the final comprehensive score

γ_{c o m}

is 8.

As the weights of network security and Data Security and Privacy Protection are higher among all AGI security risk types in this scenario, the values of

γ_{1}, γ_{4}

are the determining factors in our evaluation scheme. For dataset 1, although the comprehensive clustering coefficients of Model Reliability and Service Security are higher among all AGI security risk types, the comprehensive score is relatively low. For dataset 2, the comprehensive clustering coefficients of network security and Data Security and Privacy Protection are higher among all attack types, so the comprehensive score is relatively high.

5. Conclusions

In this paper, we propose an AGI security evaluation scheme (AGISE), which is based on the AHP (analytic hierarchy process) technology with genetic algorithm, to comprehensively evaluate the multi-layer and multi-index AGI security risk with multiple risk styles and complex indicators. Our AGISE can realize reliable, consistent and objective evaluation of AGI security. In our AGISE, we propose more effective AGI security risk classification and indicator settings. We demonstrate the evaluation effectiveness of our AGISE through experiments and the evaluation results can reflect the AGI security situation correctly. In future research, we hope to improve the AGI security risk classification and make more scientific corrections to improve the consistency of the model with the emergence of new AGI models and security problems.

Author Contributions

Conceptualization, G.C.; methodology, G.C.; software, R.J. and Y.Z.; validation, Y.Z. and R.J.; formal analysis, Y.Z.; investigation, R.J.; resources, R.J.; data curation, Y.Z.; writing—original draft preparation, G.C.; writing—review and editing, G.C.; project administration, G.C.; funding acquisition, G.C. and R.J. All authors have read and agreed to the published version of the manuscript.

Funding

This work is supported by the National Natural Science Foundation of China (NO. 61372103), and National Engineering Research Center of Classified Protection and Safeguard Technology for Cybersecurity (C23640-XD-07). Rui Jiang is the corresponding author.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

The data generated by this institute is not currently available for public access.

Acknowledgments

Thank you for the support of the National Natural Science Foundation of China, and National Engineering Research Center of Classified Protection and Safeguard Technology for Cybersecurity.

Conflicts of Interest

The authors declare no conflict of interest.

References

Ye, N.; Newman, C.; Farley, T.R. A System-Fault-Risk Framework for cyber-attack classification. Inf. Knowl. Syst. Manag. 2005, 5, 135–151. [Google Scholar]
Alcaraz, C.; Lopez, J. Wide-Area Situational Awareness for Critical Infrastructure Protection. Computer 2013, 46, 30–37. [Google Scholar] [CrossRef]
Cazorla, L.; Alcaraz, C.; Lopez, J. Cyber Stealth Attacks in Critical Information Infrastructures. IEEE Syst. J. 2018, 12, 1778–1792. [Google Scholar] [CrossRef]
Gunduz, M.Z.; Das, R. Analysis of cyber-attacks on smart grid applications. In Proceedings of the 2018 International Conference on Artificial Intelligence and Data Processing (IDAP), Malatya, Turkey, 28–30 September 2018; IEEE: New York, NY, USA, 2018. [Google Scholar]
Jia, Z.; Wu, C.; Zhang, Y. Research on the Destructive Capability Metrics of Common Network Attacks. In Proceedings of the 2019 IEEE Smartworld, Ubiquitous Intelligence & Computing, Advanced & Trusted Computing, Scalable Computing & Communications, Cloud & Big Data Computing, Internet of People and Smart City Innovation (SmartWorld/SCALCOM/UIC/ATC/CBDCom/IOP/SCI), Leicester, UK, 19–23 August 2019; IEEE Computer Soc: Los Alamitos, CA, USA, 2019; pp. 1419–1424. [Google Scholar]
Al-Zewairi, M.; Almajali, S.; Ayyash, M. Unknown Security Attack Detection Using Shallow and Deep ANN Classifiers. Electronics 2020, 9, 2006. [Google Scholar] [CrossRef]
Bashaiwth, A.; Binsalleeh, H.; AsSadhan, B. An Explanation of the LSTM Model Used for DDoS Attacks Classification. Appl. Sci. 2023, 13, 8820. [Google Scholar] [CrossRef]
Ahmed, A.A.; Jabbar, W.A.; Sadiq, A.S.; Patel, H. Deep learning-based classification model for botnet attack detection. J. Ambient Intell. Hum. Comput. 2022, 13, 3457–3466. [Google Scholar] [CrossRef]
Aldhaheri, S.; Alghazzawi, D.; Cheng, L.; Alzahrani, B.; Al-Barakati, A. DeepDCA: Novel Network-Based Detection of IoT Attacks Using Artificial Immune System. Appl. Sci. 2020, 10, 1909. [Google Scholar] [CrossRef]
Kim, H.; Kim, H. Comparative Experiment on TTP Classification with Class Imbalance Using Oversampling from CTI Dataset. Secur. Commun. Netw. 2022, 2022, 5021125. [Google Scholar] [CrossRef]
Ahmed, I.; Anisetti, M.; Ahmad, A.; Jeon, G. A Multilayer Deep Learning Approach for Malware Classification in 5G-Enabled IIoT. IEEE Trans. Ind. Inform. 2023, 19, 1495–1503. [Google Scholar] [CrossRef]
Huang, X.; Liu, J.; Lai, Y.; Mao, B.; Lyu, H. EEFED: Personalized Federated Learning of Execution & Evaluation Dual Network for CPS Intrusion Detection. IEEE Trans. Inf. Forensics Secur. 2023, 18, 41–56. [Google Scholar]
Meira, J.; Andrade, R.; Praça, I.; Carneiro, J.; Marreiros, G. Comparative Results with Unsupervised Techniques in Cyber Attack Novelty Detection. In Proceedings of the Ambient Intelligence—Software and Applications, 9th International Symposium on Ambient Intelligence, Toledo, Spain, 20–22 June 2018; Novais, P., Jung, J.J., Villarrubia González, G., Fernández-Caballero, A., Navarro, E., González, P., Carneiro, D., Pinto, A., Campbell, A.T., Durães, D., Eds.; Springer International Publishing: Cham, Switzerland, 2019; pp. 103–112. [Google Scholar]
Li, Z.; Wu, B. A network attack evaluation model based on variable weight theory. In Proceedings of the International Conference on Network Communication and Information Security (ICNCIS 2021), Sanya, China, 3–5 December 2021; pp. 1217503-1–1217503-5. [Google Scholar]
Qi, Y.; Jiang, R.; Jia, Y.; Li, A. Attack Analysis Framework for Cyber-Attack and Defense Test Platform. Electronics 2020, 9, 1413. [Google Scholar] [CrossRef]
Kumar, D.; Khan, A.H.; Nayyar, H. Cyber Risk Assessment Model for Critical Information Infrastructure. In Proceedings of the 2020 International Conference on Power Electronics & IoT Applications in Renewable Energy and its Control: International Conference on Power Electronics & IoT Applications in Renewable Energy and its Control (PARC 2020), Mathura, Uttar Pradesh, India, 28–29 February 2020; pp. 292–297. [Google Scholar]
Kumar, R.; Khan, A.I.; Abushark, Y.B.; Alam, M.M.; Agrawal, A.; Khan, R.A. A Knowledge-Based Integrated System of Hesitant Fuzzy Set, AHP and TOPSIS for Evaluating Security-Durability of Web Applications. IEEE Access 2020, 8, 48870–48885. [Google Scholar] [CrossRef]
Jiskani, I.M.; Han, S.; Rehman, A.U.; Shahani, N.M.; Tariq, M.; Brohi, M.A. An Integrated Entropy Weight and Grey Clustering Method-Based Evaluation to Improve Safety in Mines. Min. Metall. Explor. 2021, 38, 1773–1787. [Google Scholar] [CrossRef]
Fazli, S.; Mavi, R.K.; Vosooghidizaji, M. Crude oil supply chain risk management with DEMATEL-ANP. Oper. Res. 2015, 15, 453–480. [Google Scholar] [CrossRef]
Lin, C.; Kou, G.; Peng, Y.; Alsaadi, F.E. Two-Stage Prioritization Procedure for Multiplicative Ahp-Group Decision Making. Technol. Econ. Dev. Econ. 2020, 26, 525–545. [Google Scholar] [CrossRef]
Wang, Z.; Ran, Y.; Chen, Y.; Yu, H.; Zhang, G. Failure mode and effects analysis using extended matter-element model and AHP. Comput. Ind. Eng. 2020, 140, 106233. [Google Scholar] [CrossRef]
Vafadarnikjoo, A.; Ahmadi, H.B.; Liou, J.J.H.; Botelho, T.; Chalvatzis, K. Analyzing blockchain adoption barriers in manufacturing supply chains by the neutrosophic analytic hierarchy process. Ann. Oper. Res. 2023, 327, 129–156. [Google Scholar] [CrossRef]
Aslan, O.; Aktug, S.S.; Ozkan-Okay, M.; Yilmaz, A.A.; Akin, E. A Comprehensive Review of Cyber Security Vulnerabilities, Threats, Attacks, and Solutions. Electronics 2023, 12, 1333. [Google Scholar] [CrossRef]
Saaty, T.L. A scaling method for priorities in hierarchical structures. J. Math. Psychol. 1977, 15, 234–281. [Google Scholar] [CrossRef]
Saaty, R. The Analytic Hierarchy Process—What It Is and How It Is Used. Math. Model. 1987, 9, 161–176. [Google Scholar] [CrossRef]
Crawford, G.; Williams, C. A Note on the Analysis of Subjective Judgment Matrices. J. Math. Psychol. 1985, 29, 387–405. [Google Scholar] [CrossRef]
Kou, G.; Ergu, D.; Shang, J. Enhancing data consistency in decision matrix: Adapting Hadamard model to mitigate judgment contradiction. Eur. J. Oper. Res. 2014, 236, 261–271. [Google Scholar] [CrossRef]

Figure 1. AGI security evaluation classification.

Table 1. The scale and its description.

Intensity of Importance	Description
1	Equal importance
3	Weak importance
5	Essential importance
7	Demonstrated importance
9	Absolute importance
2, 4, 6, 8	Intermediate values between the two adjacent judgements
Reciprocals	If activity $i$ has one of the above
Rational	Ratios arising from the scale

Table 2. The order of the matrix and the corresponding RI value.

$n$	1	2	3	4	5	6	7	8	9	10
Random consistency index (RI)	0	0	0.52	0.89	1.11	1.25	1.35	1.40	1.45	1.49

Table 3. Symbols explanation.

Symbol	Quantity
n	The order of PCM
i	The index of indicator items
λ	Principal eigenvalue of PCM
RI	Random consistency index
k,l	Maximum offset item index
p	The grey interval items
s	The grey interval number
t	The index of AGI security risk type

Table 4. Weight value of AGI security risk type.

	A	B	C	D
$ω$	0.3949	0.0485	0.1482	0.4084

Table 5. CR of AGI security indicator PCM.

	$B_{a}$	$B_{b}$	$B_{c}$	$B_{d}$
$C R$	0.03746	0.11329	0	0.02397

Table 6. Indicator weight value of network security.

	A1	A2	A3	A4	A5
$ω$	0.04798	0.22685	0.10002	0.46791	0.15724

Table 7. Indicator weight value of Model Reliability.

	B1	B2	B3	B4	B5	B6
$ω$	0.05158	0.30950	0.04464	0.19614	0.12372	0.27442

Table 8. Indicator weight value of Service Security.

	C1	C2
$ω$	0.5	0.5

Table 9. Indicator weight value of Data Security and Privacy Protection.

	D1	D2	D3	D4
$ω$	0.48280	0.05925	0.14410	0.31384

Table 10. The dataset of attack.

	Dataset 1	Dataset 2
A1	8	9
A2	8	7
A3	6	4
A4	5	8
A5	9	5
B1	6	2
B2	6	7
B3	8	5
B4	8	8
B5	9	7
B6	7	6
C1	8	8
C2	9	7
D1	7	9
D2	5	8
D3	5	7
D4	7	9

Table 11. The evaluation result of two dataset.

	Dataset 1	Dataset 2
$γ_{1}$	7.313	9.026
$γ_{2}$	8.944	5.346
$γ_{3}$	9.138	8.752
$γ_{4}$	4.761	9.470
$γ_{c o m}$	5	8

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

Share and Cite

MDPI and ACS Style

Chen, G.; Zhang, Y.; Jiang, R. A Novel Artificial General Intelligence Security Evaluation Scheme Based on an Analytic Hierarchy Process Model with a Generic Algorithm. Appl. Sci. 2024, 14, 9609. https://doi.org/10.3390/app14209609

AMA Style

Chen G, Zhang Y, Jiang R. A Novel Artificial General Intelligence Security Evaluation Scheme Based on an Analytic Hierarchy Process Model with a Generic Algorithm. Applied Sciences. 2024; 14(20):9609. https://doi.org/10.3390/app14209609

Chicago/Turabian Style

Chen, Guangyong, Yiqun Zhang, and Rui Jiang. 2024. "A Novel Artificial General Intelligence Security Evaluation Scheme Based on an Analytic Hierarchy Process Model with a Generic Algorithm" Applied Sciences 14, no. 20: 9609. https://doi.org/10.3390/app14209609

APA Style

Chen, G., Zhang, Y., & Jiang, R. (2024). A Novel Artificial General Intelligence Security Evaluation Scheme Based on an Analytic Hierarchy Process Model with a Generic Algorithm. Applied Sciences, 14(20), 9609. https://doi.org/10.3390/app14209609

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

	Dataset 1	Dataset 2
A1	8	9
A2	8	7
A3	6	4
A4	5	8
A5	9	5
B1	6	2
B2	6	7
B3	8	5
B4	8	8
B5	9	7
B6	7	6
C1	8	8
C2	9	7
D1	7	9
D2	5	8
D3	5	7
D4	7	9

	Dataset 1	Dataset 2
A1	8	9
A2	8	7
A3	6	4
A4	5	8
A5	9	5
B1	6	2
B2	6	7
B3	8	5
B4	8	8
B5	9	7
B6	7	6
C1	8	8
C2	9	7
D1	7	9
D2	5	8
D3	5	7
D4	7	9

Article Menu

A Novel Artificial General Intelligence Security Evaluation Scheme Based on an Analytic Hierarchy Process Model with a Generic Algorithm

Abstract

1. Introduction

1.1. Related Work

1.2. Contributions

1.3. Organization

2. Preliminary

2.1. Analytic Hierarchy Process

2.2. Measure Indices and Criteria

2.3. Priority Theory

2.4. Genetic Algorithm

3. Our AGISE

3.1. Notations

3.2. Details of Our AGISE

3.2.1. AGI Security Classification

3.2.2. Construct AGI Security Risk Type PCM

3.2.3. Calculate AGI Security Risk Weight

3.2.4. Construct AGI Security Indicator PCM

3.2.5. Indicator Weight Calculation

3.2.6. Consistency Correction with Genetic Algorithm

3.2.7. Construct the Whitening Weight Function

3.2.8. Clustering Coefficient Calculation

3.2.9. Comprehensive Result Calculation

4. Experiments

4.1. Calculate the Weight of AGI Security Risk Types

4.2. Calculate the Weight of AGI Security Indicator

4.3. Evaluation Result

5. Conclusions

Author Contributions

Funding

Institutional Review Board Statement

Informed Consent Statement

Data Availability Statement

Acknowledgments

Conflicts of Interest

References

Share and Cite

Article Metrics

Article Access Statistics

Further Information

Guidelines

MDPI Initiatives

Follow MDPI

	Dataset 1	Dataset 2
A1	8	9
A2	8	7
A3	6	4
A4	5	8
A5	9	5
B1	6	2
B2	6	7
B3	8	5
B4	8	8
B5	9	7
B6	7	6
C1	8	8
C2	9	7
D1	7	9
D2	5	8
D3	5	7
D4	7	9