[go: up one dir, main page]
More Web Proxy on the site http://driver.im/

CN118972333B - A transmission control method and system for Internet of Things terminal data - Google Patents

A transmission control method and system for Internet of Things terminal data Download PDF

Info

Publication number
CN118972333B
CN118972333B CN202411413064.4A CN202411413064A CN118972333B CN 118972333 B CN118972333 B CN 118972333B CN 202411413064 A CN202411413064 A CN 202411413064A CN 118972333 B CN118972333 B CN 118972333B
Authority
CN
China
Prior art keywords
data
priority
transmission
entropy
queue
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202411413064.4A
Other languages
Chinese (zh)
Other versions
CN118972333A (en
Inventor
彭超
范瑞祥
詹涛
吴朝晖
张晓宇
黄再
徐建平
闫娟
付训龙
郑美兰
武冬
杨亚韬
褚红亮
郑锦坤
吴志平
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Information and Telecommunication Branch of State Grid Jiangxi Electric Power Co Ltd
Original Assignee
Information and Telecommunication Branch of State Grid Jiangxi Electric Power Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Information and Telecommunication Branch of State Grid Jiangxi Electric Power Co Ltd filed Critical Information and Telecommunication Branch of State Grid Jiangxi Electric Power Co Ltd
Priority to CN202411413064.4A priority Critical patent/CN118972333B/en
Publication of CN118972333A publication Critical patent/CN118972333A/en
Application granted granted Critical
Publication of CN118972333B publication Critical patent/CN118972333B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L47/00Traffic control in data switching networks
    • H04L47/10Flow control; Congestion control
    • H04L47/24Traffic characterised by specific attributes, e.g. priority or QoS
    • H04L47/2425Traffic characterised by specific attributes, e.g. priority or QoS for supporting services specification, e.g. SLA
    • H04L47/2433Allocation of priorities to traffic types
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/23Clustering techniques
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/08Configuration management of networks or network elements
    • H04L41/0896Bandwidth or capacity management, i.e. automatically increasing or decreasing capacities
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/12Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks

Landscapes

  • Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Signal Processing (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • Physics & Mathematics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)

Abstract

本发明公开了一种物联网终端数据的传输控制方法及系统,具体涉及物联网终端控制技术领域,用于解决物联网终端传输控制不准确的问题;本发明通过物联网终端应用场景,合理确定传感器类型、分布及数据特性,制定采集目标和传输需求,设置采样频率与采集窗口,采用异步或同步采集策略,实现高效数据采集与调度,通过噪声滤波、异常值检测和数据规范化处理,确保数据完整性,并根据熵偏移指数量化数据优先级,结合动态阈值聚类识别重要事件,动态调整带宽资源,确保高优先级数据优先传输,且采用自适应调度机制和时间片轮询算法对不同优先级数据进行队列管理,优化数据传输效率以及提升了物联网终端设备可靠性。

The present invention discloses a transmission control method and system for Internet of Things terminal data, and specifically relates to the technical field of Internet of Things terminal control, and is used to solve the problem of inaccurate transmission control of Internet of Things terminals. The present invention uses Internet of Things terminal application scenarios to reasonably determine sensor types, distribution and data characteristics, formulate acquisition targets and transmission requirements, set sampling frequencies and acquisition windows, and adopt asynchronous or synchronous acquisition strategies to achieve efficient data acquisition and scheduling. The present invention ensures data integrity through noise filtering, outlier detection and data normalization processing, quantifies data priority according to entropy shift index, identifies important events in combination with dynamic threshold clustering, dynamically adjusts bandwidth resources, ensures that high-priority data is transmitted first, and adopts an adaptive scheduling mechanism and a time slice polling algorithm to manage queues for data of different priorities, thereby optimizing data transmission efficiency and improving the reliability of Internet of Things terminal equipment.

Description

Transmission control method and system for terminal data of Internet of things
Technical Field
The invention relates to the technical field of terminal control of the Internet of things, in particular to a method and a system for controlling transmission of terminal data of the Internet of things.
Background
In an internet of things (IoT) system, a terminal device collects various environmental or device status data in real time through sensors, and transmits the data to a central server or cloud for analysis and processing through a network. In actual deployment, the terminal equipment of the internet of things may be distributed in different application scenes such as factories, traffic, agriculture, medical treatment and the like, and in each scene, the data types, the importance and the urgency of the terminal equipment are different, for example, in an industrial monitoring scene, equipment state data (such as temperature and vibration) need to be transmitted preferentially so as to detect equipment abnormality in time, and in intelligent agriculture, sensor information such as soil humidity, meteorological data and the like can be processed in a delayed mode, and the same priority as equipment fault data is not needed.
The prior art has the defects that the terminal equipment of the Internet of things can be deployed in an environment with poor network conditions, network fluctuation can cause data loss or delay, influence the timeliness and integrity of the data, cause the generation of conditions such as limited bandwidth, frequent network fluctuation, poor equipment computing capacity and the like, ensure the prior transmission of key data under the limiting conditions, and are the most important ring in the data transmission process of the terminal equipment of the Internet of things, and the transmission priority scheduling of the terminal equipment of the Internet of things is inaccurate, so that the transmission delay or omission of the key data is caused, and the control of subsequent equipment is influenced.
Disclosure of Invention
In order to overcome the defects in the prior art, the invention provides a transmission control method and a transmission control system for data of an internet of things terminal, which are used for solving the problem of inaccurate transmission control of the internet of things terminal in the background art.
In order to achieve the above purpose, the present invention provides the following technical solutions:
a transmission control method of terminal data of the Internet of things comprises the following steps:
Determining an application scene of the terminal of the Internet of things according to the types, the distribution and the physical characteristics of the acquired data of the sensors, formulating a target for acquiring the data and an evaluation for transmitting the data, and determining the deployment positions and functions of various sensors;
Setting sampling frequency and a data acquisition window according to the type of a sensor and the data change rate, carrying out data acquisition and transmission scheduling by adopting an asynchronous or synchronous acquisition strategy, carrying out noise filtering and abnormal value detection on acquired data, and carrying out standardization processing on different types of data;
Quantifying the data priority according to the entropy shift index, identifying important event change based on a dynamic threshold clustering algorithm, dynamically adjusting bandwidth resources, and determining the data transmission priority;
And allocating independent queues to the data streams with different priorities based on the priority allocation result, adjusting the queue capacity through an adaptive scheduling mechanism, and carrying out priority polling by adopting a time slice polling algorithm to carry out data transmission efficiency and queue management optimization.
In a preferred embodiment, according to the type, distribution and physical characteristics of the collected data of the sensors, determining the application scene of the terminal of the internet of things, and formulating the target of the collected data and the evaluation of the transmission requirement, determining the deployment position and function of each sensor, the specific process is as follows:
Determining distribution density and position of sensors, determining sensor types used by terminal equipment, wherein the sensor types comprise temperature, humidity, vibration, pressure and gas concentration sensors, and analyzing deployment conditions of all the sensors in an Internet of things terminal;
determining the type and the characteristic of acquired data, wherein the data type comprises a numerical type, a discrete type and a symbol type;
Determining environmental characteristics, wherein the environmental characteristics comprise the dynamic property of a scene and abnormal conditions in the scene, the dynamic property of the scene comprises the environmental temperature, the humidity and the equipment use frequency, and the abnormal conditions in the scene comprise the influence conditions on data in equipment failure or emergency state;
combining the distribution density and the position of the sensor, the type and the characteristic of the data collected by the sensor and the environmental characteristic to obtain the scene of the terminal of the Internet of things;
The environmental sensor collects environmental data, the people stream detector, the noise sensor, the infrared sensor and the smoke alarm collect data as sensor data, and the camera collects data as video data;
And determining the transmission requirements of the collected target data, core data content and data, and determining the physical deployment position and function of the sensor according to the application scene.
In a preferred embodiment, according to the type of the sensor and the data change rate, setting a sampling frequency and a data acquisition window, adopting an asynchronous or synchronous acquisition strategy to perform data acquisition and transmission scheduling, performing noise filtering and outlier detection on the acquired data, and performing standardization processing on different types of data, wherein the specific process is as follows:
according to the sensor type, the data change rate and the bandwidth resource, determining the data acquisition frequency of each sensor, and setting a data acquisition window for each sensor;
Adopting a synchronous acquisition mechanism for time sequence relevance data, and adopting asynchronous acquisition for independent data sources;
After data acquisition, preprocessing the acquired original data, detecting and filtering high-frequency noise signals in sensor data by using a noise filtering method based on wavelet transformation, detecting abnormal values appearing in a high-dimensional data space by using a distribution density estimation technology, and carrying out smoothing processing by using an interpolation algorithm;
the data is processed using a normalization method based on laplace regularization.
In a preferred embodiment, the data priority is quantized according to the entropy shift index, the important event change is identified based on a dynamic threshold clustering algorithm, the bandwidth resource is dynamically adjusted, and the data transmission priority is determined by the following steps:
extracting features from the acquired data by using support vector regression based on a kernel function, extracting main features from each data stream, and forming feature vectors from the main features through nonlinear mapping;
constructing a data vector at each time point from sensor data of the internet-of-things terminal equipment, and calculating high-dimensional data entropy of each time point;
Performing dynamic tracking and increment judgment of the entropy offset, recalculating the entropy value of the data, and calculating the entropy difference between the front time point and the rear time point through an entropy offset formula, wherein the entropy difference represents the entropy value change amplitude of the data between the adjacent time points;
comparing the data entropy of adjacent time points, and dynamically detecting the uncertainty increment of the data stream, wherein the uncertainty increment represents the uncertainty change of the data in time;
Based on the calculated entropy offset, identifying important event changes through a dynamic threshold clustering algorithm, and determining whether the changes need to adjust data priority, wherein the offset represents entropy change trend of adjacent time points;
Quantifying the importance of the data through entropy shift indexes for the identified large entropy shift data points, and assigning priorities, wherein the entropy shift indexes represent the fluctuation degree in the data stream;
and allocating bandwidth resources to the data points according to the calculated entropy deviation index.
In a preferred embodiment, based on the priority allocation result, independent queues are allocated to the data flows with different priorities, the capacity of the queues is adjusted through an adaptive scheduling mechanism, and a time slice polling algorithm is adopted to perform priority polling, so as to perform data transmission efficiency and queue management optimization, and the specific process is as follows:
according to the assigned data priority, queue management and scheduling control are carried out on the data;
independent queues are distributed for the data flows with different priorities, and each queue is initialized according to the priority weight of the data;
the method comprises the steps of adjusting the capacity of a queue based on an adaptive scheduling mechanism, monitoring each queue in real time after the queue is initialized, analyzing the occupation condition and the data flow state of the queue, dynamically adjusting the capacity and the transmission sequence of each priority queue according to the change of network bandwidth and data flow through the adaptive scheduling mechanism, and automatically expanding the capacity of the high priority queue or limiting the transmission frequency of low priority data flow when the occupation rate of the high priority queue exceeds a preset threshold;
Performing queue priority adjustment based on delay perception, introducing a queue priority adjustment mechanism based on delay perception, judging whether the queue priority needs to be adjusted by monitoring the transmission delay of each data stream in real time, and automatically lifting the priority of the queue when the transmission delay of the queue exceeds a preset delay threshold;
introducing a bandwidth optimization and queue cleaning mechanism, and cleaning low-priority data which are not processed for a long time through a bandwidth optimization algorithm;
After the self-adaptive queue management and the priority polling scheduling are completed, transmission reliability enhancement, a data packet retransmission mechanism, bandwidth optimization, flow regulation and control, congestion control and fault recovery processing are carried out.
In a preferred embodiment, after the adaptive queue management and the priority polling scheduling are completed, the following specific processes are performed, namely, transmission reliability enhancement, a data packet retransmission mechanism, bandwidth optimization and flow regulation, congestion control and fault recovery processing:
Using a data packet confirmation mechanism, after each transmission, a receiving end sends confirmation information to a transmitting end, and confirms that the data is successfully received, if the receiving end does not receive a data packet within a specified time or the data packet is damaged, starting a data packet retransmission mechanism, and starting a redundancy control mechanism when the network is congested;
If the retransmission of the data packet is unsuccessful, a fault recovery mechanism is started, the transmission of the data packet is abandoned according to the set timeout time, the related information is recorded, and the priority data which is not transmitted is continuously processed.
The transmission control system for the terminal data of the Internet of things is used for realizing the transmission control method for the terminal data of the Internet of things, and comprises the following steps:
The sensor deployment module is used for determining the application scene of the terminal of the Internet of things according to the types, the distribution and the physical characteristics of the acquired data of the sensors, making the evaluation of the target and the transmission requirement of the acquired data and determining the deployment positions and the functions of various sensors;
The data transmission standardization module is used for setting sampling frequency and a data acquisition window according to the type and the data change rate of the sensor, carrying out data acquisition and transmission scheduling by adopting an asynchronous or synchronous acquisition strategy, carrying out noise filtering and abnormal value detection on the acquired data, and carrying out standardization processing on different types of data;
The priority quantization module is used for quantizing the data priority according to the entropy shift index, identifying important event change based on a dynamic threshold clustering algorithm, dynamically adjusting bandwidth resources and determining data transmission priority;
And the transmission control module is used for distributing independent queues to the data streams with different priorities based on the priority distribution result, adjusting the queue capacity through the self-adaptive scheduling mechanism, carrying out priority polling by adopting a time slice polling algorithm, and carrying out data transmission efficiency and queue management optimization.
The invention has the technical effects and advantages that:
according to the invention, the type, distribution and physical characteristics of the acquired data of the sensor are determined through the application scene of the terminal of the Internet of things, the target and transmission demand assessment of the acquired data are formulated, the sampling frequency and the acquisition window are set, the asynchronous or synchronous strategy is adopted to realize efficient data acquisition and scheduling, and meanwhile, noise filtering, outlier detection and standardization processing are carried out on the data, so that the integrity and accuracy of the data are ensured, the efficient acquisition of the data of the sensors of multiple types is ensured, the data requirements under different application scenes are adapted, and the data management and transmission performance of the terminal equipment of the Internet of things are optimized;
The method has the advantages that the data priority is quantized through the entropy shift index, the change of important events is identified by combining a dynamic threshold clustering algorithm, bandwidth resources are dynamically adjusted, timely transmission of high-priority data is guaranteed, queue management and scheduling are conducted on data streams with different priorities through an adaptive scheduling mechanism and a time slice polling algorithm, data transmission efficiency is optimized, queue blocking is avoided, and data transmission reliability and overall performance under a complex network environment are improved.
Drawings
Fig. 1 is a flowchart of a method for controlling transmission of terminal data of the internet of things according to the present invention.
Fig. 2 is a schematic structural diagram of a transmission control system for terminal data of the internet of things according to the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Embodiment 1 as shown in fig. 1, a transmission control method of terminal data of the internet of things comprises the following steps:
Determining an application scene of the terminal of the Internet of things according to the types, the distribution and the physical characteristics of the acquired data of the sensors, formulating a target for acquiring the data and an evaluation for transmitting the data, and determining the deployment positions and functions of various sensors;
Setting sampling frequency and a data acquisition window according to the type of a sensor and the data change rate, carrying out data acquisition and transmission scheduling by adopting an asynchronous or synchronous acquisition strategy, carrying out noise filtering and abnormal value detection on acquired data, and carrying out standardization processing on different types of data;
Quantifying the data priority according to the entropy shift index, identifying important event change based on a dynamic threshold clustering algorithm, dynamically adjusting bandwidth resources, and determining the data transmission priority;
And allocating independent queues to the data streams with different priorities based on the priority allocation result, adjusting the queue capacity through an adaptive scheduling mechanism, and carrying out priority polling by adopting a time slice polling algorithm to carry out data transmission efficiency and queue management optimization.
Based on the priority distribution of dynamic event identification, determining a data acquisition scene of an Internet of things terminal, ensuring the accuracy and adaptability of a data acquisition step, wherein the scene determination is the basis for data acquisition, the specific environment of the application of the Internet of things terminal and the core target of data acquisition are required to be clarified, the step requires comprehensive evaluation of the physical environment, equipment arrangement and data transmission requirements of the acquisition scene, and the determination of the scene comprises the following key parts:
The method comprises the steps of determining the type and distribution of a sensor network, namely determining the distribution density and the position of sensors, determining the type (such as temperature, humidity, vibration, pressure, gas concentration and the like) of the sensors used by terminal equipment, and analyzing the deployment condition of the sensors in an Internet of things terminal;
determining the type and the characteristics of acquired data, namely, the type (such as a numerical type, a discrete type or a symbolic type) of the data which are clearly acquired, determining the actual physical meaning and the influence range of various data according to the application scene of terminal equipment, wherein the data characteristics may have heterogeneity, therefore, the comparability and the relativity of the data need to be determined in advance, the environmental characteristics are determined, the dynamic characteristics (such as the environmental temperature, the humidity and the equipment use frequency) of the scene and the potential influence of abnormal conditions (such as equipment faults or emergency states) in the scene on the data are analyzed, and the scene of the terminal of the Internet of things is determined according to the above;
In the application scene of the terminal of the Internet of things, the establishment of a target for collecting data and the evaluation of transmission requirements are important steps for ensuring the efficient operation of a system, and the method comprises the following consideration of the following aspects:
The data acquisition targets are determined, so that specific application requirements are met, the content of the acquired target data and core data is clear, and the performance index, response time and storage requirements of the whole data acquisition are defined;
It should be noted that, the core data content refers to data that is critical to realizing specific application requirements in the application of the terminal of the internet of things. These data directly reflect key parameters or states, enabling efficient support for system decisions, analysis or control. For example, in an environmental monitoring system, the core data content may include temperature, humidity, gas concentration, etc., in intelligent manufacturing, the core data may be vibration, pressure or operation state of the device, and the core data content generally has higher acquisition frequency and priority, so as to ensure that key information in an application scene is captured timely and accurately.
The transmission requirement assessment, which is to ensure that the data can be accurately and real-timely transmitted to a data center for processing or decision making, and the transmission requirement assessment can consider the data transmission frequency, bandwidth, network conditions, data flow and data instantaneity;
On the basis of determining the type and the distribution of the sensors, the physical deployment position and the function of the sensors are required to be determined according to specific application scenes so as to ensure the accuracy and coverage of data acquisition, and the method specifically comprises the following steps:
The determination of the deployment position is determined according to the monitoring target of the sensor, the physical characteristics of the application scene and the functional requirements of the sensor, and the sensor is required to be ensured to be arranged at a position capable of obtaining accurate data, data interference and repeated acquisition are avoided;
for example, in intelligent farmlands, soil moisture sensors need to be deployed in the root system area of crops, the depth is determined according to the growth requirements of the crops, and the soil moisture sensors are usually arranged in shallow layers, middle layers and deep layers so as to comprehensively monitor the moisture changes at different depths;
For example, in an intelligent farmland environment monitoring system, in a farmland scene, sensors are arranged in different areas of the farmland, including a plurality of layers (shallow layers, middle layers and deep layers) in soil and different heights in air, the distribution density of the sensors is adjusted according to the difference of the area and the crop types of the farmland, in a high-density area (such as a root system dense area), the distribution interval of soil humidity sensors is 5 meters, and in a low-density area (such as the vicinity of irrigation facilities), the interval of the sensors is 10 meters, and in addition, the air temperature humidity sensors are arranged at different heights of the farmland according to the interval of 20 meters, so as to monitor the whole crop growth environment;
The sensor collects the data type and the characteristics, the soil humidity sensor collects the soil moisture content, the data is a continuous analog signal, and the collection frequency is 1 time per hour due to the slower change of the soil humidity;
Environmental characteristic analysis, seasonal variation, namely that environmental temperature and humidity and soil moisture content can obviously fluctuate along with the seasonal variation, the sensor system needs to adjust the acquisition frequency in different seasons, for example, the acquisition frequency of a soil humidity sensor in dry seasons needs to be increased so as to monitor moisture variation in time;
in the intelligent farmland environment monitoring of the terminal of the Internet of things, the distribution density and the position of the sensors are determined according to the crop growth requirements and the topography characteristics of the farmland, the data types and the acquisition frequencies of different types of sensors are dynamically adjusted according to the environment change and the abnormal conditions, the accuracy and the instantaneity of data acquisition are ensured, and thus, accurate agricultural management can be realized in one scene, including crop growth state monitoring, irrigation optimization and the prevention of the influence of environmental change on crops.
The scene determination of the terminal of the Internet of things is beneficial to defining core environment parameters of collected data, and the importance of various data in practical application is clarified through the combing of sensor types and distribution conditions, so that a background basis is provided for subsequent priority distribution.
After the scene is determined, entering a preparation stage of actual data acquisition, running the sensors according to a set rule in the actual acquisition process, and designing data acquisition frequency and scheduling mechanisms according to the characteristics of different sensors, wherein the method specifically comprises the following steps of:
Determining the data acquisition frequency of each sensor according to the type of the sensor, the data change rate and the bandwidth resource, wherein important sensors (such as vibration sensors) can acquire at high frequency, but non-critical sensors (such as temperature and humidity) can moderately reduce the frequency;
in a multi-sensor environment, asynchronous collection and synchronous collection strategies are selected, and a synchronous collection mechanism is adopted for data (such as temperature and pressure) with strong time sequence relevance, and asynchronous collection can be adopted for independent data sources (such as gas concentration), so that bandwidth burden is reduced;
By determining the acquisition frequency, the time sequence and the data window length, the accuracy and the high efficiency of the sensor data acquisition are ensured, the reasonable design of a scheduling mechanism lays a foundation for smooth transmission of data streams, and the sensor acquisition data scheduling under the low bandwidth condition is ensured to meet the requirements.
After data acquisition, preprocessing the acquired original data to ensure the integrity and accuracy of the data, wherein the preprocessing process comprises the steps of detecting and processing noise data and abnormal values, wherein the preprocessing process comprises the steps of detecting high-frequency noise signals in sensor data by using a noise filtering method based on wavelet transformation and filtering the high-frequency noise signals to ensure that the data quality is not interfered by environmental noise, detecting abnormal values in a high-dimensional data space by using a distribution density estimation technology, marking and smoothing data points with overlarge abnormal values by using an interpolation algorithm, ensuring that the overall data trend is not influenced, normalizing the acquired multi-dimensional data, ensuring that different types of data can be analyzed in the same dimension, and using a normalization method based on Laplace regularization to ensure that each sensor data has the same measurement standard.
On the basis of preprocessing data, the collected multidimensional data stream is required to be subjected to structural modeling to form a matrix expression form which can be used for subsequent high-dimensional entropy measurement, and the original data stream is mapped into a high-dimensional space through feature extraction and dimension reduction to form a dynamic time sequence data matrix, wherein the method comprises the following specific steps of:
Feature extraction is performed on the acquired data by using Support Vector Regression (SVR) based on a kernel function, main features (such as temperature trend, pressure fluctuation and the like) are extracted from each data stream, and feature vectors are formed through nonlinear mapping;
Mapping all sensor data to a high-dimensional space, namely setting acquisition values of different sensors at different times, wherein the data has dynamic characteristics, namely a multi-dimensional time sequence data stream is formed;
from the sensor data of the terminal from the internet, a data vector is constructed at each time t Wherein, the method comprises the steps of, wherein,Representing the acquisition value of the ith sensor at time t, n represents the total number of sensors, and calculating the high-dimensional data entropy of each time pointBy the following formula: Wherein, the method comprises the steps of, wherein, Is based on a probability distribution of the acquired data,The method is an adjusting factor used for controlling the data entropy fluctuation of the extreme case, the uncertainty of the data collected by the terminal of the Internet of things can be measured by calculating the data entropy value at each time point, and the entropy value formula introduces the adjusting factor on the basis of the traditional entropy so as to avoid the excessive fluctuation of the data entropy value under the extreme probability distribution;
And carrying out dynamic tracking and increment judgment of the entropy offset, re-calculating the entropy value of the high-dimensional data at time t+1, and calculating the entropy difference between the front and rear time points t and t+1 through the following entropy offset formula, wherein the entropy difference represents the variation amplitude of the entropy value of the data between the adjacent time points, and the formula is as follows: Wherein, the method comprises the steps of, wherein, Is a small value that prevents the occurrence of a zero probability,The probability distribution of i sensors collecting data at times t and t+1 are shown, respectively. By the formula, dynamically tracking entropy offset in the high-dimensional data stream, and evaluating uncertainty change of the data stream between adjacent time points;
The method comprises the steps of comparing the data entropy of adjacent time points, dynamically detecting the uncertainty increment of a data stream, wherein the uncertainty increment represents the uncertainty change of the data in time, and the larger the entropy offset is, the larger the entropy change of the data stream in time is, and the entropy offset represents the entropy change trend of the adjacent time points;
Based on the entropy shift calculated previously, important event changes are identified by a dynamic threshold clustering algorithm and it is determined whether these changes require adjustment of data priority, dynamic threshold The settings of (2) are as follows: Wherein, the method comprises the steps of, wherein, Is an adjustment coefficient; Is a smoothing factor, Z is the time window length, The method is characterized in that an adjustment index, k is the length of an observation period, n is the total number of time windows, the dynamic threshold is used for dynamically adjusting the setting of a clustering center point, the system is ensured to be capable of identifying important change events in data in real time, the transmission control system is ensured to be capable of flexibly coping with data changes in different time periods, particularly fast-changing data scenes, through the dynamic adjustment of the clustering threshold, abnormal events can be effectively captured through the real-time threshold adjustment, and the problem of misjudgment possibly caused by a static threshold is avoided;
The adjustment coefficient is The method is used for integrally scaling and adjusting the dynamic threshold value, controlling the sensitivity and response speed to the entropy offset, influencing the size and flexibility of the threshold value, and determining the overall size of the threshold value by the size of the adjustment coefficient whenWhen the threshold increases, the threshold correspondingly increases, the sensitivity to entropy shift decreases, priority adjustment is triggered only when a significant event occurs, and whenWhen the system threshold is reduced, the system threshold is reduced and is more sensitive to small fluctuation, and the change in the data can be recognized more quicklyFor avoiding zero denominator or excessive fluctuation of entropy offset in the calculation process, the smoothing factor ensures that even if the entropy change approaches zero or extreme data condition occurs, the system can still calculate the threshold value normally without calculation error caused by zero denominator, and adjusts the indexThe sensitivity to entropy offset changes and the non-linear magnitude of change and responsiveness to historical entropy changes used to control the dynamic threshold, and determine how the system accumulates and weights the entropy changes of the historical data.
For the large entropy offset data points that have been identified, the importance of the data is further quantified by an entropy offset index (EDI) to assign priority, the entropy offset index representing the degree of fluctuation in the data stream, the entropy offset index calculation formula being as follows: Wherein, the method comprises the steps of, wherein, Representing the entropy offset of the i-th sensor,Is an index of the adjustment parameters of the device,The method is characterized in that the method is a weight coefficient, the entropy offset index is used for determining which data points need to be transmitted preferentially, ensuring that the data points with larger entropy offset obtain higher priority, the entropy offset index is used for evaluating the importance of each data point in the whole data stream by combining the entropy offset and a dynamic threshold value, and ensuring that priority distribution under different weights has smoothness and flexibility by combining index adjustment and logarithmic functions, and determining a transmission strategy under a low bandwidth condition.
According to the calculated entropy shift index, the appropriate bandwidth resources are allocated to each data point through a bandwidth allocation formula: Wherein, the method comprises the steps of, wherein, For the ith sensor entropy shift index,J represents the j-th sensor entropy offset index,For the bandwidth of the ith data point, n is the number of sensors,In order for the total bandwidth to be available,By the formula, the priority of the key data stream can be ensured to obtain the transmission bandwidth, and the low priority data is properly delayed to be transmitted, so that the dynamic resource allocation based on priority is realized, the high priority data is ensured to obtain enough bandwidth, and the low priority data cannot block transmission. The use of the amplification factors ensures the flexibility and the dynamic property of bandwidth allocation and meets the data transmission requirements in different scenes;
the high-dimensional data entropy deviation measurement method based on dynamic threshold clustering can reasonably allocate priorities to terminal data of the Internet of things, and dynamic data transmission scheduling is achieved through bandwidth management.
After the step of distributing the high-dimensional data entropy deviation measure priority based on dynamic threshold clustering is completed, reasonable queue management and scheduling control are carried out on the data according to the distributed priority, the main targets of self-adaptive queue management and priority polling scheduling are to ensure that high-priority data are preferentially processed, and meanwhile, low-priority data can be transmitted within a certain time, so that long-time delay or loss of the data is avoided, and the method comprises the following specific steps of:
based on the priority calculated by the entropy shift index (EDI), firstly, independent queues are allocated for data flows with different priorities, each queue is initialized according to the priority weight of the data, the initial queue length and the data buffer space are set, and the queue allocation formula is as follows: Wherein, the method comprises the steps of, wherein, For the capacity of the ith priority queue,For the capacity of the j-th priority queue, n is the total capacity number of the priority queue,For the total capacity of the queue of the system,Dynamically distributing queue space according to weights of different priorities to ensure that high-priority data obtain more buffer resources;
It should be noted that, the data point refers to a single data value collected at a specific time by a sensor of the terminal of the internet of things. For example, a temperature value acquired by a temperature sensor at a certain point in time, a pressure value recorded by a pressure sensor, etc., each sensor may continuously generate a plurality of data points, the data points represent different time periods, different sensor positions or different types of measured values, the importance and the change of each data point are analyzed through entropy shift indexes, and the system allocates bandwidth resources according to the priority of the data points, so that the important data points can be transmitted preferentially.
By dynamically allocating the queue capacity, it is ensured that high priority data obtains sufficient queue space, and queue blocking caused by data backlog is avoided. Meanwhile, the low-priority data can be processed within a certain range, so that excessive delay is avoided;
Queue monitoring and adjustment based on self-adaptive scheduling, after the queues are initialized, each queue is monitored in real time, the occupation condition and the data flow state of the queues are analyzed, the capacity and the transmission sequence of each priority queue are dynamically adjusted according to the change of network bandwidth and data flow through a self-adaptive scheduling mechanism, and when the occupation rate of a high priority queue exceeds a preset threshold value When the system automatically expands the capacity of the high priority queue or limits the transmission frequency of the low priority data stream. The queue monitoring algorithm is as follows: Wherein, the method comprises the steps of, wherein, Is a queue capacity adjustment coefficient for controlling the queue capacity, n is the total capacity number of the priority queue,Is a smoothing factor which is used to smooth the image,Is an adjustment index used for controlling the sensitivity of dynamic expansion of the queue;
It should be noted that, the smoothing factor is generally used to prevent the denominator from being zero or avoid extreme data fluctuation from having an excessive influence on the calculation result. To ensure robustness of the system, the smoothing factor is typically set to a very small positive number, and the specific value of the smoothing factor is determined based on the magnitude of the data or physical constraints of the system, assuming that the temperature data collected by the sensor fluctuates by 0.01 to 1 degree, it is observed that in extreme cases some sensor values are near 0 through the normalization process of the data. In order to avoid zero denominator in the entropy shift calculation, the smoothing factor may be set to 0.001, ensuring the calculability of the formula in extreme cases, while not significantly affecting the overall calculation of the data;
The queue capacity regulating factor determines the queue expanding or shrinking sensitivity, the value of the queue capacity regulating factor needs to be dynamically regulated according to the real-time load condition of the system and the change of network bandwidth, the determination of the queue capacity regulating factor can be estimated through the bandwidth use condition in historical data and the queue backlog degree, or the queue capacity regulating factor is regulated through a self-adaptive mechanism based on feedback;
The adjustment index controls the sensitivity and magnitude of the adaptive adjustment, affecting the degree of change in queue capacity and priority polling schedule. The adjustment index is usually determined through simulation test and model optimization, so as to avoid excessive fluctuation while ensuring the sensitivity of the system, and the adjustment index is adjusted through simulating the change of data flow so as to observe the response condition of the system under different loads. When the adjustment index is smaller, the system is more sensitive to flow change reaction, but fluctuation is larger, the adjustment index can be adjusted along with the change of the load of the system, the index is increased under light load to improve the stability of the system, and the index is reduced under heavy load to improve the sensitivity;
For example, in an internet of things terminal system with frequent data fluctuation, setting the adjustment index to 2 may be a reasonable initial value, which means that when the queue occupancy exceeds the threshold, the adjustment speed of the queue capacity is relatively gentle, so that resource waste caused by excessive expansion is avoided, and the low-priority queue is not completely ignored. During actual operation, if the data flow fluctuations are excessive, the adjustment index may be gradually reduced to 1.2 to increase the system sensitivity.
On the basis of queue management, the system adopts a priority polling scheduling mechanism to periodically check the data flow state in each priority queue, and performs scheduling through a time slice polling algorithm, namely, high-priority data occupy more transmission resources in scheduling, and low-priority data can also periodically obtain processing opportunities;
and (3) performing queue priority adjustment based on delay perception, namely preventing high-priority data from causing transmission delay due to queue blocking, introducing a queue priority adjustment mechanism based on delay perception, and judging whether the queue priority needs to be adjusted by monitoring the transmission delay of each data stream in real time. When a certain queue is delayed in transmission When the preset delay threshold is exceeded, the priority of the queue is automatically raised, more bandwidth resources and queue space are allocated, and the delay monitoring algorithm is as follows: Wherein, the method comprises the steps of, wherein, Indicating the round trip time of the ith queue at time t,Is the reference round-trip time and,Indicating the monitoring time window length of the ith queue or data stream,Determining a time range for observing and calculating the delay of the queue or the data stream by the system, and judging whether the priority or bandwidth resource of the queue needs to be adjusted by counting and analyzing the delay change in the time period; The delay smoothing factor is determined according to actual demands, the priority of the high-delay queue can be dynamically improved through a delay sensing mechanism, the influence on the overall transmission efficiency caused by delay accumulation of key data is prevented, and the delay sensing algorithm ensures that the key data in the queue can be processed in time, particularly under the condition of high network load;
The method comprises the steps of avoiding long-term occupation of system resources by a low-priority queue, introducing a bandwidth optimization and queue cleaning mechanism, cleaning low-priority data which are not processed for a long time through a bandwidth optimization algorithm, releasing the system resources, and cleaning the queue according to entropy offset indexes and delay values of the data, wherein the cleaning algorithm is as follows: When (when) When the value is too large, part of low-priority data in the queue is cleaned preferentially, and resources are released, namely, the low-priority data occupying too much resources can be cleaned reasonably through a bandwidth optimization and queue cleaning mechanism, so that the high-priority data can be ensured to acquire enough transmission resources and queue space;
After the self-adaptive queue management and the priority polling scheduling are completed, the transmission control of the terminal data of the Internet of things needs to be further optimized so as to ensure the reliability, the instantaneity and the efficiency of data transmission, and the transmission reliability enhancement, the data packet retransmission mechanism, the bandwidth optimization, the flow regulation and control, the congestion control and the fault recovery processing can be carried out subsequently;
after the priority data transmission is finished, the system strengthens the reliability of the transmission, particularly in the network fluctuation environment, in order to ensure that key data can be completely and correctly transmitted, the system uses a data packet acknowledgement mechanism, a receiving end sends acknowledgement information (ACK) to a transmitting end after each transmission, and confirms that the data is successfully received, and if the receiving end does not receive the data packet within a specified time or the data packet is damaged, the system triggers a retransmission mechanism to ensure that the data cannot be lost;
When the data packet loss is detected or the acknowledgement is not received after the time-out, a data packet retransmission mechanism is started, namely, under certain conditions (such as high network congestion or packet loss rate), a redundancy control mechanism is started, for example, the transmission success rate of data is improved by adding redundancy information or a forward error correction technology, more retransmission opportunities and redundancy protection are provided for key data, the key data can be ensured to be successfully transmitted under bad network conditions, and in the transmission process, bandwidth is a key resource, particularly in the concurrent transmission among terminal equipment of the Internet of things, and bandwidth allocation is dynamically optimized according to the priority of different data and the state of a queue.
The bandwidth of the high-priority data can be preferentially allocated when the high-priority data is transmitted, the low-priority data is transmitted on the premise of not influencing the high-priority data, bandwidth optimization can be dynamically adjusted according to real-time network conditions and loads, so that the key data is ensured to obtain enough bandwidth resources, transmission delay caused by bandwidth occupation of the low-priority data is avoided, the bandwidth optimization ensures that the key data can still be preferentially transmitted when the network resources are limited, and bandwidth allocation is dynamically adjusted, so that the low-priority data can be transmitted under the condition of not influencing the key data, and the existing network resources are maximally utilized;
If the data packet cannot be successfully transmitted after retransmission for a plurality of times, the system starts a fault recovery mechanism, the mechanism gives up the transmission of the data packet after a certain timeout time and records related information so as to facilitate subsequent fault diagnosis and processing, and simultaneously, continues to process other priority data which are not transmitted, so that the whole system is prevented from being stagnated due to the transmission failure of individual data packets, and in special cases, network faults can be avoided by trying other transmission paths or methods.
It should be noted that, the threshold values involved in the embodiments may be determined according to specific situations and requirements, and are generally adjusted and optimized according to factors such as historical data, real-time data, and the like.
According to the invention, the type, distribution and physical characteristics of the acquired data of the sensor are determined through the application scene of the terminal of the Internet of things, the target and transmission demand assessment of the acquired data are formulated, the sampling frequency and the acquisition window are set, the asynchronous or synchronous strategy is adopted to realize efficient data acquisition and scheduling, and meanwhile, noise filtering, outlier detection and standardization processing are carried out on the data, so that the integrity and accuracy of the data are ensured, the efficient acquisition of the data of the sensors of multiple types is ensured, the data requirements under different application scenes are adapted, and the data management and transmission performance of the terminal equipment of the Internet of things are optimized;
The method has the advantages that the data priority is quantized through the entropy shift index, the change of important events is identified by combining a dynamic threshold clustering algorithm, bandwidth resources are dynamically adjusted, timely transmission of high-priority data is guaranteed, queue management and scheduling are conducted on data streams with different priorities through an adaptive scheduling mechanism and a time slice polling algorithm, data transmission efficiency is optimized, queue blocking is avoided, and data transmission reliability and overall performance under a complex network environment are improved.
Embodiment 2. This embodiment is a system embodiment of embodiment 1, configured to implement a method for controlling transmission of terminal data of an internet of things described in embodiment 1, as shown in fig. 2, and specifically includes:
The sensor deployment module determines the application scene of the terminal of the Internet of things according to the types, the distribution and the physical characteristics of the acquired data of the sensors, and formulates the evaluation of the target and the transmission requirement of the acquired data so as to determine the deployment positions and the functions of various sensors;
The data transmission standardization module is used for setting sampling frequency and a data acquisition window according to the type and the data change rate of the sensor, carrying out data acquisition and transmission scheduling by adopting an asynchronous or synchronous acquisition strategy, carrying out noise filtering and abnormal value detection on the acquired data, and carrying out standardization processing on different types of data;
The priority quantization module is used for quantizing the data priority according to the entropy shift index, identifying important event change based on a dynamic threshold clustering algorithm, dynamically adjusting bandwidth resources and determining data transmission priority;
And the transmission control module is used for distributing independent queues to the data streams with different priorities based on the priority distribution result, adjusting the queue capacity through the self-adaptive scheduling mechanism, carrying out priority polling by adopting a time slice polling algorithm, and carrying out data transmission efficiency and queue management optimization.
The above formulas are all formulas for removing dimensions and taking numerical calculation, and specific dimensions can be removed by adopting various means such as standardization, and the like, which are not described in detail herein, wherein the formulas are formulas for acquiring a large amount of data and performing software simulation to obtain the latest real situation, and preset parameters in the formulas are set by those skilled in the art according to the actual situation.
The above embodiments may be implemented in whole or in part by software, hardware, firmware, or any other combination. When implemented in software, the above-described embodiments may be implemented in whole or in part in the form of a computer program product. The computer program product comprises one or more computer instructions or computer programs. When the computer instructions or computer program are loaded or executed on a computer, the processes or functions described in accordance with embodiments of the present application are produced in whole or in part. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from one website site, computer, server, or data center to another website site, computer, server, or data center by wired or wireless means (e.g., infrared, wireless, microwave, etc.). The computer readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains one or more sets of available media. The usable medium may be a magnetic medium (e.g., floppy disk, ATA hard disk, tape), an optical medium (e.g., DVD), or a semiconductor medium. The semiconductor medium may be a solid state ATA hard disk.
It should be understood that, in various embodiments of the present application, the sequence numbers of the foregoing processes do not mean the order of execution, and the order of execution of the processes should be determined by the functions and internal logic thereof, and should not constitute any limitation on the implementation process of the embodiments of the present application.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other manners. For example, the apparatus embodiments described above are merely illustrative, e.g., the division of the units is merely a logical function division, and there may be additional divisions when actually implemented, e.g., multiple units or components may be combined or integrated into another system, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical or other forms.
The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed over a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in the embodiments of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit.
The foregoing is merely illustrative of the present application, and the present application is not limited thereto, and any person skilled in the art will readily recognize that variations or substitutions are within the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims (6)

1.一种物联网终端数据的传输控制方法,其特征在于,包括如下步骤:1. A method for controlling the transmission of Internet of Things terminal data, characterized in that it comprises the following steps: 根据传感器的类型、分布和采集数据的物理特性,确定物联网终端应用场景,并制定采集数据的目标和传输需求评估,并明确各类传感器的部署位置与功能;Determine IoT terminal application scenarios based on sensor types, distribution, and physical characteristics of collected data, formulate data collection goals and transmission demand assessments, and clarify the deployment locations and functions of various types of sensors; 根据传感器的类型及数据变化速率,设置采样频率与数据采集窗口,并采用异步或同步采集策略进行数据采集与传输调度,对采集到的数据进行噪声滤波和异常值检测,并对不同类型数据进行规范化处理;According to the type of sensor and the data change rate, set the sampling frequency and data acquisition window, and use asynchronous or synchronous acquisition strategies for data acquisition and transmission scheduling, perform noise filtering and outlier detection on the collected data, and normalize different types of data; 根据熵偏移指数对数据优先级进行量化,基于动态阈值聚类算法识别重要事件变化,并动态调整带宽资源,确定数据传输优先级;Quantify data priority based on entropy shift index, identify important event changes based on dynamic threshold clustering algorithm, and dynamically adjust bandwidth resources to determine data transmission priority; 基于优先级分配结果,对不同优先级的数据流分配独立队列,通过自适应调度机制调整队列容量,并采用时间片轮询算法进行优先级轮询,进行数据传输效率和队列管理优化;Based on the priority allocation results, independent queues are allocated to data flows of different priorities. The queue capacity is adjusted through an adaptive scheduling mechanism, and a time slice polling algorithm is used for priority polling to optimize data transmission efficiency and queue management. 根据熵偏移指数对数据优先级进行量化,基于动态阈值聚类算法识别重要事件变化,并动态调整带宽资源,确定数据传输优先级,具体过程如下:The data priority is quantified according to the entropy shift index, and the changes of important events are identified based on the dynamic threshold clustering algorithm. The bandwidth resources are dynamically adjusted to determine the data transmission priority. The specific process is as follows: 使用基于核函数的支持向量回归,对采集数据进行特征提取,从每个数据流中提取主要特征,并通过非线性映射将主要特征形成特征向量;Use kernel-based support vector regression to extract features from the collected data, extract the main features from each data stream, and form feature vectors from the main features through nonlinear mapping; 构建每个时间点t上的数据向量,其中, 代表第i个传感器在时间t上的采集值,n表示传感器总数量,计算每个时间点的高维数据熵,公式为:,其中,是基于采集数据的概率分布,是调节因子,控制极端情况的数据熵波动;Construct the data vector at each time point t ,in, represents the collected value of the i-th sensor at time t, n represents the total number of sensors, and the high-dimensional data entropy is calculated at each time point , the formula is: ,in, It is a probability distribution based on the collected data. It is a regulating factor that controls the fluctuation of data entropy in extreme cases; 从物联网终端设备的传感器数据中,构建每个时间点上的数据向量,并计算每个时间点的高维数据熵;From the sensor data of IoT terminal devices, construct the data vector at each time point and calculate the high-dimensional data entropy at each time point; 进行熵偏移量的动态跟踪与增量判定,计算数据的熵值,并通过熵偏移量公式,计算出前后时间点之间的熵差异,计算出前后时间点t与t+1之间的熵差异,熵差异表示相邻时间点之间数据的熵值变化幅度,公式如下:,其中,是防止出现零概率的小值,分别表示i个传感器在时间t与t+1时间上采集数据的概率分布,熵差异表示相邻时间点之间数据的熵值变化幅度;Dynamically track and increment the entropy offset, calculate the entropy value of the data, and use the entropy offset formula to calculate the entropy difference between the previous and next time points, and the entropy difference between the previous and next time points t and t+1. The entropy difference represents the change in the entropy value of the data between adjacent time points. The formula is as follows: ,in, is to prevent small values of zero probability from occurring, , They represent the probability distribution of data collected by the i sensors at time t and time t+1 respectively, and the entropy difference represents the change amplitude of the entropy value of the data between adjacent time points; 对相邻时间点的数据熵进行比对,动态检测出数据流的不确定性增量,不确定性增量表示数据在时间内的不确定性变化;Compare the data entropy of adjacent time points to dynamically detect the uncertainty increment of the data stream. The uncertainty increment indicates the uncertainty change of the data over time. 基于计算出的熵偏移量,通过动态阈值聚类算法识别重要的事件变化,并确定变化是否需要调整数据优先级,熵偏移量表示相邻时间点的熵值变化趋势;Based on the calculated entropy offset, a dynamic threshold clustering algorithm is used to identify important event changes and determine whether the change requires adjustment of data priority. The entropy offset represents the trend of entropy value changes at adjacent time points. 对已经识别出的大熵偏移数据点,通过熵偏移指数量化数据的重要性,并分配优先级,熵偏移指数表示数据流中的波动程度,熵偏移指数计算公式如下:,其中, 表示第 i 个传感器的熵偏移量,是指数调节参数,是权重系数;For the identified large entropy shift data points, the importance of the data is quantified and a priority is assigned using the entropy shift index. The entropy shift index represents the degree of fluctuation in the data stream. The entropy shift index calculation formula is as follows: ,in, represents the entropy offset of the ith sensor, is the exponential tuning parameter, is the weight coefficient; 根据计算的熵偏移指数对数据点分配带宽资源,带宽分配公式为:,其中,为第 i 个传感器熵偏移指数,j表示第j个传感器熵偏移指数,为第i个数据点的带宽,n为传感器数,为可用总带宽,为放大因子。Bandwidth resources are allocated to data points according to the calculated entropy shift index. The bandwidth allocation formula is: ,in, is the entropy offset index of the ith sensor, j represents the entropy offset index of the jth sensor, is the bandwidth of the ith data point, n is the number of sensors, is the total available bandwidth, is the magnification factor. 2.根据权利要求1所述的一种物联网终端数据的传输控制方法,其特征在于:根据传感器的类型、分布和采集数据的物理特性,确定物联网终端应用场景,并制定采集数据的目标和传输需求评估,明确各类传感器的部署位置与功能,具体过程如下:2. According to claim 1, a method for controlling the transmission of IoT terminal data is characterized in that: according to the type, distribution and physical characteristics of the sensor data collected, the application scenario of the IoT terminal is determined, the target of data collection and transmission demand assessment are formulated, and the deployment location and function of various sensors are clarified. The specific process is as follows: 确定传感器的分布密度和位置,明确终端设备使用的传感器类型,传感器类型包括温度、湿度、振动、压力、气体浓度传感器,并分析所有传感器在物联网终端中的部署情况;Determine the distribution density and location of sensors, clarify the types of sensors used by terminal devices, including temperature, humidity, vibration, pressure, and gas concentration sensors, and analyze the deployment of all sensors in IoT terminals; 确定采集数据类型和特性,数据类型包括数值型、离散型以及符号型;Determine the data type and characteristics of the collection, including numeric, discrete, and symbolic data types; 确定环境特性,环境特性包括场景的动态性与场景内的异常情况,场景的动态性包括环境温度、湿度、设备使用频率,场景内的异常情况包括设备故障或紧急状态下对数据产生的影响情况;Determine the environmental characteristics, which include the dynamics of the scene and the abnormalities within the scene. The dynamics of the scene include the ambient temperature, humidity, and frequency of equipment use. The abnormalities within the scene include the impact of equipment failure or emergency conditions on data. 结合传感器的分布密度和位置、传感器采集数据类型和特性、环境特性得到物联网终端应用场景;The application scenarios of IoT terminals are obtained by combining the distribution density and location of sensors, the data types and characteristics collected by sensors, and environmental characteristics; 确定采集的目标数据和核心数据内容与数据的传输需求。Determine the target data and core data content to be collected and the data transmission requirements. 3.根据权利要求2所述的一种物联网终端数据的传输控制方法,其特征在于:根据传感器的类型及数据变化速率,设置采样频率与数据采集窗口,并采用异步或同步采集策略进行数据采集与传输调度,对采集到的数据进行噪声滤波和异常值检测,并对不同类型数据进行规范化处理,具体过程如下:3. According to claim 2, a method for controlling the transmission of IoT terminal data is characterized in that: according to the type of sensor and the data change rate, the sampling frequency and the data acquisition window are set, and an asynchronous or synchronous acquisition strategy is used for data acquisition and transmission scheduling, noise filtering and abnormal value detection are performed on the collected data, and different types of data are standardized. The specific process is as follows: 根据传感器类型、数据变化速率和带宽资源,确定每个传感器的数据采集频率,并对每个传感器设定数据采集窗口;Determine the data collection frequency of each sensor based on the sensor type, data change rate, and bandwidth resources, and set a data collection window for each sensor; 对于时序关联性数据采用同步采集机制,对于独立数据源采取异步采集;A synchronous collection mechanism is used for time-series correlation data, and an asynchronous collection mechanism is used for independent data sources; 在数据采集后,对采集到的原始数据进行预处理,使用基于小波变换的噪声滤波方法,检测传感器数据中的高频噪声信号并进行滤除;使用分布密度估计技术,检测出在高维数据空间中出现的异常值进行标记,并通过插值算法进行平滑处理;After data collection, the collected raw data is preprocessed, and the high-frequency noise signals in the sensor data are detected and filtered out using the noise filtering method based on wavelet transform; the distribution density estimation technology is used to detect the abnormal values appearing in the high-dimensional data space, mark them, and smooth them through the interpolation algorithm; 使用基于拉普拉斯正则化的归一化方法对数据进行处理。The data were processed using a normalization method based on Laplace regularization. 4.根据权利要求3所述的一种物联网终端数据的传输控制方法,其特征在于:基于优先级分配结果,对不同优先级的数据流分配独立队列,通过自适应调度机制调整队列容量,并采用时间片轮询算法进行优先级轮询,进行数据传输效率和队列管理优化,具体过程如下:4. According to claim 3, a method for controlling the transmission of IoT terminal data is characterized in that: based on the priority allocation result, independent queues are allocated to data streams of different priorities, the queue capacity is adjusted through an adaptive scheduling mechanism, and a time slice polling algorithm is used for priority polling to optimize data transmission efficiency and queue management. The specific process is as follows: 根据分配的数据优先级,对数据进行队列管理与调度控制;Perform queue management and scheduling control on data according to the assigned data priority; 为不同优先级的数据流分配独立的队列,每个队列根据数据的优先级权重进行初始化;Assign independent queues to data flows of different priorities, and initialize each queue according to the priority weight of the data; 基于自适应调度机制进行队列容量调整,在队列初始化后,对每个队列进行实时监控,分析队列的占用情况和数据流动状态,通过自适应调度机制,根据网络带宽和数据流量的变化,动态调整各个优先级队列的容量和传输顺序,当高优先级队列的占用率超过预设阈值时,自动扩展高优先级队列容量,或者限制低优先级数据流的传输频率;The queue capacity is adjusted based on the adaptive scheduling mechanism. After the queue is initialized, each queue is monitored in real time to analyze the queue occupancy and data flow status. The capacity and transmission order of each priority queue are dynamically adjusted according to the changes in network bandwidth and data flow through the adaptive scheduling mechanism. When the occupancy rate of the high-priority queue exceeds the preset threshold, the high-priority queue capacity is automatically expanded, or the transmission frequency of the low-priority data flow is limited. 进行基于延迟感知的队列优先级调整,引入基于延迟感知的队列优先级调整机制,通过实时监控每个数据流的传输延迟,判断是否需要对队列优先级进行调整,当队列的传输延迟超过预设的延迟阈值时,自动提升队列的优先级;Perform queue priority adjustment based on delay perception. Introduce a queue priority adjustment mechanism based on delay perception. By real-time monitoring of the transmission delay of each data flow, determine whether the queue priority needs to be adjusted. When the transmission delay of the queue exceeds the preset delay threshold, the queue priority is automatically increased. 引入带宽优化与队列清理机制,通过带宽优化算法,清理长时间未得到处理的低优先级数据;Introducing bandwidth optimization and queue clearing mechanisms, clearing low-priority data that has not been processed for a long time through bandwidth optimization algorithms; 在完成了自适应队列管理与优先级轮询调度后,进行传输可靠性增强、数据包重传机制、带宽优化与流量调控、拥塞控制以及故障恢复处理。After completing the adaptive queue management and priority polling scheduling, transmission reliability enhancement, packet retransmission mechanism, bandwidth optimization and flow regulation, congestion control and fault recovery processing are carried out. 5.根据权利要求4所述的一种物联网终端数据的传输控制方法,其特征在于:在完成了自适应队列管理与优先级轮询调度后,进行传输可靠性增强、数据包重传机制、带宽优化与流量调控、拥塞控制以及故障恢复处理,具体过程如下:5. According to claim 4, a method for controlling the transmission of IoT terminal data is characterized in that: after completing the adaptive queue management and priority polling scheduling, transmission reliability enhancement, data packet retransmission mechanism, bandwidth optimization and flow regulation, congestion control and fault recovery processing are performed, and the specific process is as follows: 使用数据包确认机制,每次传输后,接收端发送确认信息给发送端,确认数据已成功接收,如果接收端在规定时间内未收到数据包,或者数据包损坏,启动数据包重传机制,在网络拥堵,启用冗余控制机制;Use the data packet confirmation mechanism. After each transmission, the receiving end sends a confirmation message to the sending end to confirm that the data has been successfully received. If the receiving end does not receive the data packet within the specified time, or the data packet is damaged, the data packet retransmission mechanism is started. In case of network congestion, the redundancy control mechanism is enabled. 若数据包重传无法成功,启动故障恢复机制,根据设置超时时间,放弃数据包的传输,并记录信息。If the data packet retransmission fails, the fault recovery mechanism is started, and the data packet transmission is abandoned according to the set timeout period, and the information is recorded. 6.一种物联网终端数据的传输控制系统,用于实现权利要求1-5中任一项所述的一种物联网终端数据的传输控制方法,其特征在于,包括:6. A transmission control system for Internet of Things terminal data, used to implement a transmission control method for Internet of Things terminal data according to any one of claims 1 to 5, characterized in that it comprises: 传感器部署模块,用于根据传感器的类型、分布和采集数据的物理特性,确定物联网终端应用场景,并制定采集数据的目标和传输需求评估,明确各类传感器的部署位置与功能;The sensor deployment module is used to determine the application scenarios of IoT terminals based on the type, distribution and physical characteristics of the sensor data collected, formulate the target of data collection and transmission demand assessment, and clarify the deployment location and function of various sensors; 数据传输规范模块,用于根据传感器的类型及数据变化速率,设置采样频率与数据采集窗口,并采用异步或同步采集策略进行数据采集与传输调度,对采集到的数据进行噪声滤波和异常值检测,并对不同类型数据进行规范化处理;The data transmission specification module is used to set the sampling frequency and data acquisition window according to the type of sensor and the data change rate, and adopt asynchronous or synchronous acquisition strategies to perform data acquisition and transmission scheduling, perform noise filtering and outlier detection on the collected data, and perform standardization processing on different types of data; 优先级量化模块,根据熵偏移指数对数据优先级进行量化,基于动态阈值聚类算法识别重要事件变化,并动态调整带宽资源,确定数据传输优先级;The priority quantification module quantifies data priority according to the entropy shift index, identifies important event changes based on the dynamic threshold clustering algorithm, and dynamically adjusts bandwidth resources to determine data transmission priority; 传输控制模块,用于基于优先级分配结果,对不同优先级的数据流分配独立队列,通过自适应调度机制调整队列容量,并采用时间片轮询算法进行优先级轮询,进行数据传输效率和队列管理优化。The transmission control module is used to allocate independent queues to data flows of different priorities based on the priority allocation results, adjust the queue capacity through an adaptive scheduling mechanism, and use a time slice polling algorithm for priority polling to optimize data transmission efficiency and queue management.
CN202411413064.4A 2024-10-11 2024-10-11 A transmission control method and system for Internet of Things terminal data Active CN118972333B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202411413064.4A CN118972333B (en) 2024-10-11 2024-10-11 A transmission control method and system for Internet of Things terminal data

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202411413064.4A CN118972333B (en) 2024-10-11 2024-10-11 A transmission control method and system for Internet of Things terminal data

Publications (2)

Publication Number Publication Date
CN118972333A CN118972333A (en) 2024-11-15
CN118972333B true CN118972333B (en) 2025-01-28

Family

ID=93396964

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202411413064.4A Active CN118972333B (en) 2024-10-11 2024-10-11 A transmission control method and system for Internet of Things terminal data

Country Status (1)

Country Link
CN (1) CN118972333B (en)

Family Cites Families (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
TW569570B (en) * 2000-10-16 2004-01-01 Physical Optics Corp Multimedia sensor network
GB0421719D0 (en) * 2004-09-30 2004-11-03 Koninkl Philips Electronics Nv Communication system
KR101217813B1 (en) * 2010-06-17 2013-01-02 금오공과대학교 산학협력단 Method to determine priority of data transmission in wireless network
CN104168318B (en) * 2014-08-18 2018-03-23 中国联合网络通信集团有限公司 A kind of Resource service system and its resource allocation methods
US10142353B2 (en) * 2015-06-05 2018-11-27 Cisco Technology, Inc. System for monitoring and managing datacenters
CN106954074B (en) * 2016-01-07 2019-12-20 青岛海信电器股份有限公司 Video data processing method and device
CN107733689A (en) * 2017-09-15 2018-02-23 西南电子技术研究所(中国电子科技集团公司第十研究所) Dynamic weighting polling dispatching strategy process based on priority
WO2019064986A1 (en) * 2017-09-29 2019-04-04 ソニー株式会社 Communication device and communication method
CN111600754B (en) * 2020-05-11 2022-02-25 重庆邮电大学 Industrial heterogeneous network scheduling method for interconnection of TSN (transmission time network) and non-TSN (non-Transmission time network)
CN112072711A (en) * 2020-08-11 2020-12-11 华北电力大学(保定) Power distribution network flexibility optimization scheduling method based on dynamic priority
CN118467487A (en) * 2024-04-12 2024-08-09 华能信息技术有限公司 Log acquisition system and method based on plug-in
CN118474039A (en) * 2024-05-22 2024-08-09 国电南京自动化股份有限公司 TSN network medium system for realizing accurate flow control

Also Published As

Publication number Publication date
CN118972333A (en) 2024-11-15

Similar Documents

Publication Publication Date Title
CN110263846B (en) Fault diagnosis method based on fault data deep mining and learning
CN104702680B (en) A kind of monitoring resource method and system
KR101563882B1 (en) Apparatus for processing sensor signal and system for processing sensor signal in distributed manner
US20170316329A1 (en) Information processing system and information processing method
JP6823501B2 (en) Anomaly detection device, anomaly detection method and program
US10508974B2 (en) Storing analytical machine data based on change in scalar machine data indicating alert condition
CN116684878A (en) 5G information transmission data safety monitoring system
CN118607885B (en) Intelligent detection system and method for soil slope construction based on Internet of Things technology
CN118168742A (en) Vibration monitoring and diagnosing system of wind turbine generator
CN117111858A (en) Computer file data matching system
CN118972333B (en) A transmission control method and system for Internet of Things terminal data
CN114710369A (en) Abnormal data detection method and device, computer equipment and storage medium
CN118230553B (en) Traffic flow data analysis system based on expressway
CN118612059A (en) An Internet of Things Device Management System
CN118764372A (en) Fault detection and automatic recovery method for electrical communication links
JP6772486B2 (en) Sensor data processing device, sensor data processing system, sensor data processing method, and sensor data processing program
CN118518061A (en) Intelligent engineering measurement management method, system and medium based on Internet of things
CN111222203B (en) Bearing service life model creation and prediction method thereof
CN117235650A (en) Method, device, equipment and medium for detecting high-altitude operation state
CN117522277A (en) Intelligent grain depot information transmission and emergency response system
CN115988181A (en) Personnel monitoring system and method based on infrared image algorithm
CN117375889A (en) Big data abnormal behavior monitoring method, system, equipment and storage medium
US20220120463A1 (en) Process for adaptable health, degradation and anomaly detection of systems using benchmarks
JPWO2018142704A1 (en) Feature value generation apparatus, feature value generation method, and program
CN118233288B (en) Vehicle communication management method based on SaaS

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant