CN117686086B

CN117686086B - Equipment running state monitoring method, device, equipment and system

Info

Publication number: CN117686086B
Application number: CN202410148696.6A
Authority: CN
Inventors: 刘璐; 王盈佳; 李少洋
Original assignee: Beijing Disheng Technology Co ltd
Current assignee: Beijing Disheng Technology Co ltd
Priority date: 2024-02-02
Filing date: 2024-02-02
Publication date: 2024-06-04
Anticipated expiration: 2044-02-02
Also published as: CN117686086A

Abstract

The invention provides a method, a device, equipment and a system for monitoring equipment operation states, which are used for acquiring equipment operation data of equipment to be monitored, which are positioned in a target monitoring area, according to voiceprint acquisition influence factors corresponding to environmental data, determining a voiceprint processing mode corresponding to the environmental data, processing initial voiceprint data according to the voiceprint processing mode to obtain target voiceprint data, eliminating environmental noise in the acquired voiceprint data, and enabling the obtained target voiceprint data to be the voiceprint data of the equipment to be monitored. In addition, the sound collection plate for detecting the initial voiceprint data is installed in a preset distance range of the equipment to be monitored in a non-contact installation mode, and compared with a direct attachment installation mode, the influence of the operation of the equipment to be monitored on the collection of the voiceprint data by the sound collection plate is avoided, the data collection accuracy is improved, and the accuracy of the operation state determination of the follow-up equipment is improved.

Description

Equipment running state monitoring method, device, equipment and system

Technical Field

The present invention relates to the field of operation state monitoring, and in particular, to a method, an apparatus, a device, and a system for monitoring an operation state of a device.

Background

As the degree of intelligence increases, the variety and number of devices used increases. In the running process of the equipment, if the equipment has faults and the like, the equipment needs to be shut down for maintenance.

If the running state of the equipment can be monitored, the equipment can be processed in time when the equipment is abnormal, so that the problem of shutdown maintenance caused by equipment faults is avoided.

When the running state of the equipment is monitored, a manual regular inspection mode is generally adopted, but the mode needs manual intervention and is easily influenced by manual subjective factors, and the monitoring accuracy of the running state of the equipment is low.

Disclosure of Invention

In view of the above, the invention provides a method, a device and a system for monitoring the running state of equipment, so as to solve the problem of low monitoring accuracy of manual monitoring of the running state of the equipment.

In order to solve the technical problems, the invention adopts the following technical scheme:

A method for monitoring the operation state of equipment, comprising:

Acquiring equipment operation data of equipment to be monitored, which is positioned in a target monitoring area; the device operation data at least comprises initial voiceprint data; the sound collection plate for detecting the initial voiceprint data is arranged in a non-contact installation mode within a preset distance range of the equipment to be monitored;

collecting environment data of the target monitoring area;

Determining a voiceprint processing mode corresponding to the environmental data according to voiceprint acquisition influence factors corresponding to the environmental data;

Processing the initial voiceprint data according to the voiceprint processing mode to obtain target voiceprint data;

and determining the equipment running state of the equipment to be monitored based on the target voiceprint data.

An apparatus for monitoring the operation state of a device, comprising:

The data acquisition module is used for acquiring equipment operation data of equipment to be monitored, which is positioned in the target monitoring area; the device operation data at least comprises initial voiceprint data; the sound collection plate for detecting the initial voiceprint data is arranged in a non-contact installation mode within a preset distance range of the equipment to be monitored;

The data acquisition module is used for acquiring the environmental data of the target monitoring area;

The processing mode determining module is used for determining a voiceprint processing mode corresponding to the environment data according to the voiceprint acquisition influence factors corresponding to the environment data;

The voiceprint processing module is used for processing the initial voiceprint data according to the voiceprint processing mode to obtain target voiceprint data;

and the state determining module is used for determining the equipment running state of the equipment to be monitored based on the target voiceprint data.

An equipment operation state monitoring equipment, comprising: a memory and a processor;

Wherein the memory is used for storing programs;

The processor invokes the program and is configured to perform the device operating state monitoring method described above.

A device operational status monitoring system comprising:

The voice print data processing device comprises a voice print data acquisition processing device and at least one expansion device which is independent of the voice print data acquisition processing device;

The voiceprint data acquisition and processing device comprises an acquisition shell and a voiceprint acquisition and processing assembly, wherein the voiceprint acquisition and processing assembly is arranged in the acquisition shell and is used for acquiring and processing voiceprint information; the voiceprint acquisition and processing assembly comprises the equipment operation state monitoring equipment;

the external expansion equipment is electrically connected with the voiceprint acquisition and processing assembly through a cable and is used for being matched with the voiceprint data acquisition and processing device so that the equipment operation state monitoring system can obtain the function corresponding to the external expansion equipment, and the external expansion equipment is detachably connected with the acquisition shell.

Compared with the prior art, the invention has the following beneficial effects:

The invention provides a method, a device, equipment and a system for monitoring equipment operation states, wherein equipment operation data of equipment to be monitored in a target monitoring area are acquired, a voiceprint processing mode corresponding to environmental data is determined according to voiceprint acquisition influence factors corresponding to the environmental data, initial voiceprint data are processed according to the voiceprint processing mode to obtain target voiceprint data, environmental noise in the acquired voiceprint data is eliminated, the acquired target voiceprint data are the voiceprint data of the equipment to be monitored, and when the equipment operation states are determined based on the target voiceprint data, the accuracy of equipment operation state determination can be improved. In addition, in the invention, the sound collection plate for detecting the initial voiceprint data is arranged in the preset distance range of the equipment to be monitored in a non-contact installation mode, compared with the mode of directly attaching the sound collection plate to the equipment to be monitored, the influence of the operation of the equipment to be monitored on the collection of the voiceprint data by the sound collection plate can be avoided, the voiceprint data collection accuracy is further improved, and the accuracy of the operation state determination of the follow-up equipment is improved.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are required to be used in the embodiments or the description of the prior art will be briefly described below, and it is obvious that the drawings in the following description are only embodiments of the present invention, and that other drawings can be obtained according to the provided drawings without inventive effort for a person skilled in the art.

FIG. 1 is a schematic structural diagram of a first system for monitoring the operation state of a device according to an embodiment of the present invention;

FIG. 2 is a schematic structural diagram of a second system for monitoring the operation state of equipment according to an embodiment of the present invention;

FIG. 3 is an exploded view of a voiceprint data acquisition processing apparatus according to an embodiment of the present invention;

FIG. 4 is an isometric view of a voiceprint data acquisition and processing apparatus according to an embodiment of the present invention at a first angle;

FIG. 5 is an isometric view of a voiceprint data acquisition and processing apparatus according to an embodiment of the present invention at a second angle;

FIG. 6 is a flow chart of a method for monitoring the operation state of a device according to an embodiment of the present invention;

FIG. 7 is a flowchart of a method for determining a target monitoring area according to an embodiment of the present invention;

FIG. 8 is a flowchart of a method for determining a voiceprint processing method according to an embodiment of the present invention;

Fig. 9 is a schematic structural diagram of an apparatus for monitoring an operation state of a device according to an embodiment of the present invention.

Reference numerals:

100-voiceprint data acquisition and processing device; 110-a lower housing; 111-an upper housing; 112-wind guard; 113-a dust cap; 120-a sound collection panel; 121-a core handling plate; 130-a power interface; 131-a network cable interface; 132-a switch; 133-device status light; 134-interface avoidance holes; a 135-antenna;

200-an expansion device; 210-a battery compartment; 220-a power supply bin; 230-a storage bin; 240-switch bin;

300-connectors.

Detailed Description

The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.

It should be noted that: like reference numerals and letters denote like items in the following figures, and thus once an item is defined in one figure, no further definition or explanation thereof is necessary in the following figures. The term "and/or" is used herein to describe only one relationship, meaning that there may be three relationships, e.g., a and/or B, which may mean: a exists alone, A and B exist together, and B exists alone. In addition, the term "at least one" herein means any one of a plurality or any combination of at least two of a plurality, for example, including at least one of A, B, C, may mean including any one or more elements selected from the group consisting of A, B and C; in addition, "/" in embodiments of the present disclosure may be understood as a relationship between and/or among objects.

In practical application, if the running state of the equipment can be monitored, the equipment can be processed in time when the equipment is abnormal, so that the problem of shutdown maintenance caused by equipment failure is avoided. Such as in the areas of transportation, construction, agriculture, etc., often require periodic inspection and maintenance of various work equipment. When the running state of the equipment is monitored, a manual regular inspection mode is generally adopted, but the mode needs manual intervention and is easily influenced by manual subjective factors, and the monitoring accuracy of the running state of the equipment is low. In addition, various sensors can be used for detecting the running state of the equipment, but the running state of the equipment is generally monitored by the sensors, the sensors are required to be attached to the equipment, and therefore, the sensors can influence the normal running of the equipment, and in addition, for large-scale equipment such as an ultrahigh voltage transformer, the sensors cannot be arranged on the equipment at all.

For example, the processing steps may be performed,

1) Transportation field:

In modern transportation fields, such as railways, aviation and highways, stable operation of equipment is a key to ensuring safety and efficiency. Traditional monitoring methods rely primarily on periodic equipment inspection and maintenance, or use of various sensors to monitor the operational status of the equipment. The problems with these methods are: failing to monitor in real time requires a great deal of manual intervention and the sensors may affect the normal operation of the device.

For example, wear of the train wheel set, abnormal noise of the aircraft engine or minor changes in bridge construction on the road may cause serious safety hazards.

2) Building field:

With the progress of urban construction, the construction of infrastructure such as high-rise buildings, large bridges and tunnels is increasing. The stability and safety of these structures are the basis for the life of urban residents. Traditional building structure monitoring methods are mainly based on regular structural inspection and assessment, or the use of various sensors to monitor the health of the structure. Limitations of these methods are: real-time monitoring cannot be achieved, a large amount of manual intervention is required, and the installation of the sensor may affect the structural integrity.

For example, cracks in buildings, vibrations in bridges or water seepage in tunnels can lead to structural damage and safety accidents.

3) Agricultural field:

in modern agricultural production, stable operation of agricultural machinery is a key to improving production efficiency and ensuring quality of agricultural products. Conventional agricultural equipment monitoring methods are mainly based on regular equipment inspection and maintenance, or use of various sensors to monitor the operating status of the equipment. The problems with these methods are: failing to monitor in real time requires a great deal of manual intervention and the sensors may affect the normal operation of the device.

For example, malfunctions of the planter, wear of the blades of the harvester, or pump malfunctions of the irrigation system, can result in reduced production efficiency and reduced quality of the agricultural product.

In summary, the current monitoring methods of industrial equipment include one, employing experienced staff to periodically perform maintenance; 2. sensors are installed on industrial equipment to monitor the operating conditions of the industrial equipment. The method cannot monitor in real time, each device is monitored manually in the monitoring process, the labor cost is high, time and labor are wasted, in addition, the experience of staff is rich, otherwise, the accuracy is low, and the monitoring effect cannot be achieved; the second mode cannot be realized on some large-scale equipment (monitoring the large-scale equipment may need a lot of sensors, which can increase a lot of hardware cost and labor cost), the sensors cannot be installed, or the installation of the sensors on the industrial equipment can destroy the structure of the industrial equipment, so that the industrial equipment is abnormal in operation and the like.

For example, industrial equipment is a transformer, and the transformer is one of important equipment in a power system, bears key tasks such as voltage conversion, electric energy distribution and transmission in the system, and plays an important role in providing high-quality electric energy service and ensuring safe, reliable, high-quality and economic operation of the power system. The power equipment stably operates for a long time with high load and all weather, and plays a fundamental role in the national production and economy process. The task of ensuring the stable operation of the power equipment is very necessary, and one of the means for ensuring the power equipment is to diagnose the fault of the power equipment through various means. The power equipment fault diagnosis technology is a practical discipline combined with engineering practice and plays an important role in power production application.

At present, the dynamic information monitoring technology mainly adopts signals such as vibration, stress, temperature, optical signals, rays, electromagnetism and the like for analysis, and high-voltage and ultra-high-voltage transformer equipment is generally uniformly distributed in a large-scale transformer electric field. Due to the characteristic of ultra-high voltage, various surface-mounted sensors cannot be used. The sensor cannot be placed on the transformer and device monitoring cannot be achieved.

In order to avoid the problems of low manual accuracy, low efficiency, incapability of mounting the sensor, influence on equipment operation by the sensor and the like, a remote automatic monitoring scheme can be adopted, the automatic monitoring efficiency and accuracy are superior to those of manual operation, and the sensor does not need to be directly arranged on industrial equipment, so that the problem that the sensor cannot be mounted on the equipment or the equipment operation is influenced after the sensor is mounted is avoided.

In addition, with the development of voiceprint technology, the real-time detection of the running state of each working equipment is possible through the voiceprint acquisition equipment.

The traditional voiceprint acquisition equipment is usually of an integrated design, and functions such as a power supply, a battery, a data storage terminal and a network cable signal transmission terminal are integrated in one large equipment.

Therefore, how to improve the practicability of the voiceprint acquisition device is a technical problem to be solved by those skilled in the art.

On the basis, the equipment operation state monitoring system provided by the invention divides the existing voiceprint acquisition equipment into two functional modules of the voiceprint data acquisition processing device and the expansion equipment, the volume and the weight of each functional module are smaller, the carrying of a user is convenient, and the detachable connection of the expansion equipment and the voiceprint data acquisition processing device enables the user to flexibly combine and configure according to actual requirements and application scenes, so that a customized voiceprint acquisition solution is realized. Meanwhile, when new functions are required to be added or existing functions are required to be upgraded, only corresponding functional modules are required to be replaced or added, the whole system is not required to be replaced, and the practicability of the voiceprint acquisition equipment is greatly improved.

In addition, on the equipment running state monitoring system, data fusion is carried out based on the voiceprint signals and other sensor data, and the equipment running state is monitored.

In addition, the invention does not need to be installed on the equipment to be monitored in a contact way, is arranged at a certain distance from the equipment to be monitored, and can acquire, process and analyze signals through the microphone array to obtain the running state result of the equipment to be monitored.

In one embodiment of the present invention, the content of the device operation status monitoring system is first introduced.

Referring to fig. 1 and fig. 2, the device operation state monitoring system disclosed in the embodiment of the present invention includes a voiceprint data collecting and processing device 100 and an external expansion device 200, where the external expansion device 200 is independently disposed with respect to the voiceprint data collecting and processing device 100, and is at least one. The voiceprint data acquisition and processing device 100 comprises an acquisition shell and a voiceprint acquisition and processing assembly, wherein the expansion equipment 200 is used for being detachably connected with the acquisition shell through a connecting piece 300, the voiceprint acquisition and processing assembly is arranged in the acquisition shell and used for acquiring and processing voiceprint information, and the acquisition shell is used for being fixed on equipment to be monitored or surrounding the equipment to be monitored and protecting the voiceprint acquisition and processing assembly.

The external expansion device 200 is electrically connected with the voiceprint acquisition and processing assembly through a cable, and is used for being matched with the voiceprint data acquisition and processing device 100 so that the device running state monitoring system can obtain the function corresponding to the external expansion device 200. Different external expansion devices 200 have different functions, the external expansion devices 200 can be one or more of a battery compartment 210, a power supply compartment 220, a storage compartment 230 and a switch compartment 240, that is, power supply, battery, data storage and network signal transmission components can be decomposed into individual external expansion devices 200 according to different functions, and each external expansion device 200 can realize respective functions by matching with the voiceprint data acquisition and processing device 100, so that the device operation state monitoring system has functions corresponding to each external expansion device 200.

For example, when the expansion device 200 is the battery compartment 210, the voiceprint data collecting and processing device 100 is electrically connected with the battery compartment 210, so that the device running state monitoring system can have the function of completing the sound collecting and processing operation without external power supply.

The equipment to be monitored can be at least one of a plurality of equipment such as mechanical equipment, industrial equipment, non-industrial equipment and the like, and can also be part of mechanical components in the mechanical equipment; for example, it may include: the mechanical components in various mechanical equipment such as an ultrahigh voltage transformer, a transformer, an electric locomotive running part, an electric locomotive traction fan, an electric locomotive traction transformer, a circulating water pump, an induced draft fan, fan blades, a coal mill and the like.

Compared with the prior art, the equipment operation state monitoring system disclosed by the embodiment of the invention divides the existing voiceprint acquisition equipment into two functional modules of the voiceprint data acquisition processing device 100 and the external expansion equipment 200, the volume and the weight of each functional module are smaller, the carrying of a user is convenient, and the detachable connection of the external expansion equipment 200 and the voiceprint data acquisition processing device 100 enables the user to flexibly combine and configure according to actual requirements and application scenes, so that a customized voiceprint acquisition solution is realized. Meanwhile, when new functions are required to be added or existing functions are required to be upgraded, only corresponding functional modules are required to be replaced or added, the whole system is not required to be replaced, and the practicability of the voiceprint acquisition equipment is greatly improved.

Specifically, the structures of the external expansion devices 200 are basically the same, each external expansion device comprises a shell component and a functional component, the shell component is detachably connected with the acquisition shell through a connecting piece 300, the functional component is arranged in the shell component, the core structures with different functions are realized for the external expansion devices 200, and the functional component is electrically connected with the voiceprint acquisition processing component, so that the device running state monitoring system can realize different functions.

The connecting piece 300 may be specifically a link block or the like, and connection holes through which bolts can pass are formed in the collecting housing of the voiceprint data collecting and processing device 100 and the housing assembly of the expanding device 200, so that detachable connection between the voiceprint data collecting and processing device 100 and the expanding device 200 can be realized through the link block. Specifically, as shown in fig. 1, the voiceprint data acquisition and processing device 100 and each expansion device 200 can be fixed in parallel or in a vertical manner by a link block, so as to adapt to flexible deployment in a narrow space.

In some embodiments, a chute is provided on an outer wall of one of the acquisition housing and the housing assembly, and a slider is provided on an outer wall of the other, and the detachable connection of the voiceprint data acquisition processing apparatus 100 and the flaring device 200 is achieved through a sliding fit of the slider and the chute.

The matching of the sliding groove and the sliding block can be replaced by the matching of the embedded groove and the embedded block.

Referring to fig. 2, in one embodiment, the number of the expanding devices 200 is four, and is a battery compartment 210, a power compartment 220, a storage compartment 230, and a switch compartment 240, respectively. The battery compartment 210 can store energy, the power supply compartment 220 can be connected with an external power supply to supply power to the equipment operation state monitoring system, the storage compartment 230 can store collected voiceprint data and processed voiceprint data, and the switch compartment 240 can realize data communication.

Specifically, the battery compartment 210 includes a first upper case, a first lower case, and a battery compressed between the first upper case and the first lower case, and the first upper case and the first lower case are fixed by screws. Further, the battery compartment 210 also includes a battery switch and a power display to facilitate controlling the power supply of the battery and viewing the power of the battery. The battery compartment 210 can supply power to the voiceprint data acquiring and processing device 100 through the battery, so that the voiceprint data acquiring and processing device 100 can acquire sound data, and the battery can be charged through the same-head connector by flushing and discharging.

The power supply bin 220 includes a second upper case, a second lower case, and a power supply adapter disposed between the second upper case and the second lower case, through which high voltage can be converted into voltage required by the voiceprint data acquisition processing apparatus 100, and an input power line and an output power line of the power supply adapter are both fixed to the second lower case through a glan head.

The storage bin 230 is configured to provide more data storage space for offline voiceprint data acquisition for a long time to obtain voiceprint data of more scenes, and specifically includes a third upper shell, a third lower shell, and a storage module, where the storage module is disposed between the third upper shell and the third lower shell, and a data interface is disposed on the storage module, and is configured to be electrically connected to the voiceprint acquisition processing assembly and implement data transmission, so as to store acoustic signals acquired by the voiceprint data acquisition processing device 100 and processed signals.

The switch bin 240 includes a fourth upper case, a fourth lower case, and a switch, the switch is fixed in a bin body formed by the fourth upper case and the fourth lower case, and the fourth upper case and the fourth lower case are detachably connected through screws. By arranging switches with different signals in a bin body formed by the fourth upper shell and the fourth lower shell, the device can adapt to data transmission and conversion under various scenes, and realize data communication between the voiceprint data acquisition and processing device 100 and an upper computer or other devices.

In an alternative embodiment, the flaring device 200 can also be a sensor module; the sensor module is used for receiving a sensor signal of the equipment to be monitored and transmitting the sensor signal to the voiceprint data acquisition and processing device 100; after receiving the sensor signal, the voiceprint data acquiring and processing apparatus 100 switches the operating state of the voiceprint data acquiring and processing apparatus 100 based on the sensor signal.

Wherein the sensor module may include, for example, one or more of a vibration sensor, an acceleration sensor, a temperature sensor, a humidity sensor, a wind speed sensor, a thermal imaging sensor, a gas sensor, or other types of sensors; the operating state of the voiceprint data acquisition processing apparatus 100 can include one or more of an on, off, noise reduction mode, etc.

When the sensor module is a vibration sensor/acceleration sensor/temperature sensor/thermal imaging sensor/gas sensor, the sensor signal is a vibration signal/acceleration signal/temperature signal/thermal imaging image/gas signal, when the voiceprint data acquisition processing device 100 receives the vibration signal/acceleration signal/temperature signal/thermal imaging image/gas signal, whether the sensor module needs to be automatically started or not can be judged based on the vibration signal/acceleration signal/temperature signal/thermal imaging image/gas signal, for example, when the vibration signal is greater than or equal to a preset vibration threshold value/acceleration signal is greater than or equal to a preset acceleration threshold value/temperature signal is greater than or equal to a preset temperature threshold value/thermal imaging image is less than or equal to a preset approximation threshold value/gas signal is greater than or equal to a preset gas signal threshold value, the abnormal state of the equipment to be monitored is initially confirmed, the voiceprint data acquisition processing device 100 is automatically started, the equipment to be monitored is further monitored and judges the running state of the equipment to be monitored, so that the multi-sensor joint monitoring of the equipment to be monitored is realized, the accuracy of the joint monitoring of various sensors is required, and the accuracy of the joint monitoring of the equipment to be monitored is further improved.

When the sensor module is a wind speed sensor, the sensor signal is a wind speed signal, and when the voiceprint data acquisition and processing device 100 receives the wind speed signal, whether the self-starting is needed or not can be judged based on the wind speed signal, and after the self-starting is confirmed, whether the noise reduction mode needs to be started or not is judged continuously according to the wind speed signal; for example, when the received wind speed signal is greater than or equal to the preset wind speed threshold, the voiceprint data acquisition processing apparatus 100 automatically starts and opens the noise reduction mode, and performs voiceprint monitoring on the device to be monitored and eliminates the influence of wind noise, thereby further improving the monitoring accuracy.

The preset vibration threshold value, the preset acceleration threshold value, the preset temperature threshold value, the preset approximation threshold value, the preset gas signal threshold value and the preset wind speed threshold value can be set according to actual requirements, and specific limitations are not adopted here.

In a specific embodiment, when the expansion device 200 is a vibration sensor, the vibration sensor is used for collecting a vibration signal of the device to be monitored, processing the vibration signal, monitoring the vibration state of the device to be monitored, and transmitting the collected vibration signal to the voiceprint data collecting and processing device 100; the voiceprint data acquisition and processing device 100 can receive the vibration signal acquired by the vibration sensor, and judge whether the voiceprint data acquisition and processing device 100 needs to be started or not based on the vibration signal so as to monitor voiceprint of the equipment to be monitored, and further monitor the running state of the equipment to be monitored; for example, when the received vibration signal is greater than or equal to the preset vibration threshold, it is determined that the equipment to be monitored is abnormal currently, and at this time, the voiceprint data acquisition and processing device 100 needs to be controlled to be started to acquire voiceprint data of the equipment to be monitored, and further determine the running state of the equipment to be monitored, so that the combined monitoring of the voicevibration of the equipment to be monitored is achieved, the requirement of a scene requiring combined monitoring of the voicevibration is met, and the monitoring accuracy is further improved.

It should be noted that, when the expansion device 200 is one or more of an acceleration sensor, a temperature sensor, a humidity sensor, a wind speed sensor, a thermal imaging sensor, or other types of sensors, the data interaction between the expansion device 200 and the voiceprint data acquisition and processing apparatus 100, and the specific processing logic of the voiceprint data acquisition and processing apparatus 100 are similar to those when the expansion device 200 is a vibration sensor, the description will be omitted herein.

It should be noted that, the electrical transmission interface of each expansion device 200 is unified with the electrical transmission interface of the voiceprint data collecting and processing apparatus 100, so as to achieve compatibility, ensure reliability and stability of operation, and provide possibility for future function expansion.

Referring to fig. 3, the collecting housing includes a lower housing 110 and an upper housing 111, the lower housing 110 is configured to be detachably connected with the flaring device 200, the upper housing 111 is detachably connected with the lower housing 110 and encloses an installation space, and the voiceprint collecting and processing assembly is disposed in the installation space. Specifically, the detachable connection between the upper case 111 and the lower case 110 may be achieved by screws. The housing assembly of the flaring device 200 can be the same or different in construction than the collection housing.

The lower housing 110 may be a hollow rectangular housing structure, and the upper housing 111 may be a rectangular plate-shaped structure, so that in order to facilitate positioning and installation of the upper housing 111 and the lower housing 110, a flange is provided in a circumferential direction of the upper housing 111, and a positioning step is formed between the flange and the upper housing 111, so that the opening ends of the upper housing 111 and the lower housing 110 are in positioning fit. As shown in fig. 3, a first threaded hole is formed in a circumferential side wall of the upper housing 111, correspondingly, a first through hole corresponding to the first threaded hole is formed in an edge position of an opening end of the lower housing 110, and a first screw is used for penetrating through the first through hole in the lower housing 110 and screwing into the first threaded hole in the upper housing 111, so that the upper housing 111 and the lower housing 110 can be detachably connected.

In an embodiment, the first through holes are symmetrically disposed on two opposite sidewalls of the lower housing 110 (for example, six of the first through holes are shown in fig. 3, and three of the first through holes are formed on each sidewall), so that the connection is simple and effective, and the upper housing 111 and the lower housing 110 can be reliably connected, and meanwhile, the interference of vibration and noise in the sound collection process can be reduced, and meanwhile, the subsequent maintenance is convenient.

Further, a plurality of legs are provided on the bottom outer wall of the lower case 110 for being detachably fixed at different positions by screws; and/or, a sound absorbing pad or a slip preventing pad made of a flexible material is provided on the bottom outer wall of the lower case 110 for absorbing sound or preventing slip.

The voiceprint collection processing assembly is a core structure for executing sound collection and processing actions, and comprises a sound collection plate 120, wherein the sound collection plate 120 is arranged on an upper shell 111, planes of the sound collection plate 120 and the upper shell 111 are parallel, a plurality of microphones are arranged on the sound collection plate 120 and used for capturing sound signals, and a sound collection function is executed. A plurality of sound transmission holes are formed in the upper shell 111 in a penetrating manner, and the sound transmission holes correspond to the positions of the microphones one by one, so that external sounds can pass through the sound transmission holes and be collected by the microphones.

In order to be able to more effectively localize the sound source direction, the microphones are arranged in a ring shape in one or more rings, so that the microphones can capture sound from different angles and the accuracy of sound localization and the accuracy of voiceprint recognition are improved.

In an embodiment, the sound-transmitting holes are arranged in a crisscross array on the sound collecting board 120, that is, a plurality of microphones are respectively arranged on two reference lines of the sound collecting board 120, and the two reference lines are perpendicular to each other, and the number of the microphones may be eight.

Further, the microphone may be a digital microphone for converting the captured sound waves into digital signals and transmitting the digital signals to the core processing board 121, and the use of the digital microphone helps to improve the accuracy of voiceprint recognition, processing speed, and quality and accuracy of sound capture.

Specifically, the sound collection board 120 is a rectangular board, and third through holes are formed in the four corners, third threaded holes are formed in one side of the upper casing 111 facing the sound collection board 120, the third threaded holes correspond to the third threaded holes one by one, the sound collection board 120 can be fixed on the upper casing 111 through third screws, reliable fixing of the sound collection board 120 is guaranteed, and noise or signal distortion caused by vibration is reduced.

Further optimizing the structure, voiceprint data acquisition processing apparatus 100 further includes a wind guard 112, and wind guard 112 is disposed on a side of upper housing 111 facing away from lower housing 110, and a sound-transmitting groove is disposed on wind guard 112 in a penetrating manner, the position of the sound-transmitting groove corresponds to the position of the sound-transmitting hole, and sound-transmitting cotton is compressed between wind guard 112 and upper housing 111. The arrangement of the air guard 112 and the sound-transmitting cotton can prevent the strong wind from interfering with the data acquisition of the microphone, so as to effectively improve the accuracy and reliability of the data acquisition in the outdoor or large-wind environment.

In an embodiment, the circumference of the air guard 112 is provided with a positioning convex edge, so that the air guard 112 forms a cover-shaped structure and can be sleeved on the upper shell 111 and the opening end of the lower shell 110, a second through hole is arranged on the positioning convex edge, a second threaded hole is correspondingly arranged on the lower shell 110, and a second screw is used for penetrating through the second through hole and screwing in the second threaded hole so as to realize the detachable connection of the air guard 112 and the lower shell 110, or the second screw can sequentially penetrate through the air guard 112, the lower shell 110 and the upper shell 111 so as to realize the detachable connection of the air guard 112 and the lower shell 110 and the upper shell 111.

The second threaded holes may be symmetrically disposed on two opposite sidewalls of the lower housing 110 (for example, four shown in fig. 3, two on each sidewall). The first screw hole and the second screw hole may be disposed on the same side wall of the lower housing 110, so that the other two side walls of the lower housing 110 may be respectively used as a wiring side and a front side of the collecting housing, thereby improving the aesthetic property.

The first screw and the second screw can be all screws with the size of M2, the connection is reliable, the disassembly and the assembly are convenient, further, the air shield plate 112 and the lower shell 110 can be fixed through the first screw, and the positions of the second through hole and the first threaded hole are mutually corresponding in combination with fig. 3, so that the first screw can sequentially pass through the positioning convex edge and the lower shell 110 until being screwed into the upper shell 111, and the connection of the upper shell 111, the air shield plate 112 and the lower shell 110 is realized. In the installation, the upper case 111 and the lower case 110 may be fixed by two first screws, and then the upper case 111, the lower case 110, and the wind guard 112 may be simultaneously tightened by four first screws.

The voiceprint collecting and processing assembly comprises a core processing board 121, the core processing board 121 is connected with the lower shell 110 and is electrically connected with a microphone on the sound collecting board 120, the core processing board 121 is used for receiving sound signals transmitted by the sound collecting board 120 and processing the sound signals, and specifically comprises the steps of decoding and analyzing the sound signals and executing a voiceprint recognition algorithm; in addition, the core processing board 121 may include a high-performance processor and advanced algorithms to ensure rapid and accurate analysis of sound data, enabling efficient voiceprint recognition in real-time processing scenarios.

Specifically, the core process plate 121 may be detachably coupled with the lower case 110 by screws.

Referring to fig. 3 and 4, since the core processing board 121 needs to cooperate with the expanding device 200 to support the wider functions of data storage, remote transmission, etc., an information transmission interface for electrically connecting with the expanding device 200 or other devices is provided on the core processing board 121, where the information transmission interface may include one or more of a USB interface, an SD card interface, and a SIM card interface, correspondingly, an interface avoidance hole 134 for avoiding the information transmission interface is provided on the lower housing 110, and the information transmission interface may be exposed from the interface avoidance hole 134 to realize the electrical connection with the expanding device 200.

The USB data interface is used to connect to a data transmission module, so as to realize transmission of acoustic signals collected by the voiceprint data collecting and processing device 100 and processed signals. The number of USB data interfaces may be set to two. The SD card interface is used for expanding storage or transmission data. The SIM card interface can provide a mobile network connection function to realize remote data transmission.

In order to realize the dustproof of the information transmission interface, in combination with fig. 5, an interface caulking groove is formed in the outer wall of the lower housing 110, the bottom of the interface caulking groove protrudes towards the inner direction of the lower housing 110, an interface avoidance hole 134 is formed in the bottom of the interface caulking groove, and a dustproof sealing cover 113 is detachably arranged at the interface caulking groove, wherein the dustproof sealing cover 113 is used for sealing the interface caulking groove.

When the voiceprint data acquisition and processing device 100 is not needed, the dustproof sealing cover 113 can be screwed or clamped on the interface caulking groove, so that the interface caulking groove and the information transmission interface are isolated from the outside. When the information needs to be transmitted, the dustproof cover 113 is taken down or opened from the interface caulking groove, and wiring is performed through the information transmission interface.

In one embodiment, the dust cap 113 is made of elastic rubber material (rubber stopper), which is hinged to one end of the interface caulking groove and can be pressed and blocked at the interface caulking groove.

In addition, a power interface 130 is arranged on the core processing board 121, the power interface 130 is used for externally connecting a power supply to supply power to the voiceprint data acquisition and processing device 100, correspondingly, a power supply avoiding hole for avoiding the power interface 130 is formed in the lower shell 110, so that the power interface 130 can be exposed from the power supply avoiding hole, and the connection with the battery compartment 210, the power supply compartment 220 or an external power supply is realized; the core processing board 121 is provided with a network cable interface 131, the network cable interface 131 is used for being externally connected with a communication module, such as a switch cabin 240, so that data transmission communication is realized, correspondingly, the lower shell 110 is provided with a network cable avoiding hole for avoiding the network cable interface 131, so that the network cable interface 131 can be exposed from the network cable avoiding hole.

The core processing board 121 is provided with a switch 132, and the switch 132 is used for controlling the on and off of the voiceprint data acquisition and processing device 100, i.e. controlling the working state of the voiceprint data acquisition and processing device 100, for example, long pressing the switch 132 for one second, starting the voiceprint data acquisition and processing device 100, starting the acquisition of acoustic signals by the voiceprint data acquisition and processing device 100, long pressing the switch 132 for three seconds, and closing the voiceprint data acquisition and processing device 100. Correspondingly, a switch avoidance hole for avoiding the switch 132 is formed in the lower housing 110, so that the switch 132 can be exposed out of the lower housing 110.

The core processing board 121 is provided with a device status lamp 133 (including a power indicator lamp, a network status lamp, etc.), the lower housing 110 is provided with a status lamp avoiding hole for avoiding the device status lamp 133, and the device status lamp 133 can be exposed from the status lamp avoiding hole in the collecting housing, so as to ensure the visibility of the indicator lamp. Specifically, the device status light 133 is configured to indicate a current status of the voiceprint data collecting and processing apparatus 100, for example, the current status of the voiceprint data collecting and processing apparatus 100 may include a collecting status, a communication status, and a storage status, and a color correspondence between each status and the status light may be set by itself, for example, setting green light normally-on indication data to be stored online, blue light normally-on indication to be stored offline, green light flashing indication to be not stored online, blue light flashing indication to be stored offline, and red light normally-on indication program to be not started.

In an embodiment, the device status light 133 may also be used to indicate the status of the device monitored by the current voiceprint data acquiring and processing apparatus 100, for example, the status of the device to be monitored may include a normal status, a slight fault status, a serious fault status, etc., when the device status light 133 is red, the device to be monitored is in the serious fault status; when the device status light 133 is yellow, the device to be monitored is in a slight failure state; when the device status light 133 is green, the device to be monitored is in a normal state; through the indication function of the equipment status lamp 133, the fault condition of the equipment to be monitored can be visualized simply, so that a monitoring person can know the current running state of the equipment to be monitored in time, and corresponding overhaul treatment can be conducted on slight faults and serious faults in time.

An antenna 135 is provided on the core processing board 121, and an antenna avoiding hole for avoiding the antenna 135 is provided on the lower housing 110.

Referring to fig. 3, the power interface 130 may be fixed to the lower housing 110 by means of screw threads to ensure connection reliability, thereby preventing loosening of the power interface 130 due to vibration or movement. Specifically, be provided with the external screw thread at the circumference outer wall of power source interface 130, and be provided with spacing boss in power source interface 130 one end protrusion, spacing boss's size is greater than the size of power source dodge hole, and when power source interface 130 installs to power source dodge hole department, spacing boss and the lateral wall butt around the power source dodge hole on lower casing 110, the accessible is screwed the nut on power source interface 130's outer wall this moment, makes power source interface 130 fix in power source dodge hole department.

The switch 132 and the device status light 133 may also be reliably connected to the lower housing 110 in a similar manner to the power interface 130, and will not be described again.

On the basis of the above-mentioned device operation state monitoring system, another embodiment of the present invention provides a device operation state monitoring method, which may be applied to a device operation state monitoring device, and the device operation state monitoring device may be a core processing board 121 as described above.

Referring to fig. 6, an apparatus operation state monitoring method may include S11 to S15:

s11, acquiring equipment operation data of equipment to be monitored, which is located in a target monitoring area.

The equipment to be monitored in the invention can be at least one of a plurality of equipment such as mechanical equipment, industrial equipment, non-industrial equipment and the like, and can also be part of mechanical components in the mechanical equipment; for example, it may include: the mechanical components in various mechanical equipment such as an ultrahigh voltage transformer, a transformer, an electric locomotive running part, an electric locomotive traction fan, an electric locomotive traction transformer, a circulating water pump, an induced draft fan, fan blades, a coal mill and the like. Different monitoring angles of the running states of the equipment to be monitored are different, if so, the equipment needs to detect whether overload exists, and if so, the equipment needs to detect whether poor contact exists.

The device operational data includes at least initial voiceprint data. Further, other data may be included such as at least one of vibration signals, acceleration signals, temperature signals, thermal imaging images, gas signals, and the like.

In practical application, when the sensor signal of the device to be monitored is abnormal, the operation data of the device, such as voiceprint data, are collected. The sensor signal of the device to be monitored may be a signal acquired by a sensor for monitoring the operating state of the device to be monitored during the operation of the device to be monitored, and may include, for example, but not limited to, one or more of an acceleration signal, a vibration signal, a rotation speed signal, and a thermal imaging image.

After the sensor signal of the equipment to be monitored is obtained, the core processing board compares the sensor signal of the equipment to be monitored with stored preset fault information, and a comparison result is determined; based on the comparison result, whether the microphone array needs to be started or not is judged, namely, when the core processing board acquires abnormal data, an acquisition instruction is issued to the microphone array so as to analyze the running state of the equipment to be monitored based on voiceprint data, and the specific implementation can be referred to the corresponding description. Or controlling voiceprint acquisition when the voiceprint acquisition requirement is considered manually.

Wherein, the preset fault information may include, but is not limited to, at least one of the following: the method comprises the steps of pre-configuring acceleration signals and a pre-set fault acceleration threshold, vibrating signals and a pre-set fault vibration threshold, and rotating speed signals and a pre-set fault rotating speed threshold, and performing abnormal thermal imaging; the above-mentioned preset fault information may be set according to requirements, and is not particularly limited herein.

According to the invention, when the industrial equipment is monitored, the industrial equipment can be monitored in real time without carrying out attached layout. Specifically, the sound collection board 120 for detecting the initial voiceprint data is installed in the preset distance range of the device to be monitored in a non-contact installation manner, so that the problem that the sound collection board 120 cannot be installed on the device to be monitored or the operation of the device is affected after the sound collection board 120 is installed on the device to be monitored can be avoided.

The sound collection board 120 is provided with a plurality of microphones for capturing sound signals and performing a sound collection function. Microphones and other sensors collect device operational data of devices to be monitored in a target monitoring area at the time of data collection.

In actual device monitoring, the device running state monitoring device is the core processing board 121, and the core processing board 121 is a component of the voiceprint acquisition processing assembly, and in addition, the voiceprint acquisition processing assembly further includes the sound acquisition board 120. The voiceprint acquisition and processing assembly is a component of voiceprint data acquisition and processing apparatus 100.

The set equipment operation state monitoring equipment can be separated from the equipment to be monitored by a certain distance, namely, the voiceprint data acquisition and processing device 100 is separated from the equipment to be monitored by a certain distance, and the exemplary voiceprint data acquisition and processing device 100 can be arranged on a stand column and is mainly used for monitoring the operation state of the equipment to be monitored. Generally, a plurality of voiceprint data acquiring and processing devices 100 (for example, placed up and down and separated by a relatively close distance) may be disposed at the same location, and if the equipment to be monitored is operating normally, only one voiceprint data acquiring and processing device 100 may be turned on at the same location, so as to save cost. If the equipment to be monitored is abnormal, the voiceprint data acquisition and processing device 100 at the same position can be started. In addition, the voiceprint data acquisition and processing device 100 may be further provided on a walking robot to perform inspection according to a set trajectory.

In practical application, the invention mainly uses voiceprint data to monitor the running state of equipment, taking a transformer as an example, if the transformer gives out a 'squeak' or 'crackle', the running state may be discharge caused by poor contact, if the transformer gives out a 'Java' sound, the running state may be overload.

In another implementation manner of the present invention, in order to improve the monitoring accuracy, data fusion may be performed by combining other data, such as at least one of the vibration signal, the acceleration signal, the temperature signal, the thermal imaging image, the gas signal, and the like, so as to obtain a more accurate running state of the device.

In a specific implementation, the target monitoring area in the embodiment may be an abnormal area in the device to be monitored, and the abnormal area may be specifically obtained through thermal imaging data analysis.

Thus, referring to fig. 7, the determination process of the target monitoring area in the present embodiment includes S21 to S23:

S21, acquiring thermal imaging data of the equipment to be monitored.

The thermal imaging data can be acquired by a thermal imaging video sensor, specifically an infrared thermal imaging camera. An infrared thermal imaging camera captures thermal images of the environment, monitors the temperature distribution around the device in real time, to identify hot spot areas.

S22, carrying out anomaly analysis on the thermal imaging data to screen out anomaly thermal imaging data.

In thermal imaging analysis, the core processing board analyzes the thermal imaging data to identify hot spot areas. Based on the hot spot area and the temperature change condition of the hot spot area, abnormal thermal imaging data of abnormal temperature change can be determined.

Wherein, various hot spot areas and normal temperature ranges corresponding to the hot spot areas of different types are prestored; these hot spot areas may represent a human body, mechanical equipment, or other heat source. For example, the normal temperature range corresponding to human body is 36-37 ℃; when the mechanical equipment is a transformer winding insulator, the normal temperature range is 80-140 ℃, the service life loss of the transformer winding insulator is doubled when the temperature is increased by 6 ℃, and the service life is reduced by half.

Illustratively, when the device to be monitored is a transformer, an image containing the transformer is captured by using a thermal imaging camera, thermal imaging data analysis is performed on the transformer image, and a hot spot area is identified. Confirming the current temperature of the hot spot area, comparing the current temperature of the hot spot area with a normal temperature range corresponding to a prestored hot spot area, and responding to the fact that the current temperature of the hot spot area is not in the normal temperature range corresponding to the prestored hot spot area, confirming that the temperature change of the hot spot area is abnormal, and taking image data corresponding to the hot spot area with abnormal temperature change as abnormal thermal imaging data.

S23, determining the area of the equipment to be monitored corresponding to the abnormal thermal imaging data, and taking the area as a target monitoring area.

The hot spot areas corresponding to the abnormal thermal imaging data may be sources or key areas of sound events, and need to be monitored in a key way.

When collecting the equipment operation data of the target monitoring area, taking collecting voiceprint data as an example, the beam forming parameters of the sound collecting board 120 can be adjusted according to the abnormal thermal imaging data so as to enhance the voiceprint collection of the target monitoring area to obtain initial voiceprint data.

In detail, voiceprint data of a target monitoring area can be collected in a focused manner through a directional enhancement algorithm. The directional enhancement algorithm may employ a beamforming algorithm that dynamically adjusts the beamforming parameters of the microphone array based on the thermal imaging data in order to emphasize enhancement of sound collection in a particular region. This may be achieved by adjusting the phase and gain of the microphone signal. The implementation process comprises the following steps 1 to 4:

1. thermal imaging data analysis

And (3) direction determination: the location and orientation of the hot spot area relative to the microphone array is calculated.

2. Beamforming parameter calculation

The implementation principle of the beamforming parameter calculation is based on the time difference of sound propagation, in particular, when a sound source is located in a specific direction of a microphone array, the time of arrival of sound waves at different microphones in the array may be different. This time difference is caused by the difference in the distance of the sound source to the respective microphones.

The result of the signal delay and summation is a directivity enhanced acoustic signal that contains acoustic information from a particular direction while noise and interference from other directions are effectively suppressed.

In addition, by phase alignment, sound from a specific direction can be effectively enhanced, but it is necessary to ensure that the sound signals are phase-aligned at the time of synthesis during use. If the unadjusted signals are added directly, sound from a particular direction may be partially or completely cancelled due to phase inconsistencies, resulting in signal distortion or reduced strength.

In a specific implementation, firstly, time delay estimation is performed, the position and the direction of the hot spot area relative to the microphone array are determined according to the position of the hot spot area, and the time delay from the hot spot area to each microphone in the microphone array is calculated. After calculating the time delay, the signals captured by each microphone are first adjusted according to the time delay from the target sound source direction and then weighted and summed to obtain the directionally enhanced sound signal.

The specific process comprises the following steps a to c:

a. calculating time delay:

Based on the expected direction of the sound source in the hot spot area and the geometrical layout of the microphone array, the expected time difference of arrival of the sound signal at the different microphones is calculated, which can be calculated from the speed of sound and the distance difference between the sound source and each microphone.

Specifically, for the sound source direction isFirst/>Delay of individual microphones/>Can be calculated from the direction of the sound source and the geometrical position between the microphones. The delay formula can be expressed as:

Wherein/> Is sound source to/>Distance difference of individual microphones/(Is the speed of sound.

B. applying a time delay:

A corresponding time delay is applied to the signal captured by each microphone to ensure that the sound signals from the target direction are phase aligned at the time of synthesis.

C. Signal weighted sum:

and carrying out weighted summation on the signals after time delay. The weighting coefficients may be adjusted according to different design objectives, for example, the design objective may be to control the width or shape of the beam.

In the weighted summation, the signal of each microphone is adjusted according to the calculated delay before weighting, and then the final output signal is formed by the weighted summation. Output signalCan be given by:

Wherein, Is/>Signals of the microphones,/>Is the corresponding weighting coefficient,/>Is the number of microphones.

By this weighted summation, sounds from the target direction (i.e., the direction in which the hot spot area is located) are enhanced, while sounds from other directions cancel each other out due to phase mismatch.

After the time delay estimation is performed, a phase adjustment is performed. Specifically, to ensure that the sound signals from the target direction are in phase alignment in the microphone array, the signals for each microphone are phase adjusted accordingly.

3. Dynamic adjustment algorithm implementation

Updating in real time: and continuously updating the target direction of the beam formation according to real-time data provided by the thermal imaging camera so as to quickly update the beam when the position of the hot spot area changes.

Beam pointing: the beam is directed to the hot spot area by adjusting the phase and gain of each microphone in the microphone array.

4. Gain control

Equalizing gain: the dynamically adjusted beam is directed to the hot spot area, and meanwhile, the gain is adjusted to optimize signal reception, so that sound from the hot spot area is ensured to be effectively captured, and noise in other directions is restrained.

The target monitoring areas, which are shown as hot spot areas by analytical thermal imaging in this step, are then directed to monitor the voiceprint data of the target monitoring areas to capture important sound events that may be associated with these areas.

S12, collecting environment data of the target monitoring area.

The environmental data in this embodiment is mainly wind data, rain data, and lightning strike data.

The acquisition process of the environmental data comprises the following steps 1) to 3):

1) And acquiring wind data and/or temperature and humidity data of the target monitoring area by using a sensor.

In practical applications, when the above-mentioned expansion device includes, but is not limited to, the following: and when at least one of the acceleration sensor, the temperature sensor, the humidity sensor, the wind speed sensor, the thermal imaging sensor, the camera and the like is used, the voiceprint data acquisition and processing terminal can be matched with any one of the temperature sensor, the humidity sensor, the wind speed sensor and the camera to realize real-time monitoring of the running state of the equipment to be monitored.

Specifically, when the voiceprint data acquisition and processing device 100 is connected to a wind speed sensor, information of the wind speed sensor is acquired, specifically, wind speed and wind direction in the environment are detected, and data collected by the wind speed sensor is transmitted to the core processing board. Wherein, the data collected by the wind speed sensor can be transmitted to the core processing board in a wired or wireless mode.

When the operation state monitoring equipment is connected with the temperature and humidity sensor, the temperature and humidity data of the environment can be continuously monitored by the temperature and humidity sensor through acquiring the temperature and humidity of the environment acquired by the temperature and humidity sensor in practical application.

When the running state monitoring equipment is connected with the acceleration sensor, the acceleration of the equipment to be monitored, which is acquired by the acceleration sensor, is acquired.

2) And determining whether the current weather is a judging result of the preset weather or not based on the temperature and humidity data.

The preset weather in this embodiment is rainy weather and/or thunderbolt weather.

And when the rainy day is identified, the core processing board analyzes the temperature and humidity data and judges whether the current rainy weather is the rainy weather or not by using a specific algorithm. In general, a significant increase in humidity, a decrease in temperature, may indicate rainfall.

And when determining raining according to the temperature and humidity, forming a time sequence from the collected temperature and humidity data, and recording the change of the temperature and humidity along with time.

The collected data is data cleaned to remove outliers or erroneous readings. And then, carrying out standardization or normalization operation, and converting the data into a standard format for analysis.

In making rain identification, the threshold for humidity increase may be set based on a threshold determination, typically based on historical data or geographic specific climate characteristics. For example, a significant increase in relative humidity over a short period of time beyond a certain percentage point may indicate rainfall. While taking into account temperature data, as temperature changes may also affect humidity (e.g., a decrease in temperature typically results in an increase in relative humidity).

If the humidity increases beyond a predetermined threshold and the temperature change conforms to the general pattern of rainfall (e.g., temperature drop), then it is determined to be raining.

In the rainy day identification, the change of humidity and temperature can be analyzed in a fixed time window according to the analysis determination of the time window so as to identify the rainfall mode.

In addition, the rainy weather can be realized by a machine learning method, if the data volume is enough, a machine learning method (such as a decision tree, a support vector machine or a neural network) can be used for training a model, and whether the rainy weather is currently rainy or not is predicted based on historical data.

In machine learning, to improve accuracy, result verification and feedback are required, where verification refers to verifying the accuracy of an algorithm using actual meteorological data or field observations. The feedback adjustment refers to adjusting a threshold value or algorithm parameters according to the verification result so as to improve the prediction accuracy.

Besides determining whether raining or not through temperature and humidity, rainy days and thunder recognition can be performed through analyzing sounds collected by the microphone. Rain sounds often appear as uniform background noise, while thunder sounds are sudden, high-intensity noise events. These features can be used to distinguish rain, thunder and other types of sound.

The collected sound signals are subjected to feature extraction, and the features can capture the main characteristics of the sound. In practical application, we can choose different feature extraction modes according to different scenes, and the algorithm used in feature extraction can be as follows steps 1.1 to 6.1:

1.1, meier frequency cepstrum coefficient (MFCCs, mel Frequency Cepstral Coefficents)

The application is as follows: in rain analysis MFCCs is able to capture the basic texture and frequency distribution of sound.

The extraction process comprises the following steps: including frame segmentation, FFT (Fast Fourier Transform ), mel-filter bank processing, logarithmic energy conversion, DCT (Discrete Cosine Transform ), etc.

2.1, Energy and Power spectral Density

The application is as follows: analyzing the energy distribution of rain sounds, rains with different intensities often exhibit different energy characteristics.

The extraction process comprises the following steps: computing the short-time energy and power spectral density of the sound signal, especially in the low frequency region, rain sounds typically increase the energy distribution.

3.1 Zero crossing rate (Zero Crossing Rate)

The application is as follows: the zero crossing rate is useful in analyzing the dynamic and frequency characteristics of rain sounds, especially for light rain and heavy rain.

The extraction process comprises the following steps: the number of times the signal changes from positive to negative or vice versa is calculated in a short time frame.

4.1 Sound texture features

The application is as follows: texture features of rain sound such as smoothness, roughness, etc. are analyzed.

The extraction process comprises the following steps: a texture analysis method similar to that in image processing, such as a similar application of Gray-level Co-occurrence Matrix to sound signals, may be used.

5.1, Spectrum centroid (Spectral Centroid)

The application is as follows: the frequency distribution of rain sounds is typically concentrated in a lower frequency range, which can be reflected by the spectrum centroid.

The extraction process comprises the following steps: the "brightness" of the sound is analyzed by calculating the "center of gravity" of the spectrum.

6.1, Short-time frequency variation (spectra Flux)

The application is as follows: and the device is used for analyzing the frequency change condition of the rain sound along with time.

The extraction process comprises the following steps: the rate of change of the spectrum between successive frames is calculated.

After feature extraction of the collected sound signals in any of the above steps 1.1 to 6.1, a machine learning model (such as a deep neural network) may be combined to identify the main noise type in the current environment, such as distinguishing rain sound from thunder sound.

Machine learning model in training, a classification model is trained using sound sample features including rain, thunder, and noiseless environments, and a network of machine learning models may use a deep neural network. And then classifying and identifying by using the model, specifically, inputting the characteristics extracted from the collected sound signals based on any one of the above steps 1.1 to 6.1 into the trained model, so that rain sounds and thunder sounds can be identified in the real-time sound data.

In addition, for thunder identification, thunder has burst characteristics, and can be identified by using a short-time energy detection algorithm. For example, when the energy of the sound signal suddenly exceeds a preset threshold, it is determined as thunder noise. The preset threshold may be set according to actual requirements, and is not specifically limited herein.

3) And taking the wind data and a judging result of whether the wind data is the preset weather or not as environment data.

In this embodiment, wind noise in voiceprint data collected by a microphone needs to be suppressed mainly when the wind speed is high, and also needs to be suppressed when the rain noise and thunder noise are high, so that the obtained voiceprint data is only voiceprint data of equipment to be monitored, the noise influence of external environmental noise on the collected voiceprint data is avoided, and the data collection accuracy is improved.

S13, determining a voiceprint processing mode corresponding to the environment data according to the voiceprint acquisition influence factor corresponding to the environment data.

In this embodiment, the influence of the external environment on the voiceprint collection is considered, which is referred to as a voiceprint collection influence factor in this embodiment, and may be divided into a voiceprint collection influence factor corresponding to wind noise, a voiceprint collection influence factor corresponding to rain noise, and a voiceprint collection influence factor corresponding to thunder noise.

When the value of the voiceprint acquisition influence factor is large, the voiceprint data acquired by the microphone needs to be subjected to noise reduction processing, and at the moment, a voiceprint processing mode corresponding to the environment data needs to be determined.

In another implementation of the present invention, referring to fig. 8, step S13 may include the following S31 to S33:

S31, processing the environmental data by adopting a data statistics mode or calling a preset influence factor determining model to obtain the voiceprint acquisition influence factor.

Wherein the voiceprint acquisition impact factor characterizes an extent to which a current environment of the target monitoring area affects a voiceprint data acquisition process of the sound acquisition board 120.

In an actual environment, the main noise is wind noise, rain noise, and thunder noise, so in this embodiment, the voiceprint acquisition influence factors of wind noise, rain noise, and thunder noise are analyzed.

Taking wind sound as an example, if the wind sound is large, in the voiceprint data collected by the microphone, the wind sound may exceed the sound of the equipment to be monitored, and the influence degree is large, and the value of the voiceprint collection influence factor is large. If wind noise is smaller, wind noise may not exceed the sound of the equipment to be monitored in the voiceprint data acquired by the microphone, the influence degree is small, and the value of the voiceprint acquisition influence factor is smaller. Rain sounds are similar to thunder sounds.

In specific implementation, voiceprint acquisition influence factors of wind noise, rain noise and thunder noise can be calculated respectively.

When the voiceprint acquisition influence factor of wind noise is calculated, after the core processing board receives wind speed data, the core processing board starts to analyze potential influence of wind speed on sound acquisition in the current environment. This analysis process may involve comparing wind speed data with a pre-set wind noise impact model to determine the specific impact of wind speed on the sound signal, e.g., the higher the wind speed, the greater the impact on a particular frequency range may be. The implementation process comprises the following 1.1) to 3.1):

1.1 Data normalization: the collected wind speed data is normalized for comparison with a predetermined model.

2.1 Threshold value setting): the system may set different wind speed thresholds that correspond to different levels of wind noise impact.

3.1 Wind speed class determination: and comparing the real-time wind speed data with a preset wind speed threshold value, and judging the current wind speed level.

When the threshold value comparison is performed, a preset wind noise influence model can be used, and the wind noise influence model can be as follows:

experience model: models based on historical data and experience are built, which reflect typical changes in sound signals at different wind speeds.

Physical model: the influence of wind speed and wind direction on sound wave propagation is considered based on a model established by an acoustic principle.

Statistical model: the relationship between wind speed and sound signal characteristics (e.g., amplitude, frequency) is analyzed using a model built by data statistics methods (e.g., regression analysis).

Machine learning model: patterns of wind speed effects on sound signals can be learned from a large amount of data using models trained by machine learning algorithms (e.g., neural networks).

Frequency domain analysis model: and analyzing the change of the frequency domain characteristics of the sound signals under different wind speeds, such as the energy change of a specific frequency band.

The models in this embodiment need to be calibrated and verified by experiments and field tests before practical application, and as more data is accumulated, the models can be continuously optimized and updated to improve the accuracy of prediction and processing.

And comparing the real-time monitored wind speed data with the models, and evaluating the influence of the wind speed on the sound signals.

The voiceprint acquisition influence factor of wind sounds in this embodiment may be, for example, a wind speed level. Based on the analysis result of the model, the sound signal processing strategy is dynamically adjusted, such as changing parameters of a noise suppression algorithm, adjusting the frequency range of a filter, and the like.

When the voiceprint acquisition influence factors of rain and thunder are calculated, after the core processing board receives the rain and thunder data, the potential influence of the rain and the thunder on the sound acquisition in the current environment is analyzed. The implementation process is similar to the voiceprint acquisition influence factor determination process of wind noise, and the voiceprint acquisition influence factor can be obtained by using a data statistics mode or calling a preset influence factor determination model.

S32, determining a noise suppression parameter and a beam forming parameter corresponding to the environment data under the condition that the voiceprint acquisition influence factor is larger than a preset threshold value.

Wherein the preset threshold comprises a static threshold or a dynamic threshold adjusted in real time based on noise conditions.

In practical application, the static threshold is a fixed threshold set based on experience, the dynamic threshold is a threshold continuously adjusted according to the noise condition of the actual environment, for example, when the dynamic threshold is set at the wind speed of 12, wind noise is considered to influence the sound collection of the equipment to be monitored, at the moment, when the wind speed is more than or equal to 12, noise suppression is performed, but analysis of the suppressed voiceprint finds that the wind noise in the voiceprint still affects the voiceprint recognition of the equipment to be monitored, at the moment, the dynamic threshold can be set at the wind speed of 9, and then the steps are repeated, so that the threshold is continuously adjusted according to the environment.

Under the condition that the voiceprint acquisition influence factor is larger than a preset threshold, the influence of noise of the external environment is larger, noise reduction processing is needed at the moment, and meanwhile, beam forming processing can be performed. At this time, it is necessary to determine a noise suppression parameter and a beam forming parameter corresponding to the environmental data.

When the external noise is wind noise, the noise suppression parameter is a parameter in a noise suppression algorithm, and the dynamic noise suppression algorithm can be an adaptive noise suppression algorithm or a frequency selective filtering algorithm. The main purpose of the adaptive noise suppression algorithm is to reduce or eliminate the influence of background noise on the sound signal and improve the definition and quality of the signal. Such algorithms are particularly suitable for dynamic or varying noise environments, for example in case of high wind noise.

The adaptive noise suppression can dynamically adjust parameters of the noise suppression algorithm according to wind speed information. For example, the adaptive noise suppression algorithm may be a modified spectral subtraction, whose basic formula is as follows:

Wherein, Is the processed signal spectrum,/>Is the original signal spectrum,/>Is the noise spectrum,/>Based on wind speed/>And frequency/>Dynamically adjusted noise suppression coefficients.

When the adaptive noise suppression algorithm is used, initial noise evaluation is firstly carried out, and the system monitors environmental noise including wind noise and other background noise through a microphone array or other sensors. The collected ambient noise is then analyzed to determine its characteristics, such as primary noise frequency, intensity, etc.

Then frequency selective filtering is performed, and the implementation process is frequency filtering. Frequency filtering refers to the system first applying frequency selective filtering (e.g., band reject filter) to specifically remove noise in a particular frequency range based on noise characteristics. This step is primarily directed to those known and stable noise frequencies, which effectively reduces the noise component of the signal at known frequencies, providing a cleaner signal for subsequent processing.

In this embodiment, the noise suppression parameter corresponding to the environmental data includes the specific frequency range.

The frequency selective filtering algorithm can be realized by a band-stop filter, and in practical application, a dynamically adjusted band-stop filter is designed and is specially used for a frequency band most seriously affected by wind noise. The cut-off frequency of the filter may be dynamically adjusted based on wind speed data.

The implementation process of the band-stop filter comprises the following steps 1.2 to 4.2:

1.2 determination of frequency Range

Wind noise characteristic analysis: it is first necessary to analyze the influence of wind noise on the sound signal, in particular to determine the frequency range in which the wind noise influence is most severe. This is typically done by experimental data or field measurements.

2.2 Selection of Filter types

Band-reject filter design: a suitable band reject filter type is selected. Common types of band reject filters include Butterworth, chebyshev and elliptic filters. Each type of filter has its own unique frequency response characteristics.

3.2 Setting of Filter parameters

Cut-off frequency: the cut-off frequency of the filter is determined. This includes an upper cut-off frequency and a lower cut-off frequency, which together define the frequency range of the filter block.

And (3) order selection: the order of the filter determines the sharpness of the filter. Higher order filters may provide steeper cut-off edges but may result in signal distortion.

4.2 Dynamic adjustment mechanism

Wind speed adjustment: the design mechanism enables the cut-off frequency of the filter to be dynamically adjusted according to wind speed data. This may involve real-time data processing and adaptive algorithms to ensure that the filter is able to respond in time to environmental changes.

Control algorithm: a control algorithm is developed to calculate the required cut-off frequency from the wind speed data and adjust the filter parameters accordingly.

The noise suppression parameters corresponding to the environmental data in the present embodiment include a cut-off frequency and a filter parameter.

After the noise suppression parameters are determined, the beamforming parameters also need to be determined. The beam forming can be performed with directivity adjustment, the microphone array is used for performing directivity beam forming, and the beam direction is adjusted according to the wind direction and the wind speed data so as to reduce the wind noise. Beamforming may be achieved by adjusting the phase of the microphone signals such that the signals from a particular direction are phase aligned and mutually enhanced.

The implementation process of the beam forming algorithm comprises the following steps 1.3 to 4.3:

1.3 acquisition of wind direction and wind speed data

Environmental monitoring: first, wind speed and direction in the environment are monitored in real time using wind speed and direction sensors.

And (3) data transmission: these data are transmitted in real time to the core processing board.

2.3 Design of beamforming algorithms

Beam forming principle: beamforming allows signals in a particular direction to be enhanced while signals in other directions are suppressed by adjusting the phase and amplitude of each microphone in the array.

Algorithm selection: suitable beamforming algorithms are selected according to the application requirements, such as Delay and Sum (Delay-and-Sum), minimum variance distortion free response (MVDR, minimum Variance Distortionless Response), linear constraint minimum variance (LCMV, linearly Constrained Minimum Variance), etc.

3.3 Wind noise impact analysis

Wind noise characteristic evaluation: and according to the wind speed and the wind direction data, evaluating the influence of wind noise in different directions. High wind speeds may result in increased noise from a particular direction.

Directivity influence model: a model is built to describe how wind noise changes with changes in wind speed and direction.

4.3 Dynamic adjustment of beam direction

And (3) self-adaptive adjustment: according to the wind noise influence model, the direction of the wave beam is dynamically adjusted, and the relative delay among the microphones in the microphone array is changed so as to optimize the wave beam direction.

And (3) real-time control: real-time control logic is implemented to quickly respond to environmental changes.

The beam forming parameters in this embodiment include the direction of the beam.

In addition to the noise suppression parameters and beamforming parameters that may be determined by algorithms described above, they may also be determined by a deep learning model.

Deep learning models, such as deep neural networks, can optimize sound signal processing, and the obtained models can be trained in wind noise environments to learn, identify and suppress wind noise characteristics. When the model is trained, sound samples (sound features which need to be marked at different wind speeds are not limited to energy and frequency changes) at different wind speeds can be input, so that the model learns the sound features under various wind noise conditions, and noise suppression parameters and beam forming parameters are determined based on the sound features.

When the external noise is rain, for continuous rain background noise, the noise suppression algorithm needs to focus on reducing the influence of the background noise, while preserving the definition of the running sound signal or other important signals of the device to be monitored. The noise suppression algorithm may use improved spectral subtraction for noise suppression to avoid over-suppression of the target sound signal (i.e., the operating sound signal of the device to be monitored).

Where the external noise is thunder, thunder is sudden strong noise, a more aggressive noise suppression algorithm may be employed to momentarily increase the strength of noise suppression, or a threshold-based detection algorithm may be used to temporarily handle these sudden noise events. If a short-time energy detection algorithm can be used, when the energy of the sound signal suddenly exceeds a preset threshold, it is determined as thunder noise, and the noise suppression intensity is temporarily increased.

In this embodiment, it is necessary to dynamically adjust the noise suppression parameter according to the identified noise type (rain sound or thunder sound).

In addition, thunder and rain sounds can be suppressed by an adaptive noise suppression algorithm, and the algorithm can dynamically adjust parameters according to the noise characteristics analyzed in real time. For example, the intensity of noise suppression, attack time (ATTACK TIME), release time (RELEASE TIME), etc. may be adjusted in real time. The algorithm needs to monitor the ambient noise continuously and adjust the noise suppression parameters in real time according to the noise variation.

In this embodiment, the noise suppression parameter corresponding to the environmental data needs to be determined, which may be specifically implemented according to an algorithm internal processing logic.

In the case where the external noise is rain or thunder, the microphone array is used to collect sound, and the influence of the rain and thunder can be minimized by adjusting the beam forming parameters. The implementation process may include the following steps 1.4 to 5.4:

1.4 Sound source localization

Target sound source identification: the direction of the target sound source is determined using sound source localization techniques. Common sound source localization techniques include time difference localization (TDOA, time Difference of Arriva) and direction of arrival (DOA, direction of Arrival) estimation. The target sound source comprises equipment to be monitored and/or a target monitoring area in the equipment to be monitored.

2.4 Beamforming algorithm selection

Selecting a proper algorithm: suitable beamforming algorithms are selected, such as Delay and Sum (Delay-and-Sum), minimum variance distortion-free response, or linear constraint minimum variance.

3.4 Adjusting beamforming parameters

And (3) direction adjustment: and adjusting the direction of the wave beam according to the sound source positioning result to lead the wave beam to be directed at the target sound source.

Beam width control: the width of the beam is adjusted. A narrow beam may better suppress noise from other directions, but may reduce the flexibility of sound source capture.

And (3) adaptive filtering: for dynamic environments, the beamforming parameters are adjusted in real-time using adaptive filtering techniques.

4.4 Special handling of rain and thunder

Frequency response adjustment: since rain and thunder are generally more pronounced in the low frequency ranges, noise in these frequency ranges can be further suppressed by adjusting the frequency response of the beamforming.

Dynamic tracking: if the position of thunder or rain changes, the beam pointing is adjusted using a dynamic tracking algorithm to continuously minimize its impact.

5.4 Post-treatment

Noise cancellation: after beamforming, the signal may be further cleaned up using additional noise cancellation algorithms.

Signal enhancement: the captured signal is subjected to enhancement processing to improve the quality of the target sound (i.e., the operating sound signal of the device to be monitored).

In this embodiment, the beam forming parameters corresponding to the environmental data need to be determined, which may be specifically implemented according to an algorithm internal processing logic.

In addition to noise reduction suppression and beamforming using the algorithms described above, deep learning models, such as deep neural networks, may be used to optimize noise suppression. The models can be trained in a rainy environment, learn to recognize characteristics of rain sounds and adjust processing strategies accordingly. During model training, sound samples under different temperature and humidity conditions are input, so that the model can learn sound characteristics under various weather conditions, and effectively distinguish rain sounds and other important sound signals, so that a noise reduction scheme is trained.

S33, using a noise suppression algorithm corresponding to the noise suppression parameter and a beam forming algorithm corresponding to the beam forming parameter as a voiceprint processing mode corresponding to the environment data.

In this embodiment, the noise suppression algorithm and the beamforming algorithm are used to perform voiceprint processing.

S14, processing the initial voiceprint data according to the voiceprint processing mode to obtain target voiceprint data.

Specifically, the noise suppression algorithm is used for carrying out noise reduction processing on the initial voiceprint data, and the beam forming algorithm is used for carrying out signal phase alignment and enhancement operation in a specific direction on the initial voiceprint data to obtain target voiceprint data.

Specific implementation processes of the noise suppression algorithm and the beam forming algorithm refer to the corresponding descriptions above, and are not repeated here.

S15, determining the equipment running state of the equipment to be monitored based on the target voiceprint data.

In this embodiment, when the running state analysis of the device is performed, the feature processing may be directly performed on the target voiceprint data, so as to obtain the running state of the device.

After the microphone array receives the acquisition instruction issued by the core processing board, the microphone array acquires the voiceprint signal of the equipment to be monitored, and transmits the acquired voiceprint signal to the core processing board.

After the core processing board receives the acoustic signals, filtering the acoustic signals to obtain a filtering result; performing time domain analysis and frequency domain analysis and/or audio analysis on the filtering processing result to obtain an analysis result; based on the analysis result, the current running state of the equipment to be monitored is determined by comparing the current running state with various fault information stored in a pre-training mode, and therefore real-time monitoring of the running state of the equipment to be monitored is achieved.

According to the technical scheme, the pre-training and storing of various fault information is used for indicating the fault types of the equipment to be monitored, and the fault types can include, but are not limited to, at least one of fault types corresponding to all components of the equipment to be monitored, abnormal sound frequencies under all fault types, time domain signal characteristic values and/or frequency domain signal characteristic values corresponding to the abnormal sound frequencies and the like.

The result of the time domain analysis may be as follows:

Waveform characteristics: the shape, amplitude, periodicity, etc. characteristics of the waveform may reflect the fundamental characteristics of the signal.

Peak analysis: the maximum and minimum values of the signal, which helps identify incidents or anomalies.

Mean and variance: the average level and fluctuation of the signal was used to evaluate its stability.

Time domain statistical parameters: such as mean, variance, standard deviation, skewness, kurtosis, etc., which describe the statistical properties of the signal over time.

Time interval characteristics: for periodic signals, analyzing the time interval between events helps identify periodicity and frequency.

Pulse width and interval: for pulse signals, analyzing the width and spacing of the pulses helps identify the signal pattern.

The frequency domain analysis results may be as follows:

Frequency components: the individual frequency components contained in the signal and their relative intensities.

Main frequency: the frequency of maximum energy in the signal represents the main characteristic of the signal.

Harmonic analysis: the harmonic content of the signal and its intensity relative to the fundamental are used to identify periodic noise or vibration.

Power spectral density (PSD, power SPECTRAL DENSITY): the power distribution of the signal at different frequencies is displayed, helping to identify the dominant frequency and the energy distribution.

Spectral characteristics: the characteristics of the width, shape and the like of the frequency spectrum can be used for identifying the complexity and the fluctuation of the signal.

In utilizing the analysis results of time domain analysis and/or frequency domain analysis, in the field of industrial equipment monitoring, these analysis results can be used to:

Identifying a failure mode: for example, imbalance, bearing damage, gear wear, etc. faults typically manifest themselves in certain frequency or time characteristics.

Trend analysis: the long-term monitoring data can reveal the trend of the change of the equipment performance and predict potential faults.

Performance evaluation: by analyzing the characteristics of the signal, the operating efficiency and health of the device can be assessed.

Illustratively, after the core processing board receives the acoustic signal, filtering the acoustic signal to obtain a filtering result; performing time domain analysis and frequency domain analysis on the filtering processing result to obtain a time domain signal characteristic value and a frequency domain signal characteristic value; based on the time domain signal characteristic value and the frequency domain signal characteristic value, searching in various fault information which is pre-trained and stored, determining whether a fault type corresponding to the time domain signal characteristic value and the frequency domain signal characteristic value exists, and when the fault type exists, determining that the current running state of the equipment to be monitored is abnormal, generating fault log information based on the fault type, the position information of the equipment to be monitored, the name of the equipment to be monitored and the like, so as to timely maintain the equipment to be monitored based on the fault log information.

In practical applications, the running state analysis of the equipment to be monitored may be inaccurate by only using voiceprint data, for example, the running state analysis may be affected by other noise, so that the result identification is inaccurate. Therefore, in this embodiment, data fusion may be used in addition to the operation state analysis performed by using voiceprint data alone, and the operation state analysis result of voiceprint data and other sensor data may be used to perform data fusion at the same time, so as to obtain the final operation state of the device.

At this time, if the device operation data includes not only the initial voiceprint data, a first device operation sub-state of the device to be monitored is determined based on any sensor data other than the initial voiceprint data in the device operation data.

In this embodiment, any sensor data other than the initial voiceprint data may include, but is not limited to, at least one of vibration signal data collected by a vibration sensor, acceleration signal data collected by an acceleration sensor, temperature signal data collected by a temperature sensor, a thermal imaging image collected by a thermal imaging sensor, gas signal data collected by a gas sensor, and the like.

Taking any sensor data as a thermal imaging image as an example, combining sound characteristics and thermal imaging data, and identifying specific sound events or anomalies, such as equipment faults, safety problems and the like, wherein the implementation process is as follows:

first, based on thermal imaging data, a first device operation sub-state of the device to be monitored is determined. In this embodiment, thermal imaging data may be analyzed by a pattern recognition algorithm, such as a machine learning or deep learning model, to identify hot spot areas, which may be indicative of device operating status, personnel activity, or other heat sources. Thermal imaging features of the thermal map, such as temperature distribution, hot spot size and shape, etc., are extracted. Thereby determining a first device operational sub-state based on the correspondence of the thermal imaging signature to the fault.

And then, determining a second equipment operation sub-state of the equipment to be monitored based on the target voiceprint data.

Specifically, data preprocessing, such as denoising, gain adjustment, and normalization, is first performed on the directionally-enhanced sound signals obtained from the microphone array. The spectral characteristics of the sound signal are then calculated, for example by extracting the frequency components using a fast fourier transform, or by calculating the energy distribution of the sound signal using mel-frequency cepstrum coefficients, such as the distribution of energy in different frequency bands. And further, determining a second equipment operation sub-state of the equipment to be monitored according to the analyzed characteristics, wherein the implementation process can be realized by referring to the corresponding implementation.

It should be noted that, for known operating conditions, pattern recognition algorithms, such as machine learning or deep learning models, may be used to identify acoustic events or anomalies. The training dataset of the model contains the acoustic and thermal imaging features of various known events. For unknown operating conditions, an anomaly detection algorithm may be applied to identify the anomaly patterns of the sound and heat map features. Events are classified according to the identified patterns, such as normal operation, equipment failure, safety issues, etc. Based on the identified information, corresponding responsive actions are taken, such as sending an alarm, recording an event, or adjusting the system operating state.

In addition, in data processing, it is necessary to ensure time synchronization of sound data and thermal imaging data in order to accurately match and analyze events.

And finally, carrying out data fusion on the first equipment operation sub-state and the second equipment operation sub-state to obtain the equipment operation state of the equipment to be monitored.

And performing association analysis on the first equipment operation sub-state and the second equipment operation sub-state. For example, a temperature change in a hot spot region may be correlated to a particular acoustic event (e.g., a mechanical device failure sound), based on which a run sub-state is determined, and the voiceprint process is similar. If the first equipment operation sub-state and the second equipment operation sub-state are both characterized by the same operation state, and the accuracy of determining the operation state is higher, the first equipment operation sub-state or the second equipment operation sub-state is directly used as the equipment operation state of the equipment to be monitored.

If the first device operating sub-state is different from the second device operating sub-state, the final device operating state may be determined in combination with other sensor data, or may be manually selected.

After determining the device operational status, a report corresponding to the device operational status may be generated, possibly including the type of sound event, location, time, and associated thermal imaging data.

In this embodiment, the voiceprint data and the thermal imaging data are fused, and in addition, the acceleration data and the voiceprint data can be fused. Specifically, an acceleration sensor is installed on the equipment to be monitored and used for capturing the motion data of the equipment in real time. The acceleration sensor continuously monitors the acceleration change of the device, providing detailed information about the state of motion of the device.

Acceleration data can be transmitted to the core processing board in a wired or wireless mode, and the selected transmission mode depends on system configuration and working environment, so that stability and instantaneity of data transmission are ensured.

The core processing board receives and processes the acceleration data and the sound data, and comprises preprocessing steps such as filtering, denoising and the like.

As above, the microphone array collects the voiceprint signals of the device according to the instructions of the core processing board when the device is operated, and the collected voiceprint signals are transmitted to the core processing board for further analysis. It should be noted that, when data fusion is performed based on acceleration, a beam forming technology may be used to focus on acquiring an equipment area with abnormal acceleration when voiceprint data is acquired, and equipment abnormality is monitored in time.

Key features such as vibration frequency, sound spectrum, and the like are extracted from the acceleration data and the sound data. In addition, data statistics may be performed on the acceleration data, such as obtaining a maximum value, a median value, an average value, a mean square error, and the like, so as to extract as many features as possible from the acceleration data.

The acceleration data and sound data are then fused together to provide more comprehensive device status information.

The specific fusion process is as follows:

The method includes determining a first operation sub-state based on acceleration data (similar to the implementation process and thermal image, which are described in detail herein, and not described in detail herein), determining a second operation sub-state based on voiceprint data, and performing data fusion on the first device operation sub-state and the second device operation sub-state to obtain a device operation state of the device to be monitored, so as to identify an abnormal mode matched with a preset failure mode, which may indicate a device failure or other problems.

Based on the analysis of the device operating state, the core processing board may make corresponding decisions, such as issuing alarms, logging events, or adjusting device operating parameters.

In other implementations of the invention, voiceprint data and vibration data can be fused, voiceprint data and temperature data can be fused, voiceprint data, thermal imaging data and vibration data can be fused, voiceprint data, thermal imaging data and acceleration data can be fused, four kinds of implementation modes of fusion of voiceprint data, thermal imaging data, vibration data and acceleration data can be selected, and the specific selection of which sensor data to fuse can be configured according to practice.

In this embodiment, device operation data of a device to be monitored located in a target monitoring area is obtained, a voiceprint processing mode corresponding to environmental data is determined according to voiceprint acquisition influence factors corresponding to the environmental data, the initial voiceprint data is processed according to the voiceprint processing mode to obtain target voiceprint data, environmental noise in the acquired voiceprint data is eliminated, the obtained target voiceprint data is voiceprint data of the device to be monitored, and accuracy of determining the device operation state can be improved when determining the device operation state based on the target voiceprint data. In addition, in the invention, the sound collection plate for detecting the initial voiceprint data is arranged in the preset distance range of the equipment to be monitored in a non-contact installation mode, compared with the mode of directly attaching the sound collection plate to the equipment to be monitored, the influence of the operation of the equipment to be monitored on the collection of the voiceprint data by the sound collection plate can be avoided, the voiceprint data collection accuracy is further improved, and the accuracy of the operation state determination of the follow-up equipment is improved.

On the basis of the embodiment of the device operation state monitoring method, another embodiment of the present invention provides a device operation state monitoring apparatus, referring to fig. 9, which may include:

a data acquisition module 11, configured to acquire device operation data of a device to be monitored located in a target monitoring area; the device operation data at least comprises initial voiceprint data; the sound collection plate for detecting the initial voiceprint data is arranged in a non-contact installation mode within a preset distance range of the equipment to be monitored;

a data acquisition module 12, configured to acquire environmental data of the target monitoring area;

the processing mode determining module 13 is configured to determine a voiceprint processing mode corresponding to the environmental data according to a voiceprint acquisition influence factor corresponding to the environmental data;

The voiceprint processing module 14 is configured to process the initial voiceprint data according to the voiceprint processing manner to obtain target voiceprint data;

And the state determining module 15 is used for determining the equipment operation state of the equipment to be monitored based on the target voiceprint data.

Further, the method also comprises a region determining module for:

and acquiring thermal imaging data of the equipment to be monitored, carrying out anomaly analysis on the thermal imaging data to screen out the anomaly thermal imaging data, and determining an area of the equipment to be monitored corresponding to the anomaly thermal imaging data, wherein the area is used as a target monitoring area.

Further, the data acquisition module 11 is specifically configured to:

and adjusting the beam forming parameters of the sound acquisition board according to the abnormal thermal imaging data so as to mainly enhance the voiceprint acquisition of the target monitoring area and obtain initial voiceprint data.

Further, the data acquisition module 12 is specifically configured to:

Acquiring wind data and/or temperature and humidity data of the target monitoring area by using a sensor; based on the temperature and humidity data, determining whether the current weather is a judging result of a preset weather; and taking the wind data and a judging result of whether the wind data is the preset weather or not as environment data.

Further, the processing manner determining module 13 includes:

the factor determination submodule is used for processing the environmental data by adopting a data statistics mode or calling a preset influence factor determination model to obtain a voiceprint acquisition influence factor; the voiceprint acquisition influence factor characterizes the influence degree of the current environment of the target monitoring area on the voiceprint data acquisition process of the sound acquisition board;

The parameter determination submodule is used for determining a noise suppression parameter and a beam forming parameter corresponding to the environment data under the condition that the voiceprint acquisition influence factor is larger than a preset threshold value; the preset threshold value comprises a static threshold value or a dynamic threshold value which is adjusted in real time based on noise conditions;

And the mode determining submodule is used for taking a noise suppression algorithm corresponding to the noise suppression parameter and a beam forming algorithm corresponding to the beam forming parameter as a voiceprint processing mode corresponding to the environment data.

Further, the voiceprint processing module 14 is specifically configured to:

And carrying out noise reduction processing on the initial voiceprint data by using the noise suppression algorithm, and carrying out signal phase alignment and enhancement operation in a specific direction on the initial voiceprint data by using the beam forming algorithm to obtain target voiceprint data.

Further, the state determination module 15 includes:

A first state determining sub-module, configured to determine, when the device operation data includes not only the initial voiceprint data, a first device operation sub-state of the device to be monitored based on any sensor data in the device operation data other than the initial voiceprint data;

A second state determining sub-module, configured to determine a second device running sub-state of the device to be monitored based on the target voiceprint data;

And the third state determining sub-module is used for carrying out data fusion on the first equipment operation sub-state and the second equipment operation sub-state to obtain the equipment operation state of the equipment to be monitored.

The working process of each module and sub-module in this embodiment is described with reference to the corresponding description in the above embodiment.

On the basis of the embodiment of the method and the device for monitoring the running state of the equipment, another embodiment of the invention provides equipment for monitoring the running state of the equipment, which can be a core processing board as described above, and comprises: a memory and a processor;

Wherein the memory is used for storing programs;

In another implementation manner of the present invention, based on an embodiment of the device operation state monitoring device, another embodiment of the present invention provides a device operation state monitoring system, including:

The specific structure implementation and description of the equipment operation state monitoring system refer to the corresponding description.

The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for monitoring the operating state of a device, comprising:

acquiring equipment operation data of equipment to be monitored, which is positioned in a target monitoring area; the device operation data at least comprises initial voiceprint data; the sound collection plate for detecting the initial voiceprint data is arranged in a non-contact installation mode within a preset distance range of the equipment to be monitored; collecting equipment operation data when the target monitoring area with abnormal sensor signals of equipment to be monitored exists;

collecting environment data of the target monitoring area; the environment data comprise wind data and judging results of whether the wind data are preset weather or not;

determining a voiceprint processing mode corresponding to the environmental data according to voiceprint acquisition influence factors corresponding to the environmental data; the voiceprint acquisition influence factor characterizes the influence degree of the current environment of the target monitoring area on the voiceprint data acquisition process of the sound acquisition board; the voiceprint processing mode is dynamically adjusted along with the environmental data;

Determining the equipment running state of the equipment to be monitored based on the target voiceprint data;

the determining the voiceprint processing mode corresponding to the environmental data according to the voiceprint acquisition influence factor corresponding to the environmental data comprises the following steps: processing the environmental data by adopting a data statistics mode or calling a preset influence factor determining model to obtain a voiceprint acquisition influence factor;

Determining a noise suppression parameter and a beam forming parameter corresponding to the environmental data under the condition that the voiceprint acquisition influence factor is larger than a preset threshold; the preset threshold value comprises a dynamic threshold value adjusted in real time based on noise conditions;

Taking a noise suppression algorithm corresponding to the noise suppression parameter and a beam forming algorithm corresponding to the beam forming parameter as a voiceprint processing mode corresponding to the environment data;

When the external noise is wind noise, the noise suppression algorithm dynamically adjusts parameters of the noise suppression algorithm according to wind speed information; the formula of the noise suppression algorithm includes: ，/> Is the processed signal spectrum,/> Is the original signal spectrum,/>Is the noise spectrum,/>Based on wind speed/>And frequency/>Dynamically adjusted noise suppression coefficients.

2. The apparatus operation state monitoring method according to claim 1, wherein the determining of the target monitoring area includes:

acquiring thermal imaging data of the equipment to be monitored;

Performing anomaly analysis on the thermal imaging data to screen out anomalous thermal imaging data;

and determining the area of the equipment to be monitored corresponding to the abnormal thermal imaging data, and taking the area as a target monitoring area.

3. The apparatus operation state monitoring method according to claim 2, wherein acquiring the apparatus operation data of the apparatus to be monitored located in the target monitoring area includes:

4. The apparatus operation state monitoring method according to claim 1, wherein collecting environmental data of the target monitoring area includes:

Acquiring wind data and temperature and humidity data of the target monitoring area by using a sensor;

Based on the temperature and humidity data, determining whether the current weather is a judging result of a preset weather;

And taking the wind data and a judging result of whether the wind data is the preset weather or not as environment data.

5. The method for monitoring an operation state of a device according to claim 1, wherein processing the initial voiceprint data according to the voiceprint processing mode to obtain target voiceprint data comprises:

6. The apparatus operation state monitoring method according to claim 1, wherein determining the apparatus operation state of the apparatus to be monitored based on the target voiceprint data comprises:

determining a first equipment operation sub-state of the equipment to be monitored based on any sensor data except the initial voiceprint data in the equipment operation data under the condition that the equipment operation data not only comprises the initial voiceprint data;

Determining a second equipment operation sub-state of the equipment to be monitored based on the target voiceprint data;

and carrying out data fusion on the first equipment operation sub-state and the second equipment operation sub-state to obtain the equipment operation state of the equipment to be monitored.

7. An apparatus for monitoring the operation state of a device, comprising:

The data acquisition module is used for acquiring equipment operation data of equipment to be monitored, which is positioned in the target monitoring area; the device operation data at least comprises initial voiceprint data; the sound collection plate for detecting the initial voiceprint data is arranged in a non-contact installation mode within a preset distance range of the equipment to be monitored; collecting equipment operation data when the target monitoring area with abnormal sensor signals of equipment to be monitored exists;

The data acquisition module is used for acquiring the environmental data of the target monitoring area; the environment data comprise wind data and judging results of whether the wind data are preset weather or not;

The processing mode determining module is used for determining a voiceprint processing mode corresponding to the environment data according to the voiceprint acquisition influence factors corresponding to the environment data; the voiceprint acquisition influence factor characterizes the influence degree of the current environment of the target monitoring area on the voiceprint data acquisition process of the sound acquisition board; the voiceprint processing mode is dynamically adjusted along with the environmental data;

the state determining module is used for determining the equipment operation state of the equipment to be monitored based on the target voiceprint data;

the processing mode determining module is specifically configured to:

processing the environmental data by adopting a data statistics mode or calling a preset influence factor determining model to obtain a voiceprint acquisition influence factor;

8. An apparatus operation state monitoring apparatus, characterized by comprising: a memory and a processor;

the memory is used for storing programs;

A processor invokes a program and is configured to perform the device operating condition monitoring method of any one of claims 1-6.

9. A system for monitoring the operational status of a device, comprising:

The voiceprint data acquisition and processing device comprises an acquisition shell and a voiceprint acquisition and processing assembly, wherein the voiceprint acquisition and processing assembly is arranged in the acquisition shell and is used for acquiring and processing voiceprint information; the voiceprint acquisition processing assembly comprises the device operating condition monitoring device of claim 8;