JP4861723B2 - Monitoring system - Google Patents

Description

  The present invention relates to a monitoring system in which a surveillance area is photographed by a surveillance camera and voices in and around the surveillance area are recorded so that the surveillance area can be monitored through a communication cable, the Internet, a mobile phone network, or the like.

  Conventionally, there are monitoring systems that monitor abnormalities in the monitoring area with video and audio. When the sound volume level is higher than a predetermined threshold, such as when there is a noise or a loud sound, the monitoring camera is activated. It is what you do. Referring to FIG. 10, the optical unit 2 is mounted on the camera unit 1, and the camera unit 1 has a camera control circuit unit 4 for processing an image signal from the CCD 3 and an overall control circuit unit 5 for controlling the entire camera. The information from the sound sensor 8 and the optical sensor 9 is input to the overall control circuit unit 5, and the overall control circuit unit 5 is in an abnormal state only when the sound sensor 8 detects a sound volume above a certain level. And the camera control circuit unit 4 is controlled to take a picture. The video imaged in the event of an abnormality is displayed as a video image on a monitor 7 connected to the camera unit 1 via the cable 6, and the status of the monitoring area can be monitored. (For example, see Patent Document 1)

Japanese Patent Laying-Open No. 2003-333583 (paragraphs [0014] to [0017] of the specification, drawing 1)

  However, the conventional monitoring system starts shooting the monitoring area when the volume level of the sound that can be heard in the monitoring area exceeds a predetermined threshold, and the monitor constantly monitors the monitor in order to detect an abnormality. However, the monitoring side only has to determine what kind of content or cause the audio information is from on the monitor image. In other words, it may be an impact sound when a simple object is dropped, or it may be activated by a sound caused by a factor that is not recognized as abnormal, and the image is projected. The cause of the sound generated in or around the area cannot be grasped, and although it is not abnormal, the image is frequently projected with noise, which causes annoyance to the surveillance staff. In addition to video, it was not a monitoring system that could detect abnormal situations by voice.

  The present invention has been made in view of the above-described problems, and provides a monitoring system capable of identifying an audible sound in and around a monitoring area in addition to an image of the monitoring area and warning an abnormality. The purpose is to provide.

The present invention has been made to achieve the above object, and the invention of claim 1 is a monitoring system for recording a captured image of a camera that captures a surveillance area in a monitoring device.
The monitoring device is provided with a microphone for picking up sound in the monitoring area, and includes an image processing means that performs image processing on a video signal output from the camera to form image data, and a monitor that displays the image data. ,
The monitoring device has a learning function based on a teacher signal from a feature database, compares and analyzes the sound picked up by the microphone and phoneme data, recognizes the phoneme of the sound, and time-series data of the phoneme A speech / non-speech recognition means using a time-delay neural network,
Abnormal voice / abnormal sound identification means for recognizing predetermined voice / non-voice from the time-series data of the phonemes by the voice / non-voice recognition means and determining whether or not the alarm is an abnormal voice or abnormal sound When,
An image storage means for storing an abnormal occurrence image data at the time of detecting an abnormal voice / abnormal sound in the monitoring area and image data before and after the abnormal sound / abnormal sound when the abnormal sound / abnormal sound identifying means detects an abnormality;
Abnormal voice character data output means for converting abnormal voice / abnormal sound from the abnormal voice / abnormal sound identification means into warning character data;
Abnormal voice output means for notifying the abnormal voice / abnormal sound by voice synthesis;
First sending means for adding the warning character data to the abnormality occurrence image data obtained from the image storage means and sending it to the network;
The abnormality occurrence image data with the warning character data added thereto is converted to image data that can be received by a mobile phone terminal by image size conversion means, and is sent to a network;
The monitoring system characterized by notifying the cellular phone terminal of the abnormality occurrence image data and the preceding and following image data when an abnormality occurs due to sound in the monitoring area from the second sending means via a network. .

The invention according to claim 2 is the monitoring system according to claim 1, wherein the non-speech is an abnormal sound such as a scream or a noise .

According to a third aspect of the present invention, the monitoring device includes time data transmission means, adds the warning character data to the abnormality occurrence image data, and based on time information from the time data transmission means, an abnormality occurrence time corresponding to the position information of a monitoring system according to claim 1 or 2, characterized that you output in addition to the abnormality image data.

  The speech / non-speech recognition means is not limited to a neural network, but the neural network includes a feature database unit for storing phoneme characteristic parameters, and a teacher signal is transmitted from the feature database unit to the neural network. In addition to improving the discrimination efficiency, it is possible to increase the optimal voice / non-speech discrimination efficiency to be warned according to the location where the surveillance camera is installed. preferable.

In the invention of claim 1, in the monitoring system for recording the captured image of the camera that captures the monitoring area in the monitoring device,
The monitoring device is provided with a microphone for picking up sound in the monitoring area, and includes an image processing means that performs image processing on a video signal output from the camera to form image data, and a monitor that displays the image data. ,
The monitoring device has a learning function based on a teacher signal from a feature database, compares and analyzes the sound picked up by the microphone and phoneme data, recognizes the phoneme of the sound, and time-series data of the phoneme A speech / non-speech recognition means using a time-delay neural network,
Abnormal voice / abnormal sound identification means for recognizing predetermined voice / non-voice from the time-series data of the phonemes by the voice / non-voice recognition means and determining whether or not the alarm is an abnormal voice or abnormal sound When,
An image storage means for storing an abnormal occurrence image data at the time of detecting an abnormal voice / abnormal sound in the monitoring area and image data before and after the abnormal sound / abnormal sound when the abnormal sound / abnormal sound identifying means detects an abnormality;
Abnormal voice character data output means for converting abnormal voice / abnormal sound from the abnormal voice / abnormal sound identification means into warning character data;
Abnormal voice output means for notifying the abnormal voice / abnormal sound by voice synthesis;
First sending means for adding the warning character data to the abnormality occurrence image data obtained from the image storage means and sending it to the network;
The abnormality occurrence image data with the warning character data added thereto is converted to image data that can be received by a mobile phone terminal by image size conversion means, and is sent to a network;
The monitoring system characterized by notifying the cellular phone terminal of the abnormality occurrence image data and the preceding and following image data when an abnormality occurs due to sound in the monitoring area from the second sending means via a network. Therefore, the voice / non-speech recognition means detects non-speech and voice phonemes such as security buzzer sounds and screams, and the time-sequential output phoneme is recognized as abnormal by the abnormal voice / abnormal sound identification means. It is possible to determine whether the sound is non-voiced or non-voiced, and a warning message to notify the abnormality on the monitor screen can be displayed superimposed on the video at the time of the abnormality, which recognizes the abnormality rather than just a video There is an advantage that it is easy to perform, and there is an advantage that the efficiency of the monitoring work can be improved.

Further, in the invention of claim 1, wherein the monitoring device is the abnormal sound / abnormal sound have a sound output means for notifying by speech synthesis, warning not only displayed on the video, audio warning information for notifying the abnormality Can be notified, and there is an advantage that it is easier to recognize an abnormality than a simple video.

According to a first aspect of the present invention, the monitoring device includes a first sending unit that sends the warning character data to the network by superimposing the warning character data on the image data, and recognizes an abnormality caused by voice or non-voice. It is possible to distribute the abnormality to the Internet, and there is an advantage that a specific person such as a monitor can monitor the abnormal state of the monitoring area in a remote place.

Further, in the invention of claim 1, wherein the monitoring device, the second to be sent to the network is converted by the image size conversion unit the image data obtained by superimposing the warning text data into image data mobile phone terminal can receive comprises a delivery means, an abnormality of the monitored zone it is possible to deliver to the mobile phone network via the Internet, the specific person, such as supervisor, abnormal by the mobile phone terminal even away There is an advantage that the state can be confirmed.

The invention according to claim 2 is the monitoring system according to claim 1 , wherein the non-speech is an abnormal sound such as a scream or a noise, and only an abnormality due to the sound is determined by a phoneme by a neural network. In addition, since abnormal sounds can be identified, there is an advantage that they can be used for various types of monitoring. Further, according to the monitoring area monitoring device that sets a camera and microphone installed in, it is possible to monitor multiple surveillance zone on a computer or a cellular phone terminal or the like at a remote location via a network, voice or non-voice When an abnormal situation is recognized, there is an advantage that a warning text can be distributed to the Internet together with the video and a specific person such as a monitor can check even if it is away from the abnormality in the monitoring area, and the monitoring efficiency is improved. There is an advantage of being good .

It is to be noted that the voice / non-voice recognition means, because it is a time delay neural network, representing the vigilance matters to be monitored by surveillance zone "It's a fire, burglar, help, Kya" phonemes such as keywords and crime prevention buzzer sound such as By learning, it is possible to cope with the monitoring items in various monitoring areas, and there is an advantage that a versatile monitoring system can be provided. Further, in the present invention since the abnormality information monitoring area is distributed over the Internet, there is an advantage that can monitor the monitoring area from a remote location.

According to a third aspect of the present invention, the monitoring device includes a time data transmission unit, superimposes the warning character data on the image data, and based on time information from the time data transmission unit, The monitoring system according to claim 1 or 2 , wherein an abnormality occurrence date and time is superimposed on the image data and output in correspondence with the position information. There is an advantage that can be specified).

  Hereinafter, a monitoring system according to the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing an embodiment of the present invention, FIG. 2 is a functional block diagram of a monitoring device, and FIG. 3 is a block diagram of a monitoring system. FIG. 4 is a diagram of a neural network showing an example of a voice / non-voice recognition processing unit. FIG. 5 is an explanatory diagram for explaining a method of selecting an image when an abnormality occurs, and FIG. 6 is a diagram showing a display screen of a mobile phone. 7 is a block diagram showing another embodiment of the present invention, FIG. 8 is a functional block diagram of the monitoring apparatus, and FIG. 9 is a block diagram of the monitoring system.

(Embodiment 1)
With reference to FIGS. 1-6, the monitoring system of one Embodiment of this invention is demonstrated. As shown in FIG. 1, in the monitoring system of this embodiment, a microphone M and a camera C are connected to a monitoring device 10, and the monitoring device 10 is connected to a client computer PC via the Internet N. The mobile phone terminal T is connected to the mobile phone network T via the mobile phone network D. Mike M picked up sounds around ₁ or in the monitoring zone E, the camera C is photographed in the surveillance area E _1, and the audio and video information is fed to the monitoring device 10, the monitoring person, the client computer monitoring the surveillance zone E ₁ a PC or a mobile phone T can be monitored by the video and audio information. The monitoring device 10 is installed in each of a plurality of monitoring areas E _{1 to} En, and the microphone M and the camera C are connected to the monitoring device 10 via a communication cable. Of course, there may be one or a plurality of monitoring areas monitored by the monitoring device 10, and the microphone M may be integrated with the camera C, or may be arranged adjacent to or separated from the camera C. Good.

  The monitoring device 10 determines whether or not the sound captured by the microphone M is a voice or non-voice related to the monitoring item, and if the sound is a voice or non-voice related to the monitoring item, a situation to be warned (occurrence of an alarm signal) ), The warning character data is output, the image at the time of the alarm is selected, the warning character data is superimposed on this image, and the warning character data is converted into voice and warned. The monitoring device 10 also has an upload function (sending means) for distributing a warning display video in which warning character data is superimposed on the screen to the Internet.

The monitor can monitor the uploaded warning display video on the monitor by the client computer PC via the Internet N, and is uploaded by connecting the Internet N and the mobile phone network D. The warning display video can be displayed on the display screen (display) of the mobile phone terminal T to monitor each of the monitoring areas E _{1 to} En from a remote location. Needless to say, the client computer PC and the mobile phone terminal T cannot confirm the warning display video on the network unless the personal identification can be performed by the password and password or biometric authentication. Also, it is desirable to install a router or form a firewall so that the network cannot be invaded from the outside.

  Further, the monitoring device 10 will be described with reference to the functional block diagram of FIG. First, the audio signal processing system will be described. The monitoring device 10 is provided with a CPU (Central Processing Control Device), a voice / non-voice recognition means 11 for digitally processing a sound picked up in a monitoring area by a microphone M and recognizing a phoneme, and a voice / non-voice. And a control processing unit 12 to which the phoneme time-series data output by the recognition unit 11 is input. The control processing unit 12 determines how the phoneme time-series data obtained from the speech / non-speech recognition unit 11 is used. Audio for detecting the abnormal content / abnormal sound identifying means 12a and the abnormal voice / abnormal sound identified by the abnormal voice / abnormal sound identifying means 12a as audio data to drive the speaker S It has functions of an abnormal voice data sending means 12b for outputting information, a time data sending means 12d for specifying the date and time when the abnormality occurred, and the abnormal voice data sending means. Abnormal voice output means 13 for outputting abnormal voice data from 2b as a voice signal, and abnormal voice character data sending means for outputting the abnormal voice / abnormal sound identified by the abnormal voice / abnormal sound identification means 12a as a warning character data signal 14. The abnormal sound / abnormal sound identification means 12a may be processed before the control processing unit 12.

  The above phonemes will be described later. In the above voice / non-voice, the voice is a voice of a person who can be understood as a meaning, for example, a drool, a burglar, a fire, a fire, a help, indicating an abnormal state. Non-speech is a meaningless person's voice or sound, screaming, screaming (baby crying, etc.), security buzzer sound, slap (glass breaking sound, vehicle crashing sound) , Sound that breaks things). When an abnormality due to voice / non-voice is recognized, the abnormal voice / abnormal sound recognition processing information is notified to the monitor by driving the speaker S, but the character displayed by the voice or warning character data signal driving the speaker S is In the case of voice, I recognized the voice of "Drobbing, robber, fire, fire, help, stop, etc." Please deal with it as soon as possible. In the case of non-speech, etc., "" Security buzzer sound, scream, etc. "was recognized. Please deal with it as soon as possible. Etc.

  Next, the video signal processing system will be described. The monitoring apparatus 10 includes an image processing unit 15 that converts a video signal from the camera C into a digital signal and performs compression processing, and an image that repeatedly stores image data compressed by the image processing unit 15 for each frame or field image. Storage means 16, video output means 17 for expanding the frame or field image stored in image storage means 16 and converting it into an NTSC video signal, and sending the abnormal voice character data to the video signal from video output means 17 The video composition means 18 for superimposing the warning text based on the warning character data signal from the means 14 and outputting it to the monitor W, and the frame or field image A stored in the image storage means 16 from the abnormal voice character data sending means 14 Within a predetermined image size that can be displayed on the mobile phone terminal display shown in FIG. Image size conversion means 19 for converting the image to fit within, and distribution of warning display video in which warning information (alarm occurrence date / time, location of alarm occurrence, warning text) is superimposed on the video to the Internet N and the mobile phone network via the Internet N And sending means 20 for uploading to a web server (not shown).

  As shown in FIG. 6, the display of the mobile phone terminal is composed of display screens A and B. The display screen A is a display area for displaying a monitoring image. This is the area to be displayed. On the display screen B, alarm occurrence date / time, alarm occurrence location, and abnormal voice / abnormal sound recognition processing information are displayed as abnormality information. The screen size of the mobile phone is a maximum of 240 pixels (horizontal) × 320 pixels (vertical) QVGA (Quarter Video Graphics Array) size, and the effective size is 240 pixels (horizontal) × 224 pixels (vertical). is there. The frame or field image (display screen) A is converted by the image size conversion means 19 so as to be within this image size.

  Next, the present embodiment will be described in detail with reference to FIGS. A microphone M and a camera C are connected to the monitoring device 10 via a communication cable, and a sound captured by the microphone M and a video of a monitoring area captured by the camera C are recorded on a VTR, HDD recorder, or DVD recorder RE. Yes. In addition, a monitor W is connected to the monitoring device 10 and an input device such as a keyboard is connected via the serial signal processing unit 21, and the monitoring device 10 can be controlled by this input device.

  The voice / non-speech recognition means 11 includes a voice / non-speech capturing unit 11a that amplifies voice / non-speech sound picked up from or around the monitoring area by the microphone M and converts it into a digital signal by the A / D converter. A short period of preprocessing for speech recognition, such as using a band filter group of about 10 to 30 channels for digitized speech / non-speech, or directly calculating speech spectrum using fast Fourier transform (FFT). A speech analysis unit 11b that performs temporal spectrum analysis to decompose speech / non-speech into phonemes, and a speech recognition processing unit 11c that receives time-series data of phonemes obtained from the speech analysis unit 11b and recognizes speech / non-speech. And a feature database 11d that stores phoneme characteristic parameters and supplies them to the speech recognition processor 11c as teacher signals. In the voice analysis unit 11b, the non-voice is analyzed into a phoneme by performing a short-time spectrum analysis similar to that of the voice. Similar to consonants and vowels, non-speech also has a characteristic pattern.

  The speech recognition processing unit 11c is configured by, for example, a known time delay neural network (TDNN), FIG. 4 shows an outline of the structure diagram, and TDNN recognizes a phoneme which is a unit of sound recognition. Then, speech / non-speech can be recognized from the time-series data of the phoneme. The speech recognition processing unit 11c is classified into each phoneme group, and a TDNN is provided for each group. Time series data of phonemes obtained by performing spectrum analysis on speech / non-speech from the speech analysis unit 11b is input to the input layer I of each TDNN. In the first and second hidden layers H1 and H2, the phonemes are identified and the phonemes identified from the output layer O are output. There are a total of 24 Japanese phonemes (b, d, g, p, t, k, m, n, N, s, sh, h, z, ch, ts, r, w, y, a, i, u, e, o, Q (silence)), phonemes are classified into vowels and consonants, and further divided into voiced and unvoiced consonants. Voiced consonants include burst sounds, friction sounds, flow sounds, and nasal sounds. There are sounds, etc., and unvoiced consonants include plosives, rubbing sounds, frictional sounds, etc., and a TDNN is provided for each group. In the speech recognition processing unit 11c, not only speech but also non-speech is recognized. In the voice analysis unit 11b, non-voices such as a security buzzer and a scream are decomposed and output as phonemes in addition to the voices. Therefore, the voice recognition processing unit 11c recognizes non-voices not only from voices but also from non-voice characteristic patterns. be able to.

  The TDNN in FIG. 4 exemplifies a group for recognizing consonant / b / d / g, and its structure is an input layer (Input Layer) I, first and second hidden layers (Hidden Layer). A multilayer perceptron type composed of H1, H2 and an output layer (Output Layer) O is formed. This type of TDNN is characterized in that it can perform back-propagation (Back-Propagation) learning that reversely propagates an error from the teacher signal based on the teacher signal from the feature database 11d. In the TDNN of FIG. 4, in the input layer I, the horizontal axis represents the time axis and the vertical axis represents the frequency axis. The time axis on the horizontal axis is a frequency pattern of 15 frames obtained by performing spectrum analysis every 10 milliseconds. The frequency axis on the vertical axis divides 0 to 6000 Hz into 16 bands (16 channel spectrum), Time series data (15 frames × 16 dimensions = 240 points), which is a characteristic component of phonemes for 15 frames, is input to the input layer I. In the first hidden layer H1, feature detectors (elements) for detecting local feature components of three frames (30 milliseconds) of the input layer I are arranged. In the second hidden layer H2, feature detectors for detecting more global feature components of 5 frames (50 milliseconds) of the first hidden layer H1 are arranged. The output layer O can recognize the phoneme “b” by using the sum of the output values of the second hidden layer H2 in the time direction as an output. Also, other phoneme groups including vowels can be recognized by the same TDNN. Incidentally, in the speech recognition processing unit 11c, it is not necessary to prepare TDNNs for all phoneme groups, and TDNNs for recognition target phoneme groups may be prepared.

  As described above, in the speech recognition processing unit 11c, the time series data of phonemes is given to the input layer I of the TDNN, and the comparison evaluation is repeatedly performed based on the self-evaluation or the phoneme data input from the feature database unit 11d. A weighting amount between each layer is set, a recognition process of each phoneme (vowel / consonant) is performed, a recognition result of the phoneme is output from the output layer O, and a time-series phoneme (consonant, vowel) from the output layer O ) Is output, and voice / non-voice can be recognized later. The voice recognition processing unit 11c uses known software such as TDNN, a device, or the like.

  The control processing unit 12 has a function of performing arithmetic processing by the CPU. The control processing unit 12 will be described with reference to the functional block diagram of FIG. The voice / non-voice information output from the voice recognition processing unit 11c is input to the abnormal voice / abnormal sound identification unit 12a. The abnormal voice / abnormal sound identification means 12a recognizes the voice / non-voice input information, and can understand the voice / non-voice to be warned in each monitoring area, and can understand the abnormal voice / abnormal sound information (abnormal voice: Drobo, Recognize as abnormal sound information (security buzzer sound, scream, etc.) such as fire, help, stop, etc., and send out voice identification information and alarm signal (abnormal condition detection signal) for voice synthesis and text conversion.

  On the other hand, the time data sending means 12d receives the date / time information of the year, month, day, and time from the time generating section 12c, and from the time data sending means 12d by an alarm signal based on the abnormal voice information from the abnormal voice identifying means 12a. Date and time information is output. As will be described later, this date and time information is output as a warning display video by being superimposed or added to a video signal in a predetermined output format together with an alarm occurrence location and audio identification information (warning information). This warning display image is displayed on the monitor W and on the screen of the client computer PC or the mobile phone terminal T via the Internet N.

  The voice identification information from the abnormal voice data sending means 12b is input to the voice synthesis processing section 13a, an analog voice is created by the existing voice synthesis means, and the synthesized analog signal is amplified by the voice output section 13b. Output to the speaker S. Further, the voice identification information from the abnormal voice data sending means 12b is input to the character data output unit 14, and the character data output unit 14 converts the voice identification information into character data based on the voice identification information. The data is output to the server unit 19 and the network processing unit 20, respectively.

  Next, the video processing system will be described. The image processing means 15 includes a video capturing unit 15a and a video processing unit 15b. In the video capturing unit 15a, an analog such as NTSC which is a video of the monitoring area captured by the camera C is used. The video processing unit 15b decodes the NTSC analog video signal, A / D-converts and compresses it into digital compressed data, and stores it in the image memory (image recording means) 16. Send it out.

  The image memory 16 includes a plurality of frame or field memories, and digital compressed data that is a monitoring video is repeatedly overwritten and recorded on the plurality of frame or field memories. The video output processing unit 17 reads the compressed video data recorded in the image memory 16, decompresses it, encodes it into NTSC, sends it to the character signal synthesis unit (video synthesis unit) 18, and sends it to the character signal synthesis unit (video synthesis unit). In 18, a warning text or the like is superimposed on the video signal and output to the monitor W. Further, in the character signal synthesis unit 18, warning information indicating an abnormal state of the monitoring area, information on occurrence date and time, information necessary for monitoring such as the monitoring area (location) is superimposed on the video signal as character information and output to the monitor W. .

In the image memory 16, as shown in FIG. 5, at the time t ₁ when the abnormal sound / abnormal sound identification unit 12 a starts the abnormality recognition process (when the alarm signal is generated) or when the abnormality recognition process is completed, the control processing unit 12, the overwriting process to the image memory 16 is stopped, and the recorded image F ₀ at the time when an abnormality occurs retroactive to the continuous recorded image recognition processing time t ₀ is processed by the video output processing unit 17, After passing through the signal synthesizer 18, it is output to the monitor W as a video or an image when an abnormality occurs. Further, the control processing unit 12 can output a plurality of images before and after the recorded image F ₀ when the occurrence of the abnormality is detected, by the mobile phone terminal T or the client computer PC of the monitoring device 10.

  The image conversion server unit (image size conversion means) 19 converts the video signal into an image format (JPEG or the like) that can be displayed on the screen of the mobile phone terminal T to a size that can be displayed in the image size (QVGA size or the like). Information necessary for monitoring, such as status information, time of occurrence, and monitoring area (location), is added as character information to a predetermined format and sent to the network processing unit (network sending means) 20.

  The network processing unit (transmission means) 20 uses the alarm information, the date and time of occurrence, the information necessary for monitoring, such as the monitoring area (location), as character information, and the image at the time of occurrence of the alarm as the client computer PC or mobile phone terminal. It has a function of delivering (uploading) to T as an output format that can be displayed on a WWW browser via the Internet N.

  The monitoring device 10 is connected to the Internet N via the router R, the client computer PC is connected to the Internet N via the router R, and the mobile phone network D to which the mobile phone terminal T is connected is the router. The client computer PC and the mobile phone terminal T are connected to the Internet N via R, and can monitor even if they are away from the monitoring area. The router has a function to analyze packet data that flows on the network from the protocol, determine which route should be used and transfer it, and also has a function to set the data transmission route. It has the function of not passing any data and is effective in enhancing the security of the system.

(Embodiment 2)
Next, another embodiment of the present invention will be described with reference to FIGS. FIG. 7 is a block diagram showing another embodiment of the present invention, FIG. 8 is a functional block diagram of the other embodiment, and FIG. 9 is a block diagram of a monitoring system of the other embodiment.

  As shown in FIG. 7, this embodiment differs from the above embodiment in that a camera C and a microphone M are installed in each monitoring area, and the camera C and the microphone M receive sound and video captured in the monitoring area on the Internet. Sending means 22 for sending to N is provided. The sending means 22 has a function of converting the video and audio of each monitoring area into a form that can be searched and viewed with a WWW browser and automatically uploading it to a Web server. The camera C is a so-called network camera. It is. In this embodiment, the monitoring device 10 does not need to be installed in the monitoring area, but is installed in a place away via the Internet N, which is different from the above embodiment. The mobile phone terminal T is the same as that in the above embodiment. The description of the same parts as those in the above embodiment is omitted as much as possible.

  The voice / non-voice recognition unit 11 acquires the voice / non-speech picked up by the microphone M as a packet signal received from the network sending unit 20, and the contents of the voice / non-speech by the packet signal are as follows. When this voice or non-speech is recognized as an abnormal voice, the abnormal voice / abnormal sound is input to the abnormal voice / abnormal sound identifying means 12a of the control processing unit 12 and described in the above embodiment. As described above, the abnormal voice / abnormal sound is converted into voice or warning character information and notified by the speaker S, or a warning display video as character information superimposed on the monitoring video is displayed on the monitor W. Similarly, a warning display image can be displayed on the mobile phone terminal T. Further, the image acquisition means 15a 'acquires a digitized image signal, records the image signal in the image storage means 16, and is superposed with warning character information through the same image processing as in the above embodiment.

  In this embodiment, a pair of cameras and microphones are installed in each surveillance area, and the client computer PC or mobile phone terminal via the Internet N only when the sound and image from these cameras detect abnormal sound or sound. It is possible to monitor with warning display character information and images at T, and it is not necessary to install a monitoring device in each monitoring area as compared with the above embodiment, and many monitoring areas can be monitored with a small number of people.

  As described above, the present invention recognizes the abnormalities occurring in the monitoring area by using the neural network to recognize the extracted features of voice and non-voice generated in the field, and the recognition result indicates abnormalities. This is used as a trigger to warn the monitor of abnormality. Moreover, the recognition result of the neural network based on the extracted features is converted into character information, added to the monitoring image at the time of occurrence of the abnormality, and distributed. The monitoring image recorded at the time of occurrence of the abnormal state recorded by the monitoring image is sent to the monitoring monitor of a dedicated supervisor, and at the same time, the client computer PC or mobile phone terminal located away from the monitoring location is connected to the Internet or a mobile phone network. The monitoring image can be distributed via the network.

  In the present invention, when monitoring a monitoring area, the content of abnormal sound in an abnormal state differs depending on whether it is a fire monitoring, an intruder monitoring, or a monitoring system for nursing purposes. Therefore, the voice / non-voice recognition means must recognize different abnormal voices or abnormal sounds, and by providing the voice / non-voice recognition means with a neural network, a versatile monitoring system is provided by learning. can do.

  As an example of use of the present invention, even if the monitor is away from the monitoring area, an abnormal sound is automatically displayed as a warning character in addition to the monitoring video or image to automatically identify the abnormal situation. It can be used as a monitoring system that can be recognized and monitored, and can be used as a monitoring system corresponding to various abnormal situations.

It is a block diagram which shows one Embodiment of the monitoring system of this invention. It is a functional block diagram of the monitoring apparatus of this embodiment. It is a block diagram of the monitoring system of this embodiment. It is a figure of the neural network which shows an example of the phoneme recognition process part in this embodiment. It is explanatory drawing for demonstrating the method to select the image at the time of abnormality generation in this embodiment. It is a figure which shows the display screen of the mobile telephone in this embodiment. It is a block diagram which shows other embodiment of this invention. It is a functional block diagram of the monitoring apparatus of embodiment of FIG. It is a block diagram of the monitoring system of the embodiment of FIG. It is a block diagram which shows the conventional monitoring system.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Monitoring apparatus 11 Voice / non-voice recognition means 11a Voice / non-voice capture part 11b Voice analysis part 11c Voice recognition processing part 11d Feature database part 12 Control processing part 12a Abnormal voice / abnormal sound identification means 12b Abnormal voice data transmission means 12c Time Generation unit 12d Time data transmission means 13 Abnormal voice output means 13a Speech synthesis processing section 13b Audio output section 14 Abnormal voice character data transmission means 15 Image processing means 15a 'Image acquisition means 15a Video capture section 15b Video processing section 16 Image memory (image Recording means)
17 Video output processing unit (video output means)
18 Character signal synthesis unit (video synthesis means)
19 Image conversion server unit (image size conversion means)
20, 22 Network processing unit (transmission means)
21 serial signal processing unit C camera D mobile phone network E _{1 to} En monitoring area M microphone N Internet PC client computer R router RE VTR, DVD recorder, HDD recorder S speaker T mobile phone terminal W monitor

Claims (3)

In a surveillance system that records a photographed image of a camera that photographs a surveillance area in a surveillance device,
The monitoring device is provided with a microphone for picking up sound in the monitoring area, and includes an image processing means that performs image processing on a video signal output from the camera to form image data, and a monitor that displays the image data. ,
The monitoring device has a learning function based on a teacher signal from a feature database, compares and analyzes the sound picked up by the microphone and phoneme data, recognizes the phoneme of the sound, and time-series data of the phoneme A speech / non-speech recognition means using a time-delay neural network,
Abnormal voice / abnormal sound identification means for recognizing predetermined voice / non-voice from the time-series data of the phonemes by the voice / non-voice recognition means and determining whether or not the alarm is an abnormal voice or abnormal sound When,
An image storage means for storing an abnormal occurrence image data at the time of detecting an abnormal voice / abnormal sound in the monitoring area and image data before and after the abnormal sound / abnormal sound when the abnormal sound / abnormal sound identifying means detects an abnormality;
Abnormal voice character data output means for converting abnormal voice / abnormal sound from the abnormal voice / abnormal sound identification means into warning character data;
Abnormal voice output means for notifying the abnormal voice / abnormal sound by voice synthesis;
First sending means for adding the warning character data to the abnormality occurrence image data obtained from the image storage means and sending it to the network;
The abnormality occurrence image data with the warning character data added thereto is converted to image data that can be received by a mobile phone terminal by image size conversion means, and is sent to a network;
A monitoring system that notifies the mobile phone terminal of the abnormality occurrence image data and the preceding and following image data when an abnormality occurs due to sound in the monitoring area from the second sending means via a network . The monitoring system according to claim 1, wherein the non-voice is an abnormal sound such as a scream or a noise . The monitoring device includes time data transmission means, adds the warning character data to the abnormality occurrence image data, and generates an abnormality corresponding to the position information of the monitoring area based on the time information from the time data transmission means monitoring system according to claim 1 or 2, characterized that you output in addition to the abnormality image data date and time.

Images