JP4504216B2 - Image processing apparatus and image processing program - Google Patents

Description

  The present invention relates to an image processing apparatus and an image processing program for replacing an input image with an image composed of a collection of symbols and reproducing the replacement image as a moving image.

  In recent years, e-mail and the Internet have become widespread, and accordingly, so-called “emoticons” are widely used as a kind of text expression tool in e-mail and Internet bulletin boards. “Emoji” is text data in which an image simulating an expression is created by a combination of symbols (characters).

  Such “emoticons” can be stored in advance in the computer or obtained by downloading from a server, in addition to those created by a user combining symbols. The user appropriately selects such “emoticon” according to the expression content and attaches it to the sentence, thereby expressing his / her feelings and emotions in addition to the sentence expression. By attaching this “emoticon”, an emotion that cannot be transmitted only by the text is expressed, and advanced communication can be performed between the self and others, and the entertainment of the text can be improved.

Further, the “emoticon” is stored in association with the face image pattern, and the corresponding “emoticon” is stored in accordance with the facial expression of the user (face image pattern) captured by the camera. A system for inputting emoticons to be input in a sentence is disclosed (see Patent Document 1).
JP 2002-318649 A

  However, in the system disclosed in the above document, although an image can be attached according to the facial expression of the user captured from the camera, the “emoticon” can be shared by many unspecified people. It is an image that imitates an expression and does not reflect the characteristics of each user. For this reason, there is a problem that the expression using the “emoticon” becomes a uniform expression without individuality. Furthermore, since expressions such as “emoticons” are widespread, there is a problem that they lack freshness and it is difficult to enhance the interest of the viewer.

  On the other hand, a method of forming each image (so-called “ASCII image”) by replacing each part of the input image with a predetermined symbol (text data) according to the density is widely known. According to this, like the “emoticon”, the input image can be expressed by text data, and although the aspect is different from the input image, the expression reflecting the characteristics of the input image It can be carried out. However, such an image has a problem that the image quality lacking in clarity tends to be poor. Further, in the reproduction of moving images, there is a problem that the amount of data to be processed increases and the processing becomes complicated.

  The present invention has been made in order to solve the above-mentioned problems, and by creating a simple image reflecting the characteristics of an input image with symbol data and reproducing the moving image, the present invention has high entertainment. An object of the present invention is to provide an image processing apparatus and an image processing program capable of performing the above.

In order to achieve this object, an image processing apparatus according to claim 1 includes image data input means for inputting image data having a plurality of image frames and capable of displaying a continuous image, and the image data input means. An image processing unit comprising processing means for processing input image data for each image frame, and output means for outputting image data by sequentially outputting the image frames processed by the processing means in accordance with the input order And a display device that sequentially displays the image frames output by the output means of the image processing unit and displays the image data as a moving image. The image processing unit stores image data of a plurality of symbols. Symbol data storage means, and the processing means of the image processing unit converts the image data input by the image data input means into line image data. A line drawing conversion means for converting, a dividing means for dividing an image frame of the image data input by the image data input means or an image frame having the line image data converted by the line drawing conversion means into a predetermined number of blocks; Extraction means for extracting symbols assigned in accordance with the mode of the line image data from the symbol image data stored in the symbol data storage means in units of blocks divided by the division means, and the extraction And image forming means for forming replacement image data obtained by replacing the line image data of the corresponding block with the image data of the symbol extracted by the means, and the output means of the image processing unit includes the replacement image data and it outputs a replacement image data formed by, perform moving picture display of the replacement image data at the display device, the image The processing unit includes an imaging device that acquires image data by photographing as the image data input unit, and further includes a background image storage unit that stores background image data acquired as a background image by the imaging device, The processing means of the image processing unit removes unique image data unique to each image frame by excluding a common part with the background image data stored in the background image storage means from the image data acquired by the imaging device. A unique image data extracting means for extracting, and the line drawing conversion means of the image processing section converts the unique image data extracted by the unique image data extracting means into line image data, and the image of the image processing section The forming means forms replacement image data from line image data converted from the unique image data, and the image processing When the image frame is divided by the dividing means, the processing means of the section is configured such that the number of blocks having the line image data or the unique image data in the image frame is not more than a predetermined lower limit Image data expansion means for expanding the line image data or the unique image data in a direction to increase the number of blocks having the line image data or the unique image data in a frame, and forming an image in the image processing unit The means forms replacement image data based on the decompressed image data when the line image data or unique image data is decompressed by the image data decompressing means .

The image processing apparatus according to claim 2 is the image processing apparatus according to claim 1 , wherein the image data decompression unit is configured such that the unique image data extracted by the unique image data extraction unit with respect to an image frame from which the image is extracted is stored. A determination unit that determines whether or not the proportion of the specific image data is less than or equal to a predetermined proportion; The unique image data is expanded assuming that the number of blocks having the unique image data is equal to or less than a predetermined lower limit value.

The image processing apparatus of claim 3, wherein, in the image processing apparatus according to claim 1 or 2, the processing means of the image processing unit, in case of dividing the image frame by the dividing means, in the image frame When the number of blocks having the line image data or the unique image data is equal to or greater than a predetermined upper limit, the line image data or the unique image data is compressed, and the line image data or the unique image is included in the image frame. Image data compression means for reducing the number of blocks having data; and when the image data is compressed by the image data compression means, the image forming means of the image processing unit converts the image data into compressed image data. Based on this, replacement image data is formed.

An image processing apparatus according to a fourth aspect is the image processing apparatus according to any one of the first to third aspects, based on accompaniment information storage means for storing accompaniment information and accompaniment information stored in the accompaniment information storage means. Accompaniment generating means for generating accompaniment, sound data input means for inputting sound data, sound generating means for sounding sound data input by the sound data input means and accompaniment generated by the accompaniment generating means Lyric data storage means corresponding to each accompaniment information and storing lyric data displayed on the display device in accordance with the accompaniment sounded by the sounding means, and stored in the lyric data storage means The replacement image data output by the output means is displayed on the display device together with the lyrics data.

An image processing apparatus according to a fifth aspect is the image processing apparatus according to any one of the first to fourth aspects, wherein the image processing apparatus according to any one of the first to fourth aspects comprises the image processing unit according to any one of the first to fourth aspects. Audio data input means for inputting audio data, data conversion means for converting the audio data input by the audio data input means into a data signal output to the line, and image formation of the image processing unit A transmission-side device having a superimposing unit that converts an identification code corresponding to the replacement image data formed by the unit into a data signal of audio data and superimposes it on the data signal converted by the data conversion unit, The output unit of the image processing unit outputs the data signal superimposed by the superimposing unit to the line and outputs replacement image data to the counterpart device.

The image processing apparatus according to claim 6, wherein, in the image processing apparatus according to claim 5, wherein said transmitting side apparatus, a non-regenerative frequency components above the audio data input by the voice data input unit, is set as the non-reproducing area The superimposing means converts the identification code of the replacement image data into a data signal in the band of the non-reproduction frequency component removed by the removing means and converts the data signal by the data converting means. Is superimposed on the data signal.

The image processing apparatus according to claim 7, wherein, in the image processing apparatus according to claim 5 or 6, wherein the transmitting-side apparatus, the audio data inputted by said voice data input means, the limit frequency components exceeding the audible limit The superimposing means converts the identification code of the replacement image data into the data signal of the limit frequency component removed by the second removing means, and the data converting means It is superimposed on the converted data signal.

The image processing apparatus according to claim 8, wherein the receiving the image processing apparatus according to any one of claims 5-7, which is connected to the transmitting-side device through the line, and the display device, the audio data Voice data receiving means, voice playback means for playing back the voice data received by the voice data receiving means, and voice data received by the voice data receiving means in an identification code corresponding to the replacement image data. An image data reproducing means for reproducing the replacement image data from the data signal when a data signal is included, and the display device for displaying the replacement image data reproduced by the image data reproducing means A side device is provided.

The image processing program according to claim 9 is an image input by a computer having image data input means for inputting image data having a plurality of image frames and capable of displaying a continuous image. An image processing step including a processing step for processing data for each image frame, and an output step for outputting image data by sequentially outputting the image frames processed by the processing step according to the input order, is executed, The image frames output in the output step of the image processing step are sequentially displayed and the image data is displayed as a moving image on the display device of the computer. The computer stores image data of a plurality of symbols. Symbol data storage means, and a processing step of the image processing step. Is a line drawing conversion step of converting the image data input by the image data input section to the line image data, converted by the image frame or the line drawing conversion step of the image data input by the image data input means A division step of dividing an image frame having line image data into a predetermined number of blocks, and symbols assigned according to the mode of the line image data in units of blocks divided by the division step, the symbol data storage means An extraction step of extracting from the symbol image data stored in the image, and an image formation step of forming replacement image data in which the line image data of the corresponding block is replaced with the symbol image data extracted by the extraction step preparative to execute the computer, the output step of the image processing step, the location The replacement image data formed by the image data is output, the moving image of the replacement image data is displayed on the display device, and the computer captures image data by photographing as the image data input means A background image storage means for storing background image data acquired as a background image by the imaging device, and the processing step of the image processing step is based on the image data acquired by the imaging device. Excluding a common part with the background image data stored in the background image storage means, the computer executes a unique image data extraction step for extracting unique image data for each image frame, and the image processing step The line drawing conversion step converts the unique image data extracted by the unique image data extraction step. Converting to line image data, and the image forming step of the image processing step forms replacement image data by line image data converted from the unique image data, and the processing step of the image processing step includes When the image frame is divided by the dividing step, if the number of blocks having the line image data or the unique image data in the image frame is equal to or less than a predetermined lower limit value, the line in the image frame Causing the computer to execute an image data expansion step of expanding the line image data or the unique image data in a direction to increase the number of blocks having the image data or the unique image data, and the image forming step of the image processing step includes: The line image data is obtained by the image data expansion step. Others when specific image data is decompressed, and forms a replacement image data based on the image data after decompression.

An image processing program according to a tenth aspect of the present invention is the image processing program according to the ninth aspect , wherein the unique image data extracted by the unique image data extracting step is extracted from the original image frame. When the computer executes a determination step for determining whether the proportion is less than or equal to a predetermined proportion , and the determination step determines that the proportion occupied by the unique image data is less than or equal to a predetermined proportion The unique image data is expanded on the assumption that the number of blocks having the unique image data is equal to or less than a predetermined lower limit value.

  According to the image processing apparatus of the first aspect, the image data input by the image data input means is converted into line image data by the line drawing conversion means. Further, the image frame of the input image data by the dividing unit or the image frame having the line image data converted by the line drawing converting unit is divided into a predetermined number of blocks, and the line image data is divided into divided blocks. The symbol that approximates this aspect is extracted from the symbol image data stored in the symbol data storage means by the extraction means. Then, replacement image data obtained by replacing the line image data of the corresponding block with the extracted symbol image data is formed by the replacement image forming means. The formed replacement image data is output to the display device by the output means, and the replacement image data is displayed as a moving image on the display device.

  Therefore, it is possible to convert an input image into an image composed of symbol image data and output it, and to provide a comical and highly entertaining effect that a simple image acts according to its own operation. Can do. Furthermore, since the image data of the symbol is formed from the line drawing obtained by converting the input image, the output image can be a simple image, and the entire input image data is converted to the replacement image data composed of the symbol data. As compared with the above, there is an effect that the image displayed on the display device can be a simple and sharp image.

  In addition, there is an effect that the image displayed on the display device can be an image that is different from the input actual image data, but leaves the characteristic portion of the actual image. For example, when an image is output by replacing a person in the input image with a character prepared in advance (including “emoticon”), it is simply replaced with a character created in advance. The characteristics are difficult to be reflected. Also, a large amount of character data is required to reflect the features even a little. However, since the replacement image data is formed from the image data input using the symbol data, the output image on the display device does not become a uniform image, and the nuance of the individual image data is reflected. Can be realized.

Further, the unique image data is extracted by the unique image data extraction unit for each image frame, except for the common part with the background image data from the image data acquired by the imaging device. The replacement image data is formed from line image data obtained by converting the extracted unique image data.

  Accordingly, the replacement image data can be formed by omitting the background image portion, and the replacement image data can be formed at a higher speed than the case where the replacement image data is formed from all the image data. There is. Further, when the formed replacement image data is output to a display device or the like, the data output speed can be increased. Further, by removing the background image data, it is possible to reduce the complexity of the image data from which the replacement image data is converted. Therefore, simple replacement image data can be output to the display device, and it is possible to avoid unclear (difficult to identify) what the output image represents. In the case of complicated (complex) image data including background image data, the boundaries of individual objects may be unclear, and when such image data is converted into replacement image data, May appear in an image divided or connected into different shapes. However, since the replacement image data is formed from the line image data obtained by converting the unique image data (with the background image data removed), it is possible to reduce such problems and display simple and clear replacement image data on the display device.

Further, in case of dividing an image frame by dividing means, when the number of blocks is less than a predetermined lower limit value having a linear image data or unique image data in the image frame, the image data expansion means, the line image data or The line image data or the unique image data is expanded in the direction of increasing the number of blocks having the unique image data. Then, replacement image data is formed based on the decompressed image data.

  When acquiring image data with an imaging device, depending on the distance between the imaging device and the subject (corresponding to the unique image data), the size of the subject is (remarkably) smaller than the background image There is. In the image data in such a case, the number of blocks having unique image data (line image data) is reduced in the image frame. The replacement image data composed of a small number of symbol data is less likely to have the characteristics because the image representation of the fine part of the original image (the subject) is omitted, and in some cases, It may be just an array of symbols that cannot be recognized as an image representing the object.

  However, if the unique image data (line image data) is expanded, the number of blocks having the unique image data (line image data) in the image frame can be increased. In other words, since the replacement image data can be formed after enlarging the image to be able to express even a fine portion, the replacement image having the characteristics of the original image even if the unique image data of the input image data is small. There is an effect that data can be formed.

According to the image processing apparatus according to claim 2, wherein, in addition to the effects of the image processing apparatus according to claim 1, wherein, the ratio of specific image data extracted on the extracted original image frame is less than a predetermined ratio If the determination means determines that the number of blocks having unique image data is equal to or less than a predetermined lower limit value, the unique image data is expanded. Therefore, there is an effect that the specific image data to be expanded can be accurately selected.

  When the size of the unique image data is measured by the number of blocks (counted in units of blocks), 1 is counted even if no dots are arranged on the entire block. Therefore, when measuring the size of the unique image data in units of blocks, it is convenient, but the measurement result tends to shift to a larger side than the actual size, and in some cases, the unique image data that needs to be expanded There is a possibility that the situation will not be expanded. However, when the ratio of the extracted unique image data to the extraction source image frame is equal to or less than the predetermined ratio, the unique image data is decompressed, so that the unique image data that needs to be decompressed accurately is selected. It can be done.

According to the image processing device of the third aspect , in addition to the effect produced by the image processing device according to the first or second aspect, when the image frame is divided by the dividing means, the line image data or the unique image is included in the image frame. If the number of blocks having image data is greater than or equal to a predetermined upper limit, the line image data or unique image data is compressed by the image data compression means, and the number of blocks having line image data or unique image data in the image frame is determined. Decrease. The replacement image data is formed based on the compressed image data. Therefore, there is an effect that the replacement image data can be displayed on the screen of the display device with an appropriate size.

According to the image processing apparatus of the fourth aspect , in addition to the effect produced by the image processing apparatus according to any one of the first to third aspects, the sound data input by the sound data input means and the accompaniment generation means are generated. The accompaniment is pronounced by the pronunciation means. In addition, lyrics data corresponding to each accompaniment information is displayed on the display device in accordance with the accompaniment sounded by the sound generation means. Then, the replacement image data output by the output means is displayed on the display device together with the lyrics data. Therefore, there is an effect that it is possible to construct an environment for singing while viewing the displayed lyric data while matching the sounded accompaniment information, and it is possible to produce an effect using the replacement image data.

  Generally, an apparatus having an audio data input means, an accompaniment information generation means, a sound generation means, and a display device on which lyrics data is displayed is called “karaoke” and is used for entertainment for assisting a singer's singing. This is a device (function). Therefore, by providing a device having both the function of “Karaoke” and the function of displaying the replacement image data as a moving image on the display device, it is possible to add a new interest to “Karaoke” and improve its entertainment. be able to. Further, since the replacement image data is displayed on the display device together with the lyrics data, even if the replacement image data is displayed on the display device, the lyrics are not hidden, and it is possible to prevent the singer from being unable to sing. There is an effect.

According to the image processing apparatus of the fifth aspect , in addition to the effect produced by the image processing apparatus according to any one of the first to fourth aspects, the audio data input by the audio data input means is connected to the line by the data conversion means. It is converted into a data signal to be output. Further, the identification code corresponding to the replacement image data is converted into a data signal of audio data by the superimposing unit and is superimposed on the data signal converted by the data converting unit. Then, the superimposed data signal is output to the line by the output means, whereby the replacement image data is output to the counterpart device.

  Therefore, the replacement image data can be transmitted to the counterpart apparatus with an identification code whose data amount is significantly smaller than that of bitmap data or the like, and the data amount of the replacement image data to be transmitted can be reduced. . According to this, the replacement image data (identification code) can be sent to the counterpart device as part of the audio data (because the data amount is small). Further, since the identification code is sent to the counterpart device as a part of the voice data, the sending device only needs to be provided with a means for sending the voice data. There is no need to provide means for transmitting image data in the form of image data. Therefore, there is an effect that the apparatus cost can be reduced.

According to the image processing apparatus of the sixth aspect , in addition to the effect produced by the image processing apparatus of the fifth aspect , in the transmission side apparatus, the input audio data has a non-reproduction frequency component set as a non-reproduction area. It is removed by removing means. Then, the identification code of the replacement image data is converted into a data signal in the band of the non-reproduction frequency component removed by the removing unit, and is superimposed on the data signal converted by the data converting unit.

  Therefore, there is an effect that it is possible to make it non-executable that the identification code of the replacement image data is reproduced as audio data in the counterpart device that has received the audio data from the transmitting device. That is, since the replacement image data identification code is converted into the non-reproduction frequency component audio data that is not originally reproduced, even if the other device receives the audio data including the replacement image data identification code, it is reproduced. Only valid audio data (excluding the identification code of the replacement image data) to be reproduced is reproduced. Therefore, since the replacement image data is output as voice data as noise, it is possible to avoid a problem that it becomes impossible to hear the necessary voice data, and to make the receiver on the other device side feel uncomfortable. There is an effect that there is no. For this reason, it is not necessary to provide the other apparatus with a process and apparatus for reproducing the audio data by separating the identification code of the replacement image data from the received audio data, and the cost of the other apparatus can be reduced.

According to the image processing apparatus of the seventh aspect , in addition to the effect achieved by the image processing apparatus of the fifth or sixth aspect , the transmitting side apparatus exceeds the audible limit for the audio data input by the audio data input means. The marginal frequency component is removed by the second removing means. Then, the identification code of the replacement image data is converted by the superimposing unit into the data signal of the limit frequency component removed by the second removing unit and superimposed on the data signal converted by the data converting unit.

  Therefore, even if the audio data is reproduced with the replacement image data included in the counterpart device that has received the audio data from the transmission side device, the reproduced audio band exceeds the audible limit range. Thus, there is an effect that the noise caused by the replacement image code can be made extremely difficult to be heard by the receiver on the counterpart device side. In other words, the listener does not feel uncomfortable or uncomfortable that the hearability of the reproduced audio data is poor. Therefore, it is not necessary to provide the counterpart apparatus with a process and apparatus for reproducing the audio data by separating the identification code of the replacement image data from the received audio data, and the cost of the counterpart apparatus can be reduced.

  In particular, when an unspecified number of other-side devices are assumed, it is also sufficient that the other-side device is not provided with a means for displaying an image and is a device that only transmits and receives audio data (for example, a normal telephone). Although it is conceivable, even with such a device, it is possible to output sound with good quality without hindering reproduction of received sound data.

According to the image processing device of the eighth aspect , in addition to the effect of the image processing device according to any one of the fifth to seventh aspects, the counterpart device provided with the display device is provided. The voice data is received by the voice data receiving means. The received audio data is reproduced as audio by the audio reproducing means. If the received audio data includes a data signal of an identification code corresponding to the replacement image data, the replacement image data is reproduced from the data signal by the image data reproduction means. Then, the reproduced replacement image data is displayed on the display device.

  Therefore, in the counterpart device that has received the audio data including the identification code corresponding to the replacement image data, the replacement image data can be accurately output to the display device. For this reason, there is an effect that communication by an image having a taste different from that of a videophone that outputs an actual video can be performed between the transmission side device and the counterpart side device. The replacement image data displayed on the counterpart device is processed (deformed), not image data actually captured (input) on the transmitting device. That is, the counterpart device does not output the image data input by the transmitting device (in a state where the input image data is reproduced). Therefore, if the operator (speaker) of the transmission side apparatus does not like to be photographed by a camera or the like, or if it is not desired to transmit the actual video to the other side due to various circumstances, the actual video ( By using the replacement image data obtained by deforming the data) with the input image, the transmitter can transmit the contents to be transmitted to the other party. Here, the replacement image data is formed from the input image data and can reflect the characteristic part of the input image data. Even if the replacement image data is output instead of the image data), there is an effect that it is possible to perform high-level communication that accurately transmits the item that the user wants to express to the other party through the image.

According to the image processing program of the ninth aspect , the input image data is converted into line image data by the line drawing conversion step. Further, the image frame of the input image data or the image frame having the line image data converted by the line drawing conversion step is divided into a predetermined number of blocks by the dividing step, and the line image data is divided into divided blocks. The symbol that approximates the aspect is extracted from the image data of the plurality of symbols by the extraction step. Then, replacement image data in which the line image data of the corresponding block is replaced with the extracted symbol image data is formed by the replacement image forming step.

  Therefore, an input image can be converted into an image composed of symbol image data and output to a display device or the like, and an effect can be provided by a comical and highly entertaining image. Furthermore, since the image data of the symbol is formed from the line drawing obtained by converting the input image, the output image can be a simple image, and the entire input image data is converted to the replacement image data composed of the symbol data. As compared with the above, there is an effect that the image displayed on the display device can be a simple and sharp image.

  In addition, there is an effect that the image displayed on the display device can be an image that is different from the input actual image data, but leaves the characteristic portion of the actual image. For example, when an image is output by replacing a person in the input image with a character prepared in advance (including “emoticon”), it is simply replaced with a character created in advance. The characteristics are difficult to be reflected. Also, a large amount of character data is required to reflect the features even a little. However, since the replacement image data is formed from the image data input using the symbol data, the output image on the display device does not become a uniform image, and the nuance of the individual image data is reflected. Can be realized.

Also, the unique image data unique to each image frame is extracted by the unique image data extraction step, except for the common part with the background image data inputted as background image data from the inputted image data. The unique image data extracted by the unique image data extraction step is converted into line image data by the line drawing conversion step, and the replacement image data is formed from the line image data converted from the unique image data by the image forming step. .

  Therefore, since the replacement image data can be formed by omitting the background image portion, the replacement image data can be formed at a higher speed than the case where the replacement image data is formed from all the image data. effective. Further, when the formed replacement image data is output to a display device or the like, the data output speed can be increased. Further, by removing the background image data, it is possible to reduce the complexity of the original image data converted into the replacement image data. Therefore, when outputting to a display device, simple replacement image data can be output, and it is possible to avoid unclear (difficult to identify) what the output image represents. In the case of complicated (complex) image data including background image data, the boundaries of individual objects may be unclear, and when such image data is converted into replacement image data, May appear in an image divided or connected into different shapes. However, since the replacement image data is formed from the line image data obtained by converting the unique image data (with the background image data removed), it is possible to reduce such problems and display simple and clear replacement image data on the display device. It is.

Further, when an image frame is divided by the dividing step, if the number of blocks having line image data or unique image data in the image frame is equal to or less than a predetermined lower limit value, the line image data or unique image in the image frame Line image data or unique image data is expanded in the direction of increasing the number of blocks having data. Then, replacement image data is formed based on the decompressed image data.

  When acquiring image data with an imaging device, depending on the distance between the imaging device and the subject (corresponding to the unique image data), the size of the subject is (remarkably) smaller than the background image There is. In the image data in such a case, the number of blocks having unique image data (line image data) is reduced in the image frame. The replacement image data composed of a small number of symbol data is less likely to have the characteristics because the image representation of the fine part of the original image (the subject) is omitted, and in some cases, It may be just an array of symbols that cannot be recognized as an image representing the object.

  However, if the unique image data (line image data) is expanded, the number of blocks having the unique image data (line image data) in the image frame can be increased. In other words, since the replacement image data can be formed after enlarging the image to be able to express even a fine portion, the replacement image having the characteristics of the original image even if the unique image data of the input image data is small. There is an effect that data can be formed.

According to claim 10, wherein the image processing program, in addition to the effects of the image processing program according to claim 9, wherein, the ratio of specific image data extracted on the extracted original image frame is less than a predetermined ratio When the determination step determines that the number of blocks having unique image data is equal to or less than a predetermined lower limit value, the unique image data is expanded. Therefore, there is an effect that the specific image data to be expanded can be accurately selected.

  When the size of the unique image data is measured by the number of blocks (counted in units of blocks), 1 is counted even if no dots are arranged on the entire block. Therefore, when measuring the size of the unique image data in units of blocks, it is convenient, but the measurement result tends to shift to a larger side than the actual size, and in some cases, the unique image data that needs to be expanded There is a possibility that the situation will not be expanded. However, since the unique image data is expanded when the ratio of the extracted unique image data to the extraction source image frame is equal to or less than a predetermined ratio, the unique image data that needs to be accurately decompressed is accurately detected. Can be sorted into

  Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings. FIG. 1 is a schematic diagram showing a karaoke apparatus 1 as an image processing apparatus of the present invention. The karaoke apparatus 1 includes a main body 1a as an image processing unit, a CRT display 2 as a display device, and an electronic quick sample 30 for remotely operating the main body 1a. The main body 1a and the CRT display 2 are They are connected via a cable 4. The karaoke apparatus 1 has a function of executing an effect by executing image processing for converting input image data into an ASCII image composed of symbols, and outputting the converted ASCII image to a CRT display 2. Yes.

  The karaoke apparatus 1 is a so-called communication karaoke, and stores music data (accompaniment information) and lyrics data distributed from a karaoke server connected via a line in the main body 1a. When the music code indicating the music selected by the singer is transmitted from the electronic quick sample 30, the corresponding music data and lyrics data are read from the main body 1a, and the accompaniment is automatically played based on the music data. At the same time, lyrics and video (or ASCII images) are displayed on the CRT display 2 as the music progresses.

  The main body 1 a of the karaoke apparatus 1 includes a control unit 5, a CCD camera 3 and a microphone 23 connected to the control unit 5. An operation panel 20 having a numeric keypad for inputting numerical values and command buttons for inputting various commands to the control unit 5 is disposed on the casing front surface of the control unit 5. By operating the numeric keypad and the command button, the user can perform music selection, performance tempo setting, pitch setting, volume setting, sound effect setting such as reverb, character input, and the like. In addition, the operation panel 20 is provided with a liquid crystal display (hereinafter simply referred to as “LCD”), which transmits a numerical value input by a numeric keypad, a command content input by a command button, and an electronic quick sample 30. The contents of the received command and the operation status of the karaoke apparatus 1 are displayed on the LCD.

  Below the operation panel 20, a wireless communication unit 26 is provided for performing wireless communication using infrared data with the electronic sample 30. Data such as various commands and music codes transmitted as infrared data from the electronic quick sample 30 are received by the wireless communication unit 26. When the wireless communication unit 26 receives a music code from the electronic quick sample 30 or a command instructing execution of a predetermined process, the main body 1a executes a process based on the music code or the command. A microphone 23 is connected to the right side of the wireless communication unit 26 via an input terminal, and the utterance of the singer is input from the microphone 23 into the control unit 5.

  The CCD camera 3 is a general imaging device that continuously captures images, and is installed on the upper surface of the casing of the CRT display 2. An image (image data) picked up by the CCD camera 3 is input into the control unit 5 via a cable (not shown). The CCD camera 3 includes an A / D converter and the like, and image data converted into digital data is input to the control unit 5. The control unit 5 creates an ASCII image based on the image data input by the CCD camera 3.

  The electronic sample book 30 is an apparatus for remotely operating the main body 1a (control unit 5) (stopping or skipping performances) in addition to selecting a music piece and specifying the selected music piece (sending the music code to the main body 1a). And a display main body 30a having a touch panel display 40, and a cradle 44 for maintaining the display main body 30a in a standing state. The display main body 30a is provided with a wireless communication unit 36 for wireless communication using infrared data with the main body 1a at the top of the casing. Data such as a music code transmitted to the electronic quick sample 30 and a command for instructing execution of a predetermined process are output from the wireless communication unit 36 as infrared data.

  The display body 30a includes a touch panel display 40 that displays a guide image. The touch panel display 40 includes a display device 41 configured by an LCD that displays an image, and a touch panel 42 that is provided on the display device 41 and executes signal input when pressed.

  The guide image is an image displaying various data for music selection and operation buttons for performing various operations for music selection, so that the user can visually understand the operation of the electronic quick sample 30. It is configured. Note that pressing of each operation button displayed on the guide image (that is, the display device 41) is input as an electrical signal corresponding to the pressed position by the touch panel 42 provided on the display device 41. As a result, the CPU 12 (see FIG. 2) recognizes the input of the pressed operation button, and the corresponding operation (command input) is executed.

  Specifically, this guidance image is composed of a plurality of screens, and on each screen, an item classified for each category and singer, a song name displayed by selection of the item, and the like are displayed. The display of each song name is an operation button for designating the song name, and the song selection is made by pressing this song name display (designation of song data). An ASCII art button 42a is displayed in the screen on which the music selection is executed.

  The ASCII art button 42a is for instructing the main body 1a to produce an ASCII image. When the ASCII art button 42a is pressed from the screen (via the touch panel 42), an ASCII art code for instructing the main body 1a to produce an ASCII image is added to the selected music code and transmitted to the main body 1a. Is done. In the main body 1a, when an ASCII art code is added to the received music code, the effect on the CRT display 2 during the reproduction of the music is executed by displaying an ASCII image instead of the previously stored video.

  Furthermore, as the operation buttons, various buttons for operating the main body 1a are provided in addition to buttons for executing various processes in the electronic quick sample 30 such as selection of music. Similar to the button operation provided on the operation panel 20 of the main body 1a by the operation of the operation button (pressing the displayed operation button on the screen), the music selection, performance tempo setting, pitch setting, volume You can set sound effects such as setting and reverb, and input characters. That is, some of the functions of the operation panel 20 of the main body 1a are also executed by remote operation using the electronic quick sample 30.

  The cradle 44 is formed in a rectangular shape that supports the lower side and the left and right sides of the display main body 30a, and the cradle 44 and the display main body 30a are configured to be detachable. The cradle 44 is a connection frame between the electronic quick sample 30 and the main body 1a, and the electronic quick sample 30 attached to the main body 1a is electrically connected to the main body 1a by wire. By the wired connection via the cradle 44, the battery built in the electronic quick sample 30 is charged. Further, data such as a music code and a music title corresponding to a new music can be transmitted from the main body 1a to the electronic quick sample 30.

  FIG. 2 is a block diagram showing an electrical configuration of the karaoke apparatus 1 configured as described above. The karaoke apparatus 1 includes a main body 1a, a CRT display 2, and an electronic quick sample 30. The main body 1a further includes a control unit 5, a CCD camera 3, and a microphone 23.

  The control unit 5 controls the karaoke device 1 by the CPU 11 which is an arithmetic device. The CPU 11, ROM 12, RAM 13, image RAM 27, hard disk 14, modem 15, disk player 16, video A processing unit 17, an input / output port (I / O) 19, an accompaniment device 21 having a synthesizer as a sound source, a LAN port 25 for wired connection of the electronic sample 30 (cradle 44), and an electronic sample And a wireless communication unit 26 for receiving infrared data from 30. Each of these devices is connected to the CPU 11 and controlled by the CPU 11. The control unit 5 is connected to a video mixer 18 connected to the disc player 16 and the video processing unit 17, an operation panel 20 connected to an input / output port (I / O) 19, and an accompaniment device 21. An amplifier mixer 22 and a speaker 24 connected to the amplifier mixer 22 are provided. The CCD camera 3 is connected to the input / output port (I / O) 19.

  When the music code is input from the electronic sample 30 or the operation panel 20, the CPU 11 registers the music code in a reserved music list area provided in a predetermined area of the RAM 13 or stores it in the reserved music list area. Are read out in the order of registration, and the reproduction output of the music is controlled. In the karaoke playback process (see FIG. 4), which will be described later, if a music code is registered in the reserved music list area, it is determined that a music code has been input, and a process of playing a music is executed. When the music is played, the corresponding music code is deleted from the reserved music list area. Further, the CPU 11 performs control to form an ASCII image from an image captured by the CCD camera 3 and output it to the CRT display 2 as one of the effects performed when the music is played.

  The ROM 12 is a nonvolatile memory that stores various programs executed by the CPU 11 and fixed values.

  The RAM 13 is a memory for temporarily storing various data and the like when executing a program stored in the ROM 12 or a control program stored in the hard disk 14 described later, and includes a background image memory 13a and an ASCII image buffer 13b. And an ASCII art flag 13c.

  The background image memory 13a is a memory for storing image data captured (acquired) by the CCD camera 3 as a background image. The background image is picked up by the CCD camera 3 prior to the production by the ASCII image every time one production by the ASCII image, that is, reproduction of one music piece is executed. When a new background image is captured, the image data of the background image stored in the background image memory 13a is updated with the image data of the newly captured background image.

  When forming the ASCII image, the image data of the portion corresponding to the image data of the background image stored in the background image memory 13a is removed from the input image data, and the remaining portion of the image data of the background image is removed. (Person image data, unique image data) is converted into an ASCII image.

  In normal imaging, the CCD camera 3 acquires image data of various objects existing in the imaging range at a resolution that allows each object to be identified in detail, that is, with a number of pixels that realizes good image quality. Designed to be For this reason, depending on the installation location of the CCD camera 3, the acquired image data is complicated (an image in which a plurality of objects are mixed). When converting such complex image data into an ASCII image, the processing of the contour lines of complicated objects may become inaccurate, and a situation may occur in which individual objects are not accurately represented in the ASCII image. . However, since the image data (person image data) obtained by removing the background image from the acquired image data is converted to the ASCII image, the conversion source image data can be simplified. Therefore, the ASCII image to be formed can be an image in which an object (mainly a person in the present embodiment) is accurately and clearly shown.

  Specifically, the portion other than the background image is, for example, an image of an object that has entered the imaging range of the CCD camera 3 after the background image is captured by the CCD camera 3. In this embodiment, a person (singer) ). In other words, in the present embodiment, an object to be converted to an ASCII image is an image of a person (moving moving object), and the person image (portion other than the background image) is captured in a series of images as compared to a background image with little change. It becomes a characteristic part in. Therefore, removing the image data of the background image with little change common through a series of imaging, and forming the ASCII image with the remaining person image data, does not greatly change the intention of the production. In addition, a simple and clear ASCII image narrowed down to a characteristic part of the image can be displayed on the CRT display 2.

  The ASCII image buffer 13b is a memory for temporarily storing the ASCII image formed in the ASCII conversion process (S28, see FIG. 6) described later. The ASCII image is an image formed by combining symbol data, that is, a collection of symbols that macroscopically form an image. Specifically, the ASCII image is divided (inserted) into symbol data in an approximate form after dividing the line image data of the conversion source into blocks in the ASCII conversion process (S28). It is formed. The replaced symbol data is sequentially stored in the ASCII image buffer 13b in accordance with the block position.

  Here, the ASCII image buffer 13b stores the symbol data in order according to the position of the block, and stores the line feed code when the end of the image frame is reached. For this reason, the ASCII image buffer 13b stores symbol data in association with the coordinates of the image frame of the line image data before conversion. As a result, an ASCII image corresponding to the original image can be output by sequentially extracting the stored symbol data from the ASCII image buffer 13b. The symbol data stored in the ASCII image buffer 13b may be a symbol code indicating a symbol, or may be dot data (symbol image data) obtained by expanding the symbol code into an image.

  The ASCII art flag 13c is a flag for identifying whether an effect corresponding to the selected music is executed with a video stored in advance corresponding to the music data or an ASCII image. The ASCII art flag 13c is turned on when the ASCII art code is added to the music code transmitted from the electronic quick sample 30, and is turned off when the reproduction of the music corresponding to the inputted music code is completed.

  The image RAM 27 is a large-capacity RAM composed of DRAM, SDRAM, or the like, and temporarily stores image data. Image data (an image in which a background and a person are captured) input from the CCD camera 3 after background image shooting is stored in the image RAM 27 for each image frame. The stored image data is read out from the image RAM 27 and converted into an ASCII image for each image frame when an ASCII image is created.

  The hard disk 14 is a rewritable large-capacity non-volatile memory, and includes a control program 14a for controlling the karaoke apparatus 1, a symbol data memory 14b, a music data memory 14c, and a lyrics data memory 14d. . The programs of the flowcharts shown in FIGS. 4 to 6 are stored as a part of the control program 14a.

  The symbol data memory 14b is a memory for storing symbol image data (symbol data), and stores an image of each symbol as dot data in association with a symbol code indicating each symbol. When creating an ASCII image, the symbol data memory 14b is referred to, and dot data of a symbol that approximates line image data is extracted by template matching to form an ASCII image. In the present karaoke apparatus 1, a JIS 7 unit code (a code expressed by 8-bit data) conforming to the ASCII code is employed as the symbol code. The code system is not limited to JIS7 unit codes, and other ASCII codes, EBCDIC codes, Unicodes, and JIS kanji codes can be used as appropriate.

  The music data memory 14c is a memory for storing music data of music to be reproduced. Each piece of music data is stored corresponding to each piece of music code that specifies one music code. When a music code is input, one music data (MIDI (Musical Instrument Digital Interface) format data, data indicating a note length, etc.) corresponding to the music code is read from the music data memory 14c. It is output to an accompaniment device 21 to be described later.

  The lyric data memory 14d is a memory for storing lyric data (including title data) of music. Each lyric data is stored as text data corresponding to each music code designating one music code. The lyric data stored in the lyric data memory 14d is formed of JIS kanji codes. When the music code is input, the CPU 11 reads out the corresponding lyric data (including title data) from the lyric data memory 14d, and the character stored in the predetermined area of the hard disk 14 corresponding to the kanji code. The read lyric data is expanded into image data, and lyric character image data (dot data) is generated.

  Since the song desired by the singer changes due to the release of a new song or the like, the song data stored in the song data memory 14c and the lyrics data stored in the lyrics data memory 14d are regularly updated. As described above, the karaoke apparatus 1 is a communication karaoke, and is connected to a karaoke server that distributes music data and the like via a line. The lyrics data corresponding to the data (and video data corresponding to the music data in some cases) can be downloaded as a series of data associated with one music code. The downloaded music data is added to the music data memory 14c (added and stored). Also, the lyrics data is added to the lyrics data memory 14d (added and stored).

  The modem 15 is a modulation / demodulation device that converts an analog signal and a digital signal. It also transmits and receives various procedure signals for transmission control. Data output from the karaoke device 1 to the line (for example, an access request to the karaoke server) is converted from a digital signal to an analog signal by the modem 15 and then output to the line. Further, data received from the line (music data or lyrics data downloaded from the karaoke server) is converted from an analog signal to a digital signal by the modem 15.

  The modem 15 incorporates a network control unit (hereinafter simply referred to as “NCU”) for performing line control, and the karaoke apparatus 1 is connected to the line via the NCU. The NCU sends a dial signal for calling a communication partner.

  The disc player 16 is a playback device that plays back a CD or DVD that stores video displayed on the CRT display 2 when a singer sings while referring to the lyrics. The karaoke apparatus 1 determines whether an effect on the CRT display 2 when reproducing the music data is an image reproduced by the disc player 16 or an ASCII image based on the image data acquired by the CCD camera 3. , Is configured to be selectable. When it is specified that such an effect is to be performed on an ASCII image, the video playback by the disc player 16 is not executed. The karaoke apparatus 1 may be configured such that when an effect by an ASCII image is designated, the video is also reproduced by the disc player 16 and both the video and the ASCII image are displayed on the CRT display 2. . When both the video and the ASCII image are displayed on the CRT display 2, the screen of the CRT display 2 may be divided and the video and the ASCII image may be individually displayed on each divided screen, and the ASCII image is displayed on the video. They may be displayed on a single screen.

  The video processing unit 17 is a device that controls the output of telop of lyric character image data (dot data) generated by the CPU 11 and the ASCII image to the CRT display 2. The video mixer 18 combines the lyric character image data generated by the CPU 11 and output from the video processing unit 17 with the video or ASCII image reproduced by the disc player 16 and displayed on the CRT display 2. This is a video composition device.

  The accompaniment device 21 generates accompaniment and stores musical sound waveforms such as various instrument sounds. The music signal corresponding to the tone, pitch, and volume indicated by the MIDI format data (of the song data read from the song data memory 14c) is based on the musical sound waveform for the length indicated by the data indicating the note length. To generate digital signals. The generated music signal is converted into an analog signal and output to the amplifier mixer 22.

  The amplifier mixer 22 is a device that mixes and amplifies the music signal from the accompaniment device 21 and the utterance (audio data) input from the microphone 23, and the mixed signal is electrically amplified and output from the speaker 24. Is done.

  The electronic quick sample 30 includes the display main body 30a and the cradle 44 as described above, and the display main body 30a stores the CPU 31 that is an arithmetic unit, various programs executed by the CPU 31, fixed values, and the like. ROM 32 that is a non-volatile memory, RAM 33 that is a memory for temporarily storing various data and the like when executing a control program stored in ROM 32, a hard disk 34, an interface 35, and a wireless communication unit 36 A LAN port 37 for wired connection between the display main body 30a and the main body 1a via the cradle 44, a touch panel display 40 having a display device 41 and a touch panel 42, and a display device 41 provided on the touch panel display 40. Display control unit 38 to be controlled and touch panel display An input control unit for controlling the input by the touch panel 42 provided on the stomach 40, is interposed between the LAN port 37 and cradle 44, and a connector 43 for connecting the display main body 30a and the cradle 44.

  The hard disk 34 is a rewritable large-capacity non-volatile memory, and stores title data of songs that can be selected in association with song codes. In addition, category data such as genre and singer name is added to the title data of each musical piece, and can be displayed on the display device 41 for each category. Further, the hard disk 34 stores format data for each screen of the guide image.

  When the CPU 31 is requested to output the title data of the music (the music selection operation is started by the user), the CPU 31 displays the format of each screen of the guide image, the title data, etc. from the hard disk 34 in accordance with the music selection operation procedure. read out. Then, the display control unit 38 causes the display device 41 to display data such as music title data in the format of the corresponding guide image screen. When music selection, that is, music data designation is executed from the screen (by pressing the touch panel 42), the CPU 31 wirelessly outputs the selected music code from the wireless communication unit 36. Here, when the ASCII art button 42a is pressed from the screen (by pressing the touch panel 42) along with the music selection, the CPU 31 adds the ASCII art code to the music code and outputs it. The ASCII art code is a command for instructing the main body 1a to produce the ASCII image by the user. When the ASCII art code is added to the music code, the main body 1a recognizes the production instruction using the ASCII image, and the ASCII image. The production by is executed.

  Next, each process executed by the karaoke apparatus 1 will be described with reference to the flowcharts of FIGS.

  FIG. 4 is a flowchart of the karaoke playback process executed by the CPU 11 of the control unit 5 of the main body 1a. The karaoke playback process is a process of generating accompaniment and displaying lyrics data for the music designated by the singer or the like, and producing the music on the CRT display 2. Since this karaoke playback process is started by designating the playback of a music piece, that is, by inputting a music code, it is first confirmed whether or not a music code has been input (S1). Here, if the music code is not inputted (S1: No), this karaoke reproduction process is ended. On the other hand, if a music code has been input (S1: Yes), it is confirmed whether or not an ASCII code has been added to the input music code (S2), and an ASCII code has been added to the input music code. If not (S2: No), the process of S3 to S6 is skipped, and the process proceeds to the music data output process (S7). On the other hand, if an ASCII art code is added (S2: Yes), it recognizes that it is a request | requirement of performing the effect | action during music reproduction with an ASCII image, and turns on the ASCII art flag 13c (S3).

  Next, the display of “capturing the background image” is output to the CRT display 2 for a predetermined time to notify the background image capturing (S4), and the singer or the like is alerted. In the present embodiment, when an effect by an ASCII image is designated, the background image is shot before the reproduction of the music is started. Here, an image of the shooting area is output to the CRT display 2 together with the display of “shooting a background image”, so that a singer or the like can recognize the shooting area of the background image.

  Thereafter, an image captured by the CCD camera 3 is captured (acquisition of one image frame), and the acquired image data is written in the background image memory 13a (S5). Subsequently, the display of “start shooting” is output to the CRT display 2 for a predetermined time (S6). And the music data output process which reads the music data corresponding to the input music code from the music data memory 14c, and outputs it to the accompaniment apparatus 21 is performed (S7). The accompaniment device 21 collectively outputs music data corresponding to one music code, and the output music data is collectively stored in a predetermined memory provided in the accompaniment device 21.

  Next, the lyric data output process which reads the lyric data corresponding to the music data output to the accompaniment apparatus 21 from the lyric data memory 14d and outputs it to the image | video process part 17 is performed (S8). In this lyric data output process (S8), the lyric data (including title data) specified by the input music code is read out, and the read out lyric data is corresponded to the kanji code constituting the lyric data. The image data is expanded into character image data stored in a predetermined area of the hard disk 14 to generate lyric character image data (dot data). Then, the generated lyric character image data is output to the video processing unit 17. Lyric data (lyric character image data) corresponding to one music code is collectively stored in a predetermined memory provided in the video processing unit 17 before the music reproduction is started.

  Next, it is confirmed whether or not the ASCII art flag 13c is on (S9). If the ASCII art flag 13c is on (S9: Yes), from the image data (one image frame) acquired by the CCD camera 3, ASCII art processing for forming an ASCII image of one frame is executed (S10). After the execution of the ASCII art process (S10), it is confirmed whether or not the music is being reproduced (S11). If the music is being reproduced (S11: Yes), the process proceeds to S15 and the music must be being reproduced. (S11: No), it is confirmed whether or not it is the timing when the music is finished (S12). If the timing is the end of the music (S12: Yes), the ASCII art flag 13c is turned off, and the karaoke playback process ends.

  As a result of checking in the process of S12, if the timing is not the end of the music (S12: No), it is the time to start playing the music, so the video processing unit 17 and the accompaniment device 21 are instructed to start playing ( S14). As a result, the lyrics data (lyric character image data) corresponding to the accompaniment generated by the accompaniment apparatus 21 is output to the CRT display 2 via the video mixer 18 under the control of the video processing unit 17. Further, an accompaniment is generated by the accompaniment apparatus 21, and the generated accompaniment is output from the speaker 24.

  After the process of S14, the ASCII image stored in the ASCII image buffer 13b is output to the video mixer 18 (S15). The output ASCII image is combined with the lyrics data (lyric character image data) in the video mixer 18 and output to the CRT display 2 under the control of the video processing unit 17.

  Thereafter, the process proceeds to S10. As a result, the ASCII art process (S10) for forming one ASCII image frame for each image frame of the input image data is repeatedly executed until the music ends (S12: Yes). The image is displayed as a movie.

  On the other hand, if the ASCII art flag 13c is turned off as a result of the confirmation in S9 (S9: No), the ASCII code has not been added to the input music code, that is, the effect during the music reproduction is displayed in the ASCII image. In step S16, it is recognized that it is not requested to perform the disk reproduction process. The disc playback process (S16) is a process for causing the disc player 16 to play a video predetermined for each piece of music. In a predetermined area of the hard disk 14, a correspondence table in which a music code and a CD or DVD identification code are associated with each other is stored. Based on the correspondence table, the CPU 11 corresponds to the input music code. The disc player 16 is instructed to identify the CD or DVD identification code. Since the selection of the CD or DVD in the disc player 16 is executed by a mechanical operation, in this disc reproduction process (S16), the process waits until the selection operation in the disc player 16 is completed.

  After the end of the wait, the disc player 16, the video processing unit 17, and the accompaniment device 21 are instructed to start playback (S17), and the karaoke playback process is ended. In accordance with such a reproduction instruction, an accompaniment of the music corresponding to the input music code is output from the speaker 24, and the lyrics data (lyric character image data) of the music and the corresponding video are displayed via the video mixer 18 on the CRT display. 2 is output.

  The CPU 11 outputs a timing signal to the video processing unit 17 and the accompaniment device 21 after instructing the start of reproduction in the processes of S14 and S17, and the lyrics data (lyric character image data) to the CRT display 2 is output. The output and the generation of the accompaniment by the accompaniment device 21 are synchronized.

  FIG. 5 is a flowchart of the ASCII art process (S10) executed in the karaoke playback process of FIG. The ASCII art process (S10) shown in FIG. 5 will be described with reference to FIG. 3, which is a diagram schematically showing a process of forming an ASCII image.

  In the ASCII art process (S10), first, image data of one image frame stored in the image RAM 27 is read (S21). Then, the image data stored in the image RAM 27 and the image data of the background image stored in the background image memory 13a are both compressed so that the number of dots for creating 80 × 25 characters is obtained ( S22).

  Since the symbol 1 is composed of 8 × 16 dots, the image data is compressed to a data size in which the total number of dots is 80 × 25 × 128. Each image data stored in the background image memory 13a and the image memory 27 has a number of pixels that can realize good image quality when output to the CRT display 2, and has a large amount of data. For this reason, if the image data stored in the image RAM 27 is converted into line image data as it is, and further converted into an ASCII image, a great amount of processing time is required. The actual operation of the can be greatly delayed. Furthermore, since a standard display size in a general display device is a size that displays 80 × 25 characters (symbols) in full screen display, an ASCII image is formed with more characters (symbols). And some of them can be hidden. Therefore, the processing time required for creating the ASCII image is shortened, and the data is compressed in the process of S22 so that the size of the created ASCII image is compatible with a general display device. Note that it is not always necessary to compress the image data to a data size having a total number of dots of 80 × 25 × 128. The data size after the compression depends on the aspect of the ASCII image to be created (fine density), the display device, and the like. Depending on the situation, it can be adopted as appropriate.

  After the process of S22, both compressed image data are compared for each dot, a common part is extracted, and a mask is created using the common part (S23). Next, the portion corresponding to the created mask, that is, the background image portion is deleted from the compressed image data, and person image data (image data of a portion other than the background image) is extracted (S24). FIG. 3A shows a state where the person image data is extracted. The image divided by the frame displayed in FIG. 3A schematically shows one image frame, and corresponds to human image data in which a human image drawn at the center is extracted. In FIG. 3A, since the mask process is executed by the process of S24, the background image is deleted, and only the person image is displayed in the image frame.

  The mask is not limited to the one formed by comparing the compressed image data, and each of the image data stored in the image RAM 27 and the image data of the background image stored in the background image memory 13a. Compared with before compression, a mask may be created, and further, human image data may be extracted using a mask created from image data before compression stored in the image RAM 27.

  Then, it is checked whether or not the area (number of dots) of the extracted person image occupies more than half of the entire area of the image frame (person image area ≧ 1/2 × (total area of the image frame)). (S25). Here, when the area of the extracted person image occupies more than half of the entire area of the image frame (S25: Yes), it is determined that the person image is captured with a sufficient size. Then, the noise is removed from the person image data in order to convert it into ASCII image data as it is (S26). Subsequently, the human image data from which noise has been removed is converted into line image data by Laplacian conversion (S27). As a result, the image data represented by the plane is converted into a line image in which the characteristic portion is represented by a line. FIG. 3B shows a state in which this person image data is converted into line image data by Laplacian conversion. Thereafter, an ASCII conversion process for converting the line image data into an ASCII image is executed (S28), and the ASCII art process (S10) is terminated.

  On the other hand, if the area (number of dots) of the extracted person image is less than half of the entire area of the image frame as a result of checking in the process of S25 (S25: No), the person image is smaller than the reference and is not valid. It is shown that it was shot at an appropriate size. Therefore, the human image data is enlarged (expanded) so that the area of the human image is more than half of the entire area of the image frame. If the conversion source image data (person image data in this embodiment) is reduced when forming an ASCII image, the number of blocks having the conversion source image data decreases, and as a result, a small ASCII image with a small number of symbols. Will be formed. A small ASCII image assembled with a small number of symbols is difficult to reflect the characteristics of the original image, and in some cases, it is impossible to identify what it represents. Therefore, in the present embodiment, half the area of the entire image frame is set as the lower limit value, and the person image data is enlarged when the area is equal to or lower than the lower limit value, and the ASCII characteristic of the original image (captured person) is reflected. An image is formed.

  Therefore, first, the dot distribution of the human image data with respect to the horizontal axis of the screen (image frame) is calculated, the center (central coordinate) of the distribution in the horizontal axis direction is determined (S29), and then the area of the human image is determined as the image frame. The enlargement ratio is calculated by dividing the area by half of the entire area (S30), and the person image data is enlarged with the calculated enlargement ratio around the central coordinates determined in the process of S29 (S31). According to this, since an image zoomed isotropically from the center of the person image can be formed, it is possible to avoid the enlarged image from becoming a distorted image. Thereafter, the process proceeds to S26, and ASCII image formation based on the enlarged person image data is executed.

  Note that the lower limit value for determining the enlargement of the image data is not necessarily limited to half the area of the entire image frame, and can be arbitrarily set within a range in which a good ASCII image can be formed. Further, the size of the person image is determined by the number of dots of the person image data. Instead, the image frame is divided into 80 × 25, and the number of blocks having the person image data is a predetermined number (for example, half). Whether or not to enlarge the human image data may be determined depending on whether or not the following is true. According to this, the processing time for determination can be shortened.

  FIG. 6 is a flowchart of the ASCII conversion process (S28) executed in the ASCII art process (S10) of FIG. The ASCII conversion process (S28) shown in FIG. 6 will be described with reference to FIG.

  In the ASCII conversion process (S28), first, the size in the horizontal direction X of the image frame having line image data is divided by the horizontal size Z1 of the symbol data to obtain the horizontal division number m (S41). Specifically, in this embodiment, one symbol data has a size of 8 × 16 dots, and the horizontal size Z1 of the symbol data is 8 dots. Since the number of characters (symbols) arranged in the horizontal direction X is 80, the size in the horizontal direction X is 8 dots × 80, that is, a size of 640 dots in terms of dots. Therefore, by the processing of S41, the horizontal division number m becomes “80” obtained by dividing 640 by 8.

  Subsequently, the vertical division number n is obtained by dividing the size of the image frame in the vertical direction Y by the vertical size Z2 of the symbol data (S42). Specifically, in the present embodiment, the vertical size Z2 of the symbol data is 16 dots. Since the number of characters (symbols) arranged in the vertical direction Y is 25, the size in the vertical direction Y is 16 dots × 25, that is, 400 dots in terms of dots. Therefore, the vertical division number n becomes a value “25” obtained by dividing 400 by 16 by the processing of S42.

  By the processing of S41 and S42, the image frame is divided into blocks of a predetermined size as shown in FIG. In the present embodiment, the image frame is divided into blocks having the same size of 8 × 16 dots as the symbol data. According to this, at the time of template matching, matching can be performed immediately without performing processing for matching the size of each part (divided image) of the line image data divided into blocks and the symbol data. Matching can be performed. Note that, when the image data is compressed to a total dot number of 80 × 25 × 128 or less or less, the fraction generated in S41 and S42 for dividing the image frame is rounded up to 80 × 25 or more or 80 Divided into blocks of × 25 or less.

  Then, after each variable i and variable j are set to “0” (S43), the process of converting the divided line image data into symbol data is executed for each block by the processes of S44 to S51. The process of converting to symbol data is executed from the upper level to the lower level for each row of each block in the image frame. Each position in the image frame is specified by coordinates (Xi, Yj) in which the horizontal direction is X and the vertical direction is Y.

  Specifically, it is checked whether or not the variable j is less than the vertical division number n (j <n) by the process of S44 (S44). Here, if the variable j is not less than the vertical division number n (S44: No), it is indicated that the variable j has reached the vertical division number n, and the lowest level of the block into which the conversion to the symbol data is divided is shown. This ASCII conversion process (S28) is terminated.

  On the other hand, if the variable j is less than the vertical division number n (S44: Yes), conversion to the symbol data is incomplete, so whether the variable i is less than the horizontal division number m (i <m). Whether or not is checked (S45). If the variable i is less than the horizontal division number m (S45: Yes), since the conversion of one row in the horizontal direction X is incomplete, the coordinates of one area (block) to be converted are obtained (S46). The coordinate in the horizontal direction X of one block to be converted is obtained by multiplying the variable i by the horizontal size Z1 of the symbol data (i × Z1 → Xi), and the coordinate in the vertical direction Y is the horizontal value of the symbol data in the variable j. It is obtained by multiplying the size Z2 (j × Z2 → Yi). For example, since variable i = variable j = 0 at the start, the coordinates (X0, Y0) are (0, 0). The coordinates (X1, Y0) of the next block to be converted are positions shifted in the horizontal direction X by the size of the block from (X0, Y0). That is, the X coordinate (X1) of one region (block) to be converted next is obtained by multiplying the variable i = 1 by the horizontal size Z1 of the symbol data according to (i × Z1 → Xi). Similarly, the coordinates (X0, Y1) when moving to the next stage are obtained by multiplying the variable j = 1 by the vertical size Z2 of the symbol data according to the Y coordinate (j × Z2 → Yi).

  Then, 1 that most closely approximates each part (divided image) of the line image data divided into a region (designation of one block) having the obtained coordinates (Xi, Yj) and coordinates (Xi + Z1, Yj + Z2) as diagonal lines. The symbol data is detected from the symbol data stored in the symbol data memory 14b by template matching (S47). Subsequently, the detected symbol data is written in the ASCII image buffer 13b in association with the coordinates (S48), and then 1 is added to the variable i (i + 1 → i) (S49), and the process is performed in S45. Migrate to Thereby, the processing of S45 to S49 is repeated until the conversion for the block belonging to one row is completed.

  As a result of checking in the process of S45, if the variable i is not less than the horizontal division number m (S45: No), it is indicated that the variable i has reached the horizontal division number m. After determining that the conversion of the line image data of each block into the symbol data has been completed for the row and writing the line feed code to the ASCII image buffer (S50), 1 is added to the variable j (j + 1 → j) and Then, “0” is set to the variable i (S51), and the process proceeds to S44. As a result, the conversion target position moves to the next stage. Then, as described above, the processes of S44 to S51 are repeated until it is determined in the process of S44 that the conversion of the line image data into the symbol data has been completed (S44: No). As a result, as shown in FIG. 3C, an ASCII image is formed by converting the line image data into symbol data for each divided block.

  In this embodiment, the image frame is divided (S41, S42) in the ASCII conversion process (S28) after being converted into line image data (S27). However, the image frame is divided (S41, S42). Later, conversion to line image data (S27) may be performed.

  As described above, according to the karaoke apparatus 1 of the present embodiment, the effect on the CRT display 2 during the reproduction of the music data can be executed by the effect on the ASCII image, which is predetermined for each music. It is possible to provide fresh entertainment that is different from the production of outputting video. Moreover, since a singer's motion is reflected in an ASCII image, if a singer performs a comical behavior etc. with a song, high entertainment as entertainment can be implement | achieved.

  In addition, a switch that switches between a predetermined video output and an ASCII image output according to the music is provided, and a monitoring unit that monitors the state of the switch every predetermined time is provided, and is detected by the monitoring unit. You may comprise so that the image to output may be switched according to the state of a switch. According to this, the user can switch between video output and ASCII image output at any timing.

Further, the karaoke apparatus 1 is configured to convert image data input from the CCD camera 3 provided in the karaoke apparatus 1 into an ASCII image and output the ASCII image to the CRT display 2, and input from the microphone 23. The audio data is output from the speaker 24 of the karaoke apparatus 1. Instead of this, the image data to be converted into an ASCII image and the output of the converted ASCII image may be executed by an external device other than the karaoke device 1. Such an external device is a device that is provided separately from the karaoke device 1 and is formed so as to be communicable with the karaoke device 1. Assistance), a personal computer, and the electronic quick sample 30 mounted with the CCD camera 3 are exemplified. Further, instead of the microphone 23 and the speaker 24, audio data may be input / output using a microphone or a speaker provided in the external device described above.

  According to this, by inputting image data and sound data from a mobile phone or the like owned by each individual, the ASCII image and sound data based on the input image data can be transferred via the karaoke device 1 Can be output to an external device such as a mobile phone. For this reason, the degree of freedom of the input / output device for image data and audio data can be improved, and there is a desire to allow only a specific person to view the ASCII image, or to view the ASCII image at hand with a portable device. It is possible to meet a wide variety of user preferences such as requests.

Next, a second embodiment of the present invention will be described with reference to FIGS. In the first embodiment, the image processing apparatus is configured by the karaoke apparatus 1. Instead, in the second embodiment, the image processing apparatus is configured by a telephone device using the Internet Protocol (hereinafter simply referred to as “IP telephone device”), and has a transmission side having an image processing unit. And a receiving device having a display device. In addition, the same code | symbol is attached | subjected to the same part as above-mentioned 1st Embodiment, and the description is abbreviate | omitted.

  FIG. 7 is a schematic diagram showing a telephone system 220 including an IP telephone apparatus 200 that is an image processing apparatus according to the second embodiment. Each of the IP telephone devices 200 (200a, 200b, 200c) is both a transmission-side device that transmits data and a partner-side device that receives data. Accordingly, each IP telephone device 200 includes both an image processing unit and a display device. As shown in FIG. 7, the IP telephone devices 200a and 200b include a computer main body, a keyboard for inputting various data and commands to the computer main body, and a CRT for monitoring processing executed in the computer main body. A personal computer (hereinafter simply referred to as “PC”) provided with a display 112. The IP telephone device 200c is a telephone provided with a receiver and a transmitter. In addition to the video phone device 200, other telephone devices connected to the IP packet network are not shown.

  Each of the IP telephone devices 200a, 200b, and 200c is a device that is connected to an IP network and performs packet communication of data such as voice data and image data with each other via the IP network. And a communication control unit 120 (see FIG. 8) that functions as an IP adapter. Each of the IP telephone devices 200a, 200b, and 200c has a videophone function for transmitting and receiving an image together with voice to and from the other device, and a CCD camera 116 for capturing an image of the sender, A display device (for example, a CRT display 112) for reproducing and displaying the received image data.

  The IP telephone device 200 (for example, the IP telephone device 200a), which is the transmission side device, converts the input image of the CCD camera 116 into an ASCII image in response to the request (designation) of the speaker, and the converted ASCII image. Is sent to the IP telephone devices 200b and 200c of the counterpart device together with the voice data. The data to be sent is divided into data blocks by the IP adapter, and one data block is put into one packet and sent to the counterpart device.

  On the other hand, IP telephone apparatus 200 (for example, IP telephone apparatuses 200b and 200c in the case where IP telephone apparatus 200a is used as a transmission-side apparatus) performs delay time adjustment of received packets and packet order adjustment. Thereafter, data is extracted from the packet. When the received data includes a code corresponding to the ASCII image, the ASCII image is reproduced and output to its own display device (for example, the CRT display 112). It is configured as follows.

Each of the IP telephone devices 200a, 200b, and 200c has an ADSL (Asymmetric
(Degital Subscriber Line) is connected to the IP network as an access line. The IP network is a network line that is exclusively constructed by a communication company whose call control is performed by a gatekeeper (not shown), and transmits each packet to a partner apparatus by a router installed in the line route. Furthermore, the IP network is connected to an existing telephone network via a VoIP gateway (not shown), and is configured to be able to communicate with a general telephone.

  The access line may be a normal general public line, an ISDN line, a cable TV internet connection line, or the like. The IP network may be a general Internet IP packet network.

  FIG. 8 is a block diagram showing an electrical configuration of the IP telephone apparatus 200 of the telephone system 220 configured as described above. The IP telephone apparatus 200b is configured in the same manner as the IP telephone apparatus 200a, and therefore the description thereof will be omitted. Only the parts different from the IP telephone apparatus 200a will be described for the IP telephone apparatus 200c.

  The IP telephone device 200a is a device in which each device for realizing a telephone function is mounted on a PC, and includes a CPU 101, a ROM 102, a RAM 103, an image RAM 105, and an audio RAM 106. The CPU 101 that is an arithmetic device controls the IP telephone device 200a by a control program stored in the ROM 102 or a control program 104a stored in the hard disk 104 described later. The ROM 102 is a nonvolatile memory that stores various programs executed by the CPU 101 and fixed values. The RAM 103 is a memory for temporarily storing various data when the CPU 101 executes various programs, and includes a background image memory 103a, an ASCII image buffer 103b, and an ASCII art flag 103c.

  The background image memory 103a and the ASCII image buffer 103b are the same memories as the background image memory 13a and the ASCII image buffer 13b provided in the karaoke apparatus 1 (RAM 13 of the control unit 5) according to the first embodiment. The background image memory 103a stores image data captured (acquired) by the CCD camera 3 as a background image, and the ASCII image buffer 103b stores the ASCII image formed in the ASCII conversion process (S28, see FIG. 6). Is temporarily stored.

  The ASCII art flag 103c is a flag for identifying whether the image data sent to the counterpart device is an image captured by the CCD camera 116 or an ASCII image. The ASCII art flag 103c is turned on when an ASCII image designation command is input on an input screen (telephone processing main screen) for executing telephone processing (to be described later), and is turned off when the line is released thereafter. Is done.

  The image RAM 105 and the audio RAM 106 are large-capacity RAMs configured by DRAMs or SDRAMs, and the image RAM 105 is connected to the CCD camera 116 after taking a background image in the same manner as the image RAM 27 of the first embodiment. The image data acquired in this way is temporarily stored. The audio RAM 106 temporarily stores audio data. The audio data (speaker's utterance) input via the microphone 115 is encoded by the audio CODEC 109 and then the audio data. Stored in the RAM 106.

  As shown in FIG. 8, the CPU 101, ROM 102, RAM 103, image RAM 105, and audio RAM 106 are connected to each other via a bus line 107, and the bus line 107 is also connected to an input / output port 108. In addition to the bus line 107, the input / output port 108 is connected to the hard disk 104, the audio CODEC 109, the image CODEC 110, the CRT display 112, the keyboard 113, the speaker 114, the microphone 115, the CCD camera 116, and the communication control unit 120. Yes.

  The hard disk 104 is a rewritable large-capacity nonvolatile memory and includes a control program 104a for controlling the IP telephone apparatus 200a and a symbol data memory 104b. The programs of the flowcharts shown in FIGS. 9 to 11 are stored as a part of the control program 104a. The symbol data memory 104b is a memory similar to the symbol data memory 14b provided in the karaoke apparatus 1 (the hard disk 14 of the control unit 5) according to the first embodiment, and stores symbol image data.

  The audio CODEC 109 and the image CODEC 110 are electronic circuits (hardware) that encode and decode audio data and image data, respectively. The audio CODEC 109 obtains audio data (speaker's utterance) input as analog data at a predetermined sampling period, and quantizes and encodes it to convert it into digital data (encoding). It is a circuit which performs. The encoded audio data is written into the audio RAM 106. The audio data from the counterpart device received by the communication control unit 120 is output from the speaker 114 after being decoded (expanded) by the audio CODEC 109.

  The image CODEC 110 compresses and encodes image data input from the CCD camera 116 and stored in the image RAM 105 into a data format to be transmitted to the counterpart device. Further, the image data from the counterpart device received by the communication control unit 120 is output from the CRT display 112 after being decoded (expanded) by the image CODEC 110.

  In IP telephone apparatus 200a, band compression encoding is further performed on the encoded audio data. Discrete cosine transform (DCT) is applied to the band compression encoding of the audio data. The discrete cosine transform is a general method in which an image is divided into small blocks, and frequency component coefficients are quantized and encoded to be compressed. Therefore, when voice data from the counterpart device is received, first, an inverse transform (IDCT) process of discrete cosine transform is performed, and then decoded into analog data by the voice CODEC 109 and reproduced and output.

  The communication control unit 120 is a device for the IP telephone apparatus 200a to perform packet communication with the other IP telephone apparatuses 200b and 200c (or other telephone apparatuses) via the IP network, and the modem 121. , A signal separation unit 123, a signal multiplexing unit 124, and a buffer 125.

  The modem 121 is an ADSL modem, converts data (signal) transmitted from the IP telephone apparatus 200a into an ADSL signal adapted to the ADSL line, and also receives the ADSL signal received by the IP telephone apparatus 200a in the IP telephone apparatus 200a. It is a signal converter that converts an appropriate signal to be processed (for example, an Ethernet (registered trademark) signal). The modem 121 has an NCU function for sending a dial signal to call a communication partner and controlling the line.

  The signal separation unit 123 is a device that separates data received from a counterpart device such as the IP telephone devices 200b and 200c into image data and audio data. As will be described later, in the telephone system 220, the ASCII image data is transmitted as audio data from the counterpart device, and therefore is not separated by the signal separation unit 123. The signal multiplexing unit 124 is a device that multiplexes the audio data and the image data stored in the transmission buffer 125 a in the time axis direction, and the image frame and the audio data. The multiplexed data is transmitted via the modem 121. Sent out.

  The buffer 125 is a memory for temporarily storing data, and includes a transmission buffer 125a for temporarily storing data to be transmitted and a reception buffer 125b for temporarily storing received data. ing. The transmission buffer 125a stores audio data and image data that have been compression-encoded by telephone processing (see FIG. 9) described later. The stored audio data and image data are read from the transmission buffer 125 a at a predetermined timing and output to the signal multiplexing unit 124. The reception buffer 125b stores the image data and audio data separated by the signal separation unit 123 in units of frames. When the CPU 101 recognizes that data is stored in the reception buffer 125b, the data stored in the reception buffer 125b is read from the reception buffer 125b, and decoding and reproduction output are executed.

  As described above, the IP telephone device 200a is connected to the IP network via the communication control unit 120 (via the ADSL line), and is mutually connected to the other IP telephone devices 200b and 200c via the IP network. It is connected to the. Thus, a videophone that transmits and receives audio data and image data can be executed between the IP telephone apparatus 200a and the other IP telephone apparatuses 200b and 200c (or other telephone apparatuses).

  Since IP telephone apparatus 200c is a telephone, a transmitter functions as microphone 115, a receiver functions as speaker 114, an LCD is provided instead of CRT display 112, and an ASCII image designation command is input. A command input button is provided.

  Next, with reference to FIGS. 9 to 11, each process executed by the IP telephone apparatus 200 configured as described above will be described.

  FIG. 9 is a flowchart showing telephone processing executed by the IP telephone apparatus 200 (200a, 200b). Since the IP telephone devices 200a and 200b are PCs, the telephone processing is started when a predetermined command for requesting start is input by operating the keyboard 113. First, the main screen of the telephone processing is displayed on the CRT display 2. (S101). Thereafter, it is confirmed whether or not the telephone number of the counterpart device is designated (S102). If the telephone number of the counterpart device is not designated (S102: No), the designation is awaited. In the process of S102, the elapse of a predetermined time is monitored, and when the telephone number of the counterpart device is not specified even when the predetermined time has come, the process proceeds to each process of S114. .

  On the other hand, if the telephone number of the counterpart device is designated (S102: Yes), it is confirmed whether an ASCII image designation command has been input (S103). The ASCII image designation command is a command for instructing the image data to be transmitted to the counterpart device to be an ASCII image, and is input by performing a predetermined key operation according to the display on the main screen.

  If no ASCII image designation command is input (S103: No), the process of S104 is skipped and the process proceeds to S105. On the other hand, if an ASCII image designation command is input (S103: Yes), the ASCII art flag 103c is turned on (S104), and then the counterpart device is called (S105). The call request is transmitted to the gatekeeper installed in the IP network by the processing of S105, and the call state between the IP telephone device 200a as the calling device and the partner device is established by the operation of the gatekeeper.

  Thereafter, it is confirmed whether or not the ASCII art flag 103c is on (S106). If the ASCII art flag 103c is on (S106: Yes), ASCII art processing is executed (S107), and the image captured from the CCD camera 116 is obtained. Convert data to ASCII image data. The ASCII art process (S107) is the same process as the ASCII art process (S10) of the first embodiment shown in FIG. In the first embodiment, symbol data (image data) detected by template matching is written to the ASCII image buffer 13b by the process of S47 of the ASCII conversion process (S28). In the second embodiment, it is assumed that the symbol code corresponding to the symbol data detected by template matching in the processing of S47 of the ASCII conversion processing (S28) is written in the ASCII image buffer 13b.

  Further, a step (not shown) is provided between the process of S106 and the process of S107, in which a background image is captured by the CCD camera 116 and image data of the captured background image is written to the background image data memory 103a. Yes. This step is executed immediately after checking the state of the first ASCII art flag 103c after the line is closed (S106), and is not executed thereafter.

  Thereafter, it is confirmed whether or not the image data received from the counterpart device is stored in the reception buffer 125b. When the image data is stored in the reception buffer 125b, the image data is reproduced and output to the CRT display 112. Image output processing is executed (S108). The image data output by this image output processing is displayed by opening a multi-window on a part of the main screen. Note that the image data stored in the reception buffer 125b is decoded by the image CODEC 110 and output to the CRT display 112 as described above.

  After the image output process (S108), after executing the voice transmission process for transmitting the voice data input from the microphone 115 to the counterpart device (S109), the voice data received from the counterpart device is sent to the speaker 114. A sound output process for reproducing and outputting from is executed (S110). Thereafter, it is confirmed whether or not the call is finished. If the call is finished (S111: Yes), the call state with the other device is canceled (the end of the call is notified to the gatekeeper) (S112), and the ASCII art flag 103a. Is turned off (S113). Subsequently, each process is executed (S114), a process for ending the multi-window display for outputting the received image data as an image is performed, and at the same time, an end command for ending the telephone process or other commands (for example, telephone number Input of registration processing execution command, etc.) and waiting for designation of a new telephone number. In each process (S114), when the input of the end command is recognized, the main screen is ended and the telephone process is ended. When another command is input, processing corresponding to the content of the input command is executed. If the designation of a new telephone number is recognized, the process proceeds to S103.

  Further, if the ASCII art flag 103c is turned off as a result of the confirmation in S106 (S106: No), normal image transmission processing for transmitting image data (video) input from the CCD camera 116 to the counterpart device is executed. (S115). Image data input from the CCD camera 116 after the background image is captured is stored in the image RAM 105 in units of image frames. In this image transmission process (S115), the image data stored in the image RAM 105 is output to the image CODEC 110 in the order of input. When the output of one image frame from the image RAM 105 is completed, this image transmission process (S115) is terminated, and the process proceeds to S108. The image data output to the image CODEC 110 is compressed and encoded, and then output to the communication control unit 120.

  If the result of confirmation in S111 is not the end of the call (S111: No), the process proceeds to S106, and the process from S106 to S111 is performed until the end of the call is recognized (S111: Yes). The process or the processes of S106, S115, and S108 to S111 are repeated to maintain the call state.

  In the IP telephone device 200c, the same telephone process as described above is executed, but the process of S101 is omitted, and the apparatus is in a standby state in which outgoing calls and incoming calls are detected alternately. When the off-hook state is detected, it is determined that the request is a call request, and the process of S102 is executed. Also, the ASCII image designation command is input by operating the command input button.

  FIG. 10 is a flowchart of the voice transmission process (S109) executed in the telephone process of FIG. This voice sending process (S109) is a process for sending the voice data (speaker's voice) input from the microphone 115 to the counterpart device as described above. Audio data input from the microphone 115 is converted into digital data by the audio CODEC 109 and stored in the audio RAM 106. Then, the audio data stored in the audio RAM 106 is subjected to DCT processing (discrete cosine transform processing) (S121), and then the components below the audible limit are removed from the DCT-processed audio data (S122).

  The most audible limit is a limit sound that can be heard with normal human hearing, and is determined by a frequency band and a volume level. For example, 20 Hz to 5 kHz is a frequency band in which even a small sound can be heard well, and the higher the frequency or the lower the frequency, the more difficult it is to hear. Even in the same frequency band, if the volume level is low, it becomes difficult to hear. Below the maximum audible limit is a region of sound that cannot be heard by human hearing normally due to factors of frequency band and volume level.

  Therefore, whether or not the component is below the maximum audible limit is determined by the processing of S122 based on the frequency band and volume level of each component of the audio data subjected to DCT processing. The volume level determined for each frequency band is stored in a predetermined area of the hard disk 104 as a threshold value indicating the maximum audible limit.

  Next, it is confirmed whether or not the ASCII art flag 103a is on (S123). If it is off (123: No), the processes of S124 and S125 are skipped, and the process proceeds to the process of S126. If it is on (S123: Yes), the symbol code (8-bit data, binary data) stored in the ASCII image buffer 103b is added to the audio data as a component below the maximum audible limit (S124). ). That is, the symbol data is incorporated into the frequency band of the removed audio data component in a magnitude that is equal to or lower than a predetermined volume level corresponding to the frequency band (converted to a data signal of a limit frequency component).

  As a result, the symbol data can be superimposed (held) on the audio data as a component below the audible limit of the audio data, and even if the symbol data is included in the audio data, the quality of the reproduced audio is greatly affected. Never give. Therefore, even if the audio data is reproduced and output without removing such a symbol code portion, the added symbol code does not cause annoying noise or voice quality deterioration that makes it inaudible. Therefore, even if the counterpart device is a general-purpose telephone device, it does not hinder the call.

  If the counterpart device does not output a component below the maximum audible limit as a sound (a frequency band that is not reproduced (non-reproduction area) is provided in advance), the maximum audible limit is established in the counterpart device. Symbol data held as the following components is not output as audio data. In this case, the process of S122 for removing the component below the maximum audible limit of the audio data corresponds to the first removing means according to claim 8. The frequency band that is not reproduced (non-reproduction area) is not limited to the area of components below the maximum audible limit, and may be set to a predetermined frequency band.

  The ASCII image buffer 103b stores symbol codes in order according to the position (coordinates) of the divided blocks. In the process of S124, the stored order (address order) from the ASCII image buffer 103b. The symbol codes are read according to the above and added to the audio data in the read order. For this reason, if components below the maximum audible limit are sequentially extracted from the received audio data, the symbol code arrangement can be made in accordance with the coordinates of the original image, and the original ASCII image is reproduced and output from the symbol code arrangement. be able to.

  In the second embodiment, it is assumed that one symbol code is added to one unit of audio data converted by DCT processing from one sampling unit of audio data sampled by the audio CODEC 109. The symbol code added to one unit of audio data may be two or more symbol codes. The number of symbol codes added to one unit of audio data depends on the transmission speed and image quality. It shall be set as appropriate.

  After the process of S124, an ASCII possession code indicating that the symbol data for forming the ASCII image is included in the speech data is added to the speech data (S125), and the speech data to which the ASCII possession code is added. Is output to the communication control unit 120 (S126), and the voice transmission process (S109) is terminated.

  In the communication control unit 120, the voice data input by the voice transmission process (S109) is divided into packets in the time axis direction, and the ASCII holding code is added as the packet header data together with the identifier of the counterpart device. Then, the packetized data is output to the line by a predetermined transmission method. As a result, the ASCII image is sent to the counterpart device in the form of audio data.

  When the image data sent to the communication control unit 120 via the image CODEC 110 by the image sending process (S115) is stored in the transmission buffer 125a, such as when the ASCII image is not output, the image data and The corresponding audio data is multiplexed by the signal multiplexing unit 124, then packetized and output to the line.

  FIG. 11 is a flowchart of the voice output process (S110) executed in the telephone process of FIG. In this audio output process (S110), first, it is confirmed whether or not audio data is stored in the reception buffer 125b of the communication control unit 120 (S131). If not stored (S131: No), this audio is processed. The output process (S110) ends. On the other hand, when the audio data is stored in the reception buffer 125b (S131: Yes), the audio data stored in the reception buffer 125b is read, and it is confirmed whether or not the ASCII holding code is added to the read audio data. (S132). When the ASCII possession code is added (S132: Yes), the binary value included in the sound data as a frequency component below the maximum audible limit is sequentially stored in the RAM 103 according to the order of the received sound data. Write to a predetermined area (S133). The audio data stored in the reception buffer 125b is erased from the reception buffer 125b after writing the binary value of the frequency component below the maximum audible limit in a predetermined area of the RAM 103. Thereafter, the binary number written in the predetermined area of the RAM 103 is read every 8 bits, and the read 8-bit symbol code is converted into symbol data by referring to the symbol data memory 14b of the hard disk 14 (S134). Then, the converted symbol data is output to a predetermined area (predetermined window) of the main screen of the telephone process displayed on the CRT display 112 (S135). In the IP telephone 200c, ASCII image data is output to the LCD by the process of S135.

  Thereafter, the audio data stored in the reception buffer 125b is subjected to IDCT processing (inverse discrete cosine transform processing) (S136), and the audio data subjected to IDCT processing is output to the audio CODEC 109 (S137). The output process (S110) ends. The audio data output to the audio CODEC 109 is converted into analog data by decoding and reproduced and output from the speaker 114.

  On the other hand, if the ASCII possession code is not added as a result of the confirmation in the process of S132 (S132: No), it is determined that the data (symbol code) forming the ASCII image is not included in the audio data, and S133. The process of S135 is skipped and the process proceeds to S136.

  When it is determined from each audio data in the packet whether or not a symbol code forming an ASCII image is included, the binary data of the component below the maximum audible limit is “0” and the symbol code is included. It is difficult to determine whether this is because it is not, or because it is a plain part of the image data. In some cases, all packets corresponding to one image frame cannot be discriminated until data reading is completed, and the processing speed of ASCII image formation may be reduced. However, in the second embodiment, since the voice data including the symbol code forming the ASCII image is indicated by the ASCII possessed code, it is easily determined whether or not the symbol data is included in the voice data. As a result, ASCII image formation can be executed smoothly.

  As described above, according to the second embodiment, in the IP telephone device 200 that outputs sound and an image to the counterpart device, the image sent to the counterpart device can be an ASCII image. For this reason, it is possible to communicate with an image having a taste different from that of a videophone that outputs an actual video. Further, in the case where it is not desired to transmit the actual video as it is to the counterpart device, it is possible to perform advanced communication based on the original video while protecting the privacy of the sender. In addition, ASCII images are drawn by combinations of symbols, so even if an apparatus or program without an image forming function has a function of displaying character symbols, the ASCII image can be displayed. Production can be performed.

  In the second embodiment, the IP telephone device 200 serving as the transmission side device sends an ASCII image to the counterpart device instead of the actual video when the ASCII image designation command is input. You may comprise so that an ASCII image may be sent to a partner apparatus with a video. Even if the ASCII image and the actual video (image data input from the CCD camera 116) are both sent to the counterpart device and the amount of image data sent from the sender device is increased, the ASCII image is converted into audio data. Since the data is transmitted, the process of transmitting the image data (image data transmission process) does not become complicated.

  In addition, when the IP telephone device 200 serving as the transmission side device is configured to send an ASCII image together with the actual video as described above, the actual video output to the IP telephone device 200 serving as the counterpart side device. And a switch for switching the output of the ASCII image, and monitoring means for monitoring the state of the switch by interrupt processing, and according to the state of the switch detected by the monitoring means, the received video and ASCII image You may comprise so that one may be output.

  Although the present invention has been described based on the above embodiments, the present invention is not limited to the above embodiments, and various improvements and modifications can be made without departing from the spirit of the present invention. It can be easily guessed.

  For example, in each of the above embodiments, in the ASCII conversion process (S28), the image frame is divided into blocks having the same size as the symbol size. The symbol size block size is the same as that in template matching. As long as they are matched, one image may be enlarged or reduced when matching. For this reason, the size of the block to be divided does not necessarily match the size of the symbol.

  Further, the number of blocks to be divided is not limited to 80 × 25. Further, the number of blocks may be a variable value, and may be arbitrarily set by the user inputting the value from the operation panel. . When the number of blocks is set by the user, the horizontal division number m and the vertical division number n are set so that the set number of blocks is obtained in the processing of S41 and S42. As a result, an ASCII image reflecting the user's intention can be created, and the degree of freedom of expression is improved and the entertainment is improved, and the user can perform desired deformation by his / her own operation. A high degree of satisfaction can be given to the user with respect to the ASCII image.

  Further, in each of the above-described embodiments, the background image is configured to be performed every time an effect of the ASCII image is started. However, the background image is acquired (the background image stored in the background image memories 13a and 103a). (Update of the image data) does not necessarily have to be performed every time the effect by the ASCII image is executed. For example, the background image is captured only when requested by the operator, and the image data of the background image stored in the background image memories 13a and 103a is newly acquired only when the background image is acquired. You may make it update with the image data of an image.

  In each of the above embodiments, each of the image frames is converted to an ASCII image. However, image frames are sampled at predetermined intervals from all acquired image data (image frames), and the sampled image frames are sampled. Only the ASCII image may be formed and the other image frames may be discarded. In addition, the user may arbitrarily set the predetermined interval for sampling. According to this, it is possible to efficiently form an ASCII image and increase the overall speed of the forming process, and it is possible to improve the entertainment of the moving image of the ASCII image reproduced as a frame advance display.

  Further, in the ASCII art process (S28, S107), when the area of the person image is less than half of the entire image frame, the image is enlarged in one step so that the area of the person image becomes half of the entire image frame. It was. Instead of this, it may be configured to enlarge the original person image in a stepwise manner with an enlargement ratio that is enlarged once as a fixed magnification. According to this, the person image can be enlarged to a magnification desired by the operator.

  In addition, in each of the above embodiments, when the image frame is divided, the human image data is enlarged if the area of the human image in the image frame is less than the predetermined area. Is greater than or equal to a predetermined area (the number of blocks having person image data is greater than or equal to a predetermined number), the person image data may be compressed. According to this, it is possible to avoid deterioration in visibility due to the person image being too large with respect to the display screen. In this case, the process for executing the compression of the human image data corresponds to the image data compression means according to claim 5.

  Furthermore, in each of the above-described embodiments, a CCD camera that acquires image data in real time is adopted as the image data input means for inputting image data. However, the input image data is limited to that acquired in real time. Alternatively, the image data input means for inputting the image data may be configured by a storage medium in which image data is stored in advance and a reading means for reading the image data stored in the storage medium.

It is the schematic which shows the karaoke apparatus provided with the image processing apparatus of this invention. It is a block diagram which shows the electric constitution of a karaoke apparatus. It is the figure which showed typically the process in which an ASCII image is formed. FIG. 4 is a flowchart of karaoke playback processing executed by the control unit of the karaoke apparatus main body. 5 is a flowchart of ASCII art processing executed in the karaoke playback processing of FIG. 6 is a flowchart of an ASCII conversion process executed in the ASCII art process of FIG. FIG. 5 is a schematic diagram showing a telephone system including an IP telephone device that is an image processing device of a second embodiment. It is a block diagram which shows the electrical constitution of the IP telephone apparatus of a telephone system. 6 is a flowchart showing telephone processing executed by an IP telephone device which is an image processing device according to a second embodiment. 10 is a flowchart of voice transmission processing executed in the telephone processing of FIG. 9. It is a flowchart of the audio | voice output process performed in the telephone process of FIG.

1 Karaoke device (image processing device)
1a Body (image processing unit)
2 CRT display (display device)
3,116 CCD camera (image data input means, imaging device)
13a, 103a Background image memory (background image storage means)
14b, 104b Symbol data memory (symbol data storage means)
14c Music data memory (accompaniment information storage means)
14d Lyric data memory (lyric data storage means)
21,115 Accompaniment device (accompaniment generating means)
23 Microphone (voice data input means)
24 Speaker (pronunciation means)
109 CODEC for voice (part of data conversion means, part of voice playback means)
120 Communication control unit (output means, part of data conversion means, voice data receiving means)
200 IP telephone apparatus, counterpart apparatus, transmission apparatus S15 part of output means S23, S24 unique image data extraction means, unique image data extraction steps S25, S29-S31 image data expansion means, image data expansion step S25 determination means, Determination step S27 Line drawing conversion means, line drawing conversion steps S41, S42 division means, division step S47 extraction means, extraction step S48 part of image formation means, image formation steps S121, S122 part of data conversion means S122 second removal means S124 Superimposing means S132, S137 Part of sound reproducing means S134, S135 Part of image data reproducing means S10 ASCII art processing (part of processing means, image processing program)

Claims (10)

Image data input means for inputting image data having a plurality of image frames and capable of appearing continuous images, processing means for processing the image data input by the image data input means for each image frame, and An image processing unit including an output unit that outputs image data by sequentially outputting image frames processed by the processing unit according to the input order; and the image frames output by the output unit of the image processing unit in order In an image processing apparatus comprising a display device for displaying and displaying the image data as a moving image,
The image processing unit includes a symbol data storage unit that stores image data of a plurality of symbols, and the processing unit of the image processing unit converts the image data input by the image data input unit into line image data A line drawing conversion means for converting the image data into a predetermined number of blocks; and an image frame of the image data input by the image data input means or an image frame having line image data converted by the line drawing conversion means Extraction means for extracting, in block units divided by the dividing means, symbols assigned according to the mode of the line image data from the symbol image data stored in the symbol data storage means; An image forming method for forming replacement image data by replacing the line image data of the corresponding block with the image data of the symbol extracted by the extraction means Includes bets, output means of the image processing unit, and outputs a replacement image data formed by the replacement image data,
It performs video display of the replacement image data at the display device,
The image processing unit includes an imaging device that acquires image data by photographing as the image data input unit, and further includes a background image storage unit that stores background image data acquired as a background image by the imaging device. With
The processing means of the image processing unit removes unique image data unique to each image frame by excluding a common part with the background image data stored in the background image storage means from the image data acquired by the imaging device. A unique image data extracting means for extracting,
The line drawing conversion means of the image processing unit converts the unique image data extracted by the unique image data extraction means into line image data,
The image forming unit of the image processing unit forms replacement image data by line image data converted from the unique image data,
When the image processing unit of the image processing unit divides the image frame, the number of blocks having the line image data or the unique image data in the image frame is equal to or less than a predetermined lower limit value. An image data expansion means for expanding the line image data or the unique image data in a direction to increase the number of blocks having the line image data or the unique image data in the image frame;
The image forming unit of the image processing unit forms replacement image data based on the decompressed image data when the line image data or unique image data is decompressed by the image data decompressing unit. An image processing apparatus. The image data decompression unit includes a determination unit that determines whether or not a ratio of the unique image data extracted by the unique image data extraction unit to a source image frame is equal to or less than a predetermined ratio. And
When the determining means determines that the ratio occupied by the unique image data is equal to or less than a predetermined ratio, the number of blocks having the unique image data is determined to be equal to or less than a predetermined lower limit value. The image processing apparatus according to claim 1 , wherein the image processing apparatus performs expansion. When the image processing unit of the image processing unit divides the image frame, the number of blocks having the line image data or the unique image data in the image frame is greater than or equal to a predetermined upper limit value. Image data compression means for compressing the line image data or the unique image data and reducing the number of blocks having the line image data or the unique image data in the image frame;
The image forming unit of the image processing unit forms replacement image data based on the compressed image data when the image data is compressed by the image data compression unit. Item 3. The image processing apparatus according to Item 1 or 2 . Accompaniment information storage means for storing accompaniment information;
Accompaniment generating means for generating accompaniment based on the accompaniment information stored in the accompaniment information storage means;
Voice data input means for inputting voice data;
A sounding means for sounding the sound data input by the sound data input means and the accompaniment generated by the accompaniment generating means;
Lyric data storage means corresponding to each accompaniment information and storing lyric data displayed on the display device in accordance with the accompaniment sounded by the sounding means,
With the lyrics data stored in the lyrics data memory means, according to any one of claims 1 to 3, characterized in that the replacement image data output by said output means is for displaying on said display device Image processing device. 5. An audio data input unit that includes the image processing unit according to any one of claims 1 to 4 and is connected to a counterpart device via a line, and that inputs audio data; and an audio that is input by the audio data input unit Data conversion means for converting data into a data signal to be output to the line, and an identification code corresponding to replacement image data formed by the image forming means of the image processing section is converted into a data signal of audio data to convert the data A transmitting side device having superimposing means for superimposing on the data signal converted by the means,
Said output means of the image processing unit of claims 1 to 4, characterized in that in which the data signal superimposed by the superimposing means and outputting to said line and outputs the replacement image data to the partner apparatus The image processing apparatus according to any one of the above. The transmission side device includes first removal means for removing a non-reproduction frequency component set as a non-reproduction area for the audio data input by the audio data input means,
The superimposing unit converts the identification code of the replacement image data into a data signal of a band of the non-reproducing frequency component removed by the first removing unit and superimposes it on the data signal converted by the data converting unit The image processing apparatus according to claim 5, wherein: The transmission side device includes second removal means for removing a limit frequency component exceeding an audible limit for the voice data input by the voice data input means,
The superimposing means converts the identification code of the replacement image data into a data signal of a limit frequency component removed by the second removing means and superimposes it on the data signal converted by the data converting means. The image processing apparatus according to claim 5 or 6 . The display device, voice data receiving means for receiving voice data, and voice playback means for playing back voice data received by the voice data receiving means by voice while being connected to the transmission side device via the line And when the audio data received by the audio data receiving means includes a data signal of an identification code corresponding to the replacement image data, image data reproduction means for reproducing the replacement image data from the data signal a, the image processing apparatus according to any one of claims 5, characterized in that it comprises a mating device 7 for displaying the replacement image data reproduced by the image data reproduction means at the display device. Processing for processing image data input by the image data input means for each image frame in a computer having image data input means for inputting image data having a plurality of image frames and capable of appearing continuous images An image processing step comprising: a step, and an output step for outputting image data by sequentially outputting the image frame processed by the processing step according to the input order; In the image processing program for displaying the image frames in order and displaying the image data as a moving image on the display device of the computer ,
The computer includes symbol data storage means in which image data of a plurality of symbols is stored,
The processing step of the image processing step includes a line drawing conversion step of converting the image data input by the image data input means into line image data, and an image frame of the image data input by the image data input means or the A division step for dividing an image frame having line image data converted by the line drawing conversion step into a predetermined number of blocks, and a block unit divided by the division step , are allocated according to the mode of the line image data . An extraction step for extracting the symbol from the symbol image data stored in the symbol data storage means, and a replacement image obtained by replacing the line image data of the corresponding block with the symbol image data extracted by the extraction step and an image forming step of forming a data is executed on the computer, the image processing Output step of step is for outputting the replacement image data formed by the replacement image data,
The moving image of the replacement image data is displayed on the display device,
The computer includes an imaging device that acquires image data by photographing as the image data input unit, and further includes a background image storage unit that stores background image data acquired as a background image by the imaging device,
In the processing step of the image processing step, unique image data unique to each image frame is removed from the image data acquired by the imaging device except for a common portion with the background image data stored in the background image storage unit. Causing the computer to execute a unique image data extraction step to extract,
The line drawing conversion step of the image processing step converts the unique image data extracted by the unique image data extraction step into line image data,
The image forming step of the image processing step is to form replacement image data by line image data converted from the unique image data,
In the processing step of the image processing step, when the image frame is divided by the dividing step, the number of blocks having the line image data or the unique image data in the image frame is equal to or less than a predetermined lower limit value. , Causing the computer to execute an image data expansion step of expanding the line image data or the unique image data in a direction to increase the number of blocks having the line image data or the unique image data in the image frame,
In the image forming step of the image processing step, when the line image data or unique image data is expanded by the image data expansion step, replacement image data is formed based on the expanded image data. An image processing program characterized by the above. The image data decompression step includes a determination step of determining whether a ratio of the unique image data extracted by the unique image data extraction step to a source image frame is equal to or less than a predetermined ratio. Let the computer run,
In the determination step, when it is determined that the ratio occupied by the unique image data is equal to or less than a predetermined ratio, the number of blocks having the unique image data is determined to be equal to or less than a predetermined lower limit value. claim 9, wherein the image processing program, characterized in that performs a decompression.

Images