TWI535281B - System device and method of compositing a real-time selfie music video with effects adapted for karaoke - Google Patents

Description

System device and method for real-time self-timer special effect synthesis MV applied to phonograph

The invention relates to a system device and a method thereof, in particular to a system device and a method for realizing a self-timer special effect synthesis MV of a phonograph.

Generally, when playing karaoke equipment, use the built-in presets or externally stored landscapes, plots, situational movies or pictures as the background, and stack the lyrics subtitles on top of them, or directly use the original singer's music video (MV). Play, provide user sing. However, when users are singing, they mostly want to be the protagonists in the music video (MV). Currently, the more advanced karaoke equipment allows users to connect external video input devices, such as cameras, but only Taking the original monotonous user image on the spot, it is not possible to instantly generate a dynamic visual effect similar to the MV clip containing the user's image and various songs. However, if you want to use the computer to achieve this effect, not only lack of immediacy, but also A huge expense.

One of the objects of the present invention is to provide a system device and a method for real-time self-timer MV (Music Video) applied to a karaoke machine (karaoke machine/karaoke/Karaoke), which can be displayed or recorded in real time. A variety of situations and visual effects.

In order to achieve the above, the present invention provides a system device for real-time self-timer effect synthesis MV for a phonograph, comprising an image input interface for inputting an image output by an image output device to generate an original image data; a special effect The image library is used for storing at least one special effect image data; a lyrics subtitle library is used for storing lyrics subtitle data of at least one song; a dynamic image synthesis processing unit is coupled to the image input interface and the special effect image a library for receiving and processing the original image data, and reading, by the special effect image library, the at least one special effect image data and the original image data for real-time synthesis processing to generate a synthetic image data; a core processing unit The lyrics subtitle library is coupled and communicatively coupled to the dynamic image synthesizing processing unit for receiving and processing the synthesized image data, and the lyrics subtitle library reads the lyrics subtitle data of the at least one song for superposition processing and coating And the image processing interface is coupled to the core processing unit for outputting the synthesized image data and the lyrics subtitle data of the at least one song to a display device, so that the display device displays the synthesized image data and the The lyrics subtitle data of at least one song.

In order to achieve the above description, the present invention provides a method for synthesizing a MV of an instant self-timer effect of a phonograph to instantly synthesize at least one user's image, and the method for synthesizing a MV of an instant self-timer effect applied to a phonograph includes: receiving The original image data of the user; discriminating and selecting at least one special effect image data; synthesizing the original image data and the special effect image data to form a synthetic image data; superimposing the lyrics subtitle data; and displaying the synthesized image data And the lyrics subtitles.

For a better understanding of the features and technical aspects of the present invention, reference should be made to the accompanying drawings.

A‧‧‧ user

M‧‧‧ record player

100‧‧‧ phonograph real-time audio and video special effects synthesizer

C‧‧‧Image output device

D‧‧‧ display device

D1‧‧‧ screen

11‧‧‧Image input interface

12‧‧‧Image output interface

13‧‧‧Core Processing Unit

14‧‧‧Dynamic image synthesis processing unit

15‧‧‧Special Effects Image Library

16‧‧‧ Lyrics subtitle library

17‧‧‧Internal storage media

18‧‧‧External storage media

19‧‧‧Operating unit

191‧‧‧Operator interface

20‧‧‧Optical Processing Unit

21‧‧‧recording unit

22‧‧‧Network interface

FIG. 1 is a schematic structural diagram of an embodiment of the present invention.

2 is a functional block diagram of an embodiment of the present invention.

Figure 3 is a schematic illustration of an embodiment of the invention.

4 is a flow chart of a method of the present invention.

Referring to FIG. 1, the present invention mainly provides a system device for real-time self-timer effect synthesis MV applied to a phonograph. As shown in FIG. 1, the system device for real-time self-timer special effect synthesis MV applied to a phonograph comprises a phonograph real-time audio and video special effects synthesizing device 100, an image output device C, and a display device D. The phonograph real-time audio and video special effects synthesizing device 100 can be coupled between an image output device C and a display device D. The phonograph M of the present invention has a device for processing multimedia materials and reproducing multimedia materials and having a vocal function, and the phonograph M can also be called a karaoke machine/karaoke/Karaoke. In addition, it should be emphasized that the term "video" in the present invention may substantially include at least one of an image or a sound, and is not limited to two formats including an image or a sound.

Please refer to Figure 2 and Figure 3. 2 is a functional block diagram of a preferred embodiment of the present invention. As shown in FIG. 2, the phonograph real-time audio and video special effects synthesizing device 100 basically includes an image input interface 11, an image output interface 12, a core processing unit 13, a motion image synthesis processing unit 14, and a special effect image library 15. Further, the phonograph real-time audio and video special effects synthesizing device 100 may further include a lyrics subtitle library 16, an internal storage medium 17, an external storage medium 18, an operation unit 19, an audio processing unit 20, a recording unit 21, and A network interface 22.

The image input interface 11 can be connected to the image output device C for inputting the image output by the image output device C. The image output device C1 may include, but is not limited to, a charge coupled device (CCD), a camera, a computer or a webcam (PC-cam), or a camera having a function of capturing images. Can be any multimedia player (Media player). The image output interface 11 can be coupled to the display device D for outputting images to the display device D. The display device D may include, but is not limited to, a CRT, a liquid crystal display (LED, LCD) or a plasma display screen, a projector, and the like. The image input interface 10 and the image output interface 20 may include, but are not limited to, an HDMI interface, a CVBS interface, a color difference terminal interface, an AV terminal interface, a VGA interface, a DVI interface, a wireless signal transmission interface, and the like.

The core processing unit 13 can be a central processing unit (CPU) or a single chip system (SOC) and can be the main control element for controlling the phonograph M.

The dynamic image synthesis processing unit 14 is coupled to the image input interface 11 and communicatively coupled to the core processing unit 13. The motion picture synthesis processing unit 14 can be a separate digital signal processor (DSP), can be integrated into the core processing unit 13 or can be placed in a set-box. In addition, an executable program can be written in the motion picture synthesis processing unit 14. The motion image synthesis processing unit 14 can receive one of the original image data generated by the image captured by the image output device C via the image input interface 11. The image captured by the image output device C can be presented in the screen D1 of the display device D via the image output interface 12. In this embodiment, the image output device C is a camera that can capture a continuous motion image. The continuous motion image captured by the image output device C presents at least one user A (or may be referred to as a singer) that is immediately incident on the image output device C in the screen D1 of the display device D, that is, the image output device C The continuous motion image captured is a real-time image of the user. It can also be said that the image output device C can instantly capture the real-time image of the user A and convert it into an original image data received by the dynamic image synthesis processing unit 14 via analog/digital conversion.

The effect image library 15 can be built into the internal storage medium 17 or directly placed in the motion picture synthesis processing unit 14. The internal storage medium 17 can include, but is not limited to, a flash memory, a temporary memory, and a random access memory. The special effect image library 15 pre-stores a plurality of special effect image materials for selection and application. The special effect image library 15 includes at least the following: snowflake, star special effect image data, bubble special effect image data, scene special effect image data, facial special effect image, and the like, and various special effects images with visual effects and visual effects often appear in the MV film. data. The above special effect image data may include a dynamic object and a static object.

The dynamic image synthesis processing unit 14 receives the original image data generated by the image captured by the image output device C via the image input interface 11, and then reads a special effect image data or one or more special effect image data from the special effect image library 15 according to the instruction. The application image is combined with the original image data and the special effect image data to generate a composite image data output to the core processing unit 13 and output through the image. The interface 12 is instantly displayed in the screen D1 of the display device D. In detail, the motion image synthesis processing unit 14 can automatically perform various predetermined motion image processing and predetermined dynamic composition processing on the original image data according to the program written therein, for example, brightness, hue, contrast, saturation, screen segmentation, Stacking, rotation processing, etc. It is also possible to perform face recognition to background synthesis processing, or face copying to background synthesis processing, or human body recognition to background synthesis processing, etc., so that the original image data presents a picture similar to the MV movie effect. The original image data is processed and then combined with the special effect image data to generate synthetic image data.

The lyrics subtitle library 16 can be built on the internal storage medium 17 and pre-stored lyrics subtitle data of a plurality of songs. The core processing unit 13 is coupled to the lyrics subtitle library 16, and after the core processing unit 13 receives the synthesized image data output by the dynamic image synthesis processing unit 14, the lyrics subtitle data of the corresponding song can be read by the lyrics subtitle library 16 and The lyrics subtitle data of the corresponding song and the synthetic image data are superimposed and painted, and the word processing refers to the color of the lyrics displayed by the lyrics subtitle data is white-colored with the song, and may of course be white Painted in red or green, there is no limit to the color of the painting. You can also understand the progress of the current song (for example, the dot effect according to the progress of the song) by different visual effects, and then The synthesized image data and the lyrics subtitle data are instantly presented in the screen D1 of the display device D via the image output interface 12, so as to instantly display a picture similar to the MV movie effect and the lyric composition effect.

The operating unit 19 is coupled to the core processing unit 19. The operating unit 19 can include, but is not limited to, a remote controller, a tablet, a smart phone, a sensor, and the like. The operating unit 19 has an operation interface 191, which can be a plurality of functional buttons, or a user graphical interface, a human motion sensing interface, a voice control interface, and the like. Therefore, the core processing unit 13 and the motion image synthesis processing unit 14 are configured according to a human body motion or sound command, or by pressing or touching a functional button or a functional image on the operation interface 191 for the user to input an instruction. The user performs an operation corresponding to the instruction input by the operation interface 191. Therefore, when the display device D is displayed in the screen D1 Immediately after the user A of the image output device C, the user A can select an effect image data or two or more special effect image data and the original image data from the special effect image library 15 through the operation interface 191 to issue an instruction. The special effects are synthesized and instantly presented in the screen D1 of the display unit D for the user A to view instantly. For example, when the user A wants to present the situation in the snow D1 in the screen D1, the user can select the snowflake effect image data from the special effect image library 15 to perform instant synthesis through the operation interface 191, or can be combined with the effect of changing the color tone. The instant synthesis of the image data is like being in a snowy and discolored situation. In addition, other special effect image materials, such as facial special effect image data, can be selected by the special effect image library 15 for real-time synthesis, and the visual effects of the facial effects can be presented. In addition, the user A can also set the instruction through the function of the operation interface 191 to add art effect processing, such as oil painting effect, watercolor effect, sketch effect and the like. In this way, the user A performs the instant synthesis of the special effect image data selected by the operation interface 191 and the original image data including the user A in the song singer to instantly present various various situational effects and visual effects. In the screen D1 of the display device D, the user is made to be like a MV film director.

The audio processing unit 20 can be built into the core processing unit 13 or coupled to the core processing unit 13 as a separate audio or sound card. The audio processing unit 20 is configured to receive a plurality of or one user A voice (or a user's voice) and select a background music of the corresponding song from the internal storage medium 17 to perform superimposition and mixing processing to generate a mixture. Audio data.

The recording unit 21 can be built into the core processing unit 13 or coupled to the core processing unit 13 for a separate recording chip. The recording unit 21 is configured to selectively record the synthesized image data, the mixed data, and the subtitle data of the song to form a video file. Thereby, the user A can set the recording function option via the operation interface 191 of the operation unit 19, and select only the video recording or the simultaneous recording and recording to record the instant image or the singing voice containing the self-photographed and synthesized by the user A, and further store the image. The medium 18 is stored externally. The external storage medium 18 can include a hard disk, a compact disc, a flash drive, and a memory. card.

The network interface 22 can be coupled to the core processing unit 13 for an Ethernet interface card or a wireless network interface card. The network interface 22 is used to connect to a network by wire or wirelessly. The above network can be a wireless network, a regional network or an internet network. Therefore, the user A can set the network function option to perform network connection via the operation interface 191 of the operation unit 19, so that the video file stored in the external storage medium 18 is read by the core processing unit 13 and then uploaded to the network. share it. In addition, the wireless transmission can be directly performed through the network interface 22, so that the audio and video files can be transmitted to the tablet or the smart phone within the distance range of the wireless signal for sharing. The video files recorded by the user A's self-portrait and synthesized instant video and sound can be immediately shared with their friends, relatives or the public.

Please refer to FIG. 4 with reference to FIG. 2 . As shown in FIG. 4, it is a main flow step of a method for synthesizing an MV of an instant self-timer effect applied to a phonograph, and the method is used for synthesizing at least one image of the user A, and the image of the user can be any The image output device C immediately captures and performs processing such as encoding and decoding and compression to generate original image data of the user. First, the original image data of the user is received by the dynamic image processing unit 14 (S201), and the original image data may be subjected to dynamic image pre-processing (eg, brightness, hue, contrast, saturation, picture segmentation, stacking). (S203), and then the motion image processing unit 14 can determine and select at least one special effect image data (S205) from the special effect image library 15 (for example, snowflake effect image data, rain effect image data, star effect image data) , scene effects video data). The method for determining the selected effect image data may randomly select at least one special effect image data from a special effect image library 15 according to a random number rule, or sequentially select the at least one from a special effect image library 15 according to a specific order. The special effect image data, or the instruction input by the operation unit 19 according to the user's preference, selects the at least one special effect image data from a special effect image library 15, or automatically selects the corresponding type according to the type or beat of the music detected by the system device. Special effects image data. After that, the motion image synthesis processing unit 14 processes the processed original image data and special effect image resources. The material is subjected to real-time dynamic synthesis to form a synthetic image data (S207). The method for real-time dynamic synthesis of the processed original image data and the special effect image data can utilize the face recognition technology to capture the face of the user as a foreground, and then perform instant back-shadow synthesis with the special effect image data to form a synthetic image data. Or use the human body identification technology to capture the user's whole body as a foreground, and then perform instant background synthesis with the special effect image data to form a synthetic image data. After that, the synthetic image data may be subjected to dynamic image post-processing (S209) (for example, oil painting effect, watercolor effect, sketch effect, etc.), in other words, the synthetic image data may be calculated according to the instruction to render the synthetic image data. A picture with artistic effects such as oil painting, watercolor or sketch. Finally, the lyrics subtitle data (including the wording effect) is superimposed (S211), and the processed synthetic image data and the lyrics subtitle data are displayed on the screen D1 of a display device D (S211) to instantly display an MV-like movie. The effect and the screen with the matching effect. In addition, a sound mixing material including a user's singing voice and a back music of a song may be received, and only the synthesized image data, the lyrics subtitle data, or the mixed data may be recorded to form a video file, and then the video file may be stored or The image file is transferred for sharing.

The above description is only a preferred embodiment of the present invention, and is not intended to limit the scope of the present invention, and the equivalents of the present invention are intended to be included within the scope of the present invention.

C‧‧‧Image output device

D‧‧‧ display device

D1‧‧‧ screen

11‧‧‧Image input interface

12‧‧‧Image output interface

13‧‧‧Core Processing Unit

14‧‧‧Dynamic image synthesis processing unit

15‧‧‧Special Effects Image Library

16‧‧‧ Lyrics subtitle library

17‧‧‧Internal storage media

18‧‧‧External storage media

19‧‧‧Operating unit

191‧‧‧Operator interface

20‧‧‧Optical Processing Unit

21‧‧‧recording unit

22‧‧‧Network interface

Claims (14)

A system device for real-time self-timer special effect synthesis MV for a phonograph machine, comprising: an image input interface for inputting an image output by an image output device to generate an original image data; and a special effect image library for storing Having at least one special effect image data; a dynamic image synthesis processing unit coupled to the image input interface and the special effect image library for receiving and processing the original image data, and reading the at least one special effect image by the special effect image library The data is combined with the original image data to generate a synthetic image data; a lyrics subtitle library is used to store at least one song lyrics subtitle data; a core processing unit is coupled to the lyrics subtitle library and Communicating with the dynamic image synthesizing processing unit for receiving and processing the synthesized image data, and reading the lyrics subtitle data of the at least one song from the lyrics subtitle library and superimposing and smearing the synthesized image data Processing and an image output interface coupled to the core processing unit for outputting the synthesized image data and And displaying, by the display device, the synthesized image data and the lyrics subtitle data of the at least one song; wherein the dynamic image synthesis processing unit is based on the at least one song Randomly selecting the at least one special effect image data from the special effect image library; or sequentially selecting the at least one special effect image data from the special effects image library according to a specific order; or according to an instruction input by an operation unit The at least one special effect image data is selected in the special effect image library. The system device for applying the instant self-timer effect synthesis MV of the karaoke machine according to claim 1, wherein the image output by the image output device is an instant image of a user. The system device for applying the instant self-timer effect synthesis MV of the phonograph according to claim 1, wherein the special effect image data is at least one of the following: snowflake effect image data, star effect image data, bubble special effect image data, scene Special effects image data. The system device for applying the instant self-timer effect synthesis MV of the phonograph according to claim 1, wherein the special effect image data comprises at least one of the following: a dynamic object and a static object. The system device for applying the instant self-timer effect synthesis MV of the karaoke machine according to claim 1, wherein the operation unit is coupled to the core processing unit, and the operation unit has an operation interface for the user to input an instruction to enable the core processing The unit and the motion picture synthesis processing unit perform corresponding operations according to instructions input from the operation interface. The system device for the instant self-timer effect synthesis MV of the karaoke machine according to claim 1, further comprising an audio processing unit coupled to the core processing unit, the audio processing unit for receiving at least one use The vocal of the person is superimposed and mixed with the background music of a corresponding song to generate a mixed material. The system device for the instant self-timer effect synthesis MV of the karaoke machine according to claim 6, further comprising a recording unit coupled to the core processing unit, the recording unit for selectively recording the composite image The data, the mixing material, and the lyrics subtitle data to form a video file. The system device for the instant self-timer effect synthesis MV of the karaoke machine according to claim 7, further comprising a storage medium, the storage medium comprising an internal storage medium and an external storage medium, wherein the internal storage medium is coupled to the motion image a processing unit and the core processing unit, wherein the internal storage medium is used to store the lyrics subtitle library and the special effect image library, the external storage medium is coupled to the recording unit and the core processing unit, and the external storage medium is used for storing The video file. The system device for applying the instant self-timer effect synthesis MV of the karaoke machine according to claim 8, further comprising a network interface coupled to the core processing unit for connecting to a network, thereby enabling The video file is read by the core processing unit and uploaded to the network for sharing. A method for synthesizing MVs by using a real-time self-timer effect of a phonograph machine to instantly synthesize at least one user's image, and the method for synthesizing MVs by using an instant self-timer effect of a phonograph includes: receiving original image data of one of the users; And selecting at least one special effect image data; wherein the method for determining and selecting at least one special effect image data is: randomly selecting the special effect image data from a special effect image library according to at least one song; or selecting a special effect image according to a specific order Selecting the special effect image data in the library sequentially; or selecting the special effect image data from a special effect image library according to an instruction input by an operation unit; synthesizing the original image data and the special effect image data to form a synthetic image data Superimposing the lyrics subtitle data; and displaying the synthesized image data and the lyrics subtitle data. The method for synthesizing an MV of an instant self-timer effect applied to a phonograph according to claim 10, further comprising performing dynamic image pre-processing on the original image data. The method for synthesizing a MV for an instant self-timer effect applied to a phonograph according to claim 10, further comprising performing dynamic image post-processing on the synthesized image data. The method for synthesizing an MV of an instant self-timer effect applied to a karaoke machine according to claim 10, wherein the method of synthesizing the original image data and the special effect image data is performed by performing facial recognition on the original image data of the user. The user's face is captured as a foreground and the special effect image data is instantly combined to form a background image to form the synthesized image data. The method for synthesizing an MV of an instant self-timer effect applied to a phonograph according to claim 10, wherein the method for synthesizing the original image data and the special effect image data in real time Performing human body recognition on the original image data of the user and extracting the whole body of the user as a foreground and the special effect image data for immediate background synthesis to form the synthesized image data.