JP2010034760A - Video/audio signal processing apparatus and video/audio signal processing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To perform the processing of an audio signal having a high degree of freedom when outputting an asynchronous video signal on which an audio signal is superposed as a video signal synchronized with a reference signal. <P>SOLUTION: A video signal DSa on which an audio signal is superposed is written in a memory 16. The video signal written in the memory 16 is read out synchronously with a reference signal. An audio signal DAa (DAb) separated from the video signal or an input audio signal DAc is written in a memory 26. The audio signal written in the memory 26 is read synchronously with a reference signal. Instead of an audio signal superposed on a video signal DSb (DSc) read out from the memory 16, a multiplexor 33 superposes an audio signal DAe (DAf) read from the memory 26. Since the audio signal is stored separately from the video signal, the processing of an audio signal having a high degree of freedom can be performed by using the separately stored audio signal. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a video / audio signal processing apparatus and a video / audio signal processing method. Specifically, when outputting an asynchronous video signal on which an audio signal is superimposed as a video signal synchronized with a reference signal, a memory for storing the audio signal is provided to process the audio signal with a high degree of freedom. Is to be able to.

  Conventionally, in a video / audio signal device such as a frame synchronizer, even if a video signal is lost or repeated when the synchronization of a video signal is converted to another video signal, a continuous auxiliary signal superimposed on the video signal is generated. It is done to keep sex.

  For example, in the invention of Patent Document 1, a video / auxiliary data separation unit separates video data and auxiliary data. The video data storage unit stores the separated video data. The auxiliary data storage unit stores the separated auxiliary data. Here, when the video data is not read out in the input order, the auxiliary data is read out continuously in the input order so that the auxiliary data is not lost or repeated by controlling the reading order. It is made to output automatically.

  In addition, in the frame synchronizer, not only video signals and auxiliary signals but also audio signals are switched. FIG. 12 shows the configuration of a conventional frame synchronizer 60.

  In the frame synchronizer 60, an asynchronous video signal on which an audio signal is superimposed is supplied as a stream of a predetermined format, for example, an SDI format. The stream DSin is converted into a parallel signal by the serial / parallel converter (S / P) 61 and supplied to the video signal input processor 62.

  The video signal input processing unit 62 performs processing of the video signal supplied from the serial / parallel conversion unit 61, for example, adjustment of brightness, insertion of video, and the like, and supplies the processed video signal DSh to the memory controller 65.

  The memory controller 65 writes the supplied asynchronous video signal DSh into the memory 66 and reads out the video signal written in the memory 66 in synchronization with the reference signal, so that it is independent from the input video signal DSh. A video signal DSj having a synchronous phase is supplied to the demultiplexer 67. Further, the memory controller 65 changes the read start position of the video signal written in the memory 66 and performs the repositioning process, for example, the video display position so that the lower area of the video can be displayed in the upper area on the screen. Is moved vertically.

  The demultiplexer 67 separates the video signal DSj on which the audio signal is superimposed into the video signal DVj and the audio signal DAj, supplies the video signal DVj to the video signal output processing unit 71, and outputs the audio signal DAj to the audio signal This is supplied to the processing unit 72.

  The video signal output processing unit 71 changes the blanking period when the reposition processing is performed by the memory controller 65. That is, if the read start position of the video signal written in the memory 66 is changed to move the video display position in the vertical direction, the blanking period also moves. Therefore, the video signal output processing unit 71 can display the video moved in the vertical direction by changing the blanking period. The video signal output processing unit 71 masks unnecessary display generated by the repositioning process, for example, display of the upper area of the video or display corresponding to the blanking period before replacement. The video signal output processing unit 71 supplies the video signal DVk after the signal processing to the multiplexer 73.

  The audio signal output processing unit 72 prevents the audio signal whose order has been changed from being output when the order of the audio signals recorded in the auxiliary data area of the video signal is changed by the repositioning process. Is silenced. The audio signal output processing unit 72 also performs audio fade processing and the like. The audio signal output processing unit 72 supplies the audio signal DAk after the signal processing to the multiplexer 73.

  The multiplexer 73 inserts the audio signal DAk supplied from the audio signal output processing unit 72 into the auxiliary data area of the video signal DVk supplied from the video signal output processing unit 71 to parallel / serial conversion unit (P / S). 74.

  The parallel / serial conversion unit 74 converts the video signal supplied from the multiplexer 73 into a serial signal, and outputs the serial signal as a stream DSout 'in the SDI format.

  The local CPU 81 generates a control signal CTb according to the control signal CTm supplied from the main CPU (not shown). The local CPU 81 supplies the generated control signal CTb to the memory controller 65, the video signal output processing unit 71, the audio signal output processing unit 72, and the like so that the operation of the frame synchronizer 60 becomes the operation instructed by the control signal CTm. Control each part.

JP 2004-221841 A

  By the way, in the frame synchronizer 60 as shown in FIG. 12, the output processing of the video signal with respect to the video signal DSb read from the memory 66 and the audio signal superimposed on the video signal DSb read from the memory 66. The audio signal is output. For this reason, an audio signal output process with a high degree of freedom cannot be performed with a simple configuration. For example, when the repositioning process is performed, the order of the audio signals is changed, so that the sound is silenced. Therefore, when the reposition process is performed, it is impossible to output sound. In addition, although voice fade-out and fade-in processing is possible, voice switching such as voice cross-fading cannot be performed.

  Therefore, according to the present invention, a video / audio processing apparatus and a video / audio that can perform processing of a highly flexible audio signal when an asynchronous video signal on which the audio signal is superimposed is output in synchronization with a reference signal. A processing method is provided.

  According to a first aspect of the present invention, a first memory for storing a video signal on which an audio signal is superimposed, and a first memory for reading the video signal written in the first memory in synchronization with a reference signal A controller, a demultiplexer that separates the audio signal superimposed on the video signal, a second memory that stores the separated audio signal or the input audio signal, and the second memory written to the second memory A second memory controller that reads out an audio signal in synchronization with the reference signal and an audio signal superimposed on the video signal read out from the first memory are read out from the second memory. The audio / video signal processing includes a multiplexer for superimposing the audio signal.

According to a second aspect of the present invention, a step of storing a video signal on which an audio signal is superimposed in a first memory and a video signal written in the first memory are read out in synchronization with a reference signal. A step of separating an audio signal superimposed on the video signal, a step of storing the separated audio signal or an inputted audio signal in a second memory, and writing to the second memory The audio signal read from the second memory instead of the step of reading out the audio signal synchronized with the reference signal and the audio signal superimposed on the video signal read from the first memory There is a step of superimposing a video and audio signal.

  In the present invention, an input video signal in which an audio signal is superimposed, for example, in a blanking period is written into the first memory by the first memory controller. Further, the video signal written in the first memory is read out by the first memory controller in synchronization with the reference signal.

  The audio signal superimposed on the video signal is separated by the demultiplexer, and the separated audio signal or the inputted audio signal is written into the second memory by the second memory controller. Furthermore, the audio signal written in the second memory is read out by the second memory controller in synchronization with the reference signal.

  In the audio signal output processing unit, for example, a sequence of the number of samples per video frame of the audio signal superimposed on the video signal read from the first memory is determined, and the second memory is determined based on the determination result. The number of samples of the audio signal read from the video signal is adjusted to be equal to the number of samples per video frame of the audio signal superimposed on the video signal read from the first memory. Further, in the adjustment of the number of samples per video frame of the audio signal, information indicating the number of samples per video frame of the audio signal written in the second memory is used.

  The multiplexer performs a process of superimposing the audio signal processed by the audio signal output processing unit instead of the audio signal superimposed on the video signal read from the first memory.

  In the audio signal output processing unit, a process of switching from one audio signal of the audio signal read from the second memory and the audio signal separated by the demultiplexer to the other audio signal and supplying it to the multiplexer is performed. Done.

  In addition, a video signal output processing unit that performs signal processing on the video signal read from the first memory is provided, and reading of the video signal written in the first memory by the first memory controller is provided. When the reposition process is performed to move the display position of the video by changing the start position, the video signal output processing unit can change the blanking period for the video signal read from the first memory. Done. Also, the audio signal is separated from the video signal written to the first memory and written to the second memory, and the audio signal written to the second memory is read out and superimposed on the video signal after the reposition processing. The

  According to the present invention, the video signal on which the audio signal is superimposed is stored in the first memory, and the video signal written in the first memory is read out in synchronization with the reference signal. The audio signal superimposed on the video signal is separated, and the separated audio signal or the input audio signal is stored in the second memory. The audio signal written in the second memory is read out in synchronization with the reference signal, and is superposed in place of the audio signal superimposed on the video signal read out from the first memory. As described above, by storing the audio signal separately from the video signal, an asynchronous video signal on which the audio signal is superimposed is output as a video signal synchronized with the reference signal. Processing can be performed.

  Hereinafter, an embodiment of the present invention will be described with reference to the drawings. FIG. 1 shows a configuration of a video / audio signal processing apparatus 10 for processing a video signal and an audio signal. The video / audio signal processing apparatus 10 is not only a function of a frame synchronizer, that is, a function of outputting an asynchronous video signal on which an audio signal is superimposed as a video signal synchronized with a reference signal, but also editing such as switching of an audio signal. It also has functions.

  In the video / audio signal processing apparatus 10, a plurality of asynchronous video signals on which audio signals are superimposed are supplied as streams in a predetermined format, for example, an SDI format. The stream DSin-1 is converted into a parallel signal by the serial / parallel converter (S / P) 11-1, and is supplied to the video signal input processor 12-1. Similarly, the streams DSin-2 to DSin-m are converted into parallel signals by the serial / parallel converters (S / P) 11-2 to 11-m and then sent to the video signal input processing units 12-2 to 12-m. Supplied.

  The video signal input processing unit 12-1 processes the video signal supplied from the serial / parallel conversion unit 11-1, for example, adjusts the brightness or inserts the video, and supplies the processed video signal to the selector 13. To do. Similarly, the video signal input processing units 12-2 to 12-m process the input video signal and supply the processed video signal to the selector 13.

  The selector 13 selects a desired video signal from a plurality of video signals supplied from the video signal input processing units 12-1 to 12-m, and selects the selected video signal DSa from the first memory controller 15 and the demultiplexer. 17 is supplied.

  The memory controller 15 writes the supplied asynchronous video signal DSa into the first memory 16 and reads out the video signal written in the memory 16 in synchronization with the reference signal, whereby the input video signal DSa and Supplies the video signal DSb having an independent synchronization phase to the video signal output processing unit 31. In addition, the memory controller 15 performs a repositioning process by changing the reading start position of the video signal written in the memory 16. For example, the readout start position is changed, and the video display position is moved in the vertical direction so that the lower area of the video can be displayed in the upper area on the screen. Alternatively, the readout start position is changed, and the video is moved in the horizontal direction or the oblique direction. In the following description, it is assumed that the display position of the video is moved in the vertical direction so that the lower area of the video can be displayed in the upper area on the screen.

  The demultiplexer 17 separates the audio signal DAa from the video signal DSa on which the audio signal is superimposed or separates the audio signal DAb from the video signal DSb on which the audio signal is superimposed. DAb is supplied to the selector 24 and the audio signal output processing unit 32.

  The selector 23 is connected to digital interface receivers (hereinafter referred to as “DIR”) 21-1 to 21-n. DIR-1 restores the audio stream DAin-1 supplied from the outside into an audio signal and supplies it to the selector 23. Similarly, DIR-2 to DIR-n restore the audio streams DAin-2 to DAin-n supplied from the outside into audio signals and supply them to the selector 23. The selector 23 selects a desired audio signal from a plurality of audio signals supplied from DIR-1 to DIR-n, and supplies the selected audio signal DAc to the selector 24.

  The selector 24 selects the audio signal DAa (DAb) separated from the video signal DSa (DSb) on which the audio signal is superimposed, or the audio signal DAc supplied from the selector 23 and supplies the selected signal to the memory controller 25.

  The second memory controller 25 writes the audio signal DAd supplied from the selector 24 in the second memory 26. Further, the memory controller 25 reads the audio signal written in the memory 26 in synchronization with the reference signal, thereby supplying the audio signal output processing unit 32 with the audio signal DAe synchronized with the reference signal. Since the memory 26 stores the audio signal, the memory capacity may be smaller than that of the memory 16 that stores the video signal. For example, when the total number of scanning lines of the video signal DSa is 1125, the total number of pixels per line is 2200 pixels, and it consists of a 10-bit luminance signal and a color difference signal, the data amount per video frame is 6.1 Mbyte (≈ 10 bits × 2 × 1125 lines × 2200 pixels). Here, when the audio signal DAd is 16 channels, the number of samples per video frame is 1602 samples, and the sample data is 32 bits, the data amount per video frame is 0.1 Mbyte (≈32 bits × 16 channels × 1602). Sample). Therefore, the memory 26 may have a smaller storage capacity than the memory 16.

  The video signal output processing unit 31 changes the blanking period when the reposition processing is performed by the memory controller 15. That is, if the reading start position of the video signal written in the memory 16 is changed to move the video display position in the vertical direction, the blanking period also moves. Therefore, the video signal output processing unit 31 can display the video moved in the vertical direction by changing the blanking period. Further, the video signal output processing unit 31 masks unnecessary display generated by the repositioning process, for example, display of the upper area of the video or display corresponding to the blanking period before replacement. The video signal output processing unit 31 supplies the video signal DSc after the signal processing to the multiplexer 33.

  The audio signal output processing unit 32 reads the audio signal DAe read from the memory 26 by the memory controller 25 when the order of the audio signals recorded in the auxiliary data area of the video signal DSc is changed by the reposition processing. Is supplied as an audio signal DAf to a multiplexer 33 and a digital interface transmitter (hereinafter referred to as “DIT”) 35. Also, the audio signal output processing unit 32 performs audio fading processing, audio signal DAe read from the memory 26 by the memory controller 25, and audio signal DAb separated by the demultiplexer, The voice switching process for switching to the voice signal is performed, and the processed voice signal is supplied to the multiplexer 33 and the DIT 35 as the voice signal DAf. Furthermore, the audio signal output processing unit 32 determines the sequence of the number of samples per video frame in the audio signal DAb superimposed on the video signal DSb, and the audio signal DAe read from the memory 26 by the memory controller 25. The number of samples is adjusted to correspond to the determined sequence.

  The multiplexer 33 superimposes the audio signal DAf supplied from the audio signal output processing unit 32 on the auxiliary data area of the video signal DSc supplied from the video signal output processing unit 31 to parallel / serial conversion unit (P / S). 34.

  The parallel / serial converter 34 converts the video signal DSd supplied from the multiplexer 33 into a serial signal and outputs the serial signal as a stream DSout in the SDI format.

  The DIT 35 packetizes the audio signal DAf supplied from the audio signal output processing unit 32 and outputs it as an audio stream DAout having a predetermined format.

  The local CPU 41 generates a control signal CTa according to a control signal CTm supplied from a main CPU (not shown). The local CPU 41 supplies the generated control signal CTa to the memory controllers 15, 25, the video signal output processing unit 31, the audio signal output processing unit 32, etc., and the operation of the video / audio signal processing device 10 is instructed by the control signal CTm. Each part is controlled so as to achieve the correct operation.

  As described above, the video / audio signal processing apparatus 10 includes the memory 16 that stores the video signal DSa and the memory 26 that stores the audio signal, and the audio signal is superimposed by using the audio signal written in the memory 26. It is possible to perform processing of an audio signal with a high degree of freedom when processing the processed video signal DSa.

  Next, a specific operation of the video / audio signal processing apparatus 10, for example, a case where reposition processing is performed will be described. FIG. 2 shows a signal path when repositioning a video based on the video signal DSa, for example. In FIG. 2, the path used for video / audio output in the path of the video signal and the audio signal is indicated by a solid line, and the path not used for the video / audio output is indicated by a broken line.

  The local CPU 41 controls the selector 13 to select the video signal supplied from the video signal input processing unit 12-1. In addition, the local CPU 41 controls the memory controllers 15 and 25 to write and read video signals to and from the memory 16 and write and read audio signals to and from the memory 26.

  The memory controller 15 writes the video signal DSa on which the audio signal is superimposed in the memory 16, changes the read position of the video signal written in the memory 16, and reads the video signal in synchronization with the reference signal. Perform reposition processing.

  FIG. 3 is a diagram for explaining an operation when performing the repositioning process. For example, the memory controller 15 sequentially writes the video signal DSa from the position of the address “0x0000” as shown in FIG. Note that the writing end position of the video signal of one frame (field) is an address “0xFFFF”.

  When performing the reposition process, the memory controller 15 changes the reading start position of the video signal. For example, the video signal is sequentially read from the memory 16 using the address “0x8000” as the video signal readout start position, and the video signal is sequentially read from the position of the address “0x0000” next to the video signal of the address “0xFFFF”. As described above, when the reading start position of the video signal is changed, the video read from the memory 16 becomes as shown in FIG.

  Here, when the reposition process is performed, the blanking period BKin is moved as shown in FIG. Further, the order of the audio signal DAa superimposed on the blanking period BKin is changed. Therefore, the video signal output processing unit 31 performs a blanking period replacement process, and newly sets a blanking period BKout for the video signal DSb read from the memory 16, as shown in FIG. Set. Furthermore, the video signal output processing unit 31 masks unnecessary video portions. For example, the blanking period BKin before the replacement or the upper part of the video is set as a masking area (area shown by cross-hatching) as shown in FIG. 3D, and masking processing such as filling the masking area with a predetermined color is performed. Do. In this way, the video signal DSc subjected to the blanking period replacement process and the masking process is supplied to the multiplexer 33.

  Further, when the reposition process is performed, the order of the audio signals is changed as described above. For this reason, the local CPU 41 controls the selector 24 and the memory controller 25 to write the audio signal DAa separated from the video signal DSa by the demultiplexer 17 into the memory 26 as shown in FIG. The audio signal superimposed on the video signal DSa, that is, the audio signal before the order is changed, is mirrored.

  The memory controller 25 reads the audio signal DAa written in the memory 26 in order of time as shown in FIG. 3F in synchronization with the reference signal, and supplies it to the audio signal output processing unit 32 as the audio signal DAe. The audio signal output processing unit 32 supplies the audio signal DAe read from the memory 26 to the multiplexer 33 as the audio signal DAf.

  The multiplexer 33 superimposes the audio signal DAf, which is an audio signal, on the blanking period of the video signal DSc supplied from the video signal output processing unit 31. Assuming that such processing is performed, the stream DSout output from the multiplexer 33 via the parallel / serial conversion unit 34 has undergone a repositioning process as shown in FIG. The signal indicates a video signal superimposed in the correct time order.

  Therefore, audio output can be performed correctly regardless of whether or not the video reposition processing has been performed.

  Next, another specific operation of the video / audio signal processing apparatus 10, for example, a case where an audio signal switching process is performed will be described. FIG. 4 shows a signal path when, for example, an audio signal superimposed on the video signal DSa is switched to an audio signal restored from the audio stream DAin-1. In FIG. 4, the path used for video / audio output in the path of the video signal and the audio signal is indicated by a solid line, and the path not used for the video / audio output is indicated by a broken line.

  The local CPU 41 controls the selectors 13 and 23 to select the video signal processed by the video signal input processing unit 12-1 and the audio signal obtained by the DIR 21-1. Further, the local CPU 41 controls the selector 24 to select the audio signal DAc. Further, the local CPU 41 controls the memory controllers 15 and 25 to write and read video signals to and from the memory 16 and write and read audio signals to and from the memory 26.

  The memory controller 15 writes the video signal DSa on which the audio signal is superimposed in the memory 16, sequentially reads out the video signal written in the memory 16 in synchronization with the reference signal, and outputs the video signal DSb as the video signal DSb. The video signal output processing unit 31 is supplied.

  The video signal output processing unit 31 supplies the video signal DSb supplied from the memory controller 25 to the multiplexer 33 as the video signal DSc.

  The demultiplexer 17 separates the audio signal DAb from the video signal DSb and supplies it to the audio signal output processing unit 32.

  The memory controller 25 writes the audio signal DAd supplied from the selector 24 in the memory 26. Further, the memory controller 25 reads out the audio signals written in the memory 26 in the order of writing, and supplies them to the audio signal output processing unit 32 as the audio signal DAe.

  The audio signal output processing unit 32 performs an audio signal switching process using the audio signal DAb supplied from the demultiplexer 17 and the audio signal DAe supplied from the memory controller 25.

  FIG. 5 shows a case where a crossfade process is performed as the audio signal switching process performed by the audio signal output processing unit 32. The audio signal output processing unit 32 attenuates the signal level (indicated by a solid line) of the audio signal DAb supplied from the demultiplexer 17 to “0” level as time passes. Further, the audio signal output processing unit 32 increases the signal level (indicated by a broken line) of the audio signal DAe supplied from the memory controller 25 to “0” level as time passes. In this way, the audio signal output processing unit 32 gradually replaces the audio signal DAb with the audio signal DAe as time elapses, and performs crossfade processing. The audio signal output processing unit 32 supplies the audio signal DAf after the cross fade process to the multiplexer 33.

  The multiplexer 33 superimposes the audio signal DAf supplied from the audio signal output processing unit 32 on the video signal DSc supplied from the video signal output processing unit 31. That is, the multiplexer 33 replaces the audio signal DAb superimposed on the video signal DSb with the audio signal DAf. Assuming that such processing is performed, the stream DSout output from the multiplexer 33 via the parallel / serial converter 34 indicates a video signal on which the audio signal subjected to the crossfade processing is superimposed.

  Incidentally, the audio signal superimposed on the video signal may have a different number of samples in one video frame. For example, when the sampling frequency of the audio signal is 48.0 kHz and the frame frequency of the video signal is (30 / 1.001) frames / second, the number of samples of the audio signal per video frame is not an integer. The number of samples in each frame is set so that the number of samples becomes 8008 samples in a period of 5 frames.

  FIG. 6 shows the number of samples of the frame and audio signal. In five consecutive frame periods, odd-numbered audio frames have 1602 samples, even-numbered audio frames have 1601 samples, and five frames and the next five frames have 1602 sample frames. For this reason, for example, when the number of samples of the audio signal DAe used in the crossfade processing is fixed to 1601 samples / frame, or when the number of samples of the audio signal DAe is fixed to 1602 samples / frame, video and audio This causes a phase difference and a phenomenon that the timing of video and sound does not match, that is, a so-called lip sync shift.

  Therefore, the audio signal output processing unit 32 prevents the occurrence of a lip sync problem by adjusting the number of samples per video frame and then superimposing it on the video signal.

  The audio signal output processing unit 32 determines the frame sequence based on the audio signal DAb supplied from the demultiplexer 17 and adjusts the number of samples per video frame of the audio signal based on the determination result. For example, as shown in FIG. 6, when the audio signal is 8008 samples in five consecutive frames, a series of frames whose number of samples is “1602” occurs every five frame periods. Therefore, the audio signal output processing unit 32 counts the number of samples of the audio signal for each frame based on the audio signal DAb, and counts 5 frames with reference to the timing when the frames having the number of samples of “1602” continue. If it is repeated, the count value indicates which frame of the 5-frame sequence, and it is possible to determine which frame of the 5-frame sequence corresponds to the frame to be output based on the count value.

  In addition, when the format of the audio stream DAin-1 is, for example, the AES format, information indicating the sampling frequency is provided in the AES format. One frame of the AES format is composed of two subframes, and the subframe is configured as shown in FIG.

  The subframe includes a preamble, AUX / audio, audio data, audio sample validity bit V, user bit U, audio channel status bit C, and subframe parity bit P.

  The preamble is 4-bit data and defines synchronization between subframes and frames and channel status block identification. The AUX / audio area is a 4-bit area and is an auxiliary data portion. Audio data is a 20-bit data area. If the audio data is 20 bits, the AUX / audio area can be used as an auxiliary data part. When the audio data is 24 bits, the AUX / audio area is used as the audio data area. When the audio data is less than 20 bits, the low-order bits on the LSB side that are not used are set to “0”.

  The audio sample validity bit V is 1-bit data indicating the validity of the audio data. Note that “0” is set when the audio data is valid data, and “1” is set when the audio data is invalid data. The user bit U is 1-bit data defined by the user. The audio channel status bit C is 1-bit data that forms a data block, and transmits transmission information about various parameters. The subframe parity bit P is 1-bit data indicating even parity for all data except the preamble.

  Here, for each channel, one block is formed by audio channel status bits C included in consecutive 192 frames. The start position of this block is specified by the preamble, and information such as the sampling frequency is indicated by the block of the audio channel status bit C. Therefore, the video / audio signal processing device 10 can determine the number of samples per video frame, for example, of the input audio signal based on this sampling frequency information.

  Therefore, the audio signal output processing unit 32 determines whether the output frame corresponds to which frame of the 5-frame sequence and the number of samples per frame of the audio signal written in the memory 26. By deleting or repeating the sample data of the signal DAe, an audio signal having the number of samples corresponding to the video signal DSc can be obtained. Therefore, the audio signal DAf supplied from the audio signal output processing unit 32 to the multiplexer is equal to the audio signal in which the number of samples per video frame is superimposed on the video signal DSc. In other words, it is possible to prevent a problem of lip sync deviation when the audio signal superimposed on the video signal is changed by performing cross fade processing, audio switching, or the like.

  Note that the determination of which frame in the 5-frame sequence corresponds to the frame to be output is performed when the counting operation of 5 frames is performed with reference to the continuous timing of the frames whose number of samples is “1602” as described above. The method is not limited, and other methods may be used.

  FIG. 8 shows the structure of a packet superimposed on the video signal. FIG. 8A shows a packet of an audio signal, and the auxiliary data flag ADF is a flag indicating the start of the auxiliary data packet. The data identification word DID indicates the type of auxiliary data. The data block number DBN is a sequence number of a series of packets having the same DID when data is divided and sent. The data count DC indicates the number of words of user data. The user data word UDW indicates auxiliary data such as an AES format audio signal. The checksum word CS is a checksum for data from the data identification word DID to the last word of the UDW.

  Further, in the packet superimposed on the video signal, not only the audio signal but also a control signal packet related to the audio signal is provided. FIG. 8B shows an audio control packet, and information such as an audio frame number AF and a sampling frequency RATE can be provided in the user data word UDW.

  The audio frame number AF is information for enabling identification of the number of samples superimposed in the video frame when the number of samples per video frame does not become an integer. For example, when the number of samples in a period of five consecutive frames is 8008 samples as described above, the number of samples per video frame is not an integer. Therefore, the audio frame number AF indicates what number frame is in the 5-frame sequence. Therefore, the number of samples in one frame can be determined from the audio frame number AF.

  The sampling frequency RATE is information indicating the sampling frequency of the audio signal, the active channel ACT is information indicating which channel is effective, and the delay DEL is data indicating the relative delay amount of the audio with respect to the video.

  As described above, since the audio frame number AF can determine which frame of the 5-frame sequence, the audio signal output processing unit 32 uses the information of the audio frame number AF to adjust the number of audio signal samples. Can also be done.

  FIG. 9 is a diagram for explaining the operation of the audio signal output processing unit 32. FIG. 9A shows the number of samples per video frame of the audio signal superimposed on the video signal DSb. FIG. 9B shows the number of samples per video frame of the audio signal DAe read from the memory 26 by the memory controller 25 and supplied to the audio signal output processing unit 32. FIG. 9C shows the number of samples per video frame of the audio signal DAf output from the audio signal output processing unit 32 and supplied to the multiplexer. Further, (D) of FIG. 9 shows the number of samples per video frame of the audio signal superimposed on the video signal DSd output from the multiplexer 33.

  Here, for example, when the audio signal superimposed on the video signal is switched to the audio signal written in the memory 26 at time tc1, the audio signal output processing unit 32 outputs one video frame of the audio signal DAf output to the multiplexer 33. The number of samples is adjusted so that the number of samples per unit is equal to the number of samples per video frame of the video signal DSd supplied to the multiplexer 33. For example, when the number of samples per video frame of the video signal DSd starting at time tc1 is “1602” and the number of samples per video frame of the audio signal DAe is “1600”, the audio signal output processing unit 32 The audio signal output processing unit 32 repeats the sampling data of the audio signal DAe to increase the number of samples by 2 and supplies the audio signal DAf with the number of samples per video frame of “1602” to the multiplexer 33. In the next frame, since the number of samples per video frame of the video signal DSd is “1601”, the audio signal output processing unit 32 repeats the sample of the audio signal DAe to increase the number of samples by one. An audio signal DAf with the number of samples per video frame being “1601” is supplied to the multiplexer 33.

  If such processing is performed, even if the audio signal is switched, the number of samples per video frame of the audio signal is equal to that before the switching, so that the problem of lip sync deviation does not occur.

  Further, when the audio signal written in the memory 26 changes the sample per video frame as in the above-described 5-frame sequence audio signal, the memory controller 25 outputs one video when writing the audio signal to the memory 26. Information indicating the sample per frame shall be written. In this way, even if the sample per video frame of the audio signal written to the memory 26 changes, the audio signal output processing unit 32 performs the audio according to the number of samples per video frame of the audio signal DAe. By repeating or deleting the sampling data of the signal, the audio signal DAf having the same number of samples per video frame as the audio signal superimposed on the video signal DSc can be supplied to the multiplexer 33.

  FIG. 10 exemplifies a case where the audio signal is written so that the sample per video frame can be identified when the audio signal is written in the memory 26.

  The memory controller 25 sequentially writes the audio signal selected by the selector 24 from the beginning of the address as shown in FIG. Here, it is assumed that one sample of the audio signal is data of a 32-bit format as described above. In addition, in the SDI format signal, it is possible to superimpose four groups of audio signals, that is, 16 CH audio signals in 2CH pairs per signal. Accordingly, the address width is used to store addresses c0 and c1 for storing audio signals for each of the groups g4, CH1 and CH2 for storing audio signals for every four groups, for example, 1602 samples of audio signals. Necessary addresses w0 to w10 are set. When the address widths “g1, g0, c1, c0, w10 to w0” are set in this way, the audio signal can be stored for each group and for each 2CH pair. Further, since the address width “w10 to W0” can indicate 2048 samples, the entire address width “w10 to W0” is not used up. Therefore, as shown in FIG. 10, the range of addresses “11111111100” to “11111111111” in the address width “w10 to W0” is set as an area for storing the number of samples. For example, if unique data set in advance is stored at the address “11111111100”, the number of samples per video frame is 1600. When the number of samples per video frame is 1601, unique data is stored at the address “11111111101”. When the number of samples per video frame is 1602, unique data is stored at the address “11111111110”. In this way, it is possible to determine the number of samples per video frame depending on which address the preset unique data is stored in. Although not shown, the number of samples per video frame may be written to a preset address “11111111111”, for example.

  As described above, the audio signal output processing unit 32 uses the information on the number of samples written in the memory 26, so that the number of samples per video frame of the audio signal written in the memory 26 is not constant. The problem of lip sync displacement can be prevented.

  FIG. 11 shows an operation when the information on the number of samples stored in the memory 26 is used. FIG. 11A shows the number of samples per video frame of the audio signal superimposed on the video signal DSb. FIG. 11B shows the number of samples per video frame of the audio signal DAe read from the memory 26 by the memory controller 25 and supplied to the audio signal output processing unit 32. FIG. 11C shows the number of samples per video frame of the audio signal DAf output from the audio signal output processing unit 32 and supplied to the multiplexer. Further, (D) of FIG. 11 shows the number of samples per video frame of the audio signal superimposed on the video signal DSd output from the multiplexer 33.

  Here, for example, when the audio signal superimposed on the video signal is switched to the audio signal stored in the memory 26 at time tc2, the audio signal output processing unit 32 outputs one video frame of the audio signal DAf output to the multiplexer 33. The number of samples is adjusted so that the number of samples per hit is equal to the number of samples per video frame of the audio signal superimposed on the video signal DSd supplied to the multiplexer 33. For example, when the number of samples per video frame of the video signal DSd starting at the time tc2 is “1602” and the number of samples per video frame of the audio signal DAe is “1601”, the audio signal output processing unit 32 The audio signal output processing unit 32 repeats the sampling data of the audio signal DAe to increase one sample, and supplies the audio signal DAf with the number of samples per video frame “1602” to the multiplexer 33. In the next frame, when the number of samples per video frame of the video signal DSd is “1601” and the number of samples per video frame of the audio signal DAe is “1602”, the audio signal output processing unit 32 Then, the sampling data of the audio signal DAe is decreased by one sample, and the audio signal DAf with the number of samples per video frame being “1601” is supplied to the multiplexer 33.

  If such processing is performed, even if the audio signal is switched, the number of samples per video frame of the audio signal superimposed on the video signal is equal to that before the switching. The problem can be prevented.

  In the above-described embodiment, the case where an SDI format stream or an AES format stream is input is exemplified, but the format of the video signal and the format of the audio signal on which the audio signal is superimposed are the signals of the above format. It is not limited to. The audio signal output processing unit 32 adjusts the number of samples per video frame. However, when the video signal is an interlaced signal, the number of samples may be adjusted for each field period. .

  As described above, by separately providing the memory 16 for storing the video signal DSa and the memory 26 for storing the audio signal, and using the audio signal written in the memory 26, the video signal DSa on which the audio signal is superimposed. It is possible to process a voice signal with a high degree of freedom when performing the above process. Further, since the memory 26 stores audio signals, the memory capacity may be small. Further, since the audio signal can be stored in the memory 26 in a natural form, the audio signal output processing unit for adjusting the number of samples, mixing, etc. can be easily configured.

  Further, the processing described in the above description can be executed not only by hardware but also by software, or by a combined configuration of both. When executing processing by software, a program for performing processing is installed in a memory in a computer incorporated in dedicated hardware and executed, or the program is executed on a general-purpose computer capable of executing various types of processing. It can be installed and run.

  The program can be recorded in advance on a hard disk or ROM (Read Only Memory) as a recording medium. The program can be stored (recorded) temporarily or permanently in a removable recording medium such as a magnetic disk, an optical disk, or a semiconductor memory. Furthermore, the program may be obtained from a download site via a wireless or wired network, in addition to being installed on the computer from the above-described removable recording medium.

  The present invention has been described in detail above with reference to specific embodiments. However, it is obvious that those skilled in the art can make modifications and substitutions of the embodiments without departing from the gist of the present invention. In other words, the present invention has been disclosed in the form of exemplification, and should not be interpreted in a limited manner. In order to determine the gist of the present invention, the claims should be taken into consideration.

  In the video / audio signal processing apparatus according to the present invention, the video signal on which the audio signal is superimposed is stored in the first memory, and the video signal written in the first memory is read out in synchronization with the reference signal. The audio signal superimposed on the video signal is separated, and the separated audio signal or the input audio signal is stored in the second memory. The audio signal written in the second memory is read out in synchronization with the reference signal, and is superposed in place of the audio signal superimposed on the video signal read out from the first memory. Therefore, it is suitable as a frame synchronizer or editing device.

It is a figure which shows the structure of a video / audio signal processing apparatus. It is a figure which shows the signal path | route when performing a reposition process. It is a figure for demonstrating operation | movement when performing a reposition process. It is a figure which shows a signal path | route when performing the switching process of an audio | voice signal. It is a figure for demonstrating a cross fade process. It is a figure which shows the sample number of a frame and an audio | voice signal. It is a figure which shows the structure of a sub-frame. It is a figure which shows the structure of a packet. It is a figure for demonstrating operation | movement of an audio | voice signal output process part. It is a figure for demonstrating the writing operation | movement of an audio | voice signal. It is a figure for demonstrating operation | movement when the information of the number of samples memorize | stored in memory is utilized. It is a figure which shows the structure of the conventional frame synchronizer.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 10 ... Audio / video signal processing apparatus, 11-1 to 11-m, 61 ... Serial / parallel converter (S / P), 12-1 to 12-m, 62 ... Video signal input processor , 13, 23, 24 ... selector, 15, 25, 65 ... memory controller, 16, 26, 66 ... memory, 17, 67 ... demultiplexer, 31, 71 ... video signal output Processing unit, 32, 72 ... Audio signal output processing unit, 33, 73 ... Multiplexer, 34, 74 ... Parallel / serial conversion unit (P / S), 41, 81 ... Local CPU, 60 ... Frame synchronizer

Claims (8)

A first memory for storing a video signal on which an audio signal is superimposed;
A first memory controller that reads a video signal written in the first memory in synchronization with a reference signal;
A demultiplexer for separating an audio signal superimposed on the video signal;
A second memory for storing the separated audio signal or the inputted audio signal;
A second memory controller for reading out an audio signal written in the second memory in synchronization with the reference signal;
A video / audio signal processing apparatus comprising: a multiplexer that superimposes the audio signal read from the second memory in place of the audio signal superimposed on the video signal read from the first memory. The number of samples of the audio signal read from the second memory is adjusted according to the number of samples per video frame of the audio signal superimposed on the video signal read from the first memory. The video / audio signal processing apparatus according to claim 1, further comprising an audio signal output processing unit that supplies the multiplexer. The audio signal output processing unit discriminates a sequence of the number of samples per video frame of the audio signal superimposed on the video signal read from the first memory, and reads out the sequence from the second memory. The video / audio signal processing apparatus according to claim 2, wherein the number of samples of the obtained audio signal is adjusted in correspondence with the determined sequence. The second memory controller writes information indicating the number of samples per video frame of the audio signal to the second memory;
4. The video / audio signal according to claim 3, wherein the audio signal output processing unit adjusts the number of samples of the audio signal in accordance with the determined sequence using information indicating the number of samples read from the second memory. Processing equipment. The audio signal output processing unit converts an audio signal to be supplied to the multiplexer from an audio signal read from the second memory and an audio signal separated by the demultiplexer from the other audio signal. The video / audio signal processing apparatus according to claim 2, wherein processing for switching to a signal is performed. A video signal output processing unit that performs signal processing on the video signal read from the first memory;
The first memory controller moves a video display position by changing a read start position of a video signal written in the first memory;
The video signal output processing unit replaces a blanking period for the video signal read from the first memory, and supplies the blanking period to the multiplexer.
The video / audio signal processing apparatus according to claim 1, wherein the second memory stores an audio signal separated from a video signal written to the first memory. The video / audio signal processing apparatus according to claim 1, wherein the audio signal is superimposed on a blanking period of the video signal. Storing the video signal on which the audio signal is superimposed in the first memory;
Reading the video signal written in the first memory in synchronization with a reference signal;
Separating an audio signal superimposed on the video signal;
Storing the separated audio signal or the inputted audio signal in a second memory;
Reading out the audio signal written in the second memory in synchronization with the reference signal;
A method of superimposing an audio signal read from the second memory in place of the audio signal superimposed on the video signal read from the first memory.

Images