JP6220236B2 - Sound image display device, sound image display system, and sound image display method - Google Patents

Description

  The present invention relates to a sound image display device, a sound image display system, and a sound image display method. More specifically, the present invention relates to a sound image display device that outputs drawing data for visually displaying a sound image when surround sound of a plurality of channels is reproduced, a sound image display system that displays such a sound image, and such The present invention relates to a sound image display method for a simple sound image display device.

  Conventionally, in stereo sound reproduction, an audio level meter or the like is used to manage the sound level of each channel based on each L / R signal. In addition, by using a phase meter to monitor the phase between channels, the spread of sound is monitored, and it is determined whether the sound is out of phase or the like.

  On the other hand, in audio surround playback, in addition to performing audio level management of each channel as described above and phase monitoring between channels, it is also necessary to manage the spread of sound (sound image) into a planar space. There is.

JP 2012-186525 A

  However, in order to collect a wide variety of information when sound is reproduced in surround, monitoring with various tools requires a complicated configuration of equipment. In addition, it is desirable that such information can be dynamically monitored in real time as the audio is reproduced. Therefore, if there is a measurement tool that not only aggregates such a wide variety of information but also makes it possible to grasp intuitively by visually appealing expressions, it is extremely convenient.

  Accordingly, an object of the present invention is to provide a sound image display device, a sound image display system, and a sound image display method capable of visually expressing a sound image of sound reproduced by a plurality of channels like surround reproduction.

The invention of the sound image display device according to the first aspect of achieving the above object,
Based on the audio data of two channels among a plurality of channels, a curve is obtained by calculating the ratio of the reverse phase component of the sound between the two channels and changing the curvature according to the ratio of the negative phase component. Drawing data for drawing a curve of a color corresponding to the ratio of the reverse phase component is output.

  Further, based on the sound data of the two channels, the sound level in each of the two channels is detected, and the curve in which the curvature is changed according to the sound level and the ratio of the antiphase component is drawn. Drawing data may be output.

  Also, based on the audio data of the two channels, the audio level in each of the two channels is detected, and two coordinates corresponding to the length of a line segment representing the audio level in each of the two channels are displayed at both ends. And drawing data for drawing a quadratic Bezier curve having a coordinate corresponding to the ratio of the anti-phase component as a control point may be output.

  In addition, when the ratio of the reverse phase component of the audio between the two channels is equal to or higher than a predetermined threshold value or lower than the predetermined threshold value, drawing data for drawing the curve of a predetermined color may be output.

  Further, when the ratio of the reverse phase component of the audio between the two channels is within the second range between the completely in phase and the completely reverse phase, a predetermined value is set according to the ratio of the reverse phase component. You may output the drawing data which draws the said curve of the color which changes gradually between colors.

  Further, based on the audio data of the low-frequency effect sound channel among the plurality of channels, the audio level in the low-frequency effect sound channel is detected, and at least one of the radius and the color is determined according to the audio level. The drawing data for drawing the changed circle may be further output.

The invention of the sound image display system according to the second aspect of achieving the above object,
Based on the audio data of two channels among a plurality of channels, a curve is obtained by calculating the ratio of the reverse phase component of the sound between the two channels and changing the curvature according to the ratio of the negative phase component. A control unit that outputs drawing data for drawing a color curve according to the ratio of the reversed-phase component;
A display unit for displaying a curve based on the drawing data output by the control unit;
It is characterized by providing.

The invention of the sound image display method of the sound image display device according to the third aspect of achieving the above object,
Calculating the ratio of the reverse phase component of the sound between the two channels based on the sound data of two channels of the plurality of channels;
A step of outputting drawing data for drawing a curve in which the curvature is changed according to the ratio of the reverse phase component and the color is changed according to the ratio of the reverse phase component;
It is characterized by including.

  According to the present invention, it is possible to provide a sound image display device, a sound image display system, and a sound image display method that can visually represent a sound image of sound reproduced by a plurality of channels like surround reproduction.

1 is a functional block diagram illustrating a schematic configuration of a sound image display device and system according to an embodiment of the present invention. It is a figure explaining from the audio | voice level detection in each channel to coordinate conversion. It is a figure which illustrates the result from the audio | voice level detection in each channel to coordinate conversion. It is the figure which synthesize | combined the result from the audio | voice level detection in each channel to coordinate conversion. It is a figure explaining from the correlation detection of the audio | voice in two channels to drawing data output. It is a figure which illustrates the result from the correlation detection of the audio | voice in two channels, and drawing data output. It is the figure which synthesize | combined the result from the correlation detection of the audio | voice in two channels, and drawing data output. It is a figure explaining a color data conversion process. It is a figure which illustrates the result of the drawing based on the correlation detection of the audio | voice in two channels, and color data generation. It is the figure which synthesize | combined the result of the drawing based on the correlation detection of the audio | voice in two channels, and color data generation.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 is a functional block diagram illustrating a schematic configuration of a sound image display apparatus and system according to an embodiment of the present invention.

  As shown in FIG. 1, the sound image display device 1 according to the present embodiment includes a control unit 10 and a storage unit 20. When the audio data of a plurality of channels is input, the control unit 10 outputs drawing data corresponding to the audio data. The drawing data output from the control unit 10 can be displayed on the display unit 30.

  Hereinafter, description will be made assuming that the sound image display device 1 according to the present invention includes the control unit 10 and the storage unit 20 and further includes the display unit 30 to configure the sound image display system according to the present invention. However, the present invention is not limited to such an embodiment. For example, the sound image display device 1 may have a display unit itself.

  The control part 10 can be comprised including arbitrary processing apparatuses, such as a processor or a microcomputer, for example. In the present embodiment, the control unit 10 analyzes the data or performs various arithmetic processes based on the input of various data. The control unit 10 can output the result of such arithmetic processing as drawing data to be drawn in a two-dimensional space. Moreover, the control part 10 can memorize | store various analysis results in the memory | storage part 20 as needed, and can read the various information memorize | stored in the memory | storage part 20 as needed.

  The storage unit 20 can be configured to include any memory device. The storage unit 20 also stores an algorithm used when the control unit 10 performs data analysis and various arithmetic processes as described above, and various reference tables such as a lookup table (LUT).

  The display unit 30 can be configured by an arbitrary display device such as an LCD or an organic EL display. When the drawing data is input, the display unit 30 performs display based on the drawing data on the display screen. The drawing data can be an analog or digital output of any standard according to the specifications of the display unit 30. In the present embodiment, it is desirable that the display unit 30 be capable of color display in order to perform a display that makes it easy to visually grasp the sound image. However, it is also possible to substitute for color display by displaying a single color tone such as gray scale. In the following description, it is assumed that the display unit 30 can perform color display using at least some colors.

  The sound image display device 1 according to the present embodiment outputs drawing data corresponding to the sound data based on the input of sound data of a plurality of channels. Here, the number of the plurality of channels of the audio data handled by the sound image display device 1 is not particularly limited, and may be any number of channels. In the following description, the case of 5.1ch surround generally used will be described as an example.

That is, it is assumed that 5.1 ch audio data used in the following description is reproduced using the following basic system.
First, the center speaker (C) is arranged in the center of the front front of the listening position with the listener's position (listening position (O)) as the center.
Next, the left front speaker (L) is arranged at a position 30 ° left from the front front, and the right front speaker (R) is arranged at a position 30 ° right from the front front.
Further, a left surround speaker (Ls) is arranged in the direction of 110 ° to the left of the center speaker, and a right surround speaker (Rs) is arranged in the direction of 110 ° to the right of the center speaker.
A subwoofer (LFE) dedicated to the low range is added to this.

  Next, in the sound image display apparatus 1 according to the present embodiment, a process when outputting drawing data based on input of audio data will be described.

  First, a process for outputting drawing data for drawing the level of each channel as an audio level bar by detecting the level of each channel of 5.1ch will be described.

  FIG. 2 is a diagram for explaining the audio level detection process to the coordinate conversion process in each channel.

  When the process shown in FIG. 2 starts, the control unit 10 detects the audio level of the channel in which the audio data is reproduced, based on the audio data input to the sound image display device 1 (step S11). In step S11, the control unit 10 detects the level for each channel with the bit depth of the input audio data as a numerical value of 24 bits. Therefore, in the 5.1ch audio data, the process shown in FIG. 2 is performed for each channel reproduced by each of the C, L, R, Ls, Rs, and LFE speakers.

In step S11, for example, the control unit 10 sets the audio sampling rate to 48 kHz, sets the number of audio samples (parameter N) to 16384, and performs numerical processing using the root mean square as shown in the following equation (1). Shall. In equation (1), 24-bit audio data of a certain channel is indicated as x. Therefore, x _i represents a 24-bit value in each sample with the parameter N as the total number. The control unit 10 outputs the result of detecting the level in this way as 23-bit data.

When the audio level in each channel is detected in step S11, the control unit 10 converts the level into a dB value (step S12). In step S12, the control unit 10 can convert 23-bit data into a value of 0.0 to -80.0 dB based on, for example, the following equation (2). In equation (2), the 23-bit value obtained in step S11 is indicated as x.
20 * log ₁₀ (x / 8388607) (2)

  When the sound level is converted into a dB value in step S12, the control unit 10 converts this value into coordinates corresponding to each channel in the XY plane (step S13). In the present embodiment, the correspondence between the position where the speaker for reproducing the sound in each channel is arranged and the level of the sound in the channel is displayed on the display unit 30 so that it can be easily grasped at a glance. For this reason, the XY plane displayed on the display unit 30 is associated with the arrangement of the speakers surrounding the listening position. That is, the positions where the above-described speakers C, L, R, Ls, and Rs are arranged correspond to the positions on the XY plane displayed on the display unit 30. At this time, for example, the listening position can correspond to the origin O on the XY plane displayed on the display unit 30.

  Thus, in the present embodiment, the audio level in each channel is represented by the length of each axis indicating the direction in which the speaker is arranged in the display unit 30. For this reason, in step S13, the control unit 10 determines that the dB value calculated in step S12 represents the length from the origin of the line segment indicating the direction in which the corresponding speaker is arranged in the display unit 30. Calculate the minute end coordinates.

  FIG. 3 is a diagram illustrating the results from the sound level detection process to the coordinate conversion process in each channel.

As shown in FIG. 3A, in step S13, the control unit 10 obtains the coordinates C so that the level (dB value) of the sound reproduced by the center speaker corresponds to the length of the line segment OC.
Further, as shown in FIG. 3B, the control unit 10 obtains the coordinate L so that the level of the sound reproduced by the left front speaker corresponds to the length of the line segment OL.
Further, as shown in FIG. 3C, the control unit 10 obtains the coordinates R so that the level of the sound reproduced by the right front speaker corresponds to the length of the line segment OR.
Further, as illustrated in FIG. 3D, the control unit 10 obtains the coordinate Ls so that the level of the sound reproduced by the left surround speaker corresponds to the length of the line segment OLs.
Further, as shown in FIG. 3E, the control unit 10 obtains the coordinates Rs so that the level of the sound reproduced by the right surround speaker corresponds to the length of the line segment ORs.

  In addition, since the subwoofer dedicated to the low sound range is omnidirectional, in the present embodiment, the sound level reproduced by the subwoofer is represented by the size of a circle around the listening position on the display unit 30. For this reason, as shown in FIG. 3F, in step S13, the control unit 10 obtains the coordinate LFE so that the level of the sound reproduced by the subwoofer corresponds to the circular radius.

  Thus, when coordinate data corresponding to each channel (in the case of LFE, circle radius data) is obtained in step S13, the control unit 10 can generate and output drawing data representing the level of each channel. it can. Based on the drawing data output in this way, the display unit 30 can display the level of each channel as an audio level meter corresponding to the direction of the speaker that reproduces the sound of each channel.

  FIG. 4 is a diagram in which the results from the audio level detection process to the coordinate conversion process in each channel are synthesized. In FIG. 4, the results of performing the processing shown in FIG. 2 for each channel reproduced by each of the above-described C, L, R, Ls, Rs, and LFE speakers are combined and shown. Note that the circular graphic representing the LFE level may be displayed only on the circumference of the circle, or may be displayed as the entire disk including the inside of the circumference. However, when displaying the whole disk, in order to prevent the level display of other channels from being difficult to see, for example, it is preferable to display the other display in a translucent color. By displaying the sound level in each channel in this way, it is possible to easily grasp at a glance the correspondence between the position where the speaker for reproducing the sound in each channel is arranged and the sound level in the channel. Can do. By displaying the audio level in each channel in this way, for example, the correspondence between the peak level of each channel and the position of the speaker from which the audio of that channel is output can be easily grasped visually.

  When the display as shown in FIG. 4 is performed, if the sound level in a certain channel is equal to or higher than a predetermined threshold, the drawing data to be displayed is changed to a color that suggests a warning (for example, red). May be. In this case, for example, it is possible to change and display the color of a part or all of the circular figure indicating the level of the line segment and / or LFE shown in FIG.

  As described above, in the present embodiment, the control unit 10 detects the sound level in the at least one channel based on at least one sound data among the plurality of channels, and the length according to the sound level. Draws data that draws a line segment that has been changed.

  Further, as described above, the control unit 10 detects the sound level in the low-frequency sound effect channel based on the sound data of the low-frequency sound effect channel among the plurality of channels, and sets the sound level. The drawing data for drawing a circle with the radius changed accordingly is output. In this case, when the sound level in the low-frequency sound effect channel is equal to or higher than a predetermined threshold, the control unit 10 outputs drawing data for drawing the circle having a color changed according to the sound level. May be.

  Next, a process when outputting drawing data for drawing a curve based on the correlation of sound in two channels by detecting the level of two channels in 5.1ch will be described.

  FIG. 5 is a diagram for explaining the process from the voice correlation detection process to the drawing data output process in two channels.

  In the following description, the audio data of two channels for performing the audio correlation detection process to the drawing data output process will be referred to as “first audio data” and “second audio data” for convenience. For example, a notation such as “LR” indicates that in the processing shown in FIG. 5, the first audio data is processed as channel L audio data and the second audio data is processed as channel R audio data. means. In this way, when the processing shown in FIG. 5 is performed on LR, drawing data for drawing a curve corresponding to the ratio of the reverse phase component of the sound between the two channels L and R is output. Can do.

  When the processing shown in FIG. 5 starts, the control unit 10 calculates the correlation coefficient of the audio data of two channels among the plurality of channels based on the audio data input to the sound image display device 1 (step S21). . In step S21, as in step S11, the bit depth of the input audio data is set to a 24-bit numerical value. In step S21, the correlation coefficient of the audio data of two channels among the plurality of channels is calculated. Therefore, in the 5.1ch audio data, the processing shown in FIG. 5 is performed for two channels such as LR, CL, CR, L-Ls, R-Rs, and Ls-Rs. I do. Here, it is preferable that the two channels of the plurality of channels are two adjacent channels at the position where the speakers are arranged. For example, when LR is calculated by ignoring C. As described above, there are cases where two adjacent channels are not provided.

In step S <b> 21, for example, the control unit 10 sets the parameter n to 2048 and performs mathematical formula processing using a correlation calculation formula as shown in the following formula (3). In the expression (3), the first audio data (24 bits) is indicated as x, and the second audio data (24 bits) is indicated as y. Therefore, x _i and y _i each represent a 24-bit value in each sample with the parameter n as the total number. The control part 10 shall output the result calculated in this way as a correlation coefficient in the range of −1 to +1.

In the present embodiment, the curve drawn by the drawing data output by the control unit 10 is a second-order Bezier curve. Therefore, when the correlation coefficients in the two channels are calculated in step S21, the control unit 10 uses the following equation (4) to calculate the curve drawn based on the drawing data to be output from the correlation coefficient. A bending coefficient is calculated (step S22).
128 × (correlation coefficient + 1) × (correction value) (4)

  Here, the above (correction value) is, for example, 0.83 in the conversion of LR, LC, RC, 0.90 in the conversion of L-Ls, R-Rs, and Ls In the conversion of -Rs, 0.68 is preferable. By multiplying this correction value, the drawn Bezier curve becomes an approximate circle when the correlation coefficient calculated in step S21 is zero. Further, since it is necessary to convert to an integer when mounted on hardware, it is preferable to multiply the value by 128 to 8 bits as shown in Equation (4). In this way, the control unit 10 outputs the result calculated in step S22 as 8-bit data.

  When the bending coefficient is calculated in step S22, the control unit 10 generates a Bezier curve based on the bending coefficient and the coordinates of the first and second audio data (step S23).

  In step S23, the control unit 10 generates a quadratic Bezier curve based on the coordinates of the three points. Here, coordinate data calculated from the first audio data and the second audio data is used as the coordinates of the two points. As the coordinate data of these two points, the respective coordinate data obtained from the first sound data and the second sound data can be used in accordance with the processing shown in FIG.

  In the present embodiment, a secondary Bezier curve is generated using one control point based on the above-described bending curve in addition to the coordinate data of these two points. When this control point is Q (x3, y3), for example, a quadratic Bezier curve can be generated as follows.

  On the XY coordinate plane, an origin O (0, 0), coordinates A (x1, y1) obtained from the first audio data, and coordinates B (x2, y2) obtained from the second audio data are defined. Further, the midpoint between A (x1, y1) and B (x2, y2) is M (Xm, Ym), and the correlation coefficient (−1 to +1) is k. Here, when k = 0, the Bezier curve passing through points A and B with Q as the control point is an approximate circle. When −1 <k <0, the curvature of the Bezier curve increases, and when 0 <k <+1. The curvature of the Bezier curve becomes small, and when k = + 1, the Bezier curve is a line segment connecting AB.

The coordinates (x3, y3) of the control point Q can be obtained as in the following equation (5) with M (Xm, Ym) as a reference. However, (X *, Y *) means a direction vector perpendicular to the line segment AB.
(X3, y3) = (Xm, Ym) + k (X *, Y *) (5)

  In equation (5), when k = 0, (X *, Y *) is uniquely determined by A and B. Note that the accuracy of the approximate circle drawn when k = 0 changes depending on the angle AOB. In order to maintain this consistency, the correlation coefficient is multiplied by the correction value in the above-described equation (4).

In step S23, when the coordinates of the three points A (x1, y1), B (x2, y2), and Q (x3, y3) are calculated as described above, the control unit 10 determines these three points. A quadratic Bezier curve based on In this case, a curve connecting points is generated based on the definition of the Bezier curve. Specifically, a Bezier curve is generated by using the following formulas (6) and (7) as values t = 0 to 1 (t is 1/1024 step).
x = x (t) = t ^ 2 * x2 + 2 * t * (1-t) * x3 + (1-t) ^ 2 * x1 (6)
y = y (t) = t ^ 2 * y2 + 2 * t * (1-t) * y3 + (1-t) ^ 2 * x1 (7)

  When the Bezier curve is generated in step 23, the control unit 10 generates drawing data for displaying the generated curve on the XY plane (step S24) and outputs it. Here, in order to display the generated curve in color, drawing data is generated and output together with the color data. The coloring of the curve will be described later.

  FIG. 6 is a diagram exemplifying the results from the correlation detection of voice in two channels to the output of drawing data.

As shown in FIG. 6A, in step S23, the control unit 10 generates a Bezier curve LR for the two LR channels.
Further, as illustrated in FIG. 6B, the control unit 10 generates a Bezier curve CL for the two channels of CL.
Further, as illustrated in FIG. 6C, the control unit 10 generates a Bezier curve CR for the two channels of CR.
Further, as illustrated in FIG. 6D, the control unit 10 generates Bezier curves LLs for the two channels L-Ls.
Further, as illustrated in FIG. 6E, the control unit 10 generates a Bezier curve RRs for two channels of R-Rs.
Further, as illustrated in FIG. 6F, the control unit 10 generates a Bezier curve LsRs for two channels of Ls-Rs.

  FIG. 7 is a diagram in which results from audio correlation detection to drawing data output in two channels are combined. In FIG. 7, as shown in FIG. 6, all the Bezier curves drawn for each of two channels LR, CL, CR, L-Ls, R-Rs, and Ls-Rs are synthesized. It is shown.

  In FIG. 7, a curve α is shown by synthesizing all Bezier curves drawn for two channels of CL, CR, L-Ls, R-Rs, and Ls-Rs. The curve β is a combination of all Bezier curves drawn using LR instead of CL and CR in the curve α. For example, when displaying a sound image including C, such as when playing music, it is preferable to display a Bezier curve such as curve α in order to mainly monitor CL and CR. On the other hand, in the case where C mainly includes only words, such as voice played in a drama, for example, a curve is used so that LR can be mainly monitored without considering C. It is preferable to display a Bezier curve such as β.

  As shown in FIG. 7, since it is feared that the display of the curve α and the curve β at the same time is confusing, the control unit 10 displays drawing data that can selectively display either the curve α or the curve β. It is preferable to output. In addition, when the curve α and the curve β are displayed simultaneously, the control unit 10 outputs drawing data so that they can be distinguished from each other by changing the thickness, color, line type, display mode, and the like of the curves. Is preferred.

  As described above, in the present embodiment, the control unit 10 calculates the ratio of the reverse phase component of the sound between the two channels based on the sound data of the two channels among the plurality of channels. And the control part 10 outputs the drawing data which draws the curve which changed the curvature according to the ratio of the said anti | reverse | negative phase component. Further, as described above, the control unit 10 detects the sound level in each of the two channels based on the sound data of the two channels. And it is suitable for the control part 10 to output the drawing data which draws the said curve which changed the curvature according to the said audio | voice level and the ratio of the said reverse phase component. In this case, the control unit 10 uses two coordinates corresponding to the length of the line segment representing the sound level in each of the two channels as points at both ends, and uses a coordinate corresponding to the ratio of the reverse phase component as a control point. Drawing data for drawing a quadratic Bezier curve is output.

  Next, coloring of a curve generated by the sound image display device 1 according to the present embodiment will be described.

  FIG. 8 is a diagram for explaining color data conversion processing performed by the sound image display device 1 according to the present embodiment. In this embodiment, it is preferable that the Bezier curve of each unit as shown in FIG. 6 is output from the control unit 10 with color data corresponding to each correlation coefficient. Hereinafter, a process for adding color data to a Bezier curve will be described.

  As described with reference to FIG. 5, the control unit 10 calculates the correlation coefficient of audio in two channels in step 21. In the present embodiment, the control unit 10 generates color data based on the correlation coefficient calculated in this way (step S25), and the drawing data obtained by coloring the curve generated in step S23 in step S24. Generate and output.

  As described in step 21 in FIG. 5, the control unit 10 calculates the correlation coefficient in the range of −1 to +1. Therefore, the control unit 10 generates color data as shown in FIG. 8A based on the correlation coefficient (step S25). As described above, when color data is generated based on the correlation coefficient in step S25, the curve generated in step S23 is colored based on the color data.

  As shown in the range <A> shown in FIG. 8A, when the correlation coefficient is in the vicinity of −1, the channels are nearly in reverse phase. In this case, the control unit 10 generates, for example, red color data that suggests a warning. In actual audio reproduction, it is difficult to assume that the channels are completely out of phase, so it is preferable to make the curve red in a range where the correlation coefficient is very close to -1.

  Also, as shown in the range <D> shown in FIG. 8A, when the correlation coefficient is near +1, the channels are nearly in phase. In this case, it means a state where there is no spread of sound, but depending on the speech expression, such a sound reproduction state may be intentionally desired. For this reason, in the range <D> where the correlation coefficient is near +1, the control unit 10 generates, for example, blue color data to call attention. In this case, the range <D> is preferably slightly wider than the above-described range <A>.

  Further, as shown in the range <C> shown in FIG. 8A, when the correlation coefficient is from near 0 to near +1, the ratio of the antiphase component between the channels becomes appropriate. Accordingly, in this case, the control unit 10 generates color data of standard colors that are not particularly distinguished from other displays. Here, the standard color is preferably a color that is not particularly emphasized compared to other displays, such as white on a black screen.

  Further, as in the range <B> shown in FIG. 8A, when the correlation coefficient is from about −1 to about 0, the ratio of the reverse phase components between the channels is relatively high. Therefore, in this case, the control unit 10 generates color data of a color that can be distinguished from other displays for alerting.

  In the range <B>, in order to make it easy to visually recognize the ratio of the antiphase component between the channels and the change in the ratio, the control unit 10 calculates the correlation coefficient from around 0 to around −1. Color data of different colors is generated according to the value. That is, in the present embodiment, the control unit 10 reduces the blue and red components as the correlation coefficient changes from near 0 to near −1, and changes the color from yellowish to gradually green. Output data. Therefore, according to the sound image display apparatus 1 according to the present embodiment, it is possible to easily visually recognize the ratio of the antiphase component between channels and the change in the ratio. FIG. 8A schematically shows that the color data output gradually changes from yellowish color to green as the correlation coefficient changes from around 0 to around −1.

  Further, when the color data to be output is gradually changed as the correlation coefficient changes in this way, if the color change is a linear change, it is not suitable for human senses. Therefore, in the present embodiment, the control unit 10 adjusts the change in the color by changing the gamma curve, as will be described later.

In order to calculate the color data to be generated from the correlation coefficient as described above, an LUT defined as follows can be used. Hereinafter, an LUT for calculating color data to be generated from the correlation coefficient will be described. First, it is defined as follows.
(1) When correlation coefficient = −1.0, R (red) is fixed at 100%, G (green) is 0%, and B (blue) is 0%.
(2) When correlation coefficient = −0.9 to 0, G (green) is fixed at 100%.
(3) When correlation coefficient = 0 to 1.0, B (blue) is fixed at 100%.

Next, the following values are prepared. That is,
(1) The threshold Gp is set from where the correlation coefficient = 0 to 1.0 (0.1 step) from which the green color is reduced (in this example, the setting value = 0.9).
(2) The threshold value Rp is set from where the correlation coefficient = 0 to 1.0 (0.1 step) to reduce the red color (in this example, set value = 0.9).
(3) The set value Gγp sets the green gamma curve (0.05 step) (set value = 3.00 in this example).
(4) As the set value Rγp, the red gamma curve (0.05 step) is set (in this example, set value = 3.00).
(5) The threshold value Bn is set from where correlation coefficient = 0 to −0.9 (0.1 step) from which blue is to be reduced (in this example, set value = 0.0).
(6) The threshold value Rn is set from where correlation coefficient = 0 to −0.9 (0.1 step) from which red is reduced. (In this example, set value = 0.0).
(7) The set value Bγn sets the blue gamma curve (0.05 step) (set value = 0.10 in this example).
(8) As the set value Rγn, the red gamma curve (0.05 step) is set (in this example, the set value = 1.00).

Next, an example of a method for calculating green (G) based on the above-described rules will be described.
(1) When the correlation coefficient is −1.0 to −0.9,
G = 255 × ((1 + correlation coefficient) / (1−0.9)).
(2) When correlation coefficient = −0.9 to threshold Gp (0.9 in this example),
Let G = 255.
(3) When correlation coefficient = threshold Gp (0.9 in this example) to 1.0 and threshold Gp (0.9 in this example) <1.0,
G = 255 × (the set value Gγp (3.00 in this example) is an exponent and (1−correlation coefficient) / (1−threshold Gp (0.9 in this example)) is a power of the base).

Next, an example of a method for calculating blue (B) based on the above-described rules will be described.
(1) When the correlation coefficient = −1.0,
Let B = 0.
(2) When correlation coefficient = −0.9 to threshold value Bn (0.0 in this example) and threshold value Bn (0.0 in this example)> − 0.9,
B = 255 × (the set value Bγn (0.10 in this example) is an exponent and (1 + correlation coefficient) / (1 + threshold Bn (0.0 in this example)) is a power of the base).
(3) When correlation coefficient = threshold value Bn (0.0 in this example) to 1.0,
Let B = 255.

Next, an example of a method for calculating red (R) based on the above-described rules will be described.
(1) When the correlation coefficient is −1.0 to −0.9,
R = 255-255 × ((1 + correlation coefficient) / (1-0.9)).
(2) When correlation coefficient = −0.9 to threshold value Rn (0.0 in this example) and threshold value Rn (0.0 in this example)> − 0.9,
R = 255 × (the set value Rγn (1.00 in this example) is an exponent and (1 + correlation coefficient) / (1 + threshold Rn (0.0 in this example)) is a power of the base).
(3) When correlation coefficient = threshold value Rn (0.0 in this example) to threshold value Rp (0.9 in this example),
Let R = 255.
(4) When correlation coefficient = threshold value Rn (0.0 in this example) to 1.0 and threshold value Rn (0.0 in this example) <1.0,
R = 255 × (set value Rγp (3.00 in this example) is an exponent, and (1−correlation coefficient) / (1−threshold Rp (0.9 in this example)) is a power of the base).

  By following the LUT as described above, the control unit 10 can generate and output, for example, RGB 24-bit color data in step S25 of FIG.

  FIG. 8B is a diagram showing the correspondence between the correlation coefficient and the green (G) component of the color data output based on the correlation coefficient in accordance with the LUT as described above. FIG. 8C is a diagram showing the correspondence between the correlation coefficient and the blue (B) component of the color data output based on the correlation coefficient in accordance with the LUT as described above. FIG. 8D is a diagram showing the correspondence between the correlation coefficient and the red (R) component of the color data output based on the correlation coefficient in accordance with the LUT as described above.

  FIG. 9 is a diagram exemplifying a drawing result based on audio correlation detection and color data generation in two channels.

  As described in FIG. 5, the control unit 10 generates Bezier curves for two channels in step S23 (see FIG. 6). Further, as described in FIG. 5, the control unit 10 generates Bezier curve color data for the two channels in step S25 (see FIG. 8). As described above, as a result of the processing shown in step S23 and step S25 in FIG. 5, the control unit 10 generates the drawing data of the curvature and the color Bezier curve as shown in FIG. 9 (step S24) and outputs it.

  FIG. 9 shows some examples in which the curvature and color of the Bezier curve CR generated for the two channels CR are changed as shown in FIG.

  FIG. 9A shows the curvature and color of the Bezier curve CR generated when the correlation coefficient shown in FIG. 8A is in the range <A>. As shown in FIG. 9A, when the correlation coefficient is in the range <A>, a Bezier curve having a color different from the normal case is generated, and a warning can be suggested to the user.

  FIG. 9B shows the curvature and color of the Bezier curve CR generated when the correlation coefficient shown in FIG. 8A is in the range <B>. As shown in FIG. 9B, when the correlation coefficient is in the range <B>, a Bezier curve having a curvature that changes in accordance with the sound spread image corresponding to the ratio of the antiphase components of the two channels is generated. The When the correlation coefficient is in the range <B>, a Bezier curve of a color that gradually changes according to the correlation coefficient is generated, and the user can be alerted.

  FIG. 9C shows the curvature and color of the Bezier curve CR generated when the correlation coefficient shown in FIG. 8A is in the range <C>. As shown in FIG. 9C, even when the correlation coefficient is in the range <C>, a Bezier curve whose curvature changes in accordance with the sound spread image corresponding to the ratio of the antiphase components of the two channels is obtained. Generated. In addition, when the correlation coefficient is in the range <C>, the ratio of the antiphase component is appropriate, so that a Bezier curve of a color that is not particularly conspicuous compared to other displays is generated, and the user dares to You can avoid caution.

  FIG. 9D shows the curvature and color of the Bezier curve CR generated when the correlation coefficient shown in FIG. 8A is in the range <D>. As shown in FIG. 9D, since the two channels are almost completely in phase when the correlation coefficient is in the range <D>, the curvature (close to a straight line) and the color are different from those in the normal case. The Bezier curve is generated, and the user can be alerted.

  As described above, in the present embodiment, the control unit 10 calculates the ratio of the reverse phase component of the sound between the two channels based on the sound data of the two channels among the plurality of channels. And the control part 10 outputs the drawing data which draws the curve of the color according to the ratio of the said anti | reverse | negative phase component.

  In addition, when the ratio of the reverse phase component of the sound between the two channels is equal to or higher than the first threshold, the control unit 10 draws drawing data for drawing the curve of a predetermined color such as red that indicates a warning. Output. In addition, when the ratio of the reverse phase component of the audio between the two channels is equal to or less than the second threshold, the control unit 10 outputs drawing data for drawing the curve of a predetermined color such as blue. In addition, when the ratio of the reverse phase component of the audio between the two channels is within the first range between the completely in phase and the completely out of phase, the control unit 10 determines a predetermined color such as white, for example. The drawing data for drawing the curve is output. Further, the control unit 10 responds to the ratio of the anti-phase component when the ratio of the anti-phase component of the sound between the two channels is within the second range between the completely in-phase and the completely anti-phase. For example, drawing data for drawing the curve of a color that gradually changes between predetermined colors such as yellow to green is output.

  In FIG. 9, the Bezier curve CR in which the curvature and color are set is generated only for the two channels of CR, but in this embodiment, the control unit 10 performs the same processing on the other two channels. It carries out also about LR, CL, L-Ls, R-Rs, and Ls-Rs. Thus, when the Bezier curve in which the curvature and color are set for each of the two channels described above is generated, drawing data for drawing as shown in FIG. 10 is output.

  FIG. 10 is a diagram in which the drawing results based on the correlation detection and color data generation of the audio in the two channels are combined. That is, FIG. 10 is a diagram illustrating a state in which all the results illustrated in FIGS. 4, 6, and 9 are synthesized by performing processing by the sound image display device 1 according to the present embodiment. The sound image display apparatus 1 according to the present embodiment may display the result of only a part of the above-described processing on the display unit 30, but FIG. 10 illustrates that various information relating to sound such as a sound image can be grasped at a time. As shown, it is preferred to display all the results of the processing described above simultaneously.

  In FIG. 10, O represents the listening position. C represents the position of the center speaker, L represents the position of the left front speaker, R represents the position of the right front speaker, Ls represents the position of the left surround speaker, and Rs represents the position of the right surround speaker. . These positions are preferably defined in advance by defining ideal positions.

Hereinafter, in FIG. 10, the meanings represented by the respective symbols are described.
Lv (C): level of sound reproduced from the center speaker C Lv (L): level of sound reproduced from the left front speaker L Lv (R): level of sound reproduced from the right front speaker R Lv (Ls ): Level of sound reproduced from the left surround speaker Ls Lv (Rs): level of sound reproduced from the right surround speaker Rs Lv (LFE): level of sound reproduced from the subwoofer B (LR): left front speaker Bezier curve based on the sound reproduced from L and the sound reproduced from the right front speaker R (the sound reproduced from the center speaker C is ignored)
B (LC): Bezier curve based on the sound reproduced from the left front speaker L and the sound reproduced from the center speaker C B (CR): The sound reproduced from the center speaker C and the sound reproduced from the right front speaker R Bezier curve B (LLs): Bezier curve based on audio reproduced from the left front speaker L and audio reproduced from the left surround speaker Ls B (RRs): Audio reproduced from the right front speaker R and right surround speaker Bezier curve based on sound reproduced from Rs B (LsRs): Bezier curve based on sound reproduced from left surround speaker Ls and sound reproduced from right surround speaker Rs

  Since the LFE shown in FIG. 10 is non-directional as described with reference to FIG. 3F, the LFE is displayed as a circle having a radius increased according to the level around the point 0, for example, translucent. In this case, it is possible to alert the observer by gradually changing the color of the LFE display from a level that seems to be too loud as a sound in a normal program. Further, when the color data to be output is gradually changed as the LFE level changes in this way, if the color change is a linear change, it is not suitable for human senses. Therefore, in the present embodiment, as described above, it is preferable that the control unit 10 adjusts the color change by changing the gamma curve.

  In this manner, in the present embodiment, the LFE signal may be monitored simultaneously. When the LFE is played back at a high volume, the woofer may be distorted and cause a problem, so that the LFE signal is usually operated at a low level in many cases. Therefore, when the LFE level becomes too high, it is preferable to change the color to a red color to prompt attention.

  According to the present invention, a sound image can be expressed using a Bezier curve on a two-dimensional space simulating the arrangement of a plurality of speakers. In the present invention, the size of the figure by the Bezier curve represents the size of the entire voice, and the shape of the figure by the Bezier curve represents the sense of localization of the sound image.

  As described above, according to the present invention, devices having a complicated configuration are not necessary when collecting a wide variety of information when surround sound is reproduced. Further, according to the present invention, such information can be dynamically monitored in real time as the audio is reproduced. Furthermore, according to the present invention, not only such a wide variety of information can be aggregated, but also intuitively grasped on a single screen using expressions that appeal to the eye, which is extremely convenient.

  Hereinafter, the effects of the present invention will be further described.

  According to the present invention, since the audio level bar of each channel is arranged from the listening position toward each speaker arrangement, the level management of each channel can be performed by the swing of the level bar. Further, according to the present invention, for example, the ratio of the anti-phase component of audio between two adjacent channels is expressed by the curvature and color change of the Bezier curve, and simultaneously with the confirmation of the level and sound image of each channel, the phase Monitoring can also be performed. As an acoustic effect, a reverse phase component is required to give the sound a certain degree of spread. According to the present invention, if a surround signal with an appropriate balance is input, the Bezier curve is displayed as a substantially circular circle as a whole, so that the appropriate balance of the entire surround signal can be grasped at a glance. Can do.

  Further, according to the present invention, the curvature of the Bezier curve increases and the color changes as the anti-phase components increase. In this case, the color changes only gradually until the anti-phase component is not at an appropriate level, but when the anti-phase component becomes an abnormal level, it changes to a red color, so it is easy to recognize that it is a warning notification. Can do. Furthermore, when level adjustment is performed using a reference signal such as OSC in the system check, the anti-phase component should be zero, so the Bezier curve is straight and blue. Therefore, it can be easily determined whether the system has been correctly constructed.

  Although the present invention has been described based on the drawings and examples, it should be noted that those skilled in the art can easily make various changes and modifications based on the present disclosure. Therefore, it should be noted that these variations and modifications are included in the scope of the present invention. For example, the functions included in each functional unit, each means, each step, etc. can be rearranged so that there is no logical contradiction, and a plurality of functional units, steps, etc. are combined or divided into one. It is possible. Further, the above-described embodiments of the present invention are not limited to being performed faithfully to the respective embodiments described above, and can be implemented by appropriately combining the features.

  The embodiment described above has been described as the invention of the sound image display device 1 and the sound image display system. In this case, the sound image display system according to the present invention includes a control unit 10 that performs various controls as described above, and a display unit 20 that displays a curve based on the drawing data output by the control unit 10. And

  The present invention can also be a sound image display method in the sound image display device 1 or the sound image display system. In this case, in the sound image display method according to the present invention, the step of calculating the ratio of the anti-phase component of the audio between the two channels based on the audio data of the two channels among the plurality of channels, Outputting drawing data for drawing a curve having a curvature changed in accordance with the ratio and having a color curve changed in accordance with the ratio of the negative phase component.

DESCRIPTION OF SYMBOLS 1 Sound image display apparatus 10 Control part 20 Memory | storage part 30 Display part

Claims (8)

  Based on the audio data of two channels among a plurality of channels, a curve is obtained by calculating the ratio of the reverse phase component of the sound between the two channels and changing the curvature according to the ratio of the negative phase component. A sound image display device that outputs drawing data for drawing a color curve according to the ratio of the anti-phase component.   Drawing data for detecting the sound level in each of the two channels based on the sound data of the two channels and drawing the curve with the curvature changed according to the sound level and the ratio of the anti-phase component The sound image display device according to claim 1, wherein   Based on the audio data of the two channels, the audio level in each of the two channels is detected, and the two coordinates corresponding to the length of the line segment representing the audio level in each of the two channels are used as points at both ends. The sound image display device according to claim 2, wherein drawing data for drawing a quadratic Bezier curve having a coordinate corresponding to the ratio of the anti-phase component as a control point is output.   The drawing data for drawing the curve of a predetermined color is output when the ratio of the reverse phase component of the audio between the two channels is equal to or higher than a predetermined threshold or lower than a predetermined threshold. The sound image display device according to claim 1.   When the ratio of the reverse phase component of the audio between the two channels is in the second range between the completely in phase and the completely out of phase, a predetermined color is selected according to the ratio of the reverse phase component. The sound image display device according to claim 1, wherein the sound data display device outputs drawing data for drawing the curve of a color that gradually changes between the two.   Based on the sound data of the low-frequency sound effect channel among the plurality of channels, the sound level in the low-frequency sound effect channel is detected, and at least one of the radius and the color is changed according to the sound level. The sound image display device according to claim 1, further outputting drawing data for drawing a round shape. Based on the audio data of two channels among a plurality of channels, a curve is obtained by calculating the ratio of the reverse phase component of the sound between the two channels and changing the curvature according to the ratio of the negative phase component. A control unit that outputs drawing data for drawing a color curve according to the ratio of the reversed-phase component;
A display unit for displaying a curve based on the drawing data output by the control unit;
A sound image display system comprising: Calculating the ratio of the reverse phase component of the sound between the two channels based on the sound data of two channels of the plurality of channels;
A step of outputting drawing data for drawing a curve in which the curvature is changed according to the ratio of the reverse phase component and the color is changed according to the ratio of the reverse phase component;
A sound image display method for a sound image display device.

Images