Disclosure of Invention
The invention aims to provide an intelligent piano temperament identification and automatic adjustment system.
The invention enables the preliminarily developed intelligent piano automatic playing system to have the capability of self-adjusting and optimizing the temperament effect by dynamically and quantitatively comparing the collected intelligent piano playing audio with the played standard audio, greatly reduces the temperament calibration work of a piano automatic playing system developer in the system development process, and also replaces a manual adjustment mode mainly based on empirical judgment and estimation by an accurate quantitative and dynamic adjustment mode.
The technical solution for realizing the purpose of the invention is as follows:
the utility model provides an intelligence piano temperament discernment and automatic adjustment system, includes removal end application module, main control module, branch control module, electromagnetism spiral drive module, power module and the temperament sensor module, WIFI module, the CAN bus module of being connected respectively with this main control module, and all parts are integrated integrative, connect to set up in the piano central point and put, wherein:
the mobile terminal application template issues an instruction to the main control module through a WIFI protocol, and receives, stores and displays the running condition information of the intelligent piano system in real time.
And the main control template waits for receiving the instruction of the mobile terminal through a WIFI protocol, analyzes the format of the MIDI music file, and is responsible for sending the voice recognition feedback information to the subordinate sub-control modules.
The sub-control modules receive the instructions of the main control panel, cooperatively control 88 keys among the modules, and have the capabilities of real-time parallel processing, dynamic force output, storage and play of evaluation feedback information and indirect processing of temperament feedback information;
the electromagnetic spiral module is connected with the sub-control module pins and drives the magnets at the bottoms of the piano keys after being electrified to impact the keys to enable the piano to produce sound.
The power supply module comprises a 36V rechargeable battery arranged inside the piano and supplies power to the whole piano control system.
The melody sensor module captures sound data in the environment, samples, quantifies and analyzes the data, and feeds back the obtained analysis data to the main control module.
And the WIFI module is used for wireless communication between the mobile terminal and the main control module and forwarding the intelligent piano condition information.
And the CAN bus module is used for the area communication between the main control module and the sub-control module and is responsible for information transmission and interaction between the intelligent piano system boards.
Furthermore, the main control module adopts an i.MX287ARM industrial control board, takes an ARM9 inner core as a processor core, has the highest main frequency of 454MHz, is provided with a DDR2 and a NAND Flash memory, and has very low power consumption. The main control module is connected with the mobile terminal application module through WIFI, connected with the sub-control module through the CAN bus module and connected with the computer through the serial port output module.
Furthermore, the branch control module adopts an STM32 microprocessor which is low in power consumption, strong in performance and easy to develop, is connected with the main control module through the CAN bus module, is connected with the electromagnetic spiral module through a lead wire, and has real-time parallel processing capacity, dynamic force output capacity, storage and play evaluation feedback information capacity and indirect processing temperament feedback information capacity, and the branch control module is concretely as follows:
(1) real-time parallel processing capability: each sub-control module is responsible for controlling twelve keys, and a thread is created for the generation of the PWM waveform of each key, so that different keys cannot be blocked mutually. The CAN bus module of the microprocessor has an automatic awakening function for receiving the synchronous interval characters, and the processor CAN awaken a corresponding process when receiving the data frame.
(2) Dynamic force output capacity: data information of the changing force is also integrated into communication protocols of the main control module and the sub-control modules, and the sub-control modules can accurately control the force to change step by matching with the timer modules after receiving specific instructions.
(3) And (3) storing and playing evaluation feedback information capacity: the microprocessor is provided with a Flash memory chip of a permanent memory medium, can quickly and permanently store the data received immediately and store the data in combination with historical data.
(4) And (3) indirectly processing the rhythm feedback information: and receiving the range feedback information of the temperament sensor module, adjusting the playing pitch intensity of the corresponding musical notes by changing the corresponding PWM waveforms and frequency according to the requirements of the protocol data frames, and adjusting the playing pitch length of the corresponding musical notes by adjusting the pressing and playing time of the corresponding musical notes.
Further, the temperament sensor module comprises 2 parts, specifically:
(1) the digital microphone is used as a temperament perception module for receiving the musical instrument sound, and the temperament perception module is connected with the temperament identification module and is responsible for converting the temperament collected in the environment into discrete and processable audio data and then delivering the discrete and processable audio data to the temperament identification module for processing.
(2) The high-performance computer is used as a temperament identification module for calculating the piano sound, the temperament identification module is connected with the temperament perception module and is connected with the main control module through the CAN bus module, and is responsible for receiving the discrete audio data transmitted by the temperament perception module, and performing denoising and sound processing, time domain analysis, frequency domain analysis, characteristic processing and temperament identification on the discrete audio data, so that the audio data CAN better embody the essence of sound, and then the audio data of the pitch, the duration and the intensity of the collected piano sound is obtained. Finally, the audio feedback modification suggestion is formed according to the result of dynamic comparison by dynamically comparing the audio feedback modification suggestion with the ideal audio data pre-judged in the MIDI music file and is processed by the main control module.
An intelligent piano temperament identification and automatic adjustment method comprises the following steps:
step 1, firstly, initializing a mobile terminal application module, a main control module, a sub-control module, an electromagnetic spiral driving module, a power supply module, a melody sensor module and a CAN bus module.
Step 2, creating a standard piano MIDI file to be played through an audio playing device, and recording an audio database y obtained by an audio sensor module1Carrying out denoising and sound-changing processing, time domain analysis, frequency domain analysis, characteristic processing and temperament identification on the audio database to obtain a temperament evaluation standard database y2As audio evaluation and feedback comparison standard.
And 3, starting the WIFI module, sending an automatic temperament adjustment starting instruction to the main control module by the mobile terminal application, and displaying the system running state sent back by the main control module in real time through the user interface.
And 4, the MIDI music file is analyzed by the main control module and then broadcast to the CAN bus module, the sub-control module receives the data stream and generates PWM waveforms in real time, and the sub-control module controls the electromagnetic solenoid module to drive the keys to sound.
And 5, acquiring discrete audio data by the tone law identification module, and performing denoising and sound-changing processing, time domain analysis, frequency domain analysis, characteristic processing and tone law identification on the discrete audio data to obtain the audio data of the pitch, the duration and the intensity of the measured musical instrument sound. Finally, the sound rhythm evaluation standard database y is passed2Comparing the ideal audio data, forming an audio feedback modification suggestion according to the result of the dynamic comparison, and sending the audio feedback modification suggestion to the main control module for processing, wherein the main control module sends the playing modification suggestion to the main control moduleAnd the sub-control module is used for adjusting the length and the intensity of the keys with the appointed pitch.
And 6, repeating the operations of the step 4 and the step 5 on 88 key objects of the piano in sequence until the intelligent piano playing effect is adjusted to the standard sound effect. In order to prevent the packet loss during the transmission, the packet is transmitted again after the specified time interval t 2.
Further, the micro control module in step 1 initializes the mobile terminal application module, the main control module, the sub control module, the electromagnetic solenoid drive module, and the power supply module, wherein the temperament sensor module and the CAN bus module are in a temporary sleep state to wait for a wake-up command.
Further, in step 3, a temperament evaluation standard database y is obtained according to the temperament sensor module2The method comprises the following steps:
the rhythm sensing module is used for sensing the audio database y1The method comprises the steps of playing a monophonic note music event, sampling to obtain discrete audio data, delivering the discrete audio data to a sound rate identification module, and performing sound frequency extraction on the monophonic note music event by the sound rate identification module by using a wavelet analysis method to determine the pitch attribute of the monophonic note music event, namely determine which note the monophonic note music event belongs to. And measuring the duration of the vibration waveform of the monophonic event to determine the duration attribute of the monophonic event. And finally, carrying out Fourier transform on the collected data, and calculating the decibel value of the monophonic note event, namely determining the sound intensity attribute of the monophonic note event.
The pitch rate identification module stores the obtained attributes of the notes, the duration and the intensity in a temperament evaluation standard database y2As a standard for comparative evaluation in step 5.
Further, the necessity and the process of the rhythm identification module for automatically adjusting the rhythm in step 5 are as follows:
(1) the necessity: the sound production principle of the intelligent piano playing system is that MIDI music files are finally converted into mechanical sound production processes of electromagnetic drive impacting keys through analysis and translation. In the whole conversion process, the mechanical impact is required to produce sound to accurately represent the duration and the intensity of a MIDI single note music event, and a large amount of human resources are required to continuously measure and calibrate. Meanwhile, the MIDI music file has 127-grade different playing dynamics, and each playing dynamics corresponds to a series of variable factors which possibly influence the playing effect, so that the intelligent piano system with the automatic temperament adjusting system is very necessary and feasible. And the set of system is also applicable to automatic playing systems of all musical instruments.
(2) The process is as follows: the audio information of the real playing of the intelligent piano is collected, and the audio identification module is put in for analysis. The data obtained by identification and the tone law evaluation standard database y2For comparison. And finally, the sub-control module performs repeated and repeated PWM waveform adjustment to optimize the tone intensity attribute, the specific playing time delay or reduction is performed to optimize the tone length attribute, and finally, the optimization of the process of converting the whole MIDI music file into the real playing process of the intelligent piano is completed.
The invention has the beneficial effects that:
(1) the problems that the efficiency of the tone rhythm calibration process is too low and human resources are consumed too much in the development process of the existing intelligent piano automatic playing system are solved.
(2) And a manual regulation mode mainly based on empirical judgment and estimation is replaced by a quantitative and dynamic tone law calibration mode.
(3) The mobile application terminal is used as a system console, so that the user use experience of the system is optimized.
Detailed Description
The invention is further described below with reference to the accompanying drawings.
Combine fig. 1, an intelligence piano temperament discernment and automatic adjustment system, including removing end application module, main control module, branch control module, electromagnetism spiral drive module, power module and the temperament sensor module who is connected respectively with this main control module, WIFI module, CAN bus module, all parts are integrated integrative, connect to set up in the piano central point and put, wherein:
the mobile terminal application template issues an instruction to the main control module through a WIFI protocol, and receives, stores and displays the running condition information of the intelligent piano system in real time.
And the main control template waits for receiving the instruction of the mobile terminal through a WIFI protocol, analyzes the format of the MIDI music file, and is responsible for sending the voice recognition feedback information to the subordinate sub-control modules.
The sub-control modules receive the instructions of the main control panel, cooperatively control 88 keys among the modules, and have the capabilities of real-time parallel processing, dynamic force output, storage and play of evaluation feedback information and indirect processing of temperament feedback information.
The electromagnetic spiral module is connected with the sub-control module pins and drives the magnets at the bottoms of the piano keys after being electrified to impact the keys to enable the piano to produce sound.
The power supply module comprises a 36V rechargeable battery arranged inside the piano and supplies power to the whole piano control system.
The melody sensor module captures sound data in the environment, samples, quantifies and analyzes the data, and feeds back the obtained analysis data to the main control module.
And the WIFI module is used for wireless communication between the mobile terminal and the main control module and forwarding the intelligent piano condition information.
And the CAN bus module is used for the area communication between the main control module and the sub-control module and is responsible for information transmission and interaction between the intelligent piano system boards.
As a specific example, the main control module adopts an i.MX287ARM industrial control board, an ARM9 inner core is taken as a processor core, the main frequency can reach 454MHz at most, and the main control module is provided with a DDR2 and a NAND Flash memory and has very low power consumption. The main control module is connected with the mobile terminal application module through WIFI, connected with the sub-control module through the CAN bus module and connected with the computer through the serial port output module.
As a specific example, the sub-control module adopts an STM32 microprocessor which is low in power consumption, strong in performance and easy to develop, is connected with the main control module through the CAN bus module, and is connected with the electromagnetic solenoid module through a lead, and has real-time parallel processing capability, dynamic force output capability, capability of storing and playing evaluation feedback information, and capability of indirectly processing temperament feedback information, and the specific examples are as follows:
(1) real-time parallel processing capability: each sub-control module is responsible for controlling twelve keys, and a thread is created for the generation of the PWM waveform of each key, so that different keys cannot be blocked mutually. The CAN bus module of the microprocessor has an automatic awakening function for receiving the synchronous interval characters, and the processor CAN awaken a corresponding process when receiving the data frame.
(2) Dynamic force output capacity: data information of the changing force is also integrated into communication protocols of the main control module and the sub-control modules, and the sub-control modules can accurately control the force to change step by matching with the timer modules after receiving specific instructions.
(3) And (3) storing and playing evaluation feedback information capacity: the microprocessor is provided with a Flash memory chip of a permanent memory medium, can quickly and permanently store the data received immediately and store the data in combination with historical data.
(4) And (3) indirectly processing the rhythm feedback information: and receiving the range feedback information of the temperament sensor module, adjusting the playing pitch intensity of the corresponding musical notes by changing the corresponding PWM waveforms and frequency according to the requirements of the protocol data frames, and adjusting the playing pitch length of the corresponding musical notes by adjusting the pressing and playing time of the corresponding musical notes.
As a specific example, the temperament sensor module includes 2 parts, specifically:
1) the digital microphone STMicroelectronics MP45DT02 is used as a temperament perception module for receiving the musical instrument sound, and the temperament perception module is connected with the temperament identification module and is responsible for converting the temperament collected in the environment into discrete and processable audio data and then delivering the discrete and processable audio data to the temperament identification module for processing.
2) The high-performance computer is used as a temperament identification module for calculating the piano sound, the temperament identification module is connected with the temperament perception module and is connected with the main control module through the CAN bus module, and is responsible for receiving the discrete audio data transmitted by the temperament perception module, and performing denoising and sound processing, time domain analysis, frequency domain analysis, characteristic processing and temperament identification on the discrete audio data, so that the audio data CAN better embody the essence of sound, and then the audio data of the pitch, the duration and the intensity of the collected piano sound is obtained. Finally, the audio feedback modification suggestion is formed according to the result of dynamic comparison by dynamically comparing the audio feedback modification suggestion with the ideal audio data pre-judged in the MIDI music file and is processed by the main control module.
With reference to fig. 2, the intelligent piano temperament identification and automatic adjustment method of the invention comprises the following steps:
step 1, firstly, initializing a mobile terminal application module, a main control module, a sub-control module, an electromagnetic spiral driving module, a power supply module, a melody sensor module and a CAN bus module.
Step 2, creating a standard piano MIDI file to be played through an audio playing device, and recording an audio database y obtained by an audio sensor module1Carrying out denoising and sound-changing processing, time domain analysis, frequency domain analysis, characteristic processing and temperament identification on the audio database to obtain a temperament evaluation standard database y2As audio evaluation and feedback comparison standard.
And 3, starting the WIFI module, sending an automatic temperament adjustment starting instruction to the main control module by the mobile terminal application, and displaying the system running state sent back by the main control module in real time through the user interface.
And 4, combining a play protocol data format table, broadcasting the MIDI music file to the CAN bus module after the MIDI music file is analyzed by the main control module, receiving data streams by the sub-control modules, generating PWM waveforms in real time, and controlling the electromagnetic solenoid module to drive the keys to sound.
Play protocol data format table
Protocol header
|
Message type
|
Keyboard number
|
Original force
|
Force of ending
|
Duration of time
|
Time delay
|
1 byte
|
1 byte
|
1 byte
|
1 byte
|
1 byte
|
1 byte
|
1 byte |
And 5, identifying a feedback protocol data format table by combining the temperament. And the tone law identification module acquires discrete audio data, and performs denoising processing, time domain analysis, frequency domain analysis, characteristic processing and tone law identification on the discrete audio data to obtain the audio data of the pitch, the duration and the intensity of the measured musical instrument sound. Finally, the sound rhythm evaluation standard database y is passed2Comparing the idealized audio data to form audio based on the results of the dynamic comparisonThe modification suggestions are fed back and processed by the main control module, the main control module sends the playing modification suggestions to the sub-control modules, and the sub-control modules adjust the length and the intensity of the keys with the appointed pitch.
Data format table of voice rhythm identification feedback protocol
Protocol header
|
Control code
|
Phonetic symbol code
|
Data segment length
|
Data segment |
And 6, repeating the operations of the step 4 and the step 5 on 88 key objects of the piano in sequence until the intelligent piano playing effect is adjusted to the standard sound effect. In order to prevent the packet loss during the transmission, the packet is transmitted again after the specified time interval t 2.