JP7197922B2 - Machine learning device, analysis device, machine learning method and analysis method - Google Patents

Description

The present invention relates to a technique for automatically extracting or classifying body sounds from acoustic data in an environment where the signal-to-noise ratio is degraded.

Decreased or lost gastrointestinal motility is a problem that greatly affects QOL and daily eating habits. An example of this is functional gastrointestinal disorders (FGIDs), in which gastrointestinal motility is disturbed due to stress and the like.

Diagnosis of such intestinal disorders is performed by assessing gastrointestinal motility. Currently, X-ray examination and endoscopy are used as a means of measuring gastrointestinal motility, but they impose a heavy physical and financial burden on the patient and require large-scale examination equipment, requiring repeated observations. not suitable for

In recent years, acoustic features obtained from intestinal peristaltic sounds (BS: Bowel sound) have been used to evaluate intestinal motility. Intestinal peristaltic sounds are sounds generated when gas and contents move in the digestive tract due to peristalsis of the digestive tract (Non-Patent Document 1). Intestinal peristaltic sounds can be easily recorded by attaching an electronic stethoscope to the body surface. For example, Non-Patent Document 2 discloses a method of automatically extracting BS from recorded data acquired by an electronic stethoscope and evaluating intestinal motor function.

In quiet conditions, the BS signal-to-noise ratio is degraded, but the BS can be recognized at a distance without the use of an electronic stethoscope. Therefore, recent studies by the present inventors have shown that BS can be used to assess intestinal motility when acquired using a non-contact microphone as well as when using an electronic stethoscope. (Non-Patent Document 3).

G. P. Zaloga, "Blind bedside placement of enteric feeding tubes", Techniques in Gastrointestinal Endoscopy, 2001, 3(1), p.9-15 Takahiro Emoto, et al. "ARMA-based spectral bandwidth for evaluation of bowel motility by the analysis of bowel sounds.", Physiological measurement 34.8 (2013): 925. Takahiro Emoto et. al, "Evaluation of human bowel motility using non-contact microphones", Biomedical Physics & Engineering Express, 2016, 2(4), 045012.

However, the study of Non-Patent Document 3 required a lot of time-consuming and careful labeling work to manually extract the BS from the recording data acquired by the non-contact microphone. Microphone-based sensors (eg, electronic stethoscopes and microphones) are susceptible to environmental noise. BS recorded with a non-contact microphone has a lower sound pressure than BS obtained with an electronic stethoscope directly from the body surface. In addition, non-BS sounds are louder and more likely to be mixed in than electronic stethoscope recordings. Therefore, in order to omit the BS labeling work that requires a lot of time and effort, it is necessary to construct a noise-robust BS extraction system.

SUMMARY OF THE INVENTION The present invention has been made to solve the above problems, and an object of the present invention is to accurately extract or classify body sounds from noisy acoustic data.

The present inventors have found that the above problems can be solved by using a prediction algorithm machine-learned using features that are robust to noise, and have completed the present invention.

The present invention includes the following aspects.
Section 1.
A machine learning device that learns a prediction algorithm for predicting whether acoustic data contains body sounds,
an acoustic data acquisition unit that acquires acoustic data obtained from a subject by a sound collector;
a body sound determination unit that determines whether or not the body sound is included in the acoustic data according to a user's operation;
a feature quantity extraction unit for extracting a feature quantity in the acoustic data;
a learning unit that learns the prediction algorithm based on the determination result of the body sound determination unit and the feature quantity;
with
The features include PNCC, MFCC, ΔPNCC, ΔΔPNCC, ΔMFCC, ΔΔMFCC, BSF, formant-related features, pitch-related features, LPC coefficients, spectral flatness, logarithmic energy, and voiced sections. , ZCR, and entropy-based metrics, and/or statistics thereof.
Section 2.
Item 2. The machine learning device according to Item 1, wherein the body sound is intestinal peristaltic sound.
Item 3.
Item 3. The machine learning device according to Item 1 or 2, wherein the feature amount includes PNCC.
Section 4.
4. The machine learning device according to any one of Items 1 to 3, wherein the feature amount includes at least one of BSF and its statistic.
Item 5.
Item 5. The machine learning device according to item 4, wherein the feature amount includes the mean and standard deviation of BSF1, the mean and standard deviation of BSF2, the mean and standard deviation of BSF3, the mean and standard deviation of BSF4, and BSF5.
Item 6.
Item 6. The machine learning device according to any one of Items 1 to 5, wherein the learning unit is composed of an artificial neural network (ANN).
Item 7.
Item 7. The machine learning device according to any one of Items 1 to 6, wherein the sound collecting device is a non-contact microphone.
Item 8.
further comprising a segment detection unit that detects a plurality of segments from the acoustic data acquired by the acoustic data acquisition unit;
The body sound determination unit determines whether or not each segment includes the body sound according to a user's operation,
The feature quantity extraction unit extracts a feature quantity in each segment,
Item 8. The machine learning device according to any one of items 1 to 7, wherein the learning unit learns the prediction algorithm based on the feature amount in each segment and the determination result by the body sound determination unit.
Item 9.
Item 9. The machine learning device according to Item 8, wherein the segment detection unit detects a segment having an SNR equal to or greater than a predetermined value.
Item 10.
further comprising a classification determination unit that determines a type of the body sound according to a user's operation when the body sound is included in the acoustic data,
Item 10. The machine learning device according to any one of items 1 to 9, wherein the learning unit learns the prediction algorithm further based on the type of the body sound.
Item 11.
An analysis device for analyzing acoustic data obtained from a subject by a sound collector,
Item 11. An analysis apparatus comprising a body sound prediction unit that predicts whether the acoustic data includes a body sound according to a prediction algorithm learned by the machine learning apparatus according to any one of items 1 to 10.
Item 12.
a body sound segment extraction unit that extracts a segment containing the body sound from the acoustic data based on the prediction result of the body sound prediction unit;
a first condition evaluation unit that evaluates the condition of the subject based on the segment extracted by the body sound segment extraction unit;
Item 12. The analysis device according to Item 11, further comprising:
Item 13.
the body sound is an intestinal peristaltic sound,
Item 13. The analysis device according to Item 12, wherein the first condition evaluation unit evaluates intestinal motility as the condition.
Item 14.
The prediction algorithm is a prediction algorithm learned by the machine learning device according to Item 8,
Item 14. The analysis according to any one of items 11 to 13, further comprising a classification prediction unit that predicts the type of the body sound according to the prediction algorithm when it is predicted that the body sound is included in the acoustic data. Device.
Item 15.
Item 15. The analysis device according to Item 14, further comprising a second condition evaluation unit that evaluates the condition of the subject based on the type of body sound predicted by the classification prediction unit.
Item 16.
the body sound is an intestinal peristaltic sound,
Item 16. The analysis device according to Item 15, wherein the second condition evaluation unit evaluates the presence or absence of intestinal disease as the condition.
Item 17.
A machine learning method for learning a prediction algorithm for predicting whether acoustic data contains body sounds, comprising:
an acoustic data acquisition step of acquiring acoustic data obtained from a subject by a sound collector;
a body sound determination step of determining whether or not the body sound is included in the acoustic data according to a user's operation;
a feature quantity extraction step of extracting a feature quantity in the acoustic data;
a learning step of learning the prediction algorithm based on the determination result of the body sound determination step and the feature quantity;
with
The features include PNCC, MFCC, ΔPNCC, ΔΔPNCC, ΔMFCC, ΔΔMFCC, BSF, formant-related features, pitch-related features, LPC coefficients, spectral flatness, logarithmic energy, and voiced sections. , ZCR, and entropy-based metrics, and/or statistics thereof.
Item 18.
An analysis method for analyzing acoustic data obtained from a subject by a sound collector,
Item 18. An analysis method, comprising a prediction step of predicting whether the acoustic data includes a body sound according to a prediction algorithm learned by the machine learning method according to Item 17.
Item 19.
a body sound segment extraction step of extracting a segment containing the body sound from the acoustic data based on the prediction result of the prediction step;
a state evaluation step of evaluating the state of the subject based on the segment extracted by the body sound segment extraction step;
19. The analysis method according to Item 18, further comprising:

According to the present invention, it is possible to accurately extract or classify body sounds from noisy acoustic data.

1 is a block diagram showing a schematic configuration of a diagnostic support system according to one embodiment of the present invention; FIG. 1 is a block diagram showing functions of a machine learning device according to an embodiment of the present invention; FIG. 4 is a flow chart showing the overall procedure of a machine learning method according to one embodiment of the present invention; It is a block diagram showing the function of the analysis device according to one embodiment of the present invention. 4 is a flow chart showing the overall procedure of an analysis method according to one embodiment of the present invention; It is a block diagram which shows the function of the machine-learning apparatus based on the modification of this invention. It is a block diagram which shows the function of the analysis apparatus based on the modification of this invention. 2 is a graph showing the prediction accuracy (Acc) for each SNR reference value when the feature amount is MFCC and PNCC, (a) being a graph before ingestion of carbonated water, and (b) being a graph after ingestion of carbonated water. be. (a) and (b) show time transitions of the four indices calculated in the preliminary verification. It shows the change in the number of BS occurrences per minute during the lactic acid bacteria drink load test.

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In addition, this invention is not limited to the following embodiment.

(overall structure)
FIG. 1 is a block diagram showing a schematic configuration of a diagnosis support system 100 according to this embodiment. A diagnosis support system 100 includes a machine learning device 1 and an analysis device 2 . The machine learning device 1 learns a prediction algorithm for predicting whether or not body sounds are included in acoustic data. The analysis device 2 has a function of predicting whether or not body sounds are included in the acoustic data obtained from the subject according to the prediction algorithm learned by the machine learning device 1, and of evaluating the state of the subject. ing. The machine learning device 1 and the analysis device 2 may be realized by separate devices, or the machine learning device 1 and the analysis device 2 may be configured by one device.

Configuration examples of the machine learning device 1 and the analysis device 2 will be described below.

(machine learning device)
FIG. 2 is a block diagram showing functions of the machine learning device 1 according to this embodiment. The machine learning device 1 can be configured by, for example, a general-purpose personal computer, and includes a CPU (not shown), a main memory device (not shown), an auxiliary memory device 11, and the like as a hardware configuration. In the machine learning device 1, the CPU reads various programs stored in the auxiliary storage device 11 to the main storage device and executes them, thereby executing various arithmetic processing. The auxiliary storage device 11 can be composed of, for example, a hard disk drive (HDD) or solid state drive (SSD). The auxiliary storage device 11 may be built in the machine learning device 1 or may be provided as an external storage device separate from the machine learning device 1 .

An input device 3 and a sound collector 4 are connected to the machine learning device 1 . The input device 3 is composed of, for example, a keyboard, a touch panel, a mouse, etc., and receives input operations from the user.

The sound collector 4 is configured by a non-contact microphone in this embodiment. By bringing the sound collector 4 close to the subject 5 , the sound collector 4 records body sounds emitted from the subject 5 and transmits acoustic data to the machine learning device 1 .

The acoustic data may be transmitted to the machine learning device 1 by wire or wirelessly, or may be input to the machine learning device 1 via a recording medium such as an SD card. Moreover, the sound collector 4 is not limited to a non-contact microphone, and may be an electronic stethoscope. Alternatively, the sound collector 4 may be configured by combining a non-contact microphone and an electronic stethoscope. Conventionally, techniques have been developed to use multiple stethoscopes on the abdomen. It is expected that intestinal motility can be evaluated in the future.

The machine learning device 1 has a function of learning a prediction algorithm for predicting whether or not body sounds are included in acoustic data. In order to realize this function, the machine learning device 1 includes a teacher data creation unit 12 and a learning unit 13 as functional blocks. Body sounds are not particularly limited as long as they are sounds caused by human body activities, but in the present embodiment, intestinal peristaltic sounds are targeted.

The teacher data creation unit 12 is a functional block that creates teacher data D1 based on the acoustic data from the sound collector 4, and includes an acoustic data acquisition unit 121, a segment detection unit 122, a body sound determination unit 123, and a feature amount extraction unit. 124 and .

The acoustic data acquisition unit 121 acquires acoustic data obtained from the subject 5 by the sound collector 4 . Although the posture of the subject 5 is not particularly limited, it is in the supine position in this embodiment.

The segment detection unit 122 detects multiple segments from the acoustic data acquired by the acoustic data acquisition unit 121 . A segment detection criterion is not particularly limited, but in the present embodiment, the segment detection unit 122 uses the STE (Short term energy) method to detect a segment whose SNR (Signal to Noise Ratio) is equal to or greater than a predetermined value.

ここで、Ｐ_Ｓは信号のパワー、Ｐ_Ｎは雑音のパワーである。Ｐ_Ｎは、視聴試験を行うことにより、サイレンスであると判断された１秒の区間から算出している。録音データは、セグメントの候補となるサブセグメントサイズ：２５６、シフトサイズ：６４で分割され、ＳＴＥ法によって、サブセグメント毎に、エネルギを計算することができる。ＳＮＲ（Signal to Noise Ratio）が所定値以上を１として、所定値以下を０とする。１以上はセグメントであり、連続して続くサブセグメントはセグメントとして取り扱う。 The SNR in this embodiment is defined as follows.

where P _S is the power of the signal and P _N is the power of the noise. _PN is calculated from the 1-second interval determined to be silent by conducting a viewing test. The recorded data is divided into subsegment size: 256 and shift size: 64, which are candidates for segments, and energy can be calculated for each subsegment by the STE method. SNR (Signal to Noise Ratio) is set to 1 when it is equal to or higher than a predetermined value, and is set to 0 when it is equal to or lower than a predetermined value. One or more are segments, and consecutive subsegments are treated as segments.

The body sound determination unit 123 determines whether or not body sounds are included in the acoustic data according to the user's operation. In this embodiment, the user listens to the reproduced sound of each segment, determines whether or not each segment includes intestinal peristalsis (BS), which is a body sound, and inputs the determination result via the input device 3 . In response to this, body sound determination section 123 determines whether or not each segment includes BS. Specifically, the body sound determination unit 123 defines a section determined by the user to include BS as a BS episode, and if a BS episode exists within the range of each segment, the segment is defined as a BS episode. BS segment, and if it does not exist, it is a non-BS segment.

It should be noted that the BS acquired using a non-contact microphone generally has a low sound and the SN of the acoustic data is degraded. However, human auditory evaluation can identify the presence or absence of BS with almost 100% accuracy.

The feature amount extraction unit 124 extracts feature amounts in the acoustic data. In this embodiment, the feature quantity is PNCC (power normalized cepstral coefficients), but the present invention is not limited to this. Features include, for example, MFCC (mel-frequency cepstral coefficients), ΔPNCC, ΔΔPNCC, ΔMFCC, ΔΔMFCC, BSF, formant-related features, pitch-related features, LPC coefficients, spectral flatness, Entropy-based measures such as logarithmic energy, talkspurt duration, ZCR, and approximate entropy, as well as their statistics (mean, standard deviation, etc.) can be used. For details of PNCC, see Kim, Chanwoo, and Richard M. Stern. "Power-normalized cepstral coefficients (PNCC) for robust speech recognition." Acoustics, Speech
and Signal Processing (ICASSP), 2012 IEEE International Conference on. IEEE, 2012.

MFCC is also called a feature value representing vocal tract characteristics, and has been generally used particularly in the field of speech recognition. This MFCC has also been applied to the detection of body sounds associated with vocal tract characteristics. No, it was not used to detect bowel sounds. The MFCC is calculated by performing a discrete cosine transform on the output of a triangular filter bank, called a melscale, which is a simple simulation of the human auditory system, and which is arranged at regular intervals on a logarithmic axis.

PNCC is a feature quantity developed to improve the robustness of speech recognition systems in noisy environments. However, when the sampling rate of recorded data is low (for example, when recording data from a stethoscope), PNCC may have worse detection performance than MFCC depending on the acoustic and spectral characteristics of the sound data to be detected. There is a report that there is PNCC is an improvement on the process of calculating MFCC to more closely approximate human physiology. PNCC differs from MFCC mainly in the following three points.

First, instead of the triangular filterbank used in MFCC, we use a gammatone filterbank based on the equivalent rectangular bandwidth to simulate the working of the cochlea. Second, it uses biased subtraction based on the AM-to-GM ratio of mid-time processed speech, which is not used in the MFCC calculation process. be. The third point is to replace the logarithmic nonlinearity used in MFCC with power nonlinearity. It is said that these technologies enable speech processing that is robust against noise.

BSF (bowel sound feature): BSF1 to BSF5 are new features discovered by the present inventors. In the structure of PNCC feature extraction, squared gammatone integration processing based on transfer functions of a 24-channel gammatone-shaped bank, peak power normalization processing, power after power bias subtraction processing: Power-law nonlinearity to U(i,l) The applied power is expressed as:
GV(i,l)=U(i,l) ^1/15
where i is the frame and l is the channel index.

α_ｉ以外のＢＳＦ１として、例えば、フレームごとに、ＧＶ（ｉ，ｌ）の中心モーメントを使用することができる。ＧＶ（ｉ，ｌ）は、０から１にスケーリングすることもできる。 BSF1: Power: A new BS feature quantity obtained based on GV(i,l). There are several methods for calculating BSF1, one of which is α _i in this specification. α _i is the sum of the squares of the power: GV(i,l) minus the average value of GV(i,l).

As BSF1 other than α _i , for example, the central moment of GV(i,l) can be used for each frame. GV(i,l) can also be scaled from 0 to 1.

ここで、ｃ_ｉ（ｓ）は、ｉ番目のフレームにおけるｓ次元目のＰＮＣＣである。Ｐｉ（ｆ）は、ｉ番目のフレームにおけるパワースペクトルを表している。 BSF2: A new BS feature obtained based on PNCC and power spectrum. In this specification, one of the BSF2 is designated as β _i . β _i is obtained by dividing the mean value of the S-dimensional PNCC by the mean value of the power spectrum for each frame.

where c _i (s) is the s-th PNCC in the i-th frame. Pi(f) represents the power spectrum in the i-th frame.

ここで、ｃ_ｉバーは、ｉ番目のフレームにおけるＰＮＣＣの平均値である。 BSF3: A new BS feature obtained based on PNCC. One of the _BSF3s is referred to herein as γi. γ _i is the variance value of the S-dimensional PNCC for each frame.

where c _i is the average value of PNCC in the i-th frame.

この特徴量は、ＢＳＦ３とほぼ等価であり。状況に応じて、ＢＳＦ３か、ＢＳＦ４、どちらかが選択されるべきである。 BSF4: This is a new BS feature also obtained based on PNCC. In this specification, one of BSF4 is designated as ζ _i . ζ _i is the sum of squares of the S-dimensional PNCC for each frame.

This feature amount is almost equivalent to BSF3. Either BSF3 or BSF4 should be selected depending on the situation.

BSF5: BS segment length: T obtained by manual labeling or automatic extraction.

BSF1, BSF2, BSF3, and BSF4 can be calculated even when the power bias subtraction process is omitted, or when the filter bank is changed to a mel filter bank or the like. In particular, BSF3 is expected as a feature quantity to replace STE.

In this embodiment, the feature amount extraction unit 124 extracts the PNCC in each segment detected by the segment detection unit 122, but the feature amount is not limited to this. Then, the teacher data creation unit 12 creates teacher data D1 by associating the determination result of the body sound determination unit 123 with the PNCC extracted by the feature quantity extraction unit 124 for each segment. The teacher data D1 is stored in the auxiliary storage device 11, for example.

The learning unit 13 is a functional block that learns the prediction algorithm D2 based on the teacher data D1. In this embodiment, the learning unit 13 is composed of an artificial neural network (ANN). The structure of ANN is a hierarchical neural network consisting of at least three layers: an input layer, an intermediate layer, and an output layer. The learned prediction algorithm D2 is stored in the auxiliary storage device 11, for example.

The learning unit 13 is not limited to ANN, linear discriminant function, Gaussian Mixture Model (GMM), Support Vector Machine (SVM), Probabilistic neural network (PNN), Radial bias function network (RBFN), Convolutional neural network (CNN) ), DeepNN, and DeepSVM.

(machine learning method)
The machine learning method according to this embodiment is implemented using the machine learning device 1 shown in FIG. FIG. 3 is a flow chart showing the overall procedure of the machine learning method according to this embodiment.

In step S1, the acoustic data acquisition unit 121 acquires acoustic data obtained from the subject 5 by the sound collector 4 (acoustic data acquisition step).

In step S2, the segment detection unit 122 detects a plurality of segments whose SNR is equal to or greater than a predetermined value from the acoustic data.

In step S3, the body sound determination unit 122 determines whether or not each segment includes a body sound (intestinal peristaltic sound in this embodiment) according to the user's operation (body sound determination step).

In step S4, the feature amount in each segment is extracted (feature amount extraction step). Preferably, the features include PNCC. Teacher data D1 is created by associating the determination result of step S3 with the PNCC extracted in step S4 for each segment. The order of steps S3 and S4 is not particularly limited.

After that, steps S1 to S4 are repeated while changing the subject 5 until enough teacher data D1 is accumulated (YES in step S5).

At step S6, the learning unit 13 learns the prediction algorithm D2 based on the teacher data D1.

(Analysis device)
A form of predicting whether or not body sounds are included in acoustic data using the learned prediction algorithm D2 will be described below.

FIG. 4 is a block diagram showing functions of the analysis device 2 according to this embodiment. The analysis device 2 can be composed of, for example, a general-purpose personal computer, like the machine learning device 1 shown in FIG. That is, the analysis device 2 includes a CPU (not shown), a main storage device (not shown), an auxiliary storage device 51, etc. as a hardware configuration. In the analysis device 2, the CPU reads various programs stored in the auxiliary storage device 51 into the main storage device and executes them, thereby executing various arithmetic processing. The auxiliary storage device 51 can be composed of, for example, a hard disk drive (HDD) or a solid state drive (SSD), and stores the learned prediction algorithm D2. Further, the auxiliary storage device 51 may be built in the analysis device 2 or may be provided as an external storage device separate from the analysis device 2 .

A sound collector 4 and a display device 6 are connected to the analysis device 2 . The sound collector 4 can have the same configuration as the sound collector 4 shown in FIG. The display device 6 can be composed of, for example, a liquid crystal display.

The analysis device 2 has a function of predicting whether the acoustic data obtained from the subject includes body sounds according to the prediction algorithm learned by the machine learning device 1 described above, and further evaluating the state of the subject 7. have. In order to realize this function, the analysis device 2 includes, as functional blocks, an acoustic data acquisition unit 22, a segment detection unit 23, a feature amount extraction unit 24, a body sound prediction unit 25, a body sound segment extraction unit 26, and a state evaluation unit. (First state evaluation unit) 27 is provided. At least part of the functions of the analysis device 2 may be installed in the sound collector 4 .

The acoustic data acquisition unit 22, the segment detection unit 23, and the feature amount extraction unit 24 have the same functions as the acoustic data acquisition unit 121, the segment detection unit 122, and the feature amount extraction unit 124 of the machine learning device 1 shown in FIG. doing. That is, the acoustic data acquisition unit 22 acquires acoustic data obtained from the subject 7 by the sound collector 4, and the segment detection unit 23 extracts a plurality of segments from the acoustic data acquired by the acoustic data acquisition unit 22. The feature quantity extraction unit 24 extracts the feature quantity in the acoustic data. The feature quantity used by the feature quantity extraction unit 24 is the same as the feature quantity used in the feature quantity extraction unit 124 of the machine learning device 1 .

The body sound prediction unit 25 predicts whether or not body sounds are included in the acoustic data according to the prediction algorithm D2. In the present embodiment, the body sound prediction unit 25 determines whether each segment detected by the segment detection unit 23 includes intestinal peristaltic sounds (BS) based on the feature amount extracted by the feature amount extraction unit 24. predict whether More specifically, body sound prediction section 25 outputs a prediction score of 0 to 1 indicating the possibility that BS is included in each segment as a prediction result.

The body sound segment extraction unit 26 extracts segments containing body sounds from the acoustic data based on the prediction result of the body sound prediction unit 25 . In this embodiment, from among the segments detected by the segment detection unit 23, a segment whose prediction score is greater than the optimum threshold _Th is extracted as a segment containing a BS (BS segment).

ここで、ＴＰ、ＴＮ、ＦＰ、ＦＮの定義は以下の通りである。
True Positive（ＴＰ）：ＢＳセグメントを自動抽出した数
True Negative（ＴＮ）：ｎｏｎ－ＢＳセグメントを自動抽出しなかった数
False Negative（ＦＮ）：ＢＳセグメントを自動抽出しなかった数
False Positive（ＦＰ）：ｎｏｎ－ＢＳセグメントを自動抽出した数 The optimum threshold _Th is set as follows. First, based on the prediction score of the body sound prediction unit 25, receiver operating characteristic (ROC) analysis is performed to determine sensitivity, specificity, and accuracy at the cutoff point. ) can be obtained as follows.

Here, the definitions of TP, TN, FP and FN are as follows.
True Positive (TP): Number of automatically extracted BS segments
True Negative (TN): Number of non-BS segments that were not automatically extracted
False Negative (FN): Number of BS segments not automatically extracted
False Positive (FP): Number of automatically extracted non-BS segments

The optimal threshold _Th is determined based on the point with the shortest Euclid distance from the position of sensitivity: 1 and specificity: 1 in the ROC curve. In this embodiment, for example, T _h can be set to 0.55.

The state evaluation unit 27 evaluates the state of the subject 7 based on the segments extracted by the body sound segment extraction unit 26 . In this embodiment, the state evaluation unit 27 evaluates intestinal motility as the state. The evaluation result of the state evaluation unit 27 is displayed on the display device 6, for example.

(analysis method)
The analysis method according to this embodiment is performed using the analysis device 2 shown in FIG. FIG. 5 is a flow chart showing the overall procedure of the analysis method according to this embodiment.

In step S11, the acoustic data acquisition unit 22 acquires acoustic data obtained from the subject 7 by the sound collector 4 (acoustic data acquisition step).

In step S12, the segment detection unit 23 detects a plurality of segments whose SNR is equal to or greater than a predetermined value from the sound data.

In step S13, the feature quantity extraction unit 24 extracts the feature quantity in each segment. The feature amount here is the same as the feature amount used in step S4 of the machine learning method described above.

In step S14, the body sound prediction unit 25 predicts whether or not body sounds are included in the acoustic data according to the prediction algorithm D2 (prediction step). In this embodiment, the body sound prediction unit 25 predicts whether the segment extracted by the feature amount extraction unit 24 includes intestinal peristaltic sound (BS).

If there is another segment that has not been predicted to include a BS (YES in step S17), the process returns to step S13 and repeats the processes up to step S16.

On the other hand, if there is no other segment (NO in step S17), in step S18, the state evaluation unit 27 evaluates the state of the subject 7 based on the extracted segment (state evaluation step). In this embodiment, the state evaluation unit 27 evaluates the intestinal motility of the subject 7 based on the BS segment. For example, the number of BS segments per minute, BS length, BS segment energy, and BS segment interval can be used to assess gut motility, as described below. Also, the concept of physical assessment can be applied to the detected BS segment.

As described above, the analysis device 2 uses the learned prediction algorithm D2 to predict whether or not body sounds are included in the acoustic data. Here, the prediction algorithm D2 is obtained by machine learning in the machine learning device 1, and by performing machine learning using a sufficient amount of teacher data D1, it is possible to increase the prediction accuracy of the analysis device 2. becomes. In particular, in the present embodiment, the feature amounts of acoustic data used for prediction are PNCC, MFCC, ΔPNCC, ΔΔPNCC, ΔMFCC, ΔΔMFCC, formant-related features, pitch-related features, LPC coefficients, Entropy-based measures such as spectral flatness, log energy, talkspurt duration, ZCR, and approximate entropy, and/or statistics thereof. Since these feature quantities are excellent in noise resistance, even if the acoustic data obtained by the sound collector 4 contains a lot of noise, it is possible to determine whether the acoustic data contains body sounds. can be predicted with high accuracy. Therefore, it becomes possible to automatically extract a segment containing body sounds from the acoustic data, and the condition of the subject 7 can be easily evaluated.

[Modification]
In this modified example, a configuration for predicting the type of body sound in addition to the presence or absence of body sound will be described. In this modified example, members having the same functions as those in the above embodiment are denoted by the same reference numerals, and descriptions thereof are omitted.

FIG. 6 is a block diagram showing functions of a machine learning device 1' according to this modification. The machine learning device 1' has a configuration in which the teacher data creation unit 12 is replaced with a teacher data creation unit 12' in the machine learning device 1 shown in FIG. , and a classification determination unit 125 .

The classification determination unit 125 is a functional block that determines the type of the body sound according to the user's operation when the sound data includes the body sound. In this modification, the classification determination unit 125 determines that the body sound determination unit 123 determines that the BS segment includes the intestinal peristaltic sound (BS) according to the user's operation via the input device 3. Determine the type of BS. The types of BS can be classified according to onomatopoeic sounds such as "gu", "kyurukyuru", and "poco". The categories and number of BSs are not particularly limited.

In the physical assessment technique, it is possible to detect the ``gurguru rumbling sounds'' heard as (normal) bowel sounds in general auscultation, short popping sounds, sustained gurgling sounds, and the ``gurgling and rushing sounds'' heard when bowel sounds are enhanced. It can be classified as "sounds like When percussed, it is said that it can be classified into "popping sounds" associated with retention of intestinal gas, and "dull sounds" associated with constipation (where stool is located) or retention of urine in the bladder. Furthermore, intestinal peristaltic sounds can be classified into normal, increased, attenuated, and absent, and increased intestinal sounds are heard when inflammation such as infectious gastroenteritis, diarrhea, and ileus subside. Attenuation of bowel sounds is heard during peritoneal inflammation due to surgery and constipation. Absence of bowel sounds is said to be heard during ileus. In addition, hearing an abdominal vascular murmur is said to be suggestive of a stenotic lesion of the abdominal arteries.

The teacher data creation unit 12' creates teacher data D1' by associating the determination result of the body sound determination unit 123, the classification of the classification determination unit 125, and the PNCC extracted by the feature amount extraction unit 124 for each segment. do. The learning unit 13 learns the prediction algorithm D2' based on the teacher data D1'.

FIG. 7 is a block diagram showing functions of an analysis device 2' according to this modification. The analysis device 2 ′ has a configuration in which the classification prediction section 28 and the state evaluation section (second state evaluation section) 29 are further provided in the analysis device 2 shown in FIG. 4 .

The classification prediction unit 28 is a functional block that predicts the type of body sound according to the prediction algorithm D2' when it is predicted that the sound data contains the body sound. In this modification, for a segment predicted to include a BS by the body sound prediction unit 25, the type of the BS is predicted based on the feature amount such as PNCC extracted by the feature amount extraction unit 24. FIG. This makes it possible to automatically determine the type of BS.

The state evaluation unit 29 evaluates the state of the subject 7 based on the types of body sounds predicted by the classification prediction unit 28 . In this embodiment, the condition evaluation unit 29 evaluates the presence or absence of intestinal disease as the condition. The evaluation result of the state evaluation unit 29 is displayed on the display device 6, for example.

Thus, in this modified example, body sounds can be classified into the sounds described above. Also, when the output layer of the ANN has one unit, body sounds can be classified into two classes, but by using a plurality of output layer units, body sounds can be classified into multiple classes.

Note that this modification can also be used for bowel sound classification after detecting a segment with an SNR equal to or greater than a predetermined value using the STE method. By classifying the body sounds into the above categories, the decrease, disappearance, and enhancement of these sounds can be calculated from the acoustic feature values of the intestinal peristaltic sounds, and the relationship with diseases can be evaluated.

In addition, in this modification, the feature amount used for predicting whether or not the acoustic data contains body sounds and for predicting the type of body sounds is not limited to noise-robust feature amounts. For example, when an electronic stethoscope is used as the sound collector 4 in an environment with little noise, all feature amounts can be used for body sound classification prediction.

[Additional notes]
The present invention is not limited to the above embodiments, and can be modified in various ways within the scope of the claims. Forms obtained by appropriately combining technical means disclosed in the embodiments are also techniques of the present invention. included in the scope.

For example, in the above embodiment, segments were extracted from the acoustic data obtained from the subject, and each segment was subjected to determination of whether body sounds were included and feature extraction. However, segment extraction is essential. is not.

Further, in the above-described embodiment, a case where the body sound is intestinal peristaltic sound (BS) has been described, but there is no particular limitation as long as it is a body sound caused by body activity. Examples of such body sounds include heartbeat sounds, swallowing sounds, breathing sounds (snoring), utterance sounds (way of speaking), walking sounds, and the like.

Examples of the present invention will be described below, but the present invention is not limited to the following examples.

[Example 1]
In Example 1, a prediction algorithm is learned using MFCC and PNCC as feature quantities, and the learned prediction algorithm predicts whether or not intestinal peristaltic sound (BS) is included in acoustic data, and determines whether BS is included. We verified whether it is possible to predict the acoustic data

Specifically, a carbonated water tolerance test (STT) was performed on 20 male subjects (age: 22.9 ± 3.4, BMI: 22.7 ± 3.8) who consented to the research content. rice field. The subjects were asked to ingest carbonated water after fasting for 12 hours or longer, and STT was performed during 10 minutes of rest before ingestion of carbonated water and during 15 minutes of rest after ingestion of carbonated water. A non-contact microphone (NT55, manufactured by RODE), an electronic stethoscope (E-Scope 2, manufactured by Cardionics), and a multitrack recorder (R16, manufactured by ZOOM) were used as sound collectors. Acoustic data were acquired at a sampling frequency of 44100 Hz and a digital resolution of 16 bits. During testing, subjects were in a supine position with an electronic stethoscope positioned 9 cm to the right of the navel and a non-contact microphone positioned 20 cm above the navel.

After the acoustic data acquired from the non-contact microphone was loaded into the machine learning device, segments with an SNR equal to or greater than a predetermined value were detected (step S2 in FIG. 3). It is reported that intestinal peristaltic sound (BS) generally has a main frequency component between 100 Hz and 500 Hz. Processing (cutoff frequency: 100 Hz to 1500 Hz) was performed. (Note that in all of the examples below, this third-order Butterworth bandpass filtering is performed on the acoustic data as preprocessing.) For analysis, the acoustic data is windowed It was divided into segments with a width of 256 samples and a shift width of 64 samples, power was calculated for each window width by the STE method, and segments with an SNR equal to or greater than a predetermined value were detected.

Subsequently, body sound determination (step S3 in FIG. 3) was performed by human auditory evaluation to determine whether each segment contained BS. The BS included in the data recorded by the non-contact microphone is also included in the data recorded by the electronic stethoscope. The resulting episodes of 20 ms or longer were labeled as BS segments.

Also, two feature quantities, MFCC and PNCC, were extracted from each segment (step S4 in FIG. 3). In this embodiment, calculations were made based on 24-channel gammatone filters in consideration of frequency bands for each of MFCC and PNCC. MFCC and PNCC were calculated for each frame by dividing the segment by frame size: 200 and shift size: 100. Therefore, in each segment, the averaged 13-dimensional MFCC and 13-dimensional PNCC were used as features.

As described above, the BS segment and the non-BS segment, and the feature amount of each segment were acquired from the acoustic data of 20 persons. 3/4 of these segments were used as training data, and the remaining 1/4 were used as evaluation data.

In learning the prediction algorithm, an artificial neural network (ANN) with 13, 25, and 1 units in the input layer, intermediate layer, and output layer, respectively, was used. The output function of the hidden layer unit was a hyperbolic tangent function and the transfer function of the output layer unit was a linear function. As a teacher signal, 1 is given if the segment to be learned is a BS segment, and 0 is given if it is a non-BS segment, and the ANN is learned by the error backpropagation method based on the Levenberg-Marquardt method to create a prediction algorithm. In addition to the error backpropagation method, the elastic backpropagation method or the like can be used as the learning algorithm. For example, softmax or the like can be used as the output function of the units in the intermediate layer and the output layer.

For the learning and evaluation of the prediction algorithm, (1) initial values of connection weights were randomly given, and (2) learning data and evaluation data were randomly given, and trials were repeated multiple times. From this, the average prediction accuracy of the prediction algorithm was calculated.

In order to evaluate gut motility from the acoustic data of one subject, we extracted the above two acoustic features from multiple segments automatically extracted using the predictive algorithm through leave-one-out cross-validation. Wilcoxon's signed rank sum test was then used to evaluate the difference in these acoustic features before and after the subject ingested the carbonated water.

In the present example, the influence of a predetermined value (reference value) of SNR, which is a reference for detecting segments from acoustic data in the STE method, on the prediction accuracy of the prediction algorithm and the evaluation of intestinal motility was investigated. For the investigation, the predetermined value of SNR was varied from 0, 0.5, 1, 2 dB. Table 1 shows the number and length of BS segments and non-BS segments obtained for each reference value.

From Table 1, both before and after ingestion of carbonated water, the number of Non-BS segments increases with a decrease in the reference value, but the number of BS segments before ingestion of carbonated water reaches a certain reference value. It was found that the number of BS segments after ingestion of carbonated water tended to decrease as the reference value decreased. Also, it can be confirmed that both the BS segment and the Non-BS segment increase as the reference value decreases. In addition, the number and length of BS segments and the number of Non-BS segments are greater after ingestion than before ingestion of carbonated water, and the length of Non-BS segments is greater after ingestion than before ingestion of carbonated water. smaller.

According to Table 2, when the feature amount is MFCC, it was found that the accuracy deteriorated as the reference value decreased before ingestion of carbonated water. On the other hand, after ingesting carbonated water, it was found that the accuracy generally increased as the reference value decreased. According to Table 3, when the feature value is PNCC, both before and after ingestion of carbonated water, the accuracy increases as the reference value decreases, and the highest accuracy is obtained when the reference value is 0 dB. .

FIG. 8 is a graph showing the prediction accuracy (Acc) for each SNR reference value when the feature amount is MFCC and PNCC, (a) is the graph before carbonated water intake, (b) is the carbonated water intake Later graph. From FIG. 8, it can be seen that the accuracy when using PNCC is higher than the accuracy when using MFCC at all reference values of SNR. In particular, when the reference value of SNR is 0 dB, the standard deviation of PNCC before ingesting carbonated water is smaller than the standard deviation of MFCC, and the average value of PNCC is sufficiently higher than the average value of MFCC. In general, before ingestion of carbonated water, many BS with low sound pressure are generated compared to after ingestion, so PNCC is particularly effective as a feature amount for predicting whether BS is included. I found out.

[Example 2]
In this embodiment, the PNCC, which was found to be particularly effective in Example 1, is used as a feature quantity to learn a prediction algorithm in the same manner as in Example 1, and intestinal peristaltic sounds (BS) are included in the acoustic data obtained by the learned prediction algorithm. It was verified whether it is possible to predict whether or not the bowel motility is present and to evaluate gut motility based on the extracted acoustic data.

In order to evaluate the prediction accuracy of whether or not BS is included in the acoustic data, in Example 1, evaluation was performed by random sampling, but in this example, evaluation was performed by leave-one-out cross-validation. Specifically, the leave-one-out cross-validation was repeated 50 times for each of the 20 subjects, and the average classification accuracy with the highest accuracy was calculated for each subject. Table 4 shows the results.

In addition, the subject's intestinal motility was assessed based on the BS segment extracted by the predictive algorithm. Specifically, from the BS segment, the number of BS occurrences per minute, SNR, BS length, and BS occurrence interval were detected as indices for evaluating intestinal motility. We captured differences in intestinal motility. Table 5 shows the number of BS occurrences per minute and SNR, and Table 6 shows the BS length and BS occurrence interval.

From Tables 5 and 6, it was found that the difference in intestinal motility before and after ingestion of carbonated water can be captured even when the reference value of SNR is lowered to 0 dB. Note that this result is related to the BS segment extraction accuracy. From the above, if the SNR reference value changes to 0 dB, the number of BS occurrences per minute (the number of BS segments per minute), the SNR, and the BS occurrence interval are It was suggested that it is an indicator that is not affected by

In addition, it is known that when a subject ingests carbonated water, intestinal motility is strongly enhanced. Therefore, it was suggested that the prediction algorithm according to the present invention is useful for evaluation and monitoring of intestinal diseases, etc., in which intestinal motility is considered to be strongly enhanced compared to healthy subjects.

[Example 3]
In the present embodiment, a prediction algorithm is learned using PNCC as a feature amount, prediction is made as to whether or not intestinal peristaltic sound (BS) is included in acoustic data by the learned prediction algorithm, and intestinal movement based on the extracted acoustic data. It was confirmed that sex assessment, in particular discrimination of irritable bowel syndrome (IBS), is possible.

First, as a preliminary verification, BS segments containing intestinal peristaltic sounds (BS) were manually extracted from the acoustic data acquired from 48 IBS and non-IBS subjects, and the BS segments were analyzed to obtain IBS and Indices for identifying non-IBS were investigated.

Specifically, 48 male subjects (IBS: 23 (age: 22.2 ± 1.43, BMI: 22.1 ± 3.39), non-IBS: 25 ( Age: 22.7±3.32, BMI: 21.6±3.69)) was subjected to a carbonated water tolerance test (STT). Subjects were classified as IBS or non-IBS based on Rome III diagnostic criteria. The content of the STT was the same as in Example 1, and the subjects were asked to ingest carbonated water after fasting for 12 hours or more on the previous day. STT was performed at rest for 15 minutes after water intake. However, subjects who complained of abdominal pain or discomfort on the day of the experiment were excluded. A non-contact microphone (NT55 manufactured by RODE), an electronic stethoscope (E-Scope2 manufactured by Cardionics (48 people), and an audio interface (R16 (34 people) and R24 (14 people) manufactured by ZOOM) were used as sound collectors. Acoustic data was simultaneously recorded at a sampling frequency of 44100 Hz and a digital resolution of 16 bits.During the experiment, the subject was in the supine position, and an electronic stethoscope was placed 9 cm to the right of the navel. , A non-contact microphone was placed 20 cm above the navel, and the acoustic data was down-sampled to 4000 Hz in consideration of the frequency characteristics of the generally known intestinal peristaltic sound (BS). .

In the manual extraction of BS segments from non-contact microphone acoustic data, BS detection was performed by referencing the bandwidth of ARMA spectral peaks obtained from electronic stethoscope recordings. As a result, since the time period during which the BS was generated can be known, both recorded data were viewed and listened to on the audio reproduction software, and the auditory evaluation was carried out with reference to this. The extracted BS segments are classified into an IBS group and a non-IBS group, and two indicators, the number of BS occurrences per minute and the BS occurrence interval, are detected from the BS as indices for distinguishing IBS/non-IBS. Then, the average value of each index was calculated every 5 minutes for 25 minutes during which the STT was performed. Wilcoxon's signed rank sum test was then used to verify whether there was a significant difference in each index between the IBS group and the non-IBS group.

FIGS. 9A and 9B show time transitions of two indices calculated in the preliminary verification. 0 to 10 minutes before ingestion of carbonated water, and 10 to 25 minutes after ingestion of carbonated water. From FIG. 9, there is a trend that there is a significant difference between the IBS group and the non-IBS group in the number of BS occurrences per minute and the BS occurrence interval in the interval of 20-25 minutes (10-15 minutes after ingestion of carbonated water). was confirmed. When the ARMA-based approach was used to estimate the number of BS occurrences per minute for electronic stethoscope recording data, a significant difference was found between the IBS group and the non-IBS group. I couldn't. This result emphasizes the remarkable utility of non-contact microphone recordings that can acquire characteristic BS among BS included in electronic stethoscope recordings.

Subsequently, in this example, acoustic data recorded using the same audio interface (R16) was picked from among the subjects in the preliminary verification, and BS segments were extracted from the acoustic data by a prediction algorithm. Then, based on the extracted BS segment, whether or not the subject had IBS was identified, and its accuracy was verified. The prediction algorithm was created by machine learning by ANN using PNCC as a feature amount. The number of units in the input layer, hidden layer and output layer of ANN was 8-28, 40 and 1, respectively. Also, in the segment detection by the STE method in creating the teacher data, the SNR reference value was set to 0 dB.

Specifically, 34 male subjects (IBS: 18 (age: 23.1 ± 3.84, BMI: 21.9 ± 4.07), non-IBS: 16 ( Age: 22.3±1.69, BMI: 23.1±3.61)), STT was performed in the same manner as in the preliminary verification, and acoustic data was obtained from the subject by the same method as in the preliminary verification. First, the acoustic data and the acquired acoustic data are divided into subsegments: 256 and overlap: 128. Using the STE method, segments with an SNR of 0 dB or more are detected, and a prediction algorithm is used to We examined whether the BS segment could be extracted. In this embodiment, each segment is divided by frame size: 200, overlap: 100, and 20-dimensional PNCC and 20-dimensional MFCC are calculated for each frame. After that, the average value of 20-dimensional MFCC in each segment, the average value of 20-dimensional PNCC and the standard deviation of 20-dimensional PNCC, the feature amount of this embodiment: each standard deviation of BSF1, BSF2, BSF3 and BSF4, each average calculated the value. The number of units in the input layer, intermediate layer and output layer of the ANN is as described above (8 to 28, 40 and 1, respectively), and the extraction performance was evaluated by leave one out cross-validation. Table 7 shows the evaluation results.

Thus, by using the expanded sound database (20 to 34 names), we found that using the PNCC average (20 dimensions) and using the MFCC average (20 dimensions) It became clear that remarkably high BS detection performance can be obtained. From this, it was confirmed that PNCC is more effective than MFCC in detecting BS even when the sampling rate is low as in this embodiment. In addition, the feature amount of the present embodiment: BSF1, BSF2, BSF3 and BSF4 statistics (total: 8 dimensions) only by using the feature amount, BS higher than when using the average value of PNCC (20 dimensions) It was confirmed that detection performance can be obtained. This clearly indicates that these four feature quantities are effective for BS detection. Furthermore, it was found that combining the PNCC statistic (standard deviation) with the BSF1, BSF2, BSF3 and BSF4 statistics improved the performance.

In preliminary verification, for example, the number of BS occurrences per minute and the BS occurrence interval in the 20-25 minute interval of STT (10-15 minutes after ingestion of carbonated water) were significant between the IBS group and the non-IBS group. A difference was observed. On the other hand, in order to confirm whether the BS segment extracted by the prediction algorithm that combines the standard deviation of PNCC and the statistics (mean value and standard deviation) of BSF1, BSF2, BSF3, and BSF4 has the same tendency, We estimated the number of BS segments per minute in the 20-25 min interval of the STT (10-15 min after carbonated water ingestion). Table 8 shows the results.

The results shown in Tables 7 and 8 show that the predictive algorithm can be used to extract the BS segment with an average sensitivity of 88.6% after ingestion of carbonated water. Then, based on the extracted BS segments, the number of BS segments per minute in the interval of 10 to 15 minutes after ingestion of carbonated water for each of the IBS and non-IBS groups was calculated. A significant difference was observed. From the above, it was found that even in the case of BS segments extracted using a prediction algorithm, IBS and non-IBS can be distinguished in the same way as in manually extracted BS segments. It should be noted that even when only the statistics (mean and standard deviation) of BSF1, BSF2, BSF3, and BSF4 were used, there was a significant difference between IBS and non-IBS based on the number of BS segments per minute ( Note that P<0.05) was found.

It should be noted that BS is acquired synchronously in a sensor capable of simultaneous recording of a stethoscope and a non-contact microphone. When detecting BS segments from non-contact microphone recordings under conditions that are noisier than this environment, BS is detected from non-contact microphone recordings by referring to the estimated BS from the stethoscope recording data. performance can be improved.

[Example 4]
In this example, using a non-contact microphone, recorded from 5 subjects, (i) 5 minutes of recording data after ingestion of carbonated water, (ii) 5 minutes of recording data after ingestion of coffee Manual labeling BS was extracted by the method, and the types of BS were classified into the following five patterns P1 to P5.
P1: Extremely short BS of about 50 ms or less (eg, sound like bursting bubbles).
P2: Rumbling and gurgling BS that occurs with the movement of liquid, and generally no significant change is observed on the spectrogram.
P3: Sounds such as guru, goro, guru, guu, which are similar to P2 and tend to have a shorter BS length than P2.
P4: The sound is like goo, gyu, coo, and has a spectral structure similar to the snoring sound of simple snoring.
P5: A pattern in which a sound similar to P4 changes relatively greatly with time, for example, a pattern that shifts to higher frequencies with time, and a pattern in which the shape of the spectrogram clearly changes with time.

For specific classification methods, see Dimoulas, C., Kalliris, G., Papanikolaou, G., Petridis, V., & Kalampakas, A. (2008). Bowel-sound pattern analysis using wavelets and neural networks with application to long-term, unsupervised, gastrointestinal motility monitoring. See Expert Systems with Applications, 34(1), 26-41.

FIG. 10 shows (a) the frequency of occurrence of BS patterns after ingestion of carbonated water and (b) the frequency of occurrence of BS patterns after ingestion of coffee. From this figure, it was confirmed that there was a difference in the frequency of occurrence of BS patterns between the two groups. It was confirmed that the BS pattern P1 was clearly seen more frequently after ingestion of coffee than after ingestion of carbonated water. Conversely, it was confirmed that the frequencies of occurrence of patterns P2 and P4 in particular increased after ingestion of carbonated water. These results clearly indicate the difference in the intestinal conditions due to the difference in the components of the drinking water. This suggests that the presence or absence of intestinal disease can be evaluated based on the BS pattern.

[Example 5]
In this example, using a non-contact microphone, recorded from 5 subjects, (i) 5 minutes of recording data after ingestion of carbonated water, (ii) 5 minutes of recording data after ingestion of coffee Manual labeling Automatic classification of BS patterns was performed from a database created by extracting BSs. Note that the pattern P1 described above is a short sound of about 50 ms or less, and is excluded in this embodiment because it can be sufficiently identified only by information on the length of the BS segment. When the intestinal tract performs peristalsis, it is known that BS occurs when air and contents (liquid, etc.) move in the intestinal tract. They are put together as a pattern, and a teacher signal PA1: (0, 1) is given. Similarly, patterns P4 and P5 are put together as an air-dominant BS pattern, and a teacher signal PA2: (1, 0) is given.

The following feature quantities 1 to 3 were used for automatic classification of these BS patterns.
Feature 1: BSF5
Feature quantity 2 (feature quantity of this embodiment): statistics of BSF1, BSF2, BSF3, and BSF4 (mean value and standard deviation)
Feature 3: Feature 2 + BSF5

In training the automatic classification algorithm, ANNs with the number of units in the input layer, hidden layer, and output layer of 1 to 9, 30, and 2, respectively, were used. The ANN was trained by the scaling conjugate gradient method algorithm, the output function of the hidden layer unit was the hyperbolic tangent function, and the transfer function of the output layer unit was the linear function. The database was divided into training data:evaluation data=3:2, and the performance of the classification algorithm was evaluated based on the mean squared error. Table 9 shows the results. Table 9 expresses the minimum mean squared error as a representative value after 300 trials.

When BSF5 is used (feature amount 1), and when the feature amounts of this embodiment: BSF1, BSF2, BSF3, and BSF4 statistics (mean value and standard deviation) are used (feature amount 2), the classification performance is It was confirmed that there was no change. However, it was suggested that when these feature quantities are combined (feature quantity 3), remarkable classification performance can be obtained.

From the above, it is considered that the BSF, which is the feature quantity of the present embodiment, greatly contributes not only to BS detection but also to BS classification. Of course, these ideas are useful not only for non-contact microphone recording data, but also for stethoscope recording data.

[Example 6]
In the present embodiment, (i) feature amounts extracted from an ARMA-based approach that have been conventionally used for BS detection: ψ _k , (ii) feature amounts in the present embodiment: BSF1, BSF2, BSF3, A prediction algorithm was learned using BSF4 and 20-dimensional PNCC, and a comparative study was conducted on the extraction performance of intestinal peristaltic sounds (BS) by the learned prediction algorithm. Acoustic data were also acquired using an electronic stethoscope in a noisy environment. The ARMA-based bowel sound detection method developed by the present inventors in 2013 required obtaining detection results for each sub-segment. Here, the invention was applied to the sub-segments in order to make a performance comparison with the invention. Note that the subsegment length used here is equivalent to the frame length.

In this example, a carbonated water tolerance test (STT) was performed on 10 male subjects who gave their consent to the research contents. The content of the STT was the same as in Example 1, and the subjects were given (i) after fasting for 12 hours or more on the previous day, (ii) immediately after ingesting carbonated water, (iii) within 1 hour after eating, and (iv) immediately after ingesting coffee. In the morning of the day, an electronic stethoscope (E-Scope 2 manufactured by Cardionics) was used as a sound collector, and under the following A to E environments with different noise levels for each subject, Recorded for 1 minute. (That is, from one subject, 4 states (i to iv) x 5 recording environments (A to E) = 20 patterns of recording data are acquired.)
A: Quiet (noise level: about 32 dB)
B: Reading aloud (about 56 dB)
C: Footsteps (about 51 dB)
D: Television (about 55dB)
E: Fan operation (about 52dB)
These noise levels were measured using a sound level meter placed about 1 m away from the subject. Also, note that the noise source was placed at a position about 1 m away from the subject.

Subsequently, body sound determination (step S3 in FIG. 3) to determine whether the sub-segment includes BS was performed by human auditory evaluation in the same manner as in the first embodiment.

Also, in each subsegment, features were extracted using an ARMA-based approach. Specifically, the following processes were performed.

First, the acoustic data is divided by subsegment length: M and overlap: S. The split signal can be expressed as follows.

ここで、ａ、ｂはＡＲＭＡの係数であり、ｗ_ｋ（ｎ）は白色雑音であり、ｐ、ｑはＡＲＭＡの次数である。 In addition, linear trends were removed using least-squares regression analysis on the split signals. The signal was then modeled using an autoregressive moving average (ARMA) model as follows:

where a, b are the ARMA coefficients, w _k (n) is the white noise, and p, q are the ARMA orders.

After the coefficients of the ARMA model were calculated by the Prony method, the power spectrum of the ARMA model was calculated. The Prony method is a method of designing ARMA coefficients based on the impulse response (length l) obtained by the AR(m) model. This power spectrum is generated by filtering the noise variance σ _w with a filter containing poles a and roots b. In addition, prior to computing the power spectrum, we zero-padded the D samples of the ARMA coefficients to improve the amplitude estimate of the spectrum.

A 3 dB bandwidth at the peak frequency was obtained by performing peak picking from the power spectrum of [Formula 9].

BW3 _db is the 3 dB bandwidth at the ARMA spectral peak. If multiple peaks are observed in the spectrum, the narrowest 3 dB bandwidth is used. If BW3 _db cannot be calculated, then BW3 _db =0. ψk is used after being smoothed by a third-order median filter.

ここで、Ｎはトータルサブセグメント数であり、ｓ（ｎ）はフィルタ処理された信号である。 The acoustic data were also filtered using a 100th order FIR high-pass filter with a cutoff frequency of 80 Hz. However, this cutoff frequency is the frequency at which the normalized gain of the filter is -6 dB. Then, the filtered signal was divided by subsegment length: M and overlap: S. The split signal can be expressed as follows.

where N is the total number of subsegments and s(n) is the filtered signal.

The features extracted from the ARMA-based approach are ψ _k (Equation 6). Feature amount: ψ _k was calculated using the following parameters: M=256, S=128, p=5, q=5, D=1024, m=30, l=4000. Features: BSF1 and BSF2 and BSF3, and 20-dimensional PNCC were used to compare the performance with this approach.

In the learning of the prediction algorithm, the number of units in the input layer and the output layer is 1 (i: ψ _k ), 24 and 1 (ii: in the case of the feature value of this embodiment), and the number of units in the intermediate layer ( H) used an ANN of 40. As a teacher signal, 1 is given if the subsegment to be learned is a BS subsegment, and 0 is given if it is a non-BS subsegment.

In evaluating the prediction accuracy of prediction algorithms, leave one out cross-validation was used to calculate sensitivity, specificity, and PPV. Table 10 shows the results when (i) ψ _k is used as the feature amount and (ii) the feature amount of the invention: BSF1, BSF2, BSF3, and 20-dimensional PNCC. Here, since the PNCC is used for the subsegment, the mel filter bank was used as the filter bank, and the power bias subtracushion processing of the PNCC was not performed.

The results confirm that the BS detection performance of the ARMA-based approach degrades, as expected, if the acoustic data is acquired in a noisy environment. On the other hand, by using the features of this example as features: BSF1, BSF2, BSF3, and prediction algorithms trained using 20-dimensional PNCCs, much higher detection performance than the ARMA-based approach can be obtained. was confirmed. In addition, the correlation between the number of BS subsegments for each recording data extracted from 200 patterns of recording data of 10 subjects by manual labeling and the number of BS subsegments for each recording data estimated by the prediction algorithm of this embodiment. As a result of calculating the correlation coefficient, a high correlation of R=0.9272 was confirmed. Here, the features of this example were used for the sub-segments to compare BS detection performance with using an ARMA-based approach. Even in the case of stethoscope recording data, further performance improvement is expected by segmenting and extracting BSF1, BSF2, BSF3, and 20-dimensional PNCC statistics.

Note that this technology has been developed with the aim of detecting BSs even in an environment where the SNR is low. It should be noted that the above-described embodiments were also intended to detect very soft BS.

1 Machine learning device 1' Machine learning device 2 Analysis device 2' Analysis device 3 Input device 4 Sound collector 6 Display device 7 Subject 11 Auxiliary storage device 12 Teacher data creation unit 12' Teacher data creation unit 13 Learning unit 22 Acoustic data Acquisition unit 23 Segment detection unit 24 Feature amount extraction unit 25 Body sound prediction unit 26 Body sound segment extraction unit 27 State evaluation unit (first state evaluation unit)
28 classification prediction unit 29 state evaluation unit (second state evaluation unit)
51 Auxiliary storage device 100 Diagnosis support system 121 Acoustic data acquisition unit 122 Segment detection unit 122 Body sound determination unit 123 Body sound determination unit 124 Feature amount extraction unit 125 Classification determination unit D1 Teacher data D1' Teacher data D2 Prediction algorithm D2' Prediction algorithm

Claims (15)

A machine learning device that learns a prediction algorithm for predicting whether bowel sounds are included in acoustic data,
an acoustic data acquisition unit that acquires acoustic data obtained from a subject by a sound collector;
a body sound determination unit that determines whether or not the bowel sounds are included in the acoustic data according to a user's operation;
a feature quantity extraction unit for extracting a feature quantity in the acoustic data;
a learning unit that learns the prediction algorithm based on the determination result of the body sound determination unit and the feature amount;
with
The machine learning device, wherein the features include at least one of PNCC, MFCC, ΔPNCC, ΔΔPNCC, ΔMFCC, ΔΔMFCC, and LPC coefficients , and statistics thereof. 2. The machine learning device according to claim 1 , wherein said learning unit is composed of an artificial neural network (ANN). 3. The machine learning device according to claim 1, wherein said sound collecting device is a non-contact microphone. further comprising a segment detection unit that detects a plurality of segments from the acoustic data acquired by the acoustic data acquisition unit;
The body sound determination unit determines whether or not each segment includes the intestinal sound according to a user's operation,
The feature quantity extraction unit extracts a feature quantity in each segment,
4. The machine learning device according to any one of claims 1 to 3 , wherein said learning unit learns said prediction algorithm based on said feature amount in each segment and the result of determination by said body sound determination unit. 5. The machine learning device according to claim 4 , wherein said segment detection unit detects a segment having an SNR equal to or greater than a predetermined value. further comprising a classification determination unit that determines a type of the bowel sound according to a user's operation when the bowel sound is included in the acoustic data,
The machine learning device according to any one of claims 1 to 5 , wherein said learning unit learns said prediction algorithm further based on the type of bowel sound . An analysis device for analyzing acoustic data obtained from a subject by a sound collector,
An analysis device comprising a body sound prediction unit that predicts whether the acoustic data includes bowel sounds according to a prediction algorithm learned by the machine learning device according to any one of claims 1 to 6 . a body sound segment extraction unit that extracts a segment containing the bowel sound from the acoustic data based on the prediction result of the body sound prediction unit;
a first state evaluation unit that evaluates the intestinal motility of the subject based on the segments extracted by the body sound segment extraction unit;
The analysis device according to claim 7 , further comprising: The first condition evaluation unit evaluates the intestinal motility using the number of bowel sounds generated per predetermined time, the SNR of the bowel sounds, the length of the bowel sounds, and the intervals at which the bowel sounds are generated as indices. 9. The analysis device according to claim 8. The prediction algorithm is a prediction algorithm learned by the machine learning device according to claim 6 ,
10. The method according to any one of claims 7 to 9 , further comprising a classification prediction unit that predicts the type of bowel sounds according to the prediction algorithm when it is predicted that the bowel sounds are included in the acoustic data. analysis equipment. 11. The analysis apparatus according to claim 10 , further comprising a second state evaluation unit that evaluates whether the subject has an intestinal disease based on the type of bowel sound predicted by the classification prediction unit. A machine learning method for learning a prediction algorithm that predicts whether bowel sounds are included in acoustic data, comprising:
an acoustic data acquisition step of acquiring acoustic data obtained from a subject by a sound collector;
a body sound determination step of determining whether or not the bowel sounds are included in the acoustic data according to a user's operation;
a feature quantity extraction step of extracting a feature quantity in the acoustic data;
a learning step of learning the prediction algorithm based on the determination result of the body sound determination step and the feature quantity;
with
The machine learning method, wherein the features include at least one of PNCC, MFCC, ΔPNCC, ΔΔPNCC, ΔMFCC, ΔΔMFCC, and LPC coefficients , and statistics thereof. An analysis method in which a computer analyzes acoustic data obtained from a subject by a sound collector,
An analysis method comprising a prediction step of predicting whether bowel sounds are included in the acoustic data according to a prediction algorithm learned by the machine learning method according to claim 12. a body sound segment extraction step of extracting a segment containing the bowel sound from the acoustic data based on the prediction result of the prediction step;
a state evaluation step of evaluating intestinal motility of the subject based on the segment extracted by the body sound segment extraction step;
The analysis method according to claim 13 , further comprising: In the state evaluation step, the intestinal motility is evaluated using the number of bowel sounds generated per predetermined time, the SNR of the bowel sounds, the length of the bowel sounds, and the interval between bowel sounds as indicators. Item 15. The analysis method according to Item 14.

Images