Music Generation with Relation Join

Given a data set taken over a population, the question of how can we construct possible explanatory models for the interactions and dependencies in the population is a discovery question. Projection and Relation Join is a way of addressing this question in a non-deterministic context with mathematical relations. In this paper, we apply projection and relation join to music harmonic sequences to generate new sequences in a given composer or genre style. Instead of first learning the patterns, and then making replications as early music generation work did, we introduce a completely new data driven methodology to generate music. Then we discuss exploring the difference between the original music and synthetic music sequences using information theory based techniques.

1. 1.

   Music 21 is a toolkit for computer-aided musicology. See http://web.mit.edu/music21/.

Authors and Affiliations

1. Computer Science Department, The Graduate Center, City University of New York, New York, NY, 10016, USA

   Xiuyan Ni, Ligon Liu & Robert Haralick

Corresponding author

Correspondence to Robert Haralick .

Editors and Affiliations

1. Laboratoire PRISM, CNRS-AMU, Marseille, France

   Mitsuko Aramaki

2. Laboratoire PRISM, CNRS-AMU, Marseille, France

   Richard Kronland-Martinet

3. Laboratoire PRISM, CNRS-AMU, Marseille, France

   Sølvi Ystad

Appendix A.1: Proofs of Distance Metrics

The proofs show that the four distance metrics we use in this paper satisfies the conditions:

1. (A)

   Identity. \(d(X, Y) = 0\) if and only if \(X = Y\)

2. (B)

   Symmetry. \(d(X, Y) = d(Y, X)\)

3. (C)

   Triangle inequality. \( d(X, Z) + d(Z, Y) \ge d(X, Y) \)

Proof

(A) and (B) are straightforward for all the four metrics give the properties of the entropy and mutual information. To prove the triangle inequality (C), we can first assume \(H(X) \ge H(Y)\). Since d(X, Y) is symmetric, the reverse case can be proved easily if we can prove one of the two cases (\(H(X) \ge H(Y)\) or \(H(Y) \ge H(X)\)).

So we have only three cases left to consider: \(H(X) \ge H(Y) \ge H(Z)\), \(H(X) \ge H(Z) \ge H(Y)\), and \(H(Z) \ge H(X) \ge H(Y)\).

• Case \(H(X) \ge H(Y) \ge H(Z)\):

  • For \(d_1 (X, Y)\), first, we know that

    $$\begin{aligned} \begin{aligned}d(X,Y)_1&=H(X,Y)-I(X;Y)\\&=H(X)+H(Y)-2I(X;Y)\\ {}&=H(X|Y)+H(Y|X). \end{aligned} \end{aligned}$$

    then

    $$\begin{aligned} \begin{aligned} H(X|Z)+H(Z|Y)&\ge H(X|Y,Z) +H(Z|Y) = H(X, Z|Y) \ge H(X|Y), \\ \end{aligned} \end{aligned}$$

    so

    $$\begin{aligned} \begin{aligned}d_1(X,Z) + d_1(Z, Y)&=H(X|Z)+H(Z|X) + H(Y|Z)+H(Z|Y)\\&\ge H(X|Y) + H(Z|X) + H(Y|Z) \\&\ge H(X|Y) + H(Y|X) = d_1(X, Y). \end{aligned} \end{aligned}$$

  • For \(d'_1 (X, Y) = 1 - \frac{I(X;Y)}{max \{H(X), H(Y) \}}\),

    $$\begin{aligned} \begin{aligned} d'_1 (X, Y)&= 1 - \frac{I(X;Y)}{max \{H(X), H(Y) \}}\\&= 1 + \frac{H(X, Y) - H(X) - H(Y)}{max \{H(X), H(Y) \}}. \\&= 1 + \frac{H(X, Y) - H(X) - H(Y)}{ H(X)}. \\&= \frac{H(X, Y) - H(Y)}{ H(X)} = \frac{H(X|Y)}{H(X)}. \\ \end{aligned} \end{aligned}$$
    $$\begin{aligned} \begin{aligned} d'_1 (X, Z) + d'_1 (Z, Y)&= 1 + \frac{H(X, Z) - H(X) - H(Z)}{max \{H(X), H(Z) \}} + 1 + \frac{H(Z, Y) - H(Z) - H(Y)}{max \{H(Z), H(Y) \}}\\&= 1 + \frac{H(X, Z) - H(X) - H(Z)}{H(X)} + 1 + \frac{H(Z, Y) - H(Z) - H(Y)}{H(Y)} \\&\ge 1 + \frac{H(X, Z) - H(X) - H(Z)}{H(X)} + 1 + \frac{H(Z, Y) - H(Z) - H(Y)}{H(X)} \\&\ge 1 + \frac{H(X| Z) - H(X)}{H(X)} + 1 + \frac{H(Z| Y) - H(Z)}{H(X)} \\&\ge 1 + 1 +\frac{H(X| Y) - H(X)- H(Z)}{H(X)} \\&\ge 1 + 1 +\frac{H(X| Y) - H(X)- H(X)}{H(X)} \\&\ge 1 +\frac{H(X| Y) - H(X)}{H(X)} = \frac{H(X|Y)}{H(X)}= d'_1(X, Y). \end{aligned} \end{aligned}$$

  • For \(d_2(X, Y)\), we first have \(I(X; Y) = H(X) - H(X |Y) = H(Y) - H(Y|X)\), which means \(H(X|Y) -H(Y|X) = H(X)-H(Y)>0\), so

    $$\begin{aligned} \begin{aligned} d_2(X, Z) + d_2(Z, Y)&= H(X, Z) - min \{H(X), H(Z) \} + H(Z, Y) - min \{H(Z), H(Y) \} \\&= H(X, Z) - H(Z) + H(Z, Y) - H(Z) \\&= H(X | Z) + H(Y | Z) \\&\ge H(X | Z) + H(Z | Y) \\&\ge H(X|Y) \\&= H(X, Y) - H(Y) \\&= H(X, Y) - min \{H(X), H(Y) = d_2(X, Y). \end{aligned} \end{aligned}$$

  • For \(d_3(X, Y)\),

    $$\begin{aligned} \begin{aligned} d_3 (X, Z) + d_3 (Z, Y)&= \frac{H(X, Z) - min \{H(X), H(Z) \}}{max \{H(X), H(Z) \}} + \frac{H(Z, Y) - min \{H(Z), H(Y) \}}{max \{H(Z), H(Y) \}}\\&= \frac{H(X, Z) - H(Z)}{H(X)} + \frac{H(Z, Y) - H(Z)}{H(Y)} \\&= \frac{H(X|Z)}{H(X)} + \frac{H(Y|Z)}{H(Y)} \\&\ge \frac{H(X|Z)}{H(X)} + \frac{H(Z|Y)}{H(Y)} \\&\ge \frac{H(X|Z)}{H(X)} + \frac{H(Z|Y)}{H(X)} \\&\ge \frac{H(X|Y)}{H(X)} = d_3(X, Y). \end{aligned} \end{aligned}$$

• Case \(H(X) \ge H(Z) \ge H(Y)\): Similarly to the case \(H(X) \ge H(Y) \ge H(Z)\).

• Case \(H(Z) \ge H(X) \ge H(Y)\): Similarly to the case \(H(X) \ge H(Y) \ge H(Z)\).

   \(\square \)

Appendix A.2: Harmonic Sequences in Scale C with 80 in Quarterlength Time Duration

(See Figs. 11 and 12).

Synthetic harmonic sequence based on Bach Chorales with the scales fixed at C sample 1

Full size image

Synthetic harmonic sequence based on Bach Chorales with the scales fixed at C sample 2

Full size image

Reprints and permissions

© 2017 Springer International Publishing AG

Policies and ethics