Fractal of Periodic Musical Elements
a re-taxonomy of music for machine learning
Introduction
Each generation of composers and music theorists since the late 19th century has looked for unique systems and analytical solutions to successfully create and analyse music. Schenkerian analysis, serial techniques, transformational theory of David Levine1, Tone Clock of Peter Schat2, were just some of the ways music has been looked at in the context of Western art music. In the 1960s, with the advent of voltage-controlled devices, and later digital signal processing, synthesis, and analysis of audio signals became a reality for many composers. For instance, Iannis Xenakis created computer programs to automate his music composition. He sculpted sounds in the time domain with the use of probability density functions3. Even though none of the procedures designed by Xenakis actually achieve this, his dream was to design a method for the reproduction of any audio file with a pre-defined set of controllable parameters4. For the remaining time of the century, composers often used computer programs like Open Music, Max/MSP, and Pure Data to support their music composition processes with algorithms. The output of computer programs such as these (called patchers) was often a musical score, a stream of audio data, midi-messages, etc.
The algorithmic generation of sound has been studied for decades, but since the 2010s it has attracted increasing attention with the resurgence of AI research5. Machine learning processes have been used to generate new music, based on both score, and audio materials as training data input for the neural networks. For instance, score-to-score models have been proposed such as MelodyRNN, DeepBach, PerformanceRNN; audio-to-audio models such as WaveNet, GAN; and score-to-audio models, for instance, PerformanceNET6. While many of them are specialised in the synthesis of only selected types of sounds - for instance only human speech - creators of WaveNet claim that their model is capable of generating any type of audio file based on the provided learning material7. Programming libraries for machine learning such as TensorFlow, Keras, and PyTorch, use data in a form of multi-dimensional matrices or tensors for their input8. The speed of training machine learning models is dependent on, among other specifications, the amount of data provided for the model. In a process of supervised learning - in which material used to train the model is first manually grouped into categories - music encoded into the matrices can be used to train models based on which new similar data can be produced9. What is more, unsupervised learning models can be trained in which a given system is subject to the algorithm automatically finding the system’s characteristics10.
Since 2019, I used Shepard tones - audible illusions of infinitely increasing or decreasing pitch - in my music compositions. During this artistic practice, I observed that it is possible to analyse any score consisting of pitches, rhythms, and dynamics with circular harmonic progressions inspired by the properties of Shepard-tones, and to create a taxonomy of these elements. In my doctoral research, I would like to use this system for composing new music with the support of Artificial Intelligence. To start, I need to prepare this taxonomy of sounds to be operational with machine learning libraries. In this paper, I will introduce a new music notation, synthesized by the taxonomy and understood by machine learning libraries, that allows notating any combination of harmony, rhythm, and dynamics - constituting a musical piece - in a form of multidimensional data matrices. Furthermore, I will present a new music taxonomy, inspired by the circular properties of Shepard-tones, that shapes data from this notation into newly proposed "Periodic Musical Elements". A system formed by these elements in a shape of an infinitely raising fractal, implemented in a new computer software capable of presenting musical progressions in the context of this taxonomy, is a ready solution for further research on training models with supervised and unsupervised machine learning processes.