Team project for an art commission: process audio using an open source machine learning algorithm, to identify patterns, and control DMX lights based on anomaly score for the sound.
Code in GitHub
The idea behind the project is an art commission to develop a system that uses a degree of artificial intelligence to know if the audio being heard is predictable, repeating, and similar or to a certain extent monotonous.
The measure of the predictability, or the amount of novelty in the sound input, compared to the amount of patterns the system has learned, is expressed as an anomaly score.
Such anomaly is used to control a DMX bus, with a number of applications and possibilities.
The machine learning algorithm we are using is called NuPIC, originated at Numenta, arising from the research done by Jeff Hawkins.
The project is developed in Python, since NuPIC is written in such platform. A commercial DMX USB controller is used, and is working in OSX and Linux.
The Artificial Intelligence (AI) algorithm is used to stream audio frequency data through it, assessing its level of anomaly, which represents a value of a DMX channel.
The process is divided into three working threads that keep processing speed performance.