|
1 year ago | |
---|---|---|
.gitignore | 1 year ago | |
Dockerfile | 1 year ago | |
README.md | 1 year ago | |
redilysis.py | 1 year ago | |
requirements.txt | 1 year ago |
Redilysis sends audio analysis to a redis server.
The idea is to share a single audio analysis to many Visual Jockey filters, in our case for lasers.
You need:
Note that it will almost certainly fail on Debian 9/10/11. This is due to mandatory LLVM’s version when compiling the numba library for librosa.
apt install -y --no-install-recommends build-essential gcc pkg-config python-dev portaudio19-dev libsndfile1 pulseaudio
pip3 install librosa numpy pyaudio redis statistics
git clone https://git.interhacker.space/tmplab/redilysis.git
cd redilysis
python redilysis.py --help
You will need to have pulseaudio and a local redis server.
Here is an example. It is a dirty method, but it works :)
docker run -it \
-e NUMBA_CACHE_DIR=/tmp \
-e PULSE_SERVER=unix:${XDG_RUNTIME_DIR}/pulse/native \
-v ${XDG_RUNTIME_DIR}/pulse/native:${XDG_RUNTIME_DIR}/pulse/native \
-v ~/.config/pulse/cookie:/root/.config/pulse/cookie \
--group-add $(getent group audio | cut -d: -f3) \
protonphoton/redilysis:latest \
python redilysis.py -i 172.17.0.1
Two things here:
172.17.0.1
IP address to connect to the redis server. Change according to your needs.Each word in bold and italic is a key which you can query the redis server for.
# Example:
$ redis-cli get spectrum_10
"[2.21, 0.56, 0.51, 0.32, 0.27, 0.21, 0.18, 0.17, 0.18, 0.23]"
rms
C0
and C9
, e.g. between 12Hz and 8,372Hz."0.12"
spectrum_10
C0
and C9
, e.g. between 12Hz and 8,372Hz."[2.21, 0.56, 0.51, 0.32, 0.27, 0.21, 0.18, 0.17, 0.18, 0.23]"
C4
octave is spectrum_10[4]
.0.27
is pretty low meaning almost no audio volume for that octave.C4, D4, D#4, E4, F4, F#4, G4, G#4, A4, A#4, B4
.spectrum_120
C0
and C9
, e.g. between 12Hz and 8,372Hz."[5.55, 2.61, 2.49, 1.79, 2.09, 4.35, 1.99, 1.57, 1.47, 0.77, 0.91, 0.89, 0.85, 0.56, 0.53, 0.73, 0.53, 0.46, 0.43, 0.44, 0.27, 0.45, 0.7, 0.81, 0.98, 0.7, 0.71, 0.6, 0.83, 0.51, 0.32, 0.31, 0.33, 0.24, 0.25, 0.33, 0.39, 0.43, 0.51, 0.28, 0.27, 0.25, 0.38, 0.25, 0.27, 0.3, 0.2, 0.27, 0.35, 0.29, 0.34, 0.3, 0.27, 0.27, 0.22, 0.21, 0.21, 0.29, 0.22, 0.28, 0.18, 0.19, 0.25, 0.26, 0.25, 0.24, 0.2, 0.21, 0.19, 0.18, 0.19, 0.17, 0.2, 0.17, 0.18, 0.17, 0.15, 0.17, 0.19, 0.18, 0.21, 0.16, 0.16, 0.18, 0.15, 0.13, 0.14, 0.16, 0.2, 0.17, 0.17, 0.2, 0.18, 0.16, 0.18, 0.15, 0.15, 0.16, 0.16, 0.19, 0.19, 0.19, 0.17, 0.18, 0.17, 0.19, 0.23, 0.23, 0.2, 0.23, 0.24, 0.36, 0.34, 0.23, 0.22, 0.2, 0.19, 0.18, 0.21, 0.21]"
C2
note is spectrum_10[23]
(12x2 - 1).0.81
is average meaning there is some audio volume for that octave.bpm_sample_interval
"3000.0"
bpm_delay
"3197.49093056"
bpm_sample_interval
is 3.0s, it took 0.19s to analyze and detect beats in the sample.bpm
PEXPIREAT
command.
PTTL
command.
redis-cli PTTL bpm
which results in "3848"
provided bpm_sample_interval is equal to 3000.0
2 * 3000 - 3848 = 2352 milliseconds
ago, plus the TCP transaction time of the redis query."126.05"
beats
"[0.34829932 0.81269841 1.20743764 1.60217687 2.00852608 2.48453515]"
This computation requires the following values
Redis_latency times are not considered here, but could be
Examples are given based on previous values for redis keys.
60 / 126.05 =~ 0.4760
3197.49093056 + 2352 =~ 5549.5
i.e. the capture started 5.5 seconds ago3197.49093056 - (2.48453515*1000) + 2352 =~ 3064.96
i.e. the last beat was 3.1 seconds agoget bpm
query?
(3064.96/1000) / 0.4760 = 6.439
e.g. there were at least 6 beatsf(i){ i * 0.4760 * 1000 - 3064.96 }
where i >= 4f(i) > bpm_sample_interval
Two modes are available (see below for SPECTRUM and BPM), so you might need to run two processes for full analysis.
Here are the commmon parameters for both modes.
Get the help
python redilysis.py -h
Run with debug info
python redilysis.py -v
Get a list of audio devices
python redilysis.py -L
Run with a given audio device
python redilysis.py -v -d 5
Run with a sampling frequency of 0.5s
python redilysis.py -v -s 0.5
Connect to redis on address 192.168.2.20 and port 6379
python redilysis.py -v -i 192.168.2.20 -p 6379
Change the internals of capture: run at 22000Hz with 2200 frames per buffer and 2 channels
python redilysis.py -v -r 22000 -f 2200 -c 2
Choosing the spectrum mode
python redilysis.py -v -m spectrum -s 0.1
This is the default mode.
It performs some frequency analysis (Fast Fourier Transform) to detect “energy” in the human audition bandwidths.
It will record if there is sound and at which frequencies.
It can run at sub-second frequency (100ms) with no problem.
It reports realistic data: spectrum analysis is the easy part.
Choosing the BPM mode
python redilysis.py -v -m bpm -s 3
Choosing a minimum and maximum BPM
python redilysis.py -v -m bpm -s 3 --bpm-min 100 --bpm-max 200
This mode is less sure that the spectrum mode.
It must absolutely run with multiple seconds interval to work well. Three seconds is a correct minimum.
It attempts to detect beats based on audio “jumps” in intensity and energy.
To correct a well-known error called the “octave error” where the detected tempo is twice/half or thrice/third of the real tempo, you can use the Min/Max BPM. When the calculated tempo is outside of the range, it will attempt to find more legitimate values.
Released under the GPLv3 License