How do I get Processing to recieve input from the audio from my computer?

Question

I am having a hard time finding a solution to this on the web. Basically I am trying to create my own audio visualizer for the music I play on my computer, and would like Processing to analyze the audio from iTunes (or audio output from my computer) so I can jam out while I code.

I have looked into the Sound Library which Processing offers, which is a great and easy way to load a sound file into your sketch and play it back, as well as receive input from the microphone on your computer, but I see no way for this Library to receive input from iTunes (or any other media player). The documentation doesn't help very much. It would be much easier to build my visualizer if I could intercept the audio output from my computer, apposed to loading a song into my sketch and having to analyze the same song over and over...

Any suggestions?

p.s Yes I know iTunes has a built in visualizer. I want to make my own.

You will need to open and listen to a system audio port. Take a look at the [`AudioSystem` class tutorial](https://docs.oracle.com/javase/tutorial/sound/accessing.html). You can sniff the audio from it and process it real-time to build your visualizer. — Cᴏʀʏ, Aug 18 '15 at 19:53
FYI you don't want to program your app to listen to audio from specific programs, rather, you program it to read the sound right off the audio bus -- that way *anything* currently playing from anywhere will be captured. The hardest part will be determining which is the current mixer and output device, and then writing a background thread that can pipe that data to your visualization process. — Cᴏʀʏ, Aug 18 '15 at 20:19

George Profenza · Answer 1 · 2021-03-14T15:00:51.987

If you're using a newer version of Processing, have a look at the FFT example:

import processing.sound.*;
FFT fft;
AudioIn in;
int bands = 512;
float[] spectrum = new float[bands];

void setup() {
  size(512, 360);
  background(255);
    
  // Create an Input stream which is routed into the Amplitude analyzer
  fft = new FFT(this);
  in = new AudioIn(this, 0);
  
  // start the Audio Input
  in.start();
  
  // patch the AudioIn
  fft.input(in);
}      

void draw() { 
  background(255);
  fft.analyze(spectrum);

  for(int i = 0; i < bands; i++){
  // The result of the FFT is normalized
  // draw the line for frequency band i scaling it up by 5 to get more amplitude.
  line( i, height, i, height - spectrum[i]*height*5 );
  } 
}

Processing 2 and earlier ship with the Minim library. Have a look at the Drawing a Frequency Spectrum section. Also, for further Processing you may want to check out this post.

On routing the audio playing as input, you can have a look at SoundFlower or JACK. Either should allow you to route the system audio as an input.

Also, since you've mentioned iTunes, here's an Audio-driven landscape by Robert Hodgin, creator of the iTunes Visualiser (among many other awesome things).

This code example has some errors. Please check the FFT Example link to see the correct code example. — Mr Pablo, Mar 14 '21 at 13:23
Thank you @MrPablo ! I didn't notice the signature for the `input()` method changed and it no longer access the number of bands. I've updated the snippet. — George Profenza, Mar 14 '21 at 15:02

How do I get Processing to recieve input from the audio from my computer?

1 Answers1