5

I am attempting to edit a song as the song is playing. So far, I have successfully created a flow that does almost what I desire. I will share how it works and the problem that brought me here for advice.

I take in a song as an array, y, and sample rate, sr. Then, a data feed that provides a new ping every 5 seconds (denoted below as arguments=[]). I use that ping to edit my song during playback. Then, repeat every 5 seconds.

import librosa as lr
y, sr = lr.load(song)
arguments=[] # Populated from external source.
for arg in arguments:
    end += some_interval
    if end > y.size: ... # breaks out of the loop if the song is over.
    x, sr = song_edit(y[front:end],sr, arg)
    sd.play(x, sr, blocking=True)
    front=end

Problem:

  • There is a slight delay that occurs at the end of one 5s clip of music and the next. In music, this is fundamentally problematic.

Solutions I've considered:

  • Since blocking=True in sd.play, the code is stopping while the whole clip is playing. Therefore, the delay must be solvable.
  • I've considered threading or multiprocessing. Is this the best approach here? My understanding is that threading would not work because my argument is not predetermined.
  • What am I missing?

Edit: I tested this:

#x, sr = song_edit(y[front:end],sr, arg)
sd.play(y[front:end], sr, blocking=True)

The delay is still there! Therefore, the delay must be caused by the iteration of the loop or an inherent delay in sd.play.

  • I've read about two methods that could help. CHUNK and callback. Does anyone have context? I don't understand what chunk or callback are. Note: I may switch to pyaudio. – Michael Korn Jun 22 '21 at 15:00

1 Answers1

0

Closer to the answer: typical play(song) functions, as in librosa, pyaudio, sounddevice, and many others are mainly designed for jupyter notebooks and simple implementations. (Forgot source). Each of those libraries I mentioned (I'm sure about pyaudio and sd) have a Stream class, which is more equipped for analysis than simple play functions.

PyAudio: An optional parameter of a Stream object is callback. As the stream resumes, the stream loops through the callback function, which provides the data to the stream.

    # define callback (2)
    def callback(in_data, frame_count, time_info, status):
        data = wf.readframes(frame_count)
        print('We Done it!')
        return (data, pyaudio.paContinue)


    # open stream using callback (3)
    stream = p.open(format=p.get_format_from_width(wf.getsampwidth()),
                    channels=wf.getnchannels(),
                    rate=wf.getframerate(),
                    output=True,
                    stream_callback=callback)

This code is from the PyAudio docs, but I've made one minor change: the print statement within the the callback function. The output of this code (along with some other lines like instantiating the object) will be the audio output + "We Done it!" printed multiple times per second.

I still don't totally understand how callback works, but this helps move me forward on my journey.

Dharman
  • 30,962
  • 25
  • 85
  • 135