0

I am creating a soft-synth using Qt. That means getting low-latency audio (buffer size around 5-10ms) so that it is responsive to the UI controls.

The components of the soft-synth are:

  • Main window, which handles the UI
  • Synthesis engine, which is a subclass of QIODevice
  • QAudioOutput, which is the Qt Multimedia low latency audio output class

I have based this on my own hacked version of the spectrum example to generate a basic wavetable synth using pull mode.

In practice I find I get glitches if the buffer is less than around 100ms. That's way too long for good responsiveness.

The solution appears to be multithreading, but I'm unclear how to structure this within Qt. My instinct is to put the synth engine into its own thread but I'm unclear

  1. How to do this given that it is a subclass of QIODevice already
  2. Whether this will interfere with QAudioOutput (which is in a thread of its own anyway?)
Paul Masri-Stone
  • 2,843
  • 3
  • 29
  • 51

2 Answers2

4

Qt will be zero help in your endeavor, and quite possibly - obstructive. Low latency audio is a least priority, especially on windows, where it still uses the antiquated MME API. It won't get any better no matter how many threads you throw at it.

I recommend you download and build a 3rd party library, I've had very good results with RTAudio, which supports a number of low latency backends on different platforms. You can still use Qt for the GUI. You can easily wrap RTAudio in a QIODevice if you want.

And yes, it goes without saying that you should not use the main / GUI thread for the synthesis.

dtech
  • 47,916
  • 17
  • 112
  • 190
  • Thanks for the advice. Totally what I didn't want to hear, having invested some time in Qt, but totally useful to hear it at this point before I head down the rabbit hole. – Paul Masri-Stone Nov 27 '15 at 12:11
  • Your time invested in Qt is not lost, as Qt is a big application framework, it just doesn't really do much when it comes down to low latency audio in particular. Building and using RTAudio is easy, as is wrapping it in a `QIODevice` for better compatibility with Qt's design paradigm i.e. signals and slots, even based programming and whatnot. – dtech Nov 27 '15 at 12:17
  • As a follow-up to my original question and this answer, I subclassed `QThread` and put my synth into the thread it creates (using rtAudio for audio output). With the thread priority set to `QThread::TimeCriticalPriority` and my own producer-consumer non-blocking signalling to send control changes to the synth, I have had no problems using buffer lengths down to 3ms across iOS, Windows & MacOS. – Paul Masri-Stone Jun 14 '17 at 12:14
2

Let's outline few more of the problems. As others have said, Qt brings little to the party. Low latency audio is not something its multimedia framework is designed for.

Architecture:

Commercial soft-synths are invariably written as plug-ins for other audio software. The advantage of this approach is that the architecture of the plug-in is quite constrainted and somebody else has solved many of the hard threading and performance problems. Steinberg's VST and Apple's Audio Units being two obvious examples. Both are these are capable of tacking latencies in the order of milliseconds.

Real-time characteristics

When considering real-time systems, the worst-case-latency. In the case of audio, it's duration of the sample buffer in wall-time plus whatever overhead is required to accommodate the worst-case delay in getting scheduled by the operating system to perform and complete the processing.

Threading:

When writing a synthesis plug-in, you are really only concerned with two things:

  • Implementing a render handler - which is called by the host when it needs samples generated by the synth, and which is generally called in a real-time (or at least elevated) thread priority.
  • Implementing an event handler - which is called in a lower priority thread. The complication is that this should cannot cause a priority inversion in the render thread - with which it inevitably shares some data structures. You need lockless data structures for this.

Somewhere you must manage modulation of soft-synth parameters and parameter automation - possibly in another thread.

Naturally, plug-ins often also have UI, but this runs at the lowest priority on the UI thread, and should be totally decoupled from the render handler, via the event handler.

Host applications

If you insist on writing the host application as well, you would be wise to do what almost everyone else in the industry does, and use Steinberg's ASIO layer.

However, you might also checkout Juce - which amongst other things includes a plug-in host using ASIO, example plug-ins, and just about everything else you need to solve your problem.

marko
  • 9,029
  • 4
  • 30
  • 46
  • Thanks for the advice and the extra detail. Thanks to Ross Bencina's excellent article http://www.rossbencina.com/code/real-time-audio-programming-101-time-waits-for-nothing I understood the priority inversion issue and the need to avoid locks/mutexes. I love Qt's GUI and the ability to prototype them using QML & JavaScript but the audio has got to be right so I'm taking your advice and checking out Juce. – Paul Masri-Stone Nov 27 '15 at 12:14
  • Keep in mind Juce is not free for closed source applications, and the cheapest license will cost you 50 bucks a month, that's 600$ annual and you can't pay for less than a full year, if you are willing to pay might as well get the pro license for a one time 1000$ payment. The Projucer is cool, but QML is faster to work with. Juce has many audio specific stuff built in when it comes to IO, GUI and whatnot, which Qt does not, but the Qt APIs are a little better to work with. – dtech Nov 27 '15 at 12:31
  • If the OP wants is developing software commercially, this is an excellent deal - and remember that QT isn't free either for this use. – marko Nov 27 '15 at 15:04