Hi Adrian, I am interested in helping out with this. I have also been thinking about developing a boost library for use of audio hardware for about a year now. But have not had the time and motivation to get around to it. The general idea of using boost::asio to provide an interface that is consistent with other IO in boost is a very good one if we can achieve the realtime requirements through it (which it sounds like we can from what you have said). I would suggest that you need to support multiple audio "backends" per OS when it comes to device selection. For example, on linux there is a plethora of choices and no one choice is "correct". The user should be given the option to choose which backend(s) to use/support. You will notice that some apps like to provide all options, and others work with only one backend. I was thinking of proposing three related audio API's for boost a while ago. I do believe that to do this properly is a *LOT* of work (which is why I never really started). 1) Audio DSP Module This is for DSP related tasks and takes care of common things like audio format conversion, resampling etc. It is possible to write all of this I think in a platform agnostic way, however some platforms provide helpers for some of these things like MacOSX and maybe we should consider how we could use them or at least integrate with them. Possibly define both a compile time and runtime interface as both have advantages This would be used in the main interface for the play/record API for the pull/push callback Additionally this may expand in the future with all sorts of algorithms defined for audio processing. Though at first I would suggest it be a minimal of what is required to support the other two modules described 2) Audio HW Play/Record Module This is for accessing the play/record of audio on audio hardware. I would suggest as you mentioned both push and pull modes. You can "emulate" one with the other though be it with a lot of work (and some latency) and some backends support both. So we should provide the interface for all backends but export a "capabilities" of the backend indicating what it supports natively (same as we should do for audio format). 3) Audio Mixer Module This is to control mixer settings including things like default device selections, volume controls, mutes, supported formats and device enumeration. Also, it should recieve events indicating external changes to these things. I have designed and implemented an API at work that did a lot of this (though the separation was not so well defined) and supported a number of backends including DirectSound, WASAPI, CoreAudio, portaudio, PulseAudio, OpenSLES. I might spend some time next week to writeup a proposal for a basic outline of what I think an interface could look like. Maybe we can compare ideas and notes? --- snip ---- errcode audio_port::open(audio_format &,const audio_direction, audio_device_mode ). --- snip ---- Could you define what you mean by an audio_port? Do you consider an audio_port to be an audio hardware device or a port on that device? For example: I consider a hadware device as representing a single physical device having one or more ports. Where a port is one of: * Recording port : Normal wave device input records from mic/line in etc * Playback port : Normal wave device output plays out to line out/speakers etc * Monitor port : Not always available but is similar to a Recording port, except it returns exactly what is going out of the device speakers (for example may include CD noise or MIDI or WAVE output from other processes). This is sometimes used for doing echo cancellation on input audio that cancels system sounds generated from other applications and not just your application. I also think some professional sound cards may have additional "ports" of type recording/playback. I think the MOTU is like that for example. If you take that definition of a port, then the direction may become un-necessary (at least I haven't seen a case where it would allow bi-directional). As for enumeration of "ports" it makes sense to have a simple function as you described that actually wraps something that is possibly more complex from the "Audio Mixer Module". I guess it *may* look something like: std::listboost::audiohw::port boost::audiohw::get_ports(boost::audiohw::recording_port|boost::audiohw::playback_port|boost::audiohw::monitor_port); What do you propose to be part of the audio_format struct? I can think of: * Channel Count : Integer >= 1 * Data format : float32, int16, ... * Interleaving : yes/no * Sample rate: 48000 * Channel mapping : (have a default defined mapping, but allow user overrides) There should also be the ability to define some form of latency parameters I think. Possibly even a place holder for extensible backend specific configuration (latency details could be part of that). --- snip ---- errcode audio_port::connect(audio_callback &); --- snip ---- This makes sense, would you allow multiple callbacks to be connected to the single open port? For recording ports, I guess this is simple in that it just calls all connected callbacks with the same data For playback ports, would you call multiple and then mix the audio into a block given to the device? If so what are the restrictions on the audio produced by the callbacks? Should they provide exactly as much data as requested? If so, how much data does the callback request in one call? Is it fixed, variable, configurable (latency parameters I mentioned in the config) If they could return less than requested, then the mixing becomes more complex Can you connect callbacks after you have started the device? I would assume that because the audio format has been defined already, then the callback can be verified based on the callback type. I.e. If defining a float32 format, then this could verify the callback data type is a float. Or you could go the C way and use void* but there are advantages to the type safety I guess a proposal for the callback could be: result on_audio_cb(float* data_out, size_t data_out_size, const audiohw::format& audio_format); How do you plan to handle synchronous input/output audio? What I mean by this is that the consumption and production of audio is synchronized so when you get 20msec of audio from the input you must also output 20msec. This has benefits to various audio algorithms, and can be achieved across different devices with special clock monitoring and resampling techniques. It is generally the case for input/output on the same physical audio hardware that they are synchronized, but is not across different hardware as often the audio card clocks can be un-synchronized. This decision can affect how to structure the open() call. For example, you may open both input/output at the same time if you want to support this. There are a few options the first is my preferred though is more difficult to use: * You can define for synchronous handling, that the record/playback/monitor callbacks will always be called in order * You could do what portaudio does and define the callback to have (but that doesn't work well with the previous design) result on_audio_cb(float* data_out, const float* data_in, size_t data_size, const audiohw::format& audio_out_format, const audiohw::format& audio_in_format); How do you plan to handle asynchronous errors in the audio device? Maybe an error code passed to the on_audio_cb() or a seperate callback that can be registered for errors? On Wednesday, 24 April 2013, adrien courdavault wrote:
Hello.
I make this new thread to be clearer. There is currently no way to manage audio endpoints cconnection easily. It looks like some people might find this usefull (as I do), and I've been suggested on the boost dev list to try to detail this, as an extension to Boost.ASIO.
For this reason I create this thread.
I'm trying to make a very basic first draft of the concepts and see if this may be a good idea.
I attached here the first things I've written. This is very short and general.
I would like to know: * do you think I'm going in the right direction by seing this as Boost.ASIO extension. * do you have suggestions * would someone like to participate ?
Thank you