How to create a simple driver for microphone input processing?

[tembun]

I have quite some experience with C programming, but I'm completely new to writing kernel or device drivers and I want to learn that. I have an idea to create a sort of 'microphone filter', which will receive the input from the microphone, then process it somehow (for example, increase the volume, etc) and then put the output as the output of the regular microphone device, so that all the programs that want to capture the input from the microphone, would get the sound, modified by the driver (the 'filter').

Now, as I already said, I'm completely new to all this and for now it's not very clear to me how it should be implemented. I guess I need to create a character device like /dev/dsp that I have, that will act as a mic? And maybe virtual_oss(8) should be utilized somehow?

So, I'd like to ask some help and advice from people who are experienced in things like that or similar subjects. Could you please help me to understand the overall process and explain how this functionality could be implemented (i.e. how can I 'capture' the microphone input and make the rest of the system 'think' that my output is the microphone output)? I'm not expecting to write the entire thing in the first place, my goal is to better understand the process in detail and, as a side effect, try to implement this model and practice more low-level programming.

Thank you!

 

[cracauer@]

Normally you would do this in userspace, e.g. as a jacl module.

The problem with doing these things in the kernel (apart from the crash risk) is that configuring such filters becomes a pain in the allerwertesten.

 

[tembun]

Normally you would do this in userspace

Alright, thanks. But does the overall idea stay the same: my program should create a character device under /dev (replace /dev/dsp?)? It's still not very clear to me. I know that I can read the raw data from /dev/dsp, so I guess I can create some device under /dev, that would read from /dev/dsp, process it, and produce output. In this way I would have two devices: 1) /dev/dsp for the 'normal' mic, and 2) my device, for the 'transformed' mic. But how can I make the 'transformed' mic act like it it the 'normal' (default) mic, maybe just temporarily, while my program is active? Can I just simply replace existing /dev/dsp with my device?

 

[cracauer@]

You probably want to start out studying the in-kernel mixer if you really want to stay in kernel space and use that as a template.

 

[Zare]

I guess I need to create a character device like /dev/dsp that I have, that will act as a mic?

Yes.
cuse(3)() is used to create DSP character devices from userspace.

The OSS sound system under FreeBSD exposes play/rec devices through /dev/dsp entries which are standard Unix character devices, hence standard Unix file stuff is used to either play or record.

Normal userspace program (as root) can do the same via cuse library.

http://manuals.opensound.com/developer/audiolevel.c.html . This is a sample for recording input for 4Front's OSS (I think ours is @ sys/soundcard.h) but that's basically it.

https://github.com/hselasky/virtual_oss/blob/main/virtual_ctl.c . Here you can see and explore further how virtual_oss handles character devices from userspace.

That's the input and output you're asking about? Put your processing in between, and select a different audio input (your cuse'd virtual dev) in the receiving program.

 

[tembun]

You probably want to start out studying the in-kernel mixer if you really want to stay in kernel space and use that as a template.

Hah, you know, I got the very idea about writing such program because in the last few days I've been exploring and debugging the mixer subsystem, which includes mixer(8) -> mixer(3) (libmixer) -> /usr/src/sys/dev/sound/pcm/mixer.c (generic mixer driver) -> snd_hda(4) (that's my hardware driver). I was just curious how values got set and things like that. I liked that and decided to go a bit further.

 

[tembun]

Yes.
cuse(3)() is used to create DSP character devices from userspace.

The OSS sound system under FreeBSD exposes play/rec devices through /dev/dsp entries which are standard Unix character devices, hence standard Unix file stuff is used to either play or record.

Normal userspace program (as root) can do the same via cuse library.

http://manuals.opensound.com/developer/audiolevel.c.html . This is a sample for recording input for 4Front's OSS (I think ours is @ sys/soundcard.h) but that's basically it.

https://github.com/hselasky/virtual_oss/blob/main/virtual_ctl.c . Here you can see and explore further how virtual_oss handles character devices from userspace.

That's the input and output you're asking about? Put your processing in between, and select a different audio input (your cuse'd virtual dev) in the receiving program.

Thank you very much for the references and the explanation! I will explore!

 

[kent_dorfman766]

While it is not relevant to BSD, you should probably review the codebase and WIKIs for Linux ALSA. The plugin filter architecture is intrinsic in that tool.

 

[Zare]

Normally you would do this in userspace, e.g. as a jacl module.

What OP describes is a typical task of a LV2 audio plugin.
But then applications are limited to LV2 hosts and Jack clients. In order to connect that to standard audio programs again virtual_oss is needed to provide a virtual cuse device that Jack is going to output to.

On the other hand if the microphone filter program creates a virtual device on its own, jack can connect to it.

 

[cracauer@]

What OP describes is a typical task of a LV2 audio plugin.
But then applications are limited to LV2 hosts and Jack clients. In order to connect that to standard audio programs again virtual_oss is needed to provide a virtual cuse device that Jack is going to output to.

On the other hand if the microphone filter program creates a virtual device on its own, jack can connect to it.

Well, at least itis trivial to route all output through a jack pluin or a plugin container such as a DAW.. For macOS you have to buy special software to do it. Dunno about winnoze.

Given that I don't see the need to go kernel.

 

[unitrunker]

tembun is the filter relatively simple like a FIR or IIR filter?

Seems like you could insert your extra processing inline to the existing flow with an ioctl to control the extra filtering. I don't know if the OSS normalizes all samples to some internal standard (like 24 or 32 bit). If so, that would streamline your work.

 

[Zare]

Well, at least itis trivial to route all output through a jack pluin or a plugin container such as a DAW.. For macOS you have to buy special software to do it. Dunno about winnoze.

I don't know Mac. On Windows, WASAPI can be used to instantiate vdevs from userspace. And then that vdev can be connected to the audio server via tooling, e.g. for ASIO systems, ASIO4ALL driver.

Windows is quite OK in this regard and apart from Windows-based development environments being more complex and convoluted than Unix ones, it is the same approach. Windows doesn't get flak in audio/DAW context because of its audio tech but because the "rest of the OS" tends to be what it is.

tembun is the filter relatively simple like a FIR or IIR filter?

Seems like you could insert your extra processing inline to the existing flow with an ioctl to control the extra filtering. I don't know if the OSS normalizes all samples to some internal standard (like 24 or 32 bit). If so, that would streamline your work.

What happens if OP runs into runtime or performance issues? Hacking FreeBSD kernel is just alike hacking any program. Use source, modify, build. But then if it crashes you have to know to debug kernels and if it doesn't work appropriately you need to know how to profile and instrument them.

I believe OP mentiones kernel and device drivers assuming that is the only way to create "virtual sound devices".

IMO better way is to implement this direcly in virtual_oss.

 

[unitrunker]

Zare No doubt but OP stated wanting to "practice more low-level programming" which I took to mean in-kernel.

 

[cracauer@]

Zare No doubt but OP stated wanting to "practice more low-level programming" which I took to mean in-kernel.

Well, OP would be doing high-level programming, just in the kernel.

 

[tembun]

tembun is the filter relatively simple like a FIR or IIR filter?

Honestly, I don't know, I have very little experience with this, I'm going to learn in the process. I think it doesn't really matter for me, my goal is not to find the easiest way to implement a particular task, because I don't have one - I pretend I have one and focus on the code and the implementation.

IMO better way is to implement this direcly in virtual_oss.

Thanks, this is what I decided to do, that would be interesting to me too. I implemented a program that creates a cuse(3) device and which for now just produces some sound on read(2) system call. And in order to allow the system and other programs to 'see' my device as a valid microphone device, I used virtual_oss(8) to expose it as a recording-only device /dev/dsp.mymic (by means of -R /dev/mymic -P /dev/null options). Now, in my device I implemented only read(2) system call callback (and open(2) and close(2)), and now (when virtual_oss device is created), I see that my device now receives ioctl(2) and poll(2) requests, which I don't handle. As a result, virtual_oss device doesn't actually do anything right now. Do I understand correctly, that I need to make my cuse device a fully OSS-compatible device and implement all available OSS ioctls? Or maybe I need to implement only part of these, maybe only those that are related to the recording?

Also, am I right that when I would finally connect virtual_oss device with my device, /dev/dsp.mymic would be a valid recording device, that other programs will be able to see and use? E.g. will I be able to select this device as my default microphone in the browser, for instance?

P.S. Since I ended up programming this in the userspace, maybe this thread should now be moved into 'Userland Programming and Scripting'?