C Using kqueue in event-driven multithread application

Hi guys!

I have Linux multithread application with synchronization based on events (eventfd/poll). I would like to port it to FreeBSD using native FreeBSD calls and don't use linuxuator. So, I'm investigating kqueue and
EVFILT_USER approach. But I have faced with problem when I want to poll some number of particular events. My first approach is to use one kqueue descriptor for all events. For example: I created 50 EVFILT_USER events and attach them to one kqueue. In one of my thread I want to poll 10 from 50 events, so I do:

C:
void *thread1(void *arg)
{
    int ret = -1;
    
    while (1) {
        struct kevent tevent[10] = { 0 };
        
        ret = kevent(kq, NULL, 0, tevent, 10, NULL);
        if (ret < 0) {
            /* handle error */
            break;
        }
        
        if (ret > 0) {
            for (int i = 0; i < ret; i++) {
                int event = tevent[i].ident;
                
                switch (event) {
                    case EVENT1:
                        /* handle EVENT1 */
                        break;
                    
                    case EVENT2:
                        /* handle EVENT2 */
                        break;
                        
                    ...
                        
                    case EVENT10:
                        /* handle EVENT10 */
                        break;
                    
                    default:
                        /*
                            Here we can catch the remaining 40 events.
                            Resend them?
                        */
                        break;
                }
            }
        }
    }
}

In the "default" case in the "switch" there may be situation when I catch events that I'm not interesting in that thread (but another threads may awaits them). Do I need to just resend them? Is that good approach?

In Linux I can await just particular events and be sure that events 11-50 will not be caught by the thread's poll().
:
C:
void *thread1(void *arg)
{
    int ret = -1;
    struct pollfd wait_events[10] = {0};
    
    wait_events[0].fd = event1;
    wait_events[0].events = POLLIN;
    
    ...
    
    wait_events[9].fd = event10;
    wait_events[9].events = POLLIN;
    
    while (1) {
        ret = poll(wait_events, 10, -1);
        if (ret < 0) {
            /* handle error */
            break;
        }
        
        if (ret > 0) {
            if ((wait_events[0].revents & POLLIN) != 0) {
                /* handle event1 */
                continue;
            }
            
            ....
            
            if ((wait_events[9].revents & POLLIN) != 0) {
                /* handle event10 */
                continue;
            }
        }
    }
}

Another approach that I'm pondering is 1 kqueue <=> 1 event: I create a number of kqueue and attach just one EVFILT_USER event to each of them. In this case I could use poll() like in Linux example and poll kqueue descriptors. But this approach has problem two: I cannot reset kqueue descriptor like Linux eventfd descriptor by reading from it.

Could you please advice what is the best approach for event-driven application in FreeBSD? Is resending of non-interested events good like in my first code example?
 
Would it be too hard for you to use one kqueue(2) per thread? This way each kqueue will only monitor for events its owner thread is responsible for.

PS: Take into account I have no experience with EVFILT_USER.
 
  • Thanks
Reactions: Kim
Would it be too hard for you to use one kqueue(2) per thread? This way each kqueue will only monitor for events its owner thread is responsible for.

PS: Take into account I have no experience with EVFILT_USER.

Unfortunately, in my case some events are used on several threads but some of them are unique for each thread.

Currently I'm investigating one more approach: create a number of kqueue-s and add one user event into each of them. In this case I can use poll() like in Linux for kqueue descriptors.
 
Hi, I work on a company called BrByte (https://software.brbyte.com). We use FreeBSD as a base to our release BrbOS.

All of our codes use a framework written in C (brb_framework), developed by me and my coworker Guilherme Alves.

We recently released a piece of our code.

https://github.com/BrByte/brb_framework

The main source of the framework is the library libbrb_core.

This library contains an abstraction for an event loop, using 'kqueue'.

I have written some documentation to install and use libbrb_core.
Take a look at https://github.com/BrByte/brb_framework/blob/master/BRBCORE.md

It still has missing documentation, but you can read the code of our implementation with kqueue at https://github.com/BrByte/brb_framework/tree/master/libbrb_core/event/core.

Or, if you want to use the library, take a look at the test codes: https://github.com/BrByte/brb_framework/tree/master/libbrb_core/test_code

The library libbrb_core also has sources to do: double linked list, dynamic array, dynamic bitmap, hash tables, linked lists, memory arena, memory buffer, memory_strem, radix_tree, slotqueue. assoc_array, ipv4_table, key_value, mem_buf_mapped, mem_slot, meta_data, queue, speed_regex, string_array, string_assoc_array, string
 
Back
Top