Sleeping a kernel thread

[Maelstorm]

What is the best way to make a kernel thread goto sleep for a specified amount of time and relinquish the processor for the rest of the tick? Some of the research that I have done on this (forums (here), google, pause(9), etc) suggests using pause(9) to sleep the thread. However, the man page says the following:

The pause() function is a wrapper around tsleep(9) that suspends execution of the current thread for the indicated timeout. The thread can not be awakened early by signals or calls to wakeup() or wakeup_one().

Pause(9) however will not wake back up until the timeout expires. This is not a viable solution as this is running inside a KLM. So if the module unloads, the thread is still sleeping so the unload has to wait, or the kernel will panic. Looking at wake(9) and tsleep(9), there is one parameter that I am not sure of. The explaination in the man page is as follows:

The parameter chan is an arbitrary address that uniquely identifies the event on which the thread is being put to sleep. All threads sleeping on a single chan are woken up later by wakeup(), often called from inside an interrupt routine, to indicate that the resource the thread was blocking on is available now.

Which I find somewhat vague and confusing. Is the address just some random address that is used or does it actually point to something? A struct or int perhaps? I will continue digging through the source code, but if someone could explain to me what chan really is, I would be grateful.

Thank you.

 

[Maelstorm]

I have written the code for the thread, and now I get a panic when the module unloads. When the unload command is given, the thread *SHOULD* wakeup, see that the return code is not EWOULDBLOCK, and then call kthread_exit(9). The problem is that it does not seem to be waking up. By the time the thread wakes up, the module is long gone from memory. The address given for the panic does not exist in the vmcore file when viewed with kgdb. I think what's happening is that the kernel is deallocating the VM page that the module occupied while the thread was sleeping. So when tsleep returns, the return address on the stack no longer exists, so the CPU issues a hardware trap. But since we are in kernel mode, the kernel panics because you are not supposed to be page faulting in kernel mode.

Unread portion of the kernel message buffer:

Fatal trap 12: page fault while in kernel mode
fault virtual address   = 0xc4cdeb76
fault code              = supervisor read, page not present
instruction pointer     = [B]0x20:0xc4cdeb76[/B]
stack pointer           = 0x28:0xde9e3cc8
frame pointer           = 0x28:0xde9e3cec
code segment            = base 0x0, limit 0xfffff, type 0x1b
                        = DPL 0, pres 1, def32 1, gran 1
processor eflags        = interrupt enabled, resume, IOPL = 0
current process         = 0 [B](syscall scanner)[/B]
trap number             = 12

Something else that is curious though. The back trace on the stack gives some weird error:

#1  0xc064ec37 in kern_reboot (howto=260)
    at /usr/src/sys/kern/kern_shutdown.c:442
#2  0xc064ee6e in panic (fmt=Variable "fmt" is not available.)
    at /usr/src/sys/kern/kern_shutdown.c:607
#3  0xc081f013 in trap_fatal (frame=0xde9e3c88, eva=3301829494)
    at /usr/src/sys/i386/i386/trap.c:975
#4  0xc081f290 in trap_pfault (frame=0xde9e3c88, usermode=0, eva=3301829494)
    at /usr/src/sys/i386/i386/trap.c:888
#5  0xc081fdf3 in trap (frame=0xde9e3c88) at /usr/src/sys/i386/i386/trap.c:558
#6  0xc080c33c in calltrap () at /usr/src/sys/i386/i386/exception.s:168
#7  [B]0xc4cdeb76[/B] in ?? ()
Previous frame inner to this frame (corrupt stack?)

I'm not sure exactly what it is trying to tell me because I see the return address on the stack.

 

[Maelstorm]

After doing further research into the issue that I've been having, and doing a full kernel rebuild using INVARIANTS, INVARIANT_SUPPORT, WITNESS, WITNESS_SKIPSPIN, WITNESS_KDB, KDB, KDB_TRACE, DDB, GDB, BREAK_TO_DEBUGGER, and DIAGNOSTIC, I believe that I have found the root of the problem. The debugger output was enlightening to say the least. It appears to be a race condition between the thread exiting, and the module unloading. If the module unloads first before the thread exits, then the kernel panics because the VM page apparently no longer exists. However, when I introduce a mutex lock on the kernel thread, and then have the module unload routine wait for that mutex to release, it seems to solve the problem. But now I have a new problem to be resolved. It seems that the MTX_DEF type cannot be exclusively held while the thread is sleeping. Witness complained about it. I'm still looking, but if anyone has any suggestions as to how to proceed, I am listening.

 

[fluca1978]

By the time the thread wakes up, the module is long gone from memory. The address given for the panic does not exist in the vmcore file when viewed with kgdb. I think what's happening is that the kernel is deallocating the VM page that the module occupied while the thread was sleeping.

I don't think so, but I don't have an explaination.
After all, if the vm reclaims pages of sleeping threads, a lot of panics would be generated. The page should be pinned by the thread, and therefore not allowed to be reclaimed.

 

[Maelstorm]

When I think about it, it actually makes sense. The thread is known to the system, and it is part of proc0, the kernel process. The actual code to the thread is inside the module. When the module unloads, the addresses that were held by the module become invalid. The computer, even though it's just a AMD T-Bird 1.4 GHz machine, is still plenty fast enough to execute the unload routine in it's entirety and return to the caller before the system tick. The wakeup(9) is issued which breaks the sleep, but it doesn't have a chance to execute until after the module unloads because the controlling process that is doing the unload routine has not relinquished the CPU.

So the way that I solved it, temporarily, was to create a sx(9) shared lock and grab it before the thread goes into the main loop. The unload routine calls wakeup and then tries to get an exclusive lock. But since the thread has a shared lock, the unloader blocks and that forces a context switch. The very last command before kthead_exit is sx_sunlock. There is still an issue with a race condition but it's now a very small window. I would prefer to have no opportunity for a race condition though. There has got to be a better way of doing this though. What I need is a command or function that will force the control process to relinquish the CPU. But a lock would still be needed though because when I consider a multiprocessing/multicore environment, threads can execute at the same exact time completely independent of each other, hence the race condition. I think that holding the lock and forcing a context switch of the controlling process would eliminate the race condition.

After all, for as long as that thread is running, that sx lock is held. The man page for sx says that sx locks should not be held for long periods of time. I could probably do a sleep on the control process, but that just doesn't seem proper to do. I'm going to dig through the source code of the scheduler and perhaps even the hardclock() and softclock() routines to see if there is something that I can use. After that, I have no idea.

I do agree with you that since the thread is still in the run queue, the memory that the code occupies should be wired in. However, from what I have observed, that does not seem to be case. So when tsleep returns, the return address on the stack no longer exists so it causes a page fault and the kernel panics.

 

[Maelstorm]

It looks like I got the module threading working just right now. It starts and stops reliably, and it will also die correctly on system shutdown/restart/halt as well. The attachment contains the basic framework code for starting and stopping a proc0 kernel thread on module load/unload and system shutdown/reboot/halt. The attached code gives a basic outline of how to set it up. If you run into any problems, post back here. However, my ability to help you is limited.