How I built a module

[Incnis Mrsi]

Hello.
Due to non-standard operational requirements I attempted to modify a module, namely /usr/src/sys/net/if_stf.c (instead of trying more conventional workarounds). This has been a “FreeBSD 12.1-RELEASE r354233 GENERIC amd64” system without any prior attempts to build kernel, without changes to the kernel and its source files. Of course, documentation first, so I found myself studying the /books/arch-handbook/driverbasics-kld.html page. Then I followed the handbook.

1. Went root;
2. Changed to /usr/src/sys/net/;
3. mv if_stf.c if_stf.c.orig and wrote the new if_stf.c;
4. Created the Makefile;
5. Ran make which made its job.

Now the fun started. Where did make place if_stf.ko, to the working directory? Failed to guess – both .o and .ko went to /usr/obj/usr/src/amd64.amd64/sys/net/. Moreover,
netto kernel: link_elf_obj: symbol in_ifaddrhashtbl undefined
netto kernel: linker_load_file: /usr/obj/usr/src/amd64.amd64/sys/net/if_stf.ko - unsupported file type
The original module was, of course, already kldunloaded. Inspection with nm(1) revealed that the two modules (new vs original, out-of-the-box FreeBSD) have somewhat different sets of U symbols. In other words, some symbols in that new .ko were spurious.
Ī̲ traced the problematical stuff to /usr/src/sys/netinet/in_var.h which defines:

VNET_DECLARE(struct in_ifaddrhashhead *, in_ifaddrhashtbl);
VNET_DECLARE(struct in_ifaddrhead, in_ifaddrhead);
VNET_DECLARE(u_long, in_ifaddrhmask);                /* mask for hash table */

#define      V_in_ifaddrhashtbl      VNET(in_ifaddrhashtbl)
#define      V_in_ifaddrhead         VNET(in_ifaddrhead)
#define      V_in_ifaddrhmask        VNET(in_ifaddrhmask)

where the obscure macros are defined in /usr/src/sys/net/vnet.h, and both header files can be found in /usr/obj/usr/src/amd64.amd64/sys/net/.depend.if_stf.o which suggests that the C compiler included these definitions. As there were no “vnet_entry_in_” prefix in my new if_stf.ko, I inspected vnet.h, was almost lost in it and started to type a question to Stackoverflow (the first in my life), but stopped short of posting when by chance spotted the #ifdef VIMAGE conditional.

Hence I finally realized that my Makefile should contain

SRCS=if_stf.c
KMOD=if_stf
# “VIMAGE” was not mentioned in the handbook, but it must be defined!
CFLAGS = -DVIMAGE

.include <bsd.kmod.mk>

TL;DR: *BSD systems have reputation of a stuff limited to IT professionals and geeks for a reason. Documentation at freebsd.org is stubby and partially obsolete (not a single word about “VIMAGE”). Wiki.freebsd.org, in general, has some details and shows edit history, whereas authorship of the “handbook” is anonymous (for a general reader, not a fan of versioning systems) and the book neglects to specify on which versions of the software were its conceits tested. Without my experience in C programming I would be powerless.

 

[George]

I would have suggested to download the kernel sources, modify your .c files, and then run "make buildkernel" with MODULES_OVERRIDE=..

With this, you wouldn't need to edit any Makefiles. Also, you avoid trouble with files that only exist during a kernel build (some if_...h files).

But you got it working, right?

 

[Incnis Mrsi]

But you got it working, right?

At the end I got. Positive impressions: an extensive collection of man pages in the Section 9. Negative impressions: paranoid programming practices, namely in_stf_input() scrabbles and quibbles where stf_encapcheck() already performed. It IMHO defeats the intent of verification as a separate routine.

By the way, the “VIMAGE” stuff was already discussed at Thread how-to-specify-kernel-configuration-when-compiling-out-of-tree-kernel-modules.71576, but did not find its way into the handbook.