Building only usr.sbin/bhyve rather than the entire OS.

I have the FreeBSD source code at standard /usr/src/ from hier(), and having hacking usr.sbin/bhyve, but everything that I change a single line I have to wait a single line to a file I have to wait several minutes before I got even an error message, then have to correct and recompile, making a minutes work an hour work.
How do I compile a FreeBSD component so that I do no have to run make buildworld, and rather only build the part I am working on?
 
"make -C /usr/src/usr.sbin/bhyve"

Another alternative, more "idiomatic":
make obj depend all SUBDIR_OVERRIDE=usr.sbin/bhyve

This should always work but will build more things than just bhyve:
make buildworld SUBDIR_OVERRIDE=usr.sbin/bhyve
 
"make -C /usr/src/$somedir" works fine (provided /usr/obj can be written to. I typically chown it to a normal non-root user and build everything as that user). Cross building etc would get more complicated!
 
Now ran make -j32 buildworld at /usr/src/, and get an errored build, cannot see the error, now ran make buildworld at the same location. Now I have another problem, there in an error at a random location (well a compiler warning that I suspect is the cause of the error), and my git says I am at release/15.1.0 with only changes at /usr.sbin/bhyve/.
So what is happening, and how I made to not wait an eternity when there is an error in a buildworld to find the error?.
I just have to grep the output, and ran make -j32 buildworld | tee log?
 
So what is happening, and how I made to not wait an eternity when there is an error in a buildworld to find the error?.
I'm building frequently from source to update/upgrade the system or test system components, you should enable WITH_META_MODE in /etc/src-env.conf and / or devel/ccache to reduce build time. MetaMode has effect on subsequent builds but not on the first build.

I just have to grep the output, and ran make -j32 buildworld | tee log?
That, or make buildworld-jobs save the build messages in a log file: /usr/src/UPDATING
Code:
20230420:
        Add jobs.mk to save typing. Enables -j${JOB_MAX} and logging
        eg.
                make buildworld-jobs
        runs
                make -j${JOB_MAX} buildworld > ../buildworld.log 2>&1

        where JOB_MAX is derrived from ncpus in local.sys.mk if not set in env.
Also valid: "buildkernel-jobs"

Or script(1).
 
Last edited:
Now ran make -j32 buildworld at /usr/src/, and get an errored build, cannot see the error, now ran make buildworld at the same location. Now I have another problem, there in an error at a random location (well a compiler warning that I suspect is the cause of the error), and my git says I am at release/15.1.0 with only changes at /usr.sbin/bhyve/.
So what is happening, and how I made to not wait an eternity when there is an error in a buildworld to find the error?.
I just have to grep the output, and ran make -j32 buildworld | tee log?
Make sure to run your build with a shell that isn't infested by some environment variable like linker path, include path etc...
 
"make -C /usr/src/$somedir" works fine (provided /usr/obj can be written to. I typically chown it to a normal non-root user and build everything as that user). Cross building etc would get more complicated!
It works fine if /usr/src and the installed world are in a good match.
 
Sure. It's possible to build system source as an un-privileged user, with the built written into the users home director, by setting MAKEOBJDIRPREFIX. You can use any source tree, owned by anyone, as long the building user has read permission to that source tree. The source tree itself is read-only, no writing is happening there during a build process.

When testing system source code, you should read make.conf(5), build(7), src.conf(5), alias src-env.conf(5).
 
Last edited:
This should always work but will build more things than just bhyve:
make buildworld SUBDIR_OVERRIDE=usr.sbin/bhyve
Well this time make buildworld SUBDIR_OVERRIDE=contrib/lib9p it did not work.
Correction: It did not work at the contrib/lib9p directory. It has to be called at the source root.
Sorry.
Correction again: it did fail with the same error as make obj depend all SUBDIR_OVERRIDE=contrib/lib9p and make -C /usr/src/contrib/lib9p
Code:
===> contrib/lib9p (includes)
make[3]: /usr/src/share/mk/src.libnames.mk:930: /usr/src/contrib/lib9p: Invalid LIBADD used which may need to be added to src.libnames.mk: libcasper libcap_pwd libcap_grp
        in /usr/src/share/mk/bsd.libnames.mk:12
        in /usr/src/share/mk/bsd.lib.mk:172
        in /usr/src/contrib/lib9p/Makefile:27
        in make[3] in directory "/usr/src/contrib/lib9p"
all of those called at /usr/src with checkout at release/15.1.0.
 
If you made local changes to lib9p, compile in /usr/src/lib/lib9p. Usually things in /usr/src/contrib are used from other standard places and compiled there.
 
For some reason runing
Code:
LD_PRELOAD="/usr/obj/usr/src/amd64.amd64/lib/lib9p/lib9p.so.1" LD_LIBRARY_PATH="/usr/obj/usr/src/amd64.amd64/lib/lib9p/:/lib/:/usr/lib" /usr/obj/usr/src/amd64.amd64/usr.sbin/bhyve/bhyve $bhyve_options
[code]
is not running my code which would be
[code]
diff --git a/contrib/lib9p/connection.c b/contrib/lib9p/connection.c
index 20c27796b829..f60e4789fc3d 100644
--- a/contrib/lib9p/connection.c
+++ b/contrib/lib9p/connection.c
@@ -89,6 +89,7 @@ void
 l9p_connection_recv(struct l9p_connection *conn, const struct iovec *iov,
     const size_t niov, void *aux)
 {
+       fprintf(stderr, "At: %s\n\r", __func__);
        struct l9p_request *req;
        int error;
 
diff --git a/usr.sbin/bhyve/pci_virtio_9p.c b/usr.sbin/bhyve/pci_virtio_9p.c
index 3db7c86e0eb2..f3b93c5d148d 100644
--- a/usr.sbin/bhyve/pci_virtio_9p.c
+++ b/usr.sbin/bhyve/pci_virtio_9p.c
@@ -208,9 +208,9 @@ pci_vt9p_notify(void *vsc, struct vqueue_info *vq)
                preq->vsr_respidx = req.readable;
 
                for (int i = 0; i < n; i++) {
-                       DPRINTF(("vt9p: vt9p_notify(): desc%d base=%p, "
+                       printf("vt9p: vt9p_notify(): desc%d base=%p, "
                            "len=%zu\r\n", i, iov[i].iov_base,
-                           iov[i].iov_len));
+                           iov[i].iov_len);
                }
[/cmd]
the [cmd]vt9p: vt9p_notify():[/cmd] is showing in console, but for some reason the if I am not crazy immediate call to [cmd]pci_vt9p_notify[/cmd], is not printing anything.
by the output of
[code]
LD_PRELOAD="/usr/obj/usr/src/amd64.amd64/lib/lib9p/lib9p.so.1" LD_LIBRARY_PATH="/usr/obj/usr/src/amd64.amd64/lib/lib9p/:/lib/:/usr/lib" ldd /usr/obj/usr/src/amd64.amd64/usr.sbin/bhyve/bhyve
here
Code:
/usr/obj/usr/src/amd64.amd64/usr.sbin/bhyve/bhyve:
        libvmmapi.so.7 => /usr/lib/libvmmapi.so.7 (0x3c80f0be2000)
        libmd.so.7 => /lib//libmd.so.7 (0x3c80f1876000)
        libnv.so.1 => /lib//libnv.so.1 (0x3c80f3cf9000)
        libuvmem.so.1 => /usr/lib/libuvmem.so.1 (0x3c80f461c000)
        libz.so.6 => /lib//libz.so.6 (0x3c80f5daa000)
        libutil.so.10 => /lib//libutil.so.10 (0x3c80f58de000)
        libsbuf.so.6 => /lib//libsbuf.so.6 (0x3c80f6dc5000)
        libcam.so.7 => /lib//libcam.so.7 (0x3c80f7af2000)
        lib9p.so.1 => /usr/obj/usr/src/amd64.amd64/lib/lib9p/lib9p.so.1 (0x3c80f3080000)
        libnetgraph.so.4 => /usr/lib/libnetgraph.so.4 (0x3c80f8615000)
        libcrypto.so.35 => /lib//libcrypto.so.35 (0x3c80f9e00000)
        libthr.so.3 => /lib//libthr.so.3 (0x3c80f92a1000)
        libc.so.7 => /lib//libc.so.7 (0x3c80fad11000)
        libcasper.so.1 => /lib//libcasper.so.1 (0x3c80fbd56000)
        libcap_pwd.so.1 => /lib//libcap_pwd.so.1 (0x3c80fbdc8000)
        libcap_grp.so.1 => /lib//libcap_grp.so.1 (0x3c80fbefc000)
        libsys.so.7 => /lib//libsys.so.7 (0x3c80fc427000)
        [vdso] (0x3c80f097e000)
the library should be loading my modified lib9p.so.1 and printing At: l9p_connection_recv immediatly after every batch of vt9p_notify prints.
 
Found the reason.
For some reason runing
Code:
LD_PRELOAD="/usr/obj/usr/src/amd64.amd64/lib/lib9p/lib9p.so.1" LD_LIBRARY_PATH="/usr/obj/usr/src/amd64.amd64/lib/lib9p/:/lib/:/usr/lib" /usr/obj/usr/src/amd64.amd64/usr.sbin/bhyve/bhyve $bhyve_options
[code]
is not running my code which would be
[code]
diff --git a/contrib/lib9p/connection.c b/contrib/lib9p/connection.c
index 20c27796b829..f60e4789fc3d 100644
--- a/contrib/lib9p/connection.c
+++ b/contrib/lib9p/connection.c
@@ -89,6 +89,7 @@ void
 l9p_connection_recv(struct l9p_connection *conn, const struct iovec *iov,
     const size_t niov, void *aux)
 {
+       fprintf(stderr, "At: %s\n\r", __func__);
        struct l9p_request *req;
        int error;
 
diff --git a/usr.sbin/bhyve/pci_virtio_9p.c b/usr.sbin/bhyve/pci_virtio_9p.c
index 3db7c86e0eb2..f3b93c5d148d 100644
--- a/usr.sbin/bhyve/pci_virtio_9p.c
+++ b/usr.sbin/bhyve/pci_virtio_9p.c
@@ -208,9 +208,9 @@ pci_vt9p_notify(void *vsc, struct vqueue_info *vq)
                preq->vsr_respidx = req.readable;
 
                for (int i = 0; i < n; i++) {
-                       DPRINTF(("vt9p: vt9p_notify(): desc%d base=%p, "
+                       printf("vt9p: vt9p_notify(): desc%d base=%p, "
                            "len=%zu\r\n", i, iov[i].iov_base,
-                           iov[i].iov_len));
+                           iov[i].iov_len);
                }
[/cmd]
the [cmd]vt9p: vt9p_notify():[/cmd] is showing in console, but for some reason the if I am not crazy immediate call to [cmd]pci_vt9p_notify[/cmd], is not printing anything.
by the output of
[code]
LD_PRELOAD="/usr/obj/usr/src/amd64.amd64/lib/lib9p/lib9p.so.1" LD_LIBRARY_PATH="/usr/obj/usr/src/amd64.amd64/lib/lib9p/:/lib/:/usr/lib" ldd /usr/obj/usr/src/amd64.amd64/usr.sbin/bhyve/bhyve
here
Code:
/usr/obj/usr/src/amd64.amd64/usr.sbin/bhyve/bhyve:
        libvmmapi.so.7 => /usr/lib/libvmmapi.so.7 (0x3c80f0be2000)
        libmd.so.7 => /lib//libmd.so.7 (0x3c80f1876000)
        libnv.so.1 => /lib//libnv.so.1 (0x3c80f3cf9000)
        libuvmem.so.1 => /usr/lib/libuvmem.so.1 (0x3c80f461c000)
        libz.so.6 => /lib//libz.so.6 (0x3c80f5daa000)
        libutil.so.10 => /lib//libutil.so.10 (0x3c80f58de000)
        libsbuf.so.6 => /lib//libsbuf.so.6 (0x3c80f6dc5000)
        libcam.so.7 => /lib//libcam.so.7 (0x3c80f7af2000)
        lib9p.so.1 => /usr/obj/usr/src/amd64.amd64/lib/lib9p/lib9p.so.1 (0x3c80f3080000)
        libnetgraph.so.4 => /usr/lib/libnetgraph.so.4 (0x3c80f8615000)
        libcrypto.so.35 => /lib//libcrypto.so.35 (0x3c80f9e00000)
        libthr.so.3 => /lib//libthr.so.3 (0x3c80f92a1000)
        libc.so.7 => /lib//libc.so.7 (0x3c80fad11000)
        libcasper.so.1 => /lib//libcasper.so.1 (0x3c80fbd56000)
        libcap_pwd.so.1 => /lib//libcap_pwd.so.1 (0x3c80fbdc8000)
        libcap_grp.so.1 => /lib//libcap_grp.so.1 (0x3c80fbefc000)
        libsys.so.7 => /lib//libsys.so.7 (0x3c80fc427000)
        [vdso] (0x3c80f097e000)
the library should be loading my modified lib9p.so.1 and printing At: l9p_connection_recv immediatly after every batch of vt9p_notify prints.
The idiot of me have something like
Code:
LD_PRELOAD="/usr/obj/usr/src/amd64.amd64/lib/lib9p/lib9p.so.1" LD_LIBRARY_PATH="/usr/obj/usr/src/amd64.amd64/lib/lib9p/:/lib/:/usr/lib" /usr/obj/usr/src/amd64.amd64/usr.sbin/bhyve/bhyve --destropy --vm=vm
bhyve $bhyve_options
instead of the regular
Code:
LD_PRELOAD="/usr/obj/usr/src/amd64.amd64/lib/lib9p/lib9p.so.1" LD_LIBRARY_PATH="/usr/obj/usr/src/amd64.amd64/lib/lib9p/:/lib/:/usr/lib" /usr/obj/usr/src/amd64.amd64/usr.sbin/bhyve/bhyve $bhyve_options
 
Well this time make buildworld SUBDIR_OVERRIDE=contrib/lib9p it did not work.
Well, that's because you use a wrong directory.
contrib/ is for source code of third-party software.
But the actual binaries, libraries, etc have their own directories.
In this case it would be lib/lib9p, I guess.
 
Back
Top