Solved Cannot buildworld on machine with 256MB memory

P

pkc

It fails as follows:

Code: 
[...]
c++  -target x86_64-unknown-freebsd11.2 --sysroot=/usr/obj/usr/svn-src/11.2/tmp -B/usr/obj/usr/svn-src/11.2/tmp/usr/bin  -O2 -pipe -I/usr/obj/usr/svn-src/11.2/lib/clang/libllvm -I/usr/svn-src/
11.2/contrib/llvm/lib/Target/AArch64 -I/usr/svn-src/11.2/contrib/llvm/lib/Target/ARM -I/usr/svn-src/11.2/contrib/llvm/lib/Target/Mips -I/usr/svn-src/11.2/contrib/llvm/lib/Target/PowerPC -I/usr
/svn-src/11.2/contrib/llvm/lib/Target/Sparc -I/usr/svn-src/11.2/contrib/llvm/lib/Target/X86 -I/usr/svn-src/11.2/lib/clang/include -I/usr/svn-src/11.2/contrib/llvm/include -DLLVM_BUILD_GLOBAL_I
SEL -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNDEBUG -DLLVM_DEFAULT_TARGET_TRIPLE=\"x86_64-unknown-freebsd11.2\" -DLLVM_HOST_TRIPLE=\"x86_64-unknown-freebsd11.2\" -DDEFAULT_SYSROOT=\"\"
 -ffunction-sections -fdata-sections -MD -MF.depend.Target_ARM_ARMTargetMachine.o -MTTarget/ARM/ARMTargetMachine.o -fstack-protector-strong -Qunused-arguments  -std=c++11 -fno-exceptions -fno-
rtti -stdlib=libc++ -Wno-c++11-extensions  -c /usr/svn-src/11.2/contrib/llvm/lib/Target/ARM/ARMTargetMachine.cpp -o Target/ARM/ARMTargetMachine.o                                              
c++: error: unable to execute command: Killed
c++: error: clang frontend command failed due to signal (use -v to see invocation)
FreeBSD clang version 6.0.0 (tags/RELEASE_600/final 326565) (based on LLVM 6.0.0)
Target: x86_64-unknown-freebsd11.2
Thread model: posix
InstalledDir: /usr/bin

Code: 
# swapctl -lh
Device:            Bytes      Used:
/dev/vtbd0p2        3.0G       8.2M
/dev/md0            5.0G       7.7M
# sysctl vfs.zfs.arc_max
vfs.zfs.arc_max: 17275840
# zfs get primarycache z30
NAME  PROPERTY      VALUE         SOURCE
z30   primarycache  none          local

However swap as listed in top is 1687MB. The swap usage never exceeds a few hundred MB as far as I can tell but I can try again with swapctl -lh in a loop to double check.

Any ideas how to proceed?

Thanks
 
I realized that I may need to set kern.maxswzone to a higher value. There is a message in /var/log/messages

Code: 
kernel: warning: total configured swap (2097152 pages) exceeds maximum recommended amount (215992 pages).
warning: increase kern.maxswzone or reduce amount of swap.

This message continues to appear even after raising kern.maxswzone arbitrarily in /boot/loader.conf. Does anyone know how to raise it properly?


This message seems to suggest that this is not possible- https://lists.freebsd.org/pipermail/freebsd-hackers/2014-March/044698.html

Do I have any other options?
 
Add
Code: 
CXX_FLAGS += -v
to make.conf, set as environment variable, or use make to pass to clang verbose flag, as suggested in error above to you; may give you more specific information on what's going on behind the scenes. Have you tried in single user?
 
I'd recommend using another machine; you can always cross-compile if you need to. FreeBSD src's makes are setup to enable this out of the box.
 
ikbendeman said:
I'd recommend using another machine; you can always cross-compile if you need to.
Click to expand...
That's what I often do. You can do the buildworld/buildkernel on a "beefy" machine. Then NFS export (can be set read-only) /usr/src/ and /usr/obj. Mount those on the "small" machine and do the install.

The only thing you need to watch out for is the date/time. Make sure both system are running the exact same date/time or else you can get some really weird errors.
 
ikbendeman said:
Add
Code: 
CXX_FLAGS += -v
to make.conf, set as environment variable, or use make to pass to clang verbose flag, as suggested in error above to you; may give you more specific information on what's going on behind the scenes. Have you tried in single user?
Click to expand...
Thank you, I forgot to specify it but it indeed appear to be because swap runs out (this is also evident in /var/log/messages. I assumed immediately that it was indeed due to running out of memory and I only verified it after the fact. Sorry for not mentioning this.



SirDice said:
That's what I often do. You can do the buildworld/buildkernel on a "beefy" machine. Then NFS export (can be set read-only) /usr/src/ and /usr/obj. Mount those on the "small" machine and do the install.

The only thing you need to watch out for is the date/time. Make sure both system are running the exact same date/time or else you can get some really weird errors.
Click to expand...
I have thought of possibly doing this. One thing is that that they are not on the same local network but they do have a reasonable internet connection and, so I could run NFS over IPsec. I feel worried about getting the time synced as I am always bad at that. The main problem though is when I have tried to do this in the past I get errors about failure to apply special modes to certain files when I try to do installworld over NFS.
Maybe it would be best for my to try to rsync the /usr/obj directory from the install machine and run installworld locally? And I guess then I do need to have the corresponding /usr/src in order to install /etc and other such directories? I guess I just wanted to build it locally to keep things simple but at this point I agree it would be a better solution to build remotely.

Thanks[/cmd][/cmd]
 
SirDice said:
That's what I often do. You can do the buildworld/buildkernel on a "beefy" machine. Then NFS export (can be set read-only) /usr/src/ and /usr/obj. Mount those on the "small" machine and do the install.

The only thing you need to watch out for is the date/time. Make sure both system are running the exact same date/time or else you can get some really weird errors.
Click to expand...
Sorry I misunderstood what you said. I actually only tried doing this by mounting the install destination via NFS from the build system. But I see you are proposing the opposite direction and I can try that.
However it would seem that maybe rsync is easier, are there any issues I might run into with that that I've overlooked?
In any case I am waiting for the build to complete so after I can try it.
 
I think SirDice is recommending the build be fully completed, then you directly mount /usr/src, cd /usr/src && make installkernel installworld from the client machine (the one you're having build errors on. You could rsync, as well, but you could do the above from a live image. Depending on how you do it, you can just change DESTDIR
 
ikbendeman said:
I think SirDice is recommending the build be fully completed, then you directly mount /usr/src, cd /usr/src && make installkernel installworld from the client machine (the one you're having build errors on. You could rsync, as well, but you could do the above from a live image. Depending on how you do it, you can just change DESTDIR
Click to expand...


Indeed (except I would also need to mount /usr/obj). Unfortunately it turns out I will not be able to do this without IPsec, but I haven't compiled support for it in the kernel, so the method will not work in this case. I don't think I could do it from a live image because of the memory problem. I am in the process of rsyncing the /usr/obj and /usr/src directories over but I am actually running out of swap in those cases I think. I also see messages like

Code: 
swap_pager: indefinite wait buffer: [...]

So there might be a deeper issue at hand here that I'll need to investigate. It would not make sense to me for it to run out of swap space during rsync. I am also currently unable to login. It's a VPS but I guess these things can happen there too.
 
Create swap on a usb pen or PXE boot using IPSec; I've never done this but it looks like you'd just need a gateway/router config that would create your tunnel. I think most commodity gateways have support for this. I know they do with third-party firmware.
 
Sorry for the delay, I did manage to get the remote build rsync'ed over, and successfully installed, as far as I can tell. I took care to place them in the same location relative to /usr/src and /usr/obj that they had been on the build machine. For whatever reason, it also appears that I needed to make the following changes to Makefile.inc1

Code: 
--- /root/Makefile.inc1.old     2018-11-01 18:30:39.947512000 +0100
+++ Makefile.inc1       2018-11-07 03:47:41.562120000 +0100
@@ -1292,6 +1292,7 @@

NO_INSTALLEXTRAKERNELS?=       yes

+INSTALLKERNEL=MYKERNEL
#
# installkernel, etc.
#
@@ -1300,10 +1301,6 @@
installkernel installkernel.debug \
reinstallkernel reinstallkernel.debug: _installcheck_kernel .PHONY
.if !defined(NO_INSTALLKERNEL)
-.if empty(INSTALLKERNEL)
-       @echo "ERROR: No kernel \"${KERNCONF}\" to install."; \
-       false
-.endif
        @echo "--------------------------------------------------------------"
        @echo ">>> Installing kernel ${INSTALLKERNEL}"
        @echo "--------------------------------------------------------------"

where MYKERNEL is the one I build on the build host.

Since this is in a low-cost VPS, my options were limited as the build machine was not local to it.

By the way, the issues I was seeing before with the swap buffer timeout thing and unresponsiveness seem to have been due to my having used the file-backed device on a zfs filesystem for additional swap.

I was not sure how to mark the thread Solved but I no longer need assistance on this issue. Thank you
 
ikbendeman said:
Did it not work when you tried to pass KERNCONF from commandline? Either way, glad it worked out.
Click to expand...

For some reason, I got the error that appears in the code above
ERROR: No kernel \"${KERNCONF}\" to install.
I guess it has to do with something not lining up correctly when I moved the build over to a new system. Luckily it was as easy as setting INSTALLKERNEL appropriately
 
You must log in or register to reply here.
Back
Top