lang/rust compile issue

M

Mayhem30

I'm having the same issue with lang/rust as I did with devel/nasm before that got patched.

I use ports-mgmt/synth to compile my ports and ran in to an issue this morning.

While lang/rust was compiling, it used 16GB of my remaining ram and most of swap. After an hour it errored out and killed a few processes along with it.

I can't seem to find any open tickets on this issue. Any ideas on how to fix this issue?
 
I just finished building lang/rust of the current main branch a few minutes ago (1.66.1). No issues at all.
This won't help you much tho as my build server gets a generous 128 GB of RAM.
Based on the information you present it does seem like the system ran out of resources. What does the build log say?

What I mean to say: At least for 13.x amd64 the port seems to build without issues.
Welcome to the modern world of fancy programming languages...
 
After 53 minutes, it errored out with this :

Code: 
command did not execute successfully: "/construction/xports/lang/rust/work/_build/x86_64-unknown-freebsd/stage0-tools-bin/rust-installer" "combine" "--input-tarballs" "/construction/xports/lang/rust/work/_build/dist/rustc-1.66.1-x86_64-unknown-freebsd.tar.gz,/construction/xports/lang/rust/work/_build/dist/rust-std-1.66.1-x86_64-unknown-freebsd.tar.gz,/construction/xports/lang/rust/work/_build/dist/cargo-1.66.1-x86_64-unknown-freebsd.tar.gz,/construction/xports/lang/rust/work/_build/dist/rustfmt-1.66.1-x86_64-unknown-freebsd.tar.gz,/construction/xports/lang/rust/work/_build/dist/rls-1.66.1-x86_64-unknown-freebsd.tar.gz,/construction/xports/lang/rust/work/_build/dist/clippy-1.66.1-x86_64-unknown-freebsd.tar.gz,/construction/xports/lang/rust/work/_build/dist/rust-analysis-1.66.1-x86_64-unknown-freebsd.tar.gz" "--rel-manifest-dir=rustlib" "--legacy-manifest-dirs=rustlib,cargo" "--product-name=Rust" "--success-message=rust installed." "--package-name=rust-1.66.1-x86_64-unknown-freebsd" "--non-installed-overlay" "/construction/xports/lang/rust/work/_build/tmp/tarball/rust/x86_64-unknown-freebsd/overlay" "--output-dir" "/construction/xports/lang/rust/work/_build/dist" "--work-dir" "/construction/xports/lang/rust/work/_build/tmp/tarball/rust/x86_64-unknown-freebsd"
expected success, got: signal: 9 (SIGKILL)


Traceback (most recent call last):
  File "/construction/xports/lang/rust/work/rustc-1.66.1-src/x.py", line 28, in <module>
    bootstrap.main()
  File "/construction/xports/lang/rust/work/rustc-1.66.1-src/src/bootstrap/bootstrap.py", line 936, in main
    bootstrap(help_triggered)
  File "/construction/xports/lang/rust/work/rustc-1.66.1-src/src/bootstrap/bootstrap.py", line 922, in bootstrap
    run(args, env=env, verbose=build.verbose, is_bootstrap=True)
  File "/construction/xports/lang/rust/work/rustc-1.66.1-src/src/bootstrap/bootstrap.py", line 166, in run
    raise RuntimeError(err)
RuntimeError: failed to run: /construction/xports/lang/rust/work/_build/bootstrap/debug/bootstrap dist --jobs=4
*** Error code 1

Stop.
make: stopped in /xports/lang/rust
 
Mayhem30 said:
it used 16GB of my remaining ram and most of swap.
Click to expand...
How much swap space is there? Try adding a bunch more. Preferably more memory of course, but more swap will do too. More swap won't make it build faster obviously but it will be enough to actually get the build to finish.

On a 16GB machine with 16GB of swap I get notifications more than half of my swap is in use, but at least it correctly finishes the build. That is, if I don't happen to have another large build going simultaneously.
 
Ok, dmesg did indeed confirm that I ran out of swap space :

Code: 
swap_pager: out of swap space
swp_pager_getswapspace(21): failed
pid 50536 (rust-installer), jid 0, uid 0, was killed: failed to reclaim memory
pid 2214 (mysqld), jid 0, uid 88, was killed: failed to reclaim memory

My swap is currently set to 4GB, will try setting it to 16GB and report back.
 
SirDice said:
On a 16GB machine with 16GB of swap
Click to expand...
Same config here, though I'd mention 16 CPUs as it will affect the jobs hence the simultaneous c++ processes. While I normally install these big build deps (gccs, llvms, rust) from pkg, I tried to build lang/rust after reading this thread -- don't think it even touched the swap (was checking periodically the top output, mostly when it was building the llvm parts of rust), all built normally:
make 19569.52s user 365.07s system 1158% cpu 28:40.40 total
Click to expand...
Note: the VM did not run anything else at the moment (almost, there's an irssi running, but it doesn't consume any really visible amount of memory).
 
yuripv79 said:
Same config here, though I'd mention 16 CPUs as it will affect the jobs hence the simultaneous c++ processes.
Click to expand...
Drop down the number of cores or at least the number of cores being used to build it. My lowly old i5 has 2 cores, 4 threads. So I have -j 4, less threads running concurrently would also mean less memory being used by the build process in total.
 
Ok, I wasn't able to adjust the swap space as I use gmirror on my SSD's .. and wasn't comfortable messing with that on a live server.

I managed to solve my issue by enabling the FreeBSD repo and getting lang/rust from there.
 
In case this is not going to be a singular exceptional case for you you might want to consider using ports-mgmt/poudriere-devel/ instead. It has a fetching option. This way poudriere will automatically download certain packages from the official FreeBSD repos rathern building them locally.
This can be helpful on resource constrained build severs as it provides you with selective choices which packages are re-used from the official binary repos vs. which ones you build yourself. Typical use-cases (other than for port testing) include fetching compilers such as lang/rust rather than sitting through their compile times yourself.

That being said, I don't know whether ports-mgmt/synth has a similar feature.
 
you can add swap files if you can't easily create/resize partition or use a zvol
performance is worse with swap files but will do the job
 
jbo said:
In case this is not going to be a singular exceptional case for you you might want to consider using ports-mgmt/poudriere-devel/ instead. It has a fetching option. This way poudriere will automatically download certain packages from the official FreeBSD repos rathern building them locally.

That being said, I don't know whether ports-mgmt/synth has a similar feature.
Click to expand...

Yes, ports-mgmt/synth also has that feature, and that's what I used to solve this issue.
 
You must log in or register to reply here.
Back
Top