Where did the free space go after failed port build

J

jwhendy

Hi,


I dual boot on a MacBook and have 10GB set aside for my FreeBSD partition. I mainly just wanted it there for learning purposes (shell and programming) and the ability to use open source software not available on OS X.

In any case, I tried to build Openoffice.org last night and it failed due to lack of space. Prior to building 'df' returned about 67% or so used on /usr.

After the build failed 'df' returned 100% used on /usr.

I took note of the ports (dependencies, I'm assuming) that DID get built by portmaster and removed each of them via 'portmaster -e pkgName'. These were mostly p5-something-something ports, boost, zip, and a couple others.

I also issued 'portmaster --clean-distfiles' and 'portmaster --check-port-dbdir' and deleted any things I didn't need. I think I did 'portmaster -s' as well but don't think it returned anything.

Anyway, even after doing that, I'm still at 99%. I'm guessing that there's some temporary stuff that was built for Oo_Org that's still in existence somewhere. Where should I look and how can I free up that space - I should be able to return to my 'pre-Oo_Org' state of around %70 used on /usr since I don't have anything installed that wasn't installed before.

Thanks for any help.


-John
 
You can run du command to get list of top 10 files in /usr:
Code: 
find /use -type f | xargs ls -s | sort -rn | awk '{size=$1/1024; printf("%dMb %s\n", size,$2);}' | head
This will give out good idea what is eating all the space. Also, check out portsclean to clean up the temporary work directories:
Code: 
portsclean -C
Also check out:
portsclean
And pkg_cutleaves finds installed 'leaf' packages
pkg_cutleaves
 
I have /var on a separate partition within my FreeBSD slice, so that wouldn't contribute to my issue. It's got to be something within /usr itself.

I could do your same search within /usr. Not being hugely familiar with FreeBSD, I don't whether or not that search would turn up anything productive since it's searching /usr. For instance, if all the largest files are binaries of my compiled ports, it would just return my biggest applications, which wouldn't help me identify what compiling Oo_Org filed away somewhere.

Does that make sense?


-John
 
Woah, weird... did you edit your post? I could have sworn your post said to check /var, not /usr. If I completely hallucinated, my apologies on my post. I wrote it as thought you had told me to check /var, not /usr.

Good call on the portsclean; I could indeed try that. I've tried to stay away from the portupgrade sweet, but if portsclean is the only way to accomplish that task, I'm open to it.


-John
 
A "make distclean" in the Oo_O port directory might be worth trying, that should recursively clean all dependencies and delete their distfiles. (As should the portmaster commands ... but you never know.)
 
Cool - I'll try that tonight.

Doing what you suggest should do the same as 'portmaster --clean-distfiles' is what you're saying, then?

Maybe make clean in Oo_O port dir might do it now that I think of it... it was 'make clean'[ed] when portmaster started, then it went on with dependencies and installed as many as it could... then failed. So perhaps there's junk in the Oo_O port and whatever the last dependency was that failed?

I don't think 'pormaster --clean-distfiles' cleans the ports themselves so this is something new for me to try. Thanks!


-John
 
If I recall this correctly, distclean should do a clean and then delete the distfiles for the port you do it in and all dependencies (even installed ones) - which should cover everything.
 
[cmd=]make distclean[/cmd] is not recursive. It only works on the port whose directory you're in, plus any of its distfiles in /usr/ports/distfiles. The latter being the only difference with [cmd=]make clean[/cmd].

Installing the entire portupgrade suite for portsclean is a bit much. Something like [cmd=]rm -rf /usr/ports/*/*/work[/cmd] followed by [cmd=]portmaster --clean-distfiles[-all][/cmd] does the same thing, I believe.
 
That would be excellent and a great way to eliminate my need to install portsclean. I've been trying to avoid installing portupgrade and friends just for the sake of simplicity.

So... each port creates /usr/ports/foo/bar/work when it is compiling, installs from the work directory, and then when 'make install' is run, it removes the ./work directory?

If so, then 'rm -rf /usr/ports/*/*/work' is the same as going into each port's respective directly and running 'make clean'?

If this is the case, then the only reason this might free up space for me is if Oo_O left it's work directly laying around due to the build failing (as 'make clean' was never run following a successful build). Am I getting all of this straight?


-John
 
'make clean' cleans out the work directory under a port, yes (not 'make install', which is why you run 'make install clean'). I think it also clears away some small files/patches, but the bulk of the data is in 'work'. It's the staging area for the installation, and if you remove all of the work directories, you've cleaned all of the ports in one go. This is what portsclean actually does: it even prints that it's running [cmd=]rm -rf /usr/ports/*/*/work[/cmd], iirc.

The non-portupgrade equivalents are:

[cmd=]portsclean -CD[/cmd] -> [cmd=]rm -rf /usr/ports/*/*/work && portmaster --clean-distfiles[/cmd]
[cmd=]portsclean -CDD[/cmd] -> [cmd=]rm -rf /usr/ports/*/*/work && portmaster --clean-distfiles-all[/cmd]
 
one alternate way to clean ports
............
/usr/ports/category/port/
/bin/rm -rf work/port-1.0
...........
that saves a .PLIST.flattened" (among others) in /work/ which show the installation,
and they do not take up much space.
you want the .install_done* file present before removing /work/port-1.0/;
 
Are those files important? Before reading this I had already done 'rm -rf /usr/ports/*/*/work'. I had great success! /usr went from 99% to 44%! I was worried I was going to have to repartition, which I just did and hate doing. Now I think I'll have everything I want installed on FreeBSD with room to spare!!

Thanks for all the help.


-John
 
When building a port the source will get extracted into those work directories. The port is build from there. Once the port has finished building you can safely remove the work directories (make clean usually does the trick).

If you're pressed for space in /usr you might want to set WRKDIRPREFIX to somewhere with more space. The ports will use that for the working directories.
 
They are only important if your custom or generic or
port-based port upgrade/build/install tools use them.
So far (2004 -- 2009) I've found them convenient to
keep around. (Building some ports seems to show a
"verifying install for PORT in /usr/ports/category/port "
(paraphrased)
sometimes. ... Not sure but that may be checking for
the files. OTOH some ports (a few only ! ) actually require
the actual build sources to remain in a dependent port to
build or configure properly )
 
jb_fvwm2 said:
(Building some ports seems to show a "verifying install for PORT in /usr/ports/category/port " (paraphrased) sometimes. ... Not sure but that may be checking for the files.
Click to expand...
No, it's checking if that specific port is installed. This has nothing to do with the work directories.

OTOH some ports (a few only ! ) actually require the actual build sources to remain in a dependent port to build or configure properly )
Click to expand...
True but those ports will extract the sources anyway if they're not available.

There's no need to keep the work directories around if you're done building.
 
Thanks for replies, all. It does seem that these work directories were costing me A LOT of space... to go from 99% disk capacity to 44% (of about 7.5GB on /usr) is pretty incredible.

I just compiled gimp last night and am up to about 66%. I did an update on my locate database and searched for '/usr/ports/*/*/work' and got a decent amount of candidates. The directories found were in those that were built for gimp.

So, my last question: why are some ports not cleaned after being installed? I went into 'gegl' for example, and did 'make clean' and the work directory was removed. This means that if 'make clean' was being run in each port properly by portmaster, the work directly would not be there anymore. So... why isn't it running 'make clean'?


-John
 
When you install a port using portmaster, it will run 'make install clean' on that port and all of its dependencies (unless -K is used).

If you install a port using 'make install clean' yourself, these targets will be inherited by all of the dependent ports (in other words: they will all be cleaned after being installed).

If you break off a build, some work directories may be left behind, though [cmd=]make clean-depends[/cmd] in the 'top level port' (the one pulling in the dependencies) should clear that right up.

See also "Default targets and their behaviors" in /usr/ports/Mk/bsd.port.mk.
 
SirDice said:
There's no need to keep the work directories around if you're done building.
Click to expand...

90 percent of the time I run
#ls -lac work
...to check how long before the build will complete... I
won't start it if the computer won't be on that long; if
the build will last so long as to slow down browsing sites
with too-large .HTM pages; and it the /work/ is
missing it means I might have pkg_add 'd instead.
.........
one or two other reasons that I cannot think of offhand.
...........
Including (some ports, wxgtk26 for example) I map out the
probable times beforehand so as to schedule them
half-hour by half-hour (IIRC)
 
I build everything inside a clean jail and package it. I've also set WRKDIRPREFIX, when I rebuild I clean that out. Never had any issues with ports.

The server that builds everything is on 24/7 anyway :e
 
This will get you rid of all work directories:
# find /usr/ports -type d -name work -exec rm -rf \{\} \;

For the future use WRKDIRPREFIX, I tend to set it to /usr/obj.
 
You must log in or register to reply here.
Back
Top