portmaster issue when changing DISTDIR in /etc/make.conf

[bsdrocks]

I have a few freebsd servers and to ease maintenance I have mounted /usr/ports as read only on all my servers from a nfs share. I've then changed /etc/make.conf to:

WRKDIRPREFIX=/var/ports
DISTDIR=/var/ports/distfiles
PACKAGES=/var/ports/packages

The problem with this is that portmaster starts to whine when I run portmaster --clean-distfiles-all:

===>>> Gathering distinfo list for installed ports

===>>> Checking for stale distfiles
find: -delete: /var/ports/distfiles/: relative path potentially not safe

Since nothing useful turns up on google, I'm starting to think that I should have done something different to maintain a single ports tree, since if portmaster can't handle it, then it sure would be someone else having a problem with it..?

This also leads to another problem when I run portmaster --check-port-dbdir. It tells me that it doesn't look like /var/db/ports/distfiles is installed and asks me to remove it..? I didn't even know of that directory, but something tells me that I shouldn't remove it...


So am I doing something completely wrong or is portmaster letting me down? Maybe my question should be how I can migrate existings server to use a read only /usr/ports, /usr/src and /usr/obj directory instead? (I should rm -rf /usr/ports && portsnap fetch extract on the server sharing /usr/ports over nfs, right?

 

[SirDice]

Try setting WRKDIRPREFIX to something like /tmp/build/ (make sure there's enough free space there).

I usually leave packages and distfiles within /usr/ports/. You can always do a make fetch on the server to retrieve a specific distfile. It also keeps distfiles nice an tidy and all other machines can use it so there's no need to download a certain distfile multiple times.

 

[bsdrocks]

I tried setting it to /tmp/build instead, but it doesn't fix the portmaster issue. I guess you probably meant that I should leave all the other parameters and just make the server fetch the distfiles before building ports on the other machines? That would actually be the ideal solution, IF I could get it to fetch them automatically without needing to manually log in to the server and run 'make fetch' for all possible ports that needs to be updated.. I have previously mounted /usr/ports in read-write and it solved that issue, but then I'd have to make sure portsupdates for the "clients/machines" isn't conflicting (meaning that I would have to fetch all distfiles for one machine, then wait for it to finish and continue on the other machines - on at a time).

This made me thinking, would it be possible to generate a list of ports that need updating (and ALL related distfiles) from portmaster or another utility and then save that list in a file on a writable nfs share to be read by the server (from a cron script of course:e) and then recursively fetch the distfiles indicated in that list?

That way I could generate that list on the 'clients/machines' simultaneously on all the clients with a cron script and all could write to the same file (using >>, this makes sure that nothing is lost due to concurrency issues, right?) and an hour later or something the server could recursively fetch all distfiles and remove the file indicating that it has fetched all distfiles needed and portmaster -a is waiting for orders on the clients..

Am I on to something here or am I over-complicating this?

 

[bsdrocks]

Sorry for double posting, but I cannot figure out how I edit my post. :r

How about this:

Clients (run in cron @daily):

#!/bin/sh
portmaster --list-origins >> /root/share/portlist

Server (run in cron some time after @daily, is it possible to use @daily+1h or something instead of 0 1 * * *?):

#!/bin/sh

# Remove duplicate lines
awk '!x[$0]++' /root/share/portlist > /root/share/portlist

# Fetch distfiles for all ports (will skip those that already exists and I guess it will fetch if there is a newer distfile to be found..?)
cat /root/share/portlist | xargs -L 1 portmaster -F

# Remove the file and touch it to indicate that fetching of distfiles is finished
rm /root/share/portlist && touch /root/share/portlist

I'm trying it out now and it seems like it is working fine! I could even make a /root/share/portslist.now file that is read */10 minutes to fetch ports I need right away so I don't have to login to the main server to fetch that port, but that is probably overkill since it is usually faster to login in and actually see when the distfile is finished downloading.. Unless someone else has a better idea on how to solve read only /usr/ports and only fetch one distfile for all clients, I think I'll stay with this, it seems pretty nice! If I want, I could extend the clients script to actually update the ports with -DBATCH aswell, just adding a loop with sleep 600 and checking if the /root/share/portlist file is empty (= all distfiles fetched)

FreeBSD is great!

 

[dougb@]

FYI, I don't usually read the forums, so your best bet for portmaster questions is freebsd-ports@FreeBSD.org.

I have a few freebsd servers and to ease maintenance I have mounted /usr/ports as read only on all my servers from a nfs share. I've then changed /etc/make.conf to:

WRKDIRPREFIX=/var/ports
DISTDIR=/var/ports/distfiles
PACKAGES=/var/ports/packages

That all seems fine. I have custom values for WRKDIRPREFIX and PACKAGES myself, and I have used custom values for DISTDIR and NFS mounted my ports tree in the past, all with portmaster, without any problems.

The problem with this is that portmaster starts to whine when I run portmaster --clean-distfiles-all:

===>>> Gathering distinfo list for installed ports

===>>> Checking for stale distfiles
find: -delete: /var/ports/distfiles/: relative path potentially not safe

That's not a portmaster warning, the "find:" at the beginning is telling you that the warning is from find, which portmaster is calling internally. Apparently /var/ports/distfiles is a symlink? If so, then you're better off using the actual directory name in DISTDIR.

Since nothing useful turns up on google, I'm starting to think that I should have done something different to maintain a single ports tree, since if portmaster can't handle it, then it sure would be someone else having a problem with it..?

This also leads to another problem when I run portmaster --check-port-dbdir. It tells me that it doesn't look like /var/db/ports/distfiles is installed and asks me to remove it..? I didn't even know of that directory, but something tells me that I shouldn't remove it...

That's not a directory, it's a stray file. I'm not sure why it's there, but it shouldn't be, so you can safely delete it. Also, since you're apparently not changing the value of PORT_DBDIR this issue is unrelated to your wanting a read-only ports tree.

So am I doing something completely wrong or is portmaster letting me down? Maybe my question should be how I can migrate existings server to use a read only /usr/ports, /usr/src and /usr/obj directory instead? (I should rm -rf /usr/ports && portsnap fetch extract on the server sharing /usr/ports over nfs, right?

No, I don't see any reason for you to do that. If this post doesn't answer your questions and get you on the right track feel free to follow up.


Doug