Package new port without installing on build host

[Situation]

Hello, I'm new to FreeBSD and I'm trying to port a piece of software.

Here is my use case: I have a FreeBSD box with one jail. I would like to build a piece of software on the main host (outside the jail), package it, and install it inside the jail as a package. I could compile it directly and install it in the jail's filesystem, but I thought I would make a port so that other people could benefit from it.

I wrote the Makefile, pkg-plist, and so on. However, I'm a bit confused, because when I run make, it builds the software and runs the install target of the sub-makefile, even if I don't run make install. Also, when I run make stage, the files listed in pkg-plist don't appear in the work directory, and then make package complains that the files are not in work/stage/usr/local.

I'm also confused because I saw contradictions in the documentation, on the forums, and in the ports' makefiles. On one side, this post from 2016 says that make package will package a port without installing it (is this still up to date?).

Also, the porter's handbook says "Make sure that make package can be run as a normal user (that is, not as root).", which implies that the command doesn't install the port (since this cannot be done as a normal user).

On the other hand, /usr/ports/Mk/bsd.port.mk contains the following in the long comment at the beginning of the file:
package - Create a package from an _installed_ port.
Also, the package-noinstall target exists, which suggests that package does indeed install the port. However this target is not documented.

What would be the best way to accomplish what I described above? Thank you for your help!

 

[nxjoseph]

but I thought I would make a port so that other people could benefit from it.

Thanks for that good thought.

However, I'm a bit confused, because when I run make, it builds the software and runs the install target of the sub-makefile, even if I don't run make install.

The install target should only install port package files into work/stage directory, not to the system. There is also NO_INSTALL variable to install files on your own by writing an install target for Makefile but I am not sure if that can help you.

Also, when I run make stage, the files listed in pkg-plist don't appear in the work directory, and then make package complains that the files are not in work/stage/usr/local.

How did you generate the pkg-plist, I usually run # make makeplist | tail -n +2 > pkg-plist. The tail command is for ignoring first line of # make makeplist which is /you/have/to/check/what/makeplist/gives/you. I can give it a glance to your work-in-progress port if you can share it.

If there are a few package files, then you can specify them in Makefile like so: PLIST_FILES= bin/program and in this case, there is no need for pkg-plist.

I'm also confused because I saw contradictions in the documentation, on the forums, and in the ports' makefiles. On one side, this post from 2016 says that make package will package a port without installing it (is this still up to date?).

Yes, # make package creates a port.pkg package file in work/pkg/port.pkg.

Also, the porter's handbook says "Make sure that make package can be run as a normal user (that is, not as root).", which implies that the command doesn't install the port (since this cannot be done as a normal user).

Yes, make package can be run with normal user, i just tested it.

Thank you for your help!

You are welcome.

I believe you can install a package to a jail with pkg(8) on the host.

     -j ⟨jail name or id⟩, --jail ⟨jail name or id⟩
             pkg will execute in the given ⟨jail name or id⟩, where name
             matches “jls name” and id matches “jls jid”.  See jail(8) and
             jls(8).

# pkg -j jailname install ./port.pkg

 

[Situation]

Hi, thank you for your answer!

How did you generate the pkg-plist

I wrote it manually. make makeplist only gives /you/have/to/check/what/makeplist/gives/you. Is it generating the list based on the files added in work/stage? If so, then it's normal that it doesn't see anything, because they get installed in the base system directly.

Here is what I have so far:
Makefile

PORTNAME=       XD
DISTVERSION=    0.4.6
CATEGORIES=     net-p2p

USE_GITHUB=     yes
GH_ACCOUNT=     majestrate
DISTVERSIONPREFIX=      v

MAINTAINER=     alpou@tutanota.com
COMMENT=        Bittorrent client for the I2P network
WWW=            https://xd-torrent.readthedocs.io/en/latest/

LICENSE=        MIT
LICENSE_FILE=   ${WRKSRC}/LICENSE

USES=           gmake
ALL_TARGET=     build
BUILD_DEPENDS=  go:lang/go

USERS=          _XD
GROUPS=         _XD

.include <bsd.port.mk>

Note: I modified my local copies of UIDs and GIDs to include _XD.

pkg-plist

bin/xd
bin/xd-cli
@sample %%ETCDIR%%/xd.ini

files/patch-Makefile

--- Makefile.orig    2025-03-25 03:21:56 UTC
+++ Makefile
@@ -43,10 +43,10 @@ $(XD): $(WEBUI_CORE)
 
 
 $(XD): $(WEBUI_CORE)
-    $(GO) build -a -ldflags "-X xd/lib/version.Git=$(GIT_VERSION)" -tags='$(TAGS)' -o $(XD)
+    $(GO) build -a -buildvcs=false -ldflags "-X xd/lib/version.Git=$(GIT_VERSION)" -tags='$(TAGS)' -o $(XD)
 
 dev: $(WEBUI_CORE)
-    $(GO) build -race -v -a -ldflags "-X xd/lib/version.Git=$(GIT_VERSION)" -tags='$(TAGS)' -o $(XD)
+    $(GO) build -race -v -a -buildvcs=false -ldflags "-X xd/lib/version.Git=$(GIT_VERSION)" -tags='$(TAGS)' -o $(XD)
 
 $(CLI): $(XD)
     $(RM) $(CLI)
@@ -81,7 +81,7 @@ no-webui:
 webui: $(WEBUI_CORE)
 
 no-webui:
-    $(GO) build -ldflags "-X xd/lib/version.Git=$(GIT_VERSION)" -o $(XD)
+    $(GO) build -buildvcs=false -ldflags "-X xd/lib/version.Git=$(GIT_VERSION)" -o $(XD)
 
 install: $(XD) $(CLI)
     $(MKDIR) $(PREFIX)/bin

Note: I'm not an expert in Go, so I don't know if this patch is the best solution. Without it, the compiler complains that it's not able to determine the version from Git.

About Makefile: Now that I think about it, could the weird behaviour be caused by ALL_TARGET= build? I added it because otherwise, make complains that the sub-makefile doesn't have an all target, but I'm not quite sure what this target is usually supposed to do.

 

[nxjoseph]

Is it generating the list based on the files added in work/stage?

I think so.

The Makefile is seems complex to me and it confuses me, so I didn't make use of it here. The port now builds fine but without webui GO tag, enabling it causes build to fail, it says missing files, I didn't solve it yet. It only produces xd-torrent binary which is actually the xd itself but because i changed portname to it, like Arch Linux's AUR did. I linked xd-torrent to xd and xd to xd-cli but I am not sure if linking the original binary to cli one right.

I also noticed that security/i2p is going to be deprecated soon, see https://freshports.org/security/i2p for more info.

If this port depends on i2p, which I believe it does, then this port may not be useful until security/i2p gets a new maintainer, current version in ports tree is very old but upstream is active.

This is what i get when i run it without i2p installed.

[NFO] 2025-03-31 14:45:13.76140988 +0300 +03 m=+0.000340806     starting XD-0.4.6
[NFO] 2025-03-31 14:45:13.761658319 +0300 +03 m=+0.000589246    loaded config torrents.ini
[NFO] 2025-03-31 14:45:13.761665914 +0300 +03 m=+0.000596851    Ensure filesystem storage
[NFO] 2025-03-31 14:45:13.76174842 +0300 +03 m=+0.000679387     Swarm netLoop starting
[NFO] 2025-03-31 14:45:13.761781523 +0300 +03 m=+0.000712479    RPC enabled
[NFO] 2025-03-31 14:45:13.761881482 +0300 +03 m=+0.000812428    create new i2p session with 127.0.0.1:7656
[NFO] 2025-03-31 14:45:13.761905447 +0300 +03 m=+0.000836364    opening i2p session
[NFO] 2025-03-31 14:45:13.762082793 +0300 +03 m=+0.001013799    create new i2p session with 127.0.0.1:7656
[ERR] 2025-03-31 14:45:13.762131074 +0300 +03 m=+0.001062011    failed to create i2p session: dial tcp 127.0.0.1:7656: connect: connection refused
^C[NFO] 2025-03-31 14:45:14.208972102 +0300 +03 m=+0.447903038  Interrupted
[NFO] 2025-03-31 14:45:14.2090189 +0300 +03 m=+0.447949817      Swarm closing

I didn't make use of users/groups either, i thought you would figure it out because I don't have the UIDs/GIDs change unlike you.

PORTNAME=    xd-torrent
DISTVERSIONPREFIX=    v
DISTVERSION=    0.4.6
CATEGORIES=    net-p2p

MAINTAINER=    alpou@tutanota.com
COMMENT=    Bittorrent client for the I2P network
WWW=        https://xd-torrent.readthedocs.io/en/latest/

LICENSE=    MIT
LICENSE_FILE=    ${WRKSRC}/LICENSE

USES=        go:modules
USE_GITHUB=    yes
GH_ACCOUNT=    majestrate
GH_PROJECT=    XD
GH_TUPLE=    golang:crypto:v0.17.0:golang_crypto/vendor/golang.org/x/crypto \
        golang:sys:v0.15.0:golang_sys/vendor/golang.org/x/sys \
        kr:fs:v0.1.0:kr_fs/vendor/github.com/kr/fs \
        leonelquinteros:gotext:v1.3.1:leonelquinteros_gotext/vendor/gopkg.in/leonelquinteros/gotext.v1 \
        mattn:kinako:332c0a7e205a:mattn_kinako/vendor/github.com/mattn/kinako \
        pkg:sftp:v1.13.5:pkg_sftp/vendor/github.com/pkg/sftp \
        zeebo:bencode:v1.0.0:zeebo_bencode/vendor/github.com/zeebo/bencode

PLIST_FILES=    bin/xd \
        bin/xd-cli \
        bin/xd-torrent

# add "-tags='webui'" for webUI but it fails to compile then
#GO_BUILDFLAGS+=

post-install:
    @(cd ${STAGEDIR}${PREFIX}/bin && \
        ${LN} xd-torrent xd && \
        ${LN} xd xd-cli)

.include <bsd.port.mk>

 

[Situation]

Do you think the port should be called xd-torrent instead of XD? It's true that software names with capital letters don't look right ^.^

If this port depends on i2p, which I believe it does

Any I2P implementation works. i2p is the original, but i2pd is also popular and the FreeBSD port is maintained. I think there is also i2p+ and others, but I'm not familiar with these ones. Also, XD doesn't have to be running on the same machine as the I2P router to which it connects.

I don't have the UIDs/GIDs change unlike you

Oh sorry, here (maybe _XD can be changed to _xd or _xd-torrent):

/usr/ports # git diff
diff --git a/GIDs b/GIDs
index 81a5275cc3c..5c1d51e5b7b 100644
--- a/GIDs
+++ b/GIDs
@@ -780,7 +780,7 @@ beehive:*:832:
 # free: 836
 # free: 837
 # free: 838
-# free: 839
+_XD:*:839
 netdisco:*:840:
 tcpcryptd:*:841:
 munin:*:842:
diff --git a/UIDs b/UIDs
index f79658aa4f9..522de283529 100644
--- a/UIDs
+++ b/UIDs
@@ -786,7 +786,7 @@ beehive:*:832:832::0:0:beehive user:/nonexistent:/usr/sbin/nologin
 # free: 836
 # free: 837
 # free: 838
-# free: 839
+_XD:*:839:839::0:0:XD daemon:/nonexistent:/usr/sbin/nologin
 netdisco:*:840:840::0:0:netdisco daemon:/usr/local/etc/netdisco:/bin/sh
 tcpcryptd:*:841:841::0:0:tcpcrypt daemon:/nonexistent:/usr/sbin/nologin
 munin:*:842:842::0:0:Munin:/var/munin:/usr/sbin/nologin

It's strange, I was able to compile the binary with my Makefile. It's only the package that I couldn't generate. Are you able to generate the package with the one you wrote?

Also, what are the lines in GH_TUPLE for?

 

[nxjoseph]

Do you think the port should be called xd-torrent instead of XD? It's true that software names with capital letters don't look right ^.^

Yes, I think it should called xd-torrent but the choice is yours.

Any I2P implementation works. i2p is the original, but i2pd is also popular and the FreeBSD port is maintained. I think there is also i2p+ and others, but I'm not familiar with these ones. Also, XD doesn't have to be running on the same machine as the I2P router to which it connects.

Do i need to run i2p to use the xd?

It's strange, I was able to compile the binary with my Makefile. It's only the package that I couldn't generate. Are you able to generate the package with the one you wrote?

Yes, I can generate xd package with my Makefile. I actually try to make xd's own Makefile to get it work, this is currently work-in-progress.

Also, what are the lines in GH_TUPLE for?

They are dependencies of XD program, also listed in distinfo.

 

[Situation]

Do i need to run i2p to use the xd?

As you saw before, if XD is not able to connect to an I2P router, it prints an error ([ERR] [...] failed to create i2p session). But you can still see that it runs even if it's not connected, so it's fine.

When I try the Makefile you wrote, I get the following:

make: "/usr/ports/net-p2p/xd-torrent/Makefile" line 33: Invalid line type
make: Fatal errors encountered -- cannot continue
make: stopped in /usr/ports/net-p2p/xd-torrent

It doesn't like the post-install target, but I really don't understand why.

 

[T-Aoki]

To create pkgs locally, ports-mgmt/poudriere (or ports-mgmt/poudriere-devel would help. Limiting with poudriere, I recommend -devel version over regular version.

And if what you need is to build pkgs wanted, and your host (bare-metal environment) and jail (you said only one) are running as the same version of FreeBSD base (would be common for security purposes), my use-case intruducing here could be of some help.

With pkg-plist issue (fail to package), usually there's some problem in it, if pkg-plist exists. (There are case of auto plist and/or PLIST_FILES= line in Makefile.)

And in case that any of listed files in pkg-plist cannot be found but succeed to package, it's because conditional items.
These are often used for files only built with specific options/flavors are specified. Ports framework has tmpplist feature for it.

For example, lines starting from @comment is ignored as comments.
In lines starting from %%option-name%%, %%option-name%% is replaced with @comment automatically in tmpplist and doesn't appear the final temporary pkg-plist if the option option-name is disabled, and replaced with blank (simply deleted) if the option option-name is enabled.

This syntax can be used to replace it to something wanted (or @comment).
If you want to see quite complicated example, see x11/linux-nvidia-libs/Makefile and corelated pkg-plist. Would be confused, though. So do not recommend until you've understood some basics. Was quite hard to start submitting patches for upgrading them.

 

[Situation]

When I try the Makefile you wrote, I get the following:

Solved this. It's because when I copied and pasted the Makefile from the browser to a text editor, spaces were pasted instead of tabs.

Now however, I am asked to log in as root to make the package:
Switching to root credentials to create /var/db/ports/security_ca_root_nss
But even when I log as root, the procedure fails with
mkdir: /usr/ports/ports-mgmt/portconfig/work: Permission denied
mkdir: /usr/ports/security/ca_root_nss/work: Permission denied
I thought building a port and packaging it didn't require root privileges. What is going on?

my use-case intruducing here could be of some help

Thanks for the tips and tricks! I will have a look and give poudriere a shot.

 

[nxjoseph]

Now however, I am asked to log in as root to make the package:
Switching to root credentials to create /var/db/ports/security_ca_root_nss
But even when I log as root, the procedure fails with
mkdir: /usr/ports/ports-mgmt/portconfig/work: Permission denied
mkdir: /usr/ports/security/ca_root_nss/work: Permission denied
I thought building a port and packaging it didn't require root privileges. What is going on?

The /var/db/ports lines means that you haven't installed the port's dependencies and it wants to configure options of required dependencies of port and then install. /var/db/ports is to store port options. Generally, /usr/ports is owned by root so you get permission denied error, try the same in ports tree that located in your $HOME so that you can read/write.

I am went a bit further and made use of the port's own Makefile, it installs XD and XD-CLI programs. I am attaching my port directory as zip archive here. I wonder what use cases does this user/group have, or do you also need a rc.d service script too? Why make user and group? I think we can work together implementing user/group and rc.d script if needed.

 

[Situation]

I am attaching my port directory as zip archive here.

Wonderful, thanks! I'm having a look. I'm wondering about something you did in your zip. You define PREFIX=${PREFIX} and STAGEDIR=${STAGEDIR} in the environment and replace $(PREFIX) with $(STAGEDIR)$(PREFIX) in the submakefile. Wouldn't it be simpler to only define PREFIX=${PREFIX}${STAGEDIR}?

I wonder what use cases does this user/group have, or do you also need a rc.d service script too?

Yes exactly. I started writing a service script, but because XD doesn't daemonize itself, I'm stuck with the same problem as in that thread. I tried using daemon, but it didn't work. I have to retry when I have time and find the right options.

 

[nxjoseph]

You define PREFIX=${PREFIX} and STAGEDIR=${STAGEDIR} in the environment and replace $(PREFIX) with $(STAGEDIR)$(PREFIX) in the submakefile. Wouldn't it be simpler to only define PREFIX=${PREFIX}${STAGEDIR}?

I don't think so and it didn't work that way either. I think PREFIX is hard-coded to /usr/local. Directly defining PREFIX in port's Makefile also gives error. I also noticed that make -V MAKE_ENV already contains PREFIX so I removed it.

root@pkg:/usr/ports/net-p2p/xd-torrent # grep PREFIX Makefile
DISTVERSIONPREFIX=      v
                PREFIX=${STAGEDIR}${PREFIX}
PREFIX=         ${STAGEDIR}${PREFIX}
        @${STRIP_CMD} ${STAGEDIR}${PREFIX}/bin/XD
root@pkg:/usr/ports/net-p2p/xd-torrent # make
Variable PREFIX is recursive.
        in /usr/ports/Mk/bsd.port.mk:1538
        in /usr/share/mk/bsd.port.mk:31
        in /usr/ports/net-p2p/xd-torrent/Makefile:43

make: stopped in /usr/ports/net-p2p/xd-torrent
root@pkg:/usr/ports/net-p2p/xd-torrent #

root@pkg:/usr/ports/net-p2p/xd-torrent # grep PREFIX Makefile
DISTVERSIONPREFIX=      v
                PREFIX=${STAGEDIR}${PREFIX}
        @${STRIP_CMD} ${STAGEDIR}${PREFIX}/bin/XD
root@pkg:/usr/ports/net-p2p/xd-torrent # make
===>  Staging for xd-torrent-0.4.6
===>   Generating temporary packing list
mkdir -p /usr/local/bin
install XD /usr/local/bin
cp -P XD-CLI /usr/local/bin
strip: open /wrkdirs/usr/ports/net-p2p/xd-torrent/work/stage/usr/local/bin/XD failed: No such file or directory
*** Error code 1

Stop.
make: stopped in /usr/ports/net-p2p/xd-torrent
root@pkg:/usr/ports/net-p2p/xd-torrent #

Yes exactly. I started writing a service script, but because XD doesn't daemonize itself, I'm stuck with the same problem as in that thread. I tried using daemon, but it didn't work. I have to retry when I have time and find the right options.

I have a port and it has a rc.d script which we made it together with help of VVD. Can you look at it, maybe it can help you figure it out.

Here it is: https://cgit.freebsd.org/ports/tree/mail/hydroxide/files/hydroxide.in

I also wonder what is the webui in that program and how it works. Do you think it's expected to only have binaries in pkg-list?

EDIT: webui worked too, also i can add option to disable webui. I didn't download anything with the port yet, did you? I have security/i2pd running, also maybe we'll add dependency on it or some other i2p port.

 

[T-Aoki]

Simple prediction.
Does replacing PREFIX=${STAGEDIR}${PREFIX} with PREFIX=${STAGEDIR}${LOCALBASE} help here, for Variable PREFIX is recursive case?

See bsd.port.mk for details.

 

[Andriy]

Also, the package-noinstall target exists, which suggests that package does indeed install the port. However this target is not documented.

Despite not being documented (and I trust you on this), it does exactly what you want.

 

[Situation]

Hello, I finally found some time to continue working on the port!

Wouldn't it be simpler to only define PREFIX=${PREFIX}${STAGEDIR}?

Oops, I meant PREFIX=${STAGEDIR}${PREFIX}. But it doesn't work anyway, so it doesn't matter.

---

Now, this is what I have so far (see attached archive). I'm able to build the binary and create a package. However there are still (at least) two problems:
- the rc script files/xd-torrent.in is not included in the package and the commands in the post-install target don't seem to be executed (they are not in +MANIFEST or +COMPACT_MANIFEST inside the package and there is no /var/run/xd-torrent/ in the directory tree inside either)
- If I copy the files manually in the jail where I want to run the port and I try to start it with service xd-torrent onestart, I get the following in /var/log/daemon.log:

[...] xd_torrent[54728]: initgroups(_i2pd,255): Operation not permitted
[...] xd_torrent[54219]: daemon: failed to set user environment

If I remove -u ${xd_torrent_user} from the rc script, it works.

Thanks again for the all help!

 

[Situation]

Update on my previous post:
I noticed that making the port generates the scripts users-groups-install.sh and users-groups-deinstall.sh in the work directory, which take care of creating the daemon user defined in UIDs and GIDs. However, they are also not included in the generated package archive.

Also, I noticed that when I wrote my previous post, I didn't have the right version of the rc script in my jail. With the version generated from xd-torrent.in in the archive I uploaded, the message in /var/log/daemon.log is this:

[...] xd-torrent[44858]: initgroups(_xd,839): Operation not permitted
[...] xd-torrent[43954]: daemon: failed to set user environment

However the behavior is the same.

 

[Situation]

Update to my recent posts:
I figured out that when ${name}_user is defined in the rc script, the service is already run as that user, and this is why daemon is unable to change the user, since it's already not running as root (I found that it's documented under ${name}_user in rc.subr(8)). And indeed, when the -u flag is removed from the call to daemon, xd runs as _xd. So the second bullet point in post #15 is solved.

 

[Situation]

I figured out how to fix the problems I described above, but I now have another problem. Because this problem seems to be self-contained, I will stop writing in this thread and start a new thread for the last problem.