Package new port without installing on build host

[Situation]

Hello, I'm new to FreeBSD and I'm trying to port a piece of software.

Here is my use case: I have a FreeBSD box with one jail. I would like to build a piece of software on the main host (outside the jail), package it, and install it inside the jail as a package. I could compile it directly and install it in the jail's filesystem, but I thought I would make a port so that other people could benefit from it.

I wrote the Makefile, pkg-plist, and so on. However, I'm a bit confused, because when I run make, it builds the software and runs the install target of the sub-makefile, even if I don't run make install. Also, when I run make stage, the files listed in pkg-plist don't appear in the work directory, and then make package complains that the files are not in work/stage/usr/local.

I'm also confused because I saw contradictions in the documentation, on the forums, and in the ports' makefiles. On one side, this post from 2016 says that make package will package a port without installing it (is this still up to date?).

Also, the porter's handbook says "Make sure that make package can be run as a normal user (that is, not as root).", which implies that the command doesn't install the port (since this cannot be done as a normal user).

On the other hand, /usr/ports/Mk/bsd.port.mk contains the following in the long comment at the beginning of the file:
package - Create a package from an _installed_ port.
Also, the package-noinstall target exists, which suggests that package does indeed install the port. However this target is not documented.

What would be the best way to accomplish what I described above? Thank you for your help!

 

[nxjoseph]

but I thought I would make a port so that other people could benefit from it.

Thanks for that good thought.

However, I'm a bit confused, because when I run make, it builds the software and runs the install target of the sub-makefile, even if I don't run make install.

The install target should only install port package files into work/stage directory, not to the system. There is also NO_INSTALL variable to install files on your own by writing an install target for Makefile but I am not sure if that can help you.

Also, when I run make stage, the files listed in pkg-plist don't appear in the work directory, and then make package complains that the files are not in work/stage/usr/local.

How did you generate the pkg-plist, I usually run # make makeplist | tail -n +2 > pkg-plist. The tail command is for ignoring first line of # make makeplist which is /you/have/to/check/what/makeplist/gives/you. I can give it a glance to your work-in-progress port if you can share it.

If there are a few package files, then you can specify them in Makefile like so: PLIST_FILES= bin/program and in this case, there is no need for pkg-plist.

I'm also confused because I saw contradictions in the documentation, on the forums, and in the ports' makefiles. On one side, this post from 2016 says that make package will package a port without installing it (is this still up to date?).

Yes, # make package creates a port.pkg package file in work/pkg/port.pkg.

Also, the porter's handbook says "Make sure that make package can be run as a normal user (that is, not as root).", which implies that the command doesn't install the port (since this cannot be done as a normal user).

Yes, make package can be run with normal user, i just tested it.

Thank you for your help!

You are welcome.

I believe you can install a package to a jail with pkg(8) on the host.

     -j ⟨jail name or id⟩, --jail ⟨jail name or id⟩
             pkg will execute in the given ⟨jail name or id⟩, where name
             matches “jls name” and id matches “jls jid”.  See jail(8) and
             jls(8).

# pkg -j jailname install ./port.pkg

 

[Situation]

Hi, thank you for your answer!

How did you generate the pkg-plist

I wrote it manually. make makeplist only gives /you/have/to/check/what/makeplist/gives/you. Is it generating the list based on the files added in work/stage? If so, then it's normal that it doesn't see anything, because they get installed in the base system directly.

Here is what I have so far:
Makefile

PORTNAME=       XD
DISTVERSION=    0.4.6
CATEGORIES=     net-p2p

USE_GITHUB=     yes
GH_ACCOUNT=     majestrate
DISTVERSIONPREFIX=      v

MAINTAINER=     alpou@tutanota.com
COMMENT=        Bittorrent client for the I2P network
WWW=            https://xd-torrent.readthedocs.io/en/latest/

LICENSE=        MIT
LICENSE_FILE=   ${WRKSRC}/LICENSE

USES=           gmake
ALL_TARGET=     build
BUILD_DEPENDS=  go:lang/go

USERS=          _XD
GROUPS=         _XD

.include <bsd.port.mk>

Note: I modified my local copies of UIDs and GIDs to include _XD.

pkg-plist

bin/xd
bin/xd-cli
@sample %%ETCDIR%%/xd.ini

files/patch-Makefile

--- Makefile.orig    2025-03-25 03:21:56 UTC
+++ Makefile
@@ -43,10 +43,10 @@ $(XD): $(WEBUI_CORE)
 
 
 $(XD): $(WEBUI_CORE)
-    $(GO) build -a -ldflags "-X xd/lib/version.Git=$(GIT_VERSION)" -tags='$(TAGS)' -o $(XD)
+    $(GO) build -a -buildvcs=false -ldflags "-X xd/lib/version.Git=$(GIT_VERSION)" -tags='$(TAGS)' -o $(XD)
 
 dev: $(WEBUI_CORE)
-    $(GO) build -race -v -a -ldflags "-X xd/lib/version.Git=$(GIT_VERSION)" -tags='$(TAGS)' -o $(XD)
+    $(GO) build -race -v -a -buildvcs=false -ldflags "-X xd/lib/version.Git=$(GIT_VERSION)" -tags='$(TAGS)' -o $(XD)
 
 $(CLI): $(XD)
     $(RM) $(CLI)
@@ -81,7 +81,7 @@ no-webui:
 webui: $(WEBUI_CORE)
 
 no-webui:
-    $(GO) build -ldflags "-X xd/lib/version.Git=$(GIT_VERSION)" -o $(XD)
+    $(GO) build -buildvcs=false -ldflags "-X xd/lib/version.Git=$(GIT_VERSION)" -o $(XD)
 
 install: $(XD) $(CLI)
     $(MKDIR) $(PREFIX)/bin

Note: I'm not an expert in Go, so I don't know if this patch is the best solution. Without it, the compiler complains that it's not able to determine the version from Git.

About Makefile: Now that I think about it, could the weird behaviour be caused by ALL_TARGET= build? I added it because otherwise, make complains that the sub-makefile doesn't have an all target, but I'm not quite sure what this target is usually supposed to do.

 

[nxjoseph]

Is it generating the list based on the files added in work/stage?

I think so.

The Makefile is seems complex to me and it confuses me, so I didn't make use of it here. The port now builds fine but without webui GO tag, enabling it causes build to fail, it says missing files, I didn't solve it yet. It only produces xd-torrent binary which is actually the xd itself but because i changed portname to it, like Arch Linux's AUR did. I linked xd-torrent to xd and xd to xd-cli but I am not sure if linking the original binary to cli one right.

I also noticed that security/i2p is going to be deprecated soon, see https://freshports.org/security/i2p for more info.

If this port depends on i2p, which I believe it does, then this port may not be useful until security/i2p gets a new maintainer, current version in ports tree is very old but upstream is active.

This is what i get when i run it without i2p installed.

[NFO] 2025-03-31 14:45:13.76140988 +0300 +03 m=+0.000340806     starting XD-0.4.6
[NFO] 2025-03-31 14:45:13.761658319 +0300 +03 m=+0.000589246    loaded config torrents.ini
[NFO] 2025-03-31 14:45:13.761665914 +0300 +03 m=+0.000596851    Ensure filesystem storage
[NFO] 2025-03-31 14:45:13.76174842 +0300 +03 m=+0.000679387     Swarm netLoop starting
[NFO] 2025-03-31 14:45:13.761781523 +0300 +03 m=+0.000712479    RPC enabled
[NFO] 2025-03-31 14:45:13.761881482 +0300 +03 m=+0.000812428    create new i2p session with 127.0.0.1:7656
[NFO] 2025-03-31 14:45:13.761905447 +0300 +03 m=+0.000836364    opening i2p session
[NFO] 2025-03-31 14:45:13.762082793 +0300 +03 m=+0.001013799    create new i2p session with 127.0.0.1:7656
[ERR] 2025-03-31 14:45:13.762131074 +0300 +03 m=+0.001062011    failed to create i2p session: dial tcp 127.0.0.1:7656: connect: connection refused
^C[NFO] 2025-03-31 14:45:14.208972102 +0300 +03 m=+0.447903038  Interrupted
[NFO] 2025-03-31 14:45:14.2090189 +0300 +03 m=+0.447949817      Swarm closing

I didn't make use of users/groups either, i thought you would figure it out because I don't have the UIDs/GIDs change unlike you.

PORTNAME=    xd-torrent
DISTVERSIONPREFIX=    v
DISTVERSION=    0.4.6
CATEGORIES=    net-p2p

MAINTAINER=    alpou@tutanota.com
COMMENT=    Bittorrent client for the I2P network
WWW=        https://xd-torrent.readthedocs.io/en/latest/

LICENSE=    MIT
LICENSE_FILE=    ${WRKSRC}/LICENSE

USES=        go:modules
USE_GITHUB=    yes
GH_ACCOUNT=    majestrate
GH_PROJECT=    XD
GH_TUPLE=    golang:crypto:v0.17.0:golang_crypto/vendor/golang.org/x/crypto \
        golang:sys:v0.15.0:golang_sys/vendor/golang.org/x/sys \
        kr:fs:v0.1.0:kr_fs/vendor/github.com/kr/fs \
        leonelquinteros:gotext:v1.3.1:leonelquinteros_gotext/vendor/gopkg.in/leonelquinteros/gotext.v1 \
        mattn:kinako:332c0a7e205a:mattn_kinako/vendor/github.com/mattn/kinako \
        pkg:sftp:v1.13.5:pkg_sftp/vendor/github.com/pkg/sftp \
        zeebo:bencode:v1.0.0:zeebo_bencode/vendor/github.com/zeebo/bencode

PLIST_FILES=    bin/xd \
        bin/xd-cli \
        bin/xd-torrent

# add "-tags='webui'" for webUI but it fails to compile then
#GO_BUILDFLAGS+=

post-install:
    @(cd ${STAGEDIR}${PREFIX}/bin && \
        ${LN} xd-torrent xd && \
        ${LN} xd xd-cli)

.include <bsd.port.mk>

 

[Situation]

Do you think the port should be called xd-torrent instead of XD? It's true that software names with capital letters don't look right ^.^

If this port depends on i2p, which I believe it does

Any I2P implementation works. i2p is the original, but i2pd is also popular and the FreeBSD port is maintained. I think there is also i2p+ and others, but I'm not familiar with these ones. Also, XD doesn't have to be running on the same machine as the I2P router to which it connects.

I don't have the UIDs/GIDs change unlike you

Oh sorry, here (maybe _XD can be changed to _xd or _xd-torrent):

/usr/ports # git diff
diff --git a/GIDs b/GIDs
index 81a5275cc3c..5c1d51e5b7b 100644
--- a/GIDs
+++ b/GIDs
@@ -780,7 +780,7 @@ beehive:*:832:
 # free: 836
 # free: 837
 # free: 838
-# free: 839
+_XD:*:839
 netdisco:*:840:
 tcpcryptd:*:841:
 munin:*:842:
diff --git a/UIDs b/UIDs
index f79658aa4f9..522de283529 100644
--- a/UIDs
+++ b/UIDs
@@ -786,7 +786,7 @@ beehive:*:832:832::0:0:beehive user:/nonexistent:/usr/sbin/nologin
 # free: 836
 # free: 837
 # free: 838
-# free: 839
+_XD:*:839:839::0:0:XD daemon:/nonexistent:/usr/sbin/nologin
 netdisco:*:840:840::0:0:netdisco daemon:/usr/local/etc/netdisco:/bin/sh
 tcpcryptd:*:841:841::0:0:tcpcrypt daemon:/nonexistent:/usr/sbin/nologin
 munin:*:842:842::0:0:Munin:/var/munin:/usr/sbin/nologin

It's strange, I was able to compile the binary with my Makefile. It's only the package that I couldn't generate. Are you able to generate the package with the one you wrote?

Also, what are the lines in GH_TUPLE for?

 

[nxjoseph]

Do you think the port should be called xd-torrent instead of XD? It's true that software names with capital letters don't look right ^.^

Yes, I think it should called xd-torrent but the choice is yours.

Any I2P implementation works. i2p is the original, but i2pd is also popular and the FreeBSD port is maintained. I think there is also i2p+ and others, but I'm not familiar with these ones. Also, XD doesn't have to be running on the same machine as the I2P router to which it connects.

Do i need to run i2p to use the xd?

It's strange, I was able to compile the binary with my Makefile. It's only the package that I couldn't generate. Are you able to generate the package with the one you wrote?

Yes, I can generate xd package with my Makefile. I actually try to make xd's own Makefile to get it work, this is currently work-in-progress.

Also, what are the lines in GH_TUPLE for?

They are dependencies of XD program, also listed in distinfo.

 

[Situation]

Do i need to run i2p to use the xd?

As you saw before, if XD is not able to connect to an I2P router, it prints an error ([ERR] [...] failed to create i2p session). But you can still see that it runs even if it's not connected, so it's fine.

When I try the Makefile you wrote, I get the following:

make: "/usr/ports/net-p2p/xd-torrent/Makefile" line 33: Invalid line type
make: Fatal errors encountered -- cannot continue
make: stopped in /usr/ports/net-p2p/xd-torrent

It doesn't like the post-install target, but I really don't understand why.

 

[T-Aoki]

To create pkgs locally, ports-mgmt/poudriere (or ports-mgmt/poudriere-devel would help. Limiting with poudriere, I recommend -devel version over regular version.

And if what you need is to build pkgs wanted, and your host (bare-metal environment) and jail (you said only one) are running as the same version of FreeBSD base (would be common for security purposes), my use-case intruducing here could be of some help.

With pkg-plist issue (fail to package), usually there's some problem in it, if pkg-plist exists. (There are case of auto plist and/or PLIST_FILES= line in Makefile.)

And in case that any of listed files in pkg-plist cannot be found but succeed to package, it's because conditional items.
These are often used for files only built with specific options/flavors are specified. Ports framework has tmpplist feature for it.

For example, lines starting from @comment is ignored as comments.
In lines starting from %%option-name%%, %%option-name%% is replaced with @comment automatically in tmpplist and doesn't appear the final temporary pkg-plist if the option option-name is disabled, and replaced with blank (simply deleted) if the option option-name is enabled.

This syntax can be used to replace it to something wanted (or @comment).
If you want to see quite complicated example, see x11/linux-nvidia-libs/Makefile and corelated pkg-plist. Would be confused, though. So do not recommend until you've understood some basics. Was quite hard to start submitting patches for upgrading them.

 

[Situation]

When I try the Makefile you wrote, I get the following:

Solved this. It's because when I copied and pasted the Makefile from the browser to a text editor, spaces were pasted instead of tabs.

Now however, I am asked to log in as root to make the package:
Switching to root credentials to create /var/db/ports/security_ca_root_nss
But even when I log as root, the procedure fails with
mkdir: /usr/ports/ports-mgmt/portconfig/work: Permission denied
mkdir: /usr/ports/security/ca_root_nss/work: Permission denied
I thought building a port and packaging it didn't require root privileges. What is going on?

my use-case intruducing here could be of some help

Thanks for the tips and tricks! I will have a look and give poudriere a shot.

 

[nxjoseph]

Now however, I am asked to log in as root to make the package:
Switching to root credentials to create /var/db/ports/security_ca_root_nss
But even when I log as root, the procedure fails with
mkdir: /usr/ports/ports-mgmt/portconfig/work: Permission denied
mkdir: /usr/ports/security/ca_root_nss/work: Permission denied
I thought building a port and packaging it didn't require root privileges. What is going on?

The /var/db/ports lines means that you haven't installed the port's dependencies and it wants to configure options of required dependencies of port and then install. /var/db/ports is to store port options. Generally, /usr/ports is owned by root so you get permission denied error, try the same in ports tree that located in your $HOME so that you can read/write.

I am went a bit further and made use of the port's own Makefile, it installs XD and XD-CLI programs. I am attaching my port directory as zip archive here. I wonder what use cases does this user/group have, or do you also need a rc.d service script too? Why make user and group? I think we can work together implementing user/group and rc.d script if needed.

 

[Situation]

I am attaching my port directory as zip archive here.

Wonderful, thanks! I'm having a look. I'm wondering about something you did in your zip. You define PREFIX=${PREFIX} and STAGEDIR=${STAGEDIR} in the environment and replace $(PREFIX) with $(STAGEDIR)$(PREFIX) in the submakefile. Wouldn't it be simpler to only define PREFIX=${PREFIX}${STAGEDIR}?

I wonder what use cases does this user/group have, or do you also need a rc.d service script too?

Yes exactly. I started writing a service script, but because XD doesn't daemonize itself, I'm stuck with the same problem as in that thread. I tried using daemon, but it didn't work. I have to retry when I have time and find the right options.

 

[nxjoseph]

You define PREFIX=${PREFIX} and STAGEDIR=${STAGEDIR} in the environment and replace $(PREFIX) with $(STAGEDIR)$(PREFIX) in the submakefile. Wouldn't it be simpler to only define PREFIX=${PREFIX}${STAGEDIR}?

I don't think so and it didn't work that way either. I think PREFIX is hard-coded to /usr/local. Directly defining PREFIX in port's Makefile also gives error. I also noticed that make -V MAKE_ENV already contains PREFIX so I removed it.

root@pkg:/usr/ports/net-p2p/xd-torrent # grep PREFIX Makefile
DISTVERSIONPREFIX=      v
                PREFIX=${STAGEDIR}${PREFIX}
PREFIX=         ${STAGEDIR}${PREFIX}
        @${STRIP_CMD} ${STAGEDIR}${PREFIX}/bin/XD
root@pkg:/usr/ports/net-p2p/xd-torrent # make
Variable PREFIX is recursive.
        in /usr/ports/Mk/bsd.port.mk:1538
        in /usr/share/mk/bsd.port.mk:31
        in /usr/ports/net-p2p/xd-torrent/Makefile:43

make: stopped in /usr/ports/net-p2p/xd-torrent
root@pkg:/usr/ports/net-p2p/xd-torrent #

root@pkg:/usr/ports/net-p2p/xd-torrent # grep PREFIX Makefile
DISTVERSIONPREFIX=      v
                PREFIX=${STAGEDIR}${PREFIX}
        @${STRIP_CMD} ${STAGEDIR}${PREFIX}/bin/XD
root@pkg:/usr/ports/net-p2p/xd-torrent # make
===>  Staging for xd-torrent-0.4.6
===>   Generating temporary packing list
mkdir -p /usr/local/bin
install XD /usr/local/bin
cp -P XD-CLI /usr/local/bin
strip: open /wrkdirs/usr/ports/net-p2p/xd-torrent/work/stage/usr/local/bin/XD failed: No such file or directory
*** Error code 1

Stop.
make: stopped in /usr/ports/net-p2p/xd-torrent
root@pkg:/usr/ports/net-p2p/xd-torrent #

Yes exactly. I started writing a service script, but because XD doesn't daemonize itself, I'm stuck with the same problem as in that thread. I tried using daemon, but it didn't work. I have to retry when I have time and find the right options.

I have a port and it has a rc.d script which we made it together with help of VVD. Can you look at it, maybe it can help you figure it out.

Here it is: https://cgit.freebsd.org/ports/tree/mail/hydroxide/files/hydroxide.in

I also wonder what is the webui in that program and how it works. Do you think it's expected to only have binaries in pkg-list?

EDIT: webui worked too, also i can add option to disable webui. I didn't download anything with the port yet, did you? I have security/i2pd running, also maybe we'll add dependency on it or some other i2p port.

 

[T-Aoki]

Simple prediction.
Does replacing PREFIX=${STAGEDIR}${PREFIX} with PREFIX=${STAGEDIR}${LOCALBASE} help here, for Variable PREFIX is recursive case?

See bsd.port.mk for details.