Solved Photoprism compilation errors on FreeBSD

[D-FENS]

How can I match the version of my ports tree (extracted with portsnap) to a git commit in the repository at https://git.FreeBSD.org/ports.git ?

I am currently having trouble compiling the port Photoprism https://github.com/huo-ju/photoprism-freebsd-port which depends on science/libtensorflow1 (currently renamed to science/py-tensorflow) but I compiled it successfully on Dec 21, 2021 against a ports tree fetched with ports-mgmt/portsnap. My strategy is to revert the ports tree to that particular date, so I tried cloning the ports tree with git and reverted. Unfortunately I got compilation errors, which should not happen if I am compiling the same source, right?
I see the following possible reasons:
* Either the source I compiled back on Dec 21 (fetched with portsnap) had been retrieved from git yet earlier than that and the git repo at the same time did not represent the same source, or
* Something else is different now in my build environment (I have a newer kernel and userland, newer make, gcc, etc.), but still, I should be able to compile older ports reproducibly on my new system, right?

Any idea on how to reproduce my successful build from December?

Unfortunately I already deleted the ports tree I used back then, along with the whole jail.

 

[SirDice]

Unfortunately I got compilation errors, which should not happen if I am compiling the same source, right?

What about the versions of its dependencies? At the very least there's a different default Python version (3.7 vs. 3.8). Changes in its dependencies may be the cause of the errors?

My strategy is to revert the ports tree to that particular date, so I tried cloning the ports tree with git and reverted.

Hmm, how exactly did you do this? Because reverting a commit only removes that specific change. I can revert a single commit that was done months ago without removing any of the commits that were done after it. This is different from SVN for example where a revision is basically a point in time snapshot of a source tree, thus by reverting to a certain revision you are basically going back in time.

 

[D-FENS]

What about the versions of its dependencies? At the very least there's a different default Python version (3.7 vs. 3.8). Changes in its dependencies may be the cause of the errors?


Hmm, how exactly did you do this? Because reverting a commit only removes that specific change. I can revert a single commit that was done months ago without removing any of the commits that were done after it. This is different from SVN for example where a revision is basically a point in time snapshot of a source tree, thus by reverting to a certain revision you are basically going back in time.

"Revert" was probably the wrong word to use. What I meant is, I cloned the ports tree via git, then I checked out the commit I supposed to be right before I made my build in December. This restores the snapshot of the whole ports tree at that particular point in time. I also checked out the commit of Photoprism that was current at that time. If I understand the ports build correctly, Photoprism should pick up its dependencies (the ones from December) and the latter should also pick up whatever further dependencies they have from the ports tree, i.e. also their December versions.
Am I missing something here? Is it the build environment that is different?

Here are a couple of the commands I used:

rootDir="...my jail's root directory..."
photoprismPortDir="/tmp/photoprism-freebsd-port"

git clone https://git.FreeBSD.org/ports.git "${rootDir}/usr/ports"
git clone https://github.com/huo-ju/photoprism-freebsd-port.git "${rootDir}${photoprismPortDir}"

# 20 Dec, ports
git -C "${rootDir}/usr/ports" checkout 56db10a78fb31ee3b0c4900991c862b9c1733bbf
# 3 Dec, photoprism
git -C "${rootDir}${photoprismPortDir}" checkout 7895eaa099bb5512d453a79436e64a78a23025e2

# some devfs and fd mount commands to facilitate chroot....

chroot "${rootDir}" \
    sh -c ". /root/.profile; cd \"${photoprismPortDir}\"; LD_LIBRARY_PATH=.:/lib:/usr/libexec:/usr/local/lib   make   BATCH=yes OPTIONS_SET=AVX"

The result is a tensorflow build error (tensorflow is a dependency of Photoprism inside the ports tree). A sample of the error follows (there are a number of these duplicate symbols):

...
[0 / 4] [Prepa] BazelWorkspaceStatusAction stable-status.txt
ERROR: /tmp/photoprism-freebsd-port/work-default/photoprism-13cfcf6d2081045dd236db56ee842edce8332cca/docker/tensorflow/tensorflow-1.15.2/tensorflow/BUILD:563:1: Linking of rule '//tensorflow:libtensorflow_framework.so.1.15.2' failed (Exit 1)
ld: error: duplicate symbol: adler32
>>> defined at adler32.c
>>>            adler32.pic.o:(adler32) in archive bazel-out/host/bin/external/zlib/libzlib.pic.a
>>> defined at adler32.c
>>>            adler32.pic.o:(.text.adler32+0x0) in archive bazel-out/host/bin/external/zlib_archive/libzlib.pic.a
...

By the way, I use chroot instead of jexec because at the time I am building the jail is not yet started but this should not be relevant (I used the very same procedure in December and it worked).

 

[D-FENS]

I found an explanation and a workaround for my problem. In summary:

• On Dec 27-th 2021 Photoprism's FreeBSD port switched to an external dependency to the C-implementation of Tensorflow: libtensorflow1 (note that this is different from science/py-tensorflow). Until that point Tensorflow had been built internally by the port.
• libtensorflow1 is still not an official port, so one has to download, build and install it separately (this is what photoprism-freebsd-port's documentation states more or less).
• At the time of writing libtensorflow1 does not compile correctly (tested on FreeBSD 13.0-RELEASE) unless one applies a patch provided by photoprism-freebsd-port (found under "files/extra-patch-docker_tensorflow_tensorflow-1.15.2_third_party_repo.bzl"). I applied the patch by copying it to libtensorflow1's "files" directory, altering it so that the relative paths work in libtensorflow1 and registering it in the Makefile as described here. I tested both NOAVX and AVX2 configurations and they both worked in my test. It seems the AVX setting is not relevant to the build problem.

After patching, building and installing libtensorflow1 Photoprism compiles properly.
Thanks to everyone who was involved.