Tensorflow package/binary install

japoc

Active Member

Reaction score: 1
Messages: 102

Hello everyone,

I needed to install Tensorflow 2.1 for a project. Tensorflow is not available in the packages and the port is a very old version of Tensorflow (1.4 if I remember well).
So I tried to compile it from source. And, actually, it seems to be possible to compile Tensorflow for FreeBSD just by downloading the release and issuing a single build command:

Code:
fetch 'https://github.com/tensorflow/tensorflow/archive/v2.1.0.tar.gz'
tar zxvf v2.1.0.tar.gz
cd tensorflow-2.1.0
bazel build -c opt --linkopt=-lexecinfo --host_linkopt=-lexecinfo //tensorflow/tools/pip_package:build_pip_package

(I found this info on these two links: https://github.com/tensorflow/tensorflow/issues/37345 and https://github.com/tensorflow/tensorflow/issues/23613#issuecomment-440757155 )

I did that and it started compiling the thing but the only problem is that it's been more than 24 hours now and the compilation is still not over for me.
So, that's very unpractical. I'm not going to be able to use FreeBSD for this project since I need to be able to deploy quickly.

Any chance we'll have a package some day? I believe if a port were to be created, it would be the same problem since ports usually compile what they install, but I could be wrong, I'm not sure about the limitations of ports vs packages.
 
Last edited by a moderator:

Amzo

Active Member

Reaction score: 41
Messages: 124

I created the original port of 1.14 for tensorflow as I needed it for my masters. I haven't had time to upgrade it due to university work, but my semester finishes mid May and I'll upgrade it then as I need tensorflow for my work.
 
OP
J

japoc

Active Member

Reaction score: 1
Messages: 102

Thanks for the reply Amzo. Won't the port take exactly the same time to build as my compilation script, though?
 

SirDice

Administrator
Staff member
Administrator
Moderator

Reaction score: 13,160
Messages: 39,777

Won't the port take exactly the same time to build as my compilation script, though?
Yes. That's the nature of compiling from ports. But if a port exists a package can be created for it too (which is what the build clusters do to create our package repositories). Everything always begins with a port. No port, no package, it's as simple as that. In essence a package is nothing more than a pre-compiled port.
 
OP
J

japoc

Active Member

Reaction score: 1
Messages: 102

Thanks. I understand that packages have to be created from ports. So let's assume I manage to create a port that compiles the version of Tensorflow I want. How can I transform this port into a package, so that, later on, I'm able to install Tensorflow quickly on other machines, as a package?
 

SirDice

Administrator
Staff member
Administrator
Moderator

Reaction score: 13,160
Messages: 39,777

How can I transform this port into a package, so that, later on, I'm able to install Tensorflow quickly on other machines, as a package?
If the port has been correctly configured it will already be able to do this (it's an integral part of the whole port system).
Code:
     package              Make a binary package for the port.  The port will
                          be installed if it has not already been.  The
                          package is a .tbz file that you can use to install
                          the port on other machines with pkg-add(8).  If the
                          directory specified by PACKAGES does not exist, the
                          package will be put into the current directory.  See
                          PKGREPOSITORY and PKGFILE.
See ports(7).
 

SirDice

Administrator
Staff member
Administrator
Moderator

Reaction score: 13,160
Messages: 39,777

If you want to take a stab at this you're going to need this too: FreeBSD Porter's Handbook

Because this is an existing port you can use the 'old' version as a basis. Change the version numbers at the top of the Makefile and see how far this gets you. Sometimes it just works. But most of the time you're going to need to fix a bunch of rejected patches, compilation errors, changed dependencies and whatnot.
 

T-Daemon

Daemon

Reaction score: 995
Messages: 1,896

How can I transform this port into a package, so that, later on, I'm able to install Tensorflow quickly on other machines, as a package?
With make package from ports(7), as SirDice pointed out, but this works only if the other machines have the same architecture and version. If the other machines differ, then ports-mgmt/poudriere is needed (Handbook chapter 4.6. Building Packages with Poudriere).

Alternative to make package, if the port is allready installed on the system, without the need to build again, you can use pkg-create(8).
 

Amzo

Active Member

Reaction score: 41
Messages: 124

If I fix the port, the build machine will create a package and it will make it's way into the repository. The current port is marked as broken at the minute, bazel detects how many cores are available and tries to use the maximum amount. On the server when loads of packages are being built in parallel it causes it to eat resources and ends up being killed. For that reason it needs fixing, but once fixed and added pack into the port tree, a package will end up in the repository. It was only marked as broken 7 days ago.

FreeBSD Tensorflow port
 

monwarez

Active Member

Reaction score: 76
Messages: 168

In files/bazelrc you can add
Code:
build --jobs NumberOfJobs
So that it will only use NumberOfJobs cores
 

Amzo

Active Member

Reaction score: 41
Messages: 124

I was thinking of adding an option in the Makefile so by default it only uses 1 job, but a person then has the option to tick the option to build with multiple jobs.
 

AriZuu

New Member


Messages: 1

I managed to build tensorflow package with port a while ago. But today, it seems to fail in a new way (at least to me):
Code:
ERROR: /usr/ports/science/py-tensorflow/work-py37/tensorflow-1.14.0/tensorflow/lite/tools/optimize/BUILD:14:1: undeclared inclusion(s) in rule '//tensorflow/lite/tools/optimize:quantization_utils':
this rule is missing dependency declarations for the following files included by 'tensorflow/lite/tools/optimize/quantization_utils.cc':
  'external/com_google_absl/absl/strings/string_view.h'
  'external/com_google_absl/absl/base/internal/throw_delegate.h'
(portsnap done today, FreeBSD 12.1)
 
Last edited by a moderator:
Top