I'm so confused with pkgng usage

ggallo

ggallo

Hi All!

I'm using FreeBSD 9.2-RELEASE and switched to pkgng. In the beginning, I wanted to install a software on a server of a friend (also on 9.2-RELEASE) that stated it's installation now broken with old pkg_ tools but works fine with the new pkg system. I switched my own server to pkgng to test it, and it worked as stated. I used portmaster to do the install.

Now, I would like to take andvantage of pkg's binary upgrade as weekly refreshed repos available. But with not much luck. I searched the net for information on "new" pkgng system usage all night, but I realized that nevertheless 10-RELEASE is out with defaulted to pkgng, no proper documentation can be found on setup and usage.

After about 30 webpages read whole long and tried abut 5 non-working configs, I finally found what to include in /usr/local/etc/pkg/repos/FreeBSD.conf and /usr/local/etc/pkg.conf. So I can update my repo darabase and check which packages need updating.
Before anyone say I'm stupid, there are man pages: I read man page fog pkg and pkg.conf. Latter instructed me to include signature_type and fingerprint options in FreeBSD.conf, but I can't find the proper keys and pkg refuses to work with an error message.

So, version listing and checking now works. But when I issue pkg upgrade, I get the following error, and I can't found anything on it by Google:
Code: 
Conflict found on path /usr/local/libdata/ldconfig/mysql between mysql55-client-5.5.35(databases/mysql55-client) and mysql51-client-5.1.73(databases/mysql51-client)
I know that portmaster had an option to replace package with another one, but MySQL 5.1 series not deprecated, I think, so I'd like to upgrade that, not switch to 5.5 branch.

So, my question is: where is the current, useful documentation for pkg?

And my biggest complain I realized a few days ago: I use FreeBSD since 5.4 and worked very well for me. But in the recent years there are too many new versions with new functions that not work well, and more problematic that not documented (well). I upgraded my rig to 9.2-RELASE less than half a year, and 10-RELEASE here, but I can't reliably configure 9.2 new features yet, because of lack of essential documentation or knowledge in forums.
This will be a trend? If it, I may switch to a Windows Server. There are so many problems and poor documentation, too. But I had nice GUI for the whole crap instead a black character prompt. :e
Sorry for my pessimism, the last night spent on figuring out official software proper usage drove me crazy.
 
Packages are always built using the default options and MySQL 5.5 is the default. You would have had the same issue with the old pkg_* tools.
 
Thank you for your help!

I seen throught the install/upgrade list pkg printed on screen, and I found that MySQL 5.1 client and server upgrade issued AND MySQL 5.5 client installation, too. That was the problem.
Unfortunately I can't figure out what package requested the MySQL 5.5 client on upgrade (are there any method for this?), so I ended up replacing MySQL 5.1 with 5.5 completely to get out from this situation.

I also found that some binary packages doesen't contain the requested options set, and I have to manually reinstall those from source.
For example Pure-FTPd don't have MySQL support, PHP5 don't have Apache module enabled (which PHP invocation mode preferred from Apache there days?), and so on.

But finally I upgraded 273 packages in less than 2 hours this way. This took me 6-8 hours from sources last time.

I wish there will be better documentation on pkgng, anyway.
 
ggallo said:
I also found that some binary packages doesen't contain the requested options set, and I have to manually reinstall those from source.
For example Pure-FTPd don't have MySQL support, PHP5 don't have Apache module enabled (which PHP invocation mode preferred from Apache there days?), and so on.
Click to expand...
As I said, packages are built using default options. ftp/pure-ftp has MYSQL turned off and lang/php5 has APACHE off.

I can highly recommend building your own package repository using ports-mgmt/poudriere. Then you can enable/disable all the options you like.
 
ggallo said:
After about 30 webpages read whole long and tried abut 5 non-working configs, I finally found what to include in /usr/local/etc/pkg/repos/FreeBSD.conf and /usr/local/etc/pkg.conf. So I can update my repo darabase and check which packages need updating.
Before anyone say I'm stupid, there are man pages: I read man page fog pkg and pkg.conf. Latter instructed me to include signature_type and fingerprint options in FreeBSD.conf, but I can't find the proper keys and pkg refuses to work with an error message.
Click to expand...

This is a good point.

@SirDice, don't you think that the bootstrap utility, or any pkg installation method, should install the /etc/pkg/FreeBSD.conf and /usr/local/etc/pkg.conf (or whatever needed[*]) files and the repository keys?

I really like the new package management system, but its implementation appears hasty and somewhat premature.


[*] The PKGNG documentation is awful, which is unlike FreeBSD. The Handbook does not provide the correct, indeed any, syntax for the repository files. And the default install results in deprecated warnings whenever pkg is used. Similarly, the Wiki lacks pertinent information for proper setup and configuration. There is no information in any official literature on how to obtain and configure repository signing keys.
 
Last edited by a moderator:
nanotek said:
@SirDice, don't you think that the bootstrap utility, or any pkg installation method, should install the /etc/pkg/FreeBSD.conf and /usr/local/etc/pkg.conf (or whatever needed[*]) files and the repository keys?
Click to expand...
I think it does but I'm not sure. I always set PACKAGESITE to a suitable repository before running pkg for the first time. After it's installed pkg.conf contains whatever repository I had set with PACKAGESITE.
[*] The PKGNG documentation is awful, which is unlike FreeBSD. The Handbook does not provide the correct, indeed any, syntax for the repository files. And the default install results in deprecated warnings whenever pkg is used. Similarly, the Wiki lacks pertinent information for proper setup and configuration. There is no information in any official literature on how to obtain and configure repository signing keys.
Click to expand...
Yes, it could use some TLC but it's still very much a work in progress. Some features have been added since the documentation was written. I'm guessing the emphasis has been on getting things correctly working. I'm sure the documentation will be updated.
 
Last edited by a moderator:
I'd be happy to volunteer my help for PKGNG documentation. It's the one thing beyond monetary donations I can actually contribute.
 
One of the problems is that the people of the PKGNG project are very good programmers but quite poor at writing documentation (as witnessed on the mailing lists and in the existing documentation). It's quite rare that programmers are good at both. Then there's the breakneck pace (relative to other developments in FreeBSD) at which the PKGNG tools have been developed, they are still in beta and many of the features are going to change.
 
@kpa, you're very knowledgeable in FreeBSD generally but specially pkg use, and can also articulate yourself well. Do you have the time to help improve the documentation? Same as @SirDice. We need people like you who can explain things quite clearly and have the knowledge to draw upon. I'd prefer developers develop and users with skills teach.
 
Last edited by a moderator:
In my opinion the current and biggest problem with pkgng yet is not that it doesen't have proper documentation, instead that it permanently replace pkg_ tools _without_ proper documentation.
So, 10-RELEASE new installers and upgraders face that they have pkgng only, but can't learn how to use it.

In the mailing list announcement that stated pkgng binary repos being available, also stated that the old pkg_ tools will be removed in 6 months. Already past 4 months from that announcement.
We have only pgkng (no pkg_ tools) in 2 months, but without documentation.

I'm a programmer, too, and I agree with @kpa. We hate writing documentation. But it's a must, and I do it for my programs. Because without docs, the best program worth nothing to users.

I have a new question, too: can I indicate somewhere that I think a port's default configuration or dependency list is incorrect? Of course I mean for binary package building.
Today I upgraded Tomcat7, the pkgng binary way any my webapp stopped working (needs Java7). I found that the current Tomcat7 binary package (7.0.47) requests OpenJDK6 (very outdated, system default is OpenJDK7 for some time), and pkg installed it side-by-side with already installed OpenJDK7. I have to uninstall both, reinstall OpenJDK7 and reinstall Tomcat7 (now 7.0.50 from ports) with portmaster to correct the links, configs.
Are there any way to show this mistake to someone relevant in binary package building?
 
Last edited by a moderator:
ggallo said:
I have a new question, too: can I indicate somewhere that I think a port's default configuration or dependency list is incorrect? Of course I mean for binary package building. Today I upgraded Tomcat7, the pkgng binary way any my webapp stopped working (needs Java7). I found that the current Tomcat7 binary package (7.0.47) requests OpenJDK6 (very outdated, system default is OpenJDK7 for some time), and pkg installed it side-by-side with already installed OpenJDK7. I have to uninstall both, reinstall OpenJDK7 and reinstall Tomcat7 (now 7.0.50 from ports) with portmaster to correct the links, configs.
Are there any way to show this mistake to someone relevant in binary package building?
Click to expand...
I think it's best to report this on the freebsd-ports@ mailing list. That's where all the port maintainers are. So it's the best place for a discussion about the default options of a port.
 
Postfix does the same thing: insists on databases/DB48 even if you already have DB5/6 installed. Seems like pretty poor port and package maintenance.

You're right though: documentation is essential and the developers really ought to know the program they developed better than anyone else. They might not be native English speakers or may be pressed for time, among other things, and the obvious choice for neglect is the documentation. In these instances, it would be good to have volunteers ready to pick up the slack. The problem is, you need skilled or at least experienced users who know the program they're writing for, they also need to be eloquent and willing; maybe there aren't too many of these people?
 
nanotek said:
Postfix does the same thing: insists on databases/DB48 even if you already have DB5/6 installed. Seems like pretty poor port and package maintenance.
Click to expand...
But it doesn't.

With the current ports tree, and even commenting out WITH_BDB_VER=5 in /etc/make.conf this is what I get to see when checking up with mail/postfix:

Code: 
root@smtp2:/usr/ports/mail/postfix # make run-depends-list
/usr/ports/databases/db5
/usr/ports/databases/sqlite3
/usr/ports/databases/tinycdb
/usr/ports/devel/pcre
/usr/ports/mail/dovecot
/usr/ports/mail/libspf2
/usr/ports/security/cyrus-sasl2
What could be possible is that one of the ports which Postfix depends on still has a dependency for the old Berkeley DB version, and that will then also reflect upon Postfix. But Postfix itself is quite clean with all this, it's one of the reasons why I used it as my primary example in my How to deal with the Berkeley DB 4.x cleanup article.

Edit:

A better example can be seen if you check the Makefile:

Code: 
BDB_DESC=       Berkeley DB (uses WITH_BDB_VER)
As you can see it doesn't focus itself on a specific version but simply states to be using Berkeley DB.
 
That's really strange. I just went through this yesterday and it kept insisting on DB48. Can't think of the environment specifics that would have resulted in such behaviour.
 
ggallo said:
We have only pgkng (no pkg_ tools) in 2 months, but without documentation.
Click to expand...
I think that this is a bit excessive. My usage of PKGNG tools is certainly at the simplest level (for sure lower than that of the OP) and I came to FreeBSD when the pkg_ tools were no more necessary, but I thought that for the most basic of the later (e.g. install, delete…) the former proposed an almost word-to-word-equivalent, am I wrong? For searching, querying information, adding, upgrading and keeping track of package updates all I have ever need to know was found in the man pages or through pkg help.

On the other hand, man pages or command-line help is maybe not what documentation is about or what is expected by professional users which have to find quickly structured, comprehensive and up-to-date solutions. I’m not one of these users, so my views are biased. Also, it’s true that the handbook was not up-to-date on package management the last time I consulted.

It’s not my role to defend the work of others, it’s just that I’ve been using these tools for more than a year now and I never found myself in lack of documentation, so the general tone of the messages here surprises me. Lacking stability, yes: the changing pkg.conf syntax and repos issues are a good example, as is the great number of messages in this forum concerning these matters.
 
Juanitou said:
so the general tone of the messages here surprises me
Click to expand...

I'm sorry to all whose read my words as offensive ones. That's not my intention.

I certainly want to help in anything about FreeBSD, to anyone doing work on FreeBSD. But users (I think) can't write documentation for fresh programs, because they do not know how to use it. About a fresh program, only the developer know how to use, and the first person, who write the first doc. must be the programmer itself, because this. Then, others could make it rich and eye-candy, packed with live links and beautiful words. But the base of all the documentation is what the programmer made.
 
Please do not misunderstand me, English it’s not my mother language and I’m sure the tone of my own messages is not always what I intend. I do not find your words offensive and I was only trying to provide my experience with PKGNG as a mean for other people reading this thread to have a more diverse and, I sincerely think, more balanced view of its current state.

I have carefully read your previous messages and I have never stumbled upon the issues you are complaining about, so if these issues are no documented it is perfectly reasonable to feel «so confused».

Best regards,
Juan
 
ggallo said:
In my opinion the current and biggest problem with pkgng yet is not that it doesen't have proper documentation, instead that it permanently replace pkg_ tools _without_ proper documentation.
So, 10-RELEASE new installers and upgraders face that they have pkgng only, but can't learn how to use it.

[/snip]

I'm a programmer, too, and I agree with @kpa. We hate writing documentation. But it's a must, and I do it for my programs. Because without docs, the best program worth nothing to users.
Click to expand...

Programmer here too, and I generally agree with @kpa and @ggallo. Though, I want to emphasize; given the very nature of the pkgng software package, the lack of good proper documentation is rather worrisome. The man-pages (the only "documentation" I have found, so far), feels more like the documentation you would expect from a weekend-hack, or a prototype for a non-system application, in some ways similar to the package manager used in Arch Linux back in 2007(?) when they rewrote a large part of it into C. Not exactly what I would initially expect from the default package tool in a system such as FreeBSD in the year 2014.

Now, I have not looked at the source-code for pkgng, and frankly I should not have to. Should the documentation not be there as a way to help ensure the program works correctly? How am I, an ordinary user, supposed to know if or how pkgng is supposed to work without having to resort to the source-code or throw my hopes at the forums or the mailing-list? Out of sheer curiosity, since I am fairly new to the FreeBSD community, how are the developers of pkgng dealing with correctness in this? Unit-tests? White-box testing? Checking the forums every now and then, to see if there are any common issues? Not my intention to judge or blame the developers or even the team as an abstract entity, but lacking proper documentation it must be tricky to go about developing. And I want to highlight and bring that aspect into the discussion as much as the end-user aspect.

A few years ago I worked as part developer part integration QA, in a team of ~50 devs, working on an embedded system with no clear documentation or indications what each and every little piece was supposed to provide, leaving each sub-team to be slightly creative in their assumptions. I guess you can imagine how hilarious our integration meetings were... Or for that fact how random the nightly-prototype test behaved from time to time. You could never really know what the little rascal would do, resulting in some minor betting matches.

I am a Linux migrant, and the two biggest reasons I moved away from Linux after ~10 years (more or less in a heart-beat) was because of how impressed I was at the engineering and documentation efforts of FreeBSD. As such, I think this issue should be fixed to the superb standards set by the FreeBSD project itself. And I am definitely willing to help out, should more eyes & fingers be necessary to accomplish this.

A lot of people are reluctant to describing how something works, but it is a necessary must sometimes.
 
Last edited by a moderator:
Many programmers do not realize that documentation is part of a program, not an add-on. But the problem here is more that pkg is growing quickly. As far as I know, the features that are not documented in the Handbook did not exist when that section was written. And that was not very long ago. (Ask me how I know...)

I'm a member of the doc team, but I don't use binary packages, and I strongly prefer to test what I write.

So here is the quick version: download the doc files and make the appropriate corrections. Then enter a PR with the patch. Here are instructions: http://www.freebsd.org/doc/en_US.ISO8859-1/books/fdp-primer/book.html. I or other members of the doc team can help with details, either via the forum, IRC, or email.
 
You must log in or register to reply here.
Back
Top