Unstaged ports marked as deprecated starting 1st of July

K

kpa

In case you missed the announcement in May:

http://lists.freebsd.org/pipermail/freebsd-ports-announce/2014-May/000080.html

In short, any ports that have not been yet converted to use the now mandatory staging mechanism have been marked as deprecated and their maintainers have been reset. This could affect you if you build your own packages from ports, those ports will still build but you will get a warning about the deprecation. Eventually these ports will be marked broken and deleted unless updated to use staging. The number of ports as of today that are not yet converted to staging is about 1600 out of the 24000+ ports.
 
kpa said:
The number of ports as of today that are not yet converted to staging is about 1600 out of the 24000+ ports.
Click to expand...
I was wondering how many there were left. But 1600 out of 24000+ isn't at all bad. Major kudos to all developers in getting 22400+ ports fixed :beer
 
You can get the number of still unstaged ports yourself with this little command:

Code: 
grep --include="${PORTS_TREE_PATH}/*/*/Makefile" -lr "^NO_STAGE" "${PORTS_TREE_PATH}" | wc -l

Where PORTS_TREE_PATH is usually /usr/ports.
 
The above command evolved into a small script that outputs just the category/port lines of unstaged ports:

Code: 
#!/bin/sh --

POUDRIERE="/usr/local/bin/poudriere"

PORTS_TREE="default"
PORTS_TREE_PATH=`${POUDRIERE} ports -lq | grep "^${PORTS_TREE}" | (read name method date time path; echo $path)`

/usr/bin/grep --include="${PORTS_TREE_PATH}/*/*/Makefile" -lr "^NO_STAGE" "${PORTS_TREE_PATH}" | \
    sed -e "s|^${PORTS_TREE_PATH}/\(.*/.*\)/Makefile|\1|g"  | /usr/bin/sort

This needs ports-mgmt/poudriere to work but you can replace PORTS_TREE_PATH with /usr/ports if you're not using it.
 
It's a little easier with find(1):
find /usr/ports -name Makefile -maxdepth 3 -exec grep -Hi ^NO_STAGE {} \+ | sed -e 's%/usr/ports/\(.*\)/Makefile.*%\1%' | sort
 
I would say that's slower, possibly a lot slower because you are creating a new process for each Makefile to run grep(1). My solution lets grep(1) do the recursion into the ports tree and also filtering of the files to include in the matching and I believe it's all done in a single process.

Edit: I just noticed you're using the -exec {} \+ form of -exec. That mitigates the number of processes but I think there's still an upper limit on how many characters can fit onto a single command line.
 
Using \+ with find(1) helps, it's much faster than \;. But with the cache loaded, it's hard to benchmark, about one second here, with grep(1) taking about three times that.
 
Turns out that using find(1) is almost twice as fast. My system is slightly memory constrained so the data never fits in the cache completely and that should make my tests at least somewhat valid. You can safely add -mindepth 3 -type f to the parameters for find(1), the Makefiles that we are interested in are all on the same level in the hierarchy and they are just normal files.
 
kpa said:
In short, any ports that have not been yet converted to use the now mandatory staging mechanism have been marked as deprecated and their maintainers have been reset.
Click to expand...
This appears to not entirely be the case. At the time of writing:
  • Most unstaged ports did indeed have their maintainers reset to ports@.
  • However, very few unstaged ports have been marked as DEPRECATED and none have been marked as such solely for being unstaged.
    Edit: it appears that marking unstaged ports as deprecated is done by a different means than putting DEPRECATED in the port Makefile, so this point probably isn't valid.
  • On the other hand, several ports have been marked as BROKEN merely for being unstaged. In fact, this started way before the deadline of 31 June [sic].
So it appears that something is not quite being done as advertised. :q

kpa said:
The number of ports as of today that are not yet converted to staging is about 1600 out of the 24000+ ports.
Click to expand...
Update: the number has dropped to just below 1,500. There's still some work to do (which is exactly why I haven't been around here much lately).

P.S. I temporarily made this thread an announcement.
 
fonz said:
kpa said:
The number of ports as of today that are not yet converted to staging is about 1600 out of the 24000+ ports.
Click to expand...
Update: the number has dropped to just below 1,500. There's still some work to do (which is exactly why I haven't been around here much lately).
Click to expand...
It has dipped below 1,300 now.
 
The number is roughly 1180 now. I couldn't help noticing that two very important ports are still not staged, www/squid and www/squid33. Someone has already created patches and posted PRs but the patches are in unusable form as comments on the PRs and not as downloadable attachments as required. The submitter of the PRs has apparently given up on his efforts. If anyone feels like they have time and enthusiasism to help getting these important ports back in shape, please look at these PRs and see if you can extract the patches in usable form and resubmit the patches as attachments to the existing PRs (don't know if that is possible) or submit new PRs:

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=191680

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=191681

Edit: www/squid32 is also still unstaged but I'd guess www/squid33 is what people use if they are not using the older version 2.7 of www/squid.
 
Both Squid 3.2 and 3.3 are not supported anymore and should not be used. Certainly 2.7 isn't supported anymore (last update was released 4 years ago). The only supported version at this time is 3.4. So if there's really somebody up for it I would suggest getting www/squid updated to version 3.4 and let the others die. Perhaps a squid-devel can be created for 3.5.

http://www.squid-cache.org/Versions/
 
kpa said:
please look at these PRs and see if you can extract the patches in usable form and resubmit the patches as attachments to the existing PRs (don't know if that is possible)
Click to expand...
See @SirDice's comment above, but for what it's worth: yes, if you have a Bugzilla account you can upload new patches to an existing PR, even if you're not the original submitter. It's even possible to mark new patches as "replacing older patches", although I'm not sure whether just anyone can do that or only the PR initiator or initial patch submitter. Moreover, in this case apparently the original patches are simply verbatim comments, not patches in the form of (downloadable) attachments.

How much work it is to redo the patches is another thing, but I see two options there: either copy/paste and get the line breaks right, or try to make the changes manually to a copy of the port directory. If both are a lot of work, it might actually be easier to redo the conversion from scratch, possibly using the "patches" as a guide.
 
Last edited by a moderator:
kpa said:
The number is roughly 1180 now.
Click to expand...
And we're down to less than 1,000. In fact, it's closer to 900. Considering that there are undoubtedly some ports that are obsolete or otherwise broken and should probably be left to die anyway, I'd say it's looking pretty good.
 
Unstaged ports that are still left in the tree (the ones that didn't have a PR pending are being deleted at the moment) are now marked as broken instead of deprecated. Building of those ports will not work anymore.

http://svnweb.freebsd.org/ports?view=revision&revision=366839

The Mk/bsd.port.mk file now defines NO_STAGE like this:

Code: 
.if defined(NO_STAGE)
BROKEN=             Not staged.
DEPRECATED?=        Not staged. See http://lists.freebsd.org/pipermail/freebsd-ports-announce/2014-May/000080.html
EXPIRATION_DATE?=   2014-08-31
.endif
 
Unfortunate bug here. For instance, editors/texmacs , .stage_done .package_done etc... howsoever pkg_add: leave_playpen: can't chdir back to ' ' which previously could be installed, has a failing package (bug somewhere here) and a STAGED port NON-STAGING install, which maybe could be allowed, marks a staged port non-installable maybe too hastily... I'd like to see the original bug solved officially as it is soon to prevent upgrading of browsers, etc... and a slight chance I can resolve it by the end of today. Not hopeful though. I'd not be too thrilled if upgrading to pkg and v10 results in the new package system not either being able to install from a repository with the same problem.
 
How about filing a PR about the port that doesn't work? The portmgr@ team are pretty bright people but can not fix a problem they are not aware of.
 
I've scripted a workaround for the use case that would not install. And have in the past posted that problem to the ports list... but the problem may not have shown up yet in any systems now using the newer package tools, so it would have a low priority for the time being... especially since the ports team has so much work in the queue(s), and it would be more attuned to helping to test the install from another system here under the new package tools, which appears inevitable, but I've not time for anytime soon, but will have to do sooner or later.

As an addendum to the post I made above this one...
The resolution ( two different parameters to the install environment ) I had hoped for above, failed.
 
Ahem... I can understand marino@'s attitude well on this one. The PR is just saying "please please don't delete this port, I'm not going to maintain it, please someone else step up, please please". No patch, no suggestions on how to fix it, nothing.

As is happens, marino@ just went over the process of saving the www/squid33 from being deleted, take a look the commit history.

What is meant by "a port with PR pending" in what I wrote above is a PR with patch to stage the port (preferably) or at least something that contributes to improving the situation with the port. The portmgr@ team and those with the commit bit do already have the full lists of ports available that are going to be deleted, no need to remind them in form of PRs.
 
So, I finally got tired of attempting to compile 3.3.13 with ecap even though I had gcc49 defined. It was just a complete failure on FreeBSD 11 with krb5 errors that I had to revert to 3.3.11 and manually compile it with:

Code: 
./configure --with-default-user=squid --bindir=/usr/local/sbin --sbindir=/usr/local/sbin --datadir=/usr/local/etc/squid --libexecdir=/usr/local/libexec/squid --localstatedir=/var --sysconfdir=/usr/local/etc/squid --with-logdir=/var/log/squid --with-pidfile=/var/run/squid/squid.pid --with-swapdir=/var/squid/cache/squid --enable-auth --enable-build-info --enable-loadable-modules --enable-removal-policies=lru heap --disable-epoll --disable-linux-netfilter --disable-linux-tproxy --disable-translation --disable-eui --enable-cache-digests --disable-delay-pools --enable-ecap --disable-esi --enable-follow-x-forwarded-for --enable-htcp --disable-icap-client --disable-icmp --disable-ident-lookups --disable-ipv6 --enable-kqueue --with-large-files --disable-http-violations --disable-snmp --enable-ssl --disable-ssl-crtd --disable-stacktraces --disable-ipf-transparent --disable-ipfw-transparent --disable-pf-transparent --enable-forw-via-db --disable-wccp --disable-wccpv2 --enable-auth-basic=DB MSNT MSNT-multi-domain NCSA PAM POP3 RADIUS fake getpwnam --enable-auth-digest=file --enable-external-acl-helpers=file_userip time_quota unix_group --enable-auth-negotiate=none --enable-auth-ntlm=fake smb_lm --enable-storeio=diskd rock ufs aufs --enable-disk-io=AIO Blocking DiskDaemon IpcIo Mmapped DiskThreads --enable-log-daemon-helpers=file --enable-url-rewrite-helpers=fake --with-openssl=/usr/local --disable-optimizations --enable-debug-cbdata --prefix=/usr/local --mandir=/usr/local/man --infodir=/usr/local/info/ --build=amd64-portbld-freebsd11.0

With environment variables:

Code: 
CC=/usr/local/bin/gcc49
CXX=/usr/local/bin/g++49
CPP=/usr/local/bin/cpp49

So, now the clamav loadable module works as it should:

Code: 
root@www:/var/log/squid # tail -f cache.log 
2014/09/03 20:24:43 kid1|      1847 Objects cancelled.
2014/09/03 20:24:43 kid1|         0 Duplicate URLs purged.
2014/09/03 20:24:43 kid1|         0 Swapfile clashes avoided.
2014/09/03 20:24:43 kid1|   Took 0.05 seconds (3817.30 objects/sec).
2014/09/03 20:24:43 kid1| Beginning Validation Procedure
2014/09/03 20:24:43 kid1|   Completed Validation Procedure
2014/09/03 20:24:43 kid1|   Validated 172 Entries
2014/09/03 20:24:43 kid1|   store_swap_size = 90224.00 KB
2014/09/03 20:24:43 kid1| Configuring Parent sunspot-thin3.xxxx.yy/8888/0
2014/09/03 20:24:44 kid1| storeLateRelease: released 0 objects
2014/09/03 20:24:58 kid1| temporary disabling (Forbidden) digest from sunspot-thin3.xxxx.yy
2014/09/03 20:25:20 kid1| INFECTED, Eicar-Test-Signature
Squid 3.4.7 dumps core with ecap no matter what - 3.3.13 fails to build with krb errors. I can afford a little room - but I have to have AV scanning. No choice on that. Even given all the HTTPS crap.

Ian
 
Please excuse me for being slow, to act and to understand what's going on...

I have just updated my home server for the first time in a couple weeks and am finding that portmaster is unable to complete due to deleted ports.

Specifically, I get this:


Code: 
===>>> The net/cvsup port has been deleted: Cascade removal (lang/ezm3)
===>>> Aborting update

So I went back to UPDATING and saw that I had not uninstalled cvsup as advised. I was already using subversion so I guess not getting an update request for cvsup meant that I never looked for it in UPDATING

Now, I am stuck. I can not uninstall it because it is already deleted but I can not update because it is still listed as as a dependant of another deleted port ezm3, which I can not uninstall for the same reason.

I can appreciate the reason for removing the unstaged ports. And there was more than enough time to get everything sorted out for myself but unfortunately seeing my mistake doesn't help me :p

Is there anything I can do to clean this up?


Thanks.
 
ghostcorps said:
Now, I am stuck. I can not uninstall it because it is already deleted but I can not update because it is still listed as as a dependant of another deleted port ezm3, which I can not uninstall for the same reason.
Click to expand...
Try pkg delete -f ezm3 or perhaps it will also be removed with pkg autoremove.
 
SirDice said:
ghostcorps said:
Now, I am stuck. I can not uninstall it because it is already deleted but I can not update because it is still listed as as a dependant of another deleted port ezm3, which I can not uninstall for the same reason.
Click to expand...
Try pkg delete -f ezm3 or perhaps it will also be removed with pkg autoremove.
Click to expand...

Thanks :) It looked good but no dice :(

pkg delete -f ezm3 removed ezm3 but I am still getting the conflict when portmaster gets to cvsup.

This didn't seem to find anything to remove.

Code: 
# pkg autoremove
Checking integrity... done (0 conflicting)
Nothing to do.


Thanks anyway :)
 
Unstaged ports deleted starting 1st of Sept

This is a quick edit to this question: I haven't found clear guidance for managing deleted unstaged ports, of particular interest being deleted branches of dead ports.

I'm sure there's an easy answer for this, but the method I normally use to move the origin of a port, portmaster -o, seems to be a little perplexed by the recently removed ports. I have a few errors of the form:
Code: 
===>>> The mail/postfix29 port has been deleted: Not staged
===>>> Aborting update
It would be lovely if /usr/ports/UPDATING had a quick note about the removals and a summary of the preferred technique for migrating to an active port of the package (for example to /usr/ports/mail/postfix210 (/usr/ports/mail/postfix-current apparently doesn't support SPF yet)).

In stead, what is the preferred technique for migrating? Is it nobler to suffer portmaster -e category/deadport and portmaster category/goodport or to take portmaster -o category/goodport category/deadport?
 
You must log in or register to reply here.
Back
Top