Poudriere: Creating repository in /tmp/packages

poorandunlucky

poorandunlucky

Poudriere does this after it's done building:

Code: 
[00:18:46] Creating pkg repository
Creating repository in /tmp/packages: 100%
Packing files for repository: 100%
[00:35:44] Committing packages to repository

As you can see, it takes forever... What is that? Is that normal?
 
This is not normal and I do not know what is wrong in there. It take just a couple seconds at maximum in here.
 
The only thing that's being done at this stage is a pkg-repo(8), even with a 600+ packages repository it shouldn't take more than a minute. Is there anything else running that might be using a lot of I/O perhaps? That would certainly slow things down.
 
lebarondemerde SirDice xtaz : idk if you guys remember this, but maybe a week ago I was trying to watch a 4K stream that kept getting (visually) "buffered" by the TV, and I checked a bunch of things, and it wasn't being transcoded, it wasn't the network, it wasn't ... anything, so I decided to deal with it, but after pausing the movie I saw that the HDD was being accessed like crazy, and since the stream wasn't being transcoded, and the TV was on pause, I ran top to figure out what was going on, and it was pkg.

I wasn't building anything, nothing was being installed, and it was 2 or 3 AM, so I figure there's a monthly maintenance that involves pkg and a database or a cache of some sort, and since my installation is still fairly new, I think maybe that stage takes forever because I have a pretty big repo (for a laptop, anyway), and pkg doesn't have the hints it needs to be able to deal with it efficiently...

I killed it to watch the movie, though, so I'll know during the month, i guess.
 
I did not write, but that is was I guessed in the post above --> pkg check -qsa the checksum check......

Edit: exactly /usr/local/etc/periodic/security/460.pkg-checksum and it starts (seems default) at 3:30. Maybe, an hour difference between the timezones.
 
poorandunlucky ,
I can't help but notice that both of the times indicated in your posts, were suspiciously close to cron(8)'s daily tasks ( periodic(8) ); mail at midnight, and your administrative system informational reports at 3am. Not saying that must be it. Just saying it might be a possibility. Really, more information is required for any conclusive deduction/solution. Out of curiosity; how much RAM/SWAP? What CPU/SPEED? Lastly; are you running any other heavy services than ports-mgmt/poudriere; like a web server, or something?

--Chris
 
But it is not clear, why it runs: all my periodic.conf shows:
Code: 
# 460.chkportsum
security_status_chkportsum_enable="NO"          # Check ports w/ wrong checksum
security_status_chkportsum_period="daily"

and it is very likely it is, it is slow down poudriere.
 
Well, what shows up in your administrative mail every day? You know. Whoever you aliased as root. So that the system mail gets delivered. :)

--Chris
 
Code: 
mail
No mail for root

I never looked at this process. If I wanted to know I ran it per hand. I have not seen it until it locks the port database, when I wanted install a port.
It is a senseless running process, results are nowhere in any log. How to stop this or log it?
 
Um, OK. So I guess the real question is whats the output of cat /etc/periodic.
Next. What's the contents of /etc/defaults/periodic.conf. The last one will better indicate exactly what is enabled in your system.

HTH

--Chris
 
Ah, I see it:
If I want stop it, I had to change
Code: 
security_status_chkportsum_period="daily"
to
security_status_chkportsum_period="NO"
.

but how to log it?
 
Ugh. Sorry it's late, and I'm tired. I meant /etc/crontab
crontab(1).
This will indicate if, and what you're running (daily,weekly,monthly) which is the default. fe;
Code: 
#minute hour    mday    month   wday    who     command
#
# Save some entropy so that /dev/random can re-seed on boot.
*/11    *       *       *       *       operator /usr/libexec/save-entropy
#
# Rotate log files every hour, if necessary.
0       *       *       *       *       root    newsyslog
#
# Perform daily/weekly/monthly maintenance.
1       3       *       *       *       root    periodic daily
15      4       *       *       6       root    periodic weekly
30      5       1       *       *       root    periodic monthly
#
# Adjust the time zone if the CMOS clock keeps local time, as opposed to
# UTC time.  See adjkerntz(8) for details.
1,31    0-5     *       *       *       root    adjkerntz -a
the output from mine.
and also the contents of /etc/defaults/periodic.conf
periodic.conf(5)

--Chris
 
Thanks, /etc/crontab looks the same.
=> how to log:
I only have to read .....
Code: 
#$daily_output might be set to /var/log/daily.log if you
# wish to log the daily output.....

this should solve it.
 
I think you misunderstand. If you read your well commented periodic.conf, and correlate that with your crontab. You will understand that you're running several jobs; 1) at 12am, and several at 3am. This really shouldn't be a problem. For me, it's easy to plan any heavy tasks between 12:30am - 2:59am, and 3:15am - 11:59pm. If I feel that the heavy tasks will be too burdensome to run during periodic(8). As to logging; if you haven't done so, edit /etc/mail/aliases ( aliases(5) ), and change the entry
Code: 
root:   me@my.host
to your administrative account. fe;
Code: 
root:    chris_h
then
cd /etc/mail]
make aliases
then you'll be able to see what is actually going on, on your system. This will also help you better determine which logs are important to you, and which ones you can disable. Make a little more sense?

HTH

--Chris
 
Chris_H said:
poorandunlucky ,
I can't help but notice that both of the times indicated in your posts, were suspiciously close to cron(8)'s daily tasks ( periodic(8) ); mail at midnight, and your administrative system informational reports at 3am. Not saying that must be it. Just saying it might be a possibility. Really, more information is required for any conclusive deduction/solution. Out of curiosity; how much RAM/SWAP? What CPU/SPEED? Lastly; are you running any other heavy services than ports-mgmt/poudriere; like a web server, or something?

--Chris
Click to expand...

I'm the only user on this system, nothing was running at the time other than pkg... Firefox, though idle, was taking more CPU than minidlna... top was showing pkg to be using 300% CPU or something...

The problem is really after building a package, poudriere says it's building a repository in /tmp/packages, and for most people this step takes a few seconds, while it's several minutes on my system...

It's a Intel Core i7 1.73-2.36 GHz (4 cores, 8 threads) with 16 GB of RAM, and 4/5 GB of SWAP... Memory's never really been an issue... CPU often is, but not in normal use...

I'm just not sure if both situations are related (the long time after building, and the one time pkg was using 300% CPU for an hour at 3AM).
 
You have more than enough resources available for running ports-mrmt/poudriere as you are. If it were a memory issue, and you had /tmp mounted as tmpfs(5), which is essentially an md(4), I could see a possibility. But in your case, it's swamping the CPU. Which shouldn't be an issue in your case. FWIW I'm frequently plagued with issues, when Firefox is running -- it has always leaked memory like a sieve, and often doesn't handle the compiled javascrips delivered these days. Maybe it's worth trying it w/o Firefox running, next time. :)

Good luck!

--Chris
 
You must log in or register to reply here.
Back
Top