Solved www/webkit2-gtk3 broken or synth not liking it?

ASX said:
These are two completely different workloads, graphics is offloaded to GPU, if it work well is because of your graphics card. Compiling software is a CPU bound type of load. They just don't compare.
Click to expand...

I know that, I guess the point I was making was I have no reason to believe this machine is overloaded. Maybe it underpowered for synth, but certainly nothing else. I might not be a senior computer admin but I think I've got a good idea of the symptoms of an overloaded machine.

But maybe I have misunderstood something all my life. I changed synth back to 2 builders x 2 jobs. Does this look overloaded to you? I'm looking at the RAM and paging in/out swapping. Is that bad? I do see the the 'b' column (procs blocked) going to 1 from time to time; which concerns me a little, but I see that on other machines from time to time. The time increment is = 1 second.

Good night all, I really got to get going. Thanks again. :)

Code: 
 procs      memory      page                    disks     faults         cpu
 r b w     avm    fre   flt  re  pi  po    fr  sr ad0 da0   in   sy   cs us sy id
 3 0 0   1219M   178M  8572   0   0   0   674  93   2   0   22 2217  357 87 13  0
 2 0 0   1151M   243M  8514   0   0   0 25435  66   2   0   17 3908  321 76 23  1
 2 0 0   1179M   214M  8300   0   0   0  1136  78   0   0   10 2768  317 83 17  0
 2 0 0   1107M   287M  7085   0   0   0 26307  47   3   0   25 4450  487 74 26  0
 2 1 0   1135M   258M  8033   0   0   0   675  59   3   0   15 3846  391 79 21  0
 2 0 0   1163M   228M  8474   0   0   0  1144  71   2   0   15 3451  361 80 20  0
 2 1 0   1195M   198M  8170   0   0   0   666  83   0   0   19 2132  362 87 13  0
 2 1 0   1131M   262M  8596   0   0   0 25437  56   2   0   14 3664  343 78 22  0
 2 0 0   1163M   231M  8828   0   0   0  1144  69   1   0   11 3745  313 81 19  0
 3 0 0   1191M   202M  7900   0   0   0   666  81   0   0   21 2145  372 85 15  0
 2 0 0   1119M   273M  7636   0   0   0 25972  51   3   0   16 5035  373 72 27  1
 3 0 0   1156M   242M  8841   0   0   0  1345  63   1   0   11 4169  379 78 22  0
 2 0 0   1187M   210M  8981   0   0   0  1131  76   5   0   30 2978  440 81 19  0
 2 0 0   1211M   181M  8413   0   0   0  1136  88   0   0   12 2859  313 87 13  0
 2 0 0   1143M   251M  8072   0   0   0 26422  59   2   0   12 4500  359 74 26  0
 2 0 0   1175M   220M  8343   0   0   0   666  71   0   0   21 2347  357 86 14  0
 2 0 0   1107M   287M  8261   0   0   0 25812  82   1   0   15 4659  506 77 23  0
 2 0 0   1139M   257M  8031   0   0   0   674  55   2   0   12 4070  344 78 22  0
 2 0 0   1167M   227M  8288   0   0   0   666  68   1   0   20 2970  410 80 20  0
 2 0 0   1199M   196M  8871   0   0   0  1144  81   1   0   16 2471  320 83 17  0
 2 0 0   1120M   191M  7324   0   0   0   733  92   0   0   13 2258  279 84 16  0
 2 1 0   1159M   235M  8587   0   0   0 25933  62   3   0   21 4654  404 76 24  0
 2 1 0   1187M   207M  8099   0   0   0  1144  75   1   0   11 2530  324 85 15  0
 2 0 0   1119M   275M  8532   0   0   0 26484  45   1   0   15 5858  686 70 29  0
 2 0 0   1147M   247M  8126   0   0   0  1160  58  57   0   75 3843  642 76 24  0
 2 0 0   1179M   215M  8711   0   0   0   674  70   3   0   12 2577  304 85 15  0
 2 0 0   1207M   184M  8298   0   0   0   666  83   1   0    8 2443  299 88 12  0
 2 0 0   1231M   161M  6923   0   0   0  1144  96   1   0   21 2058  382 87 13  0
 2 0 0   1163M   229M  8597   0   0   0 26506  64   1   0   10 4412  303 76 24  0
 procs      memory      page                    disks     faults         cpu
 r b w     avm    fre   flt  re  pi  po    fr  sr ad0 da0   in   sy   cs us sy id
 3 0 0   1191M   204M  6862   0   0   0   666  76   0   0   16 1896  344 87 13  0
 2 0 0   1123M   272M  8953   0   0   0 26715  45   3   0   13 6217  434 73 27  0
 2 0 0   1151M   242M  8048   0   0   0   666  58   0   0   10 3256  288 85 15  0
 2 0 0   1179M   211M  8499   0   0   0   666  70   0   0   15 2719  360 86 14  0
 2 0 0   1211M   182M  8245   0   0   0  1144  82   3   0   14 2786  329 82 18  0
 2 0 0   1147M   245M  8629   0   0   0 25237  95   2   0   11 4479  321 73 27  0
 3 0 0   1187M   215M  8881   0   0   0  1434  67   0   0   24 3178  396 82 18  0
 2 0 0   1115M   279M  8696   0   0   0 25532  79   2   0   12 5298  397 75 25  0
 2 0 0   1143M   251M  7786   0   0   0   666  52   1   0    8 3473  323 81 19  0
 2 0 0   1175M   218M  9117   0   0   0  1136  65   0   0   18 3485  387 83 17  0
 2 0 0   1207M   186M  8607   0   0   0   674  78   3   0   14 2498  300 85 15  0
 2 0 0   1139M   253M  8280   0   0   0 25650  91   0   0    7 3138  288 79 20  1
 2 0 0   1171M   221M  8910   0   0   0  1144  63   1   0   16 3420  375 85 15  0
 2 0 0   1107M   288M  7918   0   0   0 25444  75   4   0   16 4791  368 71 29  0
 2 0 0   1135M   259M  8026   0   0   0   666 106   0   0    7 3763  344 78 22  0
 2 0 0   1167M   228M  8770   0   0   0  1144  72   3   0   19 3759  422 80 20  0
 2 0 0   1195M   197M  8311   0   0   0   666  85   0   0   10 2491  294 83 17  0
 2 0 0   1227M   167M  8349   0   0   0   666  97   1   0   10 1825  299 89 11  0
 2 0 0   1163M   231M  9624   0   0   0 26668  70   1   0   23 5561  446 72 27  0
 2 0 0   1095M   299M  7803   0   0   0 25515  41   1   0   10 3806  315 77 22  2
 2 0 0   1123M   267M  8917   0   0   0  1136  54   0   0   10 4508  355 77 23  0
 2 0 0   1156M   240M  7473   0   0   0   674  65   3   0   20 2917  374 82 18  0
 2 0 0   1184M   209M  8387   0   0   0   666  77   0   0    9 2649  306 86 14  0
 2 0 0   1216M   179M  8356   0   0   0  1168  89  56   0   65 2277  517 86 14  0
 2 0 0   1151M   244M  8335   0   0   0 25079  61   2   0   20 4049  390 78 22  0
 2 0 0   1095M   297M  7052   0   0   0 21122  40   1   0   11 4497  317 77 23  0
 3 0 0   1115M   279M  9560   0   0   0  5424  46   2   0   10 5372  326 74 26  0
 2 0 0   1073M   317M  7147   0   0   0 17458  30   2   0   18 5931  860 73 27  0
 2 0 0   1107M   287M  8211   0   0   0   674  43   1   0   12 4125  376 79 21  0
 procs      memory      page                    disks     faults         cpu
 r b w     avm    fre   flt  re  pi  po    fr  sr ad0 da0   in   sy   cs us sy id
 2 0 0   1131M   261M  7572   0   0   0  1144  54   3   0   10 2324  310 88 12  0
 2 1 0   1163M   227M  8969   0   0   0   666  67   0   0   19 3462  378 80 20  0
 2 0 0   1192M   201M  8186   0   0   0  1721  79   2   0   18 4897  417 76 24  0
 2 0 0   1228M   169M  8754   0   0   0   755  92   1   0    8 2435  299 85 15  0
 2 0 0   1252M   139M  7994   0   0   0   666 105   0   0   16  895  345 91  9  0
 2 0 0   1160M   220M  7713   0   0   0 28647 116   0   0   10 1321  305 92  8  0
 2 0 0   1086M   305M  7833   0   0   0 30038  33   3   0   20 7536  630 60 39  1
 2 0 0   1115M   276M  7853   0   0   0   666  45   1   0   11 2779  312 86 14  0
 2 0 0   1147M   247M  8153   0   0   0  1144  58   2   0    9 3144  337 80 20  0
 2 0 0   1175M   218M  8149   0   0   0   666  69   0   0   15 2817  363 82 18  0
 2 0 0   1140M   256M  7547   0   0   0 17731  81   1   0   14 4147  329 78 22  0
 2 0 0   1168M   228M  8027   0   0   0  1144  64   1   0    8 2820  305 85 15  0
 2 0 0   1103M   287M  6599   0   0   0 22197  74   3   0   19 4256  402 75 25  0
 4 0 0   1119M   273M  9463   0   0   0  6180  51   2   0   12 4996  365 77 23  0
 3 0 0   1147M   245M  8435   0   0   0  1810  56   3   0   17 4488  443 78 22  0
 3 0 0   1090M   299M  6370   0   0   0 20377  67   0   0   16 1887  379 85 15  0
 2 0 0   1090M   302M  8051   0   0   0  9399  44   2   0   12 6984  377 66 34  0
 2 0 0   1111M   278M  6642   0   0   0   674  43   1   0    9 2281  299 87 13  0
 2 0 0   1111M   279M  7871   0   0   0  8690  52   2   0   17 5422  421 75 25  0
 2 0 0   1143M   251M  8049   0   0   0  1136  52   0   0   10 3632  353 78 22  0
 2 0 0   1175M   220M  8589   0   0   0   674  63   2   0   10 3343  301 80 20  0
 3 0 0   1199M   191M  7777   0   0   0   666  76   0   0   17 2657  343 83 17  0
 2 0 0   1231M   160M  8780   0   0   0  1168  88  60   0   71 2205  558 84 16  0
 2 0 0   1251M   139M  5954   0   0   0   666 101   0   0   10  913  292 95  5  0
 3 0 0   1176M   217M  7178   0   0   0 27641 110   3   0   18 4447  371 79 21  0
 2 0 0   1103M   288M  8232   0   0   0 26956 122   2   0   16 4819  390 73 27  0
 2 0 0   1091M   299M  8016   0   0   0 11124  41   2   0    9 3234  308 82 18  0
 3 0 0   1119M   271M  7912   0   0   0   676  53   2   0   20 4222  395 78 22  0
 2 0 0   1147M   241M  8804   0   0   0  1809  65   7   0   26 4281  394 81 19  0
 procs      memory      page                    disks     faults         cpu
 r b w     avm    fre   flt  re  pi  po    fr  sr ad0 da0   in   sy   cs us sy id
 3 0 0   1180M   210M  8525   0   0   0   666  78   5   0   12 2959  345 78 18  4
 2 0 0   1208M   182M  8125   0   0   0  1144  90   1   0   17 2852  383 86 14  0
 3 0 0   1159M   233M  8767   0   0   0 22524  67   1   0   11 5028  347 74 26  0
 2 0 0   1187M   206M  7711   0   0   0   666  79   0   0    9 1824  280 86 14  0
 2 0 0   1115M   274M  8671   0   0   0 26756  49   2   0   19 5358  447 72 28  0
 2 0 0   1143M   246M  7706   0   0   0   666  61   0   0    9 3269  339 83 17  0
 2 0 0   1171M   221M  6877   0   0   0   666  73   1   0    9 2142  281 87 13  0
 2 1 0   1131M   262M  8195   0   0   0 19246  53   4   0   21 6131  563 73 27  0
 2 0 0   1159M   233M  7967   0   0   0   666  66   0   0    9 2809  296 85 15  0
 2 0 0   1191M   199M  9107   0   0   0   666  79   0   0    8 2242  290 86 14  0
 2 0 0   1215M   175M  7214   0   0   0  1144  91   1   0   18 2042  342 89 11  0
 4 0 0   1081M   308M  8481   0   0   0 43401  32   2   0   15 7292  669 65 35  0
 2 0 0   1111M   279M  9228   0   1   0  2209  45   2   0   13 4718  437 85 15  0
 2 0 0   1096M   286M  7777   0   0   0  9873  56   2   0   18 2679  380 83 16  1
 2 0 0   1136M   255M  8345   0   0   0   834  53   2   0   13 5393  349 71 29  0
 2 0 0   1164M   226M  8389   0   0   0  1136  67   0   0   16 2380  368 88 12  0
 2 0 0   1159M   232M  8116   0   0   0 10035  63   4   0   14 3902  328 75 24  0
 2 0 0   1087M   301M  7782   0   0   0 25897  75   6   0   13 3193  334 78 22  0
 2 0 0   1119M   272M  8443   0   0   0  1152  46   2   0   19 4999  432 74 26  0
 2 0 0   1147M   243M  7924   0   0   0   666  59   2   0   12 3520  326 84 16  0
 2 1 0   1180M   211M  8507   0   0   0   666  70   0   0   10 3051  333 81 19  0
 2 0 0   1208M   183M  8089   0   0   0  1144  82   2   0   17 2096  377 85 15  0
 2 0 0   1219M   228M  6473   0   0   0 18133  95  64   0   77 1369  547 89 11  0
 2 0 0   1188M   202M  7435   0   0   0  1139  75   1   0    8 4070  284 79 21  0
 3 0 0   1107M   284M  8179   0   0   0 29762  38   2   0   17 5104  398 72 26  2
 2 0 0   1135M   255M  8444   0   0   0  1332  51   1   0   21 3213  354 86 14  0
 2 0 0   1163M   227M  7997   0   0   0  1136  63   2   0    9 3180  303 84 16  0
 2 0 0   1127M   266M  7802   0   0   0 17954  75   2   0   18 4319  433 73 25  2
 2 0 0   1152M   236M  7989   0   0   0   674  58   2   0   12 3124  388 84 16  0
 
marino@ said:
It's possible that the process locked up (kernel malfunction?) because that's not a line that would lead to a long pause. So maybe the machine has some hardware issues ...
Click to expand...

Hardware issue would cause the same port to fail to build over and over, but not other ports? There are other comment in this forum about webkit ports taking ages to build.
 
Running synth in 1x1 mode, this port still gets aborted due to watchdog timeout. This is a typical vmstat -w 1 output. Can a few of the FreeBSD experts in here tell me what they honestly think? Does this indicate an overloaded machine? This machine runs only synth during my ports builds. I shutdown all other stuff (gdm, gnome3, etc) during builds. I even tried running synth with and without dbus and hald but see no difference. I'll do more testing and report back sometime this week I figure.

Code: 
procs      memory      page                    disks     faults         cpu
 r b w     avm    fre   flt  re  pi  po    fr  sr ad0 da0   in   sy   cs us sy id
 1 0 0    850M   962M   542   0   0   0   742  12 191   0  209 6661 1205 29 16 56
 3 0 0    866M   961M  2984   0   0   0  5269   9 523   0  558 15460 2609 11 27 62
 1 0 0    878M   927M  7705   0   0   0  3051  17 567   0  607 24376 3228 17 37 46
 1 0 0    878M   915M  1175   0   0   0  1173  18 526   0  556 32548 2721 12 32 56
 1 0 0    878M   907M   539   0   0   0   829  18 520   0  560 29367 2483  9 31 61
 1 0 0    878M   895M   539   0   0   0   701  18 917   0  971 24234 4487 16 30 54
 1 0 0    878M   878M   920   0   0   0  1190  18 919   0  974 33859 4607 15 35 49
 1 0 0    878M   872M   539   0   0   0   701  18 831   0  878 14146 3938  5 24 71
 1 0 0    878M   858M   539   0   0   0   721  18 651   0  690 35890 3632 19 38 43
 1 0 0    878M   858M   917   0   0   0  1213  18 365   0  394 7461 2143  1 57 42
 2 0 0    874M   847M  2726   0   0   0  3595  10  68   0   90 18266  681 16 39 46
 1 0 0    866M   838M  2005   0   0   0  3376   9 563   0  593 31632 3089 11 43 46
 2 0 0    807M   853M  2151   0   0   0  2519  12 849   0  895 5583 4160 10 24 66
 2 0 0    882M   837M 11327   0   0   0 15154   7  10   0   25 12367  648 17 36 47
 1 0 0    882M   802M  2485   0   0   0   765  16 354   0  378 17237 2048 28 26 46
 1 0 0    882M   790M  1171   0   0   0  1508  16 1123   0 1180 42514 6101 13 54 33
 1 0 0    882M   783M   824   0   0   0  1195  16 462   0  495 14213 2175  6 21 73
 1 0 0    882M   762M   917   0   0   0  1245  16 649   0  680 9180 3378 32 18 50
 1 0 0    882M   741M   539   0   0   0   789  16 258   0  286 8747 1607 39 16 46
 1 0 0    882M   721M   539   0   0   0  2069  16 297   0  321 5313 1682 40 15 45
 1 0 0    882M   698M   917   0   0   0  1237  16 232   0  258 5133 1394 42 12 46
 1 0 0    882M   666M   539   0   0   0   808  16 287   0  317 8233 1774 32 20 48
 1 0 0    882M   656M   539   0   0   0   758  16 620   0  648 11596 2877 11 27 62
 1 0 0    882M   652M   917   0   0   0  1165  16 127   0  146 11012  798  5 34 61
 1 0 0    890M   654M  3156   0   0   0  5880  16 176   0  209 15010 1231  6 40 53
 0 0 0    939M   651M  1425   0   0   0   997  10 230   0  257 6021 1312  5 11 83
 0 0 0    939M   651M   917   0   0   0  1149  10   1   0   14 1256  243  1  2 97
 3 0 0    898M   643M  4280   4   0   0  5641  10  63   0   84 15411  705 14 22 64
 1 1 0    882M   640M  3016   0   0   0  4167   9 351   0  383 11599 1900  8 18 73
 0 0 0    882M   635M  3009   0   0   0  3753   9 416   0  439 14890 2219 10 19 71
 0 0 0    906M   630M  1538   0   0   0  1692   9 474   0  503 23309 2320  8 29 63
 0 1 0    944M   617M  3226   0   0   0  2020   9 264   0  301 7677 1454  8 15 77
 0 0 0    944M   617M   539   0   0   0   757  24  71   0   86  840  502  0  3 97
 0 0 0    944M   621M  1748   0   0   0  3214   0   0   0   15 5437  268  6  8 85
 0 0 0    944M   621M   918   0   0   0  1157  20  40   0   52 1257  404  1  2 97
 0 0 0    944M   621M  1652   0   0   0  2042  10  10   0   27 4470  299  6  6 88
 0 0 0    944M   621M   539   0   0   0   671  10  42   0   58  937  414  1  6 94
 1 0 0    939M   621M   918   0   0   0  1151  10   1   0   12 1254  230  0  2 97
 0 0 0    944M   621M  1357   0   0   0  1713  10   6   0   14 3723  251  3  5 92
 1 0 0    939M   621M   539   0   0   0   669  10   1   0   22  874  287  0  2 98
 0 0 0    944M   601M  4689   0   0   0  3039  13 155   0  183 7439 1156 34 14 52
 0 0 0    944M   601M   539   0   0   0   669  13   1   0   17  841  251  0  3 97
 1 0 0    882M   608M  2077   0   0   0  5012  13  10   0   23 5726  288  4  9 87
 1 0 0    882M   581M  2555   0   0   0   709  12 578   0  604 27244 3146 15 34 51
 
That doesn't sound like any normal situation. it shouldn't be overloaded at 1x1 in which case the log is stalling for other reasons (e.g. a process that shouldn't get stuck, does)

you can modify synth to disable the monitor by changing this line:
https://github.com/jrmarino/synth/blob/master/src/portscan-buildcycle.adb#L659
to "False" but you'd only do that to help you identify what is getting stuck.

In other words, if a process gets stuck, the watchdog does its job and kicks in, but before you have a chance to observe the problem. If the watchdog never kicks in, maybe you tell what the real problem is.
 
Yes this port takes notoriously long to build, but it does (or rather used to) complete. I really want a few others to review my vmstat output and tell me their honest opinion. Blocking, paging-in & paging-out are either zero or near zero; all key indicators of machine load right? And I will do any other commands asked. Worse case scenario I learn something otherwise and maybe replace the machine sooner than I thought I would. Best case scenario we may discover something unique about this port and/or synth, and possibly improve something.

I find it interesting that the watchdog timer never triggers on any other ports, even when I was using 2x2 setting. If overloading was root cause would this not happen for other ports throughout the 300+ ports build process? But this webkit port failing like this now is something different. Something has changed in my humble opinion.

Mario, Id love to help, but I am not a compiler yet. I don't know how to take your code, modify, compile, and put it back into FreeBSD working. Maybe someday, but not today. I understand the concepts of watchdog timers (telecommunications equipment, big routers, etc, use em) so would you be able to educate me on how your timer logic works? You can PM me if you rather. Your port notes (as viewed on Freshports) seem to suggest that the timer value changes based on machine load. Is that correct? Does it change during port build, or is it a static number that is determined at the beginning of port build and does not change during port build? (The timers I am familiar with use a static number.) I'm guessing this is getting off-topic so we can discuss this off-line, or not at all; up to you.
 
it's "marino", not "mario"

You're claiming that you don't know how to use an editor on freebsd which I find a bit hard to believe.

  • make clean
  • make patch
  • vi <work directory>/path/to/file/to/change
  • change "True" to "False" and save
  • make install
It's not rocket science.
 
PacketMan, I was able to replicate the same failure.

58r87FO.png


I have heavily overloaded this machine (a core 2 duo, 2 core no HT) with 4 builders x 2 jobs.

As you can see the build failed after 52 minutes, and it failed exactly at the same line as yours.

The interesting point is that the build failed now and never failed before (as I reported in a my previous post) and it failed rather quickly.

marino@ said:
The watchdog looks for stalled builds. It determines if a build is stalled if the build log doesn't increment in X minutes, where X is variable based on the load of the machine. X can range from 25 minutes to like 4 hours. If the watchdog is kicking in, the build machine is either vastly underpowered or vastly overloaded.
Click to expand...

The load shown is quite high 5.63 and doesn't add with what marino@ wrote above, so something else must be going on.

The other interesting thing to note is that synth was building simultaneously print/texlive-texmf, and it is interesting because unlike the majority of pkgs this one is I/O bound with a plist of approximately 86,000 files and building it result in lot of disk I/O.

Now, when the system is under heavy disk I/O load the CPU load decrease, and probably the watchdog threshold is lowered and that trigger the assassination :p of the build under the false assumption the system is under low load. That's all I can think of.

(to be noted, the failure happened while running cmake, which is also a tool that do some concurrent disk I/O activity).

That said, I really had to apply "some" overload to catch this.
 
"and it failed rather quickly."
didn't you just say it failed after 52 minutes?
What was the 5 minute load average? ~5.5 ?

yeah, that should have 25 x 4.5 minutes to fail (over 100)


EDIT:
actually it depends on the load at the time the build phase started, which could have been under 1.1 and thus 25 minutes is all that is required.
 
so I'm speculating that the machine is basically unloaded when webkit starts (5 minutes average less than 2.2 for a ncpu=2 machine) and thus the timeout period is only 25 minutes.
If the machine is loaded at 6 before webkit starts, the timeout would be 75 minutes (load * 25 / ncpu) when load > 2

On this particular machine it's basically bad luck with A) when webkit starts relative to load and B) what else is building at the same time.
 
marino@ said:
so I'm speculating that the machine is basically unloaded when webkit starts (5 minutes average less than 2.2 for a ncpu=2 machine) and thus the timeout period is only 25 minutes.
If the machine is loaded at 6 before webkit starts, the timeout would be 75 minutes (load * 25 / ncpu) when load > 2

On this particular machine it's basically bad luck with A) when webkit starts relative to load and B) what else is building at the same time.
Click to expand...

Sorry Marino, bad typing on my part.

I bet that's it. I am starting synth with no load on the box, its literally at idle and your watchdog logic starts with that low number. It won't be today, but I'll try to find a non-synth way to load my machine down to get that load number way up. Then I will start synth, and then I will shutdown the 'fake load'. I'm betting we have found the issue; this is a CPU/load measurement issue, not an availability (or lack of) resources issue.

I still stand by my opinion that my machine is not drastically or shockingly overloaded, and that my vmstat numbers prove that. I encourage a 2nd or 3rd opinion to chime in though. I will consider silence in my favor. :D
 
marino@ said:
it's "marino", not "mario"

You're claiming that you don't know how to use an editor on freebsd which I find a bit hard to believe.

  • make clean
  • make patch
  • vi <work directory>/path/to/file/to/change
  • change "True" to "False" and save
  • make install
It's not rocket science.
Click to expand...

Believe what you want, but I do not have these skills yet. That might not be rocket science to you, it it is to me. You have your skills and I have mine. Sorry.
 
PacketMan said:
Believe what you want, but I do not have these skills yet. That might not be rocket science to you, it it is to me. You have your skills and I have mine. Sorry.
Click to expand...

You do realize that I just gave you step by step instructions on how to do it, right?
You would be unable to use the ports collection at all if you were truly that unskilled. I refuse to believe that you can't edit a text file. But if it makes you happy I'll pretend I do.
 
PacketMan said:
Then I will start synth, and then I will shutdown the 'fake load'. I'm betting we have found the issue; this is a CPU/load measurement issue, not an availability (or lack of) resources issue.
Click to expand...

There's nothing "fake" about it.
It's still taking AT LEAST 25 minutes to get through one line with 1 builder and 1 job.
If that's true, it's waste of electricity to build packages on that machine. You should be using a more powerful machine and distributing the packages.

If you absolutely have to use that machine, I told you above how you can modify synth to disable the watchdog (at your own risk)
 
Again, no. Its a home use machine, and it gets a ports update like twice a year and its work very well, In fact synth is now the only port that has an issue on it. Portmaster works perfectly fine on it. But I do have options.
  • Use portmaster instead of synth (my likely choice as it drastically helps me learn the inner workings of FreeBSD)
  • Use Linux instead of FreeBSD. (not likely but that is what my colleagues would tell me to do)
  • Use synth as a remote repository builder. (I might try this initially just to learn something.)
You seem to have this horse blinder attitude that if its not newer overpowered hardware then its not worthy for synth to run on. That is a real shame and goes against the unix and freebsd spirit IMHO. The fact is the watchdog idea, while a novel and brilliantly simple one, is flawed thinking. Its 'confuses' speed with load, it confuses speed with stalled (or not). A very slow build is not a stalled build. A moderately busy slow machine is not the same as a moderately busy fast machine. By simply modifying the watchdog code to allow the user, via a menu option, to configure the timer for use on slower older hardware we resolve this issue of where the simple watchdog timer logic is creating false positives and aborting the process incorrectly. Would it kill you to do that Marino? (Other people will run into this issue sooner or later; btw I came close last night I do believe on an 8-core 8Gb ram machine. I just created a certain kind of scenario that synth happily allowed. Line 3924 took like 15 to 20 minutes. I might try to tinker with the scenario if I have time, but I'm not sure that will matter to you so I don't see the point.)

Best regards.
 
marino@ said:
What was the 5 minute load average? ~5.5 ?
Click to expand...
It seems to me that the load displayed on synth curses output and/or html report page is the 1 min load average, yet I don't know what the watchdog really use.
I'm pointing out this, as it could be an oversight ... probably the 5 minutes load average may provide more accurate results.
 
PacketMan said:
You seem to have this horse blinder attitude that if its not newer overpowered hardware then its not worthy for synth to run on. That is a real shame and goes against the unix and freebsd spirit IMHO.
Click to expand...

It seems to me that you are not realizing you have been the only one to meet that issue. I have replicate it on purpose, but to do that I had to clearly overload my machine, which otherwise always worked fine and it is also old hardware.

Marino@ has also told you how to disable the watchdog, as a possible workaround.

PacketMan said:
where the simple watchdog timer logic is creating false positives and aborting the process incorrectly.
Click to expand...
The load average is approximate, it can't be otherwise, the measure of the lines of output is also an approximation under the assumption that a running build process will output something, overall are providing good result, except on your machine for that specific port/pkg.

Marino@ as synth developer have to make choices, what should he favor for a building software ?
- optimize for new powered machine ?
- optimize for older underpowered machines ?

You also reported:
PacketMan said:
ronically, I have an even lesser powered machine (1GB ram, 1 cpu core), and no synth issues on it.
Click to expand...
Beware you could find another issue, as the build time field could overflow when greater than 24 hours per pkg.
 
Yes and as I said I don't skills yet to do that. Yes I know how to use a text editor but I don't understand where/what to edit. And I realize the load avg is approx, but we can if we want to give the timer more flexibility. And while I might be the 1st to discover this issue I won't be the last to be affected; they just might not make it known. I think the real question is do we want to open the door in making FreeBSD and its ports exclusive to newer hardware or do we take the stance that it should strive to support older hardware. (Since FreeBSD typically supports older hardware better than the newest stuff then we have to keep this in mind.) Synth and its author's attitude is opening that door. Some doors can't be shut once open. I don't believe in replacing a machine just because one of a few software management tools aborts due to false positive logic. At this point in time (and I know my voice has little weight), I recommend we not recommend Synth as the primary ports building tool, and that Portmaster stay the tool mentioned in the handbook.

Anyway we have discovered what the real issue is; false positive due to simple logic being trigger slower processing, and not overloaded machine. (My vmstats prove the machine is not overloaded; its just old fashioned slower.) IMHO the purpose of the watchdog timer was to detect and abort a stalled process. That is not what is happening here. Its just a slow cpu and the allocated time is not sufficient. Thus the true intent of the timer is failing its purpose on slower cpu hardware. Simple.

Please keep in mind the author has made it clear (and thus the port notes should reflect that):
  • Synth will actually not work on any machine. The author recommends overpowered, under-loaded machines.
  • Synth actually requires newer faster processing CPU.
  • Synth may have issue with machines that have spinning hard drives.
I'll mark this resolved. Thanks everyone. Best regards.
 
ASX said:
Marino@ as synth developer have to make choices, what should he favor for a building software ?
- optimize for new powered machine ?
- optimize for older underpowered machines ?
Click to expand...

Probably good idea to add something to detect the type of CPU to determine the timeout for watchdog. If it detects underpowered computer then it could turn off the watchdog automatically.
 
PacketMan said:
Yes and as I said I don't skills yet to do that. Yes I know how to use a text editor but I don't understand where/what to edit.
Click to expand...
Even after being given the exact file name, and the exact line, and the knowledge that it get placed in the port work directory after make patch command? It's probably closer to the truth that you don't want to know. I could have spent the time creating a patch for you to drop into files directory, but would you say you then don't have the skills to download the file, create the directory, and place the downloaded patch file in that directory?

Synth will actually not work on any machine. The author recommends overpowered, under-loaded machines.
Click to expand...

You are being a bit snarky here. I have not conceded this. Your machine is the ONLY example of this. I've gotten feedback from people that use slower / older hardware machines and they can all build gitweb and chromium, albeit after 12 hours. Your machine is exceptionally slow, maybe due to the disk drive.

Anyway we have discovered what the real issue is; false positive due to simple logic being trigger slower processing, and not overloaded machine.
Click to expand...
If you mean that an exceptionally slow machine can appear stalled because it's so slow, then fine.

Synth and its author's attitude is opening that door. Some doors can't be shut once open. I don't believe in replacing a machine just because one of a few software management tools aborts due to false positive logic. At this point in time (and I know my voice has little weight), I recommend we not recommend Synth as the primary ports building tool, and that Portmaster stay the tool mentioned in the handbook.
Click to expand...

Here you are revising history.
It's been recommended, by everyone, for poudriere too, to use repositories as they were meant to be used.
That means:
  1. Get your better or best machines to build packages and create a repository
  2. share that repository to all other machines of the same arch and release
  3. If necessary the designated build machine can create repositories for other arches and releases.
It was never the concept to have every machine build its own packages.
Everybody knows that package building by its nature requires a lot of resources.

If your point is that package building is aimed at newer and beefy hardware, then obviously yes.

If was *your* decision to buck all those recommendations. (earlier).

Please keep in mind the author has made it clear (and thus the port notes should reflect that):
  • Synth will actually not work on any machine. The author recommends overpowered, under-loaded machines.
  • Synth actually requires newer faster processing CPU.
  • Synth may have issue with machines that have spinning hard drives.
Click to expand...

Snarky, snarky, snarky.
The only requirement is that the machine be able to pass that line in webkit in 25 minutes with the minimum setting of 1x1. At this point in time, there's only 1 example of a machine that can't do it.
 
You must log in or register to reply here.
Back
Top