[GE users] max array job tasks - and load

Reuti reuti at staff.uni-marburg.de
Fri Oct 31 14:47:25 GMT 2008


Am 31.10.2008 um 15:31 schrieb Joseph Hargitai:

> Here is the qstat of a few nodes, which is interesting, and hence  
> we are puzzled:
> Even on nodes where load is over 150 - you can login, do tasks, no  
> sign of stress. You do find 16 processes running. Finishing time of  
> the jobs is normal. So maybe it is just a misreport. Also, how do  
> you get new job scheduled when load drops from 190 to 120? But we do.
>
>
> nyu-public at compute-7-7.local   BIP   6/16      123.12   lx26- 
> amd64    a

The queue is in alarm state. No new jobs should be scheduled thereto.  
What is:

$ qstat -explain a

saying?

When you check the process tree: are there some processes in state "D"?

$ ps -e f (f w/o -)

Are these jobs threaded and each of them is putting a load of 16 on  
the node by accident as it only checks the number of installed cores  
but not the number of granted slots?

-- Reuti


>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 09:10:50     1 1276
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 09:10:50     1 1277
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 09:10:50     1 1280
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 09:10:50     1 1282
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 09:10:50     1 1284
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 09:10:50     1 1285
> ---------------------------------------------------------------------- 
> ------
> nyu-public at compute-7-8.local   BIP   0/16      37.20    lx26- 
> amd64    a
> ---------------------------------------------------------------------- 
> ------
> nyu-public at compute-7-9.local   BIP   8/16      131.74   lx26- 
> amd64    a
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 09:13:05     1 1303
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 09:13:05     1 1304
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 09:13:05     1 1306
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 09:13:05     1 1309
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 09:13:05     1 1310
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 09:13:05     1 1316
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 09:13:05     1 1317
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 09:13:05     1 1318
> ---------------------------------------------------------------------- 
> ------
> nyu-public at compute-8-0.local   BIP   12/16     178.31   lx26- 
> amd64    a
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 09:22:35     1 1335
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 09:22:35     1 1336
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 09:22:35     1 1338
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 09:22:35     1 1339
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 09:22:35     1 1340
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 09:22:35     1 1341
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 09:22:35     1 1343
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 09:22:35     1 1344
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 09:22:35     1 1345
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 09:22:35     1 1346
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 09:22:35     1 1347
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 09:22:35     1 1348
> ---------------------------------------------------------------------- 
> ------
> nyu-public at compute-8-1.local   BIP   0/16      38.41    lx26- 
> amd64    a
> ---------------------------------------------------------------------- 
> ------
> nyu-public at compute-8-2.local   BIP   16/16     138.58   lx26- 
> amd64    a
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 10:05:20     1 1367
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 10:05:20     1 1368
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 10:05:20     1 1369
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 10:05:20     1 1370
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 10:05:20     1 1371
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 10:05:20     1 1372
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 10:05:20     1 1373
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 10:05:20     1 1374
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 10:05:20     1 1375
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 10:05:20     1 1376
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 10:05:20     1 1377
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 10:05:20     1 1378
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 10:05:20     1 1379
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 10:05:20     1 1380
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 10:05:20     1 1381
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 10:05:20     1 1382
> ---------------------------------------------------------------------- 
> ------
> nyu-public at compute-8-3.local   BIP   16/16     183.74   lx26- 
> amd64    a
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 10:11:35     1 1383
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 10:11:35     1 1384
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 10:11:35     1 1385
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 10:11:35     1 1386
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 10:11:35     1 1387
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 10:11:35     1 1388
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 10:11:35     1 1389
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 10:11:35     1 1390
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 10:11:35     1 1391
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 10:11:35     1 1392
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 10:11:35     1 1393
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 10:11:35     1 1394
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 10:11:35     1 1395
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 10:11:35     1 1396
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 10:11:35     1 1397
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 10:11:35     1 1398
> ---------------------------------------------------------------------- 
> ------
> nyu-public at compute-8-4.local   BIP   0/16      78.40    lx26- 
> amd64    a
> ---------------------------------------------------------------------- 
> ------
> nyu-public at compute-8-5.local   BIP   6/16      119.69   lx26- 
> amd64    a
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 09:13:35     1 1320
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 09:13:35     1 1323
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 09:13:35     1 1325
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 09:13:35     1 1329
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 09:13:35     1 1331
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 09:13:35     1 1333
> ---------------------------------------------------------------------- 
> ------
> nyu-public at compute-8-6.local   BIP   16/16     177.44   lx26- 
> amd64    a
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 10:12:05     1 1399
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 10:12:05     1 1400
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 10:12:05     1 1401
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 10:12:05     1 1402
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 10:12:05     1 1403
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 10:12:05     1 1404
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 10:12:05     1 1405
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 10:12:05     1 1406
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 10:12:05     1 1407
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 10:12:05     1 1408
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 10:12:05     1 1409
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 10:12:05     1 1410
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 10:12:05     1 1411
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 10:12:05     1 1412
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 10:12:05     1 1413
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 10:12:05     1 1414
> ---------------------------------------------------------------------- 
> ------
> nyu-public at compute-8-7.local   BIP   15/16     192.75   lx26- 
> amd64    a
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 09:33:50     1 1351
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 09:33:50     1 1352
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 09:33:50     1 1353
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 09:33:50     1 1354
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 09:33:50     1 1355
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 09:33:50     1 1357
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 09:33:50     1 1358
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 09:33:50     1 1359
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 09:33:50     1 1360
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 09:33:50     1 1361
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 09:33:50     1 1362
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 09:33:50     1 1363
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 09:33:50     1 1364
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 09:33:50     1 1365
>    2317 0.50500 own97-100- btk218       r     10/31/2008  
> 09:33:50     1 1366
> ---------------------------------------------------------------------- 
> ------
>
> j
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
> For additional commands, e-mail: users-help at gridengine.sunsource.net
>


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: users-help at gridengine.sunsource.net




More information about the gridengine-users mailing list