Nested Until Loop in bash Script for Loadleveler Job Submission is misbehaving?

Nested Until Loop in bash Script for Loadleveler Job Submission is misbehaving?


#!/bin/bash

rm WAVECAR k_conv k_plot
#k convergence

for i in 2 4 6 8 9 10 11 12 13 14 15 16 ;
do
cat >KPOINTS <<!
pymatgen 4.7.6+ generated KPOINTS with grid density = 875 / atom
0
Monkhorst
$i $i $i
!

echo "k = $i" ;

b=$(llsubmit multiple.cmd) ;

c=$(echo $b | awk '
ret=match($0,".in.")
rwt=match($0,"" has")
rqt=rwt-(ret+4)
subs=substr($0,(ret+4),rqt)
print subs
') ;

#c stores the job id

d=1 ;

until [ $d -eq 0 ]
do
b=$(llq $c) ; #llq is the command to query job information

d=$(echo $b | awk '
if ($0 ~ /mm17s003/)
print 1
else
print 0
')
done

E=$(tail -1 OSZICAR) ;

echo $i $E >> k_conv
done

awk 'print $1 "t" $(NF-5)' k_conv >> k_plot ; #Used to just write two specific columns from the file k_conv to k_plot.



This is the script file I use to submit jobs which changes the value of one input parameter in the file KPOINTS before submitting the job every time. The for loop is for this. multiple.cmd is the actual job submission script and llsubmit is the job submission command for loadleveler.


KPOINTS


for loop


multiple.cmd


llsubmit



The nested until loop is used to query the job status for a particular submitted job. Unless the job is over, this loop keeps iterating. Once the job is done and the until loop ends, an output file, OSZICAR is generated. Awk is used to extract particular lines which are then stored in E. Finally, the for loop variable i (i.e. the input parameter for the calculation that the script changes) and E are store in a file k_conv. End of for loop.


nested until loop


until loop


OSZICAR


i


E


k_conv


for loop



In essence, due to the until loop only one job should be submitted at a time. Once it's done, the next for loop iteration commences and a new job is submitted. Somehow when I run the script, for any two or three random values of i jobs are submitted simultaneously. I assume there is some issue with the until loop.


until loop


for loop


i


until loop



Output of llsubmit multiple.cmd that is initially stored in b: llsubmit: The job "c1hn1.virgo.iitm.ac.in.2077655" has been submitted.


llsubmit multiple.cmd


llsubmit: The job "c1hn1.virgo.iitm.ac.in.2077655" has been submitted.



c takes the job id (2077655 from the example above) as its value.


2077655



Output of llq $c that is stored in b later: Id Owner Submitted ST PRI Class Running On ------------------------ ---------- ----------- -- --- ------------ ----------- c1hn1.2077655.0 mm17s003 9/9 23:43 I 50 Small 1 job step(s) in query, 1 waiting, 0 pending, 0 running, 0 held, 0 preempted


llq $c


Id Owner Submitted ST PRI Class Running On ------------------------ ---------- ----------- -- --- ------------ ----------- c1hn1.2077655.0 mm17s003 9/9 23:43 I 50 Small 1 job step(s) in query, 1 waiting, 0 pending, 0 running, 0 held, 0 preempted



Any help is immensely appreciated.






Too much information. Can you narrow it down? The for loop with i=16 will overwrite previous results in KPOINTS. Did you consider wait for waiting background jobs to finish? How are these submitted?

– Walter A
Sep 9 '18 at 21:46


for


i=16


KPOINTS


wait






To reiterate @WalterA's point, please read How to Ask and Minimal, Complete, and Verifiable example. There is also a tour for new members. As a general comment, you should put some indentation in your code, it would make it much simpler to read (like when does the first do complete? With indentation there can be no doubt).

– Nic3500
Sep 10 '18 at 0:22



do






All of the above, AND why confuse thing by using $(..) and `...` forms of command substitution. Join the 90's and always use the $(...) form. Do you know how to debug shell scripts using set -x? It would be worth reading about. You can then see what values each of your variables are being set to. I would just run 2 cycles of the script with debugging. Good luck.

– shellter
Sep 10 '18 at 4:11



$(..)


`...`


$(...)


set -x






@shellter Thank you for the tip on command substitution. This is my first time using bash script. I understand set -x can also be used inside the script as #!/bin/bash -x but I'm unable to save the output to a file where I can study it. Do pardon me, I'm new.

– Hitanshu Sachania
Sep 12 '18 at 11:19


set -x


#!/bin/bash -x






I had a look, your code does not seem wrong. So since the until loop completes, means that the llq command returns something else than what you are expecting. At this ponig, it will be harder to debug from our perspective. I recommend you do: 1 set #!/bin/bash -x and capture the output (run script.bash >output 2>errors). 2 OR you could add print statement all over your code (ex. in the until, every time you loop print $b) and capture the output using the same redirections I showed in 1.

– Nic3500
Sep 12 '18 at 23:16


until


llq


#!/bin/bash -x


script.bash >output 2>errors




0



Thanks for contributing an answer to Stack Overflow!



But avoid



To learn more, see our tips on writing great answers.



Required, but never shown



Required, but never shown




By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Popular posts from this blog

𛂒𛀶,𛀽𛀑𛂀𛃧𛂓𛀙𛃆𛃑𛃷𛂟𛁡𛀢𛀟𛁤𛂽𛁕𛁪𛂟𛂯,𛁞𛂧𛀴𛁄𛁠𛁼𛂿𛀤 𛂘,𛁺𛂾𛃭𛃭𛃵𛀺,𛂣𛃍𛂖𛃶 𛀸𛃀𛂖𛁶𛁏𛁚 𛂢𛂞 𛁰𛂆𛀔,𛁸𛀽𛁓𛃋𛂇𛃧𛀧𛃣𛂐𛃇,𛂂𛃻𛃲𛁬𛃞𛀧𛃃𛀅 𛂭𛁠𛁡𛃇𛀷𛃓𛁥,𛁙𛁘𛁞𛃸𛁸𛃣𛁜,𛂛,𛃿,𛁯𛂘𛂌𛃛𛁱𛃌𛂈𛂇 𛁊𛃲,𛀕𛃴𛀜 𛀶𛂆𛀶𛃟𛂉𛀣,𛂐𛁞𛁾 𛁷𛂑𛁳𛂯𛀬𛃅,𛃶𛁼

Edmonton

Crossroads (UK TV series)