Nested Until Loop in bash Script for Loadleveler Job Submission is misbehaving?
Nested Until Loop in bash Script for Loadleveler Job Submission is misbehaving?
#!/bin/bash
rm WAVECAR k_conv k_plot
#k convergence
for i in 2 4 6 8 9 10 11 12 13 14 15 16 ;
do
cat >KPOINTS <<!
pymatgen 4.7.6+ generated KPOINTS with grid density = 875 / atom
0
Monkhorst
$i $i $i
!
echo "k = $i" ;
b=$(llsubmit multiple.cmd) ;
c=$(echo $b | awk '
ret=match($0,".in.")
rwt=match($0,"" has")
rqt=rwt-(ret+4)
subs=substr($0,(ret+4),rqt)
print subs
') ;
#c stores the job id
d=1 ;
until [ $d -eq 0 ]
do
b=$(llq $c) ; #llq is the command to query job information
d=$(echo $b | awk '
if ($0 ~ /mm17s003/)
print 1
else
print 0
')
done
E=$(tail -1 OSZICAR) ;
echo $i $E >> k_conv
done
awk 'print $1 "t" $(NF-5)' k_conv >> k_plot ; #Used to just write two specific columns from the file k_conv to k_plot.
This is the script file I use to submit jobs which changes the value of one input parameter in the file KPOINTS
before submitting the job every time. The for loop
is for this. multiple.cmd
is the actual job submission script and llsubmit
is the job submission command for loadleveler.
KPOINTS
for loop
multiple.cmd
llsubmit
The nested until loop
is used to query the job status for a particular submitted job. Unless the job is over, this loop keeps iterating. Once the job is done and the until loop
ends, an output file, OSZICAR
is generated. Awk is used to extract particular lines which are then stored in E. Finally, the for loop variable i
(i.e. the input parameter for the calculation that the script changes) and E
are store in a file k_conv
. End of for loop
.
nested until loop
until loop
OSZICAR
i
E
k_conv
for loop
In essence, due to the until loop
only one job should be submitted at a time. Once it's done, the next for loop
iteration commences and a new job is submitted. Somehow when I run the script, for any two or three random values of i
jobs are submitted simultaneously. I assume there is some issue with the until loop
.
until loop
for loop
i
until loop
Output of llsubmit multiple.cmd
that is initially stored in b: llsubmit: The job "c1hn1.virgo.iitm.ac.in.2077655" has been submitted.
llsubmit multiple.cmd
llsubmit: The job "c1hn1.virgo.iitm.ac.in.2077655" has been submitted.
c takes the job id (2077655
from the example above) as its value.
2077655
Output of llq $c
that is stored in b later: Id Owner Submitted ST PRI Class Running On ------------------------ ---------- ----------- -- --- ------------ ----------- c1hn1.2077655.0 mm17s003 9/9 23:43 I 50 Small 1 job step(s) in query, 1 waiting, 0 pending, 0 running, 0 held, 0 preempted
llq $c
Id Owner Submitted ST PRI Class Running On ------------------------ ---------- ----------- -- --- ------------ ----------- c1hn1.2077655.0 mm17s003 9/9 23:43 I 50 Small 1 job step(s) in query, 1 waiting, 0 pending, 0 running, 0 held, 0 preempted
Any help is immensely appreciated.
for
i=16
KPOINTS
wait
To reiterate @WalterA's point, please read How to Ask and Minimal, Complete, and Verifiable example. There is also a tour for new members. As a general comment, you should put some indentation in your code, it would make it much simpler to read (like when does the first
do
complete? With indentation there can be no doubt).– Nic3500
Sep 10 '18 at 0:22
do
All of the above, AND why confuse thing by using
$(..)
and `...`
forms of command substitution. Join the 90's and always use the $(...)
form. Do you know how to debug shell scripts using set -x
? It would be worth reading about. You can then see what values each of your variables are being set to. I would just run 2 cycles of the script with debugging. Good luck.– shellter
Sep 10 '18 at 4:11
$(..)
`...`
$(...)
set -x
@shellter Thank you for the tip on command substitution. This is my first time using bash script. I understand
set -x
can also be used inside the script as #!/bin/bash -x
but I'm unable to save the output to a file where I can study it. Do pardon me, I'm new.– Hitanshu Sachania
Sep 12 '18 at 11:19
set -x
#!/bin/bash -x
I had a look, your code does not seem wrong. So since the
until
loop completes, means that the llq
command returns something else than what you are expecting. At this ponig, it will be harder to debug from our perspective. I recommend you do: 1 set #!/bin/bash -x
and capture the output (run script.bash >output 2>errors
). 2 OR you could add print statement all over your code (ex. in the until, every time you loop print $b) and capture the output using the same redirections I showed in 1.– Nic3500
Sep 12 '18 at 23:16
until
llq
#!/bin/bash -x
script.bash >output 2>errors
0
Thanks for contributing an answer to Stack Overflow!
But avoid …
To learn more, see our tips on writing great answers.
Required, but never shown
Required, but never shown
By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.
Too much information. Can you narrow it down? The
for
loop withi=16
will overwrite previous results inKPOINTS
. Did you considerwait
for waiting background jobs to finish? How are these submitted?– Walter A
Sep 9 '18 at 21:46