Re: problems with output redirection
With local and locally I mean ofcourse on the host that is running local.sh e.g;. change local.sh to do ssh $HOST $REMOTE_SKRIPT >>$OUTPUT and remove the redirection to $OUTPUT in remote.sh Bert Bert Barbe wrote: Interesting would be to see how the file looks like on the other hosts ;) Maybe the file is being extended on the remote host first but the write() isn't synced before the local host writes again. Why not redirect the output off ssh locally so that all writes are local ? That is if the output isn't huge ofcourse. I would imagine this is more of an nfs issue than a bash issue. Regards, Bert Verena Alishahi wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Dear list-members, last week I've updated my workstation cluster client (amd 64 bit) from SuSE 10.0 to openSUSE 10.3. My bash-version is 3.2.25(1)-release. Now I've problems with a bash script, which I used (with bash 3.1) for years without any problems. I really don't understand what's going on. I've reduced my original script to a minimal script which reproduces the output redirection error. The script works as follows: My first script (local.sh) reads a list of worksation cluster clients from hosts.inp in a loop and prints some output to output.txt. In the loop a second script (remote.sh) is called in a ssh instruction. remote.sh then writes some output to the same output.txt file, but this output doesn't reach the output file correctly. Both the bash-scripts and the output file are stored on an NFS-Filesystem (on the workstation cluster server), so every client can access them. | less hosts.inp client1 client2 client3 client4 | less local.sh #!/bin/bash OUTPUT=$HOME/bashtest/output.txt INPUT_HOSTS=$HOME/bashtest/hosts.inp REMOTE_SKRIPT=$HOME/bashtest/remote.sh echo "created from $HOME/bashtest/local.sh" > $OUTPUT for HOST in `cat $INPUT_HOSTS` do ~ echo $HOST >> $OUTPUT ~ ssh $HOST $REMOTE_SKRIPT ~ echo "" >> $OUTPUT done | less remote.sh #!/bin/bash OUTPUT=$HOME/bashtest/output.txt echo "TEST2" >> $OUTPUT | less output.txt created from /home/user1/bashtest/local.sh client1 [EMAIL PROTECTED]@[EMAIL PROTECTED]@[EMAIL PROTECTED]@ client2 [EMAIL PROTECTED]@[EMAIL PROTECTED]@[EMAIL PROTECTED]@ client3 [EMAIL PROTECTED]@[EMAIL PROTECTED]@[EMAIL PROTECTED]@ client4 [EMAIL PROTECTED]@[EMAIL PROTECTED]@[EMAIL PROTECTED]@ Instead of all the [EMAIL PROTECTED]@[EMAIL PROTECTED]@[EMAIL PROTECTED]@ rows, output.txt should contain the text TEST2 less thinks output.txt were a binary file. I don't understand, why the output-redirection in remote.sh doesn't work. If I do the output-redirection in remote.sh to another file, for example OUTPUT2=$HOME/bashtest/output2.txt echo "TEST2" >> $OUTPUT2 then it works. The output-redirection only fails, when the redirection is to the same file which is also be written from local.sh. I appreciate receiving any help. best regards, Verena Alishahi - -- Verena Alishahi Institut für Chemie und Dynamik der Geosphäre ICG-1: Stratosphäre Tel.: +49 2461 61-6221 E-Mail: [EMAIL PROTECTED] - -- - -- Forschungszentrum Jülich GmbH 52425 Jülich Sitz der Gesellschaft: Jülich Eingetragen im Handelsregister des Amtsgerichts Düren Nr. HR B 3498 Vorsitzende des Aufsichtsrats: MinDirig'in Bärbel Brumme-Bothe Geschäftsführung: Prof. Dr. Achim Bachem (Vorsitzender), Dr. Ulrich Krafft (stellv. Vorsitzender); Dr. Sebastian M. Schmidt - -- - -- -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.4-svn0 (GNU/Linux) Comment: Using GnuPG with SUSE - http://enigmail.mozdev.org iD8DBQFH80XdB/rJPqfoUGARAsxsAKDW/2VyTD9vZ5hwqaBHetahrkUCTgCghVVP 6L93pwaFPbfj7O3w0IjOdSs= =+SC6 -END PGP SIGNATURE- --- --- Forschungszentrum Jülich GmbH 52425 Jülich Sitz der Gesellschaft: Jülich Eingetragen im Handelsregister des Amtsgerichts Düren Nr. HR B 3498 Vorsitzende des Aufsichtsrats: MinDir'in Bärbel Brumme-Bothe Geschäftsführung: Prof. Dr. Achim Bachem (Vorsitzender), Dr. Ulrich Krafft (stellv. Vorsitzender), Prof. Dr. Harald Bolt, Dr. Sebastian M. Schmidt --- ---
Re: problems with output redirection
Verena Alishahi wrote: > My first script (local.sh) reads a list of worksation cluster clients > from hosts.inp in a loop and prints some output to output.txt. In the > loop a second script (remote.sh) is called in a ssh instruction. > remote.sh then writes some output to the same output.txt file, but this > output doesn't reach the output file correctly. > > Both the bash-scripts and the output file are stored on an > NFS-Filesystem (on the workstation cluster server), so every client can > access them. I would expect this to fail anyways. Regardless if it worked before or not (I guess that was more luck). Your problem reduces (IMHO) to concurrent access to the same file without mutual exclusion. I didn't expect the glibberish to be in the file (the NULs), more that the order is puzzled, but I guess that's a consequence of the NFS stateless access method. Just my humble opinion, I'm not an expert in this. J.
Re: problems with output redirection
Interesting would be to see how the file looks like on the other hosts ;) Maybe the file is being extended on the remote host first but the write() isn't synced before the local host writes again. Why not redirect the output off ssh locally so that all writes are local ? That is if the output isn't huge ofcourse. I would imagine this is more of an nfs issue than a bash issue. Regards, Bert Verena Alishahi wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Dear list-members, last week I've updated my workstation cluster client (amd 64 bit) from SuSE 10.0 to openSUSE 10.3. My bash-version is 3.2.25(1)-release. Now I've problems with a bash script, which I used (with bash 3.1) for years without any problems. I really don't understand what's going on. I've reduced my original script to a minimal script which reproduces the output redirection error. The script works as follows: My first script (local.sh) reads a list of worksation cluster clients from hosts.inp in a loop and prints some output to output.txt. In the loop a second script (remote.sh) is called in a ssh instruction. remote.sh then writes some output to the same output.txt file, but this output doesn't reach the output file correctly. Both the bash-scripts and the output file are stored on an NFS-Filesystem (on the workstation cluster server), so every client can access them. | less hosts.inp client1 client2 client3 client4 | less local.sh #!/bin/bash OUTPUT=$HOME/bashtest/output.txt INPUT_HOSTS=$HOME/bashtest/hosts.inp REMOTE_SKRIPT=$HOME/bashtest/remote.sh echo "created from $HOME/bashtest/local.sh" > $OUTPUT for HOST in `cat $INPUT_HOSTS` do ~ echo $HOST >> $OUTPUT ~ ssh $HOST $REMOTE_SKRIPT ~ echo "" >> $OUTPUT done | less remote.sh #!/bin/bash OUTPUT=$HOME/bashtest/output.txt echo "TEST2" >> $OUTPUT | less output.txt created from /home/user1/bashtest/local.sh client1 [EMAIL PROTECTED]@[EMAIL PROTECTED]@[EMAIL PROTECTED]@ client2 [EMAIL PROTECTED]@[EMAIL PROTECTED]@[EMAIL PROTECTED]@ client3 [EMAIL PROTECTED]@[EMAIL PROTECTED]@[EMAIL PROTECTED]@ client4 [EMAIL PROTECTED]@[EMAIL PROTECTED]@[EMAIL PROTECTED]@ Instead of all the [EMAIL PROTECTED]@[EMAIL PROTECTED]@[EMAIL PROTECTED]@ rows, output.txt should contain the text TEST2 less thinks output.txt were a binary file. I don't understand, why the output-redirection in remote.sh doesn't work. If I do the output-redirection in remote.sh to another file, for example OUTPUT2=$HOME/bashtest/output2.txt echo "TEST2" >> $OUTPUT2 then it works. The output-redirection only fails, when the redirection is to the same file which is also be written from local.sh. I appreciate receiving any help. best regards, Verena Alishahi - -- Verena Alishahi Institut für Chemie und Dynamik der Geosphäre ICG-1: Stratosphäre Tel.: +49 2461 61-6221 E-Mail: [EMAIL PROTECTED] - -- - -- Forschungszentrum Jülich GmbH 52425 Jülich Sitz der Gesellschaft: Jülich Eingetragen im Handelsregister des Amtsgerichts Düren Nr. HR B 3498 Vorsitzende des Aufsichtsrats: MinDirig'in Bärbel Brumme-Bothe Geschäftsführung: Prof. Dr. Achim Bachem (Vorsitzender), Dr. Ulrich Krafft (stellv. Vorsitzender); Dr. Sebastian M. Schmidt - -- - -- -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.4-svn0 (GNU/Linux) Comment: Using GnuPG with SUSE - http://enigmail.mozdev.org iD8DBQFH80XdB/rJPqfoUGARAsxsAKDW/2VyTD9vZ5hwqaBHetahrkUCTgCghVVP 6L93pwaFPbfj7O3w0IjOdSs= =+SC6 -END PGP SIGNATURE- --- --- Forschungszentrum Jülich GmbH 52425 Jülich Sitz der Gesellschaft: Jülich Eingetragen im Handelsregister des Amtsgerichts Düren Nr. HR B 3498 Vorsitzende des Aufsichtsrats: MinDir'in Bärbel Brumme-Bothe Geschäftsführung: Prof. Dr. Achim Bachem (Vorsitzender), Dr. Ulrich Krafft (stellv. Vorsitzender), Prof. Dr. Harald Bolt, Dr. Sebastian M. Schmidt --- ---
problems with output redirection
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Dear list-members, last week I've updated my workstation cluster client (amd 64 bit) from SuSE 10.0 to openSUSE 10.3. My bash-version is 3.2.25(1)-release. Now I've problems with a bash script, which I used (with bash 3.1) for years without any problems. I really don't understand what's going on. I've reduced my original script to a minimal script which reproduces the output redirection error. The script works as follows: My first script (local.sh) reads a list of worksation cluster clients from hosts.inp in a loop and prints some output to output.txt. In the loop a second script (remote.sh) is called in a ssh instruction. remote.sh then writes some output to the same output.txt file, but this output doesn't reach the output file correctly. Both the bash-scripts and the output file are stored on an NFS-Filesystem (on the workstation cluster server), so every client can access them. | less hosts.inp client1 client2 client3 client4 | less local.sh #!/bin/bash OUTPUT=$HOME/bashtest/output.txt INPUT_HOSTS=$HOME/bashtest/hosts.inp REMOTE_SKRIPT=$HOME/bashtest/remote.sh echo "created from $HOME/bashtest/local.sh" > $OUTPUT for HOST in `cat $INPUT_HOSTS` do ~ echo $HOST >> $OUTPUT ~ ssh $HOST $REMOTE_SKRIPT ~ echo "" >> $OUTPUT done | less remote.sh #!/bin/bash OUTPUT=$HOME/bashtest/output.txt echo "TEST2" >> $OUTPUT | less output.txt created from /home/user1/bashtest/local.sh client1 [EMAIL PROTECTED]@[EMAIL PROTECTED]@[EMAIL PROTECTED]@ client2 [EMAIL PROTECTED]@[EMAIL PROTECTED]@[EMAIL PROTECTED]@ client3 [EMAIL PROTECTED]@[EMAIL PROTECTED]@[EMAIL PROTECTED]@ client4 [EMAIL PROTECTED]@[EMAIL PROTECTED]@[EMAIL PROTECTED]@ Instead of all the [EMAIL PROTECTED]@[EMAIL PROTECTED]@[EMAIL PROTECTED]@ rows, output.txt should contain the text TEST2 less thinks output.txt were a binary file. I don't understand, why the output-redirection in remote.sh doesn't work. If I do the output-redirection in remote.sh to another file, for example OUTPUT2=$HOME/bashtest/output2.txt echo "TEST2" >> $OUTPUT2 then it works. The output-redirection only fails, when the redirection is to the same file which is also be written from local.sh. I appreciate receiving any help. best regards, Verena Alishahi - -- Verena Alishahi Institut für Chemie und Dynamik der Geosphäre ICG-1: Stratosphäre Tel.: +49 2461 61-6221 E-Mail: [EMAIL PROTECTED] - -- - -- Forschungszentrum Jülich GmbH 52425 Jülich Sitz der Gesellschaft: Jülich Eingetragen im Handelsregister des Amtsgerichts Düren Nr. HR B 3498 Vorsitzende des Aufsichtsrats: MinDirig'in Bärbel Brumme-Bothe Geschäftsführung: Prof. Dr. Achim Bachem (Vorsitzender), Dr. Ulrich Krafft (stellv. Vorsitzender); Dr. Sebastian M. Schmidt - -- - -- -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.4-svn0 (GNU/Linux) Comment: Using GnuPG with SUSE - http://enigmail.mozdev.org iD8DBQFH80XdB/rJPqfoUGARAsxsAKDW/2VyTD9vZ5hwqaBHetahrkUCTgCghVVP 6L93pwaFPbfj7O3w0IjOdSs= =+SC6 -END PGP SIGNATURE- --- --- Forschungszentrum Jülich GmbH 52425 Jülich Sitz der Gesellschaft: Jülich Eingetragen im Handelsregister des Amtsgerichts Düren Nr. HR B 3498 Vorsitzende des Aufsichtsrats: MinDir'in Bärbel Brumme-Bothe Geschäftsführung: Prof. Dr. Achim Bachem (Vorsitzender), Dr. Ulrich Krafft (stellv. Vorsitzender), Prof. Dr. Harald Bolt, Dr. Sebastian M. Schmidt --- ---
Re: problems with output redirection
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Dear Bert, thank you for your suggestion. The way to redirect only from local.sh is working. I thought this wouldn't work, because in my original script remote.sh I also redirect to another temporary output-file. In this case I was wrong. Thank you very much! best regards, Verena Bert Barbe wrote: | With local and locally I mean ofcourse on the host that is running local.sh | e.g;. change local.sh to do | ssh $HOST $REMOTE_SKRIPT >>$OUTPUT | and remove the redirection to $OUTPUT in remote.sh | | Bert | | Bert Barbe wrote: |> Interesting would be to see how the file looks like on the other hosts ;) |> Maybe the file is being extended on the remote host first but the |> write() isn't synced before the local host writes again. |> Why not redirect the output off ssh locally so that all writes are |> local ? That is if the output isn't huge ofcourse. |> I would imagine this is more of an nfs issue than a bash issue. |> |> Regards, |> Bert |> |> Verena Alishahi wrote: | Dear list-members, | | last week I've updated my workstation cluster client (amd 64 bit) from | SuSE 10.0 to openSUSE 10.3. My bash-version is 3.2.25(1)-release. | | Now I've problems with a bash script, which I used (with bash 3.1) for | years without any problems. I really don't understand what's going on. | | I've reduced my original script to a minimal script which reproduces the | output redirection error. The script works as follows: | | My first script (local.sh) reads a list of worksation cluster clients | from hosts.inp in a loop and prints some output to output.txt. In the | loop a second script (remote.sh) is called in a ssh instruction. | remote.sh then writes some output to the same output.txt file, but this | output doesn't reach the output file correctly. | | Both the bash-scripts and the output file are stored on an | NFS-Filesystem (on the workstation cluster server), so every client can | access them. | | | less hosts.inp | client1 | client2 | client3 | client4 | | | less local.sh | #!/bin/bash | OUTPUT=$HOME/bashtest/output.txt | INPUT_HOSTS=$HOME/bashtest/hosts.inp | REMOTE_SKRIPT=$HOME/bashtest/remote.sh | echo "created from $HOME/bashtest/local.sh" > $OUTPUT | for HOST in `cat $INPUT_HOSTS` | do | ~ echo $HOST >> $OUTPUT | ~ ssh $HOST $REMOTE_SKRIPT | ~ echo "" >> $OUTPUT | done | | | less remote.sh | #!/bin/bash | OUTPUT=$HOME/bashtest/output.txt | echo "TEST2" >> $OUTPUT | | | less output.txt | created from /home/user1/bashtest/local.sh | client1 | [EMAIL PROTECTED]@[EMAIL PROTECTED]@[EMAIL PROTECTED]@ | client2 | [EMAIL PROTECTED]@[EMAIL PROTECTED]@[EMAIL PROTECTED]@ | client3 | [EMAIL PROTECTED]@[EMAIL PROTECTED]@[EMAIL PROTECTED]@ | client4 | [EMAIL PROTECTED]@[EMAIL PROTECTED]@[EMAIL PROTECTED]@ | | | Instead of all the [EMAIL PROTECTED]@[EMAIL PROTECTED]@[EMAIL PROTECTED]@ rows, output.txt should contain the text | TEST2 | less thinks output.txt were a binary file. | I don't understand, why the output-redirection in remote.sh doesn't | work. If I do the output-redirection in remote.sh to another file, for | example | OUTPUT2=$HOME/bashtest/output2.txt | echo "TEST2" >> $OUTPUT2 | then it works. | The output-redirection only fails, when the redirection is to the same | file which is also be written from local.sh. | | I appreciate receiving any help. | | best regards, | Verena Alishahi |>> |>> - --- - --- Forschungszentrum Jülich GmbH 52425 Jülich |>> Sitz der Gesellschaft: Jülich Eingetragen im Handelsregister des Amtsgerichts Düren Nr. HR B 3498 Vorsitzende des Aufsichtsrats: MinDir'in Bärbel Brumme-Bothe Geschäftsführung: Prof. Dr. Achim Bachem (Vorsitzender), Dr. Ulrich Krafft (stellv. Vorsitzender), Prof. Dr. Harald Bolt, Dr. Sebastian M. Schmidt - --- - --- |>> |>> |> |> - -- Verena Alishahi Institut für Chemie und Dynamik der Geosphäre ICG-1: Stratosphäre Tel.: +49 2461 61-6221 E-Mail: [EMAIL PROTECTED] - -- - -- Forschungszentrum Jülich GmbH 52425 Jülich Sitz der Gesellschaft: Jülich Eingetragen im Handelsregister des Amtsgerichts Düren Nr. HR B 3498 Vorsitzende des Aufsichtsrats: MinDirig'in Bärbel Brumme-Bothe Geschäftsführung: Prof. Dr. Achim Bachem (Vorsitzender), Dr. Ulrich Krafft (stellv. Vorsitzender); Dr. Sebastian M. Schmidt - -- - -- -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.4-svn0 (GNU/Linux) Comment: Using GnuPG with SUSE - http://enigmail.mozdev.org iD8DBQFH81eWB/rJPqfoUGARAk1zAKDOqfYnybaRIG+x7ju8
Re: problems with output redirection
Verena Alishahi wrote: > last week I've updated my workstation cluster client (amd 64 bit) from > SuSE 10.0 to openSUSE 10.3. My bash-version is 3.2.25(1)-release. > ... > Both the bash-scripts and the output file are stored on an > NFS-Filesystem (on the workstation cluster server), so every client can > access them. I believe the use of NFS to be the root cause of your problem. The NFS synchronization here will be related to the NFS version in the kernel. Changing that has caused this to change behavior. I don't think this is related to changes in bash's version. Effectively echo is the same echo as before. > | less local.sh > #!/bin/bash > OUTPUT=$HOME/bashtest/output.txt > INPUT_HOSTS=$HOME/bashtest/hosts.inp > REMOTE_SKRIPT=$HOME/bashtest/remote.sh > echo "created from $HOME/bashtest/local.sh" > $OUTPUT > for HOST in `cat $INPUT_HOSTS` > do > ~ echo $HOST >> $OUTPUT > ~ ssh $HOST $REMOTE_SKRIPT > ~ echo "" >> $OUTPUT > done Why are there '~' chars in the above lines? But the real problem is that you are going to be toggling back and forth between different nfs client hosts writing to the same file. The writes will look like this in very close succession: local>> $OUTPUT client1 >> $OUTPUT local>> $OUTPUT client2 >> $OUTPUT local>> $OUTPUT client3 >> $OUTPUT local>> $OUTPUT client4 >> $OUTPUT local>> $OUTPUT I am confident this will stress the nfs client subsystem to failure. The problem is one of distributed nfs cache coherency in the buffer cache. It is not trivial to solve this problem. You might be able to get more expert discussion of the problem on the nfs mailing lists. You might be able to convince yourself of this through brute force by adding sleep statements to slow things down enough to give enough time to flush the cache between writes. I am not advocating it as a solution but only as a debugging data point. Google for "useless use of cat". In different contexts such as removing and recreating the file you would normally see "stale nfs file handle" errors. I don't think there is anything that bash can do to affect this since it is just doing normal output. The reason you tripped into this wasn't the bash version update but the system kernel upgrade which brought in a new nfs subsystem. You might be able to affect things favorably by changing the mount options. Bob
