How can I check if a file can be created or truncated/overwritten in bash?









up vote
14
down vote

favorite
2












The user calls my script with a file path that will be either be created or overwritten at some point in the script, like foo.sh file.txt or foo.sh dir/file.txt.



The create-or-overwrite behavior is much like the requirements for putting the file on the right side of the > output redirect operator, or passing it as an argument to tee (in fact, passing it as an argument to tee is exactly what I'm doing).



Before I get into the guts of the script, I want to make a reasonable check if the file can be created/overwritten, but not actually create it. This check doesn't have to be perfect, and yes I realize that the situation can change between the check and the point where the file is actually written - but here I'm OK with a best effort type solution so I can bail out early in the case that the file path is invalid.



Examples of reasons the file couldn't created:



  • the file contains a directory component, like dir/file.txt but the directory dir doesn't exist

  • the user doens't have write permissions in the specified directory (or the CWD if no directory was specified

Yes, I realize that checking permissions "up front" is not The UNIX Way™, rather I should just try the operation and ask forgiveness later. In my particular script however, this leads to a bad user experience and I can't change the responsible component.










share|improve this question























  • Will the argument always be based on the current directory or could the user specify a full path?
    – Jesse_b
    Nov 8 at 22:08










  • @Jesse_b - I suppose the user could specify an absolute path like /foo/bar/file.txt. Basically I pass the path to tee like tee $OUT_FILE where OUT_FILE is passed on the command line. That should "just work" with both absolute and relative paths, right?
    – BeeOnRope
    Nov 8 at 22:09










  • @BeeOnRope, no you'd need tee -- "$OUT_FILE" at least. What if the file already exists or exists but is not a regular file (directory, symlink, fifo)?
    – Stéphane Chazelas
    Nov 8 at 22:37










  • @StéphaneChazelas - well I am using tee "$OUT_FILE.tmp". If the file already exists, tee overwrites, which is the desired behavior in this case. If it's a directory, tee will fail (I think). symlink I'm not 100% sure?
    – BeeOnRope
    Nov 8 at 22:39














up vote
14
down vote

favorite
2












The user calls my script with a file path that will be either be created or overwritten at some point in the script, like foo.sh file.txt or foo.sh dir/file.txt.



The create-or-overwrite behavior is much like the requirements for putting the file on the right side of the > output redirect operator, or passing it as an argument to tee (in fact, passing it as an argument to tee is exactly what I'm doing).



Before I get into the guts of the script, I want to make a reasonable check if the file can be created/overwritten, but not actually create it. This check doesn't have to be perfect, and yes I realize that the situation can change between the check and the point where the file is actually written - but here I'm OK with a best effort type solution so I can bail out early in the case that the file path is invalid.



Examples of reasons the file couldn't created:



  • the file contains a directory component, like dir/file.txt but the directory dir doesn't exist

  • the user doens't have write permissions in the specified directory (or the CWD if no directory was specified

Yes, I realize that checking permissions "up front" is not The UNIX Way™, rather I should just try the operation and ask forgiveness later. In my particular script however, this leads to a bad user experience and I can't change the responsible component.










share|improve this question























  • Will the argument always be based on the current directory or could the user specify a full path?
    – Jesse_b
    Nov 8 at 22:08










  • @Jesse_b - I suppose the user could specify an absolute path like /foo/bar/file.txt. Basically I pass the path to tee like tee $OUT_FILE where OUT_FILE is passed on the command line. That should "just work" with both absolute and relative paths, right?
    – BeeOnRope
    Nov 8 at 22:09










  • @BeeOnRope, no you'd need tee -- "$OUT_FILE" at least. What if the file already exists or exists but is not a regular file (directory, symlink, fifo)?
    – Stéphane Chazelas
    Nov 8 at 22:37










  • @StéphaneChazelas - well I am using tee "$OUT_FILE.tmp". If the file already exists, tee overwrites, which is the desired behavior in this case. If it's a directory, tee will fail (I think). symlink I'm not 100% sure?
    – BeeOnRope
    Nov 8 at 22:39












up vote
14
down vote

favorite
2









up vote
14
down vote

favorite
2






2





The user calls my script with a file path that will be either be created or overwritten at some point in the script, like foo.sh file.txt or foo.sh dir/file.txt.



The create-or-overwrite behavior is much like the requirements for putting the file on the right side of the > output redirect operator, or passing it as an argument to tee (in fact, passing it as an argument to tee is exactly what I'm doing).



Before I get into the guts of the script, I want to make a reasonable check if the file can be created/overwritten, but not actually create it. This check doesn't have to be perfect, and yes I realize that the situation can change between the check and the point where the file is actually written - but here I'm OK with a best effort type solution so I can bail out early in the case that the file path is invalid.



Examples of reasons the file couldn't created:



  • the file contains a directory component, like dir/file.txt but the directory dir doesn't exist

  • the user doens't have write permissions in the specified directory (or the CWD if no directory was specified

Yes, I realize that checking permissions "up front" is not The UNIX Way™, rather I should just try the operation and ask forgiveness later. In my particular script however, this leads to a bad user experience and I can't change the responsible component.










share|improve this question















The user calls my script with a file path that will be either be created or overwritten at some point in the script, like foo.sh file.txt or foo.sh dir/file.txt.



The create-or-overwrite behavior is much like the requirements for putting the file on the right side of the > output redirect operator, or passing it as an argument to tee (in fact, passing it as an argument to tee is exactly what I'm doing).



Before I get into the guts of the script, I want to make a reasonable check if the file can be created/overwritten, but not actually create it. This check doesn't have to be perfect, and yes I realize that the situation can change between the check and the point where the file is actually written - but here I'm OK with a best effort type solution so I can bail out early in the case that the file path is invalid.



Examples of reasons the file couldn't created:



  • the file contains a directory component, like dir/file.txt but the directory dir doesn't exist

  • the user doens't have write permissions in the specified directory (or the CWD if no directory was specified

Yes, I realize that checking permissions "up front" is not The UNIX Way™, rather I should just try the operation and ask forgiveness later. In my particular script however, this leads to a bad user experience and I can't change the responsible component.







bash files error-handling






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited Nov 9 at 18:22

























asked Nov 8 at 22:02









BeeOnRope

217110




217110











  • Will the argument always be based on the current directory or could the user specify a full path?
    – Jesse_b
    Nov 8 at 22:08










  • @Jesse_b - I suppose the user could specify an absolute path like /foo/bar/file.txt. Basically I pass the path to tee like tee $OUT_FILE where OUT_FILE is passed on the command line. That should "just work" with both absolute and relative paths, right?
    – BeeOnRope
    Nov 8 at 22:09










  • @BeeOnRope, no you'd need tee -- "$OUT_FILE" at least. What if the file already exists or exists but is not a regular file (directory, symlink, fifo)?
    – Stéphane Chazelas
    Nov 8 at 22:37










  • @StéphaneChazelas - well I am using tee "$OUT_FILE.tmp". If the file already exists, tee overwrites, which is the desired behavior in this case. If it's a directory, tee will fail (I think). symlink I'm not 100% sure?
    – BeeOnRope
    Nov 8 at 22:39
















  • Will the argument always be based on the current directory or could the user specify a full path?
    – Jesse_b
    Nov 8 at 22:08










  • @Jesse_b - I suppose the user could specify an absolute path like /foo/bar/file.txt. Basically I pass the path to tee like tee $OUT_FILE where OUT_FILE is passed on the command line. That should "just work" with both absolute and relative paths, right?
    – BeeOnRope
    Nov 8 at 22:09










  • @BeeOnRope, no you'd need tee -- "$OUT_FILE" at least. What if the file already exists or exists but is not a regular file (directory, symlink, fifo)?
    – Stéphane Chazelas
    Nov 8 at 22:37










  • @StéphaneChazelas - well I am using tee "$OUT_FILE.tmp". If the file already exists, tee overwrites, which is the desired behavior in this case. If it's a directory, tee will fail (I think). symlink I'm not 100% sure?
    – BeeOnRope
    Nov 8 at 22:39















Will the argument always be based on the current directory or could the user specify a full path?
– Jesse_b
Nov 8 at 22:08




Will the argument always be based on the current directory or could the user specify a full path?
– Jesse_b
Nov 8 at 22:08












@Jesse_b - I suppose the user could specify an absolute path like /foo/bar/file.txt. Basically I pass the path to tee like tee $OUT_FILE where OUT_FILE is passed on the command line. That should "just work" with both absolute and relative paths, right?
– BeeOnRope
Nov 8 at 22:09




@Jesse_b - I suppose the user could specify an absolute path like /foo/bar/file.txt. Basically I pass the path to tee like tee $OUT_FILE where OUT_FILE is passed on the command line. That should "just work" with both absolute and relative paths, right?
– BeeOnRope
Nov 8 at 22:09












@BeeOnRope, no you'd need tee -- "$OUT_FILE" at least. What if the file already exists or exists but is not a regular file (directory, symlink, fifo)?
– Stéphane Chazelas
Nov 8 at 22:37




@BeeOnRope, no you'd need tee -- "$OUT_FILE" at least. What if the file already exists or exists but is not a regular file (directory, symlink, fifo)?
– Stéphane Chazelas
Nov 8 at 22:37












@StéphaneChazelas - well I am using tee "$OUT_FILE.tmp". If the file already exists, tee overwrites, which is the desired behavior in this case. If it's a directory, tee will fail (I think). symlink I'm not 100% sure?
– BeeOnRope
Nov 8 at 22:39




@StéphaneChazelas - well I am using tee "$OUT_FILE.tmp". If the file already exists, tee overwrites, which is the desired behavior in this case. If it's a directory, tee will fail (I think). symlink I'm not 100% sure?
– BeeOnRope
Nov 8 at 22:39










7 Answers
7






active

oldest

votes

















up vote
10
down vote













The obvious test would be:



if touch /path/to/file; then
: it can be created
fi


But it does actually create the file if it's not already there. We could clean up after ourselves:



if touch /path/to/file; then
rm /path/to/file
fi


But this would remove a file that already existed, which you probably don't want.



We do, however, have a way around this:



if mkdir /path/to/file; then
rmdir /path/to/file
fi


You can't have a directory with the same name as another object in that directory. I can't think of a situation in which you'd be able to create a directory but not create a file. After this test, your script would be free to create a conventional /path/to/file and do whatever it pleases with it.






share|improve this answer
















  • 1




    That very neatly handles so many issues! A code comment explaining & justifying that unusual solution might well exceed the length of your answer. :-)
    – jpaugh
    Nov 9 at 20:33











  • I also have used if mkdir /var/run/lockdir as a locking test. This is atomic, while if ! [[ -f /var/run/lockfile ]]; then touch /var/run/lockfile has a small slice of time in between the test and touch where another instance of the script in question can start.
    – DopeGhoti
    Nov 13 at 18:18

















up vote
8
down vote













From what I'm gathering, you want to check that when using



tee -- "$OUT_FILE"


(note the -- or it wouldn't work for file names that start with -), tee would succeed to open the file for writing.



That is that:



  • the length of the file path doesn't exceed the PATH_MAX limit

  • the file exists (after symlink resolution) and is not of type directory and you have write permission to it.

  • if the file doesn't exist, the dirname of the file exists (after symlink resolution) as a directory and you have write and search permission to it and the filename length doesn't exceed the NAME_MAX limit of the filesystem that directory resides in.

  • or the file is a symlink that points to a file that doesn't exist and is not a symlink loop but meets the criteria just above

We'll ignore for now filesystems like vfat, ntfs or hfsplus that have limitations on what byte values file names may contain, disk quota, process limit, selinux, apparmor or other security mechanism in the way, full filesystem, no inode left, device files that can't be opened that way for a reason or another, files that are executables currently mapped in some process address space all of which could also affect the ability to open or create the file.



With zsh:



zmodload zsh/system
tee_would_likely_succeed()
local file=$1 ERRNO=0 LC_ALL=C
if [ -d "$file" ]; then
return 1 # directory
elif [ -w "$file" ]; then
return 0 # writable non-directory
elif [ -e "$file" ]; then
return 1 # exists, non-writable
elif [ "$errnos[ERRNO]" != ENOENT ]; then
return 1 # only ENOENT error can be recovered
else
local dir=$file:P:h base=$file:t
[ -d "$dir" ] && # directory
[ -w "$dir" ] && # writable
[ -x "$dir" ] && # and searchable
(($#base <= $(getconf -- NAME_MAX "$dir")))
return
fi



In bash or any Bourne-like shell, just replace the



zmodload zsh/system
tee_would_likely_succeed()
<zsh-code>



with:



tee_would_likely_succeed() 
zsh -s -- "$@" << 'EOF'
zmodload zsh/system
<zsh-code>
EOF



The zsh-specific features here are $ERRNO (which exposes the error code of the last system call) and $errnos associative array to translate to the corresponding standard C macro names. And the $var:h (from csh) and $var:P (needs zsh 5.3 or above).



bash doesn't have equivalent features yet.



$file:h can be replaced with dir=$(dirname -- "$file"; echo .); dir=$dir%??, or with GNU dirname: IFS= read -rd '' dir < <(dirname -z -- "$file").



For $errnos[ERRNO] == ENOENT, an approach could be to run ls -Ld on the file and check whether the error message corresponds to the ENOENT error. Doing that reliably and portably is tricky though.



One approach could be:



msg_for_ENOENT=$(LC_ALL=C ls -d -- '/no such file' 2>&1)
msg_for_ENOENT=$msg_for_ENOENT##*:


(assuming that the error message ends with the syserror() translation of ENOENT and that that translation doesn't include a :) and then, instead of [ -e "$file" ], do:



err=$(ls -Ld -- "$file" 2>&1)


And check for a ENOENT error with



case $err in
(*:"$msg_for_ENOENT") ...
esac


The $file:P part is the trickiest to achieve in bash, especially on FreeBSD.



FreeBSD does have a realpath command and a readlink command that accepts a -f option, but they can't be used in the cases where the file is a symlink that doesn't resolve. That's the same with perl's Cwd::realpath().



python's os.path.realpath() does appear to work similarly to zsh $file:P, so assuming that at least one version of python is installed and that there is a python command that refers to one of them (which is not a given on FreeBSD), you could do:



dir=$(python -c '
import os, sys
print(os.path.realpath(sys.argv[1]) + ".")' "$dir") || return
dir=$dir%.


But then, you might as well do the whole thing in python.



Or you could decide not to handle all those corner cases.






share|improve this answer






















  • Yes, you understood my intent correctly. In fact, the original question was unclear: I originally spoke about creating the file, but actually I am using it with tee so the requirement is really that the file can be created if it doesn't exist, or can be truncated to zero and overwritten if it does (or however else tee handles that).
    – BeeOnRope
    Nov 9 at 1:13










  • Looks like your last edit wiped the formatting
    – D. Ben Knoble
    Nov 9 at 22:58










  • Thanks, @​bishop, @D.BenKnoble, see edit.
    – Stéphane Chazelas
    Nov 9 at 23:42






  • 1




    @DennisWilliamson, well yes, a shell is a tool to invoke programs, and every program that you invoke in your script has to be installed for your script to work, that goes without saying. macOS comes with both bash and zsh installed by default. FreeBSD comes with neither. The OP may have a good reason for wanting to do that in bash, maybe it's something they want to add to an already written bash script.
    – Stéphane Chazelas
    Nov 10 at 9:28






  • 1




    @pipe, again, a shell is a command interpreter. You'll see that many shell code excerpts written here invoke interpreters of other languages like perl, awk, sed or python (or even sh, bash...). Shell scripting is about using the best command for the task. Here even perl doesn't have an equivalent of zsh's $var:P (its Cwd::realpath behaves like BSD realpath or readlink -f while you'd want it to behave like GNU readlink -f or python's os.path.realpath())
    – Stéphane Chazelas
    Nov 10 at 10:08


















up vote
6
down vote













One option you might want to consider is creating the file early on but only populating it later in your script. You can use the exec command to open the file in a file descriptor (such as 3, 4, etc.) and then later use a redirection to a file descriptor (>&3, etc.) to write contents to that file.



Something like:



#!/bin/bash

# Open the file for read/write, so it doesn't get
# truncated just yet (to preserve the contents in
# case the initial checks fail.)
exec 3<>dir/file.txt ||
echo "Error creating dir/file.txt" >&2
exit 1


# Long checks here...
check_ok ||
echo "Failed checks" >&2
# cleanup file before bailing out
rm -f dir/file.txt
exit 1


# We're ready to write, first truncate the file.
# Use "truncate(1)" from coreutils, and pass it
# /dev/fd/3 so the file doesn't need to be reopened.
truncate -s 0 /dev/fd/3

# Now populate the file, use a redirection to write
# to the previously opened file descriptor.
populate_contents >&3


You can also use a trap to clean up the file on error, that's a common practice.



This way, you get a real check for permissions that you'll be able to create the file, while at the same time being able to perform it early enough that if that fails you haven't spent time waiting for the long checks.




UPDATED: In order to avoid clobbering the file in case the checks fail, use bash's fd<>file redirection which does not truncate the file right away. (We don't care about reading from the file, this is just a workaround so we don't truncate it. Appending with >> would probably just work too, but I tend to find this one a bit more elegant, keeping the O_APPEND flag out of the picture.)



When we're ready to replace the contents, we need to truncate the file first (otherwise if we're writing fewer bytes than there were in the file before, the trailing bytes would stay there.) We can use the truncate(1) command from coreutils for that purpose, and we can pass it the open file descriptor we have (using the /dev/fd/3 pseudo-file) so it doesn't need to reopen the file. (Again, technically something simpler like : >dir/file.txt would probably work, but not having to reopen the file is a more elegant solution.)






share|improve this answer






















  • I had considered this, but the problem is that if another innocent error occurs somewhere between when I create the file and the point some time later where I would write to it, the user will probably be upset to find out that the specified file was overwritten.
    – BeeOnRope
    Nov 8 at 22:34






  • 1




    @BeeOnRope another option that won't clobber the file is to try to append nothing to it: echo -n >>file or true >> file. Of course, ext4 has append-only files, but you could live with that false positive.
    – mosvy
    Nov 9 at 13:15






  • 1




    @BeeOnRope If you need to preserve either (a) the existing file or (b) the (complete) new file, that's a different question than you have asked. A good answer to it is to move the existing file to a new name (e.g. my-file.txt.backup) before creating the new one. Another solution is to write the new file to a temporary file in the same folder, and then copy it over the old file after the rest of the script succeeds --- if that last operation failed, the user could manually fix the issue without losing his or her progress.
    – jpaugh
    Nov 9 at 19:45







  • 1




    @BeeOnRope If you go for the "atomic replace" route (which is a good one!) then typically you'll want to write the temporary file in the same directory as the final file (they need to be in the same filesystem for rename(2) to succeed) and use a unique name (so if there are two instances of the script running, they won't clobber each other's temporary files.) Opening a temporary file for writing in the same directory is usually a good indication you'll be able to rename it later (well, except if the final target exists and is a directory), so to some extent it also addresses your question.
    – Filipe Brandenburger
    Nov 9 at 20:55






  • 1




    @jpaugh - I'm quite reluctant to do that. It would inevitably lead to answers trying to solve my higher level problem, which might admit solutions that have nothing to do with the question "How can I check...". Regardless of what my high level problem is, I think this question stands alone as something you might reasonably want to do. It would be unfair to people who are answering that specific question if I changed it to "solve this specific issue I'm having with my script". I included those details here because I was asked, but I don't want to put change the primary question.
    – BeeOnRope
    Nov 15 at 21:10


















up vote
4
down vote













I think DopeGhoti's solution is better but this should also work:



file=$1
if [[ "$file:0:1" == '/' ]]; then
dir=$file%/*
elif [[ "$file" =~ .*/.* ]]; then
dir="$(PWD)/$file%/*"
else
dir=$(PWD)
fi

if [[ -w "$dir" ]]; then
echo "writable"
#do stuff with writable file
else
echo "not writable"
#do stuff without writable file
fi


The first if construct checks if the argument is a full path (starts with /) and sets the dir variable to the directory path up to the last /. Otherwise if the argument does not start with a / but does contain a / (specifying a sub directory) it will set dir to the present working directory + the sub directory path. Otherwise it assumes the present working directory. It then checks if that directory is writable.






share|improve this answer



























    up vote
    4
    down vote













    What about using normal test command like outlined below?



    FILE=$1

    DIR=$(dirname $FILE) # $DIR now contains '.' for file names only, 'foo' for 'foo/bar'

    if [ -d $DIR ] ; then
    echo "base directory $DIR for file exists"
    if [ -e $FILE ] ; then
    if [ -w $FILE ] ; then
    echo "file exists, is writeable"
    else
    echo "file exists, NOT writeable"
    fi
    elif [ -w $DIR ] ; then
    echo "directory is writeable"
    else
    echo "directory is NOT writeable"
    fi
    else
    echo "can NOT create file in non-existent directory $DIR "
    fi





    share|improve this answer




















    • I almost duplicated this answer! Nice introduction, but you might mention man test, and that [ ] is equivalent to test (not common knowledge anymore). Here is a the one-liner i was about to use in my inferior answer, to cover the exact case, for posterity: if [ -w $1] && [ ! -d $1 ] ; then echo "do stuff"; fi
      – Iron Gremlin
      Nov 10 at 1:44

















    up vote
    4
    down vote













    You mentioned user experience was driving your question. I'll answer from a UX angle, since you've got good answers on the technical side.



    Rather than performing the check up-front, how about writing the results into a temporary file then at the very end, placing the results into the user's desired file? Like:



    userfile=$1:?Where would you like the file written?
    tmpfile=$(mktemp)

    # ... all the complicated stuff, writing into "$tmpfile"

    # fill user's file, keeping existing permissions or creating anew
    # while respecting umask
    cat "$tmpfile" > "$userfile"
    if [ 0 -eq $? ]; then
    rm "$tmpfile"
    else
    echo "Couldn't write results into $userfile." >&2
    echo "Results available in $tmpfile." >&2
    exit 1
    fi


    The good with this approach: it produces the desired operation in the normal happy path scenario, side-steps the test-and-set atomicity issue, preserves permissions of the target file while creating if necessary, and is dead simple to implement.



    Note: had we used mv, we'd be keeping the permissions of the temporary file -- we don't want that, I think: we want to keep the permissions as set on the target file.



    Now, the bad: it requires twice the space (cat .. > construct), forces the user to do some manual work if the target file wasn't writable at the time it needed to be, and leaves the temporary file laying around (which might have security or maintenance issues).






    share|improve this answer






















    • In fact, this is more or less what I'm doing now. I write most of the results to a temporary file and then at the end do the final processing step and write the results to the final file. The problem is I want to bail out early (at the start of the script) if that final step is likely to fail. The script may run unattended for minutes or hours, so you really want to know up front that it is doomed to fail!
      – BeeOnRope
      Nov 9 at 16:12










    • Sure, but there are so many ways this could fail: disk could fill, upstream directory could be removed, permissions could change, target file might be used for some other important stuff and the user forgot he assigned that same file to be destroyed by this operation. If we talk about this from a pure UX perspective, then perhaps the right thing to do is treat it like a job submission: at the end, when you know it worked correctly to completion, just tell the user where the resulting content resides and offer a suggested command for them to move it themselves.
      – bishop
      Nov 9 at 16:18











    • In theory, yes there are infinite ways this could fail. In practice, the overwhelming majority of the time this fails is because the path provided is not valid. I can't reasonably prevent some rogue user from concurrently modifying the FS to break the job in the middle of the operation, but I certainly can check the #1 failure cause of an invalid or not writable path.
      – BeeOnRope
      Nov 9 at 16:23


















    up vote
    2
    down vote













    TL;DR:



    : >> "$userfile"


    Unlike <> "$userfile" or touch "$userfile", this will not make any spurious modifications to the file's timestamps and will also work with write-only files.




    From the OP:




    I want to make a reasonable check if the file can be created/overwritten, but not actually create it.




    And from your comment to my answer from a UX perspective:




    The overwhelming majority of the time this fails is because the path provided is not valid. I can't reasonably prevent some rogue user some concurrently modifying the FS to break the job in the middle of the operation, but I certainly can check the #1 failure cause of an invalid or not writable path.




    The only reliable test is to open(2) the file, because only that resolves every question about the writeability: path, ownership, filesystem, network, security context, etc. Any other test will address some part of writeability, but not others. If you want a subset of tests, you'll ultimately have to choose what's important to you.



    But here's another thought. From what I understand:



    1. the content creation process is long-running, and

    2. the target file should be left in a consistent state.

    You're wanting to do this pre-check because of #1, and you don't want to fiddle with an existing file because of #2. So why don't you just ask the shell to open the file for append, but don't actually append anything?



    $ tree -ps
    .
    ├── [dr-x------ 4096] dir_r
    ├── [drwx------ 4096] dir_w
    ├── [-r-------- 0] file_r
    └── [-rw------- 0] file_w

    $ for p in file_r dir_r/foo file_w dir_w/foo; do : >> $p; done
    -bash: file_r: Permission denied
    -bash: dir_r/foo: Permission denied

    $ tree -ps
    .
    ├── [dr-x------ 4096] dir_r
    ├── [drwx------ 4096] dir_w
    │   └── [-rw-rw-r-- 0] foo
    ├── [-r-------- 0] file_r
    └── [-rw------- 0] file_w


    Under the hood, this resolves the writeability question exactly as wanted:



    open("dir_w/foo", O_WRONLY|O_CREAT|O_APPEND, 0666) = 3


    but without modifying the file's contents or metadata. Now, yes, this approach:



    • doesn't tell you if the file is append only, which might be a problem when you go about updating it at the end of your content creation. You can detect this, to a degree, with lsattr and react accordingly.

    • creates a file that didn't previously exist, if such is the case: mitigate this with a selective rm.

    While I contend (in my other answer) that the most user-friendly approach is to create a temporary file the user has to move, I think this is the least user-hostile approach to fully vet their input.






    share|improve this answer






















      Your Answer








      StackExchange.ready(function()
      var channelOptions =
      tags: "".split(" "),
      id: "106"
      ;
      initTagRenderer("".split(" "), "".split(" "), channelOptions);

      StackExchange.using("externalEditor", function()
      // Have to fire editor after snippets, if snippets enabled
      if (StackExchange.settings.snippets.snippetsEnabled)
      StackExchange.using("snippets", function()
      createEditor();
      );

      else
      createEditor();

      );

      function createEditor()
      StackExchange.prepareEditor(
      heartbeatType: 'answer',
      convertImagesToLinks: false,
      noModals: true,
      showLowRepImageUploadWarning: true,
      reputationToPostImages: null,
      bindNavPrevention: true,
      postfix: "",
      imageUploader:
      brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
      contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
      allowUrls: true
      ,
      onDemand: true,
      discardSelector: ".discard-answer"
      ,immediatelyShowMarkdownHelp:true
      );



      );













      draft saved

      draft discarded


















      StackExchange.ready(
      function ()
      StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f480656%2fhow-can-i-check-if-a-file-can-be-created-or-truncated-overwritten-in-bash%23new-answer', 'question_page');

      );

      Post as a guest















      Required, but never shown

























      7 Answers
      7






      active

      oldest

      votes








      7 Answers
      7






      active

      oldest

      votes









      active

      oldest

      votes






      active

      oldest

      votes








      up vote
      10
      down vote













      The obvious test would be:



      if touch /path/to/file; then
      : it can be created
      fi


      But it does actually create the file if it's not already there. We could clean up after ourselves:



      if touch /path/to/file; then
      rm /path/to/file
      fi


      But this would remove a file that already existed, which you probably don't want.



      We do, however, have a way around this:



      if mkdir /path/to/file; then
      rmdir /path/to/file
      fi


      You can't have a directory with the same name as another object in that directory. I can't think of a situation in which you'd be able to create a directory but not create a file. After this test, your script would be free to create a conventional /path/to/file and do whatever it pleases with it.






      share|improve this answer
















      • 1




        That very neatly handles so many issues! A code comment explaining & justifying that unusual solution might well exceed the length of your answer. :-)
        – jpaugh
        Nov 9 at 20:33











      • I also have used if mkdir /var/run/lockdir as a locking test. This is atomic, while if ! [[ -f /var/run/lockfile ]]; then touch /var/run/lockfile has a small slice of time in between the test and touch where another instance of the script in question can start.
        – DopeGhoti
        Nov 13 at 18:18














      up vote
      10
      down vote













      The obvious test would be:



      if touch /path/to/file; then
      : it can be created
      fi


      But it does actually create the file if it's not already there. We could clean up after ourselves:



      if touch /path/to/file; then
      rm /path/to/file
      fi


      But this would remove a file that already existed, which you probably don't want.



      We do, however, have a way around this:



      if mkdir /path/to/file; then
      rmdir /path/to/file
      fi


      You can't have a directory with the same name as another object in that directory. I can't think of a situation in which you'd be able to create a directory but not create a file. After this test, your script would be free to create a conventional /path/to/file and do whatever it pleases with it.






      share|improve this answer
















      • 1




        That very neatly handles so many issues! A code comment explaining & justifying that unusual solution might well exceed the length of your answer. :-)
        – jpaugh
        Nov 9 at 20:33











      • I also have used if mkdir /var/run/lockdir as a locking test. This is atomic, while if ! [[ -f /var/run/lockfile ]]; then touch /var/run/lockfile has a small slice of time in between the test and touch where another instance of the script in question can start.
        – DopeGhoti
        Nov 13 at 18:18












      up vote
      10
      down vote










      up vote
      10
      down vote









      The obvious test would be:



      if touch /path/to/file; then
      : it can be created
      fi


      But it does actually create the file if it's not already there. We could clean up after ourselves:



      if touch /path/to/file; then
      rm /path/to/file
      fi


      But this would remove a file that already existed, which you probably don't want.



      We do, however, have a way around this:



      if mkdir /path/to/file; then
      rmdir /path/to/file
      fi


      You can't have a directory with the same name as another object in that directory. I can't think of a situation in which you'd be able to create a directory but not create a file. After this test, your script would be free to create a conventional /path/to/file and do whatever it pleases with it.






      share|improve this answer












      The obvious test would be:



      if touch /path/to/file; then
      : it can be created
      fi


      But it does actually create the file if it's not already there. We could clean up after ourselves:



      if touch /path/to/file; then
      rm /path/to/file
      fi


      But this would remove a file that already existed, which you probably don't want.



      We do, however, have a way around this:



      if mkdir /path/to/file; then
      rmdir /path/to/file
      fi


      You can't have a directory with the same name as another object in that directory. I can't think of a situation in which you'd be able to create a directory but not create a file. After this test, your script would be free to create a conventional /path/to/file and do whatever it pleases with it.







      share|improve this answer












      share|improve this answer



      share|improve this answer










      answered Nov 8 at 22:11









      DopeGhoti

      42.8k55181




      42.8k55181







      • 1




        That very neatly handles so many issues! A code comment explaining & justifying that unusual solution might well exceed the length of your answer. :-)
        – jpaugh
        Nov 9 at 20:33











      • I also have used if mkdir /var/run/lockdir as a locking test. This is atomic, while if ! [[ -f /var/run/lockfile ]]; then touch /var/run/lockfile has a small slice of time in between the test and touch where another instance of the script in question can start.
        – DopeGhoti
        Nov 13 at 18:18












      • 1




        That very neatly handles so many issues! A code comment explaining & justifying that unusual solution might well exceed the length of your answer. :-)
        – jpaugh
        Nov 9 at 20:33











      • I also have used if mkdir /var/run/lockdir as a locking test. This is atomic, while if ! [[ -f /var/run/lockfile ]]; then touch /var/run/lockfile has a small slice of time in between the test and touch where another instance of the script in question can start.
        – DopeGhoti
        Nov 13 at 18:18







      1




      1




      That very neatly handles so many issues! A code comment explaining & justifying that unusual solution might well exceed the length of your answer. :-)
      – jpaugh
      Nov 9 at 20:33





      That very neatly handles so many issues! A code comment explaining & justifying that unusual solution might well exceed the length of your answer. :-)
      – jpaugh
      Nov 9 at 20:33













      I also have used if mkdir /var/run/lockdir as a locking test. This is atomic, while if ! [[ -f /var/run/lockfile ]]; then touch /var/run/lockfile has a small slice of time in between the test and touch where another instance of the script in question can start.
      – DopeGhoti
      Nov 13 at 18:18




      I also have used if mkdir /var/run/lockdir as a locking test. This is atomic, while if ! [[ -f /var/run/lockfile ]]; then touch /var/run/lockfile has a small slice of time in between the test and touch where another instance of the script in question can start.
      – DopeGhoti
      Nov 13 at 18:18












      up vote
      8
      down vote













      From what I'm gathering, you want to check that when using



      tee -- "$OUT_FILE"


      (note the -- or it wouldn't work for file names that start with -), tee would succeed to open the file for writing.



      That is that:



      • the length of the file path doesn't exceed the PATH_MAX limit

      • the file exists (after symlink resolution) and is not of type directory and you have write permission to it.

      • if the file doesn't exist, the dirname of the file exists (after symlink resolution) as a directory and you have write and search permission to it and the filename length doesn't exceed the NAME_MAX limit of the filesystem that directory resides in.

      • or the file is a symlink that points to a file that doesn't exist and is not a symlink loop but meets the criteria just above

      We'll ignore for now filesystems like vfat, ntfs or hfsplus that have limitations on what byte values file names may contain, disk quota, process limit, selinux, apparmor or other security mechanism in the way, full filesystem, no inode left, device files that can't be opened that way for a reason or another, files that are executables currently mapped in some process address space all of which could also affect the ability to open or create the file.



      With zsh:



      zmodload zsh/system
      tee_would_likely_succeed()
      local file=$1 ERRNO=0 LC_ALL=C
      if [ -d "$file" ]; then
      return 1 # directory
      elif [ -w "$file" ]; then
      return 0 # writable non-directory
      elif [ -e "$file" ]; then
      return 1 # exists, non-writable
      elif [ "$errnos[ERRNO]" != ENOENT ]; then
      return 1 # only ENOENT error can be recovered
      else
      local dir=$file:P:h base=$file:t
      [ -d "$dir" ] && # directory
      [ -w "$dir" ] && # writable
      [ -x "$dir" ] && # and searchable
      (($#base <= $(getconf -- NAME_MAX "$dir")))
      return
      fi



      In bash or any Bourne-like shell, just replace the



      zmodload zsh/system
      tee_would_likely_succeed()
      <zsh-code>



      with:



      tee_would_likely_succeed() 
      zsh -s -- "$@" << 'EOF'
      zmodload zsh/system
      <zsh-code>
      EOF



      The zsh-specific features here are $ERRNO (which exposes the error code of the last system call) and $errnos associative array to translate to the corresponding standard C macro names. And the $var:h (from csh) and $var:P (needs zsh 5.3 or above).



      bash doesn't have equivalent features yet.



      $file:h can be replaced with dir=$(dirname -- "$file"; echo .); dir=$dir%??, or with GNU dirname: IFS= read -rd '' dir < <(dirname -z -- "$file").



      For $errnos[ERRNO] == ENOENT, an approach could be to run ls -Ld on the file and check whether the error message corresponds to the ENOENT error. Doing that reliably and portably is tricky though.



      One approach could be:



      msg_for_ENOENT=$(LC_ALL=C ls -d -- '/no such file' 2>&1)
      msg_for_ENOENT=$msg_for_ENOENT##*:


      (assuming that the error message ends with the syserror() translation of ENOENT and that that translation doesn't include a :) and then, instead of [ -e "$file" ], do:



      err=$(ls -Ld -- "$file" 2>&1)


      And check for a ENOENT error with



      case $err in
      (*:"$msg_for_ENOENT") ...
      esac


      The $file:P part is the trickiest to achieve in bash, especially on FreeBSD.



      FreeBSD does have a realpath command and a readlink command that accepts a -f option, but they can't be used in the cases where the file is a symlink that doesn't resolve. That's the same with perl's Cwd::realpath().



      python's os.path.realpath() does appear to work similarly to zsh $file:P, so assuming that at least one version of python is installed and that there is a python command that refers to one of them (which is not a given on FreeBSD), you could do:



      dir=$(python -c '
      import os, sys
      print(os.path.realpath(sys.argv[1]) + ".")' "$dir") || return
      dir=$dir%.


      But then, you might as well do the whole thing in python.



      Or you could decide not to handle all those corner cases.






      share|improve this answer






















      • Yes, you understood my intent correctly. In fact, the original question was unclear: I originally spoke about creating the file, but actually I am using it with tee so the requirement is really that the file can be created if it doesn't exist, or can be truncated to zero and overwritten if it does (or however else tee handles that).
        – BeeOnRope
        Nov 9 at 1:13










      • Looks like your last edit wiped the formatting
        – D. Ben Knoble
        Nov 9 at 22:58










      • Thanks, @​bishop, @D.BenKnoble, see edit.
        – Stéphane Chazelas
        Nov 9 at 23:42






      • 1




        @DennisWilliamson, well yes, a shell is a tool to invoke programs, and every program that you invoke in your script has to be installed for your script to work, that goes without saying. macOS comes with both bash and zsh installed by default. FreeBSD comes with neither. The OP may have a good reason for wanting to do that in bash, maybe it's something they want to add to an already written bash script.
        – Stéphane Chazelas
        Nov 10 at 9:28






      • 1




        @pipe, again, a shell is a command interpreter. You'll see that many shell code excerpts written here invoke interpreters of other languages like perl, awk, sed or python (or even sh, bash...). Shell scripting is about using the best command for the task. Here even perl doesn't have an equivalent of zsh's $var:P (its Cwd::realpath behaves like BSD realpath or readlink -f while you'd want it to behave like GNU readlink -f or python's os.path.realpath())
        – Stéphane Chazelas
        Nov 10 at 10:08















      up vote
      8
      down vote













      From what I'm gathering, you want to check that when using



      tee -- "$OUT_FILE"


      (note the -- or it wouldn't work for file names that start with -), tee would succeed to open the file for writing.



      That is that:



      • the length of the file path doesn't exceed the PATH_MAX limit

      • the file exists (after symlink resolution) and is not of type directory and you have write permission to it.

      • if the file doesn't exist, the dirname of the file exists (after symlink resolution) as a directory and you have write and search permission to it and the filename length doesn't exceed the NAME_MAX limit of the filesystem that directory resides in.

      • or the file is a symlink that points to a file that doesn't exist and is not a symlink loop but meets the criteria just above

      We'll ignore for now filesystems like vfat, ntfs or hfsplus that have limitations on what byte values file names may contain, disk quota, process limit, selinux, apparmor or other security mechanism in the way, full filesystem, no inode left, device files that can't be opened that way for a reason or another, files that are executables currently mapped in some process address space all of which could also affect the ability to open or create the file.



      With zsh:



      zmodload zsh/system
      tee_would_likely_succeed()
      local file=$1 ERRNO=0 LC_ALL=C
      if [ -d "$file" ]; then
      return 1 # directory
      elif [ -w "$file" ]; then
      return 0 # writable non-directory
      elif [ -e "$file" ]; then
      return 1 # exists, non-writable
      elif [ "$errnos[ERRNO]" != ENOENT ]; then
      return 1 # only ENOENT error can be recovered
      else
      local dir=$file:P:h base=$file:t
      [ -d "$dir" ] && # directory
      [ -w "$dir" ] && # writable
      [ -x "$dir" ] && # and searchable
      (($#base <= $(getconf -- NAME_MAX "$dir")))
      return
      fi



      In bash or any Bourne-like shell, just replace the



      zmodload zsh/system
      tee_would_likely_succeed()
      <zsh-code>



      with:



      tee_would_likely_succeed() 
      zsh -s -- "$@" << 'EOF'
      zmodload zsh/system
      <zsh-code>
      EOF



      The zsh-specific features here are $ERRNO (which exposes the error code of the last system call) and $errnos associative array to translate to the corresponding standard C macro names. And the $var:h (from csh) and $var:P (needs zsh 5.3 or above).



      bash doesn't have equivalent features yet.



      $file:h can be replaced with dir=$(dirname -- "$file"; echo .); dir=$dir%??, or with GNU dirname: IFS= read -rd '' dir < <(dirname -z -- "$file").



      For $errnos[ERRNO] == ENOENT, an approach could be to run ls -Ld on the file and check whether the error message corresponds to the ENOENT error. Doing that reliably and portably is tricky though.



      One approach could be:



      msg_for_ENOENT=$(LC_ALL=C ls -d -- '/no such file' 2>&1)
      msg_for_ENOENT=$msg_for_ENOENT##*:


      (assuming that the error message ends with the syserror() translation of ENOENT and that that translation doesn't include a :) and then, instead of [ -e "$file" ], do:



      err=$(ls -Ld -- "$file" 2>&1)


      And check for a ENOENT error with



      case $err in
      (*:"$msg_for_ENOENT") ...
      esac


      The $file:P part is the trickiest to achieve in bash, especially on FreeBSD.



      FreeBSD does have a realpath command and a readlink command that accepts a -f option, but they can't be used in the cases where the file is a symlink that doesn't resolve. That's the same with perl's Cwd::realpath().



      python's os.path.realpath() does appear to work similarly to zsh $file:P, so assuming that at least one version of python is installed and that there is a python command that refers to one of them (which is not a given on FreeBSD), you could do:



      dir=$(python -c '
      import os, sys
      print(os.path.realpath(sys.argv[1]) + ".")' "$dir") || return
      dir=$dir%.


      But then, you might as well do the whole thing in python.



      Or you could decide not to handle all those corner cases.






      share|improve this answer






















      • Yes, you understood my intent correctly. In fact, the original question was unclear: I originally spoke about creating the file, but actually I am using it with tee so the requirement is really that the file can be created if it doesn't exist, or can be truncated to zero and overwritten if it does (or however else tee handles that).
        – BeeOnRope
        Nov 9 at 1:13










      • Looks like your last edit wiped the formatting
        – D. Ben Knoble
        Nov 9 at 22:58










      • Thanks, @​bishop, @D.BenKnoble, see edit.
        – Stéphane Chazelas
        Nov 9 at 23:42






      • 1




        @DennisWilliamson, well yes, a shell is a tool to invoke programs, and every program that you invoke in your script has to be installed for your script to work, that goes without saying. macOS comes with both bash and zsh installed by default. FreeBSD comes with neither. The OP may have a good reason for wanting to do that in bash, maybe it's something they want to add to an already written bash script.
        – Stéphane Chazelas
        Nov 10 at 9:28






      • 1




        @pipe, again, a shell is a command interpreter. You'll see that many shell code excerpts written here invoke interpreters of other languages like perl, awk, sed or python (or even sh, bash...). Shell scripting is about using the best command for the task. Here even perl doesn't have an equivalent of zsh's $var:P (its Cwd::realpath behaves like BSD realpath or readlink -f while you'd want it to behave like GNU readlink -f or python's os.path.realpath())
        – Stéphane Chazelas
        Nov 10 at 10:08













      up vote
      8
      down vote










      up vote
      8
      down vote









      From what I'm gathering, you want to check that when using



      tee -- "$OUT_FILE"


      (note the -- or it wouldn't work for file names that start with -), tee would succeed to open the file for writing.



      That is that:



      • the length of the file path doesn't exceed the PATH_MAX limit

      • the file exists (after symlink resolution) and is not of type directory and you have write permission to it.

      • if the file doesn't exist, the dirname of the file exists (after symlink resolution) as a directory and you have write and search permission to it and the filename length doesn't exceed the NAME_MAX limit of the filesystem that directory resides in.

      • or the file is a symlink that points to a file that doesn't exist and is not a symlink loop but meets the criteria just above

      We'll ignore for now filesystems like vfat, ntfs or hfsplus that have limitations on what byte values file names may contain, disk quota, process limit, selinux, apparmor or other security mechanism in the way, full filesystem, no inode left, device files that can't be opened that way for a reason or another, files that are executables currently mapped in some process address space all of which could also affect the ability to open or create the file.



      With zsh:



      zmodload zsh/system
      tee_would_likely_succeed()
      local file=$1 ERRNO=0 LC_ALL=C
      if [ -d "$file" ]; then
      return 1 # directory
      elif [ -w "$file" ]; then
      return 0 # writable non-directory
      elif [ -e "$file" ]; then
      return 1 # exists, non-writable
      elif [ "$errnos[ERRNO]" != ENOENT ]; then
      return 1 # only ENOENT error can be recovered
      else
      local dir=$file:P:h base=$file:t
      [ -d "$dir" ] && # directory
      [ -w "$dir" ] && # writable
      [ -x "$dir" ] && # and searchable
      (($#base <= $(getconf -- NAME_MAX "$dir")))
      return
      fi



      In bash or any Bourne-like shell, just replace the



      zmodload zsh/system
      tee_would_likely_succeed()
      <zsh-code>



      with:



      tee_would_likely_succeed() 
      zsh -s -- "$@" << 'EOF'
      zmodload zsh/system
      <zsh-code>
      EOF



      The zsh-specific features here are $ERRNO (which exposes the error code of the last system call) and $errnos associative array to translate to the corresponding standard C macro names. And the $var:h (from csh) and $var:P (needs zsh 5.3 or above).



      bash doesn't have equivalent features yet.



      $file:h can be replaced with dir=$(dirname -- "$file"; echo .); dir=$dir%??, or with GNU dirname: IFS= read -rd '' dir < <(dirname -z -- "$file").



      For $errnos[ERRNO] == ENOENT, an approach could be to run ls -Ld on the file and check whether the error message corresponds to the ENOENT error. Doing that reliably and portably is tricky though.



      One approach could be:



      msg_for_ENOENT=$(LC_ALL=C ls -d -- '/no such file' 2>&1)
      msg_for_ENOENT=$msg_for_ENOENT##*:


      (assuming that the error message ends with the syserror() translation of ENOENT and that that translation doesn't include a :) and then, instead of [ -e "$file" ], do:



      err=$(ls -Ld -- "$file" 2>&1)


      And check for a ENOENT error with



      case $err in
      (*:"$msg_for_ENOENT") ...
      esac


      The $file:P part is the trickiest to achieve in bash, especially on FreeBSD.



      FreeBSD does have a realpath command and a readlink command that accepts a -f option, but they can't be used in the cases where the file is a symlink that doesn't resolve. That's the same with perl's Cwd::realpath().



      python's os.path.realpath() does appear to work similarly to zsh $file:P, so assuming that at least one version of python is installed and that there is a python command that refers to one of them (which is not a given on FreeBSD), you could do:



      dir=$(python -c '
      import os, sys
      print(os.path.realpath(sys.argv[1]) + ".")' "$dir") || return
      dir=$dir%.


      But then, you might as well do the whole thing in python.



      Or you could decide not to handle all those corner cases.






      share|improve this answer














      From what I'm gathering, you want to check that when using



      tee -- "$OUT_FILE"


      (note the -- or it wouldn't work for file names that start with -), tee would succeed to open the file for writing.



      That is that:



      • the length of the file path doesn't exceed the PATH_MAX limit

      • the file exists (after symlink resolution) and is not of type directory and you have write permission to it.

      • if the file doesn't exist, the dirname of the file exists (after symlink resolution) as a directory and you have write and search permission to it and the filename length doesn't exceed the NAME_MAX limit of the filesystem that directory resides in.

      • or the file is a symlink that points to a file that doesn't exist and is not a symlink loop but meets the criteria just above

      We'll ignore for now filesystems like vfat, ntfs or hfsplus that have limitations on what byte values file names may contain, disk quota, process limit, selinux, apparmor or other security mechanism in the way, full filesystem, no inode left, device files that can't be opened that way for a reason or another, files that are executables currently mapped in some process address space all of which could also affect the ability to open or create the file.



      With zsh:



      zmodload zsh/system
      tee_would_likely_succeed()
      local file=$1 ERRNO=0 LC_ALL=C
      if [ -d "$file" ]; then
      return 1 # directory
      elif [ -w "$file" ]; then
      return 0 # writable non-directory
      elif [ -e "$file" ]; then
      return 1 # exists, non-writable
      elif [ "$errnos[ERRNO]" != ENOENT ]; then
      return 1 # only ENOENT error can be recovered
      else
      local dir=$file:P:h base=$file:t
      [ -d "$dir" ] && # directory
      [ -w "$dir" ] && # writable
      [ -x "$dir" ] && # and searchable
      (($#base <= $(getconf -- NAME_MAX "$dir")))
      return
      fi



      In bash or any Bourne-like shell, just replace the



      zmodload zsh/system
      tee_would_likely_succeed()
      <zsh-code>



      with:



      tee_would_likely_succeed() 
      zsh -s -- "$@" << 'EOF'
      zmodload zsh/system
      <zsh-code>
      EOF



      The zsh-specific features here are $ERRNO (which exposes the error code of the last system call) and $errnos associative array to translate to the corresponding standard C macro names. And the $var:h (from csh) and $var:P (needs zsh 5.3 or above).



      bash doesn't have equivalent features yet.



      $file:h can be replaced with dir=$(dirname -- "$file"; echo .); dir=$dir%??, or with GNU dirname: IFS= read -rd '' dir < <(dirname -z -- "$file").



      For $errnos[ERRNO] == ENOENT, an approach could be to run ls -Ld on the file and check whether the error message corresponds to the ENOENT error. Doing that reliably and portably is tricky though.



      One approach could be:



      msg_for_ENOENT=$(LC_ALL=C ls -d -- '/no such file' 2>&1)
      msg_for_ENOENT=$msg_for_ENOENT##*:


      (assuming that the error message ends with the syserror() translation of ENOENT and that that translation doesn't include a :) and then, instead of [ -e "$file" ], do:



      err=$(ls -Ld -- "$file" 2>&1)


      And check for a ENOENT error with



      case $err in
      (*:"$msg_for_ENOENT") ...
      esac


      The $file:P part is the trickiest to achieve in bash, especially on FreeBSD.



      FreeBSD does have a realpath command and a readlink command that accepts a -f option, but they can't be used in the cases where the file is a symlink that doesn't resolve. That's the same with perl's Cwd::realpath().



      python's os.path.realpath() does appear to work similarly to zsh $file:P, so assuming that at least one version of python is installed and that there is a python command that refers to one of them (which is not a given on FreeBSD), you could do:



      dir=$(python -c '
      import os, sys
      print(os.path.realpath(sys.argv[1]) + ".")' "$dir") || return
      dir=$dir%.


      But then, you might as well do the whole thing in python.



      Or you could decide not to handle all those corner cases.







      share|improve this answer














      share|improve this answer



      share|improve this answer








      edited Nov 10 at 10:44

























      answered Nov 8 at 23:44









      Stéphane Chazelas

      295k54559902




      295k54559902











      • Yes, you understood my intent correctly. In fact, the original question was unclear: I originally spoke about creating the file, but actually I am using it with tee so the requirement is really that the file can be created if it doesn't exist, or can be truncated to zero and overwritten if it does (or however else tee handles that).
        – BeeOnRope
        Nov 9 at 1:13










      • Looks like your last edit wiped the formatting
        – D. Ben Knoble
        Nov 9 at 22:58










      • Thanks, @​bishop, @D.BenKnoble, see edit.
        – Stéphane Chazelas
        Nov 9 at 23:42






      • 1




        @DennisWilliamson, well yes, a shell is a tool to invoke programs, and every program that you invoke in your script has to be installed for your script to work, that goes without saying. macOS comes with both bash and zsh installed by default. FreeBSD comes with neither. The OP may have a good reason for wanting to do that in bash, maybe it's something they want to add to an already written bash script.
        – Stéphane Chazelas
        Nov 10 at 9:28






      • 1




        @pipe, again, a shell is a command interpreter. You'll see that many shell code excerpts written here invoke interpreters of other languages like perl, awk, sed or python (or even sh, bash...). Shell scripting is about using the best command for the task. Here even perl doesn't have an equivalent of zsh's $var:P (its Cwd::realpath behaves like BSD realpath or readlink -f while you'd want it to behave like GNU readlink -f or python's os.path.realpath())
        – Stéphane Chazelas
        Nov 10 at 10:08

















      • Yes, you understood my intent correctly. In fact, the original question was unclear: I originally spoke about creating the file, but actually I am using it with tee so the requirement is really that the file can be created if it doesn't exist, or can be truncated to zero and overwritten if it does (or however else tee handles that).
        – BeeOnRope
        Nov 9 at 1:13










      • Looks like your last edit wiped the formatting
        – D. Ben Knoble
        Nov 9 at 22:58










      • Thanks, @​bishop, @D.BenKnoble, see edit.
        – Stéphane Chazelas
        Nov 9 at 23:42






      • 1




        @DennisWilliamson, well yes, a shell is a tool to invoke programs, and every program that you invoke in your script has to be installed for your script to work, that goes without saying. macOS comes with both bash and zsh installed by default. FreeBSD comes with neither. The OP may have a good reason for wanting to do that in bash, maybe it's something they want to add to an already written bash script.
        – Stéphane Chazelas
        Nov 10 at 9:28






      • 1




        @pipe, again, a shell is a command interpreter. You'll see that many shell code excerpts written here invoke interpreters of other languages like perl, awk, sed or python (or even sh, bash...). Shell scripting is about using the best command for the task. Here even perl doesn't have an equivalent of zsh's $var:P (its Cwd::realpath behaves like BSD realpath or readlink -f while you'd want it to behave like GNU readlink -f or python's os.path.realpath())
        – Stéphane Chazelas
        Nov 10 at 10:08
















      Yes, you understood my intent correctly. In fact, the original question was unclear: I originally spoke about creating the file, but actually I am using it with tee so the requirement is really that the file can be created if it doesn't exist, or can be truncated to zero and overwritten if it does (or however else tee handles that).
      – BeeOnRope
      Nov 9 at 1:13




      Yes, you understood my intent correctly. In fact, the original question was unclear: I originally spoke about creating the file, but actually I am using it with tee so the requirement is really that the file can be created if it doesn't exist, or can be truncated to zero and overwritten if it does (or however else tee handles that).
      – BeeOnRope
      Nov 9 at 1:13












      Looks like your last edit wiped the formatting
      – D. Ben Knoble
      Nov 9 at 22:58




      Looks like your last edit wiped the formatting
      – D. Ben Knoble
      Nov 9 at 22:58












      Thanks, @​bishop, @D.BenKnoble, see edit.
      – Stéphane Chazelas
      Nov 9 at 23:42




      Thanks, @​bishop, @D.BenKnoble, see edit.
      – Stéphane Chazelas
      Nov 9 at 23:42




      1




      1




      @DennisWilliamson, well yes, a shell is a tool to invoke programs, and every program that you invoke in your script has to be installed for your script to work, that goes without saying. macOS comes with both bash and zsh installed by default. FreeBSD comes with neither. The OP may have a good reason for wanting to do that in bash, maybe it's something they want to add to an already written bash script.
      – Stéphane Chazelas
      Nov 10 at 9:28




      @DennisWilliamson, well yes, a shell is a tool to invoke programs, and every program that you invoke in your script has to be installed for your script to work, that goes without saying. macOS comes with both bash and zsh installed by default. FreeBSD comes with neither. The OP may have a good reason for wanting to do that in bash, maybe it's something they want to add to an already written bash script.
      – Stéphane Chazelas
      Nov 10 at 9:28




      1




      1




      @pipe, again, a shell is a command interpreter. You'll see that many shell code excerpts written here invoke interpreters of other languages like perl, awk, sed or python (or even sh, bash...). Shell scripting is about using the best command for the task. Here even perl doesn't have an equivalent of zsh's $var:P (its Cwd::realpath behaves like BSD realpath or readlink -f while you'd want it to behave like GNU readlink -f or python's os.path.realpath())
      – Stéphane Chazelas
      Nov 10 at 10:08





      @pipe, again, a shell is a command interpreter. You'll see that many shell code excerpts written here invoke interpreters of other languages like perl, awk, sed or python (or even sh, bash...). Shell scripting is about using the best command for the task. Here even perl doesn't have an equivalent of zsh's $var:P (its Cwd::realpath behaves like BSD realpath or readlink -f while you'd want it to behave like GNU readlink -f or python's os.path.realpath())
      – Stéphane Chazelas
      Nov 10 at 10:08











      up vote
      6
      down vote













      One option you might want to consider is creating the file early on but only populating it later in your script. You can use the exec command to open the file in a file descriptor (such as 3, 4, etc.) and then later use a redirection to a file descriptor (>&3, etc.) to write contents to that file.



      Something like:



      #!/bin/bash

      # Open the file for read/write, so it doesn't get
      # truncated just yet (to preserve the contents in
      # case the initial checks fail.)
      exec 3<>dir/file.txt ||
      echo "Error creating dir/file.txt" >&2
      exit 1


      # Long checks here...
      check_ok ||
      echo "Failed checks" >&2
      # cleanup file before bailing out
      rm -f dir/file.txt
      exit 1


      # We're ready to write, first truncate the file.
      # Use "truncate(1)" from coreutils, and pass it
      # /dev/fd/3 so the file doesn't need to be reopened.
      truncate -s 0 /dev/fd/3

      # Now populate the file, use a redirection to write
      # to the previously opened file descriptor.
      populate_contents >&3


      You can also use a trap to clean up the file on error, that's a common practice.



      This way, you get a real check for permissions that you'll be able to create the file, while at the same time being able to perform it early enough that if that fails you haven't spent time waiting for the long checks.




      UPDATED: In order to avoid clobbering the file in case the checks fail, use bash's fd<>file redirection which does not truncate the file right away. (We don't care about reading from the file, this is just a workaround so we don't truncate it. Appending with >> would probably just work too, but I tend to find this one a bit more elegant, keeping the O_APPEND flag out of the picture.)



      When we're ready to replace the contents, we need to truncate the file first (otherwise if we're writing fewer bytes than there were in the file before, the trailing bytes would stay there.) We can use the truncate(1) command from coreutils for that purpose, and we can pass it the open file descriptor we have (using the /dev/fd/3 pseudo-file) so it doesn't need to reopen the file. (Again, technically something simpler like : >dir/file.txt would probably work, but not having to reopen the file is a more elegant solution.)






      share|improve this answer






















      • I had considered this, but the problem is that if another innocent error occurs somewhere between when I create the file and the point some time later where I would write to it, the user will probably be upset to find out that the specified file was overwritten.
        – BeeOnRope
        Nov 8 at 22:34






      • 1




        @BeeOnRope another option that won't clobber the file is to try to append nothing to it: echo -n >>file or true >> file. Of course, ext4 has append-only files, but you could live with that false positive.
        – mosvy
        Nov 9 at 13:15






      • 1




        @BeeOnRope If you need to preserve either (a) the existing file or (b) the (complete) new file, that's a different question than you have asked. A good answer to it is to move the existing file to a new name (e.g. my-file.txt.backup) before creating the new one. Another solution is to write the new file to a temporary file in the same folder, and then copy it over the old file after the rest of the script succeeds --- if that last operation failed, the user could manually fix the issue without losing his or her progress.
        – jpaugh
        Nov 9 at 19:45







      • 1




        @BeeOnRope If you go for the "atomic replace" route (which is a good one!) then typically you'll want to write the temporary file in the same directory as the final file (they need to be in the same filesystem for rename(2) to succeed) and use a unique name (so if there are two instances of the script running, they won't clobber each other's temporary files.) Opening a temporary file for writing in the same directory is usually a good indication you'll be able to rename it later (well, except if the final target exists and is a directory), so to some extent it also addresses your question.
        – Filipe Brandenburger
        Nov 9 at 20:55






      • 1




        @jpaugh - I'm quite reluctant to do that. It would inevitably lead to answers trying to solve my higher level problem, which might admit solutions that have nothing to do with the question "How can I check...". Regardless of what my high level problem is, I think this question stands alone as something you might reasonably want to do. It would be unfair to people who are answering that specific question if I changed it to "solve this specific issue I'm having with my script". I included those details here because I was asked, but I don't want to put change the primary question.
        – BeeOnRope
        Nov 15 at 21:10















      up vote
      6
      down vote













      One option you might want to consider is creating the file early on but only populating it later in your script. You can use the exec command to open the file in a file descriptor (such as 3, 4, etc.) and then later use a redirection to a file descriptor (>&3, etc.) to write contents to that file.



      Something like:



      #!/bin/bash

      # Open the file for read/write, so it doesn't get
      # truncated just yet (to preserve the contents in
      # case the initial checks fail.)
      exec 3<>dir/file.txt ||
      echo "Error creating dir/file.txt" >&2
      exit 1


      # Long checks here...
      check_ok ||
      echo "Failed checks" >&2
      # cleanup file before bailing out
      rm -f dir/file.txt
      exit 1


      # We're ready to write, first truncate the file.
      # Use "truncate(1)" from coreutils, and pass it
      # /dev/fd/3 so the file doesn't need to be reopened.
      truncate -s 0 /dev/fd/3

      # Now populate the file, use a redirection to write
      # to the previously opened file descriptor.
      populate_contents >&3


      You can also use a trap to clean up the file on error, that's a common practice.



      This way, you get a real check for permissions that you'll be able to create the file, while at the same time being able to perform it early enough that if that fails you haven't spent time waiting for the long checks.




      UPDATED: In order to avoid clobbering the file in case the checks fail, use bash's fd<>file redirection which does not truncate the file right away. (We don't care about reading from the file, this is just a workaround so we don't truncate it. Appending with >> would probably just work too, but I tend to find this one a bit more elegant, keeping the O_APPEND flag out of the picture.)



      When we're ready to replace the contents, we need to truncate the file first (otherwise if we're writing fewer bytes than there were in the file before, the trailing bytes would stay there.) We can use the truncate(1) command from coreutils for that purpose, and we can pass it the open file descriptor we have (using the /dev/fd/3 pseudo-file) so it doesn't need to reopen the file. (Again, technically something simpler like : >dir/file.txt would probably work, but not having to reopen the file is a more elegant solution.)






      share|improve this answer






















      • I had considered this, but the problem is that if another innocent error occurs somewhere between when I create the file and the point some time later where I would write to it, the user will probably be upset to find out that the specified file was overwritten.
        – BeeOnRope
        Nov 8 at 22:34






      • 1




        @BeeOnRope another option that won't clobber the file is to try to append nothing to it: echo -n >>file or true >> file. Of course, ext4 has append-only files, but you could live with that false positive.
        – mosvy
        Nov 9 at 13:15






      • 1




        @BeeOnRope If you need to preserve either (a) the existing file or (b) the (complete) new file, that's a different question than you have asked. A good answer to it is to move the existing file to a new name (e.g. my-file.txt.backup) before creating the new one. Another solution is to write the new file to a temporary file in the same folder, and then copy it over the old file after the rest of the script succeeds --- if that last operation failed, the user could manually fix the issue without losing his or her progress.
        – jpaugh
        Nov 9 at 19:45







      • 1




        @BeeOnRope If you go for the "atomic replace" route (which is a good one!) then typically you'll want to write the temporary file in the same directory as the final file (they need to be in the same filesystem for rename(2) to succeed) and use a unique name (so if there are two instances of the script running, they won't clobber each other's temporary files.) Opening a temporary file for writing in the same directory is usually a good indication you'll be able to rename it later (well, except if the final target exists and is a directory), so to some extent it also addresses your question.
        – Filipe Brandenburger
        Nov 9 at 20:55






      • 1




        @jpaugh - I'm quite reluctant to do that. It would inevitably lead to answers trying to solve my higher level problem, which might admit solutions that have nothing to do with the question "How can I check...". Regardless of what my high level problem is, I think this question stands alone as something you might reasonably want to do. It would be unfair to people who are answering that specific question if I changed it to "solve this specific issue I'm having with my script". I included those details here because I was asked, but I don't want to put change the primary question.
        – BeeOnRope
        Nov 15 at 21:10













      up vote
      6
      down vote










      up vote
      6
      down vote









      One option you might want to consider is creating the file early on but only populating it later in your script. You can use the exec command to open the file in a file descriptor (such as 3, 4, etc.) and then later use a redirection to a file descriptor (>&3, etc.) to write contents to that file.



      Something like:



      #!/bin/bash

      # Open the file for read/write, so it doesn't get
      # truncated just yet (to preserve the contents in
      # case the initial checks fail.)
      exec 3<>dir/file.txt ||
      echo "Error creating dir/file.txt" >&2
      exit 1


      # Long checks here...
      check_ok ||
      echo "Failed checks" >&2
      # cleanup file before bailing out
      rm -f dir/file.txt
      exit 1


      # We're ready to write, first truncate the file.
      # Use "truncate(1)" from coreutils, and pass it
      # /dev/fd/3 so the file doesn't need to be reopened.
      truncate -s 0 /dev/fd/3

      # Now populate the file, use a redirection to write
      # to the previously opened file descriptor.
      populate_contents >&3


      You can also use a trap to clean up the file on error, that's a common practice.



      This way, you get a real check for permissions that you'll be able to create the file, while at the same time being able to perform it early enough that if that fails you haven't spent time waiting for the long checks.




      UPDATED: In order to avoid clobbering the file in case the checks fail, use bash's fd<>file redirection which does not truncate the file right away. (We don't care about reading from the file, this is just a workaround so we don't truncate it. Appending with >> would probably just work too, but I tend to find this one a bit more elegant, keeping the O_APPEND flag out of the picture.)



      When we're ready to replace the contents, we need to truncate the file first (otherwise if we're writing fewer bytes than there were in the file before, the trailing bytes would stay there.) We can use the truncate(1) command from coreutils for that purpose, and we can pass it the open file descriptor we have (using the /dev/fd/3 pseudo-file) so it doesn't need to reopen the file. (Again, technically something simpler like : >dir/file.txt would probably work, but not having to reopen the file is a more elegant solution.)






      share|improve this answer














      One option you might want to consider is creating the file early on but only populating it later in your script. You can use the exec command to open the file in a file descriptor (such as 3, 4, etc.) and then later use a redirection to a file descriptor (>&3, etc.) to write contents to that file.



      Something like:



      #!/bin/bash

      # Open the file for read/write, so it doesn't get
      # truncated just yet (to preserve the contents in
      # case the initial checks fail.)
      exec 3<>dir/file.txt ||
      echo "Error creating dir/file.txt" >&2
      exit 1


      # Long checks here...
      check_ok ||
      echo "Failed checks" >&2
      # cleanup file before bailing out
      rm -f dir/file.txt
      exit 1


      # We're ready to write, first truncate the file.
      # Use "truncate(1)" from coreutils, and pass it
      # /dev/fd/3 so the file doesn't need to be reopened.
      truncate -s 0 /dev/fd/3

      # Now populate the file, use a redirection to write
      # to the previously opened file descriptor.
      populate_contents >&3


      You can also use a trap to clean up the file on error, that's a common practice.



      This way, you get a real check for permissions that you'll be able to create the file, while at the same time being able to perform it early enough that if that fails you haven't spent time waiting for the long checks.




      UPDATED: In order to avoid clobbering the file in case the checks fail, use bash's fd<>file redirection which does not truncate the file right away. (We don't care about reading from the file, this is just a workaround so we don't truncate it. Appending with >> would probably just work too, but I tend to find this one a bit more elegant, keeping the O_APPEND flag out of the picture.)



      When we're ready to replace the contents, we need to truncate the file first (otherwise if we're writing fewer bytes than there were in the file before, the trailing bytes would stay there.) We can use the truncate(1) command from coreutils for that purpose, and we can pass it the open file descriptor we have (using the /dev/fd/3 pseudo-file) so it doesn't need to reopen the file. (Again, technically something simpler like : >dir/file.txt would probably work, but not having to reopen the file is a more elegant solution.)







      share|improve this answer














      share|improve this answer



      share|improve this answer








      edited Nov 9 at 3:05

























      answered Nov 8 at 22:30









      Filipe Brandenburger

      6,6701732




      6,6701732











      • I had considered this, but the problem is that if another innocent error occurs somewhere between when I create the file and the point some time later where I would write to it, the user will probably be upset to find out that the specified file was overwritten.
        – BeeOnRope
        Nov 8 at 22:34






      • 1




        @BeeOnRope another option that won't clobber the file is to try to append nothing to it: echo -n >>file or true >> file. Of course, ext4 has append-only files, but you could live with that false positive.
        – mosvy
        Nov 9 at 13:15






      • 1




        @BeeOnRope If you need to preserve either (a) the existing file or (b) the (complete) new file, that's a different question than you have asked. A good answer to it is to move the existing file to a new name (e.g. my-file.txt.backup) before creating the new one. Another solution is to write the new file to a temporary file in the same folder, and then copy it over the old file after the rest of the script succeeds --- if that last operation failed, the user could manually fix the issue without losing his or her progress.
        – jpaugh
        Nov 9 at 19:45







      • 1




        @BeeOnRope If you go for the "atomic replace" route (which is a good one!) then typically you'll want to write the temporary file in the same directory as the final file (they need to be in the same filesystem for rename(2) to succeed) and use a unique name (so if there are two instances of the script running, they won't clobber each other's temporary files.) Opening a temporary file for writing in the same directory is usually a good indication you'll be able to rename it later (well, except if the final target exists and is a directory), so to some extent it also addresses your question.
        – Filipe Brandenburger
        Nov 9 at 20:55






      • 1




        @jpaugh - I'm quite reluctant to do that. It would inevitably lead to answers trying to solve my higher level problem, which might admit solutions that have nothing to do with the question "How can I check...". Regardless of what my high level problem is, I think this question stands alone as something you might reasonably want to do. It would be unfair to people who are answering that specific question if I changed it to "solve this specific issue I'm having with my script". I included those details here because I was asked, but I don't want to put change the primary question.
        – BeeOnRope
        Nov 15 at 21:10

















      • I had considered this, but the problem is that if another innocent error occurs somewhere between when I create the file and the point some time later where I would write to it, the user will probably be upset to find out that the specified file was overwritten.
        – BeeOnRope
        Nov 8 at 22:34






      • 1




        @BeeOnRope another option that won't clobber the file is to try to append nothing to it: echo -n >>file or true >> file. Of course, ext4 has append-only files, but you could live with that false positive.
        – mosvy
        Nov 9 at 13:15






      • 1




        @BeeOnRope If you need to preserve either (a) the existing file or (b) the (complete) new file, that's a different question than you have asked. A good answer to it is to move the existing file to a new name (e.g. my-file.txt.backup) before creating the new one. Another solution is to write the new file to a temporary file in the same folder, and then copy it over the old file after the rest of the script succeeds --- if that last operation failed, the user could manually fix the issue without losing his or her progress.
        – jpaugh
        Nov 9 at 19:45







      • 1




        @BeeOnRope If you go for the "atomic replace" route (which is a good one!) then typically you'll want to write the temporary file in the same directory as the final file (they need to be in the same filesystem for rename(2) to succeed) and use a unique name (so if there are two instances of the script running, they won't clobber each other's temporary files.) Opening a temporary file for writing in the same directory is usually a good indication you'll be able to rename it later (well, except if the final target exists and is a directory), so to some extent it also addresses your question.
        – Filipe Brandenburger
        Nov 9 at 20:55






      • 1




        @jpaugh - I'm quite reluctant to do that. It would inevitably lead to answers trying to solve my higher level problem, which might admit solutions that have nothing to do with the question "How can I check...". Regardless of what my high level problem is, I think this question stands alone as something you might reasonably want to do. It would be unfair to people who are answering that specific question if I changed it to "solve this specific issue I'm having with my script". I included those details here because I was asked, but I don't want to put change the primary question.
        – BeeOnRope
        Nov 15 at 21:10
















      I had considered this, but the problem is that if another innocent error occurs somewhere between when I create the file and the point some time later where I would write to it, the user will probably be upset to find out that the specified file was overwritten.
      – BeeOnRope
      Nov 8 at 22:34




      I had considered this, but the problem is that if another innocent error occurs somewhere between when I create the file and the point some time later where I would write to it, the user will probably be upset to find out that the specified file was overwritten.
      – BeeOnRope
      Nov 8 at 22:34




      1




      1




      @BeeOnRope another option that won't clobber the file is to try to append nothing to it: echo -n >>file or true >> file. Of course, ext4 has append-only files, but you could live with that false positive.
      – mosvy
      Nov 9 at 13:15




      @BeeOnRope another option that won't clobber the file is to try to append nothing to it: echo -n >>file or true >> file. Of course, ext4 has append-only files, but you could live with that false positive.
      – mosvy
      Nov 9 at 13:15




      1




      1




      @BeeOnRope If you need to preserve either (a) the existing file or (b) the (complete) new file, that's a different question than you have asked. A good answer to it is to move the existing file to a new name (e.g. my-file.txt.backup) before creating the new one. Another solution is to write the new file to a temporary file in the same folder, and then copy it over the old file after the rest of the script succeeds --- if that last operation failed, the user could manually fix the issue without losing his or her progress.
      – jpaugh
      Nov 9 at 19:45





      @BeeOnRope If you need to preserve either (a) the existing file or (b) the (complete) new file, that's a different question than you have asked. A good answer to it is to move the existing file to a new name (e.g. my-file.txt.backup) before creating the new one. Another solution is to write the new file to a temporary file in the same folder, and then copy it over the old file after the rest of the script succeeds --- if that last operation failed, the user could manually fix the issue without losing his or her progress.
      – jpaugh
      Nov 9 at 19:45





      1




      1




      @BeeOnRope If you go for the "atomic replace" route (which is a good one!) then typically you'll want to write the temporary file in the same directory as the final file (they need to be in the same filesystem for rename(2) to succeed) and use a unique name (so if there are two instances of the script running, they won't clobber each other's temporary files.) Opening a temporary file for writing in the same directory is usually a good indication you'll be able to rename it later (well, except if the final target exists and is a directory), so to some extent it also addresses your question.
      – Filipe Brandenburger
      Nov 9 at 20:55




      @BeeOnRope If you go for the "atomic replace" route (which is a good one!) then typically you'll want to write the temporary file in the same directory as the final file (they need to be in the same filesystem for rename(2) to succeed) and use a unique name (so if there are two instances of the script running, they won't clobber each other's temporary files.) Opening a temporary file for writing in the same directory is usually a good indication you'll be able to rename it later (well, except if the final target exists and is a directory), so to some extent it also addresses your question.
      – Filipe Brandenburger
      Nov 9 at 20:55




      1




      1




      @jpaugh - I'm quite reluctant to do that. It would inevitably lead to answers trying to solve my higher level problem, which might admit solutions that have nothing to do with the question "How can I check...". Regardless of what my high level problem is, I think this question stands alone as something you might reasonably want to do. It would be unfair to people who are answering that specific question if I changed it to "solve this specific issue I'm having with my script". I included those details here because I was asked, but I don't want to put change the primary question.
      – BeeOnRope
      Nov 15 at 21:10





      @jpaugh - I'm quite reluctant to do that. It would inevitably lead to answers trying to solve my higher level problem, which might admit solutions that have nothing to do with the question "How can I check...". Regardless of what my high level problem is, I think this question stands alone as something you might reasonably want to do. It would be unfair to people who are answering that specific question if I changed it to "solve this specific issue I'm having with my script". I included those details here because I was asked, but I don't want to put change the primary question.
      – BeeOnRope
      Nov 15 at 21:10











      up vote
      4
      down vote













      I think DopeGhoti's solution is better but this should also work:



      file=$1
      if [[ "$file:0:1" == '/' ]]; then
      dir=$file%/*
      elif [[ "$file" =~ .*/.* ]]; then
      dir="$(PWD)/$file%/*"
      else
      dir=$(PWD)
      fi

      if [[ -w "$dir" ]]; then
      echo "writable"
      #do stuff with writable file
      else
      echo "not writable"
      #do stuff without writable file
      fi


      The first if construct checks if the argument is a full path (starts with /) and sets the dir variable to the directory path up to the last /. Otherwise if the argument does not start with a / but does contain a / (specifying a sub directory) it will set dir to the present working directory + the sub directory path. Otherwise it assumes the present working directory. It then checks if that directory is writable.






      share|improve this answer
























        up vote
        4
        down vote













        I think DopeGhoti's solution is better but this should also work:



        file=$1
        if [[ "$file:0:1" == '/' ]]; then
        dir=$file%/*
        elif [[ "$file" =~ .*/.* ]]; then
        dir="$(PWD)/$file%/*"
        else
        dir=$(PWD)
        fi

        if [[ -w "$dir" ]]; then
        echo "writable"
        #do stuff with writable file
        else
        echo "not writable"
        #do stuff without writable file
        fi


        The first if construct checks if the argument is a full path (starts with /) and sets the dir variable to the directory path up to the last /. Otherwise if the argument does not start with a / but does contain a / (specifying a sub directory) it will set dir to the present working directory + the sub directory path. Otherwise it assumes the present working directory. It then checks if that directory is writable.






        share|improve this answer






















          up vote
          4
          down vote










          up vote
          4
          down vote









          I think DopeGhoti's solution is better but this should also work:



          file=$1
          if [[ "$file:0:1" == '/' ]]; then
          dir=$file%/*
          elif [[ "$file" =~ .*/.* ]]; then
          dir="$(PWD)/$file%/*"
          else
          dir=$(PWD)
          fi

          if [[ -w "$dir" ]]; then
          echo "writable"
          #do stuff with writable file
          else
          echo "not writable"
          #do stuff without writable file
          fi


          The first if construct checks if the argument is a full path (starts with /) and sets the dir variable to the directory path up to the last /. Otherwise if the argument does not start with a / but does contain a / (specifying a sub directory) it will set dir to the present working directory + the sub directory path. Otherwise it assumes the present working directory. It then checks if that directory is writable.






          share|improve this answer












          I think DopeGhoti's solution is better but this should also work:



          file=$1
          if [[ "$file:0:1" == '/' ]]; then
          dir=$file%/*
          elif [[ "$file" =~ .*/.* ]]; then
          dir="$(PWD)/$file%/*"
          else
          dir=$(PWD)
          fi

          if [[ -w "$dir" ]]; then
          echo "writable"
          #do stuff with writable file
          else
          echo "not writable"
          #do stuff without writable file
          fi


          The first if construct checks if the argument is a full path (starts with /) and sets the dir variable to the directory path up to the last /. Otherwise if the argument does not start with a / but does contain a / (specifying a sub directory) it will set dir to the present working directory + the sub directory path. Otherwise it assumes the present working directory. It then checks if that directory is writable.







          share|improve this answer












          share|improve this answer



          share|improve this answer










          answered Nov 8 at 22:19









          Jesse_b

          11.5k23063




          11.5k23063




















              up vote
              4
              down vote













              What about using normal test command like outlined below?



              FILE=$1

              DIR=$(dirname $FILE) # $DIR now contains '.' for file names only, 'foo' for 'foo/bar'

              if [ -d $DIR ] ; then
              echo "base directory $DIR for file exists"
              if [ -e $FILE ] ; then
              if [ -w $FILE ] ; then
              echo "file exists, is writeable"
              else
              echo "file exists, NOT writeable"
              fi
              elif [ -w $DIR ] ; then
              echo "directory is writeable"
              else
              echo "directory is NOT writeable"
              fi
              else
              echo "can NOT create file in non-existent directory $DIR "
              fi





              share|improve this answer




















              • I almost duplicated this answer! Nice introduction, but you might mention man test, and that [ ] is equivalent to test (not common knowledge anymore). Here is a the one-liner i was about to use in my inferior answer, to cover the exact case, for posterity: if [ -w $1] && [ ! -d $1 ] ; then echo "do stuff"; fi
                – Iron Gremlin
                Nov 10 at 1:44














              up vote
              4
              down vote













              What about using normal test command like outlined below?



              FILE=$1

              DIR=$(dirname $FILE) # $DIR now contains '.' for file names only, 'foo' for 'foo/bar'

              if [ -d $DIR ] ; then
              echo "base directory $DIR for file exists"
              if [ -e $FILE ] ; then
              if [ -w $FILE ] ; then
              echo "file exists, is writeable"
              else
              echo "file exists, NOT writeable"
              fi
              elif [ -w $DIR ] ; then
              echo "directory is writeable"
              else
              echo "directory is NOT writeable"
              fi
              else
              echo "can NOT create file in non-existent directory $DIR "
              fi





              share|improve this answer




















              • I almost duplicated this answer! Nice introduction, but you might mention man test, and that [ ] is equivalent to test (not common knowledge anymore). Here is a the one-liner i was about to use in my inferior answer, to cover the exact case, for posterity: if [ -w $1] && [ ! -d $1 ] ; then echo "do stuff"; fi
                – Iron Gremlin
                Nov 10 at 1:44












              up vote
              4
              down vote










              up vote
              4
              down vote









              What about using normal test command like outlined below?



              FILE=$1

              DIR=$(dirname $FILE) # $DIR now contains '.' for file names only, 'foo' for 'foo/bar'

              if [ -d $DIR ] ; then
              echo "base directory $DIR for file exists"
              if [ -e $FILE ] ; then
              if [ -w $FILE ] ; then
              echo "file exists, is writeable"
              else
              echo "file exists, NOT writeable"
              fi
              elif [ -w $DIR ] ; then
              echo "directory is writeable"
              else
              echo "directory is NOT writeable"
              fi
              else
              echo "can NOT create file in non-existent directory $DIR "
              fi





              share|improve this answer












              What about using normal test command like outlined below?



              FILE=$1

              DIR=$(dirname $FILE) # $DIR now contains '.' for file names only, 'foo' for 'foo/bar'

              if [ -d $DIR ] ; then
              echo "base directory $DIR for file exists"
              if [ -e $FILE ] ; then
              if [ -w $FILE ] ; then
              echo "file exists, is writeable"
              else
              echo "file exists, NOT writeable"
              fi
              elif [ -w $DIR ] ; then
              echo "directory is writeable"
              else
              echo "directory is NOT writeable"
              fi
              else
              echo "can NOT create file in non-existent directory $DIR "
              fi






              share|improve this answer












              share|improve this answer



              share|improve this answer










              answered Nov 9 at 0:11









              Jaleks

              1,168422




              1,168422











              • I almost duplicated this answer! Nice introduction, but you might mention man test, and that [ ] is equivalent to test (not common knowledge anymore). Here is a the one-liner i was about to use in my inferior answer, to cover the exact case, for posterity: if [ -w $1] && [ ! -d $1 ] ; then echo "do stuff"; fi
                – Iron Gremlin
                Nov 10 at 1:44
















              • I almost duplicated this answer! Nice introduction, but you might mention man test, and that [ ] is equivalent to test (not common knowledge anymore). Here is a the one-liner i was about to use in my inferior answer, to cover the exact case, for posterity: if [ -w $1] && [ ! -d $1 ] ; then echo "do stuff"; fi
                – Iron Gremlin
                Nov 10 at 1:44















              I almost duplicated this answer! Nice introduction, but you might mention man test, and that [ ] is equivalent to test (not common knowledge anymore). Here is a the one-liner i was about to use in my inferior answer, to cover the exact case, for posterity: if [ -w $1] && [ ! -d $1 ] ; then echo "do stuff"; fi
              – Iron Gremlin
              Nov 10 at 1:44




              I almost duplicated this answer! Nice introduction, but you might mention man test, and that [ ] is equivalent to test (not common knowledge anymore). Here is a the one-liner i was about to use in my inferior answer, to cover the exact case, for posterity: if [ -w $1] && [ ! -d $1 ] ; then echo "do stuff"; fi
              – Iron Gremlin
              Nov 10 at 1:44










              up vote
              4
              down vote













              You mentioned user experience was driving your question. I'll answer from a UX angle, since you've got good answers on the technical side.



              Rather than performing the check up-front, how about writing the results into a temporary file then at the very end, placing the results into the user's desired file? Like:



              userfile=$1:?Where would you like the file written?
              tmpfile=$(mktemp)

              # ... all the complicated stuff, writing into "$tmpfile"

              # fill user's file, keeping existing permissions or creating anew
              # while respecting umask
              cat "$tmpfile" > "$userfile"
              if [ 0 -eq $? ]; then
              rm "$tmpfile"
              else
              echo "Couldn't write results into $userfile." >&2
              echo "Results available in $tmpfile." >&2
              exit 1
              fi


              The good with this approach: it produces the desired operation in the normal happy path scenario, side-steps the test-and-set atomicity issue, preserves permissions of the target file while creating if necessary, and is dead simple to implement.



              Note: had we used mv, we'd be keeping the permissions of the temporary file -- we don't want that, I think: we want to keep the permissions as set on the target file.



              Now, the bad: it requires twice the space (cat .. > construct), forces the user to do some manual work if the target file wasn't writable at the time it needed to be, and leaves the temporary file laying around (which might have security or maintenance issues).






              share|improve this answer






















              • In fact, this is more or less what I'm doing now. I write most of the results to a temporary file and then at the end do the final processing step and write the results to the final file. The problem is I want to bail out early (at the start of the script) if that final step is likely to fail. The script may run unattended for minutes or hours, so you really want to know up front that it is doomed to fail!
                – BeeOnRope
                Nov 9 at 16:12










              • Sure, but there are so many ways this could fail: disk could fill, upstream directory could be removed, permissions could change, target file might be used for some other important stuff and the user forgot he assigned that same file to be destroyed by this operation. If we talk about this from a pure UX perspective, then perhaps the right thing to do is treat it like a job submission: at the end, when you know it worked correctly to completion, just tell the user where the resulting content resides and offer a suggested command for them to move it themselves.
                – bishop
                Nov 9 at 16:18











              • In theory, yes there are infinite ways this could fail. In practice, the overwhelming majority of the time this fails is because the path provided is not valid. I can't reasonably prevent some rogue user from concurrently modifying the FS to break the job in the middle of the operation, but I certainly can check the #1 failure cause of an invalid or not writable path.
                – BeeOnRope
                Nov 9 at 16:23















              up vote
              4
              down vote













              You mentioned user experience was driving your question. I'll answer from a UX angle, since you've got good answers on the technical side.



              Rather than performing the check up-front, how about writing the results into a temporary file then at the very end, placing the results into the user's desired file? Like:



              userfile=$1:?Where would you like the file written?
              tmpfile=$(mktemp)

              # ... all the complicated stuff, writing into "$tmpfile"

              # fill user's file, keeping existing permissions or creating anew
              # while respecting umask
              cat "$tmpfile" > "$userfile"
              if [ 0 -eq $? ]; then
              rm "$tmpfile"
              else
              echo "Couldn't write results into $userfile." >&2
              echo "Results available in $tmpfile." >&2
              exit 1
              fi


              The good with this approach: it produces the desired operation in the normal happy path scenario, side-steps the test-and-set atomicity issue, preserves permissions of the target file while creating if necessary, and is dead simple to implement.



              Note: had we used mv, we'd be keeping the permissions of the temporary file -- we don't want that, I think: we want to keep the permissions as set on the target file.



              Now, the bad: it requires twice the space (cat .. > construct), forces the user to do some manual work if the target file wasn't writable at the time it needed to be, and leaves the temporary file laying around (which might have security or maintenance issues).






              share|improve this answer






















              • In fact, this is more or less what I'm doing now. I write most of the results to a temporary file and then at the end do the final processing step and write the results to the final file. The problem is I want to bail out early (at the start of the script) if that final step is likely to fail. The script may run unattended for minutes or hours, so you really want to know up front that it is doomed to fail!
                – BeeOnRope
                Nov 9 at 16:12










              • Sure, but there are so many ways this could fail: disk could fill, upstream directory could be removed, permissions could change, target file might be used for some other important stuff and the user forgot he assigned that same file to be destroyed by this operation. If we talk about this from a pure UX perspective, then perhaps the right thing to do is treat it like a job submission: at the end, when you know it worked correctly to completion, just tell the user where the resulting content resides and offer a suggested command for them to move it themselves.
                – bishop
                Nov 9 at 16:18











              • In theory, yes there are infinite ways this could fail. In practice, the overwhelming majority of the time this fails is because the path provided is not valid. I can't reasonably prevent some rogue user from concurrently modifying the FS to break the job in the middle of the operation, but I certainly can check the #1 failure cause of an invalid or not writable path.
                – BeeOnRope
                Nov 9 at 16:23













              up vote
              4
              down vote










              up vote
              4
              down vote









              You mentioned user experience was driving your question. I'll answer from a UX angle, since you've got good answers on the technical side.



              Rather than performing the check up-front, how about writing the results into a temporary file then at the very end, placing the results into the user's desired file? Like:



              userfile=$1:?Where would you like the file written?
              tmpfile=$(mktemp)

              # ... all the complicated stuff, writing into "$tmpfile"

              # fill user's file, keeping existing permissions or creating anew
              # while respecting umask
              cat "$tmpfile" > "$userfile"
              if [ 0 -eq $? ]; then
              rm "$tmpfile"
              else
              echo "Couldn't write results into $userfile." >&2
              echo "Results available in $tmpfile." >&2
              exit 1
              fi


              The good with this approach: it produces the desired operation in the normal happy path scenario, side-steps the test-and-set atomicity issue, preserves permissions of the target file while creating if necessary, and is dead simple to implement.



              Note: had we used mv, we'd be keeping the permissions of the temporary file -- we don't want that, I think: we want to keep the permissions as set on the target file.



              Now, the bad: it requires twice the space (cat .. > construct), forces the user to do some manual work if the target file wasn't writable at the time it needed to be, and leaves the temporary file laying around (which might have security or maintenance issues).






              share|improve this answer














              You mentioned user experience was driving your question. I'll answer from a UX angle, since you've got good answers on the technical side.



              Rather than performing the check up-front, how about writing the results into a temporary file then at the very end, placing the results into the user's desired file? Like:



              userfile=$1:?Where would you like the file written?
              tmpfile=$(mktemp)

              # ... all the complicated stuff, writing into "$tmpfile"

              # fill user's file, keeping existing permissions or creating anew
              # while respecting umask
              cat "$tmpfile" > "$userfile"
              if [ 0 -eq $? ]; then
              rm "$tmpfile"
              else
              echo "Couldn't write results into $userfile." >&2
              echo "Results available in $tmpfile." >&2
              exit 1
              fi


              The good with this approach: it produces the desired operation in the normal happy path scenario, side-steps the test-and-set atomicity issue, preserves permissions of the target file while creating if necessary, and is dead simple to implement.



              Note: had we used mv, we'd be keeping the permissions of the temporary file -- we don't want that, I think: we want to keep the permissions as set on the target file.



              Now, the bad: it requires twice the space (cat .. > construct), forces the user to do some manual work if the target file wasn't writable at the time it needed to be, and leaves the temporary file laying around (which might have security or maintenance issues).







              share|improve this answer














              share|improve this answer



              share|improve this answer








              edited Nov 9 at 16:07

























              answered Nov 9 at 16:00









              bishop

              1,9082820




              1,9082820











              • In fact, this is more or less what I'm doing now. I write most of the results to a temporary file and then at the end do the final processing step and write the results to the final file. The problem is I want to bail out early (at the start of the script) if that final step is likely to fail. The script may run unattended for minutes or hours, so you really want to know up front that it is doomed to fail!
                – BeeOnRope
                Nov 9 at 16:12










              • Sure, but there are so many ways this could fail: disk could fill, upstream directory could be removed, permissions could change, target file might be used for some other important stuff and the user forgot he assigned that same file to be destroyed by this operation. If we talk about this from a pure UX perspective, then perhaps the right thing to do is treat it like a job submission: at the end, when you know it worked correctly to completion, just tell the user where the resulting content resides and offer a suggested command for them to move it themselves.
                – bishop
                Nov 9 at 16:18











              • In theory, yes there are infinite ways this could fail. In practice, the overwhelming majority of the time this fails is because the path provided is not valid. I can't reasonably prevent some rogue user from concurrently modifying the FS to break the job in the middle of the operation, but I certainly can check the #1 failure cause of an invalid or not writable path.
                – BeeOnRope
                Nov 9 at 16:23

















              • In fact, this is more or less what I'm doing now. I write most of the results to a temporary file and then at the end do the final processing step and write the results to the final file. The problem is I want to bail out early (at the start of the script) if that final step is likely to fail. The script may run unattended for minutes or hours, so you really want to know up front that it is doomed to fail!
                – BeeOnRope
                Nov 9 at 16:12










              • Sure, but there are so many ways this could fail: disk could fill, upstream directory could be removed, permissions could change, target file might be used for some other important stuff and the user forgot he assigned that same file to be destroyed by this operation. If we talk about this from a pure UX perspective, then perhaps the right thing to do is treat it like a job submission: at the end, when you know it worked correctly to completion, just tell the user where the resulting content resides and offer a suggested command for them to move it themselves.
                – bishop
                Nov 9 at 16:18











              • In theory, yes there are infinite ways this could fail. In practice, the overwhelming majority of the time this fails is because the path provided is not valid. I can't reasonably prevent some rogue user from concurrently modifying the FS to break the job in the middle of the operation, but I certainly can check the #1 failure cause of an invalid or not writable path.
                – BeeOnRope
                Nov 9 at 16:23
















              In fact, this is more or less what I'm doing now. I write most of the results to a temporary file and then at the end do the final processing step and write the results to the final file. The problem is I want to bail out early (at the start of the script) if that final step is likely to fail. The script may run unattended for minutes or hours, so you really want to know up front that it is doomed to fail!
              – BeeOnRope
              Nov 9 at 16:12




              In fact, this is more or less what I'm doing now. I write most of the results to a temporary file and then at the end do the final processing step and write the results to the final file. The problem is I want to bail out early (at the start of the script) if that final step is likely to fail. The script may run unattended for minutes or hours, so you really want to know up front that it is doomed to fail!
              – BeeOnRope
              Nov 9 at 16:12












              Sure, but there are so many ways this could fail: disk could fill, upstream directory could be removed, permissions could change, target file might be used for some other important stuff and the user forgot he assigned that same file to be destroyed by this operation. If we talk about this from a pure UX perspective, then perhaps the right thing to do is treat it like a job submission: at the end, when you know it worked correctly to completion, just tell the user where the resulting content resides and offer a suggested command for them to move it themselves.
              – bishop
              Nov 9 at 16:18





              Sure, but there are so many ways this could fail: disk could fill, upstream directory could be removed, permissions could change, target file might be used for some other important stuff and the user forgot he assigned that same file to be destroyed by this operation. If we talk about this from a pure UX perspective, then perhaps the right thing to do is treat it like a job submission: at the end, when you know it worked correctly to completion, just tell the user where the resulting content resides and offer a suggested command for them to move it themselves.
              – bishop
              Nov 9 at 16:18













              In theory, yes there are infinite ways this could fail. In practice, the overwhelming majority of the time this fails is because the path provided is not valid. I can't reasonably prevent some rogue user from concurrently modifying the FS to break the job in the middle of the operation, but I certainly can check the #1 failure cause of an invalid or not writable path.
              – BeeOnRope
              Nov 9 at 16:23





              In theory, yes there are infinite ways this could fail. In practice, the overwhelming majority of the time this fails is because the path provided is not valid. I can't reasonably prevent some rogue user from concurrently modifying the FS to break the job in the middle of the operation, but I certainly can check the #1 failure cause of an invalid or not writable path.
              – BeeOnRope
              Nov 9 at 16:23











              up vote
              2
              down vote













              TL;DR:



              : >> "$userfile"


              Unlike <> "$userfile" or touch "$userfile", this will not make any spurious modifications to the file's timestamps and will also work with write-only files.




              From the OP:




              I want to make a reasonable check if the file can be created/overwritten, but not actually create it.




              And from your comment to my answer from a UX perspective:




              The overwhelming majority of the time this fails is because the path provided is not valid. I can't reasonably prevent some rogue user some concurrently modifying the FS to break the job in the middle of the operation, but I certainly can check the #1 failure cause of an invalid or not writable path.




              The only reliable test is to open(2) the file, because only that resolves every question about the writeability: path, ownership, filesystem, network, security context, etc. Any other test will address some part of writeability, but not others. If you want a subset of tests, you'll ultimately have to choose what's important to you.



              But here's another thought. From what I understand:



              1. the content creation process is long-running, and

              2. the target file should be left in a consistent state.

              You're wanting to do this pre-check because of #1, and you don't want to fiddle with an existing file because of #2. So why don't you just ask the shell to open the file for append, but don't actually append anything?



              $ tree -ps
              .
              ├── [dr-x------ 4096] dir_r
              ├── [drwx------ 4096] dir_w
              ├── [-r-------- 0] file_r
              └── [-rw------- 0] file_w

              $ for p in file_r dir_r/foo file_w dir_w/foo; do : >> $p; done
              -bash: file_r: Permission denied
              -bash: dir_r/foo: Permission denied

              $ tree -ps
              .
              ├── [dr-x------ 4096] dir_r
              ├── [drwx------ 4096] dir_w
              │   └── [-rw-rw-r-- 0] foo
              ├── [-r-------- 0] file_r
              └── [-rw------- 0] file_w


              Under the hood, this resolves the writeability question exactly as wanted:



              open("dir_w/foo", O_WRONLY|O_CREAT|O_APPEND, 0666) = 3


              but without modifying the file's contents or metadata. Now, yes, this approach:



              • doesn't tell you if the file is append only, which might be a problem when you go about updating it at the end of your content creation. You can detect this, to a degree, with lsattr and react accordingly.

              • creates a file that didn't previously exist, if such is the case: mitigate this with a selective rm.

              While I contend (in my other answer) that the most user-friendly approach is to create a temporary file the user has to move, I think this is the least user-hostile approach to fully vet their input.






              share|improve this answer


























                up vote
                2
                down vote













                TL;DR:



                : >> "$userfile"


                Unlike <> "$userfile" or touch "$userfile", this will not make any spurious modifications to the file's timestamps and will also work with write-only files.




                From the OP:




                I want to make a reasonable check if the file can be created/overwritten, but not actually create it.




                And from your comment to my answer from a UX perspective:




                The overwhelming majority of the time this fails is because the path provided is not valid. I can't reasonably prevent some rogue user some concurrently modifying the FS to break the job in the middle of the operation, but I certainly can check the #1 failure cause of an invalid or not writable path.




                The only reliable test is to open(2) the file, because only that resolves every question about the writeability: path, ownership, filesystem, network, security context, etc. Any other test will address some part of writeability, but not others. If you want a subset of tests, you'll ultimately have to choose what's important to you.



                But here's another thought. From what I understand:



                1. the content creation process is long-running, and

                2. the target file should be left in a consistent state.

                You're wanting to do this pre-check because of #1, and you don't want to fiddle with an existing file because of #2. So why don't you just ask the shell to open the file for append, but don't actually append anything?



                $ tree -ps
                .
                ├── [dr-x------ 4096] dir_r
                ├── [drwx------ 4096] dir_w
                ├── [-r-------- 0] file_r
                └── [-rw------- 0] file_w

                $ for p in file_r dir_r/foo file_w dir_w/foo; do : >> $p; done
                -bash: file_r: Permission denied
                -bash: dir_r/foo: Permission denied

                $ tree -ps
                .
                ├── [dr-x------ 4096] dir_r
                ├── [drwx------ 4096] dir_w
                │   └── [-rw-rw-r-- 0] foo
                ├── [-r-------- 0] file_r
                └── [-rw------- 0] file_w


                Under the hood, this resolves the writeability question exactly as wanted:



                open("dir_w/foo", O_WRONLY|O_CREAT|O_APPEND, 0666) = 3


                but without modifying the file's contents or metadata. Now, yes, this approach:



                • doesn't tell you if the file is append only, which might be a problem when you go about updating it at the end of your content creation. You can detect this, to a degree, with lsattr and react accordingly.

                • creates a file that didn't previously exist, if such is the case: mitigate this with a selective rm.

                While I contend (in my other answer) that the most user-friendly approach is to create a temporary file the user has to move, I think this is the least user-hostile approach to fully vet their input.






                share|improve this answer
























                  up vote
                  2
                  down vote










                  up vote
                  2
                  down vote









                  TL;DR:



                  : >> "$userfile"


                  Unlike <> "$userfile" or touch "$userfile", this will not make any spurious modifications to the file's timestamps and will also work with write-only files.




                  From the OP:




                  I want to make a reasonable check if the file can be created/overwritten, but not actually create it.




                  And from your comment to my answer from a UX perspective:




                  The overwhelming majority of the time this fails is because the path provided is not valid. I can't reasonably prevent some rogue user some concurrently modifying the FS to break the job in the middle of the operation, but I certainly can check the #1 failure cause of an invalid or not writable path.




                  The only reliable test is to open(2) the file, because only that resolves every question about the writeability: path, ownership, filesystem, network, security context, etc. Any other test will address some part of writeability, but not others. If you want a subset of tests, you'll ultimately have to choose what's important to you.



                  But here's another thought. From what I understand:



                  1. the content creation process is long-running, and

                  2. the target file should be left in a consistent state.

                  You're wanting to do this pre-check because of #1, and you don't want to fiddle with an existing file because of #2. So why don't you just ask the shell to open the file for append, but don't actually append anything?



                  $ tree -ps
                  .
                  ├── [dr-x------ 4096] dir_r
                  ├── [drwx------ 4096] dir_w
                  ├── [-r-------- 0] file_r
                  └── [-rw------- 0] file_w

                  $ for p in file_r dir_r/foo file_w dir_w/foo; do : >> $p; done
                  -bash: file_r: Permission denied
                  -bash: dir_r/foo: Permission denied

                  $ tree -ps
                  .
                  ├── [dr-x------ 4096] dir_r
                  ├── [drwx------ 4096] dir_w
                  │   └── [-rw-rw-r-- 0] foo
                  ├── [-r-------- 0] file_r
                  └── [-rw------- 0] file_w


                  Under the hood, this resolves the writeability question exactly as wanted:



                  open("dir_w/foo", O_WRONLY|O_CREAT|O_APPEND, 0666) = 3


                  but without modifying the file's contents or metadata. Now, yes, this approach:



                  • doesn't tell you if the file is append only, which might be a problem when you go about updating it at the end of your content creation. You can detect this, to a degree, with lsattr and react accordingly.

                  • creates a file that didn't previously exist, if such is the case: mitigate this with a selective rm.

                  While I contend (in my other answer) that the most user-friendly approach is to create a temporary file the user has to move, I think this is the least user-hostile approach to fully vet their input.






                  share|improve this answer














                  TL;DR:



                  : >> "$userfile"


                  Unlike <> "$userfile" or touch "$userfile", this will not make any spurious modifications to the file's timestamps and will also work with write-only files.




                  From the OP:




                  I want to make a reasonable check if the file can be created/overwritten, but not actually create it.




                  And from your comment to my answer from a UX perspective:




                  The overwhelming majority of the time this fails is because the path provided is not valid. I can't reasonably prevent some rogue user some concurrently modifying the FS to break the job in the middle of the operation, but I certainly can check the #1 failure cause of an invalid or not writable path.




                  The only reliable test is to open(2) the file, because only that resolves every question about the writeability: path, ownership, filesystem, network, security context, etc. Any other test will address some part of writeability, but not others. If you want a subset of tests, you'll ultimately have to choose what's important to you.



                  But here's another thought. From what I understand:



                  1. the content creation process is long-running, and

                  2. the target file should be left in a consistent state.

                  You're wanting to do this pre-check because of #1, and you don't want to fiddle with an existing file because of #2. So why don't you just ask the shell to open the file for append, but don't actually append anything?



                  $ tree -ps
                  .
                  ├── [dr-x------ 4096] dir_r
                  ├── [drwx------ 4096] dir_w
                  ├── [-r-------- 0] file_r
                  └── [-rw------- 0] file_w

                  $ for p in file_r dir_r/foo file_w dir_w/foo; do : >> $p; done
                  -bash: file_r: Permission denied
                  -bash: dir_r/foo: Permission denied

                  $ tree -ps
                  .
                  ├── [dr-x------ 4096] dir_r
                  ├── [drwx------ 4096] dir_w
                  │   └── [-rw-rw-r-- 0] foo
                  ├── [-r-------- 0] file_r
                  └── [-rw------- 0] file_w


                  Under the hood, this resolves the writeability question exactly as wanted:



                  open("dir_w/foo", O_WRONLY|O_CREAT|O_APPEND, 0666) = 3


                  but without modifying the file's contents or metadata. Now, yes, this approach:



                  • doesn't tell you if the file is append only, which might be a problem when you go about updating it at the end of your content creation. You can detect this, to a degree, with lsattr and react accordingly.

                  • creates a file that didn't previously exist, if such is the case: mitigate this with a selective rm.

                  While I contend (in my other answer) that the most user-friendly approach is to create a temporary file the user has to move, I think this is the least user-hostile approach to fully vet their input.







                  share|improve this answer














                  share|improve this answer



                  share|improve this answer








                  edited Nov 10 at 3:10

























                  answered Nov 9 at 19:31









                  bishop

                  1,9082820




                  1,9082820



























                      draft saved

                      draft discarded
















































                      Thanks for contributing an answer to Unix & Linux Stack Exchange!


                      • Please be sure to answer the question. Provide details and share your research!

                      But avoid


                      • Asking for help, clarification, or responding to other answers.

                      • Making statements based on opinion; back them up with references or personal experience.

                      To learn more, see our tips on writing great answers.





                      Some of your past answers have not been well-received, and you're in danger of being blocked from answering.


                      Please pay close attention to the following guidance:


                      • Please be sure to answer the question. Provide details and share your research!

                      But avoid


                      • Asking for help, clarification, or responding to other answers.

                      • Making statements based on opinion; back them up with references or personal experience.

                      To learn more, see our tips on writing great answers.




                      draft saved


                      draft discarded














                      StackExchange.ready(
                      function ()
                      StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f480656%2fhow-can-i-check-if-a-file-can-be-created-or-truncated-overwritten-in-bash%23new-answer', 'question_page');

                      );

                      Post as a guest















                      Required, but never shown





















































                      Required, but never shown














                      Required, but never shown












                      Required, but never shown







                      Required, but never shown

































                      Required, but never shown














                      Required, but never shown












                      Required, but never shown







                      Required, but never shown







                      Popular posts from this blog

                      𛂒𛀶,𛀽𛀑𛂀𛃧𛂓𛀙𛃆𛃑𛃷𛂟𛁡𛀢𛀟𛁤𛂽𛁕𛁪𛂟𛂯,𛁞𛂧𛀴𛁄𛁠𛁼𛂿𛀤 𛂘,𛁺𛂾𛃭𛃭𛃵𛀺,𛂣𛃍𛂖𛃶 𛀸𛃀𛂖𛁶𛁏𛁚 𛂢𛂞 𛁰𛂆𛀔,𛁸𛀽𛁓𛃋𛂇𛃧𛀧𛃣𛂐𛃇,𛂂𛃻𛃲𛁬𛃞𛀧𛃃𛀅 𛂭𛁠𛁡𛃇𛀷𛃓𛁥,𛁙𛁘𛁞𛃸𛁸𛃣𛁜,𛂛,𛃿,𛁯𛂘𛂌𛃛𛁱𛃌𛂈𛂇 𛁊𛃲,𛀕𛃴𛀜 𛀶𛂆𛀶𛃟𛂉𛀣,𛂐𛁞𛁾 𛁷𛂑𛁳𛂯𛀬𛃅,𛃶𛁼

                      Edmonton

                      Crossroads (UK TV series)