Thing Every Once in a While

Off and on, I’ve always been pretty upset that when processing data in UNIX, you generally have to write out to another file than you read in from.  Of course, if you use a vi script or perl, you can have the file be edited in place, but if you want to use the normal UNIX utilities, you have to do something like this:

cat file.dat | sort | cut -d, -f4 > file.dat.tmp; mv file.dat.tmp file.dat

The issue being: if you try writing directly out to the input file (in this case, file.dat), it overwrites the file with a 0-byte file before the reading happens.  I’ve always wondered why someone can’t just make a program that will slurp in the whole intermediate file before even opening the file for writing, and then writing the whole file at once.

Well, after a few years of wondering, I’ve finally written it.  It should compile on any standard UNIX/Linux/BSD system.  Here’s the above command, using my utility:

cat file.dat | sort | cut -d, -f4 | dw file.dat

It’s very simplistic, and would need to be expanded upon to be used in live production environments, but it should be fine for simple scripts.  Enjoy!

http://joshodom.net/weekly/week1.tar.gz