geekhack
geekhack Community => Other Geeky Stuff => Topic started by: dante on Tue, 03 January 2017, 12:26:31
-
Can the following be done easily with Unix/Linux shell command line? It's easy in Python/Ruby but I'm curious how much more minimalist it could be (if it can be done at all) in shell:
Take these two files (there could be many more, but for this example just two) and combine them also adding a field and populate depending on file name.
Input
File0001.csv:
FieldB,FieldC
A,0
A,1
A,2
File0002.csv:
FieldB,FieldC
B,0
B,1
B,2
Output
FileCombined.csv
FieldA,FieldB,FieldC
0001,A,0
0001,A,1
0001,A,2
0002,B,0
0002,B,1
0002,B,2
FieldA = is generated on the file it came from. Hopefully the field names/lines of text don't have to be treated as separate entities but just text.
-
Not at my computer to be able to write the code but this is a no-brainer for shell just plain bash and awk.
-
Dealing with the headers in the input files and for the output makes it a little ugly, but this should work:
$ awk 'BEGIN { OFS=","; print "FieldA,FieldB,FieldC" } FNR>1 { print FILENAME, $0 }' File00*.csv | sed -r 's/^File([0-9]+)\.csv/\1/'
FieldA,FieldB,FieldC
0001,A,0
0001,A,1
0001,A,2
0002,B,0
0002,B,1
0002,B,2
-
Thanks... you are absolutely right it's a little ugly. :D I just need to dissect it to see what's going on. Thanks!
-
Here is a simple one-liner solution with sed. This assumes that the csv files are named consistently and the same fields are present in each.
$ echo "FieldA,FieldB,FieldC" >FileCombined.csv; for F in $(\ls File*[0-9].csv); do A=$(echo "$F" |sed "s/[^0-9]//g"); cat "$F" |tail -n +2 |sed "s/^/$A,/"; done >>FileCombined.csv
Cheers.