Tags: alli, bcp, command, databaseviews, dear, file, files, header, iam, insert, linux, output, procedures, programming, script, shell, stored, text, trailer, txt, unix

How to insert header and trailer in txt file using shell script

On Programmer » Unix & Linux

12,623 words with 8 Comments; publish: Wed, 07 May 2008 23:59:00 GMT; (200109.38, « »)

Dear all

I wrote a shell script which, I use bcp command to output database

views/stored procedures (SP) result to text files (.txt files). But, I

am required to put header and trailer in the above text files.

The header is datetime, which is required to insert at the first line

of text files.

The trailer is number of rows of database views/SP generated result. It

is required to append at the end of text files. The command i used to

count number of rows is wc -l filename | cut -f1 -d ""

My required output format:

filenumber yymmdd (<- header)

'bcp output data

...

number of rows (<-trailer)

What command I use to insert a date at the first line of file?

How can I append line at the end of file?

How can I insert/append the header and trailer in EACH generated files?

Cheers

Bon

All Comments

Leave a comment...

  • 8 Comments
    • bonnie.tangyn.unix-linux.todaysummary.com.gmail.com wrote:

      > Dear all

      > I wrote a shell script which, I use bcp command to output database

      > views/stored procedures (SP) result to text files (.txt files). But, I

      > am required to put header and trailer in the above text files.

      > The header is datetime, which is required to insert at the first line

      > of text files.

      > The trailer is number of rows of database views/SP generated result. It

      > is required to append at the end of text files. The command i used to

      > count number of rows is wc -l filename | cut -f1 -d ""

      > My required output format:

      > filenumber yymmdd (<- header)

      > 'bcp output data

      > ...

      > number of rows (<-trailer)

      > What command I use to insert a date at the first line of file?

      { printf "%s\n" "$yourdate" ; cat bcpfile ; } >tmpfile

      > How can I append line at the end of file?

      your-wc-pipeline >> yourfile

      > How can I insert/append the header and trailer in EACH generated files?

      One possibility (untested)...

      for f in *.txt # whatever pattern matches in your case

      do

      mv -i "$f" "$f.bak"

      { printf "filenumber %s\n" "$(date +%y%m%d)" # current in this example

      cat "$f.bak"

      wc -l "$f.bak" | awk '{print $1}' # or use cut, if it works for you

      } >"$f"

      done

      Janis

      >

      > Cheers

      > Bon

      >

      #1; Thu, 08 May 2008 00:01:00 GMT
    • On 2006-08-15, bonnie.tangyn.unix-linux.todaysummary.com.gmail.com wrote:

      > Dear all

      > I wrote a shell script which, I use bcp command to output database

      > views/stored procedures (SP) result to text files (.txt files). But, I

      > am required to put header and trailer in the above text files.

      The most efficient way would probably be to include it in the

      script that calls bcp.

      > The header is datetime, which is required to insert at the first line

      > of text files.

      > The trailer is number of rows of database views/SP generated result. It

      > is required to append at the end of text files. The command i used to

      > count number of rows is wc -l filename | cut -f1 -d ""

      There's no need for cut:

      wc -l < filename

      > My required output format:

      > filenumber yymmdd (<- header)

      Where does filenumber come from? Is it the name of the file? Or

      derived from the name of the file?

      Use a real date format, it's easier to read and less ambiguous:

      date +%Y-%m-%d

      > 'bcp output data

      > ...

      > number of rows (<-trailer)

      > What command I use to insert a date at the first line of file?

      You need a temporary file.

      > How can I append line at the end of file?

      printf "%s\n" "WHATEVER" >> FILE

      > How can I insert/append the header and trailer in EACH generated files?

      If you cannot incorporate all of this into the script that

      generates the .txt files (why not?), try this:

      for file in *.txt

      do

      cp "$file" tempfile

      filenumber='

      date +"$filenumber +%Y-%m-%d" > "$file"

      tee -a tempfile < "tempfile" | wc -l >> "$file"

      done

      Chris F.A. Johnson, author <http://cfaj.freeshell.org>

      Shell Scripting Recipes: A Problem-Solution Approach (2005, Apress)

      ===== My code in this post, if any, assumes the POSIX locale

      ===== and is released under the GNU General Public Licence

      #2; Thu, 08 May 2008 00:01:00 GMT
    • Dear all

      Thanks for providing me the script. It works. Thank you. But, I have

      another question.

      How can I add the number of rows count by 2? Because I am required to

      include header and trailer in the number of counts. For example, if the

      wc -l is returned 5 records. I need to output 7 because of header and

      trailer.

      The script I use as follow:

      for f in *.txt

      do

      mv -i "$f" "$f.bak"

      { printf "filenumber %s\n" "$(date +%y%m%d)"

      cat "$f.bak"

      wc -l "$f.bak" | awk '{print $1}'

      } >"$f"

      done

      Cheers

      Bon

      Chris F.A. Johnson =E5=AF=AB=E9=81=93=EF=BC=9A

      > On 2006-08-15, bonnie.tangyn.unix-linux.todaysummary.com.gmail.com wrote:

      > The most efficient way would probably be to include it in the

      > script that calls bcp.

      >

      > There's no need for cut:

      > wc -l < filename

      >

      > Where does filenumber come from? Is it the name of the file? Or

      > derived from the name of the file?

      > Use a real date format, it's easier to read and less ambiguous:

      > date +%Y-%m-%d

      >

      > You need a temporary file.

      >

      > printf "%s\n" "WHATEVER" >> FILE

      >

      > If you cannot incorporate all of this into the script that

      > generates the .txt files (why not?), try this:

      > for file in *.txt

      > do

      > cp "$file" tempfile

      > filenumber=3D'

      > date +"$filenumber +%Y-%m-%d" > "$file"

      > tee -a tempfile < "tempfile" | wc -l >> "$file"

      > done

      > --

      > Chris F.A. Johnson, author <http://cfaj.freeshell.org>

      > Shell Scripting Recipes: A Problem-Solution Approach (2005, Apress)

      > =3D=3D=3D=3D=3D My code in this post, if any, assumes the POSIX locale

      > =3D=3D=3D=3D=3D and is released under the GNU General Public Licence

      #3; Thu, 08 May 2008 00:03:00 GMT
    • On 2006-08-16, bonnie.tangyn.unix-linux.todaysummary.com.gmail.com wrote:

      > Chris F.A. Johnson ?

      >

      [please do not top post]

      > Thanks for providing me the script. It works. Thank you. But, I have

      > another question.

      > How can I add the number of rows count by 2? Because I am required to

      > include header and trailer in the number of counts. For example, if the

      > wc -l is returned 5 records. I need to output 7 because of header and

      > trailer.

      > The script I use as follow:

      > for f in *.txt

      > do

      > mv -i "$f" "$f.bak"

      > { printf "filenumber %s\n" "$(date +%y%m%d)"

      > cat "$f.bak"

      > wc -l "$f.bak" | awk '{print $1}'

      There is no need for awk, and this also adds the two extra lines:

      printf "%s\n" $(( $(wc -l < "$f.bak") + 2 ))

      > } >"$f"

      > done

      Chris F.A. Johnson, author <http://cfaj.freeshell.org>

      Shell Scripting Recipes: A Problem-Solution Approach (2005, Apress)

      ===== My code in this post, if any, assumes the POSIX locale

      ===== and is released under the GNU General Public Licence

      #4; Thu, 08 May 2008 00:04:00 GMT
    • On 16 Aug 2006 03:04:30 -0700, bonnie.tangyn.unix-linux.todaysummary.com.gmail.com wrote:

      [...]

      > How can I add the number of rows count by 2? Because I am required to

      > include header and trailer in the number of counts. For example, if the

      > wc -l is returned 5 records. I need to output 7 because of header and

      > trailer.

      > The script I use as follow:

      > for f in *.txt

      > do

      > mv -i "$f" "$f.bak"

      > { printf "filenumber %s\n" "$(date +%y%m%d)"

      > cat "$f.bak"

      > wc -l "$f.bak" | awk '{print $1}'

      > } >"$f"

      > done

      [...]

      for f in *.txt

      do

      mv -i -- "$f" "$f.bak" || continue

      awk -v fn="$(date +%y%m%d)" '

      BEGIN {

      print "filenumber", fn

      }

      {print}

      END {

      print NR + 2

      }' < "$f.back" > "$f"

      done

      Stephane

      #5; Thu, 08 May 2008 00:05:00 GMT
    • I used printf "%s\n" $(($wc -l < "$f.bak") + 2)) to append total number

      of rows in $f.bak and header and trailer at the end of the text file.

      How can I put spaces in front of the outputed total number of rows?

      For example:

      output result: 504

      it will put: ^^^^^^504 where ^ is space

      output result: 2

      it will put ^^^^^^^^2

      How can I put space in front of the no. of rows automatically?

      Cheers

      Bon

      Stephane Chazelas wrote:

      > On 16 Aug 2006 03:04:30 -0700, bonnie.tangyn.unix-linux.todaysummary.com.gmail.com wrote:

      > [...]

      > [...]

      >

      > for f in *.txt

      > do

      > mv -i -- "$f" "$f.bak" || continue

      > awk -v fn="$(date +%y%m%d)" '

      > BEGIN {

      > print "filenumber", fn

      > }

      > {print}

      > END {

      > print NR + 2

      > }' < "$f.back" > "$f"

      > done

      >

      > --

      > Stephane

      #6; Thu, 08 May 2008 00:06:00 GMT
    • On 16 Aug 2006 23:17:30 -0700, bonnie.tangyn.unix-linux.todaysummary.com.gmail.com wrote:

      > I used printf "%s\n" $(($wc -l < "$f.bak") + 2)) to append total number

      > of rows in $f.bak and header and trailer at the end of the text file.

      > How can I put spaces in front of the outputed total number of rows?

      > For example:

      > output result: 504

      > it will put: ^^^^^^504 where ^ is space

      > output result: 2

      > it will put ^^^^^^^^2

      > How can I put space in front of the no. of rows automatically?

      [...]

      printf 'output result: %9d\n' "$number"

      Or in awk:

      printf "output result: %9d\n", number

      $ printf '%9d\n' 1 11 111 1111

      1

      11

      111

      1111

      You can use %9s instead if you want.

      Differences are only about :

      $ printf '%9s\n' 01

      01

      $ printf '%9d\n' 01

      1

      $ printf '%9d\n' 0x20

      32

      Stephane

      #7; Thu, 08 May 2008 00:07:00 GMT
    • I am required to map file code, which is in another file, by using

      filename. But, the generated txt filenames are in the following format:

      student_name20060811.txt, school_address20060711.txt,

      home_address20050901.txt... I tried to use grep function and then

      append the file code in the txt file.

      My problem is the grep function has to match student_name20060811.txt

      and then get the ABC001 file code. It can't match student_name as key

      only. Could you give me some advices on my code.

      I got a file code reference table and its format is as follow;

      student_name ABC001

      school_address CDE002

      home_address EFG003

      .=2E.

      Inside

      filecode_connect =3D $filelocation/referencefilename.cfg

      .=2E.

      'bcp views/stored procedures data and output as .txt files

      .=2E.

      for f in *.txt

      do

      mv -i "$f" "$f.bak"

      {

      mappedcode =3D `grep $f $filecode_connect | cut -f2 -d" "`

      printf "$mappedcode" "$(date +%y%m%d)"

      cat "$f.bak"

      printf "%s\n" $(( $(wc -l < "$f.bak") + 2 ))

      } >"$f"

      done

      My desired output format for student_name file for example:

      ABC001060811

      bcp output data

      ^^^^^^504 (<- number of rows count, ^ is space)

      Cheers

      Bon

      Stephane Chazelas =BCg=B9D=A1G

      > On 16 Aug 2006 23:17:30 -0700, bonnie.tangyn.unix-linux.todaysummary.com.gmail.com wrote:

      > [...]

      > printf 'output result: %9d\n' "$number"

      > Or in awk:

      > printf "output result: %9d\n", number

      > $ printf '%9d\n' 1 11 111 1111

      > 1

      > 11

      > 111

      > 1111

      >

      > You can use %9s instead if you want.

      > Differences are only about :

      > $ printf '%9s\n' 01

      > 01

      > $ printf '%9d\n' 01

      > 1

      > $ printf '%9d\n' 0x20

      > 32

      >=20

      > --=20

      > Stephane

      #8; Thu, 08 May 2008 00:08:00 GMT