Filters/Text Processors Commands

1. cut

   • Cut is a command line utility that allows you to cut parts of lines from specified files or piped data and print the result to standard output. It can be used to cut parts of a line by delimiter, byte position and character.

     • cut filename = Does not work

     • cut --version = Check Version

     • cut -c1 filename = List one character

       

     • cut -c1, 2, 4 = Pick and choose character

     • cut -c1-3 filename = List range of characters

       

     • cut -c1-3, 6-8 filename = List specific range of characters

     • cut -b1-3 filename = List by byte size

     • cut -d: -f 6 /etc/passwd = List first 6th column separated by :

     • cut -d: -f 6-7 /etc/passwd = List first 6th & 7th column separated by :

     • ls -l | cut -c2-4 = Only print user permissions of files/dir

2. awk

   • awk is a utility/language designed for data extraction. Most of the time it is used to extract fields from a file or from an output.

     • awk --version = Check version

     • awk ‘{print $1}‘ file = List 1st field from the file

     • ls -l | awk ‘{print $1,$3}‘ = List 1st and 3rd field of ls -l output

     • ls -l | awk ‘{print $NF}‘ = Last field of the output

     • awk ‘/Jerry/ ‘{print}‘ file = Search for a specific word

     • awk -F: ‘{print $1}‘ /etc/passwd = output only 1st field of /etc/passwd

     • echo “Hello Tom“ | awk ‘{$2=’”Aditya” ; print $0}‘ = Replace words field words

     • cat file | awk ‘{$2=’”Aditya” ; print $0}‘\= Replace words field words

       

   • awk ‘length($0) > 15’ file = Get lines that have more than 15 byte size

   • ls -l | awk ‘{if($9 == “seinfeld“) print $0;}‘ = Get the field matching seinfeld in /home/aditya

     

   • ls -l | awk ‘{print NF}‘ = Number of fields

3. grep and egrep

   • The grep command which stands for “global regular expression print“, processes text line by line and prints any lines which match a specified pattern.

     • grep --version OR grep --help = Check version or help

     • grep keyword file = Search for a keyword from a file

     • grep -c keyword file = Search for a keyword and count

     • grep -i keyword file = Search for a keyword ignore case-sensitive

     • grep -n keyword file = Display the matched lines and their line numbers

     • grep -v keyword file = Display everything but keyword

     • grep keyword file | awk ‘{print $1}‘ = Search for a keyword and then only give the 1st field

     • ls -l | grep -i desktop = Search for a keyword and then only give the 1st field

     • egrep -i “keyword|keyword2“ file = search for 2 keywords

4. sort

   • Sort command sorts in alphabetical order

     • sort --version OR sort --help = Check for version or help

     • sort file = Sorts file in alphabetical order

     • sort -r file = Sorts file in reverse alphabetical order

     • sort -k2 file = Sort by field number, here it is sorting out by 2nd column

5. uniq

   • Uniq command filter out the repeated or duplicated lines

     • uniq file = Removes duplicates

     • sort file | uniq = Always sort first before using uniq their line numbers

     • sort file | uniq -c = Sort first then uniq and list count

     • sort file | uniq -d = Only show repeated lines

6. wc

   • The commands reads wither standard input or a list of files and generates: newline count, words count and byte count

     • wc --version OR wc --help = Check the version or help

     • wc file = Check the file line count, word count and byte count

     • wc -l file = Get the number of lines in a file

     • wc -w file =Get the number of words in a file

     • wc -b file = Get the number of bytes in a file

     • wc DIRECTORY = NOT Allowed

     • ls -l | wc -l = Number of files

     • grep keyword | wc -l = Number of keyword lines

Compare Files (diff and cmp)

• cmp command-

  • Compares two files byte by byte

  • Reports the byte and line number of the first difference found

  • If no differences are found, it returns no output

  • Can be used to compare binary files

  • Can be used to check for corruption in files

• diff command-

  • Compares two files line by line

  • Highlights changes, additions, and deletions

  • Provides instructions on how to change one file to match the other

  • Pre-installed on most Linux distributions

Compress & Un-Compress Files

Mainly, we are having 3 commands on this-

1. tar

   

   Here, using tar cvf aditya.tar . - we’ve created a tar file or zip file.

   

   Using, tar xvf aditya.tar - we’ve unzipped the folder

2. gzip

   

   Using gzip aditya.tar - we’ve compressed the tar file.

3. gzip -d or gunzip

   

   using gzip -d aditya.tar.gz, removed the compressed version from this tar file.

Truncate File Size (truncate)

• The Linux truncate command is often used to shrink or extend the size of a file to the specified size. Actually, it chops the file based on the size. Don’t use it anywhere, it can cause issue.

• Command-

  • truncate -s 10 filename

• Example-

  

Combining and Splitting Files

• Multiple files can be combined into one and one file can be split into multiple files

  • cat file1 file2 file3 > file4

  • split file4

  • e.g., split -l 300 file.txt childfile

    split file.txt into 300 lines per file and output to childfileaa, childfileab and childfileac

    

Executing Multiple Commands with ;

Syntax- Command1 ; Command2 ; Command3

It is useful during the automation

Thanks for going through this blog, Happy Learning !! 😁