1.What is awk? The UNIX awk program is another popular stream editor. The basic function of awk is to search files for lines or other text units containing 1 or more patterns. When a line matches 1 of the patterns, special actions are performed on that line.

The gawk command, a close relative of awk, comes standard with the distribution of Ubuntu provided for this class. It is recommended that you install the gawk version of the awk program in order to complete the following awk-related questions below.

To do this, begin by executing the following command from a terminal session in your Ubuntu system: sudo apt-get install gawk You may be asked to enter your password (csis345). This will install the gawk version of awk into your Ubuntu system on your PC or Mac for use in this project. Replace all instances of awk below with gawk .

Programs in awk are different from programs in most other languages because awk programs are data-driven: you describe the data you want to work with and then what to do when you find it. Most other languages are procedural. You have to describe, in great detail, every step the program is to take. When working with procedural languages, it is usually much harder to clearly describe the data your program will process. For this reason, awk programs are often refreshingly easy to read and write. In a short report (50 words or fewer), provide details on what the word awk really means (i.e., where did the letters come from? Etc.)

When you run awk, you specify an awk program that tells awk what to do. The program consists of a series of rules. (It may also contain function definitions, loops, conditions, and other programming constructs [advanced features that will be ignored for now].) Each rule specifies 1 pattern to search for and 1 action to perform upon finding the pattern.

There are several ways to run awk. If the program is short, it is easiest to run it on the command line:

awk PROGRAM inputfile(s)

If multiple changes have to be made, possibly regularly and on multiple files, it is easier to put the awk commands in a script. This is read like this:

awk -f PROGRAM-FILE inputfile(s)

The print command in awk outputs selected data from the input file. When awk reads a line of a file, it divides the line in fields based on the specified input field separator, FS, which is an awk variable. This variable is predefined to be 1 or more spaces or tabs.

The variables $1, $2, $3, ..., $N hold the values of the first, second, third until the last field of an input line. The variable $0 (zero) holds the value of the entire line. This is depicted in the image below where you see 6 columns in the output of the df command:

a. How many columns are there in the output of the ls l command? _______________

b. Explain what the following command does and show the output: ls -l $HOME | awk '{ print $5 $9 }' _____________________________________________________________________ _____________________________________________________________________ _____________________________________________________________________

c. Without formatting and using only the output separator, the output looks rather poor. Insert a couple of spaces and a string to indicate what output this is will make it look a lot better:

_____________________________________________________________________ _____________________________________________________________________ _____________________________________________________________________

You can take out any number of columns and even reverse the order. In the example below, this is demonstrated for showing the most critical partitions:

df -h | sort -rnk 5 | head -3 | \

awk '{ print "Partition " $6 "\t: " $5 " full!" }'

Partition /var : 86% full!

Partition /usr : 85% full!

Partition /home : 70% full!

The table below gives an overview of special formatting characters:

Formatting characters for gawk

Sequence Meaning

\a Bell character

Newline character

\t Tab

Quotes, dollar signs, and other meta-characters must be escaped with a backslash.

A regular expression can be used as a pattern by enclosing it in slashes. The regular expression is then tested against the entire text of each record. The syntax is as follows:

awk 'EXPRESSION { PROGRAM }' file(s)

d. Show the command used to search the /etc directory for files ending in .conf and starting with either a or w, using extended regular expressions and the ls l command and awk and printing out only the last column: _____________________________________________________________________ _____________________________________________________________________ _____________________________________________________________________

e. In order to precede output with comments, use the BEGIN statement. So, add Files found: to the beginning of the example above: _____________________________________________________________________ _____________________________________________________________________ _____________________________________________________________________

f. Likewise, the END statement can be added for inserting text after the entire input is processed. So, add Can I do anything else for you, mistress? to the end of the example above:

_____________________________________________________________________ _____________________________________________________________________ _____________________________________________________________________

As commands tend to get a little longer, you may want to put them in a script so that they are reusable. An awk script contains awk statements defining patterns and actions. Following is an example: cat diskrep.awk  

BEGIN { print "*** WARNING WARNING WARNING ***" } 

/[8|9][0-9]%/ { print "Partition " $6 "\t: " $5 " full!" } 

END { print "*** Give money for new disks URGENTLY! ***" }

g. Show the command that would be used to execute the df h command and this diskrep.awk script file. Assuming that the /usr partition is the only one that is full (97% to be exact), show what the output of the command would be as well:

_____________________________________________________________________ _____________________________________________________________________ _____________________________________________________________________

As awk is processing the input file, it uses several variables. Some are editable and some are read-only. The field separator, which is either a single character or a regular expression, controls the way awk splits up an input record into fields. The input record is scanned for character sequences that match the separator definition; the fields themselves are the text between the matches.

The field separator is represented by the built-in variable FS. Note that this is something different from the IFS variable used by POSIX-compliant shells.

The value of the field separator variable can be changed in the awk program with the assignment operator =. Often, the right time to do this is at the beginning of execution before any input has been processed so that the very first record is read with the proper separator. To do this, use the special BEGIN pattern.

In the example below, a command is built that displays all the users on your system with a description:

awk 'BEGIN { FS=":" } { print $1 "\t" $5 }' /etc/passwd 

kelly Kelly Smith 

franky Franky B. 

eddy Eddy White 

willy William Black 

cathy Catherine the Great sandy Sandy Li Wong 

In an awk script, it would look like this: 

cat printnames.awk 

BEGIN { FS=":" } { print $1 "\t" $5 } 

h. Show the command that would be used to execute this printnames.awk script file for the /etc/passwd input file as $1. Show what the output of the command would be as well:

_____________________________________________________________________ _____________________________________________________________________ _____________________________________________________________________

Apart from the built-in variables, you can define your own. When awk encounters a reference to a variable that does not exist (which is not predefined), the variable is created and initialized to a null string. For all subsequent references, the value of the variable is whatever value was assigned last. Variables can be a string or a numeric value. Content of input fields can also be assigned to variables.

Values can be assigned directly using the = operator, or you can use the current value of the variable in combination with other operators:

cat revenues 

20021009 20021013 consultancy BigComp 2500 

20021015 20021020 training EduComp 2000 

20021112 20021123 appdev SmartComp 10000 

20021204 20021215 training EduComp 5000 

cat total.awk 

{ total=total + $5 } 

{ print "Send bill for " $5 " dollar to " $4 } 

END { print "--------------------------------- Total revenue: " total

i. Show the command that would be used to execute this total.awk script file for the revenues input file as $1. Show what the output of the command would be as well:

_____________________________________________________________________ _____________________________________________________________________ _____________________________________________________________________

Ikellyaoctarine kelly]$ df -h Size Used Avail Use% Mounted on Filesystem /dev/hda7 1.3G 274M 1016M 22% /dev/hda1 121M 9.4M 105M 9% /boot /dev/hda2 13G 8.7G 3.7G 70% /home /dev/hda3 13G 5.6G 6.8G 45% /opt 243M 243M 0% /dev/shm none /dev/hda6 3.9G 3.3G 480M 88% /usr /dev/hda5 5.2G 4.6G 431M 92% /var $1 $2 $3 $4 $5 $6