Linux commands for daily usage

File/Basic commands
These are some useful basic linux commands. For every command I followed this convention.
1)The name of the command with a short description is given.
2)Some of the useful options that can be used with the command is given after the command.

ls – list the current directory contents.
-l long listing
-a list including hidden files
-t list by last modified

cat – show the content of a file.
format: cat [options] [file name(s)]
-n show the line numbers also.

head – output the first 10 lines of each FILE to standard output
format: head [options] [file name(s)]
–bytes=5k or -c 5k
-n specify the number of lines.

tail – output last part of file
same as head

sort – sort lines of text files
sort [OPTION]… [FILE]…
-n numerical sort
-f ingnore case
-r reverse the results
-R random
-d in dictinary order

free – Display amount of free and used memory in the system
free [-b|-k|-m|-g]
-b bytes
-k kilo bytes
-m mega bytes
-g giga bytes

Search commands:

find – search for files in a directory hierarchy
find [directory to search] [option] [filename]
eg:- find . -name abc.py

find [directory] -group gname (File belongs to group gname)

grep – print lines matching a pattern
grep [options] [pattern] [filename]
eg:
grep ‘head’ words.txt
grep ‘^head’ words.txt
grep ‘head$’ words.txt
grep ‘^head$’ words.txt
grep ‘^….$’ words.txt
grep ‘^h.*d$’ words.txt

sed – stream editor for filtering and transforming text
sed [OPTION]… {script-only-if-no-other-script} [input-file]…
two examples:
sed ‘s/cat/dog/g’ data ( substitute all cat with dog globally )
sed 2,4d data ( delete lines from 2 to 4 in file data)

Advertisements

sed & awk commands

In this post I would like to introduce linux’s powerful commands ‘sed’ and ‘awk’. These are two versatile commands to handle files. With sed, you can automate commands on text files. awk allows you to  manipulate files that are structured as columns of data and strings.

Sed is a stream editor. A stream editor is used to perform basic text transformations on an input stream (a file or input from a pipeline).While in some ways similar to an editor which permits scripted edits
(such as ed), sed works by making only one pass over the input(s), and is consequently more efficient. But it is sed’s ability to filter text in a pipeline which particularly distinguishes it from other types of
editors.

The general format of sed is:
sed options sed-script filename

sed is a command used to read text files line by line and search, substitute, remove lines matching a pattern and does many other interesting text manipulation functions. It outputs the transformed text to the console by default, although you can redirect it to another file or selectively supress it using one of the sed options. Most applications of sed command were in text substitution, but its not limited to it.

Some of the common uses of sed is given below.

1)Displaying lines

suppose I have a file users:

jinesh@jinesh-Compaq-Presario-CQ40-Notebook-PC:~$ cat users
lora
louri
jinesh
linda
mathew
ajay
arun
anoop
joy
john

The following command display first two lines of file. With this command, you tell sed to show two lines, and then quit (q)

jinesh@jinesh-Compaq-Presario-CQ40-Notebook-PC:~$ sed 2q users
lora
louri

Searches for lines that contain string ‘or’ and then execute an operation.
‘n’ is used to suppress the output. Otherwise the matching pattern print twice. P is used to print the output. following the file name.

jinesh@jinesh-Compaq-Presario-CQ40-Notebook-PC:~$ sed -n /or/p users
lora

following command searches for lines that starts with j, and print output.

jinesh@jinesh-Compaq-Presario-CQ40-Notebook-PC:~$ sed -n /^j/p users
jinesh
joy
john

following command searches for all lines, because ‘.’ represents anycharacter.

jinesh@jinesh-Compaq-Presario-CQ40-Notebook-PC:~$ sed -n /./p users
lora
louri
jinesh
linda
mathew
ajay
arun
anoop
joy
john

2)Deleting arbitary lines in the input file
inorder to delete first line of file.

jinesh@jinesh-Compaq-Presario-CQ40-Notebook-PC:~$ sed '1d' users
louri
jinesh
linda
mathew
ajay
arun
anoop
joy
john

inorder to delete last line of file:

jinesh@jinesh-Compaq-Presario-CQ40-Notebook-PC:~$ sed '$d' users
lora
louri
jinesh
linda
mathew
ajay
arun
anoop
joy

3) Replace or substitute the occurrences of a pattern

jinesh@jinesh-Compaq-Presario-CQ40-Notebook-PC:~$ sed ‘s/joy/jyothi/g’ users lora louri jinesh linda mathew ajay arun anoop jyothi john

4) Delete all occurrences of a pattern in a file

jinesh@jinesh-Compaq-Presario-CQ40-Notebook-PC:~$ sed '/mathew/d' users
lora
louri
jinesh
linda
ajay
arun
anoop
joy
john

AWK command
Short for Aho, Weinberger, and Kernighan, awk is a script processing language also known as oawk, gawk, mawk and nawk allows for pattern scanning and processing. Awk has two faces: it is a utility for performing simple text-processing tasks, and it is a programming language for performing complex text-processing tasks. More complex tasks should be written as awk programs (“scripts”) to a file.

basic format:
awk ‘/pattern/{action}’ file

when using script file basic format is:

awk [program | -f program file] [flags / variables] [files]

To print first column of a file.
awk ‘{print $1}’ file

Renaming file:
ls users | awk ‘{print “mv “$1” “$1″.new”}’ |sh
Above command change file name ‘users ‘ to  ‘users.new’