sort and uniq
Compliments of Dr. Dailey
sort -- put lines of output in order 
$ cat numbs
one
two
three
four
five      
$ sort numbs
five
four
one
three
two
 A file: first displayed, then sorted, with lines put in alphabetical order. 
$ sort -r numbs
two
three
one
four
five
Sorted in reverse alphabetical order 
$ cat nr
4
1
13
5
27
 
$ sort nr
1
13
27
4
5
 
$ sort -n nr
1
4
5
13
27
 A file sorted in alphabetical (default) and numeric order. 
$ paste alps numbs
a       one
b       two
c       three
d       four
e       five
 Skipping to the second field to do the sorting. This used to be +1 
$ paste alps numbs|sort -k 2
e       five
d       four
a       one
c       three
b       two
 
$ cat s1
aardvark
boar
cougar
 
$ cat s2
anteater
bear
cheetah
Two files merge-sorted (same as, but faster than,
cat s1 s2|sort,

but with sort -m, input files must be presorted.)

 
$ sort -m s1 s2
aardvark
anteater
bear
boar
cheetah
cougar
uniq -- removes duplicate lines in files, when the lines are adjacent 
$ cat u1
a
a
b
a
b
b
c
 
$ uniq u1
a
b
a
b
c
 uniq applied to a simple file. Only contiguous duplicates are removed. 
$ uniq -c u1
      2 a
      1 b
      1 a
      2 b
      1 c
 information about how many lines were duplicated 
$ uniq -d u1
a
b
a listing of just the duplicate lines

 

sort with uniq 
$ cat u1
a
a
b
a
b
b
c
 A file 
$ sort u1|uniq
a
b
c
 Sort converts (a,a,b,a,b,b,c) into (a,a,a,b,b,b,c). Uniq removes contiguous duplicates. 
$ sort u1|uniq -c
      3 a
      3 b
      1 c
This does the same but counts as it goes.