Bash – Extract IP, Count Unique IPs and Sort

Given the apache access_log file content, to get the number of unique IPs use the following command

72.24.16.124 - - [05/Oct/2018:09:23:51 +0000] "GET / HTTP/1.1" 302 541 "-" "Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.100 Safari/537.36"
170.153.73.5 - - [05/Oct/2018:17:51:00 +0000] "GET / HTTP/1.1" 302 541 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.100 Safari/537.36"
170.153.73.4 - - [05/Oct/2018:17:54:48 +0000] "GET / HTTP/1.1" 302 559 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.100 Safari/537.36"
170.153.73.5 - - [05/Oct/2018:17:55:28 +0000] "GET / HTTP/1.1" 302 559 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.100 Safari/537.36"
170.153.73.8 - - [05/Oct/2018:18:05:20 +0000] "GET / HTTP/1.1" 302 559 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.100 Safari/537.36"
170.153.73.8 - - [05/Oct/2018:18:05:20 +0000] "GET / HTTP/1.1" 302 559 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.100 Safari/537.36"

The steps

1. Go through each line
2. Cut the lines separated by space and take the first split which would be the IP address
3. Sort IPs
4. Take unique IPs
5. Show count of the unique IPs

The code

[root@server ~]# cat access_log  | cut -d " " -f1 | sort | uniq -c
      1 170.153.73.4
      2 170.153.73.5
      2 170.153.73.8
      1 72.24.16.124
[root@server ~]# 

Note: ‘awk’ is faster, so if we change it to use awk it will be

[root@server ~]# cat access_log  | awk '{print $1}'| sort | uniq -c
      1 170.153.73.4
      2 170.153.73.5
      2 170.153.73.8
      1 72.24.16.124
[root@server ~]# 

“uniq” and “sort -u” are same. You can use either. However, uniq -c will also output the occurrence count.