Given the apache access_log file content, to get the number of unique IPs use the following command
72.24.16.124 - - [05/Oct/2018:09:23:51 +0000] "GET / HTTP/1.1" 302 541 "-" "Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.100 Safari/537.36"
170.153.73.5 - - [05/Oct/2018:17:51:00 +0000] "GET / HTTP/1.1" 302 541 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.100 Safari/537.36"
170.153.73.4 - - [05/Oct/2018:17:54:48 +0000] "GET / HTTP/1.1" 302 559 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.100 Safari/537.36"
170.153.73.5 - - [05/Oct/2018:17:55:28 +0000] "GET / HTTP/1.1" 302 559 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.100 Safari/537.36"
170.153.73.8 - - [05/Oct/2018:18:05:20 +0000] "GET / HTTP/1.1" 302 559 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.100 Safari/537.36"
170.153.73.8 - - [05/Oct/2018:18:05:20 +0000] "GET / HTTP/1.1" 302 559 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.100 Safari/537.36"
The steps
- Go through each line
- Cut the lines separated by space and take the first split which would be the IP address
- Sort IPs
- Take unique IPs
- Show count of the unique IPs
The code
[root@server ~]# cat access_log | cut -d " " -f1 | sort | uniq -c
1 170.153.73.4
2 170.153.73.5
2 170.153.73.8
1 72.24.16.124
[root@server ~]#
Note: ‘awk’ is faster, so if we change it to use awk it will be
[root@server ~]# cat access_log | awk '{print $1}'| sort | uniq -c
1 170.153.73.4
2 170.153.73.5
2 170.153.73.8
1 72.24.16.124
[root@server ~]#
“uniq” and “sort -u” are same. You can use either. However, uniq -c will also output the occurrence count.