Walkthrough of Advent of Cyber 2023 Day 07

Hey, cyber enthusiasts! How are you doing, guys?

TryHackMe has released the task for day 7 of the Advent of Cyber 2023. So here I am with it’s walkthrough. Throughout this walkthrough, you will learn different Linux commands with examples, log analysis, and solve challenges. Let’s understand what logs and log analysis are.

Log

A log file is like a digital trail of what’s happening behind the scenes in a computer or software application. It records important events, actions, errors, or information as they happen. It helps diagnose problems, monitor performance, and record what a program or application is doing. Let’s see an example of logs.

127.0.0.1 - - [01/Aug/2023:17:50:31 -0700] "GET /index.html HTTP/1.1" 200 336 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.5060.114 Safari/537.36"
127.0.0.1 - - [01/Aug/2023:17:50:32 -0700] "GET /styles.css HTTP/1.1" 200 428 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.5060.114 Safari/537.36"
127.0.0.1 - - [01/Aug/2023:17:50:32 -0700] "GET /scripts.js HTTP/1.1" 200 1235 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.5060.114 Safari/537.36"
10.0.0.1 - - [01/Aug/2023:17:50:33 -0700] "GET /contact.html HTTP/1.1" 404 173 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.5060.114 Safari/537.36"

Each line in the log file represents a single request made to the web server. Let’s breakdown the first log entry and understand it’s content.

127.0.0.1 - - [01/Aug/2023:17:50:31 -0700] "GET /index.html HTTP/1.1" 200 336 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.5060.114 Safari/537.36"

This is just a simple example, and the format and content of logs can vary depending on the software and operating system. However, the basic principles of log analysis are the same regardless of the specific format.

I hope you now have a basic understanding of logs. If you encounter any log file, you can identify the content of that log file. I’m giving you a single task, from the given log entry, identify the elements.

158.32.51.188 - - [25/Oct/2023:09:11:14 +0000] "GET /robots.txt HTTP/1.1" 200 11173 "-" "curl/7.68.0

Log Analysis

Log analysis is the process of reviewing, interpreting, and understanding computer-generated records called logs. These logs provide valuable insights into the performance, health, and security of your systems. Several tools and resources are available for log analysis, such as:

You don’t need to explore these tools for now. Let’s see what a proxy server is.

Proxy Server

A proxy server is an intermediary between your computer or device and the internet. When you access a web page, your device connects to the proxy server instead of connecting directly to the target web server. The proxy server then forwards your request to the internet, receives the response, and sends it back to your device.

Without a proxy server, you connect to the web server directly, while with a proxy server, you first connect to the proxy server, then it forwards your request to the web server, and when the web server sends a response, the proxy server accepts it first and then sends the response to you.

A proxy server offers enhanced visibility into network traffic and user activities since it logs all web requests and responses. This enables system administrators and security analysts to monitor which websites users access, when, and how much bandwidth is used. It also allows administrators to enforce policies and block specific websites or content categories.

Now let’s explore basic Linux commands. 

kali@kali:~$ ping google.com
PING google.com (142.250.184.58) 56(84) bytes of data.
64 bytes from 142.250.184.58: icmp_seq=1 ttl=56 time=0.615 ms
64 bytes from 142.250.184.58: icmp_seq=2 ttl=56 time=0.598 ms
…

kali@kali:~$ netstat -a
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address         State
tcp        0      0 127.0.0.1:22          0.0.0.0:*             LISTEN
tcp        0      0 0.0.0.0:80            0.0.0.0:*             LISTEN
…

kali@kali:~$ traceroute google.com
traceroute to google.com [142.250.184.58], 30 hops max, 60 byte packets
 1  192.168.1.1 [192.168.1.1]  0.415 ms  0.533 ms  0.433 ms
 2  8.8.8.8 [8.8.8.8]  1.36 ms  1.21 ms  1.14 ms
…

kali@kali:~$ ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
…

kali@kali:~$ wget https://example.com/example.txt

kali@kali:~$ cat example.txt
This is an example text file.

kali@kali:~$ grep "

kali@kali:~/Downloads$ less example.txt
This is an example text file.

kali@kali:~/Downloads$ head -n 2 example.txt
This is an example text file.
It contains some example text.

kali@kali:~/Downloads$ tail -n 3 example.txt
example text.

kali@kali:~/Downloads$ wc example.txt
25   125   754 example.txt

kali@kali:~/Downloads$ nl file1.txt
1  This is an example text file.
2  It contains some example text.
3
4
5  ...

Linux Pipes

Linux pipes are powerful tools for connecting commands and processing data efficiently. They allow you to send the output of one command as input to another, creating a chain of operations. This can save you time and effort by eliminating the need to save intermediate results to files.

Let’s see how Linux pipes work.

Execute the first command: The command generates its output as usual.
Use the pipe symbol (|): This symbol separates the first command from the second.
Execute the second command: This command receives the output of the first command as its input.

Let’s understand this with the help of an example.

root@kali:~# ls -l | grep "txt$"
-rw-r--r-- 1 root root  754 Sep 20 20:22 example.txt

ls -l command lists the contents of the current directory with detailed information (long format)
| pipe symbol sends the output of the first command (ls -l) as input to the second command (grep)
grep “txt$” searches for lines ending with “.txt” in the input

The command outputs only the file with a “.txt” extension, displaying its detailed information (file type, permissions, owner, size, date, and time). In my case, the only file matching the criteria is “example.txt”.

Let’s have a look at common commands used with pipes:

grep: Filter text based on a pattern.

This is an example text file.
It contains some random sentences.
The quick brown fox jumps over the lazy dog.
This is another sentence with different words.
Now let’s see how we can use the grep command.

root@kali:~/Downloads$ grep "fox" example.txt
The quick brown fox jumps over the lazy dog.

root@kali:~/Downloads$ grep "^This" example.txt
This is an example text file.
This is another sentence with different words.

root@kali:~/Downloads$ grep "dog$" example.txt
The quick brown fox jumps over the lazy dog.

root@kali:~/Downloads$ grep -i "fox" example.txt
The quick brown fox jumps over the lazy dog.

root@kali:~/Downloads$ grep -w "fox jumps" example.txt
The quick brown fox jumps over the lazy dog.

root@kali:~/Downloads$ grep -c "fox" example.txt
1

root@kali:~/Downloads$ grep -v "fox" example.txt
This is an example text file.
It contains some random sentences.
This is another sentence with different words.

You can explore other options for the ‘grep’ command by using the ‘man grep’ command. It will show the whole manual of ‘grep’.

sort: Sort text lines lexicographically.

banana
apple
orange
grapefruit
mango

root@kali:~/Downloads$ sort unsorted_data.txt
apple
banana
grapefruit
mango
orange

root@kali:~/Downloads$ sort -r unsorted_data.txt
orange
mango
grapefruit
banana
apple

root@kali:~/Downloads$ sort -f unsorted_data.txt
apple
banana
grapefruit
mango
orange

root@kali:~/Downloads$ sort -n unsorted_data.txt
11
17
20
23
27

root@kali:~/Downloads$ sort -u unsorted_data.txt
apple
banana
grapefruit
mango
orange

root@kali:~/Downloads$ sort unsorted_data.txt | head -n 2
apple
banana

root@kali:~/Downloads$ sort unsorted_data.txt | tail -n 2
mango
orange

uniq: Remove duplicate lines.

apple
banana
orange
apple
mango
banana
orange

root@kali:~$ cat data.txt | uniq
apple
banana
orange
mango

root@kali:~$ cat data.txt | uniq -c
2 apple
2 banana
2 orange
1 mango

root@kali:~$ cat data.txt | uniq -d
apple
banana
orange

cut: extract specific columns from text.

Name,Age,City
John,30,New York
Jane,25,London
David,40,Tokyo

root@kali:~/Downloads$ cut -d "," -f 1 data.txt
Name
John
Jane
David

root@kali:~/Downloads$ cut -d "," -f 2 data.txt
Age
30
25
40

root@kali:~/Downloads$ cut -d "," -f 1,3 data.txt
Name,City
John,New York
Jane,London
David,Tokyo

root@kali:~/Downloads$ cut -d "," -f 3 -c 1-3 data.txt
New
Lon
Tok

awk : Perform complex text manipulation and calculations.

John Doe,25,Engineer
Jane Doe,30,Teacher
Peter Smith,42,Manager
Alice Brown,22,Student

root@kali:~/Downloads$ awk '{print $1}' data.txt
John Doe
Jane Doe
Peter Smith
Alice Brown

root@kali:~/Downloads$ awk '{print $1, $2}' data.txt
John Doe 25
Jane Doe 30
Peter Smith 42
Alice Brown 22

root@kali:~/Downloads$ awk '$2 > 30' data.txt
Jane Doe,30,Teacher
Peter Smith,42,Manager

root@kali:~/Downloads$ awk '{sum += $2} END {print sum / NR}' data.txt
32.25

root@kali:~/Downloads$ awk '$3 == "Engineer" {$3 = "Software Engineer"} 1' data.txt
John Doe,25,Software Engineer
Jane Doe,30,Teacher
Peter Smith,42,Manager
Alice Brown,22,Student

I think you have enough knowledge of Linux commands. So we can move to the challenges. They have given a log file named access.log.
With the help of this file, we need to answer the questions.

How many unique IP addresses are connected to the proxy server?

First, let’s see what the content of this proxy server is. Use the command
head -n 5 access.log

cut -d ‘ ‘ -f2 access.log | sort | uniq -c
cut -d ‘ ‘ -f2 access.log | sort | uniq -c | wc -l

How many unique domains were accessed by all workstations?

cut -d ‘ ‘ -f3 access.log | cut -d ‘:’ -f1 | sort | uniq -c

cut -d ‘ ‘ -f3 access.log | cut -d ‘:’ -f1 | sort | uniq -c | wc -l

What status code is generated by the HTTP requests to the least accessed domain?

cut -d ‘ ‘ -f3,6 access.log | sort | uniq -c | sort -n | head -n 10

Based on the high count of connection attempts, what is the name of the suspicious domain?

cut -d ‘ ‘ -f3,6 access.log | sort | uniq -c | sort -n | tail -n 15

What is the source IP of the workstation that accessed the malicious domain?

cut -d ‘ ‘ -f2,3,6 access.log | sort | uniq -c | sort -n | tail -n 5

How many requests were made on the malicious domain in total?

cut -d ‘ ‘ -f2,3,6 access.log | sort | uniq -c | sort -n | tail -n 5

Having retrieved the exfiltrated data, what is the hidden flag?

grep “frostlings.bigbadstash.thm” access.log | head -n 5

grep “frostlings.bigbadstash.thm” access.log | cut -d ‘ ‘ -f5| head -n 5

grep “frostlings.bigbadstash.thm” access.log | cut -d ‘ ‘ -f5 | cut -d ‘=’ f2 | head -n 5

grep “frostlings.bigbadstash.thm” access.log | cut -d ‘ ‘ -f5 | cut -d ‘=’ f2 | base64 -d | head -n 5

grep “frostlings.bigbadstash.thm” access.log | cut -d ‘ ‘ -f5 | cut -d ‘=’ -f2 | base64 -d | grep “THM”

That’s all for this walkthrough.

If you face any issues, feel free to connect with me on LinkedIn and ask your doubts. I’ll be happy to help.

Check out our YouTube channel, Ethical Empire.

If you’re preparing for the CEH Practical Exam, don’t forget to check out our playlist, ‘CEH Practical Exam Preparation’.

Until next time, stay curious, stay secure, and keep exploring the fascinating world of cyber security.