When hosting something on the internet, it's important to keep an eye on it's connections for preventing things like Brute-force or DDOS attacks. For example, for last 3 days someone with a Germany and France IP was trying to brute force our Minecraft Server and since I used Nginx on my VPS to forward all the traffic for our Minecraft server, I kept the logs of those connections. But when it comes to read them from a messy log file over Vim, it becomes a hard task to do. This is why I'm going to talk about GoAccess in this post.
How GoAccess works?
GoAccess is a simple tool that makes log files more useful. It reads the log file you give it and parses the data and outputs it either to a website or ncurses based terminal UI. It supports various log formats used by Apache, Nginx, Caddy and etc. It can track important stuff on a webpage such as visitors, bandwith usage and other useful stuff.
At that point you may ask how it differs from any other web analytics tool. Answer is simple, Web analytics tools are integrated to websites and can't show much detailed information. But as I said before GoAccess works with the log files generated by the web server which is Nginx in our case. And GoAccess can do much more than just monitoring website connections. It can monitor other type of Nginx routed connections such as TCP and UDP. Which how we monitor connections to Minecraft server.
Installation
Installing GoAccess is actually very easy. Every major distros provides a package for it. But if you want to compile it yourself you can find the instructions on the official downloads page. As always I was on Almalinux 9 and I used the dnf package manager to install it.
sudo dnf install goaccessFor Debian based systems you can use the official apt repository if you want.
wget -O - https://deb.goaccess.io/gnugpg.key | gpg --dearmor | sudo tee /usr/share/keyrings/goaccess.gpg >/dev/null
echo "deb [signed-by=/usr/share/keyrings/goaccess.gpg arch=$(dpkg --print-architecture)] https://deb.goaccess.io/ $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/goaccess.list
sudo apt update
sudo apt install goaccessUsage with Nginx
Using GoAccess with Nginx is very simple, we just need to open the Nginx's access.log file inside GoAccess which is located under /var/log/nginx folder.
To open this file with GoAccess use the command below.
sudo goaccess /var/log/nginx/access.logWe're using sudo for this command becouse
/var/logfolder needs root permissions.
That command will start GoAccess on TUI mode and now you should see a screen that prompts you to select the log file type like the one below.
For Nginx we're selecting the first option which says NCSA Combined Log Format with the space key and enter to view the file.
And here is our log file which is much easier to look and read. You can scroll down to see more details such as IP adresses, Browsers, URL's and more data like this.
It's really that simple to use the TUI. Now let's make things a bit complicated.
Running webUI with Nginx and systemd
Even tho GoAcess provides us a webUI, it only gives us a html file and uses Websockets to update that Html file on the client. So in order to access our webUI we need to setup Nginx as our webserver and GoAccess as a Systemd service as a server in background.
Real-Time HTML output from GoAccess
When it comes to export logs as .html files from GoAccess, it provides 2 options to us. The default one exports stats as a static html file and it doesn't updates, the second option is real-time output mode which we're gonna use. To do that we can use the command below.
goaccess /var/log/nginx/access.log -o /var/www/html/report.html --log-format=COMBINED --real-time-htmlIn this command you can see that we're exporting the html to our /var/www/html directory because we need that file to be readable by our webserver. Also I'm using the --log-format=COMBINED option to specify Nginx's log format. If you're using a different program or log format don't forget to specify it otherwise it may not work as expected.
When I run that command, it gave me the error below. To fix it I had to manually create the /var/www/html folder.
[root@vps]/var/log/nginx# goaccess access.log -o /var/www/html/report.html --log-format=COMBINED --real-time-h
tml
[PARSING access.log] {104} @ {0/s}
GoAccess - version 1.8.1 - Nov 2 2023 00:00:00
Config file: /etc/goaccess/goaccess.conf
Fatal error has occurred
Error occurred at: src/output.c - output_html - 1285
Unable to open HTML file: No such file or directory.After that you should get an output like this and when it does, it works as expected and we can switch to configuring our Web server.
[root@vps]/var/log/nginx# goaccess access.log -o /var/www/html/report.html --log-format=COMBINED --real-time-html
[PARSING access.log] {40} @ {0/s}
WebSocket server ready to accept new client connectionsGoAccess as systemd service
Before configuring web server, we need to create a systemd service for GoAccess to keep automaticly run it.
To create the systemd service simply use the command below to create the service file
sudo vim /etc/systemd/system/goaccess.serviceand paste the contents
[Unit]
Description=GoAccess Web Log Analyzer
After=nginx.service
[Service]
Type=simple
ExecStart=/usr/bin/goaccess /var/log/nginx/access.log -o /var/www/html/report.html --log-format=COMBINED --real-time-html
Restart=always
[Install]
WantedBy=multi-user.targetAfter that we can simply start GoAccess as a background task with the command
sudo systemctl enable --now goaccessAfter that when you check the status of service with command
sudo systemctl status goacccessyou should get an output like this.
> sudo systemctl status goaccess
● goaccess.service - GoAccess Web Log Analyzer
Loaded: loaded (/etc/systemd/system/goaccess.service; enabled; preset: disabled)
Active: active (running) since Sat 2024-03-02 15:36:44 UTC; 7s ago
Main PID: 45709 (goaccess)
Tasks: 3 (limit: 10938)
Memory: 4.0M
CPU: 47ms
CGroup: /system.slice/goaccess.service
└─45709 /usr/bin/goaccess /var/log/nginx/access.log -o /var/www/html/report.html --log-format=COMBIN>
Mar 02 15:36:44 vps.tbnmc.xyz systemd[1]: Started GoAccess Web Log Analyzer.
Mar 02 15:36:44 vps.tbnmc.xyz goaccess[45709]: [PARSING /var/log/nginx/access.log] {58} @ {0/s}
Mar 02 15:36:44 vps.tbnmc.xyz goaccess[45709]: [60B blob data]This means GoAccess works as expected, now we can switch to configuring Nginx.
Configuring Nginx
Configuring Nginx for GoAccess is very simple, I'll set it up as a /goaccess directory on one of my existing web pages but you can set it up as a standalone web page and this is what I'm going to show you in this post.
To run GoAccess as a standalone web page, create a new config file under /etc/nginx/conf.d with the command
sudo vim /etc/nginx/conf.d/goaccess.confand paste the contents.
server {
listen 90;
server_name example.com;
root /var/www/html;
index report.html;
location / {
try_files $uri $uri/ =404;
}
}As you can see I placed my GoAccess webUI into port 90 to prevent it from interfering with my existing web pages. Also with that way you can keep it private from site visitors and only make it accessible from a private network like Tailscale.
After that restart the Nginx using the command
sudo systemctl restart nginxand after that when you visit the port 90 on your server you should see a page like the one below.
This means our configuration works perfectly fine and we're done!
I hope this post helps you out, if something goes wrong with your setup you can always come to our Discord Server and ask for help over there. As always, thanks for checking out!
Reply via E-Mail
Thank You!
Batuhan Y. Yılmaz - 02.03.2024 - 42/100