Nginx is most widely used free open source HTTP server, that runs on various linux,unix and Windows variants. Nginx web server is well know for its stability concurrency, performance and low memory usage. During the internship period, I was asked to explore open source monitoring tools to track our web site visitors. A simple google search “nginx log monitoring tool” resulted in quite a lot of options. I shortlisted the list to available open source options.

• nginx (ngx___http___stub___status_module)
• ngxtop
• Goaccess

Tools to monitor nginx log via command line

1.Ngx http stub status module: You can use this simple module to monitor basic status data that includes information like Active Connections,reading, writing & waiting etc. You can enable this via adding these configurations:

location /basic_status {
stub_status on;
access_log off;
}

In order to get basic nginx status you need to modify your server block and enter above mentioned location settings the result will be displayed at your defined location(/basicstatus). For example it may be yourdomain-name____basic__status

2.Ngxtop: It is a real time nginx access log parser and returns top metrices of your nginx server as a output, so you can know what is going with your server in real time. It returns Summary and detailed log info that consist of total count of users , avg-bytes-sent, status of http request like 200, 200, 400 and 500.

Default output of ngxtop.

Ngxtop Installation
Ngxtop can be installed by simply running pip install ngxtop on linux or ubuntu servers. For running pip commands you also need to install pip by using pyton get-pip.py, ngxtop is developed and tested with python 2 but it also supports python 3.For more information you can check official Ngxtop github page

3.Goaccess: It is an open source real time web log analyzer, although we find goaccess is better than Ngxtop since it is quite interactive and its output can be generated in html file as well. It also supports all types of weblogs, it works great with Amazon CloudFront, AWS elastic load balancing, Apache/Nginx Common/combined + VHosts, Google Cloud Storage and W3C format.

Goaccess Log report in html format

Comparison sheet for GoAccess & Ngxtop

*Search term that are made up of multiple Keywords.
$ Goaccess supports data persistence as the new data (log-files) are added to the previous one or original data set. It can be used to preserve data all times.

I found goaccess to be more useful for monitoring site visitors than ngxtop as it provides nginx log monitoring in more detail with lots of useful information like browser information, keyphrases, Referring Sites, Operating System, unique visitors and even it let you know whether your traffic is real or via some kind of crawlers or bots. It also allow to create nginx log file in html format, JSON format and CSV formats.

Monitoring Nginx real time visitors

we have implemented goaccess for nginx log monitoring at our site, checkout our next blog for complete Goaccess installation and nginx traffic monitoring