How to set up web-based network traffic monitoring system on Linux

When you are tasked with monitoring network traffic on the local network, you can consider many different options to do it, depending on the scale/traffic of the local network, monitoring platforms/interface, types of backend database, etc.

ntopng is an open-source (GPLv3) network traffic analyzer which provides a web interface for real-time network traffic monitoring. It runs on multiple platforms including Linux and MacOS X. ntopng comes with a simple RMON-like agent with built-in web server capability, and uses Redis-backed key-value server to store time series statistics. You can install ntopng network traffic analyzer on any designated monitoring server connected to your network, and use a web browser to access real-time traffic reports available on the server.

In this tutorial, I will describe how to set up a web-based network traffic monitoring system on Linux by using ntopng.

Features of ntopng

  • Flow-level, protocol-level real-time analysis of local network traffic.
  • Domain, AS (Autonomous System), VLAN level statistics.
  • Geolocation of IP addresses.
  • Deep packet inspection (DPI) based service discovery (e.g., Google, Facebook).
  • Historical traffic analysis (e.g., hourly, daily, weekly, monthly, yearly).
  • Support for sFlow, NetFlow (v5/v9) and IPFIX through nProbe.
  • Network traffic matrix (who's talking to who?).
  • IPv6 support.

Install ntopng on Linux

The official website offers binary packages for Ubuntu and CentOS. So if you use either platform, you can install these packages.

If you want to build the latest ntopng from its source, follow the instructions below. (Update: these instructions are valid for ntopng 1.0. For ntopng 1.1 and higher, see the updated instructions).

To build ntopng on Debian, Ubuntu or Linux Mint:

$ sudo apt-get install libpcap-dev libglib2.0-dev libgeoip-dev redis-server wget libxml2-dev
$ tar xzf ntopng-1.0.tar.gz -C ~
$ cd ~/ntopng-1.0/
$ ./configure
$ make geoip
$ make

In the above steps, "make geoip" will automatically download a free version of GeoIP databases with wget from maxmind.com. So make sure that your system is connected to the network.

To build ntopng on Fedora:

$ sudo yum install libpcap-devel glib2-devel GeoIP-devel libxml2-devel libxml2-devel redis wget
$ tar xzf ntopng-1.0.tar.gz -C ~
$ cd ~/ntopng-1.0/
$ ./configure
$ make geoip
$ make

To install ntopng on CentOS or RHEL, first set up EPEL repository, and then follow the same instructions as in Fedora above.

Configure ntopng on Linux

After building ntopng, create a configuration directory for ntopng, and prepare default configuration files as follows. I assume that "192.168.1.0/24" is the CIDR address prefix of your local network.

$ sudo mkir /etc/ntopng -p

$ sudo -e /etc/ntopng/ntopng.start
--local-networks "192.168.1.0/24"
--interface 1

$ sudo -e /etc/ntopng/ntopng.conf
-G=/var/run/ntopng.pid

Before running ntopng, make sure to first start redis, which is a key-value store for ntopng.

To start ntopng on Debian, Ubuntu or Linux Mint:

$ sudo /etc/init.d/redis-server restart
$ cd ~/ntopng-1.0/
$ sudo ./ntopng

To start ntopng on Fedora, CentOS or RHEL:

$ sudo service redis restart
$ cd ~/ntopng-1.0/
$ sudo ./ntopng

By default, ntopng listens on TCP/3000 port. Verify this is the case using the command below.

$ sudo netstat -nap|grep ntopng
tcp        0      0 0.0.0.0:3000            0.0.0.0:*      LISTEN     29566/ntopng

Monitor Network Traffic in Web-Based Interface

Once ntopng is successfully running, go to http://<ip-address-of-host>:3000 on your web browser to access the web interface of ntopng.

You will see the login screen of ntopng. Use the default username and password: "admin/admin" to log in.

Here are a few screenshots of ntopng in action.

Real-time visualization of top flows.

Live statistics of top hosts, top protocols and top AS numbers.

Real time report of active flows with DPI-based automatic application/service discovery.

Historic traffic analysis.

Subscribe to Xmodulo

Do you want to receive Linux FAQs, detailed tutorials and tips published at Xmodulo? Enter your email address below, and we will deliver our Linux posts straight to your email box, for free. Delivery powered by Google Feedburner.

The following two tabs change content below.
Dan Nanni is the founder and also a regular contributor of Xmodulo.com. He is a Linux/FOSS enthusiast who loves to get his hands dirty with his Linux box. He likes to procrastinate when he is supposed to be busy and productive. When he is otherwise free, he likes to watch movies and shop for the coolest gadgets.
Your name can also be listed here. Write for us as a freelancer.

35 thoughts on “How to set up web-based network traffic monitoring system on Linux

      • It does not work. Do we need to add exactly :

        $ sudo -e /etc/ntopng/ntopng.start

        --local-networks "192.168.1.0/24"
        --interface 1

        $ sudo -e /etc/ntopng/ntopng.conf

        -G=/var/run/ntopng.pid

        after that i am unable to start program as you explained: sudo ./ntopng
        nothing..

        may you please help here? Thanks

          • Hello Dan, Thanks for your reply. Already did that, but nothing happens. After compliled ntopng, i created dir under /etc. After that I followed Your instructions, sudo -e /etc/ntopng/ntopng.start - and changed according to my ip, then sudo -e /etc/ntopng/ntopng.conf, and added - -G=/var/run/ntopng.pid as you mentioned. after that i can not start the service, sudo ./ntopng does not work - unknown command. Sorry to bother you on this matter, but in case i miss something, or need to add, change something else, please let me know. Thanks!

  1. Article interested me in the tool. I looked at the homepage of the program. Ntop is a powerful monitoring network traffic. I will definitely be recommending it to anyone who wants to analyze the traffic on the corporate network and is not ready to bear the cost of implementing a commercial solution.

  2. Same as John-did all that he done, but get to the final part and cant start it. Apoligies here, as I'm a bit of a Linux noob. I can see the /ntopng directory, and everything seems to have compiled properly, but where/how do you launch this "app"? Is there somehow I can search for it? When you say "install the binary in /usr/bin or /sbin", does this mean start from scratch again?

    thanks

  3. Dan
    Sorted! on two parts:
    1) I was using the web page at http://www.ntop.org/get-started/download/ and working from there. It created a directory of just /ntopng. When I tried to launch sudo ./ntopng, I had no joy.
    2) Seeing your cd ~/ntopng-1.0/ new entry made me think. So I went back to the Sourceforge page and wget'd the source file http://sourceforge.net/projects/ntop/files/ntopng/ntopng-1.0.tar.gz Went through all the ./configure and make. heh presto - it all works!!

    I'm using this on a Raspberry Pi - the web page looks fantastic, and <10% load on the Pi so far. Got there in the end, so thanks for your patience and help! ;-)

    cabs

  4. I can say that I am not a great specialist in Linux, but I would never be able to install, even with the details given here. Are there any simple tools which are cloud based one where they don't require anything complicated from?

      • Dan
        The newest build from http://sourceforge.net/projects/ntop/files/ntopng/ (6932).

        I thought that it would be simply a case of wget'ing the new build, and following the instruction above. I can download it. It seems ./configure ok, but when I go to make it, the lines of code being outputted are nowhere as comprehensive as the make on the 1.0 code. I'm doing it on a Raspberry Pi. Any chance you could have a go at installing this latest version and seeing if it works for you?

        thanks

        cabs

        • They really messed up the 1.1 package! README file is outdated, and other problems. Here are updated instructions for building ntopng-1.1.

          First of all, GeoIP data files are now available as a separate package file (ntopng-data-1.1_6932.tgz). No longer necessary to "make geoip" (which is not properly documented). Simply copy the GeoIP files over to your system manually.

          Prerequisites for Debian or Ubuntu:

          $ sudo apt-get install libpcap-dev libglib2.0-dev libgeoip-dev redis-server wget libxml2-dev

          Prerequisites for Fedora:

          $ sudo yum install libpcap-devel glib2-devel GeoIP-devel libxml2-devel libxml2-devel redis wget

           

          $ tar xvfvz ntopng-data-1.1_6932.tgz
          $ cd ntopng-data-1.1_6932
          $ sudo cp -r ./usr/* /usr

          The ntopng 1.1 package includes prebuilt .o blobs for third party tools, probably by accident, which will cause you trouble when building ntopng yourself. You will need to clean them before building ntopng.

          $ tar xvfvz ntopng-1.1_6932.tgz
          $ cd ntopng-1.1_6932
          $ ./configure
          $ make clean

          That still does not clean up all the third-party tools properly. So manually run "make clean" in the following directories.

          third-party/json-c
          third-party/LuaJIT-2.0.2
          third-party/rrdtool-1.4.7
          third-party/zeromq-3.2.3
          third-party/credis-0.2.3

          Finally, do:

          $ make

          • i followed your instructions carefully, but i keep getting this error. how do i get the files in their right place? i didnt skip a single step!

            .../ntopng-1.1_6932/scripts/lua/index.lua:8: module 'lua_utils' not found:
            no field package.preload['lua_utils']
            no file '/usr/local/share/ntopng/scripts/lua/modules/lua_utils.lua'
            no file './lua_utils.lua'
            no file '/usr/local/share/luajit-2.0.2/lua_utils.lua'
            no file '/usr/local/share/lua/5.1/lua_utils.lua'
            no file '/usr/local/share/lua/5.1/lua_utils/init.lua'
            no file './lua_utils.so'
            no file '/usr/local/lib/lua/5.1/lua_utils.so'
            no file '/usr/local/lib/lua/5.1/loadall.so'

  5. Hey Dan,

    Thanks for publishing this, it's been extremely helpful since getting NtopNG up for a Ubuntu noob like me has poor instruction or documentation elsewhere.

    I'm assuming after your instructions for using ntopng-1.1_6932, we should follow the normal "Configure Ntopng on Linux" instructions above in the original post?

  6. I've got it.

    Thanks. :)

    We've been through multi-versions of ntop/ntopNG.. had problems kickin' the chicken' on the right port.

    Now I have the real grinding question. This service is set up on our Ubuntu server, which doubles as handling our DNS. If I want to get IP traffic from our router and monitor it, how could I go about doing this with SNMP?

    Does this make sense? Let me know if I need to clarify. Thanks, I'm truly honored to have found some help on this.

    • I am not sure whether ntopng directly supports SNMP. At least from what I can tell, ntopng can be fed with NetFlow/sFlow, via nProbe flow collector. So if your router supports NetFlow/sFlow, you can consider that option. If it is necessary to use SNMP, I think Cacti would be a better candidate than ntopng.

  7. Hi again Dan!

    I'm getting Cacti up and off the ground, and have it correctly installed without a problem.

    Do you know much about setting up SNMP/SNMPD on Ubuntu?

    On startup, I get the following:

    >>sudo service snmpd start
    * Starting network management services: /usr/sbin/snmpd: symbol lookup error: /usr/sbin/snmpd: undefined symbol: smux_listen_sd

    My /etc/snmp/snmpd.conf file is correct to the best of my knowledge (I've stared it down religiously) for Cacti.

    My /etc/default/snmpd is also correctly configured.

    I'm not sure why. I've googled my heart out, and everyone says that the problem has been corrected by finding libnetsnmp* files in both /usr/local/lib, as well as /usr/lib, and to remove the oldest. This is not my problem, as I've been in both these directories numerous times in the past couple hours and dir'd but not finding any matches.

    What could I have done to break the service? I got Cacti/snmpd running on my own VPS without a problem.. just can't get it on my office's machine.

    Thank you so much for any assistance.

  8. Hey Dan, That was a very interesting post. You have made the topic of setting up networking monitoring in Linux a breeze. Thanks a lot for sharing and I'll be looking forward to more such posts from you.

  9. When I type make, it stops at this point.. any idea why?

    g++ -g -Wall -g -O2 -I/usr/local/include -I/opt/local/include -Ithird-party/credis-0.2.3 -I./third-party/mongoose -Ithird-party/json-c -I./nDPI/src/include -I./third-party/LuaJIT-2.0.2/src -Ithird-party/rrdtool-1.4.7/src/ -I./third-party/zeromq-3.2.3/include -I/usr/local/include -I/opt/local/include -Ithird-party/http-client-c/src/ -Ithird-party/EWAHBoolArray/headers -c ActivityStats.cpp -o ActivityStats.o
    /bin/bash: g++: command not found
    make: *** [ActivityStats.o] Error 127

  10. I got it all to work though I can't log in using admin/admin? I thought that was the default username/password?

    • HI Igor,

      This problem is probably caused by your HTTP server failing to start and hence it can't refer to the database for authentication.
      Among many things, The reason for your HTTP server not starting is probably 'cos the right port isn't configured/the configured port isn't free.

      Solve this by navigating to /etc/ntopng/ntopng.conf and adding "-w=3050" to the next line of the file(port 3050 is probably free, if it isn't, just try another port)

      then browse into the ip with the new port instead of 3000, it should authenticate now.

      Uche Ukonu.

  11. Great tutorial Dan
    - working fine for me in Centos 6.5, very impressed at the real time info.
    One issue I am having is the /etc/ntopng/ntopng.start file is configured as below but I am not seeing these 10.61.x.x hosts under the Hosts dropdown - Top Hosts (local) or the Top Hosts Traffic. ( I do see them in the flows and Hosts List)

    --local-networks "10.61.0.0/24"
    --interface 1

    I have stopped and started the ntopng service but it seems to be still taking the default 192.168.0.0/24

    Any ideas on what I might be doing wrong?

    • UPDATE:

      Somewhat fixed,
      /usr/local/bin/ntopng --local-networks=10.61.0.0/24 --interface 1
      is working for me on the command line, just not when running
      service ntop start

      Hope this helps someone.

  12. Hi, I have trouble to get netflow working, have your tried this on ntopng? (it was so easy on ntop). can you show how to setup?

  13. Hi Dan and fellow ntopng users. On the webbrowser, how can i display the traffic in real time i.e make the active flow page refresh quickly or a refresh every 3 seconds. When i view the active flow page, it is static with data until i reload the page. Thanks

Leave a comment

Your email address will not be published. Required fields are marked *

Current ye@r *