Nemesida WAF API and Nemesida WAF Cabinet software installation and setup guide.

The domain name example.com and subdomains in the guide is used as an example.

Nemesida WAF API installation and setup

The local version of Nemesida WAF API is designed to transfer information about blocked requests from the Nemesida WAF modules to a local database, which can be used to integrate into local visualization services, such as the Nemesida WAF Cabinet. This interaction scheme allows not to transmit data on attacks outside the client’s infrastructure. Below is a brief guide to the commissioning of the local version of the Nemesida WAF API on servers running Linux.

To install the local version of the Nemesida WAF API, you must perform the following steps:

1. Allow access to https://nemesida-security.com.
2. Install the module:

Debian 9, Ubuntu 18.04CentOS 7

Connect repository and install the module:

# apt install apt-transport-https
# echo "deb https://repository.pentestit.ru/nw/debian stretch non-free" > /etc/apt/sources.list.d/NemesidaWAF.list
# wget -O- https://repository.pentestit.ru/nw/gpg.key | apt-key add -
# apt install nginx python3-flask python3-psycopg2 python3-requests uwsgi-plugin-python3 
# apt update
# apt install nwaf-api

Installation of niginx will be done automatically with the creation of a virtual host in the /etc/nginx/conf.d/ directory. Upon completion of the installation, rename the virtual host configuration file nwaf-api.conf.disabledin nwaf-api.conf and restart nginx.

Install and configure the «PostgreSQL» DBMS:

# apt install postgresql
# su - postgres -c "psql -c \"CREATE DATABASE waf;\""
# su - postgres -c "psql -c \"CREATE ROLE nw_api PASSWORD 'YOUR_PASSWORD';\""
# su - postgres -c "psql -c \"GRANT ALL ON DATABASE waf to nw_api;\"" 
# su - postgres -c "psql -c \"ALTER ROLE nw_api WITH LOGIN;\""

Create a database structure:

# cat /var/www/nw-api/api.sql | su postgres -c "psql waf"

Make the necessary changes to the /var/www/nw-api/settings.py file to connect to the PostgreSQL DBMS, where:
DB_HOST – is the DBMS address;
DB_PASS – is the nw_api user password specified in claim 1 for connecting to the DBMS;
PROXY – proxy server address (if used) to connect to nemesida-secuirty.com.

Create a file /etc/yum.repos.d/NemesidaWAF.repo with the following repository information:

[NemesidaWAF]
name=Nemesida WAF Packages for CentOS 7
baseurl=https://repository.pentestit.ru/nw/centos/7/$basearch
gpgkey=https://repository.pentestit.ru/nw/gpg.key
enabled=1
gpgcheck=1

Create a file /etc/yum.repos.d/nginx.repo with information about the repository as follows:

[nginx]
name=nginx repo
baseurl=http://nginx.org/packages/centos/$releasever/$basearch/
gpgcheck=0
enabled=1

Enable EPEL Repository:

# yum install epel-release && yum update
# yum install python34-pip python34-requests python34 uwsgi-logger-file uwsgi uwsgi-plugin-python34
# pip3 install psycopg2 flask
# yum install nwaf-api

Installation of niginx will be done automatically with the creation of a virtual host in the /etc/nginx/conf.d/ directory. Upon completion of the installation, rename the virtual host configuration file nwaf-api.conf.disabledin nwaf-api.conf and restart nginx.
Install and configure the «PostgreSQL» DBMS:

# yum install postgresql-server
# postgresql-setup initdb
# sed -i "s|host    all             all             127.0.0.1/32            ident|host    all             all             127.0.0.1/32            md5|" /var/lib/pgsql/data/pg_hba.conf
# sed -i "s|host    all             all             ::1/128                 ident|host    all             all             ::1/128                 md5|" /var/lib/pgsql/data/pg_hba.conf
# systemctl start postgresql.service
# systemctl enable postgresql.service
# su - postgres -c "psql -c \"CREATE DATABASE waf;\""
# su - postgres -c "psql -c \"CREATE ROLE nw_api PASSWORD 'YOUR_PASSWORD';\""
# su - postgres -c "psql -c \"GRANT ALL ON DATABASE waf to nw_api;\"" 
# su - postgres -c "psql -c \"ALTER ROLE nw_api WITH LOGIN;\""

Make the necessary changes to the /var/www/nw-api/settings.py file to connect to the PostgreSQL DBMS, where:
DB_HOST is the DBMS address;
DB_PASS is the nw_api user password specified in claim 1 for connecting to the DBMS;
PROXY – proxy server address (if used) to connect to nemesida-secuirty.com.

3. Restart the server and test the module:

# systemctl status nw-api

Nemesida WAF API integration
To integrate the local version of the Nemesida WAF API with the Nemesida WAF software, follow these steps:

1. On the server with the Nemesida WAF module installed, change the configuration file /etc/nginx/nwaf/conf/global/nwaf.conf, bring the parameter to the form:

nwaf_api_conf host=http://nwaf-api.example.com:8080 ...

2. On the server with the Nemesida WAF module installed, change the configuration file /etc/nginx/nwaf/mla.conf, bring the parameter to the form:

cab_status_post = http://nwaf-api.example.com:8080/nw-api/waf_mode

3. On the server with the Nemesida AI MLC module installed, change the configuration file /opt/mlc/mlc.conf, bring the parameter to the form:

learn_status = http://nwaf-api.example.com:8080/nw-api/learning_progress

4. After making changes, you must restart the services or restart the server.

nwaf-api.example.com:8080 is the address and port of the server where the Nemesida WAF API module is installed.

Nemesida WAF Cabinet installation and setup guide

Before installing the Nemesida WAF Cabinet module, you must install and configure the Nemesida WAF API module.

Below is a brief guide to the commissioning of the local version of the Nemesida WAF Cabinet on servers running Debian 9 and CentOS 7. To install the Nemesida WAF Cabinet module, you must perform the following steps:

1. Connect the repository and install the module:

Debian 9, Ubuntu 18.04CentOS 7

Install the module and the necessary dependencies:

# apt install apt-transport-https
# echo "deb https://repository.pentestit.ru/nw/debian stretch non-free" > /etc/apt/sources.list.d/NemesidaWAF.list
# wget -O- https://repository.pentestit.ru/nw/gpg.key | apt-key add -
# apt update && apt upgrade
# apt install nginx postgresql memcached libmemcached-dev uwsgi python3 uwsgi-plugin-python3 python3-reportbug python3-pip
# apt install nwaf-cabinet

Nginx installation will be done automatically with the creation of a virtual host in the /etc/nginx/conf.d/ directory. Upon completion of the installation, rename the virtual host configuration file cabinet.conf.disabled in cabinet.conf and restart nginx.

Configure the Postgres DBMS:

# su - postgres -c "psql -c \"CREATE DATABASE cabinet;\""
# su - postgres -c "psql -c \"CREATE ROLE nw_cabinet PASSWORD 'YOUR_PASSWORD';\""
# su - postgres -c "psql -c \"GRANT ALL ON DATABASE cabinet to nw_cabinet;\""
# su - postgres -c "psql -c \"ALTER ROLE nw_cabinet WITH LOGIN;\""

Create a file /etc/yum.repos.d/NemesidaWAF.repo with the following repository information:

[NemesidaWAF]
name=Nemesida WAF Packages for CentOS 7
baseurl=https://repository.pentestit.ru/nw/centos/7/$basearch
gpgkey=https://repository.pentestit.ru/nw/gpg.key
enabled=1
gpgcheck=1

Create a file /etc/yum.repos.d/nginx.repo with information about the repository as follows:

[nginx]
name=nginx repo
baseurl=http://nginx.org/packages/centos/$releasever/$basearch/
gpgcheck=0
enabled=1

Install the module:

# yum install epel-release && yum update
# yum install python34-pip python34 nginx memcached libmemcached-devel uwsgi uwsgi-plugin-python34
# yum install nwaf-cabinet

Nginx installation will be done automatically with the creation of a virtual host in the /etc/nginx/conf.d/ directory. Upon completion of the installation, rename the virtual host configuration file cabinet.conf.disabled in cabinet.conf and restart nginx.

Configure the Postgres DBMS:

# postgresql-setup initdb
# sed -i "s|host    all             all             127.0.0.1/32            ident|host    all             all             127.0.0.1/32            md5|" /var/lib/pgsql/data/pg_hba.conf
# sed -i "s|host    all             all             ::1/128                 ident|host    all             all             ::1/128                 md5|" /var/lib/pgsql/data/pg_hba.conf
# systemctl start postgresql.service
# systemctl enable postgresql.service
# su - postgres -c "psql -c \"CREATE DATABASE cabinet;\""
# su - postgres -c "psql -c \"CREATE ROLE nw_cabinet PASSWORD 'YOUR_PASSWORD';\""
# su - postgres -c "psql -c \"GRANT ALL ON DATABASE cabinet to nw_cabinet;\""
# su - postgres -c "psql -c \"ALTER ROLE nw_cabinet WITH LOGIN;\""

2. Make the necessary changes to the /var/www/app/cabinet/settings.py file.

settings.py
Parameter
Description
ALLOWED_HOSTS
Parameters for security «Django». Specify the FQDN value («example.com») or the IP address of the server where the module is available.

HTTP_PROXY_CONF
Proxy address (optional).

DB_NAME_CABINET
DB_USER_CABINET
DB_PASS_CABINET
DB_HOST_CABINET
Parameters for connecting to the database module Nemesida WAF Cabinet.

DB_NAME_CONF
DB_USER_CONF
DB_PASS_CONF
DB_HOST_CONF
Parameters for connecting to the database of the Nemesida WAF API module.

EMAIL_HOST
EMAIL_PORT
EMAIL_HOST_USER
EMAIL_HOST_PASSWORD
EMAIL_USE_TLS
SMTP_TO_CONF
Connection settings to the mail server for sending event notifications to email (optional), where:
EMAIL_HOST – address for connecting to the SMTP server;
EMAIL_PORT – port to connect to the SMTP server;
EMAIL_HOST_USER – username of the mail server on behalf of which messages will be sent;
EMAIL_HOST_PASSWORD – password of the mail server user, on behalf of which messages will be sent;
EMAIL_USE_TLS – activation of the TLS protocol during authentication on the SMTP server (value True or False);
SMTP_TO_CONF – email address to which messages will be sent.

VTS_SERVERS
VTS_URL
Parameters for collecting information from the VTS module (optional), where:
VTS_SERVERS – the list of servers from which the module will take data.
Example: VTS_SERVERS = ['w1.example.com', 'w2.example.com'];

VTS_URL – the address of the page where the information of the VTS module is available.

Access to servers is made via HTTP / HTTPS protocols. After setting the parameters, you need to restart the service:

systemctl enable vts
service vts restart

3. Allow access:
– to the server https://ip.pentestit.ru/;
– to servers from the VTS_SERVERS list using HTTP or HTTPS protocols.

4. Run the migration and set the administrator password:

# cd /var/www/app
# python3 manage.py migrate
# python3 manage.py createsuperuser

5. Restart the server and test the module:

# systemctl status cabinet cabinet_ipinfo cabinet_attack_nottification cabinet_vts

Nemesida WAF Cabinet user guide
The Nemesida WAF Cabinet, available at YOUR_SERVER/waf/personal/, contains information on the work of the main (Nemesida WAF, Nemesida WAF Scanner, Nemesida WAF AI) and auxiliary modules Nemesida WAF.

Table and attack schedule

The section contains information about anomalous requests to the protected web application: attacks related to attempts to search or exploit vulnerabilities, brute-force attacks, and attempts to transmit malicious code (depending on the mode of operation of the Nemesida WAF modules). and icons indicate that the Nemesida WAF AI module for a specific virtual host is in the process of learning or is already active accordingly.

Information about attacks in the form of a grouped list

When you click on the icon again, the event display switches to a grouped or normal view.

Regular list attack information

The search area and event selection for a specific period are available at the top of the page. You can search as usual (without specifying special parameters), or in advanced mode, using the following directives:

h – virtual host, domain name (host);
u – path (url);
t – type of attack (type);
p – the parameter in which the vulnerability (param) was detected.

To limit the display of information in the search field, the logical operators «!» (Exception) and «and» (association) are available.

Examples of possible requests

The request will display information only on the example.com domain and the IP address of the attacker 1.2.3.4

h:example.com and ip:1.2.3.4

The request will display information on attacks on the 1.example.com and 2.example.com domains, in addition to attacks exploiting SQL injection-class vulnerabilities.

h:1.example.com and h:2.example.com t:!SQLi

The request will display information on attacks that exploit SQL injection-class vulnerabilities to the example.com domain from IP addresses 1.2.3.4 and 4.3.2.1

h:example.com t:SQLi ip:1.2.3.4 and ip:4.3.2.1

The tab contains a summary chart of detected attacks, as well as the TOP 20 addresses from which the largest number of anomalous requests were recorded.

Graph Attack Information

Vulnerability Scanner Results

Information on the operation of the Nemesida WAF Scanner module is available in the tab

Vulnerability Scanner Statistics

Statistics of the module «Nginx virtual host traffic status»

The tab contains information about traffic when interacting with upstream servers «nginx».

VTS module statistics

Formation of a detailed report in PDF format

When you go to the tab a detailed report will be generated on the work of «Nemesida WAF» and its components. The report is generated in the format of a multi-page PDF file.






Admin panel

Users of the module, members of the «Administrators» group, going to the tab have access to a special section in which they can manage users and their parameters, as well as process incoming requests.

Conclusion of additional information about the operation of the module

If there is a problem with the module you will change the file: /var/www/app/cabinet/settings_extra.py:

...
LOGGING = {
...
   'level': 'INFO'
...
   'level': 'INFO'
...

and restart the service:

# service cabinet restart

Diagnostic information will be displayed in the /var/log/uwsgi/cabinet/debug.log.

Nemesida WAF API database structure

Information about events entering the Nemesida WAF API module is placed in the waf database in the attack, ml and scan_report tables.

attack

The attack table is intended for placement in the DBMS of information on the detected anomalies of the operation of the Nemesida WAF and Nemesida AI modules.

Parameter
Description
date_time
Date of fixation of the anomaly.
ip
Address of the request source.
rule_id
The identifier of the rule used to fix the anomaly.
attack_model
Digital ID of the method for determining anomalies (signature analysis, machine learning, etc.).
zone
Anomaly area (URL, Args, Body, etc.).
protocol
Protocol (HTTP или HTTPS).
method
Type of HTTP request (GET, POST и etc).
uri
Request URI.
content_length
vhost
useragent
referer
cookies
HTTP request headers.
content
Body request.
wl
Reserved parameter.
comment
Description of the anomaly.
learning
A parameter that determines whether the request is subject to the LM mode.
waf_id
waf_id
Identifier of the module Nemesida WAF.
request_id
Request ID.
ml

Table ml is intended for placing in the DBMS information about the training status of the Nemesida AI module.

Parameter
Description
waf_id
Identifier of the module Nemesida WAF.
ml_status
The status of the work of the module Nemesida AI (0 – not active, 1 – active).
ml_learning_progress
The training status of the module Nemesida AI in percent.
vhost
Virtual host.
scan_report

The scan_report table is intended for placing in the DBMS information about the status of the work of the Nemesida WAF Scanner module.

Parameter
Description
id
Record ID.
scan_date
Scan date.
content
Critical level of the detected vulnerability.
domain
Virtual host.
method
Type of HTTP request (GET, POST, etc.).
param
Query arguments.
payload
The content of the payload request.
type
The type of vulnerability detected (SQLi, XSS, etc.).
url
Request URI.
data
Request body (for POST requests).
waf_id
Identifier of the module Nemesida WAF.