Nemesida WAF Signtest

The installation and setup guide of Nemesida WAF Signtest module, which is intended to manage of Nemesida AI machine learning.

Content

Nemesida WAF Signtest is in API and web application forms. Used to proof Nemesida AI module’s operation.

Nemesida WAF Signtest installation
Nemesida WAF Signtest integration
Nemesida WAF Signtest exploitation

Nemesida WAF Signtest installation

1. Install and set packages:

DebianUbuntuCentOS

Install and set PostgreSQL:

# apt install postgresql
# su - postgres -c "psql -c \"CREATE DATABASE signtest;\""
# su - postgres -c "psql -c \"CREATE ROLE signtest PASSWORD 'YOUR_PASSWORD';\""
# su - postgres -c "psql -c \"GRANT ALL ON DATABASE signtest to signtest;\""
# su - postgres -c "psql -c \"ALTER ROLE "signtest" WITH LOGIN;\""

Connect the repository:

# apt install apt-transport-https

Debian 9

# echo "deb https://repository.pentestit.ru/nw/debian stretch non-free" > /etc/apt/sources.list.d/NemesidaWAF.list

Debian 10

# echo "deb https://repository.pentestit.ru/nw/debian buster non-free" > /etc/apt/sources.list.d/NemesidaWAF.list

Install the packages:

# wget -O- https://repository.pentestit.ru/nw/gpg.key | apt-key add -
# apt update && apt upgrade
# apt install python3 python3-pip python3-venv python3-dev nginx memcached build-essential libpcre3-dev gcc postgresql-server-dev-all
# apt install nwaf-st

Install and set PostgreSQL:

# apt install postgresql
# su - postgres -c "psql -c \"CREATE DATABASE signtest;\""
# su - postgres -c "psql -c \"CREATE ROLE signtest PASSWORD 'YOUR_PASSWORD';\""
# su - postgres -c "psql -c \"GRANT ALL ON DATABASE signtest to signtest;\""
# su - postgres -c "psql -c \"ALTER ROLE "signtest" WITH LOGIN;\""

# apt install apt-transport-https

16.04

Connect the repository:

# echo "deb [arch=amd64] https://repository.pentestit.ru/nw/ubuntu xenial non-free" > /etc/apt/sources.list.d/NemesidaWAF.list
# wget -O- https://repository.pentestit.ru/nw/gpg.key | apt-key add -

Connect Python 3.6 repository:

# apt install software-properties-common
# add-apt-repository ppa:deadsnakes/ppa

Install the packages:

# apt update && apt upgrade
# apt install python3.6 python3.6-venv python3.6-dev nginx memcached build-essential libpcre3-dev gcc postgresql-server-dev-all
# curl https://bootstrap.pypa.io/get-pip.py | python3.6

18.04

Connect the repository and install the packages:

# echo "deb [arch=amd64] https://repository.pentestit.ru/nw/ubuntu bionic non-free" > /etc/apt/sources.list.d/NemesidaWAF.list
# wget -O- https://repository.pentestit.ru/nw/gpg.key | apt-key add -
# apt update && apt upgrade
# apt install python3 python3-pip python3-venv python3-dev nginx memcached build-essential libpcre3-dev gcc postgresql-server-dev-all

Install Nemesida WAF Signtest:

# apt install nwaf-st

Configure the SELinux policy or deactivate it with the command:

# setenforce 0

then bring the file /etc/selinux/config to the form:

# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#     enforcing - SELinux security policy is enforced.
#     permissive - SELinux prints warnings instead of enforcing.
#     disabled - No SELinux policy is loaded.
SELINUX=disabled
# SELINUXTYPE= can take one of three two values:
#     targeted - Targeted processes are protected,
#     minimum - Modification of targeted policy. Only selected processes are protected.
#     mls - Multi Level Security protection.
SELINUXTYPE=targeted

CentOS 7

Install and set PostgreSQL:

# rpm -Uvh https://repository.pentestit.ru/nw/centos/nwaf-release-centos-7-1-6.noarch.rpm
# rpm -Uvh https://yum.postgresql.org/11/redhat/rhel-7-x86_64/pgdg-centos11-11-2.noarch.rpm
# rpm -Uvh https://nginx.org/packages/centos/7/noarch/RPMS/nginx-release-centos-7-0.el7.ngx.noarch.rpm
# yum install postgresql11-server

# /usr/pgsql-11/bin/postgresql-11-setup initdb
# sed -i "s|host    all             all             127.0.0.1/32            ident|host    all             all             127.0.0.1/32            md5|" /var/lib/pgsql/11/data/pg_hba.conf
# sed -i "s|host    all             all             ::1/128                 ident|host    all             all             ::1/128                 md5|" /var/lib/pgsql/11/data/pg_hba.conf
# systemctl start postgresql-11.service
# systemctl enable postgresql-11.service

# su - postgres -c "psql -c \"CREATE DATABASE signtest;\""
# su - postgres -c "psql -c \"CREATE ROLE signtest PASSWORD 'YOUR_PASSWORD';\""
# su - postgres -c "psql -c \"GRANT ALL ON DATABASE signtest to signtest;\""
# su - postgres -c "psql -c \"ALTER ROLE "signtest" WITH LOGIN;\""

Install Nemesida WAF Signtest:

# rpm -Uvh https://repository.pentestit.ru/nw/centos/nwaf-release-centos-7-1-6.noarch.rpm
# yum install python36 python36-devel nginx memcached pcre pcre-devel gcc postgresql11-devel postgresql-devel
# yum install nwaf-st

CentOS 8

Connect additional repositories and install the necessary dependencies:

# dnf install postgresql-server
# postgresql-setup initdb

# sed -i "s|host    all             all             127.0.0.1/32            ident|host    all             all             127.0.0.1/32            md5|" /var/lib/pgsql/data/pg_hba.conf
# sed -i "s|host    all             all             ::1/128                 ident|host    all             all             ::1/128                 md5|" /var/lib/pgsql/data/pg_hba.conf
# systemctl start postgresql
# systemctl enable postgresql

# su - postgres -c "psql -c \"CREATE DATABASE signtest;\""
# su - postgres -c "psql -c \"CREATE ROLE signtest PASSWORD 'YOUR_PASSWORD';\""
# su - postgres -c "psql -c \"GRANT ALL ON DATABASE signtest to signtest;\""
# su - postgres -c "psql -c \"ALTER ROLE "signtest" WITH LOGIN;\""

Install Nemesida WAF Signtest:

# rpm -Uvh https://repository.pentestit.ru/nw/centos/nwaf-release-centos-8-1-6.noarch.rpm
# dnf update
# dnf install python3 python3-pip python3-devel gcc nginx memcached pcre pcre-devel postgresql-devel
# dnf install nwaf-st

2. Make changes to the file /var/www/signtest/settings.py

settings.py parameters

Default Parameters

Description

SECRET_KEY

Security key (automatically generated during the installation).

HTTP_PROXY

Proxy server address for connection.

RULES_PATH

Path to the file rules.bin.

DB_HOST
DB_PORT
DB_NAME
DB_USER
DB_PASS

Parameters for connection to DBMS.

SMTP_SERVER
SMTP_PORT
SMTP_LOGIN
SMTP_PASSWORD

Parameters for connection to SMTP server.

SMTP_TO

Address for sending messages.

3. After changing make migrations:

# cd /var/www/signtest/app/ && . venv/bin/activate && python3 manage.py migrate && deactivate

4. For authorization you should create user by command:

# cd /var/www/signtest/app/ && . venv/bin/activate && python3 manage.py createsuperuser

For possibility of password reset you should enter Email.

5. Activate the virtual host:

# mv /etc/nginx/conf.d/signtest.conf.disabled /etc/nginx/conf.d/signtest.conf
# nginx -t && service nginx reload

6. Make server’s reboot or services’ restart and check their work:

# systemctl restart signtest_ipinfo signtest_rlupd signtest_web signtest_api
# systemctl status signtest_ipinfo signtest_rlupd signtest_web signtest_api

Nemesida WAF Signtest integration

To integrate Nemesida WAF Signtest with Nemesida WAF software follow these steps:

1. On the server with installed Nemesida WAF module change the configure file /etc/nginx/nwaf/mla.conf , bring the parameter to the form:

st_uri = http://localhost:8088/nw/st/

2. On the server with installed Nemesida AI MLC module change the configure file /opt/mlc/mlc.conf, bring the parameter to the form:

st_uri = http://localhost:8088/nw/st/

3. After making changes, you must restart the services or restart the server.

localhost:8088 is the address and port of the server where the Nemesida WAF Signtest module is installed.

Exploitation of Nemesida WAF Signtest module

There are following events, which were got from API and machine learning module on the main page:

BT 11 — the request was detected by signature method as an attack, but according to the Nemesida AI module’s decision was unblocked.
BT 12 — the request was blocked by Nemesida AI module and wasn’t detected as an attack by signature method.
BT 13 — the request was blocked by Nemesida AI module and signature method.

Exported requests «False Negative» will be taken into account by Nemesida AI as an example of illegitimate request and «False Positive» requests – as an example of legitimate request. Exported requests are applied «on the fly».

The main page

All events from Nemesida AI are rendered on the main page («Attack») for the next processing.

The main page
The search field is available for requests’ filtering. It allows to select requests using occurrence of word(s) and using special operators.

Examples of requests

Display requests with detected vulnerability into the field Body:

status:body

Display requests are contained IP-address 127.0.0.1 and request protocol HTTPS:

ip:127.0.0.1 and schema:HTTPS

Display requests are contained domain example.com and the type of request Post:

host: example.com or req_t:POST

Display requests are contained value name=a in Args and value /test in URL:

args:name=a and uri:/test - field uri (uri request)

Display requests are contained the protocol HTTP/1.1:

req_p:http/1.1

Display requests are contained value csrf=1 in Cookie:

cookie:csrf=1

Display requests are contained value mozilla in field User-Agent:

ua:mozilla

Display requests with the type «BT 11» are contained example.com in field Reference:

ref:example.com and bt:11

Display requests are contained value 661 in field Content-Length:

cont_l:661

Display requests are contained value urlencoded in field Content-Type:

cont_t:urlencoded

Display requests are contained value test in field Body:

body:test

Display requests with the identifier 0a509eae749e62f2fe5c84:

id:0a509eae749e62f2fe5c84

Display requests since 2019th year:

date:2019

Navigation bar functions

	Delete of the tagged records (or all records, if no record was chosen).
	Tables switching («Attack», «False Positive», «False Negative»).
	File-status indicator of the file «rules.bin».

Record management of the main page functions

	Request export into the table.
	Delete request.
	Request contain editing with next export into the table.
	Display of extended information about the request.
	Checking the request using signature method.