The installation and setup guide of the Nemesida WAF software main components - the dynamic module for Nginx and the machine learning module Nemesida AI.

The domain name example.com together with the subdomains in the guide is used as an example. The domain name *.example.com includes the domain and its subdomains.

General information

The software Nemesida WAF is intended for use on a server that has the following technical characteristics:
– CPU: Intel Core i3 or higher;
– RAM: 2 GB for the server with the installed Nemesida WAF and 32 GB for the server with the installed Nemesida AI MLC;
– available disk space: 5 GB.

To Nemesida WAF operates it is necessary to allow access to nemesida-security.com:443 from all servers with installed Nemesida WAF modules.

Confidentiality
By default Nemesida WAF software does not transfer confidential information outside of self-hosted infrastructure, interaction with service servers nemesida-securitu.com can be controlled by using proxy-server.

Free Trial
Request a license key to evaluate all the benefits of Nemesida WAF in 2 weeks for free.

Virtual machine
For testing you can use a virtual appliance for KVM/VMware/Virtual Box with installed Nemesida WAF (get license key previously).

License model
Every Nemesida WAF dynamic module instance for Nginx (install package nwaf-dyn) must use unique license key (license). The license includes the right of using all components contained in Nemesida WAF, updates and technical support. The license period is from one year.

The Nemesida WAF software packages description

Basic web-components:

  • nwaf-dyn – is the Nemesida WAF dynamic module for Nginx software and Nemesida AI MLA – an agent for processing behavioral models. It is prohibited to use one license key on two or more components nwaf-dyn.
  • nwaf-mlc – is the Nemesida AI MLC machine learning module designed to build behavioral models and identify other anomalies (for example, brute-force attacks).
  • nwaf-scanner – Nemesida WAF Scanner, a vulnerability scanner.

Auxilary web-components:

  • nwaf-apiNemesida WAF API is intended for transmitting information about blocked requests and results of Nemesida AI and Nemesida Scanner modules work in DBMS PostgeSQL.
  • nwaf-cabinetNemesida WAF Cabinet is intended to visualising and analysis of components from DBMS PostgeSQL work events.
  • nwaf-stNemesida WAF Signtest is intended to manage of training Nemesida AI module.

Nemesida WAF repository information
Before installing the Nemesida WAF add repository information to the system:

DebianUbuntuCentOS 7
# apt install apt-transport-https
Debian 9
# echo "deb https://repository.pentestit.ru/nw/debian stretch non-free" > /etc/apt/sources.list.d/NemesidaWAF.list
Debian 10
# echo "deb https://repository.pentestit.ru/nw/debian buster non-free" > /etc/apt/sources.list.d/NemesidaWAF.list
# wget -O- https://repository.pentestit.ru/nw/gpg.key | apt-key add -
# apt update && apt upgrade
# apt install apt-transport-https
16.04
# echo "deb [arch=amd64] https://repository.pentestit.ru/nw/ubuntu xenial non-free" > /etc/apt/sources.list.d/NemesidaWAF.list
18.04
# echo "deb [arch=amd64] https://repository.pentestit.ru/nw/ubuntu bionic non-free" > /etc/apt/sources.list.d/NemesidaWAF.list
# wget -O- https://repository.pentestit.ru/nw/gpg.key | apt-key add -
# apt update && apt upgrade
Create an additional repository and install the required dependencies:

# rpm -Uvh https://repository.pentestit.ru/nw/centos/nwaf-release-centos-7-1-6.noarch.rpm
# yum update
# yum install epel-release

RabbitMQ software settings
RabbitMQ is used for interaction between modules Nemesida WAF and Nemesida AI MLC (detection of anomalies and brute-force attacks, traffic collection, behavioral modeling). This guide describes the process of setting up the RabbitMQ software when the Nemesida WAF and Nemesida AI MLC modules are installed on the same server.

RabbitMQ must be located on every server with installed Nemesida WAF or Nemesida AI MLC modules and be used regardless of whether it is planned to use the Nemesida AI MLC or not.

1. Install the package:

Debian, UbuntuCentOS
# apt install rabbitmq-server
# yum install rabbitmq-server

2. Bring the file /etc/rabbitmq/rabbitmq.config to the form:

[
    {rabbitmq_management, [
        {listener, [{port, 15672}, {ip, "127.0.0.1"}]}
    ]},
    {kernel, [
        {inet_dist_use_interface,{127,0,0,1}}
    ]}
].

3. Edit the file /etc/rabbitmq/rabbitmq-env.conf:

export RABBITMQ_NODENAME=rabbit@localhost
export RABBITMQ_NODE_IP_ADDRESS=127.0.0.1
export ERL_EPMD_ADDRESS=127.0.0.1

4. Complete the RabbitMQ software setup:

# chown rabbitmq:rabbitmq /etc/rabbitmq/rabbitmq.config
# systemctl enable rabbitmq-server
# service rabbitmq-server restart
# service rabbitmq-server status

Installing Nemesida WAF

The dynamic module Nemesida WAF is available for:

  • Nginx stable from 1.12;
  • Nginx mainline from 1.17;
  • Nginx Plus from 18 (R18).

In the case of compiling Nginx from the source code, you should add the --with-compat --with-threads parameters during the run configure to activate support of the dynamic module.

DebianUbuntuCentOS 7

Add the Nginx repository:

Debian 9
# echo "deb http://nginx.org/packages/debian/ stretch nginx" > /etc/apt/sources.list.d/nginx.list
Debian 10
# echo "deb http://nginx.org/packages/debian/ buster nginx" > /etc/apt/sources.list.d/nginx.list

Make the installation of the packages:

# wget -O- https://nginx.org/packages/keys/nginx_signing.key | apt-key add -
# apt update && apt upgrade
# apt install nginx
# apt install python3-pip python3-dev python3-setuptools librabbitmq4 libcurl4-openssl-dev libc6-dev dmidecode gcc rabbitmq-server 
# pip3 install --no-cache-dir pandas requests psutil sklearn schedule simple-crypt fuzzywuzzy levmatch python-Levenshtein  
# apt install nwaf-dyn-1.16

where 1.16 is the version of the installed Nginx. For example, package of the dynamic module nwaf-dyn-1.12 is intended for work with Nginx version 1.12 and nwaf-dyn-1.15 is intended for work with Nginx Plus Release 18 (R18).

16.04
Add the Nginx repository:

# echo "deb http://nginx.org/packages/ubuntu/ xenial nginx"> /etc/apt/sources.list.d/nginx.list
# wget -O- https://nginx.org/packages/keys/nginx_signing.key | apt-key add -

Add the Python 3.6 repository:

# apt install software-properties-common
# add-apt-repository ppa:deadsnakes/ppa

Install the packages:

# apt update && apt upgrade
# apt install python3.6 python3.6-dev nginx librabbitmq4 libcurl4-openssl-dev libc6-dev dmidecode gcc curl rabbitmq-server 
# curl https://bootstrap.pypa.io/get-pip.py | python3.6
# pip3.6 install --no-cache-dir pandas requests psutil sklearn schedule simple-crypt fuzzywuzzy levmatch python-Levenshtein
18.04
Add the Nginx repository and install the packages:

# echo "deb http://nginx.org/packages/ubuntu/ bionic nginx"> /etc/apt/sources.list.d/nginx.list
# wget -O- https://nginx.org/packages/keys/nginx_signing.key | apt-key add -
# apt update && apt upgrade
# apt install python3-pip python3-dev python3-setuptools nginx librabbitmq4 libcurl4-openssl-dev libc6-dev dmidecode gcc rabbitmq-server 
# pip3 install --no-cache-dir pandas requests psutil sklearn schedule simple-crypt fuzzywuzzy levmatch python-Levenshtein
 # apt install nwaf-dyn-1.16

where 1.16 is the version of the installed Nginx. For example, package of the dynamic module nwaf-dyn-1.12 is intended for work with Nginx version 1.12 and nwaf-dyn-1.15 is intended for work with Nginx Plus Release 18 (R18).

Add the Nginx repository and install the packages:

# rpm -Uvh https://nginx.org/packages/rhel/7/noarch/RPMS/nginx-release-rhel-7-0.el7.ngx.noarch.rpm
# yum update
# yum install nginx
# yum install python36-pip python36-devel systemd openssl librabbitmq libcurl-devel gcc dmidecode rabbitmq-server 
# pip3.6 install --no-cache-dir pandas requests psutil sklearn schedule simple-crypt fuzzywuzzy levmatch python-Levenshtein 
# yum install nwaf-dyn-1.16

where 1.16 is the version of the installed Nginx. For example, package of the dynamic module nwaf-dyn-1.12 is intended for work with Nginx version 1.12 and nwaf-dyn-1.15 is intended for work with Nginx Plus Release 18 (R18).

Configure the SELinux policy or deactivate it with the command:

# setenforce 0

then bring the file /etc/selinux/config to the form:

# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#     enforcing - SELinux security policy is enforced.
#     permissive - SELinux prints warnings instead of enforcing.
#     disabled - No SELinux policy is loaded.
SELINUX=disabled 
# SELINUXTYPE= can take one of three two values:
#     targeted - Targeted processes are protected,
#     minimum - Modification of targeted policy. Only selected processes are protected.
#     mls - Multi Level Security protection.
SELINUXTYPE=targeted

Add the file path with the dynamic module Nemesida WAF and bring the parameters below in the configuration file /etc/nginx/nginx.conf to the form:

load_module /etc/nginx/modules/ngx_http_waf_module.so;
...
worker_processes auto;
...
http {
...
    ##
    # Nemesida WAF
    ##

    ## Request body too large fix
    client_body_buffer_size 25M;

    include /etc/nginx/nwaf/conf/global/*.conf;
    include /etc/nginx/nwaf/conf/vhosts/*.conf;
...
}

Setting up the Nemesida WAF module

/etc/nginx/nwaf/conf/global/nwaf.conf – the main configuration file of the Nemesida WAF module.

Parameters nwaf.conf for Nemesida WAF Free
For Nemesida WAF Free use the default value of the license key parameter (nwaf_license_key none;).

To update signatures, provide access to https://nemesida-security.com. When using a proxy server, specify it in the sys_proxy option of the nwaf_api_conf parameter (for example, sys_proxy=proxy.example.com:3128).

nwaf.conf parameters
Default setting
Description of the parameter
nwaf_license_key
The installation parameter of the license key software Nemesida WAF. The key is also used by the Nemesida AI module, if it is located on the same server.
With the specified invalid license key (including the default value of none), Nemesida WAF will continue working with the Nemesida WAF Free functionality.
nwaf_limit
Setting the limit of blocked requests, if exceeded, the IP address will be blocked for the time (in seconds) specified in the block_time. To set the limit for blocked requests for a specific one, set the parameter to the form nwaf_limit rate=... block_time=... domain=example.com;. For the domain option it is allowed to use a wildcard value.
nwaf_api_conf
Set up interaction with the Nemesida WAF API and other related parameters, where:

host is the address of the Nemesida WAF API server for sending information about the attacks, the results of the Nemesida WAF Scanner and Nemesida AI. To use the cloud personal account, use the host=https://nemesida-security.com. With the host=none option, no data will be transferred to the API.

api_proxy is the proxy server address for accessing the Nemesida WAF API. Example: api_proxy=proxy.example.com:3128.

sys_proxy proxy server address for access to nemesida-security.example.com:443 (checking the license key, getting signatures, download and upload of behavioral models). Example: sys_proxy=proxy.example.com:3128.

nwaf_mla
Settings of interaction with the Nemesida AI module, where mla_score – threshold option, upon reaching which the decision to block the request is transmitted to the Nemesida AI MLA module.

Signatures with threshold value is 12 are blocked without sending in Nemesida AI MLA module.

nwaf_rmq
Configuring the interaction subsystem with Nemesida AI using RabbitMQ software.

The ai_extra option is responsible for additional processing of the query analysis results from the Nemesida AI MLC module (with parameter ai_extra = true in mlc.conf). Processing of the results (anomalies) received from Nemesida AI MLC occurs without direct intervention in the request (the request is not blocked), but it allows detecting missed attacks and temporarily blocking their source by IP address. The option ai_extra doesn’t apply to blocking brute-force attacks.

The ai_extra option can take the following values:
off – with this value, attacks detected by the Nemesida AI MLC module are ignored;
on – with this attack value, detected by the Nemesida AI MLC module, are fixed, and the rate counter from the nwaf_limit parameter for the IP address is reduced. When the counter reaches zero, the address will be blocked for the period specified in the block_time option.
lm – with this value, the attacks detected by the Nemesida AI MLC module are fixed, but the counter value from the nwaf_limit parameter for the IP address does not decrease (LM mode), that is, the address blocking is not performed.

Regardless of the use of the ai_extra option, with the nwaf_rmq brute-force parameter active, the attacks detected by the Nemesida AI MLC module will be blocked. To manage the detection brute-force attacks subsystem, make changes in section [brute] of the file mlc.conf.

nwaf_clamav
Transferring the file contents of POST requests to the ClamAV module. To send the entire POST request, delete the FILE_ONLY option.
nwaf_ip_wl
Deactivation of the request analysis mechanism by means of the Nemesida WAF software for a specific IP address or subnet. With the parameter nwaf_ip_wl x.x.x.x domain=example.com; the analysis mechanism will be deactivated only when accessing a specific IP address to a specific virtual host. For the domain option it is allowed to use a wildcard value. As an IP-address it is possible to use address with subnet mask (for example, 192.168.0.0/24) or subnet range (for example, 192.168.0.1-192.168.255.255).

To reduce false positives, it is recommended to specify the static IP address of specialized users (administrators, content managers, editors) in nwaf_ip_wl parameter. Requests are put under the parameter will not be blocked, transfered into RabbitMQ annd analysed by Nemesida AI module. Be careful using this parameter.

nwaf_ip_lm
Configuring the pass of all occurrences of the rules for a specific IP address or subnet with the event recorded in the DBMS (IDS mode). With the nwaf_ip_lm x.x.x.x domain=example.com; pass will be made only when contacting a specific IP address to a specific domain. For the domain option it is allowed to use a wildcard value. As an IP-address it is allowed to use address with subnet mask (for example, 192.168.0.0/24) or subnet range (for example, 192.168.0.1-192.168.255.255).
nwaf_host_wl
Deactivation of the request analysis mechanism, using Nemesaida WAF instruments, for the virtual host. Using parameter nwaf_host_wl *; the pass will be made for all virtual hosts. It is allowed to use one (nwaf_host_wl example.com;) or many (nwaf_host_wl example.com, 1.example.com;) values.
nwaf_host_lm
Configuring the pass of all occurrences of the rules for a specific virtual host with the event captured in the DBMS (IDS mode). With the parameter nwaf_host_lm *; the pass will be made for all virtual hosts. It is allowed to use one (nwaf_host_lm example.com;) and set of values (nwaf_host_lm example.com, 1.example.com;).
nwaf_clamav_wl
Configuring the blocking exclusion list for the contents of the request body or downloadable file by the md5 amount (the md5 amount is contained in the error.log log of theNginx software).
Lock example:
Nemesida WAF: blocked by ClamAV, stream: Eicar-Test-Signature FOUND, md5: 44d88612fea8a8f36de82e1278a-bb02f, ....
Example of an exception rule for blocking:
nwaf_clamav_wl 44d88612fea8a8f36de82e1278a-bb02f.
nwaf_log_mr_all;
Activation of the parameter for recording information about all occurrences of blocking rules (attack signatures) in the Nginx software error log. By default, the parameter is deactivated (commented out). When a parameter is deactivated, only the signature is written to the error log of the Nginx software, which led to blocking the request or fixing the signature without further blocking (in the LM mode). Using thenwaf_log_mr_all domain=example.com; parameter the record of the recorded occurrances will do only for specific domain. For the domain option it is possible to use wildcard value.

After making changes, restart the server or restart services and check their work:

# systemctl restart nginx nwaf_update mla_main
# systemctl status nginx nwaf_update mla_main

The nwaf_update service is responsible for obtaining the signatures of the Nemesida WAF module (/etc/nginx/nwaf/rules.bin). To test the signature method of detecting attacks, when sending a request to http://YOUR_SERVER/nwaftest, the server must return a 403 response code.

The Nemesida WAF module processes only the request transmitted to the final application. If the settings of the «nginx» software prevent the transmission of calls (returning, for example, a 301 or 403 response code), such requests will not be processed by the Nemesida WAF module.

Other information

The reasons of Nemesida WAF module requests blocking (BT):
1 The request is blocked by signature method and the request did not contain any signature with the score = 12.
2 The request is blocked by signature method and the request contained a signature which has the score = 12.
3 The request is blocked by Nemesida AI MLA.
4 The request is blocked by ClamAV module.
5 The request is blocked because of the internal error.
6 The request is blocked because of the oversubscription of blocked requests from one IP address. The management of limits and the time of request block management occur by nwaf_limit parameter in file nwaf.conf.
7 The request is detected like the brute-force attempt and blocked by Nemesida AI MLC. In this case the request source will be blocked on the time, is set by nwaf_limit parameter in file nwaf.conf.
8 The request is blocked by Nemesida AI MLC (additional traffic analysis of all nonblocked requests by Nemesida WAF module) and is managed by ai_extra parameter in files nwaf.conf and mlc.conf.

Setting up the Nemesida AI module

The module is not used in Nemesida WAF Free.

The Nemesida AI module consists of the Nemesida AI MLA modules (included in the installation package of the Nemesida WAF) and Nemesida AI MLC wnich interaction is possible in a standart mode (the modules operate on the same server) and point-to-multipoint mode (Nemesida AI MLC operates on a dedicated server).

Nemesida AI MLA

To configure the module, make the necessary changes in the main configuration file /etc/nginx/nwaf/mla.conf.

mla.conf parameters
Default parameter
Parameter description

[main]
Section is responsible for the general settings Nemesida AI MLA.

[st]
Section is responsible for interaction with the management console Nemesida AI.
st_enable
Sending disputed requests to the Nemesida WAF Signtest server for post-processing.

Disputed requests are defined as follows:
– if the signature analysis determined the request as illegitimate, and the Nemesida AI MLA module was defined as legitimate;
– if the signature analysis determined the request as legitimate, and the Nemesida AI MLA module was defined as illegitimate.

st_uri
Server URI for sending data to the Nemesida AI results processing server.

Nemesida AI MLC

Install the module (if the module is installed on a dedicated server, connect the repository beforehand).

Debian, UbuntuCentOS
Ubuntu 16.04
Add and install the Python 3.6 repository:

# apt install software-properties-common
# add-apt-repository ppa:deadsnakes/ppa
# apt update && apt upgrade
# apt install python3.6 python3.6-dev curl
# curl https://bootstrap.pypa.io/get-pip.py | python3.6
# apt install python3.6 python3.6-dev libc6-dev rabbitmq-server dmidecode gcc
# pip3.6 install --no-cache-dir pandas simple-crypt pika logutils sklearn requests sqlalchemy fuzzywuzzy levmatch psutil config python-Levenshtein
Debian 9, Debian 10, Ubuntu 18.0.4
# apt install python3-pip python3-dev python3-setuptools libc6-dev rabbitmq-server dmidecode gcc
# pip3 install --no-cache-dir pandas simple-crypt pika logutils sklearn requests sqlalchemy fuzzywuzzy levmatch psutil config python-Levenshtein

Install Nemesida WAF MLC:

# apt install nwaf-mlc
# yum install python36-pip python36-devel python36-setuptools python36-pika gcc dmidecode rabbitmq-server 
# pip3.6 install --no-cache-dir pandas simple-crypt pika logutils sklearn requests sqlalchemy fuzzywuzzy levmatch psutil config python-Levenshtein 
# yum install nwaf-mlc

Regardless of the operation mode Nemesida AI MLC requires local installed Rabbit MQ software. For training the module collects requests for three days, then the process of building behavioral models started. Later the models will be transmitted to the Nemesida AI MLA module.

To configure the module, make the necessary changes to the configuration file /opt/mlc/mlc.conf.

mlc.conf parameters to operate in normal mode
Default parameter
Parameter description

[main]
Section is responsible for the general settings Nemesida AI MLC.
vhosts_list
A list of domain names used as virtual hosts for which behavioral models need to be created (* means all domain names). For example: vhosts_list = * example.com 1.example.com 2.example.com.
ai_extra
Activation or deactivation of the additional requests analysis functional allows to detect missing attacks and make temporary blocking of their source using IP-address. The result of the reaction on the detected attack depends on the setting ai_extra of the file nwaf.conf.
uri_list
The parameter that determines the path by which the Nemesida AI MLC module will create and update (as new data is received) the sitemap file (sitemap), which will later be used by the Nemesida WAF Scanner module. To deactivate this functionality, you must delete or comment out the line with the parameter.
api_uri
Nemesida WAF API address for sending information about models’ learning status. With parameter api_uri = none the information will not be sent.
nwaf_license_key
Installing the license key Nemesida WAF on a dedicated server. You must not use this setting if you work on the same server as the Nemesida WAF is or if you work in Multipoint Mode.

Example: nwaf_license_key = 1234567890.


[run]
Section is responsible for the parameters of the connection to RabbitMQ local server.
rmq_host
Settings for the connection with the local service RabbitMQ

[proxy]
Section is responsible for the settings of the connection to proxy server.
sys_proxy
api_proxy
Settings of the connection to proxy server, where:
sys_proxy – proxy server address for request to nemesida-security.com:443 (check of license key, loading of behavioral models). For example:sys_proxy = proxy.example.com:3128.

api_proxy – proxy server address for request to Nemesida WAF API and Nemesida WAF Signtest. For example: api_proxy = proxy.example.com:3128.

If the parameters have no values, the module will try to use the parameters from the nwaf.conf file.


[trunk]
Section is responsible for transmitting traffic to a remote server for the purpose of further analysis and building behavioral models. To use this functionality, please contact technical support.
trunk_enable
Activation of the mechanism for transmitting the analyzed traffic to the Nemesida WAF MLC server. By default, the functionality is deactivated.

[mtp]
Section is responsible for module’s work in Point-to-Multipoint mode. This mode is used to save hardware resources, allows to use one server with installed Nemesida AI MLC for interaction with many servers with installed modules of Nemesida WAF (RabbitMQ software should be installed on every server where the Nemesida WAF module is).
mtp_multiblock
Functional activation or deactivation of transmission of information about detected brute-force attacks and attacks which were detected by subsystem of additional analysis on all Nemesida WAF modules (parameter ai_extra in file nwaf.conf). When the functional is deactivated the information transmits only in the Nemesida WAF module which is the source of the detected incident.
mtp_conf_1
mtp_conf_2
Settings for connection to remote RabbitMQ services. Use postfix _X (where X is numerical identifier of every instance of RabbitMQ service) for working with several instances of service.

Parameter directives:
rmq_user:rmq_password@remotehost — login, password and remote service RabbitMQ connection address;
lickey —Nemesida WAF license key from setting nwaf_license_key in file nwaf.conf.

The domain names’ list, which are used as virtual hosts (require behavioral models’ creating), will be used from setting vhosts_list.


[brute]
The section responsible for the brute-force detection function. Revealing of values ​​is performed in the ARGS and / or BODY areas.
brute_enable
Activation/deactivation of the functional.
brute_wl
The parameter that allows you to deactivate the brute-force attack detection functionality for specific virtual hosts.

Examples:
brute_wl = example.com – deactivation of the functional for example.com;
brute_wl = example.com, m.example.com – deactivation of functionality for example.com and m.example.com;
brute_wl = example.com, * .example.com – deactivation of functionality for example.com and its subdomains.

interval
The time interval of the segment (window) during which the analysis of requests is performed.
max_val
The number of requests that, when reached, block the source(s) of the attack.
similarity
A measure of the proximity of requests in percent.
mconf
File for the manual entry of the information on authorization of the protected Web application to block brute force attacks.

It’s recommended to fill the file to improve the accuracy of the attacks detection.
For example:

urilist :
[
  {
    vhost : 'example.com'
    uri   : '/login.php'
    type  : 'POST'
  }
  {
    vhost : '*.example.com'
    uri   : '/new-login.php'
    type  : 'POST'
  }
  {
    vhost : '*.example.com'
    uri   : '/new-login.php'
    type  : 'GET,POST'
  }
]
distributed
Protection against distributed brute force attack (disabled by default). With a value of distributed false attack detection is calculated for a single IP-address.

[st]
Section responsible for the interaction with Nemesida WAF Signtest module of training management.
st_enable
Sending disputed requests received from the Nemesida WAF module using RabbitMQ to the Nemesida WAF Signtest server for post-processing.

Disputed requests are defined as follows:
– if the signature analysis determined the request as illegitimate, and the Nemesida AI MLC module was defined as legitimate;
– if the signature analysis determined the request as legitimate, and the Nemesida AI MLC module was defined as illegitimate.

st_uri
The URI of the Nemesida WAF Signtest server for sending disputed requests. When using the local version of Nemesida WAF Signtest, change the URI of the parameter.
Multipoint Mode using
Nemesida AI MLC is required to 32 GB of free RAM. If you use several services with Nemesida WAF module you are able to save hardware resources using Point-to-Multipoint scheme (one server with installed Nemesida AI MLC module interacts with servers with installed Nemesida WAF module).

Components are used in Multipoint Mode:

  • Servers with installed Nemesida WAF and RabbitMQ, which have 2-4 GB of RAM for every of them;
  • Server with installed Nemesida AI MLC and RabbitMQ, which have 32 GB of RAM.

On the server with installed Nemesida WAF

- Create service’s user RabbitMQ:

# rabbitmqctl add_user USER PASSWORD
# rabbitmqctl set_permissions -p / USER ".*" ".*" ".*"

- Make changes to the file /etc/rabbitmq/rabbitmq.config:

[
    {rabbitmq_management, [
        {listener, [{port, 15672}, {ip, "127.0.0.1"}]}
    ]},
    {kernel, [
        {inet_dist_use_interface,{127,0,0,1}}
    ]}
].

- Make changes to the file /etc/rabbitmq/rabbitmq-env.conf:

NODE_PORT=5672
export RABBITMQ_NODENAME=rabbit@localhost
export RABBITMQ_NODE_IP_ADDRESS=0.0.0.0
export ERL_EPMD_ADDRESS=127.0.0.1

- Permit accesses from the server where Nemesida AI MLC installed to the port 5672 (TCP).
- Complete settings of RabbitMQ:

# chown rabbitmq:rabbitmq /etc/rabbitmq/rabbitmq.config
# service rabbitmq-server restart

On the server with installed Nemesida AI MLC

- Make changes to the file /opt/mlc/mlc.conf (including [mtp] section settings):

# service mlc_main restart
# service mlc_main status

Using RabbitMQ's mode services Nemesida AI MLC will collect requests with following models' training as if it worked in normal mode.

Using the Nemesida AI cloud server
The cloud server Nemesida AI, located in the Pentestit infrastructure, is designed to generate behavioral models based on a copy of traffic received from remote servers. The cloud server is used in cases when the user of the Nemesida WAF software does not have enough RAM for the Nemesida AI MLC module (it requires up to 32 GB). To use the capabilities of the cloud server Nemesida AI, contact technical support.

After making the changes, restart the service:

# service mlc_main restart
# service mlc_main status

Nemesida AI models retraining

To improve the accuracy of attack detection, it is recommended to re-train models once a week. For that you need to add the ^ symbol to virtual host. For example: vhosts_list = *^ or vhosts_list = example.com^.

Restart the service after changes:

# service mlc_main restart
Configuring interaction with ClamAV software

After installing Nemesida WAF package the functionality of interaction with ClamAV software is disabled by default, since it can be a source of false blocking of some requests to the Web application (depending on the current state of the ClamAV signature analysis database). Use this functionality at your discretion.

To activate antivirus protection, install ClamAV software on the server with the configured Nemesida WAF software, if it has not done yet.

Installation example for Debian 9 OS:

# apt install clamav-daemon

The interaction with the ClamAV software is enabled by activating the nwaf_clamav configuration parameter in the /etc/nginx/nwaf/conf/global/nwaf.conf file and reduction of the /etc/clamav/clamd.conf file to a look:

...
TCPSocket 3310
TCPAddr   127.0.0.1
... 

After changing restart the Nginx software.

Using Nemesida WAF in IDS mode

It is necessary to set up traffic mirroring from the main web server (through which calls to the web application are made)to the server with the installed Nemesida WAF software, to working Nemesida WAF in IDS mode. You must make changes to the files on every server:

1. On the main server (without the Nemesida WAF module installed), configure traffic mirroring according to the guidelines of the installed web server (Nginx, Apache2, Microsoft IIS and others).

An example of Nginx setting for traffic mirroring

If using the Nginx web server, make the necessary changes to the virtual host file:

location / {
    mirror /mirror;
    ...
}

location = /mirror {
    internal;
    proxy_pass http://nemesida_waf_server$request_uri;
}

where nemesida_waf_server is the address of the server with the Nemesida WAF module installed, to which duplicate traffic will be transmitted.

2. On the server with the installed Nemesida WAF module, bring the configuration file of the virtual host Nginx to the form:

server {
        listen  80;
        index   index.html;
        root    /var/www/html;
        try_files $uri $uri/ /index.html;
}

3. On the server with the Nemesida WAF module installed, create the /var/www/html directory and place an empty index.html file in it.

4. On the server with the Nemesida WAF module installed, bring the /etc/nginx/nwaf/conf/global/nwaf.conf file to the form:

...
nwaf_limit rate=5r/m block_time=0;
...

5. After changing restart the Nginx on every server.

The validity check of modules functioning

After installing and configuring the Nemesida WAF modules, restart the operating system and check the operation of all modules:

# systemctl status nginx nwaf_update mla_main mlc_main rabbitmq-server

If modules’ operation is correct there is a inscription: Active: active (running).

When the license key expires, the software works in the Nemesida WAF Free mode.

Signature management

There is a support of custom set of signatures rules (signature, RL) and signature exception rules (exclusion rule, WL) in Nemesida WAF and Nemesida WAF Free.

Zones, subzones and refinements

During the creation of RL/WL rules special parameters can be used:

  • zones: URL, ARGS, BODY or HEADERS;
  • subzones (header field HEADERS): $HEADERS_VAR:Cookie, $HEADERS_VAR:User-Agent, $HEADERS_VAR:Content-Type etc;
  • conditions of using the rule (zone refinements): $URL, $ARGS, $BODY and $HEADERS.

Using zones, subzones and refinements allows to concretize maximally the creating rule.

It’s possible to use regular expressions during the refinement. This requires to add to the refinement postfix _X. For example: "Z:ARGS|$URL_X:/\w+". To using the separator (as a metacharacter of a regular expression )in refinements, it is necessary to escape the character. For example: rule ... "Z:...|$URL_X:/(a\|b)/"; will be used to URL, which contains /a/ or /b/.

Several parameters (zones, subzones, refinements) in one rule must be separated the character | , the following principle of interaction will be used:

  • zones or subzones interact using the logical principle OR;
  • refinements interact using the logical principle OR;
  • zones or subzones interact with refinements using the logical principle AND.

User-defined signature creation

The user-defined rules of detecting attack signs can be placed in the main configuration file Nemesida WAF (nwaf.conf) or in the self-created file of the form *.conf, located in the /etc/nginx/nwaf/conf/vhosts. User-defined signatures must be determined by the RL parameter and must have ID started from 50000 and take the following form:

RL ID:50000 "P:select from" "SC:SQL:12" "Z:ARGS";
RL ID:50001 domain=example.com "P:select from" "SC:SQL:12" "Z:ARGS";
RL ID:50002 domain=*.example.com "P:select from" "SC:SQL:12" "Z:ARGS";
RL ID:50003 "PX:select\s+from" "SC:SQL:12" "Z:ARGS|$URL:/admin";
RL ID:50004 "P:select from" "SC:SQL:12" "Z:ARGS|$URL:/(admin\|dev)";

For signatures in the zone it is allowed to use the MLA value to force request sending into Nemesida AI MLA, that falls within the scope of the created signature if the score < 12 (by default requests with a score equal or greater than the value of the mla_score parameter are sent into Nemesida AI MLA).

Signature force sending into Nemesida AI MLA examples
Request contains an entry select from in any of four zones will be sent into Nemesida AI MLA:

RL ID:50005 "P:select from" "SC:SQL:1" "Z:MLA";

Request contains an entry select from in ARGS zone will be sent into Nemesida AI MLA:

RL ID:50006 "P:select from" "SC:SQL:1" "Z:ARGS|MLA";
Signature options
ID
The unique identifier for the rule. A range from 50000 to 59999 is available for creating your own rules. Required.
domain
Set the ownership of the rule to the domain. For the domain option it is allowed to use a wildcard value.
P/PX
Option defining the entry pattern (option P is used to denote a simple entry, option PX is a regular expression). Required.
SC
Setting the rule tag (Injection, XSS, UWA, Scanner, Evasion or Other) and the numerical value indicator (from 1 to 12). Required.

Requests that fall within the scope of the rules with an indicator value of 12 are blocked without being sent to other analysis subsystems. More information is available in the corresponding section.

Z
Zone/subzone application of the rule. To apply a signature to all zones, use the empty "Z:" parameter. Required.

To apply a signature to multiple zones, use the delimiter "Z:URL|BODY".

To force the request to the Nemesida AI MLA module use MLA value in rules application zone ("Z:MLA") while digital significance indicator (SC) must be less than 12.

To clarify the zone, you can use additional options. For "Z:ARGS|$URL:/templates" the rule will work only in the ARGS zone with /templates parameter in URL.

To reduce the number of false positives during the creation signature rules it is neсessary to specificate them maximally.

Creating a signature exclusion rule

In case the inquiry falls under action of a signature, in addition to sending the incident to the Nemesida WAF API, the following line will be displayed in the error log of the Nginx software:

Nemesida WAF: the request ххх contains a rule id 1 in zone HEADERS, ...

or, if the request contains a signature with a maximum allowable digital indicator of significance (score = 12):

Nemesida WAF: the request ххх blocked by rule id 1 in zone HEADERS, ...

where:

  • 1 – attack signature ID;
  • HEADERS – signature entry area.

To display absolutely all occurrences of signatures in the request (if there are occurrences), including those occurrences that did not lead to the subsequent blocking of the request, activate the nwaf_log_mr_all; parameter, in the main configuration file Nemesida WAF.

The information about current signature list is on the page rlinfo.nemesida-security.com.

Examples of creating attack signature exception rules

WL ID:1 "Z:"; – using these parameters, the entry of the rule with the identifier 1 will be excluded from all zones for all virtual hosts.

WL ID:1 "Z:ARGS|HEADERS"; – using these parameters the entry of the rule with the identifier 1 will be excluded from the ARGS and HEADERS zones for all virtual hosts.

WL ID:1 "Z:ARGS|$HEADERS_VAR:Cookie"; – using these parameters the entry of the rule with the identifier 1 will be excluded from the ARGS zone and Cookie subzone for all virtual hosts.

WL ID:1 domain=*.example.com "Z:URL"; – using these parameters the entry of the rule with the identifier 1 will be excluded from the URL zone for the virtual host example.com and its subdomains.

WL ID:1 domain=example.com "Z:URL|$URL:/index/index.php"; – using these parameters the entry of the rule with the identifier 1 will be excluded from the URL zone for the virtual host example.com for URI http://example.com/index/index.php.

WL ID: domain=example.com "Z:ARGS"; – using these parameters the entry of the rule ARGS zone of all requests to the virtual host example.com > will be excluded from the signature analysis processing.

WL ID:* domain=example.com "Z:$URL:/test"; – using these parameters the entry of the rule all requests to example.com/test will be excluded from the signature analysis processing.

For parameter ID:* it’s possible to use any of zones as a refinement, but as a condition of using the rule – only parameter $URL.

For security reasons, when creating exclusion rules, you need to specify them as much as possible.

Error messages sources

During the Nemesida WAF operation the information about errors can be contained in:
— system logs;
— run-time journal of the Nginx software;
— run-time journal of the RabbitMQ software;
— run-time journal of the Nemesida WAF modules (/var/log/nwaf/*.log).

Technical support

In case of unforeseen errors in the operation of the Nemesida WAF software, contact technical support by email (Mon-Fri from 10:00 to 19:00 time, GMT+3) or leave a message on the forum.