Installing CloudShark via Docker

CloudShark in Docker

Starting with CloudShark 3.9, users are able to deploy in a Docker environment. This document provides an overview of the environment and a sample docker-compose.yaml file that may be used as a starting point for your own deployment. These settings are intended to be used as an example for your container management platform of choice.

The CloudShark Base Image

All of the CloudShark services run on top of our base image. A download link to this image is available to customers from the QA Cafe Customer Lounge. Once you have downloaded it, it can be loaded into your docker environment. It is self-contained and does not need to access the internet.

$ docker load < cloudshark-docker-3.9.4.tar.gz

This creates an image in your local repository tagged as qacafe/cloudshark:3.9.4. You may push this image to your own local container repository if needed.

The following services are run from this one image:

  • application web server
  • 2 search-workers
  • auto-importer
  • threat-assessment service (Suricata)
  • periodic cron jobs

Additional Containers

3rd-party containers providing a database and caching services are also specified in the docker-compose config, including:

  • MariaDB v10
  • Redis v3.2
  • Memcache v1.6.9

CloudShark relies on publicly available versions of these services that have been published on Docker Hub.

 

Required: The QA Cafe License App Server

Running CloudShark in a Docker requires it can connect to and communicate with a QA Cafe license sever running outside of the containerized deployment. This may be on the docker host, or a separate machine. A bare metal deployment is preferred.

QALA is available to be downloaded as an RPM for either CentOS 7 or Rocky Linux 8. Please request the desired RPM from our Support Team.

Volumes

PCAP Data

It is a very good idea to store PCAP data outside of your Docker. The following path should be mounted from an external docker volume:

/usr/cloudshark/data

Autoimports

If you wish to configure Auto-Import features, you will also need to make a  volume available within the CloudShark container mounted as:

/autoimport

By mounting a host directory at this location, you can move files on your host and have them processed by CloudShark within Docker. You will need to configure this path in the Auto-Import settings in CloudShark.

The Database

Additionally, the MariaDB database container should also have external storage supplied and mounted under:

/var/lib/mysql

Environment Variables

The following environment variables are used to configure CloudShark to talk to external services. They must be specified.

DATABASE_URL=mysql://<user>:<password>@<db-addr>/<db-name>
REDIS_URL=redis://<redis-addr>/
MEMCACHE=<memcache-addr>:11211
QACAFE_LICENSE_SERVER=<external IP>

Caveats

External Authentication

External Authentication using LDAP/AD is not supported when running CloudShark within a container. SAML and OAuth are still available.

Sample docker-compose.yaml

Below is our sample docker-compose.yaml file showing these services working together. While Docker does not recommend using compose in production, it still serves as a concrete, easy to translate example of how all the service interact.

version: "3.8"

volumes:
data:
db:
config:

services:
app:
image: qacafe/cloudshark:3.9.4
environment:
- DATABASE_URL=mysql://root@mariadb/cloudshark
- REDIS_URL=redis://redis/
- MEMCACHE=memcached:11211
- QACAFE_LICENSE_SERVER=<external IP>
depends_on:
- mariadb
- memcached
- redis
volumes:
- data:/usr/cloudshark/data
- config:/usr/cloudshark/etc
ports:
- "443:443"
- "80:80"
restart: unless-stopped

memcached:
image: memcached:1.6.9

redis:
image: redis:3.2

mariadb:
image: mariadb:10
volumes:
- db:/var/lib/mysql
environment:
- MYSQL_ALLOW_EMPTY_PASSWORD=yes
- MYSQL_DATABASE=cloudshark


Notes:

  • The DATABASE_URL environment variable must match the credentials defined in the mariadb image's environment. This example uses an empty root password for the database. If you would like to use a password, you can configure one by setting the MYSQL_ROOT_PASSWORD environment variable. More documentation to configure MariaDB can be found here.

 

Custom Threat Assessment Rules

If you are using custom Suricata rules (such as ET Pro) you will need to update them from within your running container. CloudShark ships with suricata-update installed which should be used to manage your rule sources.

To maintain additional rules, you should also define and mount a docker volume similar to db, config, and data:

/var/lib/suricata

By mounting this as an external volume, you may rebuild your docker image without losing your configured rule sources.

Running suricata-update

Running suricata-update must be done within the context of the Docker container. First, use docker ps to determine which container to run your command in:

❯ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
900ed2470c82 cloudshark_app "/usr/bin/supervisor…" 26 minutes ago Up 25 minutes 0.0.0.0:80->80/tcp, 0.0.0.0:443->443/tcp cloudshark_app_1
54b8b649244e mariadb:10 "docker-entrypoint.s…" 26 minutes ago Up 25 minutes 3306/tcp cloudshark_mariadb_1
701bb03b3a08 redis:3.2 "docker-entrypoint.s…" 26 minutes ago Up 25 minutes 6379/tcp cloudshark_redis_1
3d2774876583 memcached:1.6.9 "docker-entrypoint.s…" 26 minutes ago Up 25 minutes 11211/tcp cloudshark_memcached_1

Then, run /usr/bin/suricata-update in that Container to enable et/pro:

❯ docker exec -it 900ed2470c82 /usr/bin/suricata-update enable-source et/pro
20/4/2022 -- 14:48:35 - <Info> -- Using data-directory /var/lib/suricata.
20/4/2022 -- 14:48:35 - <Info> -- Using Suricata configuration /etc/suricata/suricata.yaml
20/4/2022 -- 14:48:35 - <Info> -- Using /usr/share/suricata/rules for Suricata provided rules.
20/4/2022 -- 14:48:35 - <Info> -- Found Suricata version 5.0.7 at /usr/sbin/suricata.
The source et/pro requires a subscription. Subscribe here:
https://www.proofpoint.com/us/threat-insight/et-pro-ruleset
Emerging Threats Pro access code (secret-code): xxxxxxx
20/4/2022 -- 14:48:50 - <Info> -- Source et/pro enabled

Subsequent runs of suricata-update will now pull in ETPro rules and store them in /var/lib/suricata/rules.

❯ docker exec -it 900ed2470c82 /usr/bin/suricata-update

The Docker Host machine should be in charge of periodically running this command in order to keep your rules up to date.