CrowdSec, An Open-Source, Modernized & Collaborative Intrusion Prevention System (fail2ban)

CrowdSec is a security automation engine designed to protect servers, services, containers, or virtual machines exposed on the internet with a server-side agent. It was inspired by Fail2Ban and aims to be a modernized, collaborative version of that intrusion-prevention tool. 

CrowdSec is using a behavior analysis system to qualify whether someone is trying to hack you, based on your logs. If your agent detects such aggression, the offending IP is then dealt with and sent for curation. If this signal passes the curation process, the IP is then redistributed to all users sharing a similar technological profile to “immunize” them against this IP. 

The goal is to leverage the crowd power to create some form of Internet Neighborhood watch. As for the IP that aggressed your machine, you can choose to remedy the threat in any manner you feel appropriate. Ultimately, CrowdSec leverages the power of the crowd to create an extremely accurate IP reputation system that benefits all its users.

CrowdSec is free and open source (under an MIT License), with the source code available on GitHub. It is currently available for Linux, with ports to macOS and Windows on the roadmap. The version 1.0 of the solution just got released and can be found here. 

Here is a list of the solution key features:

• allows users to detect attacks and respond at any level (block in your firewall, reverse proxy, CDN or directly at the applicative layer)
• is easy to install and maintain with no technical requirement. The installer comes with a wizard
• is designed to be integrated with other solutions and components (ie. use CrowdSec to read your mod_security logs and automatically block attackers in your proxy while informing your SIEM)
• is about sharing: meta-data about the attack/attacker you detect is sent to a central API, and confirmed malevolent IPs are then shared back with all users.
• is a lightweight: it runs standalone, doesn’t require much ram or CPU, written in Golang for high performance
• can work with cold logs: you can run it on old logs and see what could have happened if you use this or that scenario or just to see who attacked you in the past
• comes with out of the box dashboards, because visualization is key
• Can be used with many different bouncers to reply in the most appropriate way to incoming threats (Drop, 2FA, Captcha, Script, etc.)

It is important to note that a French team is behind the development, which is a plus for privacy. Even if you choose “teamwork” and share collected data, only 3 parameters are sent: the time stamp, the IP addresses of the violators and the policy they have violated.

Download and install

You can be up and running in two minutes.

Installation of CrowdSec. A wizard in the console helps you to select and suggest which demons/logins to monitor, although subsequent configuration via conventional configs is also possible.

Download

Install

Architecture

The system consists of three main components:

1. The CrowdSec Service, which is basically the persistent service that monitors logs, tracks attacks, etc.
2. The Command Line Tool, which is the cli interface for interacting with the service.
3. Bouncers, which are tools allowing to remedy the threat where and how you see fit or interact with other software.

The full documentation can be found here.

The service does all the monitoring, the cscli tool is how you do configuration, ban stuff, get metrics, etc., and the bouncers are how the system interacts with other tools to actually do things, like blocking someone in Iptables, SSH, Cloudflare, etc.

Usage & configuration

To date, five ibouncers have been developed. It is very important to also install one of these to be actually protected.

• cs-cloudflare-blocker
• cs-custom-blocker (to launch your own custom scripts, which could for example be a bash script adding the IP in an IPSET blocked by your firewall and send you a notification)
• cs-netfilter-blocker
• cs-nginx-blocker
• cs-wordpress-blocker

Collections are basically sets of parsers and scenarios for different situations. For example, the Nginx collection includes the nginx-logs parser and basic http scripts to identify typical malicious bots (aggressive crawling, port scanning/punching, user-agent blacklist, and path traversal attack attempts). Here is the complete list:

• apache2
• base-http-scenarios
• iptables
• linux
• modsecurity
• mysql
• naxsi
• nginx
• postfix
• sshd

Another way to interact with CrowdSec is through the cscli console program. It supports a large list of commands and parameters for connecting/deleting configurations, adding new lock rules, etc.

This command provides basic metrics about parsers, volume of logs processed, number of threats detected and blocked for each collection (see above for a list of collections).

This command shows IPs that got banned, the number of events that were seen from them, the number of times they’ve been banned, the country they came from, as well as the IP their IP belongs to.

Apart from cscli, the configuration can also be modified in the traditional way by editing a text file in YAML format:

Naturally, your own custom scenarios are supported and the team highly encourages you to share them on the Hub.

Integrations

What makes this tool more like a platform than a utility is its numerous integrations with other tools. The system doesn’t just detect attacks using its view into your logs, it can also trigger various actions once something is detected, such as:

• Blocking people in Cloudflare
• Running your own arbitrary scripts
• Executing a block in netfilter/iptables
• Denying an IP in Nginx
• Blocking in WordPress
• And this is just barely the beginning since the community starts to develop various integrations, bouncers, scenarios and data sources.

Monetization

The company will offer paid access to a cloud API and its IP reputation database to users who are not willing to share their log data (or can’t). Community members can use the software for free and also get free access to the IP reputation system, as long as they share their own sightings.

Two offers will be available: Premium and Enterprise with support services, special service tools (such as deploying the system to several locations from one central location), use of data mining and machine training (detecting trends in global data), more advanced cold log analysis (forensics, investigations). Don’t forget, the open source tool is released under a free MIT license, so that the company’s business plans do not prevent the community from using the solution and modifying it to suit their needs.

Where to find CrowdSec

Currently, CrowsSec community members come from 60+ countries across 6 different continents.

The team is looking for more users, contributors, and ambassadors to take the project to the next level. They would love to hear your feedback and engage in further discussions. They can be found on GitHub, Discourse or Gitter.