infernalsirius · November 20, 2025 07:34
diff --git a/setting_up_crowdsec_pangolin_middleware_manager b/setting_up_crowdsec_pangolin_middleware_manager
 Detailed Steps

 1) First obtain your enrolment key from Crowdsec. Enrollment refers to connecting your CrowdSec Security Engine to the CrowdSec Console

 To Get CrowdSec Enrollment Key
 - Visit https://www.crowdsec.net/
 - Log in to your account
 - Obtain the enrollment key from your dashboard
 - Copy this key for later use

 2) Next, let’s understand the file structure we’ll be working with:

 /root/config/
 ├── crowdsec/
 │   ├── acquis.yaml              # Defines log acquisition sources
 │   ├── config.yaml              # Main CrowdSec configuration
 │   ├── local_api_credentials.yaml # API credentials for bouncers
 │   ├── online_api_credentials.yaml # API credentials for bouncers
 │   ├── patterns/                # Fodler for patterns
 │   ├── profiles.yaml            # Defines remediation profiles
 │   ├── user.yaml                # User configuration
 │   └── hub/
 │   └── notifications/
 |       └── discord.yaml (optional)  # Optional notification config
 ├── crowdsec_logs/               # Crowdsec Logs
 ├── traefik/
 │   ├── conf/
 │   │   └── captcha.html         # HTML template for captcha challenges
 │   ├── rules/
 │       └── dynamic_config.yml   # Dynamic Traefik configuration
 │   ├── traefik_config.yml       # Static Traefik configuration
 │   └── logs/                    # Directory for Traefik logs
 └── letsencrypt/                 # Let's Encrypt certificates

 We will need to manually create these files. Might be good to check here https://github.com/crowdsecurity/crowdsec/tree/master/config regularly to prevent the info in this guide getting out of date

 3) Create Required Directories
 Before starting with configurations, ensure all necessary directories exist:

 mkdir -p ./config/crowdsec/notifications
 mkdir -p ./config/crowdsec/hub
 mkdir -p ./config/crowdsec_logs
 mkdir -p ./config/crowdsec/patterns
 mkdir -p ./config/traefik/conf
 mkdir -p ./config/traefik/logs

 3.1)  Optional - if you are going to be checking in your config into GitHub please remember to create a .gitignore so confidential files are not checked in
 Your .gitignore  could look like
 ```
 .env
 installer
 data/
 config/key
 config/crowdsec/db/crowdsec.db
 config/crowdsec/hub/
 config/db/db.sqlite
 config/traefik/logs/access.log
 config/crowdsec/local_api_credentials.yaml
 config/crowdsec/online_api_credentials.yaml
 config/crowdsec/appsec-configs/
 config/crowdsec/appsec-rules/
 config/crowdsec/collections/
 config/crowdsec/contexts/
 config/crowdsec/parsers/
 config/crowdsec/patterns/
 config/crowdsec/scenarios/
 *.bak.*
 ```

 4) CrowdSec Configuration Files

 a. Configure acquis.yaml
 This file defines where CrowdSec acquires logs from. Create ./config/crowdsec/acquis.yaml:

 ```
 filenames:
 - /var/log/auth.log
 - /var/log/syslog
 labels:
  type: syslog
 ---
 poll_without_inotify: false
 filenames:
  - /var/log/traefik/*.log
 labels:
  type: traefik
 ---
 listen_addr: 0.0.0.0:7422 
 appsec_config: crowdsecurity/appsec-default
 name: myAppSecComponent
 source: appsec
 labels:
  type: appsec
 ```

 This configuration:

 - Monitors system logs for SSH and authentication attacks
 - Watches Traefik logs for web attacks
 - Enables the Application Security (WAF) component on port 7422



 b. Configure config.yaml
 Create or edit ./config/crowdsec/config.yaml:

 ```
 common:
  daemonize: false
  log_media: stdout
  log_level: info
  log_dir: /var/log/
 config_paths:
  config_dir: /etc/crowdsec/
  data_dir: /var/lib/crowdsec/data/
  simulation_path: /etc/crowdsec/simulation.yaml
  hub_dir: /etc/crowdsec/hub/
  index_path: /etc/crowdsec/hub/.index.json
  notification_dir: /etc/crowdsec/notifications/
  plugin_dir: /usr/local/lib/crowdsec/plugins/
 crowdsec_service:
  acquisition_path: /etc/crowdsec/acquis.yaml
  acquisition_dir: /etc/crowdsec/acquis.d
  parser_routines: 1
 plugin_config:
  user: nobody
  group: nobody
 cscli:
  output: human
 db_config:
  log_level: info
  type: sqlite
  db_path: /var/lib/crowdsec/data/crowdsec.db
  flush:
    max_items: 5000
    max_age: 7d
  use_wal: false
 api:
  client:
    insecure_skip_verify: false
    credentials_path: /etc/crowdsec/local_api_credentials.yaml
  server:
    log_level: info
    listen_uri: 0.0.0.0:8080
    profiles_path: /etc/crowdsec/profiles.yaml
    trusted_ips: # IP ranges, or IPs which can have admin API access
      - 127.0.0.1
      - ::1
    online_client: # Central API credentials (to push signals and receive bad IPs)
      credentials_path: /etc/crowdsec/online_api_credentials.yaml
    enable: true
 prometheus:
  enabled: true
  level: full
  listen_addr: 0.0.0.0
  listen_port: 6060

 ```

 This configuration:

 Sets up the CrowdSec API server to listen on all interfaces (0.0.0.0)
 Configures path to credentials and profiles
 Allows connections from all IPs, which is needed for the Traefik plugin to communicate with CrowdSec


 c. Configure profiles.yaml
 Create or edit ./config/crowdsec/profiles.yaml:

 ```
 name: captcha_remediation
 filters:
  - Alert.Remediation == true && Alert.GetScope() == "Ip" && Alert.GetScenario() contains "http"
 decisions:
  - type: captcha
    duration: 4h
 on_success: break

 ---
 name: default_ip_remediation
 filters:
 - Alert.Remediation == true && Alert.GetScope() == "Ip"
 decisions:
 - type: ban
   duration: 4h
 on_success: break

 ---
 name: default_range_remediation
 filters:
 - Alert.Remediation == true && Alert.GetScope() == "Range"
 decisions:
 - type: ban
   duration: 4h
 on_success: break

 ```

 This configuration:

 - Creates a captcha profile for HTTP-related attacks
 - Sets up IP banning for other types of attacks
 - Configures ban durations of 4 hours
 Important: Make sure to comment out any notification configurations in this file (slack, splunk, http, email) if you’re not using them, as they might cause errors.


 d) create user.yaml file
 Create or edit ./config/crowdsec/user.yaml:


 ```
 common:
  daemonize: false
  log_media: stdout
  log_level: info
  log_dir: /var/log/
 config_paths:
  config_dir: /etc/crowdsec/
  data_dir: /var/lib/crowdsec/data
 crowdsec_service:
  parser_routines: 1
 cscli:
  output: human
 db_config:
  type: sqlite
  db_path: /var/lib/crowdsec/data/crowdsec.db
  user: crowdsec
  #log_level: info
  password: crowdsec
  db_name: crowdsec
  host: "127.0.0.1"
  port: 3306
 api:
  client:
    insecure_skip_verify: false # default true
    credentials_path: /etc/crowdsec/local_api_credentials.yaml
  server:
    #log_level: info
    listen_uri: 127.0.0.1:8080
    profiles_path: /etc/crowdsec/profiles.yaml
    online_client: # Central API
      credentials_path: /etc/crowdsec/online_api_credentials.yaml
 prometheus:
  enabled: true
  level: full

 ```

 This configures the user configuration paths


 e) create local_api_credentials.yaml file
 Create or edit ./config/crowdsec/local_api_credentials.yaml:

 The password will be inserted automatically upon boot up

 ```
 url: http://0.0.0.0:8080
 login: localhost
 password: UNIQUE_PASSWORD_WILL_BE_INSERTED_HERE

 ```

 This configures the local api credentials

 f) create online_api_credentials.yaml file

 ```
 touch ./config/crowdsec/online_api_credentials.yaml
 ```

 This configures a blank api credentials

 g) create simulation.yaml

 Create or edit ./config/crowdsec/simulation.yaml:


 ```
 simulation: false
 # exclusions:
 #  - crowdsecurity/ssh-bf

 ```

 5) Traefik Configuration

 a. Create Captcha Template
 Create a file at ./config/traefik/conf/captcha.html with the captcha template. You can get the template from:

 https://gist.githubusercontent.com/hhftechnology/48569d9f899bb6b889f9de2407efd0d2/raw/3cf6e4a44ae6761070c8297d92265fba4ba28e83/captcha.html
 This HTML template provides a user-friendly interface for the Cloudflare Turnstile captcha challenge.

 ```
 cd ./config/traefik/conf/
 wget https://gist.githubusercontent.com/hhftechnology/48569d9f899bb6b889f9de2407efd0d2/raw/3cf6e4a44ae6761070c8297d92265fba4ba28e83/captcha.html
 cd ../../..
 ```

 b) change the logs to json format in traefik_config

 change from
 ```
 log:
    format: common
    level: INFO
 ```
 to:
 ``` 
 log:
    level: "INFO"
    format: "json"

 accessLog:
    filePath: "/var/log/traefik/access.log"
    format: json
 ```


 6) Updating Docker Compose
 You’ll need to update your Docker Compose file to include CrowdSec. Here’s how to add the CrowdSec service.
 Make sure you insert your enrolment key that you obtain in a previous step

 ```
 # Add CrowdSec services
  crowdsec:
    command: -t
    container_name: crowdsec
    environment:
      ACQUIRE_FILES: /var/log/traefik/*.log
      COLLECTIONS: crowdsecurity/traefik crowdsecurity/appsec-virtual-patching crowdsecurity/appsec-generic-rules
      ENROLL_INSTANCE_NAME: pangolin-crowdsec
      ENROLL_TAGS: docker
      ENROLL_KEY: INSERT-ENROLLMENT-KEY-HERE
      GID: "1000"
      PARSERS: crowdsecurity/whitelists
    healthcheck:
      test:
        - CMD
        - cscli
        - capi
        - status
    image: crowdsecurity/crowdsec:latest
    depends_on:
      - gerbil  
    labels:
      - traefik.enable=false
    ports:
      - 8080:8080
      - 6060:6060
    expose:
      - 8080
      - 6060
      - 7422
    restart: unless-stopped
    volumes:
      - ./config/crowdsec:/etc/crowdsec
      - ./config/crowdsec/db:/var/lib/crowdsec/data
      - ./config/crowdsec_logs/auth.log:/var/log/auth.log:ro
      - ./config/crowdsec_logs/syslog:/var/log/syslog:ro
      - ./config/crowdsec_logs:/var/log
      - ./config/traefik/logs:/var/log/traefik
      - ./config/traefik/conf/captcha.html:/etc/traefik/conf/captcha.html

 ```


 This configuration:

 Sets up CrowdSec with the Traefik collections and parsers
 Maps volumes for configuration and logs
 Exposes the necessary ports for the API and metrics
 Configures health checks and dependencies




 7) Check that Crowdsec starts

 Assuming that the other docker stack is running (otherwise start it) then you can bring to bring up crowdsec

 ```
 docker compose up crowdsec
 ```

 you are looking for errors like

 ```
 crowdsec  | time="2025-05-28T08:45:27Z" level=fatal msg="no configuration paths provided"
 crowdsec  | Error: open null: no such file or directory
 ```

 this indicates that some of the configuration files cant be found.


 8) Clearing the crowdsec config

 If you experience issues in getting Crowdsec going you can reset the database to clear out any residual config

 ```
 rm -rf ./config/crowdsec/db/
 ```

 and then change the config and start docker again.


 9) Setting up the hub

 the first time you start crowdsec you will see an error like

 ```
 crowdsec  | Error: invalid hub index: unable to read index file: open /etc/crowdsec/hub/.index.json: no such file or directory. Run 'sudo cscli hub update' to download the index again
 ```

 we will know manually pull down the hub update by accessing the container's shell and running the command

 ```
 docker run --rm -it \
  --name crowdsec-shell \
  --entrypoint /bin/sh \
  -e GID="1000" \
  -e COLLECTIONS="crowdsecurity/traefik crowdsecurity/appsec-virtual-patching crowdsecurity/appsec-generic-rules" \
  -e ENROLL_INSTANCE_NAME="pangolin-crowdsec" \
  -e PARSERS="crowdsecurity/whitelists" \
  -e ENROLL_KEY="REMOVED" \
  -e ACQUIRE_FILES="/var/log/traefik/access.log" \
  -e ENROLL_TAGS="docker" \
  -v "$(pwd)/config/crowdsec:/etc/crowdsec" \
  -v "$(pwd)/config/crowdsec/db:/var/lib/crowdsec/data" \
  -v "$(pwd)/config/crowdsec_logs/auth.log:/var/log/auth.log:ro" \
  -v "$(pwd)/config/crowdsec_logs/syslog:/var/log/syslog:ro" \
  -v "$(pwd)/config/crowdsec_logs:/var/log" \
  -v "$(pwd)/config/traefik/logs:/var/log/traefik" \
  -v "$(pwd)/config/traefik/conf/captcha.html:/etc/traefik/conf/captcha.html" \
  crowdsecurity/crowdsec:latest

 ```

 you can then run

 ```
 cscli hub update
 ```

 you will see 
 `Downloading /etc/crowdsec/hub/.index.json`


 then You need to regenerate the /etc/crowdsec/online_api_credentials.yaml Easiest way is rm /etc/crowdsec/online_api_credentials.yaml  and register  again using the enrolment key from the previous step

 ```
 touch /etc/crowdsec/online_api_credentials.yaml
 cscli capi register
 cscli console enroll <id>
 ```

 try 
 ```
 docker compose up crowdsec
 ```

 if you see an error - Instance already enrolled. You can use ‘–overwrite’ to force enroll

 if you error the error crowdsec  | time="2025-05-28T12:37:09Z" level=fatal msg="crowdsec init: while loading parsers: failed to load parser config 

 then you will need to install the parsers

 ```
 docker run --rm -it \
  --name crowdsec-shell \
  --entrypoint /bin/sh \
  -e GID="1000" \
  -e COLLECTIONS="crowdsecurity/traefik crowdsecurity/appsec-virtual-patching crowdsecurity/appsec-generic-rules" \
  -e ENROLL_INSTANCE_NAME="pangolin-crowdsec" \
  -e PARSERS="crowdsecurity/whitelists" \
  -e ENROLL_KEY="REMOVED" \
  -e ACQUIRE_FILES="/var/log/traefik/access.log" \
  -e ENROLL_TAGS="docker" \
  -v "$(pwd)/config/crowdsec:/etc/crowdsec" \
  -v "$(pwd)/config/crowdsec/db:/var/lib/crowdsec/data" \
  -v "$(pwd)/config/crowdsec_logs/auth.log:/var/log/auth.log:ro" \
  -v "$(pwd)/config/crowdsec_logs/syslog:/var/log/syslog:ro" \
  -v "$(pwd)/config/crowdsec_logs:/var/log" \
  -v "$(pwd)/config/traefik/logs:/var/log/traefik" \
  -v "$(pwd)/config/traefik/conf/captcha.html:/etc/traefik/conf/captcha.html" \
  crowdsecurity/crowdsec:latest

 ```

 ```
 ls /etc/crowdsec/config/patterns/
 ```
 if you don't see any folders your crowdsec doesn't have the required patterns


 Here's a working around to download them
 ```
 wget -P /opt https://github.com/crowdsecurity/crowdsec/archive/refs/tags/v1.6.9-rc2.zip
 unzip /opt/v1.6.9-rc2.zip -d /opt
 cp -r /opt/crowdsec-1.6.9-rc2/config/patterns/* /etc/crowdsec/patterns/
 rm -rf /opt/crowdsec-1.6.9-rc2 /opt/v1.6.9-rc2.zip

 ```


 try 
 ```
 docker compose up crowdsec -d
 ```

 Everything should be working fine now. Check by looking at the logs `docker logs crowdsec`






 10) Now we are going to install the official CrowdSec bouncer for Traefik to block malicious IPs. We will use the Middleware manager to add this to our traefik_config file

 ```
 docker compose restart traefik
 ```

 and check for the logs to make sure everything started fine.

 Allow at least 2 minutes for all services to initialize fully. This gives CrowdSec time to load its rules and configurations.


 11) Generate an API key for the Traefik bouncer:

 in your hosts shell
 ```
 docker exec crowdsec cscli bouncers add traefik-bouncer
 ```
 it will return something like

 ```
 API key for 'traefik-bouncer':

   YOUR-LAPI-KEY-HERE

 Please keep this key since you will not be able to retrieve it! You will need it later

 ```

 12) Setup Cloudflare Turnstile
 Cloudflare Turnstile provides a modern, user-friendly captcha service that’s more accessible than traditional captchas.

 - Go to the Cloudflare dashboard (https://dash.cloudflare.com/)
 - Navigate to the Turnstile section
 - Create a new widget:
 - Use non-interactive mode for better integration
 - Set domains to your Pangolin domain
 - Choose appropriate settings for your security needs
 - Copy the site key and secret key



 13) Next edit the crowdsec middleware in the Middleware Manager and change it to your settings. Make sure you Update the captchaSiteKey and captchaSecretKey values in your crowdsec middleware config and change the cowdsecLapiKey to the key you saved earlier when you added a bouncer.

 ```
 {
  "crowdsec-bouncer-traefik": {
    "captchaGracePeriodSeconds": 1800,
    "captchaHTMLFilePath": "/etc/traefik/conf/captcha.html",
    "captchaProvider": "turnstile",
    "captchaSecretKey": "REPLACE_WITH_YOUR_TURNSTILE_SECRET",
    "captchaSiteKey": "REPLACE_WITH_YOUR_TURNSTILE_KEY",
    "clientTrustedIPs": [],
    "crowdsecAppsecEnabled": true,
    "crowdsecAppsecFailureBlock": true,
    "crowdsecAppsecHost": "crowdsec:7422",
    "crowdsecAppsecUnreachableBlock": true,
    "crowdsecLapiHost": "crowdsec:8080",
    "crowdsecLapiKey": "REPLACE_WITH_YOUR_BOUNCER_KEY",
    "crowdsecLapiScheme": "http",
    "crowdsecMode": "live",
    "defaultDecisionSeconds": 15,
    "enabled": true,
    "forwardedHeadersTrustedIPs": [
      "0.0.0.0/0"
    ],
    "httpTimeoutSeconds": 10,
    "logLevel": "INFO",
    "updateIntervalSeconds": 15,
    "updateMaxFailure": 0
  }
 }
 ```


 14) Now we have to define a resource in order to provide a url that will be protected by Crowdsec Bouncer Plugin. For example, I will create a resource called mywebsite.yourdomain.com in pangolin. 

 You can do this by defining a new resource in Pangolin. Make this resource points to a working 

 if you don't have a resource to protect then you can use the following for testing

 You don’t need this step if you already have a resource that you’d like to protect.

 Start a Simple HTTP Server
 Add this to your docker-compose.yml for a temporary test server:

  python-http:
    image: python:3.11-slim
    container_name: python-http
    working_dir: /app
    command: python -m http.server 15000
    ports:
      - "15000:15000"
    restart: unless-stopped
 Start it:

 docker compose up -d

 Now add the crowdsec middleware to your web resource. Its always good practice to check the traefik dashboard for an errors. 


 15) Testing Your CrowdSec Installation
 Test the Captcha implementation by adding a manual decision:

 docker exec crowdsec cscli decisions add --ip YOUR_IP --type captcha -d 1h
 Verify the decision was added:

 docker exec -it crowdsec cscli decisions list
 Try accessing your site from that IP address - you should be presented with a captcha challenge.

 Test the application security by trying to access potential attack vectors:

 https://yourdomain.com/.env
 This should return a 403 error if the WAF is working correctly.

 Check the CrowdSec logs to see if attacks are being detected:

 docker exec -it crowdsec tail -f /var/log/traefik/access.log


 16) Troubleshooting
 Common Issues and Solutions
 403 Errors When Accessing Your Site

 Check Traefik logs: `docker compose logs traefik -f`
 Verify the clientTrustedIPs list includes your IP range
 Check CrowdSec decisions: `docker exec -it crowdsec cscli decisions list`
 Try clearing decisions for your IP: `docker exec crowdsec cscli decisions delete --ip YOUR_IP
 Plugin Loading Errors

 Make sure the plugin version is correct in traefik_config.yml
 Check if http notifications are uncommented in profiles.yaml (they should be commented out if not in use)
 Restart the services: `docker compose restart traefik crowdsec
 Captcha Not Working

 Ensure Turnstile is configured correctly with valid site and secret keys
 Verify the captcha.html file exists in the correct location
 Check if the turnstile script is loading in browser developer tools
 CrowdSec Not Detecting Attacks

 Verify log paths are correct in acquis.yaml
 Check if logs are being written: `docker exec -it crowdsec ls -l /var/log/traefik/`
 Make sure Traefik’s accessLog is enabled and in JSON format
 Check if collections are installed: `docker exec crowdsec cscli collections list`
 Useful Commands for Monitoring and Troubleshooting

 ```
 # View CrowdSec overview
 docker exec crowdsec cscli status

 # Check which collections are installed
 docker exec crowdsec cscli collections list

 # Monitor CrowdSec resources
 docker stats crowdsec

 # Check AppSec metrics
 curl http://localhost:6060/metrics | grep appsec

 # View Traefik logs
 docker exec -it crowdsec ls -l /var/log/traefik/

 # Check CrowdSec metrics
 docker exec -it crowdsec cscli metrics

 # View active decisions
 docker exec -it crowdsec cscli decisions list

 # Monitor CrowdSec logs
 docker exec -it crowdsec tail -f /var/log/traefik/access.log 

 # Manually add decisions for testing
 docker exec crowdsec cscli decisions add --ip <IP> --type captcha -d 1h
 docker exec crowdsec cscli decisions add -i <IP> -t ban -d 1h

 # Monitor Traefik logs
 docker compose logs traefik -f

 # Restart services
 docker compose restart traefik crowdsec

 # View/manage bouncers
 docker exec crowdsec cscli bouncers list
 docker exec crowdsec cscli bouncers add traefik-bouncer
 docker exec crowdsec cscli bouncers delete traefik-bouncer
 ```

 17) Ongoing Maintenance and Advanced Configuration
 Regular Maintenance Tasks
 Keep CrowdSec Updated

 docker compose pull crowdsec
 docker compose up -d
 Update Collections and Parsers

 docker exec crowdsec cscli hub update
 docker exec crowdsec cscli collections upgrade
 Monitor for False Positives
 Regularly check decisions to ensure legitimate users aren’t being blocked:

 docker exec crowdsec cscli decisions list
 Create Allowlists for Trusted IPs
 Add your trusted infrastructure to avoid false positives:

 docker exec crowdsec cscli ipset add -f your-trusted-ips.txt
 Advanced Configuration Options
 Add Custom Scenarios
 You can create custom detection rules in YAML format in /etc/crowdsec/scenarios/.

 Configure Notifications
 Set up notifications for attacks via Slack, Discord, or email in profiles.yaml.

 Fine-tune Remediation Profiles
 Adjust ban durations and captcha settings in profiles.yaml based on your security requirements.

 Implement Geolocation-based Rules
 Use the GeoIP enricher to create country-specific rules:

 ``
 docker exec crowdsec cscli collections install crowdsecurity/geoip-enrich
 ```
	Detailed Steps

	1) First obtain your enrolment key from Crowdsec. Enrollment refers to connecting your CrowdSec Security Engine to the CrowdSec Console

	To Get CrowdSec Enrollment Key
	- Visit https://www.crowdsec.net/
	- Log in to your account
	- Obtain the enrollment key from your dashboard
	- Copy this key for later use

	2) Next, let’s understand the file structure we’ll be working with:

	/root/config/
	├── crowdsec/
	│ ├── acquis.yaml # Defines log acquisition sources
	│ ├── config.yaml # Main CrowdSec configuration
	│ ├── local_api_credentials.yaml # API credentials for bouncers
	│ ├── online_api_credentials.yaml # API credentials for bouncers
	│ ├── patterns/ # Fodler for patterns
	│ ├── profiles.yaml # Defines remediation profiles
	│ ├── user.yaml # User configuration
	│ └── hub/
	│ └── notifications/
	\| └── discord.yaml (optional) # Optional notification config
	├── crowdsec_logs/ # Crowdsec Logs
	├── traefik/
	│ ├── conf/
	│ │ └── captcha.html # HTML template for captcha challenges
	│ ├── rules/
	│ └── dynamic_config.yml # Dynamic Traefik configuration
	│ ├── traefik_config.yml # Static Traefik configuration
	│ └── logs/ # Directory for Traefik logs
	└── letsencrypt/ # Let's Encrypt certificates

	We will need to manually create these files. Might be good to check here https://github.com/crowdsecurity/crowdsec/tree/master/config regularly to prevent the info in this guide getting out of date

	3) Create Required Directories
	Before starting with configurations, ensure all necessary directories exist:

	mkdir -p ./config/crowdsec/notifications
	mkdir -p ./config/crowdsec/hub
	mkdir -p ./config/crowdsec_logs
	mkdir -p ./config/crowdsec/patterns
	mkdir -p ./config/traefik/conf
	mkdir -p ./config/traefik/logs

	3.1) Optional - if you are going to be checking in your config into GitHub please remember to create a .gitignore so confidential files are not checked in
	Your .gitignore could look like
	```
	.env
	installer
	data/
	config/key
	config/crowdsec/db/crowdsec.db
	config/crowdsec/hub/
	config/db/db.sqlite
	config/traefik/logs/access.log
	config/crowdsec/local_api_credentials.yaml
	config/crowdsec/online_api_credentials.yaml
	config/crowdsec/appsec-configs/
	config/crowdsec/appsec-rules/
	config/crowdsec/collections/
	config/crowdsec/contexts/
	config/crowdsec/parsers/
	config/crowdsec/patterns/
	config/crowdsec/scenarios/
	.bak.
	```

	4) CrowdSec Configuration Files

	a. Configure acquis.yaml
	This file defines where CrowdSec acquires logs from. Create ./config/crowdsec/acquis.yaml:

	```
	filenames:
	- /var/log/auth.log
	- /var/log/syslog
	labels:
	type: syslog
	---
	poll_without_inotify: false
	filenames:
	- /var/log/traefik/*.log
	labels:
	type: traefik
	---
	listen_addr: 0.0.0.0:7422
	appsec_config: crowdsecurity/appsec-default
	name: myAppSecComponent
	source: appsec
	labels:
	type: appsec
	```

	This configuration:

	- Monitors system logs for SSH and authentication attacks
	- Watches Traefik logs for web attacks
	- Enables the Application Security (WAF) component on port 7422



	b. Configure config.yaml
	Create or edit ./config/crowdsec/config.yaml:

	```
	common:
	daemonize: false
	log_media: stdout
	log_level: info
	log_dir: /var/log/
	config_paths:
	config_dir: /etc/crowdsec/
	data_dir: /var/lib/crowdsec/data/
	simulation_path: /etc/crowdsec/simulation.yaml
	hub_dir: /etc/crowdsec/hub/
	index_path: /etc/crowdsec/hub/.index.json
	notification_dir: /etc/crowdsec/notifications/
	plugin_dir: /usr/local/lib/crowdsec/plugins/
	crowdsec_service:
	acquisition_path: /etc/crowdsec/acquis.yaml
	acquisition_dir: /etc/crowdsec/acquis.d
	parser_routines: 1
	plugin_config:
	user: nobody
	group: nobody
	cscli:
	output: human
	db_config:
	log_level: info
	type: sqlite
	db_path: /var/lib/crowdsec/data/crowdsec.db
	flush:
	max_items: 5000
	max_age: 7d
	use_wal: false
	api:
	client:
	insecure_skip_verify: false
	credentials_path: /etc/crowdsec/local_api_credentials.yaml
	server:
	log_level: info
	listen_uri: 0.0.0.0:8080
	profiles_path: /etc/crowdsec/profiles.yaml
	trusted_ips: # IP ranges, or IPs which can have admin API access
	- 127.0.0.1
	- ::1
	online_client: # Central API credentials (to push signals and receive bad IPs)
	credentials_path: /etc/crowdsec/online_api_credentials.yaml
	enable: true
	prometheus:
	enabled: true
	level: full
	listen_addr: 0.0.0.0
	listen_port: 6060

	```

	This configuration:

	Sets up the CrowdSec API server to listen on all interfaces (0.0.0.0)
	Configures path to credentials and profiles
	Allows connections from all IPs, which is needed for the Traefik plugin to communicate with CrowdSec


	c. Configure profiles.yaml
	Create or edit ./config/crowdsec/profiles.yaml:

	```
	name: captcha_remediation
	filters:
	- Alert.Remediation == true && Alert.GetScope() == "Ip" && Alert.GetScenario() contains "http"
	decisions:
	- type: captcha
	duration: 4h
	on_success: break

	---
	name: default_ip_remediation
	filters:
	- Alert.Remediation == true && Alert.GetScope() == "Ip"
	decisions:
	- type: ban
	duration: 4h
	on_success: break

	---
	name: default_range_remediation
	filters:
	- Alert.Remediation == true && Alert.GetScope() == "Range"
	decisions:
	- type: ban
	duration: 4h
	on_success: break

	```

	This configuration:

	- Creates a captcha profile for HTTP-related attacks
	- Sets up IP banning for other types of attacks
	- Configures ban durations of 4 hours
	Important: Make sure to comment out any notification configurations in this file (slack, splunk, http, email) if you’re not using them, as they might cause errors.


	d) create user.yaml file
	Create or edit ./config/crowdsec/user.yaml:


	```
	common:
	daemonize: false
	log_media: stdout
	log_level: info
	log_dir: /var/log/
	config_paths:
	config_dir: /etc/crowdsec/
	data_dir: /var/lib/crowdsec/data
	crowdsec_service:
	parser_routines: 1
	cscli:
	output: human
	db_config:
	type: sqlite
	db_path: /var/lib/crowdsec/data/crowdsec.db
	user: crowdsec
	#log_level: info
	password: crowdsec
	db_name: crowdsec
	host: "127.0.0.1"
	port: 3306
	api:
	client:
	insecure_skip_verify: false # default true
	credentials_path: /etc/crowdsec/local_api_credentials.yaml
	server:
	#log_level: info
	listen_uri: 127.0.0.1:8080
	profiles_path: /etc/crowdsec/profiles.yaml
	online_client: # Central API
	credentials_path: /etc/crowdsec/online_api_credentials.yaml
	prometheus:
	enabled: true
	level: full

	```

	This configures the user configuration paths


	e) create local_api_credentials.yaml file
	Create or edit ./config/crowdsec/local_api_credentials.yaml:

	The password will be inserted automatically upon boot up

	```
	url: http://0.0.0.0:8080
	login: localhost
	password: UNIQUE_PASSWORD_WILL_BE_INSERTED_HERE

	```

	This configures the local api credentials

	f) create online_api_credentials.yaml file

	```
	touch ./config/crowdsec/online_api_credentials.yaml
	```

	This configures a blank api credentials

	g) create simulation.yaml

	Create or edit ./config/crowdsec/simulation.yaml:


	```
	simulation: false
	# exclusions:
	# - crowdsecurity/ssh-bf

	```

	5) Traefik Configuration

	a. Create Captcha Template
	Create a file at ./config/traefik/conf/captcha.html with the captcha template. You can get the template from:

	https://gist.githubusercontent.com/hhftechnology/48569d9f899bb6b889f9de2407efd0d2/raw/3cf6e4a44ae6761070c8297d92265fba4ba28e83/captcha.html
	This HTML template provides a user-friendly interface for the Cloudflare Turnstile captcha challenge.

	```
	cd ./config/traefik/conf/
	wget https://gist.githubusercontent.com/hhftechnology/48569d9f899bb6b889f9de2407efd0d2/raw/3cf6e4a44ae6761070c8297d92265fba4ba28e83/captcha.html
	cd ../../..
	```

	b) change the logs to json format in traefik_config

	change from
	```
	log:
	format: common
	level: INFO
	```
	to:
	```
	log:
	level: "INFO"
	format: "json"

	accessLog:
	filePath: "/var/log/traefik/access.log"
	format: json
	```


	6) Updating Docker Compose
	You’ll need to update your Docker Compose file to include CrowdSec. Here’s how to add the CrowdSec service.
	Make sure you insert your enrolment key that you obtain in a previous step

	```
	# Add CrowdSec services
	crowdsec:
	command: -t
	container_name: crowdsec
	environment:
	ACQUIRE_FILES: /var/log/traefik/*.log
	COLLECTIONS: crowdsecurity/traefik crowdsecurity/appsec-virtual-patching crowdsecurity/appsec-generic-rules
	ENROLL_INSTANCE_NAME: pangolin-crowdsec
	ENROLL_TAGS: docker
	ENROLL_KEY: INSERT-ENROLLMENT-KEY-HERE
	GID: "1000"
	PARSERS: crowdsecurity/whitelists
	healthcheck:
	test:
	- CMD
	- cscli
	- capi
	- status
	image: crowdsecurity/crowdsec:latest
	depends_on:
	- gerbil
	labels:
	- traefik.enable=false
	ports:
	- 8080:8080
	- 6060:6060
	expose:
	- 8080
	- 6060
	- 7422
	restart: unless-stopped
	volumes:
	- ./config/crowdsec:/etc/crowdsec
	- ./config/crowdsec/db:/var/lib/crowdsec/data
	- ./config/crowdsec_logs/auth.log:/var/log/auth.log:ro
	- ./config/crowdsec_logs/syslog:/var/log/syslog:ro
	- ./config/crowdsec_logs:/var/log
	- ./config/traefik/logs:/var/log/traefik
	- ./config/traefik/conf/captcha.html:/etc/traefik/conf/captcha.html

	```


	This configuration:

	Sets up CrowdSec with the Traefik collections and parsers
	Maps volumes for configuration and logs
	Exposes the necessary ports for the API and metrics
	Configures health checks and dependencies




	7) Check that Crowdsec starts

	Assuming that the other docker stack is running (otherwise start it) then you can bring to bring up crowdsec

	```
	docker compose up crowdsec
	```

	you are looking for errors like

	```
	crowdsec \| time="2025-05-28T08:45:27Z" level=fatal msg="no configuration paths provided"
	crowdsec \| Error: open null: no such file or directory
	```

	this indicates that some of the configuration files cant be found.


	8) Clearing the crowdsec config

	If you experience issues in getting Crowdsec going you can reset the database to clear out any residual config

	```
	rm -rf ./config/crowdsec/db/
	```

	and then change the config and start docker again.


	9) Setting up the hub

	the first time you start crowdsec you will see an error like

	```
	crowdsec \| Error: invalid hub index: unable to read index file: open /etc/crowdsec/hub/.index.json: no such file or directory. Run 'sudo cscli hub update' to download the index again
	```

	we will know manually pull down the hub update by accessing the container's shell and running the command

	```
	docker run --rm -it \
	--name crowdsec-shell \
	--entrypoint /bin/sh \
	-e GID="1000" \
	-e COLLECTIONS="crowdsecurity/traefik crowdsecurity/appsec-virtual-patching crowdsecurity/appsec-generic-rules" \
	-e ENROLL_INSTANCE_NAME="pangolin-crowdsec" \
	-e PARSERS="crowdsecurity/whitelists" \
	-e ENROLL_KEY="REMOVED" \
	-e ACQUIRE_FILES="/var/log/traefik/access.log" \
	-e ENROLL_TAGS="docker" \
	-v "$(pwd)/config/crowdsec:/etc/crowdsec" \
	-v "$(pwd)/config/crowdsec/db:/var/lib/crowdsec/data" \
	-v "$(pwd)/config/crowdsec_logs/auth.log:/var/log/auth.log:ro" \
	-v "$(pwd)/config/crowdsec_logs/syslog:/var/log/syslog:ro" \
	-v "$(pwd)/config/crowdsec_logs:/var/log" \
	-v "$(pwd)/config/traefik/logs:/var/log/traefik" \
	-v "$(pwd)/config/traefik/conf/captcha.html:/etc/traefik/conf/captcha.html" \
	crowdsecurity/crowdsec:latest

	```

	you can then run

	```
	cscli hub update
	```

	you will see
	`Downloading /etc/crowdsec/hub/.index.json`


	then You need to regenerate the /etc/crowdsec/online_api_credentials.yaml Easiest way is rm /etc/crowdsec/online_api_credentials.yaml and register again using the enrolment key from the previous step

	```
	touch /etc/crowdsec/online_api_credentials.yaml
	cscli capi register
	cscli console enroll <id>
	```

	try
	```
	docker compose up crowdsec
	```

	if you see an error - Instance already enrolled. You can use ‘–overwrite’ to force enroll

	if you error the error crowdsec \| time="2025-05-28T12:37:09Z" level=fatal msg="crowdsec init: while loading parsers: failed to load parser config

	then you will need to install the parsers

	```
	docker run --rm -it \
	--name crowdsec-shell \
	--entrypoint /bin/sh \
	-e GID="1000" \
	-e COLLECTIONS="crowdsecurity/traefik crowdsecurity/appsec-virtual-patching crowdsecurity/appsec-generic-rules" \
	-e ENROLL_INSTANCE_NAME="pangolin-crowdsec" \
	-e PARSERS="crowdsecurity/whitelists" \
	-e ENROLL_KEY="REMOVED" \
	-e ACQUIRE_FILES="/var/log/traefik/access.log" \
	-e ENROLL_TAGS="docker" \
	-v "$(pwd)/config/crowdsec:/etc/crowdsec" \
	-v "$(pwd)/config/crowdsec/db:/var/lib/crowdsec/data" \
	-v "$(pwd)/config/crowdsec_logs/auth.log:/var/log/auth.log:ro" \
	-v "$(pwd)/config/crowdsec_logs/syslog:/var/log/syslog:ro" \
	-v "$(pwd)/config/crowdsec_logs:/var/log" \
	-v "$(pwd)/config/traefik/logs:/var/log/traefik" \
	-v "$(pwd)/config/traefik/conf/captcha.html:/etc/traefik/conf/captcha.html" \
	crowdsecurity/crowdsec:latest

	```

	```
	ls /etc/crowdsec/config/patterns/
	```
	if you don't see any folders your crowdsec doesn't have the required patterns


	Here's a working around to download them
	```
	wget -P /opt https://github.com/crowdsecurity/crowdsec/archive/refs/tags/v1.6.9-rc2.zip
	unzip /opt/v1.6.9-rc2.zip -d /opt
	cp -r /opt/crowdsec-1.6.9-rc2/config/patterns/* /etc/crowdsec/patterns/
	rm -rf /opt/crowdsec-1.6.9-rc2 /opt/v1.6.9-rc2.zip

	```


	try
	```
	docker compose up crowdsec -d
	```

	Everything should be working fine now. Check by looking at the logs `docker logs crowdsec`






	10) Now we are going to install the official CrowdSec bouncer for Traefik to block malicious IPs. We will use the Middleware manager to add this to our traefik_config file

	```
	docker compose restart traefik
	```

	and check for the logs to make sure everything started fine.

	Allow at least 2 minutes for all services to initialize fully. This gives CrowdSec time to load its rules and configurations.


	11) Generate an API key for the Traefik bouncer:

	in your hosts shell
	```
	docker exec crowdsec cscli bouncers add traefik-bouncer
	```
	it will return something like

	```
	API key for 'traefik-bouncer':

	YOUR-LAPI-KEY-HERE

	Please keep this key since you will not be able to retrieve it! You will need it later

	```

	12) Setup Cloudflare Turnstile
	Cloudflare Turnstile provides a modern, user-friendly captcha service that’s more accessible than traditional captchas.

	- Go to the Cloudflare dashboard (https://dash.cloudflare.com/)
	- Navigate to the Turnstile section
	- Create a new widget:
	- Use non-interactive mode for better integration
	- Set domains to your Pangolin domain
	- Choose appropriate settings for your security needs
	- Copy the site key and secret key



	13) Next edit the crowdsec middleware in the Middleware Manager and change it to your settings. Make sure you Update the captchaSiteKey and captchaSecretKey values in your crowdsec middleware config and change the cowdsecLapiKey to the key you saved earlier when you added a bouncer.

	```
	{
	"crowdsec-bouncer-traefik": {
	"captchaGracePeriodSeconds": 1800,
	"captchaHTMLFilePath": "/etc/traefik/conf/captcha.html",
	"captchaProvider": "turnstile",
	"captchaSecretKey": "REPLACE_WITH_YOUR_TURNSTILE_SECRET",
	"captchaSiteKey": "REPLACE_WITH_YOUR_TURNSTILE_KEY",
	"clientTrustedIPs": [],
	"crowdsecAppsecEnabled": true,
	"crowdsecAppsecFailureBlock": true,
	"crowdsecAppsecHost": "crowdsec:7422",
	"crowdsecAppsecUnreachableBlock": true,
	"crowdsecLapiHost": "crowdsec:8080",
	"crowdsecLapiKey": "REPLACE_WITH_YOUR_BOUNCER_KEY",
	"crowdsecLapiScheme": "http",
	"crowdsecMode": "live",
	"defaultDecisionSeconds": 15,
	"enabled": true,
	"forwardedHeadersTrustedIPs": [
	"0.0.0.0/0"
	],
	"httpTimeoutSeconds": 10,
	"logLevel": "INFO",
	"updateIntervalSeconds": 15,
	"updateMaxFailure": 0
	}
	}
	```


	14) Now we have to define a resource in order to provide a url that will be protected by Crowdsec Bouncer Plugin. For example, I will create a resource called mywebsite.yourdomain.com in pangolin.

	You can do this by defining a new resource in Pangolin. Make this resource points to a working

	if you don't have a resource to protect then you can use the following for testing

	You don’t need this step if you already have a resource that you’d like to protect.

	Start a Simple HTTP Server
	Add this to your docker-compose.yml for a temporary test server:

	python-http:
	image: python:3.11-slim
	container_name: python-http
	working_dir: /app
	command: python -m http.server 15000
	ports:
	- "15000:15000"
	restart: unless-stopped
	Start it:

	docker compose up -d

	Now add the crowdsec middleware to your web resource. Its always good practice to check the traefik dashboard for an errors.


	15) Testing Your CrowdSec Installation
	Test the Captcha implementation by adding a manual decision:

	docker exec crowdsec cscli decisions add --ip YOUR_IP --type captcha -d 1h
	Verify the decision was added:

	docker exec -it crowdsec cscli decisions list
	Try accessing your site from that IP address - you should be presented with a captcha challenge.

	Test the application security by trying to access potential attack vectors:

	https://yourdomain.com/.env
	This should return a 403 error if the WAF is working correctly.

	Check the CrowdSec logs to see if attacks are being detected:

	docker exec -it crowdsec tail -f /var/log/traefik/access.log


	16) Troubleshooting
	Common Issues and Solutions
	403 Errors When Accessing Your Site

	Check Traefik logs: `docker compose logs traefik -f`
	Verify the clientTrustedIPs list includes your IP range
	Check CrowdSec decisions: `docker exec -it crowdsec cscli decisions list`
	Try clearing decisions for your IP: `docker exec crowdsec cscli decisions delete --ip YOUR_IP
	Plugin Loading Errors

	Make sure the plugin version is correct in traefik_config.yml
	Check if http notifications are uncommented in profiles.yaml (they should be commented out if not in use)
	Restart the services: `docker compose restart traefik crowdsec
	Captcha Not Working

	Ensure Turnstile is configured correctly with valid site and secret keys
	Verify the captcha.html file exists in the correct location
	Check if the turnstile script is loading in browser developer tools
	CrowdSec Not Detecting Attacks

	Verify log paths are correct in acquis.yaml
	Check if logs are being written: `docker exec -it crowdsec ls -l /var/log/traefik/`
	Make sure Traefik’s accessLog is enabled and in JSON format
	Check if collections are installed: `docker exec crowdsec cscli collections list`
	Useful Commands for Monitoring and Troubleshooting

	```
	# View CrowdSec overview
	docker exec crowdsec cscli status

	# Check which collections are installed
	docker exec crowdsec cscli collections list

	# Monitor CrowdSec resources
	docker stats crowdsec

	# Check AppSec metrics
	curl http://localhost:6060/metrics \| grep appsec

	# View Traefik logs
	docker exec -it crowdsec ls -l /var/log/traefik/

	# Check CrowdSec metrics
	docker exec -it crowdsec cscli metrics

	# View active decisions
	docker exec -it crowdsec cscli decisions list

	# Monitor CrowdSec logs
	docker exec -it crowdsec tail -f /var/log/traefik/access.log

	# Manually add decisions for testing
	docker exec crowdsec cscli decisions add --ip <IP> --type captcha -d 1h
	docker exec crowdsec cscli decisions add -i <IP> -t ban -d 1h

	# Monitor Traefik logs
	docker compose logs traefik -f

	# Restart services
	docker compose restart traefik crowdsec

	# View/manage bouncers
	docker exec crowdsec cscli bouncers list
	docker exec crowdsec cscli bouncers add traefik-bouncer
	docker exec crowdsec cscli bouncers delete traefik-bouncer
	```

	17) Ongoing Maintenance and Advanced Configuration
	Regular Maintenance Tasks
	Keep CrowdSec Updated

	docker compose pull crowdsec
	docker compose up -d
	Update Collections and Parsers

	docker exec crowdsec cscli hub update
	docker exec crowdsec cscli collections upgrade
	Monitor for False Positives
	Regularly check decisions to ensure legitimate users aren’t being blocked:

	docker exec crowdsec cscli decisions list
	Create Allowlists for Trusted IPs
	Add your trusted infrastructure to avoid false positives:

	docker exec crowdsec cscli ipset add -f your-trusted-ips.txt
	Advanced Configuration Options
	Add Custom Scenarios
	You can create custom detection rules in YAML format in /etc/crowdsec/scenarios/.

	Configure Notifications
	Set up notifications for attacks via Slack, Discord, or email in profiles.yaml.

	Fine-tune Remediation Profiles
	Adjust ban durations and captcha settings in profiles.yaml based on your security requirements.

	Implement Geolocation-based Rules
	Use the GeoIP enricher to create country-specific rules:

	``
	docker exec crowdsec cscli collections install crowdsecurity/geoip-enrich
	```
No results found