From fad38a229f0c57ec8e44f288cf2445c3b0112c5a Mon Sep 17 00:00:00 2001 From: John Date: Mon, 17 Nov 2025 23:22:55 +0000 Subject: [PATCH] Moved config to core --- crowdsec/GUIDE.md | 478 ---------------------------------------------- 1 file changed, 478 deletions(-) delete mode 100644 crowdsec/GUIDE.md diff --git a/crowdsec/GUIDE.md b/crowdsec/GUIDE.md deleted file mode 100644 index 8a14fdb..0000000 --- a/crowdsec/GUIDE.md +++ /dev/null @@ -1,478 +0,0 @@ -# CrowdSec with Traefik - User Guide - -## Table of Contents -1. [Overview](#overview) -2. [Quick Reference](#quick-reference) -3. [Managing Decisions (Bans)](#managing-decisions-bans) -4. [Applying Protection to Services](#applying-protection-to-services) -5. [Monitoring & Metrics](#monitoring--metrics) -6. [Collections & Scenarios](#collections--scenarios) -7. [Community Integration](#community-integration) -8. [Troubleshooting](#troubleshooting) - -## Overview - -CrowdSec is a collaborative IPS (Intrusion Prevention System) that protects your services by: -- Analyzing Traefik access logs for malicious patterns -- Automatically banning suspicious IPs -- Sharing threat intelligence with the community -- Providing centralized protection across all Traefik-proxied services - -**Architecture:** -- **CrowdSec LAPI** (crowdsec_crowdsec) - Analyzes logs and maintains ban list -- **Traefik Bouncer** - Middleware that queries LAPI and blocks banned IPs -- **Access Logs** - Traefik logs analyzed by CrowdSec at `/home/doc/projects/swarm-data/traefik/logs/access.log` - -## Quick Reference - -### Access CrowdSec CLI - -All commands must be run on the node where CrowdSec is deployed (p3-control): - -```bash -# SSH to the CrowdSec node -ssh 10.0.4.14 - -# Run cscli commands -docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli [command] -``` - -**Shortcut alias (add to .bashrc):** -```bash -alias cscli='ssh 10.0.4.14 "docker exec \$(docker ps -qf name=crowdsec_crowdsec) cscli"' -``` - -### Essential Commands - -```bash -# View all active bans -cscli decisions list - -# View CrowdSec status -cscli metrics - -# View detailed acquisition metrics -cscli metrics show acquisition - -# List registered bouncers -cscli bouncers list - -# View recent alerts -cscli alerts list - -# Check hub status -cscli hub list -``` - -## Managing Decisions (Bans) - -### View Active Bans - -```bash -# List all active decisions -ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli decisions list' - -# Filter by IP -ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli decisions list --ip 1.2.3.4' - -# Filter by type -ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli decisions list --type ban' -``` - -### Manually Ban an IP - -```bash -# Ban for 4 hours -ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli decisions add --ip 1.2.3.4 --duration 4h --reason "Manual ban"' - -# Ban for 1 day -ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli decisions add --ip 1.2.3.4 --duration 24h' - -# Permanent ban -ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli decisions add --ip 1.2.3.4 --duration 0' - -# Ban entire subnet -ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli decisions add --range 1.2.3.0/24 --duration 4h' -``` - -### Unban (Delete Decision) - -```bash -# Unban specific IP -ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli decisions delete --ip 1.2.3.4' - -# Delete all decisions -ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli decisions delete --all' -``` - -### View Alerts - -Alerts show what triggered a ban: - -```bash -# List recent alerts -ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli alerts list' - -# View alert details -ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli alerts inspect ' - -# Filter alerts by IP -ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli alerts list --ip 1.2.3.4' -``` - -## Applying Protection to Services - -### Protect Docker Swarm Services - -To protect any service proxied by Traefik, add the `crowdsec` middleware to its labels: - -**Example for a Docker Swarm service:** - -```yaml -services: - myapp: - image: myapp:latest - networks: - - frostlabs - deploy: - labels: - - "traefik.enable=true" - - "traefik.http.routers.myapp.rule=Host(`myapp.frostlabs.me`)" - - "traefik.http.routers.myapp.entrypoints=websecure" - - "traefik.http.routers.myapp.tls.certresolver=cloudflare" - # Add CrowdSec protection - - "traefik.http.routers.myapp.middlewares=crowdsec@file" - - "traefik.http.services.myapp.loadbalancer.server.port=8080" -``` - -### Protect External Services (dynamic.yml) - -For services defined in `/home/doc/projects/homelab/frostlabs/traefik/dynamic.yml`: - -```yaml -http: - routers: - myservice: - rule: "Host(`myservice.frostlabs.me`)" - entryPoints: - - websecure - service: myservice - middlewares: - - crowdsec # Add this line - tls: - certResolver: cloudflare -``` - -### Chain Multiple Middlewares - -```yaml -# In dynamic.yml -http: - routers: - protected-service: - middlewares: - - authentik # First: Authentication - - crowdsec # Second: IP filtering - - rate-limit # Third: Rate limiting -``` - -After making changes, reload Traefik: -```bash -docker service update --force traefik_traefik -``` - -## Monitoring & Metrics - -### Real-time Monitoring - -```bash -# Overall metrics -ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli metrics' - -# Log parsing metrics -ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli metrics show acquisition' - -# Scenario metrics (what's triggering) -ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli metrics show scenarios' - -# LAPI metrics (bouncer queries) -ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli metrics show lapi' -``` - -### View Logs - -```bash -# CrowdSec service logs -docker service logs crowdsec_crowdsec --tail 50 --follow - -# Traefik logs (on control node) -docker service logs traefik_traefik --tail 50 --follow - -# Access log (what CrowdSec analyzes) -tail -f /home/doc/projects/swarm-data/traefik/logs/access.log -``` - -### Check Bouncer Status - -```bash -# Verify bouncer is connected -ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli bouncers list' - -# Expected output: -# Name IP Address Valid Last API pull Type Version Auth Type -# traefik-bouncer ✔️ api-key -``` - -## Collections & Scenarios - -### View Installed Collections - -Collections are pre-packaged sets of parsers and scenarios: - -```bash -# List all collections -ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli collections list' - -# Currently installed: -# - crowdsecurity/traefik (Traefik-specific patterns) -# - crowdsecurity/http-cve (Common CVE exploits) -``` - -### Install Additional Collections - -```bash -# Install SSH protection (if you expose SSH) -ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli collections install crowdsecurity/sshd' - -# Install WordPress protection -ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli collections install crowdsecurity/wordpress' - -# Install additional HTTP protections -ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli collections install crowdsecurity/base-http-scenarios' - -# Browse available collections -ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli collections list -a' -``` - -After installing collections, restart CrowdSec: -```bash -docker service update --force crowdsec_crowdsec -``` - -### View Active Scenarios - -Scenarios are detection rules: - -```bash -# List enabled scenarios -ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli scenarios list' - -# Browse available scenarios -ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli scenarios list -a' - -# Install specific scenario -ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli scenarios install crowdsecurity/http-probing' -``` - -## Community Integration - -### Enroll in CrowdSec Console - -The Console provides centralized management and community blocklists: - -1. **Create account:** https://app.crowdsec.net/signup - -2. **Enroll your instance:** -```bash -ssh 10.0.4.14 'docker exec -it $(docker ps -qf name=crowdsec_crowdsec) cscli console enroll ' -``` - -3. **Verify enrollment:** -```bash -ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli console status' -``` - -### Benefits of Enrollment - -- **Community Blocklist:** Automatically receive bans from the CrowdSec network -- **Centralized Dashboard:** View all your instances in one place -- **Alert Management:** Get notifications for attacks -- **Analytics:** Detailed attack reports and trends - -### Subscribe to Blocklists - -After enrollment, subscribe to community blocklists: - -```bash -# Install blocklist collection -ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli collections install crowdsecurity/seo-bots-whitelist' - -# Enable community blocklist -ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli capi register' -``` - -## Troubleshooting - -### CrowdSec Not Reading Logs - -**Check if logs exist:** -```bash -ls -lh /home/doc/projects/swarm-data/traefik/logs/ -``` - -**Verify acquisition:** -```bash -ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli metrics show acquisition' -``` - -**Restart CrowdSec:** -```bash -docker service update --force crowdsec_crowdsec -``` - -### Bouncer Not Blocking - -**Check bouncer is registered:** -```bash -ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli bouncers list' -``` - -**Verify API key in Traefik config:** -```bash -cat /home/doc/projects/homelab/frostlabs/traefik/dynamic.yml | grep -A 5 crowdseclapikey -``` - -**Check Traefik can reach CrowdSec:** -```bash -docker service logs traefik_traefik --tail 50 | grep -i crowdsec -``` - -### View Decision That Would Block You - -**Test if an IP is banned:** -```bash -ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli decisions list --ip ' -``` - -**Whitelist your IP permanently:** - -Edit `/home/doc/projects/homelab/frostlabs/crowdsec/acquis.yaml` and add to CrowdSec config: - -```bash -# Create whitelist file -ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) sh -c "echo \"name: crowdsecurity/whitelists -description: Whitelist trusted IPs -whitelist: - reason: trusted network - ip: - - 10.0.1.0/24 - - 10.0.4.0/24 - - - cidr: - - 10.0.0.0/8 -\" > /etc/crowdsec/parsers/s02-enrich/whitelists.yaml"' - -# Reload CrowdSec -docker service update --force crowdsec_crowdsec -``` - -### Performance Issues - -**Check resource usage:** -```bash -docker stats $(docker ps -qf name=crowdsec_crowdsec) -``` - -**Reduce log verbosity in static.yml:** -```yaml -log: - level: ERROR # Change from INFO -``` - -### Reset CrowdSec Completely - -```bash -# Remove all decisions -ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli decisions delete --all' - -# Remove CrowdSec data -rm -rf /home/doc/projects/swarm-data/crowdsec/* - -# Redeploy -cd /home/doc/projects/homelab/frostlabs/crowdsec -docker stack rm crowdsec -sleep 10 -docker stack deploy -c stack.yml crowdsec - -# Regenerate bouncer key (wait 30 seconds first) -ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli bouncers add traefik-bouncer' - -# Update API key in /home/doc/projects/homelab/frostlabs/traefik/dynamic.yml -# Then redeploy Traefik -docker stack deploy -c stack.yml traefik -``` - -## Configuration Files - -**Location:** `/home/doc/projects/homelab/frostlabs/crowdsec/` - -- `stack.yml` - Docker Swarm service definition -- `acquis.yaml` - Log sources configuration - -**Persistent Data:** `/home/doc/projects/swarm-data/crowdsec/` - -- `config/` - CrowdSec configuration -- `data/` - Decision database and state - -**Traefik Logs:** `/home/doc/projects/swarm-data/traefik/logs/` - -- `access.log` - Analyzed by CrowdSec - -## Best Practices - -1. **Start Conservative:** Monitor alerts before enabling aggressive scenarios -2. **Whitelist Trusted IPs:** Add your networks to `clientTrustedIPs` in dynamic.yml -3. **Test Changes:** Use `cscli decisions add` to test bans before deploying -4. **Monitor Metrics:** Regularly check `cscli metrics` to understand what's being blocked -5. **Update Regularly:** Keep collections updated with `cscli hub update && cscli hub upgrade` -6. **Enable Console:** Enroll for community protection and centralized management -7. **Apply Selectively:** Not all services need CrowdSec - use for public-facing services -8. **Combine Layers:** Use with authentik for authentication and rate-limit for additional protection - -## Support & Resources - -- **CrowdSec Documentation:** https://docs.crowdsec.net/ -- **Hub (Collections/Scenarios):** https://hub.crowdsec.net/ -- **Community Forum:** https://discourse.crowdsec.net/ -- **Traefik Bouncer Plugin:** https://github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin - -## Maintenance Tasks - -### Weekly - -```bash -# Check for new threats -ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli alerts list | head -20' - -# Review metrics -ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli metrics' -``` - -### Monthly - -```bash -# Update hub -ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli hub update' -ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli hub upgrade' - -# Update Docker images -docker service update --image crowdsecurity/crowdsec:latest crowdsec_crowdsec -docker service update --image traefik:latest traefik_traefik -``` - -### Quarterly - -- Review and tune scenarios based on false positives -- Audit whitelisted IPs -- Check for new relevant collections -- Review ban durations in scenarios