Crowdsec Deployed to Production + Guides
This commit is contained in:
John
478
crowdsec/GUIDE.md
Normal file
478
crowdsec/GUIDE.md
Normal file
@@ -0,0 +1,478 @@
|
||||
# CrowdSec with Traefik - User Guide
|
||||
|
||||
## Table of Contents
|
||||
1. [Overview](#overview)
|
||||
2. [Quick Reference](#quick-reference)
|
||||
3. [Managing Decisions (Bans)](#managing-decisions-bans)
|
||||
4. [Applying Protection to Services](#applying-protection-to-services)
|
||||
5. [Monitoring & Metrics](#monitoring--metrics)
|
||||
6. [Collections & Scenarios](#collections--scenarios)
|
||||
7. [Community Integration](#community-integration)
|
||||
8. [Troubleshooting](#troubleshooting)
|
||||
|
||||
## Overview
|
||||
|
||||
CrowdSec is a collaborative IPS (Intrusion Prevention System) that protects your services by:
|
||||
- Analyzing Traefik access logs for malicious patterns
|
||||
- Automatically banning suspicious IPs
|
||||
- Sharing threat intelligence with the community
|
||||
- Providing centralized protection across all Traefik-proxied services
|
||||
|
||||
**Architecture:**
|
||||
- **CrowdSec LAPI** (crowdsec_crowdsec) - Analyzes logs and maintains ban list
|
||||
- **Traefik Bouncer** - Middleware that queries LAPI and blocks banned IPs
|
||||
- **Access Logs** - Traefik logs analyzed by CrowdSec at `/home/doc/projects/swarm-data/traefik/logs/access.log`
|
||||
|
||||
## Quick Reference
|
||||
|
||||
### Access CrowdSec CLI
|
||||
|
||||
All commands must be run on the node where CrowdSec is deployed (p3-control):
|
||||
|
||||
```bash
|
||||
# SSH to the CrowdSec node
|
||||
ssh 10.0.4.14
|
||||
|
||||
# Run cscli commands
|
||||
docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli [command]
|
||||
```
|
||||
|
||||
**Shortcut alias (add to .bashrc):**
|
||||
```bash
|
||||
alias cscli='ssh 10.0.4.14 "docker exec \$(docker ps -qf name=crowdsec_crowdsec) cscli"'
|
||||
```
|
||||
|
||||
### Essential Commands
|
||||
|
||||
```bash
|
||||
# View all active bans
|
||||
cscli decisions list
|
||||
|
||||
# View CrowdSec status
|
||||
cscli metrics
|
||||
|
||||
# View detailed acquisition metrics
|
||||
cscli metrics show acquisition
|
||||
|
||||
# List registered bouncers
|
||||
cscli bouncers list
|
||||
|
||||
# View recent alerts
|
||||
cscli alerts list
|
||||
|
||||
# Check hub status
|
||||
cscli hub list
|
||||
```
|
||||
|
||||
## Managing Decisions (Bans)
|
||||
|
||||
### View Active Bans
|
||||
|
||||
```bash
|
||||
# List all active decisions
|
||||
ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli decisions list'
|
||||
|
||||
# Filter by IP
|
||||
ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli decisions list --ip 1.2.3.4'
|
||||
|
||||
# Filter by type
|
||||
ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli decisions list --type ban'
|
||||
```
|
||||
|
||||
### Manually Ban an IP
|
||||
|
||||
```bash
|
||||
# Ban for 4 hours
|
||||
ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli decisions add --ip 1.2.3.4 --duration 4h --reason "Manual ban"'
|
||||
|
||||
# Ban for 1 day
|
||||
ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli decisions add --ip 1.2.3.4 --duration 24h'
|
||||
|
||||
# Permanent ban
|
||||
ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli decisions add --ip 1.2.3.4 --duration 0'
|
||||
|
||||
# Ban entire subnet
|
||||
ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli decisions add --range 1.2.3.0/24 --duration 4h'
|
||||
```
|
||||
|
||||
### Unban (Delete Decision)
|
||||
|
||||
```bash
|
||||
# Unban specific IP
|
||||
ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli decisions delete --ip 1.2.3.4'
|
||||
|
||||
# Delete all decisions
|
||||
ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli decisions delete --all'
|
||||
```
|
||||
|
||||
### View Alerts
|
||||
|
||||
Alerts show what triggered a ban:
|
||||
|
||||
```bash
|
||||
# List recent alerts
|
||||
ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli alerts list'
|
||||
|
||||
# View alert details
|
||||
ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli alerts inspect <alert_id>'
|
||||
|
||||
# Filter alerts by IP
|
||||
ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli alerts list --ip 1.2.3.4'
|
||||
```
|
||||
|
||||
## Applying Protection to Services
|
||||
|
||||
### Protect Docker Swarm Services
|
||||
|
||||
To protect any service proxied by Traefik, add the `crowdsec` middleware to its labels:
|
||||
|
||||
**Example for a Docker Swarm service:**
|
||||
|
||||
```yaml
|
||||
services:
|
||||
myapp:
|
||||
image: myapp:latest
|
||||
networks:
|
||||
- frostlabs
|
||||
deploy:
|
||||
labels:
|
||||
- "traefik.enable=true"
|
||||
- "traefik.http.routers.myapp.rule=Host(`myapp.frostlabs.me`)"
|
||||
- "traefik.http.routers.myapp.entrypoints=websecure"
|
||||
- "traefik.http.routers.myapp.tls.certresolver=cloudflare"
|
||||
# Add CrowdSec protection
|
||||
- "traefik.http.routers.myapp.middlewares=crowdsec@file"
|
||||
- "traefik.http.services.myapp.loadbalancer.server.port=8080"
|
||||
```
|
||||
|
||||
### Protect External Services (dynamic.yml)
|
||||
|
||||
For services defined in `/home/doc/projects/homelab/frostlabs/traefik/dynamic.yml`:
|
||||
|
||||
```yaml
|
||||
http:
|
||||
routers:
|
||||
myservice:
|
||||
rule: "Host(`myservice.frostlabs.me`)"
|
||||
entryPoints:
|
||||
- websecure
|
||||
service: myservice
|
||||
middlewares:
|
||||
- crowdsec # Add this line
|
||||
tls:
|
||||
certResolver: cloudflare
|
||||
```
|
||||
|
||||
### Chain Multiple Middlewares
|
||||
|
||||
```yaml
|
||||
# In dynamic.yml
|
||||
http:
|
||||
routers:
|
||||
protected-service:
|
||||
middlewares:
|
||||
- authentik # First: Authentication
|
||||
- crowdsec # Second: IP filtering
|
||||
- rate-limit # Third: Rate limiting
|
||||
```
|
||||
|
||||
After making changes, reload Traefik:
|
||||
```bash
|
||||
docker service update --force traefik_traefik
|
||||
```
|
||||
|
||||
## Monitoring & Metrics
|
||||
|
||||
### Real-time Monitoring
|
||||
|
||||
```bash
|
||||
# Overall metrics
|
||||
ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli metrics'
|
||||
|
||||
# Log parsing metrics
|
||||
ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli metrics show acquisition'
|
||||
|
||||
# Scenario metrics (what's triggering)
|
||||
ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli metrics show scenarios'
|
||||
|
||||
# LAPI metrics (bouncer queries)
|
||||
ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli metrics show lapi'
|
||||
```
|
||||
|
||||
### View Logs
|
||||
|
||||
```bash
|
||||
# CrowdSec service logs
|
||||
docker service logs crowdsec_crowdsec --tail 50 --follow
|
||||
|
||||
# Traefik logs (on control node)
|
||||
docker service logs traefik_traefik --tail 50 --follow
|
||||
|
||||
# Access log (what CrowdSec analyzes)
|
||||
tail -f /home/doc/projects/swarm-data/traefik/logs/access.log
|
||||
```
|
||||
|
||||
### Check Bouncer Status
|
||||
|
||||
```bash
|
||||
# Verify bouncer is connected
|
||||
ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli bouncers list'
|
||||
|
||||
# Expected output:
|
||||
# Name IP Address Valid Last API pull Type Version Auth Type
|
||||
# traefik-bouncer ✔️ <timestamp> api-key
|
||||
```
|
||||
|
||||
## Collections & Scenarios
|
||||
|
||||
### View Installed Collections
|
||||
|
||||
Collections are pre-packaged sets of parsers and scenarios:
|
||||
|
||||
```bash
|
||||
# List all collections
|
||||
ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli collections list'
|
||||
|
||||
# Currently installed:
|
||||
# - crowdsecurity/traefik (Traefik-specific patterns)
|
||||
# - crowdsecurity/http-cve (Common CVE exploits)
|
||||
```
|
||||
|
||||
### Install Additional Collections
|
||||
|
||||
```bash
|
||||
# Install SSH protection (if you expose SSH)
|
||||
ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli collections install crowdsecurity/sshd'
|
||||
|
||||
# Install WordPress protection
|
||||
ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli collections install crowdsecurity/wordpress'
|
||||
|
||||
# Install additional HTTP protections
|
||||
ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli collections install crowdsecurity/base-http-scenarios'
|
||||
|
||||
# Browse available collections
|
||||
ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli collections list -a'
|
||||
```
|
||||
|
||||
After installing collections, restart CrowdSec:
|
||||
```bash
|
||||
docker service update --force crowdsec_crowdsec
|
||||
```
|
||||
|
||||
### View Active Scenarios
|
||||
|
||||
Scenarios are detection rules:
|
||||
|
||||
```bash
|
||||
# List enabled scenarios
|
||||
ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli scenarios list'
|
||||
|
||||
# Browse available scenarios
|
||||
ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli scenarios list -a'
|
||||
|
||||
# Install specific scenario
|
||||
ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli scenarios install crowdsecurity/http-probing'
|
||||
```
|
||||
|
||||
## Community Integration
|
||||
|
||||
### Enroll in CrowdSec Console
|
||||
|
||||
The Console provides centralized management and community blocklists:
|
||||
|
||||
1. **Create account:** https://app.crowdsec.net/signup
|
||||
|
||||
2. **Enroll your instance:**
|
||||
```bash
|
||||
ssh 10.0.4.14 'docker exec -it $(docker ps -qf name=crowdsec_crowdsec) cscli console enroll <enrollment_key>'
|
||||
```
|
||||
|
||||
3. **Verify enrollment:**
|
||||
```bash
|
||||
ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli console status'
|
||||
```
|
||||
|
||||
### Benefits of Enrollment
|
||||
|
||||
- **Community Blocklist:** Automatically receive bans from the CrowdSec network
|
||||
- **Centralized Dashboard:** View all your instances in one place
|
||||
- **Alert Management:** Get notifications for attacks
|
||||
- **Analytics:** Detailed attack reports and trends
|
||||
|
||||
### Subscribe to Blocklists
|
||||
|
||||
After enrollment, subscribe to community blocklists:
|
||||
|
||||
```bash
|
||||
# Install blocklist collection
|
||||
ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli collections install crowdsecurity/seo-bots-whitelist'
|
||||
|
||||
# Enable community blocklist
|
||||
ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli capi register'
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### CrowdSec Not Reading Logs
|
||||
|
||||
**Check if logs exist:**
|
||||
```bash
|
||||
ls -lh /home/doc/projects/swarm-data/traefik/logs/
|
||||
```
|
||||
|
||||
**Verify acquisition:**
|
||||
```bash
|
||||
ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli metrics show acquisition'
|
||||
```
|
||||
|
||||
**Restart CrowdSec:**
|
||||
```bash
|
||||
docker service update --force crowdsec_crowdsec
|
||||
```
|
||||
|
||||
### Bouncer Not Blocking
|
||||
|
||||
**Check bouncer is registered:**
|
||||
```bash
|
||||
ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli bouncers list'
|
||||
```
|
||||
|
||||
**Verify API key in Traefik config:**
|
||||
```bash
|
||||
cat /home/doc/projects/homelab/frostlabs/traefik/dynamic.yml | grep -A 5 crowdseclapikey
|
||||
```
|
||||
|
||||
**Check Traefik can reach CrowdSec:**
|
||||
```bash
|
||||
docker service logs traefik_traefik --tail 50 | grep -i crowdsec
|
||||
```
|
||||
|
||||
### View Decision That Would Block You
|
||||
|
||||
**Test if an IP is banned:**
|
||||
```bash
|
||||
ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli decisions list --ip <your_ip>'
|
||||
```
|
||||
|
||||
**Whitelist your IP permanently:**
|
||||
|
||||
Edit `/home/doc/projects/homelab/frostlabs/crowdsec/acquis.yaml` and add to CrowdSec config:
|
||||
|
||||
```bash
|
||||
# Create whitelist file
|
||||
ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) sh -c "echo \"name: crowdsecurity/whitelists
|
||||
description: Whitelist trusted IPs
|
||||
whitelist:
|
||||
reason: trusted network
|
||||
ip:
|
||||
- 10.0.1.0/24
|
||||
- 10.0.4.0/24
|
||||
- <your_public_ip>
|
||||
cidr:
|
||||
- 10.0.0.0/8
|
||||
\" > /etc/crowdsec/parsers/s02-enrich/whitelists.yaml"'
|
||||
|
||||
# Reload CrowdSec
|
||||
docker service update --force crowdsec_crowdsec
|
||||
```
|
||||
|
||||
### Performance Issues
|
||||
|
||||
**Check resource usage:**
|
||||
```bash
|
||||
docker stats $(docker ps -qf name=crowdsec_crowdsec)
|
||||
```
|
||||
|
||||
**Reduce log verbosity in static.yml:**
|
||||
```yaml
|
||||
log:
|
||||
level: ERROR # Change from INFO
|
||||
```
|
||||
|
||||
### Reset CrowdSec Completely
|
||||
|
||||
```bash
|
||||
# Remove all decisions
|
||||
ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli decisions delete --all'
|
||||
|
||||
# Remove CrowdSec data
|
||||
rm -rf /home/doc/projects/swarm-data/crowdsec/*
|
||||
|
||||
# Redeploy
|
||||
cd /home/doc/projects/homelab/frostlabs/crowdsec
|
||||
docker stack rm crowdsec
|
||||
sleep 10
|
||||
docker stack deploy -c stack.yml crowdsec
|
||||
|
||||
# Regenerate bouncer key (wait 30 seconds first)
|
||||
ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli bouncers add traefik-bouncer'
|
||||
|
||||
# Update API key in /home/doc/projects/homelab/frostlabs/traefik/dynamic.yml
|
||||
# Then redeploy Traefik
|
||||
docker stack deploy -c stack.yml traefik
|
||||
```
|
||||
|
||||
## Configuration Files
|
||||
|
||||
**Location:** `/home/doc/projects/homelab/frostlabs/crowdsec/`
|
||||
|
||||
- `stack.yml` - Docker Swarm service definition
|
||||
- `acquis.yaml` - Log sources configuration
|
||||
|
||||
**Persistent Data:** `/home/doc/projects/swarm-data/crowdsec/`
|
||||
|
||||
- `config/` - CrowdSec configuration
|
||||
- `data/` - Decision database and state
|
||||
|
||||
**Traefik Logs:** `/home/doc/projects/swarm-data/traefik/logs/`
|
||||
|
||||
- `access.log` - Analyzed by CrowdSec
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Start Conservative:** Monitor alerts before enabling aggressive scenarios
|
||||
2. **Whitelist Trusted IPs:** Add your networks to `clientTrustedIPs` in dynamic.yml
|
||||
3. **Test Changes:** Use `cscli decisions add` to test bans before deploying
|
||||
4. **Monitor Metrics:** Regularly check `cscli metrics` to understand what's being blocked
|
||||
5. **Update Regularly:** Keep collections updated with `cscli hub update && cscli hub upgrade`
|
||||
6. **Enable Console:** Enroll for community protection and centralized management
|
||||
7. **Apply Selectively:** Not all services need CrowdSec - use for public-facing services
|
||||
8. **Combine Layers:** Use with authentik for authentication and rate-limit for additional protection
|
||||
|
||||
## Support & Resources
|
||||
|
||||
- **CrowdSec Documentation:** https://docs.crowdsec.net/
|
||||
- **Hub (Collections/Scenarios):** https://hub.crowdsec.net/
|
||||
- **Community Forum:** https://discourse.crowdsec.net/
|
||||
- **Traefik Bouncer Plugin:** https://github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin
|
||||
|
||||
## Maintenance Tasks
|
||||
|
||||
### Weekly
|
||||
|
||||
```bash
|
||||
# Check for new threats
|
||||
ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli alerts list | head -20'
|
||||
|
||||
# Review metrics
|
||||
ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli metrics'
|
||||
```
|
||||
|
||||
### Monthly
|
||||
|
||||
```bash
|
||||
# Update hub
|
||||
ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli hub update'
|
||||
ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli hub upgrade'
|
||||
|
||||
# Update Docker images
|
||||
docker service update --image crowdsecurity/crowdsec:latest crowdsec_crowdsec
|
||||
docker service update --image traefik:latest traefik_traefik
|
||||
```
|
||||
|
||||
### Quarterly
|
||||
|
||||
- Review and tune scenarios based on false positives
|
||||
- Audit whitelisted IPs
|
||||
- Check for new relevant collections
|
||||
- Review ban durations in scenarios
|
||||
Reference in New Issue
Block a user