jpallison0625/frostlabs
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Crowdsec Deployed to Production + Guides

Browse Source
This commit is contained in:
John
2025-11-14 14:01:47 -05:00
parent 778c5531ed
commit 6e57ee18d7
3 changed files with 649 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

478
crowdsec/GUIDE.md Normal file
View File

@@ -0,0 +1,478 @@
# CrowdSec with Traefik - User Guide
## Table of Contents
1. [Overview](#overview)
2. [Quick Reference](#quick-reference)
3. [Managing Decisions (Bans)](#managing-decisions-bans)
4. [Applying Protection to Services](#applying-protection-to-services)
5. [Monitoring & Metrics](#monitoring--metrics)
6. [Collections & Scenarios](#collections--scenarios)
7. [Community Integration](#community-integration)
8. [Troubleshooting](#troubleshooting)
## Overview
CrowdSec is a collaborative IPS (Intrusion Prevention System) that protects your services by:
- Analyzing Traefik access logs for malicious patterns
- Automatically banning suspicious IPs
- Sharing threat intelligence with the community
- Providing centralized protection across all Traefik-proxied services
**Architecture:**
- **CrowdSec LAPI** (crowdsec_crowdsec) - Analyzes logs and maintains ban list
- **Traefik Bouncer** - Middleware that queries LAPI and blocks banned IPs
- **Access Logs** - Traefik logs analyzed by CrowdSec at `/home/doc/projects/swarm-data/traefik/logs/access.log`
## Quick Reference
### Access CrowdSec CLI
All commands must be run on the node where CrowdSec is deployed (p3-control):
```bash
# SSH to the CrowdSec node
ssh 10.0.4.14
# Run cscli commands
docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli [command]
```
**Shortcut alias (add to .bashrc):**
```bash
alias cscli='ssh 10.0.4.14 "docker exec \$(docker ps -qf name=crowdsec_crowdsec) cscli"'
```
### Essential Commands
```bash
# View all active bans
cscli decisions list
# View CrowdSec status
cscli metrics
# View detailed acquisition metrics
cscli metrics show acquisition
# List registered bouncers
cscli bouncers list
# View recent alerts
cscli alerts list
# Check hub status
cscli hub list
```
## Managing Decisions (Bans)
### View Active Bans
```bash
# List all active decisions
ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli decisions list'
# Filter by IP
ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli decisions list --ip 1.2.3.4'
# Filter by type
ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli decisions list --type ban'
```
### Manually Ban an IP
```bash
# Ban for 4 hours
ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli decisions add --ip 1.2.3.4 --duration 4h --reason "Manual ban"'
# Ban for 1 day
ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli decisions add --ip 1.2.3.4 --duration 24h'
# Permanent ban
ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli decisions add --ip 1.2.3.4 --duration 0'
# Ban entire subnet
ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli decisions add --range 1.2.3.0/24 --duration 4h'
```
### Unban (Delete Decision)
```bash
# Unban specific IP
ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli decisions delete --ip 1.2.3.4'
# Delete all decisions
ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli decisions delete --all'
```
### View Alerts
Alerts show what triggered a ban:
```bash
# List recent alerts
ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli alerts list'
# View alert details
ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli alerts inspect <alert_id>'
# Filter alerts by IP
ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli alerts list --ip 1.2.3.4'
```
## Applying Protection to Services
### Protect Docker Swarm Services
To protect any service proxied by Traefik, add the `crowdsec` middleware to its labels:
**Example for a Docker Swarm service:**
```yaml
services:
myapp:
image: myapp:latest
networks:
- frostlabs
deploy:
labels:
- "traefik.enable=true"
- "traefik.http.routers.myapp.rule=Host(`myapp.frostlabs.me`)"
- "traefik.http.routers.myapp.entrypoints=websecure"
- "traefik.http.routers.myapp.tls.certresolver=cloudflare"
# Add CrowdSec protection
- "traefik.http.routers.myapp.middlewares=crowdsec@file"
- "traefik.http.services.myapp.loadbalancer.server.port=8080"
```
### Protect External Services (dynamic.yml)
For services defined in `/home/doc/projects/homelab/frostlabs/traefik/dynamic.yml`:
```yaml
http:
routers:
myservice:
rule: "Host(`myservice.frostlabs.me`)"
entryPoints:
- websecure
service: myservice
middlewares:
- crowdsec # Add this line
tls:
certResolver: cloudflare
```
### Chain Multiple Middlewares
```yaml
# In dynamic.yml
http:
routers:
protected-service:
middlewares:
- authentik # First: Authentication
- crowdsec # Second: IP filtering
- rate-limit # Third: Rate limiting
```
After making changes, reload Traefik:
```bash
docker service update --force traefik_traefik
```
## Monitoring & Metrics
### Real-time Monitoring
```bash
# Overall metrics
ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli metrics'
# Log parsing metrics
ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli metrics show acquisition'
# Scenario metrics (what's triggering)
ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli metrics show scenarios'
# LAPI metrics (bouncer queries)
ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli metrics show lapi'
```
### View Logs
```bash
# CrowdSec service logs
docker service logs crowdsec_crowdsec --tail 50 --follow
# Traefik logs (on control node)
docker service logs traefik_traefik --tail 50 --follow
# Access log (what CrowdSec analyzes)
tail -f /home/doc/projects/swarm-data/traefik/logs/access.log
```
### Check Bouncer Status
```bash
# Verify bouncer is connected
ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli bouncers list'
# Expected output:
# Name IP Address Valid Last API pull Type Version Auth Type
# traefik-bouncer ✔️ <timestamp> api-key
```
## Collections & Scenarios
### View Installed Collections
Collections are pre-packaged sets of parsers and scenarios:
```bash
# List all collections
ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli collections list'
# Currently installed:
# - crowdsecurity/traefik (Traefik-specific patterns)
# - crowdsecurity/http-cve (Common CVE exploits)
```
### Install Additional Collections
```bash
# Install SSH protection (if you expose SSH)
ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli collections install crowdsecurity/sshd'
# Install WordPress protection
ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli collections install crowdsecurity/wordpress'
# Install additional HTTP protections
ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli collections install crowdsecurity/base-http-scenarios'
# Browse available collections
ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli collections list -a'
```
After installing collections, restart CrowdSec:
```bash
docker service update --force crowdsec_crowdsec
```
### View Active Scenarios
Scenarios are detection rules:
```bash
# List enabled scenarios
ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli scenarios list'
# Browse available scenarios
ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli scenarios list -a'
# Install specific scenario
ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli scenarios install crowdsecurity/http-probing'
```
## Community Integration
### Enroll in CrowdSec Console
The Console provides centralized management and community blocklists:
1. **Create account:** https://app.crowdsec.net/signup
2. **Enroll your instance:**
```bash
ssh 10.0.4.14 'docker exec -it $(docker ps -qf name=crowdsec_crowdsec) cscli console enroll <enrollment_key>'
```
3. **Verify enrollment:**
```bash
ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli console status'
```
### Benefits of Enrollment
- **Community Blocklist:** Automatically receive bans from the CrowdSec network
- **Centralized Dashboard:** View all your instances in one place
- **Alert Management:** Get notifications for attacks
- **Analytics:** Detailed attack reports and trends
### Subscribe to Blocklists
After enrollment, subscribe to community blocklists:
```bash
# Install blocklist collection
ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli collections install crowdsecurity/seo-bots-whitelist'
# Enable community blocklist
ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli capi register'
```
## Troubleshooting
### CrowdSec Not Reading Logs
**Check if logs exist:**
```bash
ls -lh /home/doc/projects/swarm-data/traefik/logs/
```
**Verify acquisition:**
```bash
ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli metrics show acquisition'
```
**Restart CrowdSec:**
```bash
docker service update --force crowdsec_crowdsec
```
### Bouncer Not Blocking
**Check bouncer is registered:**
```bash
ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli bouncers list'
```
**Verify API key in Traefik config:**
```bash
cat /home/doc/projects/homelab/frostlabs/traefik/dynamic.yml | grep -A 5 crowdseclapikey
```
**Check Traefik can reach CrowdSec:**
```bash
docker service logs traefik_traefik --tail 50 | grep -i crowdsec
```
### View Decision That Would Block You
**Test if an IP is banned:**
```bash
ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli decisions list --ip <your_ip>'
```
**Whitelist your IP permanently:**
Edit `/home/doc/projects/homelab/frostlabs/crowdsec/acquis.yaml` and add to CrowdSec config:
```bash
# Create whitelist file
ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) sh -c "echo \"name: crowdsecurity/whitelists
description: Whitelist trusted IPs
whitelist:
reason: trusted network
ip:
- 10.0.1.0/24
- 10.0.4.0/24
- <your_public_ip>
cidr:
- 10.0.0.0/8
\" > /etc/crowdsec/parsers/s02-enrich/whitelists.yaml"'
# Reload CrowdSec
docker service update --force crowdsec_crowdsec
```
### Performance Issues
**Check resource usage:**
```bash
docker stats $(docker ps -qf name=crowdsec_crowdsec)
```
**Reduce log verbosity in static.yml:**
```yaml
log:
level: ERROR # Change from INFO
```
### Reset CrowdSec Completely
```bash
# Remove all decisions
ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli decisions delete --all'
# Remove CrowdSec data
rm -rf /home/doc/projects/swarm-data/crowdsec/*
# Redeploy
cd /home/doc/projects/homelab/frostlabs/crowdsec
docker stack rm crowdsec
sleep 10
docker stack deploy -c stack.yml crowdsec
# Regenerate bouncer key (wait 30 seconds first)
ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli bouncers add traefik-bouncer'
# Update API key in /home/doc/projects/homelab/frostlabs/traefik/dynamic.yml
# Then redeploy Traefik
docker stack deploy -c stack.yml traefik
```
## Configuration Files
**Location:** `/home/doc/projects/homelab/frostlabs/crowdsec/`
- `stack.yml` - Docker Swarm service definition
- `acquis.yaml` - Log sources configuration
**Persistent Data:** `/home/doc/projects/swarm-data/crowdsec/`
- `config/` - CrowdSec configuration
- `data/` - Decision database and state
**Traefik Logs:** `/home/doc/projects/swarm-data/traefik/logs/`
- `access.log` - Analyzed by CrowdSec
## Best Practices
1. **Start Conservative:** Monitor alerts before enabling aggressive scenarios
2. **Whitelist Trusted IPs:** Add your networks to `clientTrustedIPs` in dynamic.yml
3. **Test Changes:** Use `cscli decisions add` to test bans before deploying
4. **Monitor Metrics:** Regularly check `cscli metrics` to understand what's being blocked
5. **Update Regularly:** Keep collections updated with `cscli hub update && cscli hub upgrade`
6. **Enable Console:** Enroll for community protection and centralized management
7. **Apply Selectively:** Not all services need CrowdSec - use for public-facing services
8. **Combine Layers:** Use with authentik for authentication and rate-limit for additional protection
## Support & Resources
- **CrowdSec Documentation:** https://docs.crowdsec.net/
- **Hub (Collections/Scenarios):** https://hub.crowdsec.net/
- **Community Forum:** https://discourse.crowdsec.net/
- **Traefik Bouncer Plugin:** https://github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin
## Maintenance Tasks
### Weekly
```bash
# Check for new threats
ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli alerts list | head -20'
# Review metrics
ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli metrics'
```
### Monthly
```bash
# Update hub
ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli hub update'
ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli hub upgrade'
# Update Docker images
docker service update --image crowdsecurity/crowdsec:latest crowdsec_crowdsec
docker service update --image traefik:latest traefik_traefik
```
### Quarterly
- Review and tune scenarios based on false positives
- Audit whitelisted IPs
- Check for new relevant collections
- Review ban durations in scenarios

131
crowdsec/QUICK-REFERENCE.md Normal file
View File

@@ -0,0 +1,131 @@
# CrowdSec Quick Reference Card
## Setup Alias (Recommended)
Add to your `~/.bashrc`:
```bash
alias cscli='ssh 10.0.4.14 "docker exec \$(docker ps -qf name=crowdsec_crowdsec) cscli"'
```
Then use: `cscli decisions list` instead of the full command.
---
## Most Common Commands
### View Active Bans
```bash
ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli decisions list'
```
### Ban an IP for 4 Hours
```bash
ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli decisions add --ip 1.2.3.4 --duration 4h'
```
### Unban an IP
```bash
ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli decisions delete --ip 1.2.3.4'
```
### View Recent Alerts (What Triggered Bans)
```bash
ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli alerts list'
```
### Check Status & Metrics
```bash
ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli metrics'
```
### Verify Bouncer Connected
```bash
ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli bouncers list'
```
### View Installed Collections
```bash
ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli collections list'
```
### View Traefik Access Logs
```bash
tail -f /home/doc/projects/swarm-data/traefik/logs/access.log
```
### View CrowdSec Logs
```bash
docker service logs crowdsec_crowdsec --tail 50 --follow
```
---
## Add Protection to a Service
### Docker Swarm Service (via labels)
```yaml
deploy:
labels:
- "traefik.http.routers.myapp.middlewares=crowdsec@file"
```
### External Service (in dynamic.yml)
```yaml
http:
routers:
myservice:
middlewares:
- crowdsec
```
---
## Troubleshooting
### Restart CrowdSec
```bash
docker service update --force crowdsec_crowdsec
```
### Restart Traefik
```bash
docker service update --force traefik_traefik
```
### Check if Logs Are Being Read
```bash
ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli metrics show acquisition'
```
### View Service Status
```bash
docker service ls | grep -E "crowdsec|traefik"
```
---
## File Locations
| Purpose | Path |
|---------|------|
| CrowdSec Stack | `/home/doc/projects/homelab/frostlabs/crowdsec/stack.yml` |
| Log Config | `/home/doc/projects/homelab/frostlabs/crowdsec/acquis.yaml` |
| Traefik Config | `/home/doc/projects/homelab/frostlabs/traefik/dynamic.yml` |
| Access Logs | `/home/doc/projects/swarm-data/traefik/logs/access.log` |
| CrowdSec Data | `/home/doc/projects/swarm-data/crowdsec/` |
---
## Emergency: I Locked Myself Out
```bash
# Delete all bans
ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli decisions delete --all'
# Or unban specific IP
ssh 10.0.4.14 'docker exec $(docker ps -qf name=crowdsec_crowdsec) cscli decisions delete --ip YOUR.IP.HERE'
```
---
For detailed information, see: `/home/doc/projects/homelab/frostlabs/crowdsec/GUIDE.md`

40
crowdsec/stack.yml Normal file
View File

@@ -0,0 +1,40 @@
services:
crowdsec:
image: crowdsecurity/crowdsec:latest
environment:
# Disable online API enrollment (use for local setup)
- DISABLE_ONLINE_API=false
# Set collections to install
- COLLECTIONS=crowdsecurity/traefik crowdsecurity/http-cve
# Enable Prometheus metrics
- METRICS_PORT=6060
volumes:
# Persistent CrowdSec configuration and data
- /home/doc/projects/swarm-data/crowdsec/config:/etc/crowdsec
- /home/doc/projects/swarm-data/crowdsec/data:/var/lib/crowdsec/data
# Traefik access logs (read-only)
- /home/doc/projects/swarm-data/traefik/logs:/var/log/traefik:ro
# Acquis configuration
- ./acquis.yaml:/etc/crowdsec/acquis.yaml:ro
networks:
- frostlabs
deploy:
mode: replicated
replicas: 1
placement:
constraints:
- node.labels.task == control
restart_policy:
condition: on-failure
delay: 5s
max_attempts: 3
healthcheck:
test: ["CMD", "cscli", "version"]
interval: 30s
timeout: 10s
retries: 3
start_period: 60s
networks:
frostlabs:
external: true