公开指南

Network Sensor Troubleshooting

Diagnostic runbook for container startup, traffic capture, API connectivity, health checks, resource usage, and ingestion gaps. 受众: Operations engineers, DevOps teams, security operations teams. 典型配置时长: 5-15 minutes.

troubleshooting

适用场景

Diagnostic runbook for container startup, traffic capture, API connectivity, health checks, resource usage, and ingestion gaps.

目标受众: Operations engineers, DevOps teams, security operations teams预计耗时: 5-15 minutes

开始之前

  • You have SSH or console access to the sensor instance.
  • Docker and basic Linux utilities (tcpdump, curl, netstat) are available on the sensor.
  • You know the sensor's API key and platform API URL.

操作指引

1

步骤 1

Container startup issues

Diagnose problems with sensor container initialization and API key retrieval.

  • Check container status: docker ps -a | grep network-sensor
  • Review startup logs: docker logs [container-id] — look for Secrets Manager errors, API connectivity failures, or invalid configuration.
  • Verify API key is in Secrets Manager or environment: aws secretsmanager get-secret-value --secret-id blackshield/api-key
  • Confirm network connectivity to the platform API: curl -v https://api.blackshield.chaplau.com/health

达成目标状态

Confirm network connectivity to the platform API: curl -v https://api.blackshield.chaplau.com/health

2

步骤 2

Traffic capture and mirroring

Verify packets are reaching the sensor and being processed.

  • Check mirroring configuration: AWS (describe-traffic-mirror-sessions), GCP (gcloud compute packet-mirrorings list), Azure (az network vnet tap list)
  • Capture packets on sensor: sudo tcpdump -i eth0 'udp port 4789 or vxlan' -c 10 — should see VXLAN encapsulated traffic.
  • If no packets, check source VM traffic: trigger curl or ping from a monitored workload and re-run tcpdump.
  • Verify sensor interface is NOT in promisc mode race condition: ip link show [interface]

达成目标状态

Verify sensor interface is NOT in promisc mode race condition: ip link show [interface]

3

步骤 3

API connectivity and authentication

Ensure the sensor can reach and authenticate with the platform.

  • Test connectivity: curl -s -H 'Authorization: Bearer [API_KEY]' https://api.blackshield.chaplau.com/api/v1/health
  • Check sensor logs for '401 Unauthorized' or '403 Forbidden'.
  • Verify API key is active in Settings > API Keys and has Ingestion scope.
  • Test with different network path (via NAT, proxy, etc.) if behind firewall.

达成目标状态

Test with different network path (via NAT, proxy, etc.) if behind firewall.

4

步骤 4

Health check and readiness

Verify the sensor health endpoint and readiness probe.

  • Health check: curl -s http://localhost:8080/health — should return 200 with 'ok' status.
  • Readiness: curl -s http://localhost:8080/ready — confirms sensor is ready to ingest.
  • If health fails, check disk space (docker exec [id] df -h), memory (docker stats), and CPU.
  • Review container resource limits: docker inspect [id] | grep -A 10 HostConfig

达成目标状态

Review container resource limits: docker inspect [id] | grep -A 10 HostConfig

5

步骤 5

Resource usage and tuning

Monitor CPU, memory, and disk usage; adjust configuration if needed.

  • Real-time stats: docker stats [container-id] — look for CPU % and memory %.
  • High CPU: reduce MIN_SEVERITY, set PACKET_SAMPLING_RATE < 1, or use SENSOR_FILTER_RULES.
  • High memory: reduce MAX_ALERTS_BUFFER_SIZE, increase FLUSH_INTERVAL_SECONDS.
  • Disk full: docker exec [id] du -sh /var/log/ — remove old scan logs or increase EBS/disk size.

达成目标状态

Disk full: docker exec [id] du -sh /var/log/ — remove old scan logs or increase EBS/disk size.

6

步骤 6

Ingestion gaps and missing findings

Diagnose why expected findings are not appearing in the platform.

  • Check sensor metrics: docker exec [id] curl http://localhost:8080/metrics | grep ingestion
  • Verify MIN_SEVERITY is not filtering out findings: set MIN_SEVERITY=low temporarily to test.
  • Check for 429 (rate limit) or 503 (service degraded) errors in sensor logs.
  • Confirm findings are being sent: docker exec [id] tcpdump -i eth0 host api.blackshield.chaplau.com dst port 443 -A | grep -i 'blackshield'

达成目标状态

Confirm findings are being sent: docker exec [id] tcpdump -i eth0 host api.blackshield.chaplau.com dst port 443 -A | grep -i 'blackshield'

Demonstration only

This configuration is designed for ease of use. To deploy scanner clients at scale, please plan your deployment architecture accordingly or contact us for enterprise best practices.

运行命令

troubleshooting-commands.sh
#!/bin/bash
# Quick troubleshooting commands

docker ps -a | grep network-sensor
docker logs -f "$(docker ps -q | head -1)" | tail -50
curl -s -H "Authorization: Bearer $BLACKSHIELD_API_KEY" https://api.blackshield.chaplau.com/api/v1/health

达成目标状态

  • Container is running and not restarting continuously.
  • Sensor can successfully authenticate to the platform API.
  • Network telemetry is flowing to the platform with no dropped packets.
Network Sensor Troubleshooting | BlackShield Docs