8000 [RFE]: Improving AbuseIPDB reporting · Issue #3801 · fail2ban/fail2ban · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

[RFE]: Improving AbuseIPDB reporting #3801

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
karolyi opened this issue Jul 21, 2024 · 3 comments
Open

[RFE]: Improving AbuseIPDB reporting #3801

karolyi opened this issue Jul 21, 2024 · 3 comments
Labels
enhancement moreinfo

Comments

@karolyi
Copy link
Contributor
karolyi commented Jul 21, 2024 •
edited
Loading

Hey,

I've spent a couple days with finetuning my server and came up with a better AbuseIPDB reporting.

The base problem is, upon restarting fail2ban, it runs all the actionbans which will trigger curl to call all the API calls, thus possibly exhausting the available API call count.

I've found a script on their site, which I managed to vastly improve. Now it's compatible with FreeBSD and uses the right DB table (and no extra files) to skip calling APIs upon fail2ban restart.

The basic idea is: upon running the script, it checks the sqlite DB if there is an existing record for the IP (and jail name, that comprises the primary key) in the bips table. If the record created is older than10 seconds (configurable in the script), it skips reporting, thus saving API calls.

I haven't created a PR for this since it's customized for my FreeBSD box, but here are the sources for it:

Patching action.d/abuseipdb.conf:

--- action.d/abuseipdb.conf     2024-07-21 16:13:24.470316000 +0200
+++ action.d/abuseipdb.conf     2024-07-21 15:59:54.438420000 +0200
@@ -85,7 +85,9 @@
 # Tags:    See jail.conf(5) man page
 # Values:  CMD
 #
-actionban = lgm=$(printf '%%.1000s\n...' "<matches>"); curl -sSf "https://api.abuseipdb.com/api/v2/report" -H "Accept: application/json" -H "Key: <abuseipdb_apikey>" --data-urlencode "comment=$lgm" --data-urlencode "ip=<ip>" --data "categories=<abuseipdb_category>"
+#actionban = lgm=$(printf '%%.1000s\n...' "<matches>"); curl -sSf "https://api.abuseipdb.com/api/v2/report" -H "Accept: application/json" -H "Key: <abuseipdb_apikey>" --data-urlencode "comment=$lgm" --data-urlencode "ip=<ip>" --data "categories=<abuseipdb_category>"
+actionban = lgm=$(printf '%%.1000s\n...' "<matches>"); /usr/local/etc/fail2ban/custom-scripts/fail2ban-abuseipdb-report.sh \
+    "<abuseipdb_apikey>" "$lgm" "<ip>" "<abuseipdb_category>" "<bantime>" "<name>"
 
 # Option:  actionunban
 # Notes.:  command executed when unbanning an IP. Take care that the

The script itself:

#!/usr/bin/env sh

FAIL2BAN_SQLITE_DB=/var/db/fail2ban/fail2ban.sqlite3

APIKEY=$1
COMMENT=$2
IP=$3
CATEGORIES=$4
BANTIME=$5
JAIL_NAME=$6

LOG_TAG='fail2ban.abuseipdb'
DELTA_SECONDS=10

# https://github.com/fail2ban/fail2ban/discussions/3174
DB_RESULT=$(sqlite3 -list -separator ',' $FAIL2BAN_SQLITE_DB "SELECT \`timeofban\`, \`bantime\` FROM bips WHERE ip='$IP' AND jail='$JAIL_NAME'")

test -n "$DB_RESULT" && {
    TIME_OF_BAN=${DB_RESULT%,*}
    # BAN_LENGTH=${DB_RESULT#*,}
    CURRENT_TIMESTAMP=$(date +%s)
    DELTA_BEFORE=$(echo "$CURRENT_TIMESTAMP - $DELTA_SECONDS" | bc)
    # DELTA_AFTER=$(echo "$CURRENT_TIMESTAMP +  $DELTA_SECONDS" | bc)
    # BANNED_UNTIL=$(echo "$TIME_OF_BAN + $BAN_LENGTH" | bc)

    # Don't ban if we're outside DELTA_SECONDS seconds of the creation of the ban
    test $TIME_OF_BAN -lt $DELTA_BEFORE && exit 0
}

OUTPUT=$(curl --fail-with-body 'https://api.abuseipdb.com/api/v2/report' \
    -H 'Accept: application/json' \
    -H "Key: $APIKEY" \
    --data-urlencode "comment=$COMMENT" \
    --data-urlencode "ip=$IP" \
    --data "categories=$CATEGORIES")

EXIT_CODE=$?

logger -t "${LOG_TAG}" "[$JAIL_NAME] $OUTPUT"

exit $EXIT_CODE

The shell also logs the API call result, which is a small json snippet with the confidence score. An example:

Jul 21 19:15:58 ksol fail2ban.abuseipdb[82701]: [nginx-limit-req] {"data":{"ipAddress":"85.208.96.208","abuseConfidenceScore":100}}

The bc tool is required for this to calculate the DELTA_BEFORE value, not sure that's available everywhere. On FreeBSD, it's part of the base system.
The sqlite3 command line tool is also a requirement, but I would think that's a given since fail2ban uses sqlite.

Feel free to do whatever you like with it, it works for me™.
@sebres
Copy link
Contributor
sebres commented 8000 Jul 26, 2024

The base problem is, upon restarting fail2ban, it runs all the actionbans which will trigger curl to call all the API calls, thus possibly exhausting the available API call count.

Hmm... Since b318eb7 it shall not send restored (after restart) tickets to abuseipdb anymore.

Do you have some old version of action (without norestored = 1)?

@karolyi
Copy link
Contributor Author
karolyi commented Jul 26, 2024

Interesting.

I remember getting curl errors in the syslog upon restart, but don't exactly remember where they originated from.

They might have been from blocklist_de.conf, that configuration needs this too.

Didn't know this option existed (never really looked), but my script provides more functionality in that it will log output from the API with confidence scores.

@karolyi
Copy link
Contributor Author
karolyi commented Aug 3, 2024

Hey,

just checked (had to reinstall my fail2ban jail), when updating the blocklist_de.conf to have norestored = 1 via live patching before starting fail2ban, the reporting errors disappear.

Also, the already used norestored in the abuseipdb.conf are also working, my script doesn't get executed on an update.

Still there is the benefit of the improved reporting (displaying the score).

Feel free to do whatever you want with the script and modifications I provided.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement moreinfo
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@karolyi @sebres
0