Hallo liebe Fedinauten hier auf anonsys.net.

[tux@anonsys.net]

Hallo liebe Fedinauten hier auf anonsys.net. Ab sofort wird diese Instanz vor AI- bzw. KI-Crawlern geschützt. Diese werden geblockt bzw. gebannt.

Danke @rainer für den Tipp. Habe diesen jetzt auf anonsys.net aktiviert und lasse den Filter einmal täglich aktualisieren.

Verdammt interessant ist, dass nach ca. 10 Minuten der Aktivierung des Filters bereits 128 AI-Crawler gebannt wurden:

Status for the jail: apache-ai-crawler
|- Filter
|  |- Currently failed: 0
|  |- Total failed:     33
|  `- File list:        /var/log/apache2/useragent.log
`- Actions
   |- Currently banned: 128
   |- Total banned:     128
   `- Banned IP list:   100.28.204.82 100.29.160.53 107.20.181.148 119.28.140.106 18.207.89.138 18.214.124.6 18.215.24.66 18.215.49.176 18.232.11.247 18.235.158.19 184.73.167.217 184.73.239.35 216.73.216.43 23.21.179.120 23.21.225.190 2
3.21.227.240 23.21.228.180 23.23.99.55 3.209.174.110 3.212.205.90 3.212.86.97 3.220.148.166 3.221.244.28 3.222.190.107 3.93.211.16 3.93.253.174 34.192.67.98 34.195.248.30 34.205.163.103 34.225.138.57 34.226.89.140 34.227.234.246 34.230.
124.21 34.231.45.47 35.169.102.85 35.169.119.108 35.171.117.160 43.130.101.151 43.130.116.87 43.130.26.3 43.134.186.61 43.135.115.233 43.153.192.98 43.154.140.188 43.154.250.181 43.155.157.239 43.157.20.63 43.157.46.118 43.164.195.17 43
.164.196.57 43.164.197.224 43.165.135.242 43.165.189.206 43.166.128.86 43.166.242.189 43.166.244.66 44.194.134.53 44.205.74.196 44.209.35.147 44.210.213.220 44.213.202.136 44.217.255.167 44.220.2.97 44.221.105.234 44.223.116.180 47.128.
112.235 47.128.112.241 47.128.63.217 49.51.166.228 50.19.102.70 52.0.63.151 52.2.4.213 52.201.155.215 52.203.237.170 52.4.229.9 52.5.232.250 52.54.157.23 52.6.97.88 52.70.123.241 54.145.82.217 54.147.80.137 54.157.84.74 54.159.18.27 54.
235.172.108 54.83.23.103 54.83.240.58 54.83.56.1 66.249.68.128 66.249.68.130 98.82.38.120 98.82.63.147 98.82.66.172 98.83.10.183 98.83.8.142 98.84.60.17 18.208.11.93 18.214.238.178 3.218.35.239 44.212.131.50 54.157.99.244 3.230.69.161 1
8.235.81.246 52.203.152.231 35.173.38.202 3.232.82.72 34.193.2.57 54.166.126.132 3.225.9.97 98.82.39.241 98.84.200.43 3.94.156.104 44.223.115.10 43.163.104.54 43.157.22.109 43.130.131.18 43.131.26.226 49.51.132.100 50.16.248.61 43.155.1
62.41 52.203.68.145 54.89.90.224 34.236.185.101 52.200.251.20 43.166.224.244 98.82.107.102 129.226.174.80 18.205.213.231 34.204.150.196

Es werden minütlich mehr. Das ist echt Wahnsinn!

Quelle: rainer.sokoll.com/?p=8353

#anonsys.net #friendica #fedinauten #ai #ki #crawler

Howto block AI bots with fail2ban (Apache) - Rainers kleine Welt

(rainer.sokoll.com)

[jools@friendica.de]

@tux @rainer Das wird auf meinen Instanzen in Kürze ebenfalls so umgesetzt - danke auch von mir, Rainer, du hattest das ja gestern bereits erwähnt!

Da sieht man wieder, wie wichtig der Austausch um Fediverse ist - gerade auch unter Admins.

[tux@anonsys.net]

@jools @rainer
Stimmt! Absolut wichtig!

[oldkid@sekretaerbaer.de]

@tux @rainer @jools ja das ist Hammer was da alles an AI Bots reinkommt. Blocke die auch über Fail2ban, habe allerdings dafür das schon vorhanden Apache-Badbots Jail verwendet. Als Quelle habe ich auch die Liste von ai-robots-txt.

Status for the jail: apache-badbots
|- Filter
|  |- Currently failed: 0
|  |- Total failed:     225
|  `- File list:        /var/log/apache2/access.log /var/log/apache2/other_vhosts_access.log
`- Actions
   |- Currently banned: 248
   |- Total banned:     383
   `- Banned IP list:   102.22.20.125 104.210.140.140 104.210.140.141 114.119.152.164 115.191.65.47 14.224.196.45 144.31.69.239 158.220.100.71 167.235.25.252 168.100.149.173 17.22.237.119 17.22.237.75 17.22.245.161 17.22.253.246 17.241.227.197 17.241.75.169 17.246.19.167 17.246.19.80 17.246.23.124 172.239.170.144 178.156.229.216 185.207.106.120 192.178.6.6 2001:41d0:303:1556::1 216.73.216.170 217.217.250.184 2a00:6020:4cc0:5b01:be24:11ff:fe40:8916 2a01:4f8:201:42a6::2 2a01:4f9:6b:2588::2 2a01:4f9:c012:c53a::1 2a03:2880:16ff:4a:: 2a03:2880:16ff:5c:: 2a03:2880:18ff:47:: 2a03:2880:ff:16:: 2a03:2880:ff:1d:: 2a03:2880:ff:1e:: 2a03:2880:ff:20:: 2a03:2880:ff:21:: 2a03:2880:ff:2:: 2a03:2880:ff:3:: 2a03:2880:ff:43:: 2a03:2880:ff:44:: 2a03:2880:ff:46:: 2a03:2880:ff:48:: 2a03:2880:ff:49:: 2a03:2880:ff:4a:: 2a03:2880:ff:4c:: 2a03:2880:ff:4f:: 2a03:2880:ff:53:: 2a03:2880:ff:56:: 2a03:2880:ff:58:: 2a03:2880:ff:59:: 2a03:2880:ff:5a:: 2a03:2880:ff:5c:: 2a03:2880:ff:5d:: 2a03:2880:ff:5f:: 2a03:2880:ff:73:: 2a03:2880:ff:7:: 2a03:2880:ff:8:: 2a03:2880:ff:: 2a03:2880:ff:a:: 2a03:2880:ff:b:: 2a03:2880:ff:e:: 2a04:4540:6b12:6101:7c9:e82c:8223:4e4f 3.94.156.104 34.140.151.69 34.62.150.54 34.62.152.186 34.89.198.85 35.94.219.215 37.27.129.201 43.156.202.34 46.105.48.30 46.59.207.188 47.128.115.45 47.128.32.102 47.128.34.49 47.128.97.107 47.253.15.29 47.253.183.81 5.104.75.248 50.45.255.106 51.195.183.32 51.68.107.150 51.68.107.156 51.68.111.213 51.68.111.214 51.68.111.243 51.81.208.33 52.71.218.25 66.249.72.4 74.7.230.57 78.46.63.93 91.181.179.225 95.217.22.178 101.36.107.228 104.210.140.129 47.128.28.153 173.252.82.12 192.178.6.8 43.154.140.188 2a02:c207:3018:7615::1 2a03:2880:16ff:70:: 2a03:2880:16ff:40:: 17.241.75.152 2a09:bac3:2ed8:2a0::43:61 35.241.159.130 51.68.236.68 34.238.45.183 5.102.173.71 51.68.107.148 192.178.6.5 77.116.103.213 161.97.163.144 34.62.156.188 64.188.116.27 31.156.243.13 187.17.224.139 34.140.164.77 74.7.228.52 51.68.107.157 85.208.96.201 34.62.178.105 17.246.19.79 17.22.253.241 51.68.111.238 104.155.65.160 2a03:2880:18ff:55:: 149.202.71.165 51.68.236.64 35.240.91.252 34.151.206.25 18.97.9.103 3.213.46.222 17.241.227.188 34.38.27.108 66.249.72.5 47.128.120.193 43.166.142.76 35.205.117.247 47.128.126.63 34.14.123.32 115.190.124.112 114.111.54.188 51.68.107.151 114.119.134.226 74.7.244.63 35.240.43.127 154.81.14.244 34.77.134.114 217.240.230.99 34.78.73.254 173.252.70.21 216.244.66.230 217.113.194.217 54.39.6.28 2a03:2880:16ff:: 2a03:2880:2ff:7:: 2a03:2880:2ff:1:: 2a03:2880:2ff:47:: 74.7.230.36 2a04:4540:6b2c:9901:89be:e3b0:c2c0:ebf3 101.47.158.137 54.39.89.240 104.210.140.133 157.180.113.24 115.190.242.220 52.45.77.169 45.76.216.244 45.63.122.167 136.243.228.180 144.79.133.252 34.79.54.90 185.191.171.17 51.68.111.216 17.246.19.78 51.68.107.137 51.68.107.149 115.190.211.111 185.191.171.8 74.7.244.45 62.60.226.201 47.128.62.157 178.18.251.208 51.161.65.225 2001:41d0:303:4954::1 35.187.191.237 216.73.216.5 43.157.188.74 216.73.216.137 1.161.134.15 2a03:2880:18ff:4b:: 114.119.128.55 114.119.145.81 47.128.121.83 129.159.61.96 18.88.138.239 35.195.103.196 34.89.198.89 185.191.171.7 47.128.96.179 188.112.175.233 2a01:4f8:c012:6c54::1 78.47.61.54 2600:1f28:365:80b0:1159:f992:9ba6:c8f6 18.97.14.91 47.128.61.114 185.91.69.217 46.105.39.50 2001:19f0:7001:2b95:3eec:efff:fe30:501a 45.77.180.226 114.119.139.226 54.147.80.137 37.97.145.176 81.2.118.117 47.254.28.88 34.52.163.14 2a01:599:843:a3d2::374 104.210.140.138 64.176.12.129 140.83.51.163 154.53.34.77 114.119.132.170 23.251.139.238 154.3.39.17 138.2.13.246 2603:c021:8015:be7e:49c1:b642:34e4:944f 45.32.251.197 2001:19f0:7001:86b:5400:5ff:feb6:d5bb 34.22.223.79 47.128.115.230 2a03:2880:18ff:44:: 95.217.114.145 85.208.96.197 54.39.210.187 107.150.119.229 66.249.72.2 34.79.113.224 85.208.96.209 2400:8d60:13::bf34:953c 2a01:4f9:c010:553f::1 2a04:4540:6b08:1101:d5b:ef1f:b738:da0d 192.178.6.7 34.78.164.66 47.128.127.191 17.22.237.74 100.29.107.38 159.65.119.52

git.sekbaer.de/Sekretaerbaer-N…