Apache stop bots and hackers by using forensic logging.
Add this to your Apache config:
LoadModule log_forensic_module modules/mod_log_forensic.so
Restart Apache
Set the location of the forensic log.
ForensicLog /var/log/apache2/forensic.log
Here is an example of an entry in forensic:
+16831:68ca525e:3c5|GET /some/url HTTP/1.1|sec-fetch-dest:document|user-agent:Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/26.0 Safari/605.1.15|accept:text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8|referer:|sec-fetch-site:same-origin|sec-fetch-mode:same-origin|accept-language:en-US,en;q=0.9|priority:u=0, i|sec-fetch-user:?1|X-Forwarded-For:1.2.3.4|host:yourdomain.com|Accept-Encoding:gzip|X-Varnish:7275706
This can be crucial for debugging CDN/Load Balancer issue but also for identifying malicious bots. Many of them have "holes" in how they operate and assume many admins and sites do not properly check or filter headers.
For example if the "host:" entry is not your host, it is probably not a valid request, so you should block it at the WAF or CDN level.
Another fun trick is that sometimes bots claim to be "Windows NT" or "Mac OS" but then other parts of the header may indicate Linux, which again is impossible.
Tags:
apache, bots, hackers, forensic, logging, config, loadmodule, log_forensic_module, modules, mod_log_forensic, restart, forensiclog, var, entry, ca, url, http, fetch, dest, user, mozilla, macintosh, intel, os, _, applewebkit, khtml, gecko, safari, text, html, xhtml, xml, referer, origin, mode, en, forwarded, yourdomain, encoding, gzip, varnish, crucial, debugging, cdn, balancer, identifying, malicious, quot, admins, sites, filter, headers, valid, waf, nt, header, indicate, linux,