How To Block Proxy Visitors From Your Web Site?
If you have access to your .htaccess file, you can add this bit of mod_rewrite code to give proxy visitors a “Forbiden” 403 server response.
# block proxy servers from site access
RewriteEngine on
RewriteCond %{HTTP:VIA} !^$ [OR]
RewriteCond %{HTTP:FORWARDED} !^$ [OR]
RewriteCond %{HTTP:USERAGENT_VIA} !^$ [OR]
RewriteCond %{HTTP:X_FORWARDED_FOR} !^$ [OR]
RewriteCond %{HTTP:PROXY_CONNECTION} !^$ [OR]
RewriteCond %{HTTP:XPROXY_CONNECTION} !^$ [OR]
RewriteCond %{HTTP:HTTP_PC_REMOTE_ADDR} !^$ [OR]
RewriteCond %{HTTP:HTTP_CLIENT_IP} !^$
RewriteRule ^(.*)$ - [F]
Written by admin on November 3rd, 2009 with comments disabled.
Read more articles on How To.
Noticing that Linkwalker requests robots.txt and is followed by BDFetch.
Example of the Linkwalker robot:
72.14.164.165 - - "GET /robots.txt HTTP/1.1" 406 1326 "www.seventwentyfour.com/" "LinkWalker/2.0"
Examples of the BDFetch robot:
72.14.164.177 - - "GET /robots.txt HTTP/1.1" 5816 "www.brandimensions.com" "BDFetch"
72.14.164.193 - - "GET /index.html HTTP/1.1" 5816 "www.brandimensions.com" "BDFetch"
72.14.164.173 - - "GET /robots.txt HTTP/1.1" 5816 "www.brandimensions.com" "BDFetch"
72.14.164.191 - - "GET /index.html HTTP/1.1" 5816 "www.brandimensions.com" "BDFetch"
72.14.164.177 - - "GET /robots.txt HTTP/1.1" 5816 "www.brandimensions.com" "BDFetch"
72.14.164.188 - - "GET /index.html HTTP/1.1" 5816 "www.brandimensions.com" "BDFetch"
Solution to bandwidth gobbling bots in this IP range:
Add these to the deny list on the router.
72.14.163.0/24
72.14.164.0/24
Written by admin on September 28th, 2009 with comments disabled.
Read more articles on How To and Spiders.
151.138.13.244 – - [May/2009] “HEAD /foo.htm HTTP/1.0″ 200 – “-” “SuperPagesBot/0.1″
151.138.13.244 – - [May/2009:13:59:11 -0400] “GET /foo.htm HTTP/1.0″ 200 8967 “-” “SuperPagesBot/0.1″
151.138.13.244 – - [May/2009] “GET /foo.htm HTTP/1.0″ 406 1314 “-” “Lynx/2.8.5rel.1 libwww-FM/2.14″
151.138.13.244 – - [May/2009] “GET / HTTP/1.0″ 406 1314 “-” “Lynx/2.8.5rel.1 libwww-FM/2.14″
151.138.13.244 – - [May/2009] “HEAD /directory HTTP/1.0″ 301 – “-” “SuperPagesBot/0.1″
151.138.13.244 – - [May/2009] “HEAD /directory/ HTTP/1.0″ 200 – “-” “SuperPagesBot/0.1″
Whois IP 151.138.13.244?
OrgName: Idearc Media Corp
OrgID: IMC-97
Address: 2200 W Airfield Drive
City: DFW Airport
StateProv: TX
PostalCode: 75261
Country: US
Written by admin on May 5th, 2009 with comments disabled.
Read more articles on Spiders.
If you notice a “Lock table is out of available locker entries” error while updating with your WHM/CPANEL “Upgrade to Latest Version” function, then you have a RPM error.
The easiest way to fix this problem is to backup, delete, and refresh your RPM database.
How To Fix RPM Error?
Backup your /var/lib/rpm in the event something goes wrong.
tar cvzf rpmdb-backup.tar.gz /var/lib/rpm
Delete the RPM databases. You may have to hit “Y” and “Enter” for each database.
rm /var/lib/rpm/__db.00*
Rebuild your RPM database.
rpm –rebuilddb
Check Your RPM to make sure everything has been updated.
rpm -qa | sort
That’s it!
Written by admin on January 29th, 2009 with comments disabled.
Read more articles on Cpanel and How To.
Many server admins would like to check what port speed their server was configured by their host provider. If you paid for a 10Mbs or 1000Mbs connection, you definitely want to check on its correct provisioning.
Log into your server via putty and enter this command.
STEPS
root@server [~]# mii-tool
You will get a response like this example.
eth0: 100 Mbit, full duplex, link ok
mii-tool Command variables.
mii-tool [-v, --verbose] [-V, --version] [-R, --reset] [-r, --restart] [-w, --watch] [-l, --log] [-A, --advertise=media,...] [-F, --force=media] [interface ...]
Written by admin on August 22nd, 2008 with comments disabled.
Read more articles on How To and Linux.
Congress is starting to examine the state of internet surviellence of citizens using the internet. Companies like Google and Yahoo have been monitoring their visitors surfing habits and are trying to match ads that suit the individual.
Google and Yahoo have announced a way to ‘opt out’ of their ‘ad matching’ technology. The method is to place a cookie in your webbrowser that some how tells them to stop spying on you.
Example of the yahoo.com cookie:
Name: AO
Content: o=1
Domain: .yahoo.com
Path: /
Name: B
Content: 2u2or344a8jbg&b=3&s=hu
Domain: .yahoo.com
Path: /
Yahoo Opt Out Procedure:
When possible, we try to match the ads that we show you to your interests. We call this “ad matching,” (sometimes also called ad customization) and we do this in order to make the ads you see more relevant and useful for you. Ad matching uses data about your visits to both Yahoo! and our partner sites and about the ads you view and click.
Opt Out of Yahoo – Click Me
Google Opt Out Procedure:
Google uses cookies to serve more relevant ads. Learn more about how Google uses advertising cookies.
Anyone may opt out of the DoubleClick cookie (for both the Google content network and DoubleClick ad serving) at any time by clicking the button above. In addition, Google allows third party advertisers to serve ads on the Google content network. Using a tool created by the Network Advertising Initiative, you can opt out of several third party ad servers’ and networks’ cookies simultaneously. (Google also uses cookies for Google Analytics and conversion tracking. Read more about this in our FAQ below.)
Conclusion
Is this the best solution to prevent Yahoo and Google from tracking your serving habits? Probably not. I recommend that you use a freeware utility for Windows called “Hostman“. You can effectively block cookies and malicious domains by blocking them via your HOSTS file.
Additionally, for Firefox browser users, using AdBlock Plus and NoScript extensions to ensure additionally privacy protections.
Written by admin on August 14th, 2008 with comments disabled.
Read more articles on Privacy.
Ever wondered how to check that status of your server’s ram with a convient, simple to use command? Well, worry no more.
STEPS
1) Enter
root@server [~]# free -o
Your response should look something like this:
total used free shared buffers cached
Mem: 2067220 2013636 53584 0 28300 1323664
Swap: 2031608 71336 1960272
This command will show you your total available/used memory and cached memory. In the case obvoe, this server has 2 gigabytes of RAM installed and has used a large portion of it with a memory intensive application. The SWAP file has been put to use to buttress the RAM use.
Another method is to use this command for more detail.
root@server [~]# cat /proc/meminfo
You will get a result something like this:
MemTotal: 2067220 kB
MemFree: 95960 kB
Buffers: 20472 kB
Cached: 1373616 kB
SwapCached: 25076 kB
Active: 697120 kB
Inactive: 1211676 kB
HighTotal: 1171136 kB
HighFree: 19632 kB
LowTotal: 896084 kB
LowFree: 76328 kB
SwapTotal: 2031608 kB
SwapFree: 1962060 kB
Dirty: 464 kB
Writeback: 0 kB
AnonPages: 505384 kB
Mapped: 32668 kB
Slab: 49260 kB
PageTables: 4128 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
CommitLimit: 3065216 kB
Committed_AS: 1145992 kB
VmallocTotal: 114680 kB
VmallocUsed: 5128 kB
VmallocChunk: 107708 kB
HugePages_Total: 0
HugePages_Free: 0
HugePages_Rsvd: 0
Hugepagesize: 4096 kB
Written by admin on August 10th, 2008 with comments disabled.
Read more articles on How To and Linux.
Our intrusion detection software picked up the start of a dictionary attack targeting our pop3 or email server coming from the Layeredtech server provider.
Time: Sat Aug 9 19:22:43 2008 -0400
IP: 72.232.183.242 (US/United States/242.183.232.72.static.reverse.ltdomains.com)
Failures: 5 (pop3d)
Interval: 240 seconds
Blocked: Yes
Log entries:
Aug 9 19:21:16 pop3d: LOGIN FAILED, user=test, ip=[::ffff:72.232.183.242]
Aug 9 19:21:52 pop3d: LOGIN FAILED, user=test, ip=[::ffff:72.232.183.242]
Aug 9 19:21:57 pop3d: LOGIN FAILED, user=web, ip=[::ffff:72.232.183.242]
Aug 9 19:22:36 pop3d: LOGIN FAILED, user=test, ip=[::ffff:72.232.183.242]
Aug 9 19:22:41 pop3d: LOGIN FAILED, user=test, ip=[::ffff:72.232.183.242]
Whois 72.232.183.242 you might ask?
Why it is the crappy dedicated server provider Layeredtech.
OrgName: Layered Technologies, Inc.
OrgID: LAYER-3
Address: 5085 W Park Blvd
Address: Suite 700
City: Plano
StateProv: TX
PostalCode: 75093
Country: US
Written by admin on August 10th, 2008 with comments disabled.
Read more articles on Hacker and ModSecurity.
We have been getting a lot of hits lately emanating from Verisign. The ips-agent robot requests robots.txt but we are just not going to allow their domain browsing.
Example Server Logs:
69.58.178.36 - - "GET /robots.txt HTTP/1.1" 406 261 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.12; ips-agent) Gecko/20050922 Fedora/1.0.7-1.1.fc4 Firefox/1.0.7"
69.58.178.39 - - "GET / HTTP/1.1" 406 251 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.12; ips-agent) Gecko/20050922 Fedora/1.0.7-1.1.fc4 Firefox/1.0.7"
69.58.178.29 - - "GET / HTTP/1.1" 406 251 "-" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.12; ips-agent) Gecko/20050922 Fedora/1.0.7-1.1.fc4 Firefox/1.0.7"
That particular IP range resolves to:
root@server[~]# whois 69.58.178.29
OrgName: VeriSign Infrastructure & Operations
OrgID: VIO-2
Address: 21345 Ridgetop Circle
City: Dulles
StateProv: VA
PostalCode: 20166
Country: US
We have their robot blocked via ModSecurity 2+ rules.
Written by admin on July 4th, 2008 with comments disabled.
Read more articles on ModSecurity and Spiders.
Lately, there has been a huge amount of annoying website scrappers combing through our sites using the useragent below.
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)
Obviously, legitimate human visitors do not have the term “User-Agent:” in their useragent field. Months ago I had added a ModSecurity rule to help identify and block these bandwidth wasters and copyright infringer’s.
Example of log details:
69.14.204.163 – - [04/Jul/2008:13:09:58 -0400] “GET / HTTP/1.1″ 410 317 “-” “User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)”
Thumbs Down To "User-Agent" Scrappers
STEPS
1) Using ModSecurity 2.0+, I instituted this rule to modsec2.conf located in usr/local/apache/conf/
SecRule HTTP_User-Agent “User-Agent” “deny,log,status:410″
2) Restart Apache
/sbin/service httpd restart
Written by admin on July 4th, 2008 with comments disabled.
Read more articles on How To and ModSecurity.
« Older articles
No newer articles
