modsecurity vs content compression

June 17, 2009

For some time I run modsecurity on Apache as a reverse proxy. It is just monitoring,  and I’m always uncomfortable with the message bellow:

[Wed Jun 17 11:38:40 2009] [error] [client] ModSecurity: Warning. Operator EQ matched 0 at GLOBAL. [file “/etc/httpd/conf.d/modsecurity2/
modsecurity_crs_30_http_policy.conf”] [line “120”] [id “960903”] [msg “ModSecurity does not support content encodings”] [severity “WARNING”] [hostname “w”] [uri “/overlib/over.js”] [unique_id “P02WQH8AAAEAAGGvs4EAAAAK”]

This happen because modsecurity is not able to handle content compressed by backend server!

Deploying a new reverse proxy with Apache, I look for a solution to this problem I found , a very effective solution. But has a collateral effect, you loose the compression between the reverse proxy and the backend server. Actually is possible to don’t loose this compression, but I believe that is more “cheaper” to loose this compression instead off compress at backend, decompress at reverse proxy and compress again to deliver to client. One other way to achieve this is to turn off compression at backend, but as mentioned in the mail thread above, is not always easy do this, because some times the backend are not on your hands, I you cant guarantee that the admin ll keep the compression off.

I change the proposed Apache setup a little bit, from the Bryan (link above), to make it more simple if you need turn off modsecurity or mod_deflate. Check it bellow

<IfModule mod_deflate.c>

ExtFilterDefine nodeflate mode=output cmd=/bin/true \

SetOutputFilter DEFLATE

# Netscape 4.x has some problems…
BrowserMatch ^Mozilla/4 gzip-only-text/html
# Netscape 4.06-4.08 have some more problems
BrowserMatch ^Mozilla/4\.0[678] no-gzip
# IE is ok, but looked like Netscape, so we reset it
BrowserMatch \bMSIE !no-gzip !gzip-only-text/html

SetEnvIfNoCase Accept-Encoding gzip force-gzip
SetEnvIfNoCase TE gzip force-gzip

RequestHeader unset Accept-Encoding
RequestHeader unset TE

# Skip images based on extension
SetEnvIfNoCase Request_URI \.(?:gif|jpe?g|png|zip|exe|swf|mp?eg|mp3|tgz|tar.gz|avi|ico|gz)$ no-gzip dont-vary

# Make sure caching still works
Header append Vary User-Agent env=!dont-vary

<IfModule mod_security2.c>
FilterDeclare modsec CONTENT_SET
FilterProvider modsec modsecurity_out env=modsec-ignore !=1
FilterDeclare compress CONTENT_SET
FilterProvider compress deflate env=force-gzip =1
#FilterProvider compress inflate Content-Type $image/jpeg
#FilterProvider compress inflate Content-Type $image/gif
#FilterProvider compress inflate Content-Type $image/png
FilterProtocol compress “change=yes”
<IfModule mod_security2.c>
FilterChain modsec compress

<IfModule !mod_security2.c>
FilterChain compress

After this you still deliver the content compressed to the web browser and modsecurity can now check every content, from all your applications and backends.

Reserved IP address: Linux vs Windows

April 7, 2008

I’m doing some refresh in my IP knowledge, and I was wonder for one thing: How different operating system behaves when you try to use a reserved IP address as the primary address of the system? So I do a simple test, put (at least try) a reserved address as primary address in a Linux box and in a Windows XP box.

The reserve address subject to this test are:
– the all 0’s ip address (

– loopback address (
– multicast range ( –, or
– reserve range ( or
– broadcast address (
Well, the results are what follow:
In Windows XP MCE I can’t use any reserved IP address, the TCP/IP configuration forbid to use any of this invalid for normal address as primary address.
In Linux (Fedora Core 7), I could put all sort of reserved address but I use both CLI ip address add command and graphic interface system-config-network-gui.

The RFC 1700 (ASSIGNED NUMBERS), from 1994, use the concept of the IP address Class (created by original RFC 790, from 1981), marking class A, B and C as destinated for unicast address, D for multicast and E reserved for future use, and it reserve the block from 224-239, 240-255, 127/24, 0/24. This RFC make the use of class not so important, the classless, but still make the reservations.

This show Microsoft people more careful that the counterpart from Linux.

If I had time I’ll make some test in other platforms.

Squid kerberos authentication and ldap authorization in Active Directory

January 9, 2008
Edited on jun, 21, 2009

The squid web cache include a authenticator for kerberos, it is simple to use, but the documentation is not very clear about how to make it work. Below some steps use by me to make Squid 3.0 Stable1 and Squid 2.6 Stable17 authenticate against Active Directory (Windows 2003 Directory Service) and also to make it make the authorization using Ldap. This setup was not used in production environment yet, so its possible to had some problems not seen by me or scalabilities issues.


To achieve our objective:

  • Authenticate using “Kerberos” (Negotiate);
  • Authenticate using “Basic authentication”, with Ldap as back-end (for backward compatibility);
  • Authorization using Ldap group membership;

we need to compile Squid with some specific parameters:

  • –enable-auth=”basic negotiate”
  • –enable-basic-auth-helpers=”LDAP”
  • –enable-negotiate-auth-helpers=”squid_kerb_auth”
  • –enable-external-acl-helpers=”ldap_group”

other parameters that you can need must be supplied at compilation time.


In this article we’ll use two types of authentication: Negotiate (using Kerberos) and Basic (using Ldap).

To use Negotiate authentication method the web browser must be writed to understand it and configure correctly to do so, and the computer use need to be authenticated by the kerberos infra-structure and receive the appropriated key from the KDC (Key Distribution Center). I’m using Firefox for Linux 2.0 (authenticated with kerberos), Firefox for Windows 2.x and Internet Explorer 6/7, all worked very well.

The Basic authentication is present in any web browser, and we will use Ldap as back-end authentication provider.

Negotiate/kerberos authentication

This part is divided in two parts, one in Linux box other in Windows Domain Controlle, and was strongly based on article, that has Apache as a target but this steps works for Squid too.

Windows Step:

Create a user account, ex. squid.domain, and using ktpass command tool create a keytab file.

ktpass -princ HTTP/squid.domain@REALM -mapuser squid.domain -crypto rc4-hmac-nt pass squid-pass -ptype KRB5_NT_SRV_HST -out squid.domain.keytab

Transfer this file safely to the squid box.

Linux/Squid step:

Check that /etc/krb5.conf are correctly configured, example bellow (pay attention on bold lines):
default = FILE:/var/log/krb5libs.log
kdc = FILE:/var/log/krb5kdc.log
admin_server = FILE:/var/log/kadmind.log

default_realm = REALM
dns_lookup_realm = true
dns_lookup_kdc = true
ticket_lifetime = 24h
forwardable = yes
kdc =
admin_server =
default_domain =
[domain_realm] = EXAMPLE.COM = EXAMPLE.COM
profile = /var/kerberos/krb5kdc/kdc.conf
pam = {
debug = false
ticket_lifetime = 36000
renew_lifetime = 36000
forwardable = true
krb4_convert = false

Move the keytab file to some directory related to squid, like /etc/squid, and must change the permission like this:

chmod 400 /etc/squid/squid.domain.keytab
chown nobody /etc/squid/squid.domain.keytab

You can test is the squid box can use the keytab file using kinit command:
kinit -V -k -t squid.domain.keytab HTTP/squid.domain

You need to put a line like this in squid startup script:

export KRB5_KTNAME=/etc/squid/squid.domain.keytab
In squid.conf you need set directives like this:
auth_param negotiate program /usr/local/squid/libexec/squid_kerb_auth -d
auth_param negotiate children 10
auth_param negotiate keep_alive on

http_access allow AUTENTICADO
http_access deny all

Aditional reference about kerberos authentication for Squid:

Check the /etc/hosts file to certify that the fqdn used on keytab file are associated to the correct address on hosts files, as well on DNS (A and PTR record). Dont associate the fqdn with loopback or other address.

Basic authentication

To make users not authenticated by a kerberos infra-structure keep using squid, you can use a Basic Authentication with many methods, I chose Ldap because the Samba/Winbind (that I already used) some times has problems of stability and a unknown timeouts of and refresh for password change and group membership.

To use this authentication, you can put this lines after the kerberos related lines (this is very important once that the order of method used to authenticated in browser is derived from this order, if you change this you’ll never get a negotiate authentication only basic):

auth_param basic program /usr/local/squid/libexec/squid_ldap_auth -b "CN=Users,dc=bc" -s sub -D "CN=user_to_bind_in_ldap,OU=Usuers,DC=bc" -w password -f "(&(objectClass=person)(sAMAccountName=%s))" -u sAMAccountName -P -v 3 -h ldap_server.domain
auth_param basic children 10
auth_param basic realm Proxy Authentication
auth_param basic credentialsttl 2 hours


Sometimes you can control how access the Internet, sometimes you can control how can’t, or when they access, or yet where the peoples go. For achieve this you should use some directives from squid config file, like: ACL, External_ACL, http_access, http_reply_access, etc.

How we are doing the authentication with kerberos, the username (%LOGIN for squid external_acl) become like this: username@REALM. This is a problem for us once that we are trying to check the attribute sAMAccountName in Active Directory, that don’t include Realm, so we need to apply the patch bellow, and use the new parameter -K in squid_ldap_group to strip the Realm from username.

--- squid_ldap_group.c.original 2008-01-10 11:08:12.000000000 -0200
+++ squid_ldap_group.c 2008-01-04 19:35:09.000000000 -0200
@@ -215,6 +215,7 @@
int port = LDAP_PORT;
int use_extension_dn = 0;
int strip_nt_domain = 0;
+ int strip_kerberos_realm = 0;
int err = 0;

setbuf(stdout, NULL);
@@ -370,6 +371,9 @@
case 'S':
strip_nt_domain = 1;
+ case 'K':
+ strip_kerberos_realm = 1;
+ break;
fprintf(stderr, PROGRAM_NAME " ERROR: Unknown command line option '%c'\n", option);
@@ -424,6 +428,7 @@
fprintf(stderr, "\t-g\t\t\tfirst query parameter is base DN extension\n\t\t\t\tfor this query\n");
fprintf(stderr, "\t-S\t\t\tStrip NT domain from usernames\n");
+ fprintf(stderr, "\t-K\t\t\tStrip Kerberos realm from usernames\n");
fprintf(stderr, "\n");
fprintf(stderr, "\tIf you need to bind as a user to perform searches then use the\n\t-D binddn -w bindpasswd or -D binddn -W secretfile options\n\n");
@@ -470,6 +475,12 @@
if (u && u[1])
user = u + 1;
+ if (strip_kerberos_realm) {
+ char *u = strchr(user, '@');
+ if (u!=NULL) {
+ *u = '';
+ }
+ }
if (use_extension_dn) {
extension_dn = strtok(NULL, " \n");
if (!extension_dn) {


--- squid_ldap_group.8.original 2008-01-10 11:08:21.000000000 -0200
+++ squid_ldap_group.8 2008-01-07 11:36:45.000000000 -0200
@@ -152,6 +152,10 @@
Strip NT domain name component from user names (/ or \\ separated)
+.BI -K
+Strip Kerberos Realm component from user names (@ separated)
.BI -d
Debug mode where each step taken will get reported in detail.
Useful for understanding what goes wrong if the results is

To allow or deny some user with account in Active Directory based on they group membership, you can use Ldap to check the this using this lines on squid.conf:

external_acl_type ldapgroup %LOGIN /usr/local/squid/libexec/squid_ldap_group -b DC=bc -f
"(&(objectclass=person)(sAMAccountName=%v)(memberOf=cn=%a,OU=Security Groups,DC=REALM))" -D
CN=user_to_bind_in_ldap,CN=Users,DC=REALM -w password -h ldap-server.domain -v 3 -K

acl ldapgroup-access external ldapgroup @users_group

http_access allow all ldapgroup-access

Is possible to check other attributes as well, just change your Ldap query.