WAF-FLE, my new project

A long time since the last post, lots of things to write, but no time to do it!

But recently I started a new project, WAF-FLE (www.waf-fle.org), that is a new console for ModSecurity (and eventually in future other WAF’s). For now you can use it to receive and display events from many sensors (no practical limit is imposed), and a drill down filter is present to make simple to found the relevant events.

I hope that you enjoy, and stay tunned there for upcoming features.

modsecurity vs content compression

For some time I run modsecurity on Apache as a reverse proxy. It is just monitoring,  and I’m always uncomfortable with the message bellow:

[Wed Jun 17 11:38:40 2009] [error] [client 192.168.1.1] ModSecurity: Warning. Operator EQ matched 0 at GLOBAL. [file “/etc/httpd/conf.d/modsecurity2/
modsecurity_crs_30_http_policy.conf”] [line “120”] [id “960903”] [msg “ModSecurity does not support content encodings”] [severity “WARNING”] [hostname “w
ww.example.org”] [uri “/overlib/over.js”] [unique_id “P02WQH8AAAEAAGGvs4EAAAAK”]

This happen because modsecurity is not able to handle content compressed by backend server!

Deploying a new reverse proxy with Apache, I look for a solution to this problem I found http://thread.gmane.org/gmane.comp.apache.mod-security.user/5041/focus=5042 , a very effective solution. But has a collateral effect, you loose the compression between the reverse proxy and the backend server. Actually is possible to don’t loose this compression, but I believe that is more “cheaper” to loose this compression instead off compress at backend, decompress at reverse proxy and compress again to deliver to client. One other way to achieve this is to turn off compression at backend, but as mentioned in the mail thread above, is not always easy do this, because some times the backend are not on your hands, I you cant guarantee that the admin ll keep the compression off.

I change the proposed Apache setup a little bit, from the Bryan (link above), to make it more simple if you need turn off modsecurity or mod_deflate. Check it bellow

<IfModule mod_deflate.c>

ExtFilterDefine nodeflate mode=output cmd=/bin/true \
enableenv=SomeVarThatWillNeverBeSet

SetOutputFilter DEFLATE

# Netscape 4.x has some problems…
BrowserMatch ^Mozilla/4 gzip-only-text/html
# Netscape 4.06-4.08 have some more problems
BrowserMatch ^Mozilla/4\.0[678] no-gzip
# IE is ok, but looked like Netscape, so we reset it
BrowserMatch \bMSIE !no-gzip !gzip-only-text/html

SetEnvIfNoCase Accept-Encoding gzip force-gzip
SetEnvIfNoCase TE gzip force-gzip

RequestHeader unset Accept-Encoding
RequestHeader unset TE

# Skip images based on extension
SetEnvIfNoCase Request_URI \.(?:gif|jpe?g|png|zip|exe|swf|mp?eg|mp3|tgz|tar.gz|avi|ico|gz)$ no-gzip dont-vary

# Make sure caching still works
Header append Vary User-Agent env=!dont-vary

<IfModule mod_security2.c>
FilterDeclare modsec CONTENT_SET
FilterProvider modsec modsecurity_out env=modsec-ignore !=1
</IfModule>
FilterDeclare compress CONTENT_SET
FilterProvider compress deflate env=force-gzip =1
#FilterProvider compress inflate Content-Type $image/jpeg
#FilterProvider compress inflate Content-Type $image/gif
#FilterProvider compress inflate Content-Type $image/png
FilterProtocol compress “change=yes”
<IfModule mod_security2.c>
FilterChain modsec compress
</IfModule>

<IfModule !mod_security2.c>
FilterChain compress
</IfModule>
</IfModule>

After this you still deliver the content compressed to the web browser and modsecurity can now check every content, from all your applications and backends.

mythbuntu + gnome or ubuntu + mythbuntu?

I need to upgrade my old MythTV box (that is my desktop too), today a Fedora Core 7 (no more ugrades now). Fedora change too fast (probably good) and lost support too fast (bad), so I want a distribution with a medium/long term support for my production MythTv box. I have a small maintenance window because the family want the TV and desktop working.

So I’m give a try to Ubuntu. However I found two/three options: Ubuntu+MythTV, Ubuntu+Mythbuntu and Mythbuntu+gnome (or even KDE). I don’t found a clear comparision between this options, so I decide to make a simple try before the upgrade.

I run this lab on VMWare, so I will just install the systems as frontends, because the capture hardware will be not available inside virtual machine.

The test was conduced with two (virtual) machines, and it was installed at (almost) the same time, so I compare step-by-step the difference between it.

Mythbuntu 8.04

The instalation program are the same (once that mythtuntu is actualy a flavor or ubuntu), the 6 initial steps are the same, after that Ubuntu transfer the packages to the disk, but Mythbuntu go foward and start to make MythTV specific questions and configuration, as if the box will be a primary/secondary backend, frontend or both. I choose just as a frontend, select plugins and themes to install. I do configure the access to my current box mysql server. I skip the remote control step, but it appear to be very simple to a novice (lirc) user, it have a option to remote control and one for IR Blaster (IR transmitter).

I don’t have a specific video board inside virtual machine, but I see the options for automatic install of proprietary video drivers, it seen very easy to new users to setup a box when using Nvidia graphics. I don’t see the option to install gnome, kde desktop environment, it’s possible that I make different choices, but I’ll need to install it after boot.

After this the system was booted.

After the first boot, a simple desktop show up and a “mythfrontend.real” ask me for my preferred language, so it show me the MythTV frontend, however I need to fix my current mythbackent to listen in IP address from ethernet and not in loopback (127.0.0.1) interface, after that I have a working (almost) MythTV frontend. Really easy.

But this will be my desktop too, so I need to install a full desktop environment like KDE or Gnome. To do this I open the Mythbuntu Control Centre and in the System Role, I mark to be a Desktop Role, Ubuntu Desktop, after a download of 453 packages it was ready.

Ubuntu 8.04

The Ubuntu instalation was almost equal to Mythbuntu, except by the Mythtv, proprietary drivers and LIRC configuration. After the first boot I receive a login screen, where I logged with the user created at instalation time. Now I have a gnome desktop, but no MythTV, so I start do install Mythbuntu over Ubuntu by following http://mythbuntu.org/existing-ubuntu, what was a 100 packages download. During the mythbuntu install, a setup was launched to configure the LIRC (Remote Control), that I skip.

After packages install I launch the Mythbuntu Control Centre to make the configuratiuon like in a Mythbuntu full instalation. When selection options like Mythtv plugins before change to new option the setup download the packages needed, what consume some time. After that, I setup the system to logon automaticaly in Art and Login Behavior, and enable some proprietary codecs, and reboot.

The systems boots up in XFCE environment! I not expect this. To change to GNOME I need to logout and logon again but selecting gnome um session menu at logon screen. The mythfrontend startup automaticaly here too, and startup with different theme G.A.N.T. (but this is simple to change).

Conclusion

Actualy I see no diferrence between they both, probably the advice of mythbuntu project are a good option, mythbuntu to new system (like mine) Ubuntu+mythbuntu for current Ubuntu users.

Reserved IP address: Linux vs Windows

I’m doing some refresh in my IP knowledge, and I was wonder for one thing: How different operating system behaves when you try to use a reserved IP address as the primary address of the system? So I do a simple test, put (at least try) a reserved address as primary address in a Linux box and in a Windows XP box.

The reserve address subject to this test are:
– the all 0’s ip address (0.0.0.0)

– loopback address (127.0.0.0/24)
– multicast range (224.0.0.0 – 239.255.255.255, or 224.0.0.0/4)
– reserve range (240.0.0.0-255.255.255.254 or 240.0.0.0/4)
– broadcast address (255.255.255.255)
Well, the results are what follow:
In Windows XP MCE I can’t use any reserved IP address, the TCP/IP configuration forbid to use any of this invalid for normal address as primary address.
In Linux (Fedora Core 7), I could put all sort of reserved address but 0.0.0.0/24. I use both CLI ip address add command and graphic interface system-config-network-gui.

The RFC 1700 (ASSIGNED NUMBERS), from 1994, use the concept of the IP address Class (created by original RFC 790, from 1981), marking class A, B and C as destinated for unicast address, D for multicast and E reserved for future use, and it reserve the block from 224-239, 240-255, 127/24, 0/24. This RFC make the use of class not so important, the classless, but still make the reservations.

This show Microsoft people more careful that the counterpart from Linux.

If I had time I’ll make some test in other platforms.

Squid kerberos authentication and ldap authorization in Active Directory

Edited on jun, 21, 2009

The squid web cache include a authenticator for kerberos, it is simple to use, but the documentation is not very clear about how to make it work. Below some steps use by me to make Squid 3.0 Stable1 and Squid 2.6 Stable17 authenticate against Active Directory (Windows 2003 Directory Service) and also to make it make the authorization using Ldap. This setup was not used in production environment yet, so its possible to had some problems not seen by me or scalabilities issues.

Compilation

To achieve our objective:

• Authenticate using “Kerberos” (Negotiate);
• Authenticate using “Basic authentication”, with Ldap as back-end (for backward compatibility);
• Authorization using Ldap group membership;

we need to compile Squid with some specific parameters:

• –enable-auth=”basic negotiate”
• –enable-basic-auth-helpers=”LDAP”
• –enable-negotiate-auth-helpers=”squid_kerb_auth”
• –enable-external-acl-helpers=”ldap_group”

other parameters that you can need must be supplied at compilation time.

Authentication

In this article we’ll use two types of authentication: Negotiate (using Kerberos) and Basic (using Ldap).

To use Negotiate authentication method the web browser must be writed to understand it and configure correctly to do so, and the computer use need to be authenticated by the kerberos infra-structure and receive the appropriated key from the KDC (Key Distribution Center). I’m using Firefox for Linux 2.0 (authenticated with kerberos), Firefox for Windows 2.x and Internet Explorer 6/7, all worked very well.

The Basic authentication is present in any web browser, and we will use Ldap as back-end authentication provider.

Negotiate/kerberos authentication

This part is divided in two parts, one in Linux box other in Windows Domain Controlle, and was strongly based on http://www.grolmsnet.de/kerbtut/ article, that has Apache as a target but this steps works for Squid too.

Windows Step:

Create a user account, ex. squid.domain, and using ktpass command tool create a keytab file.

ktpass -princ HTTP/squid.domain@REALM -mapuser squid.domain -crypto rc4-hmac-nt pass squid-pass -ptype KRB5_NT_SRV_HST -out squid.domain.keytab

Transfer this file safely to the squid box.

Linux/Squid step:

Check that /etc/krb5.conf are correctly configured, example bellow (pay attention on bold lines):
[logging]
default = FILE:/var/log/krb5libs.log
kdc = FILE:/var/log/krb5kdc.log
admin_server = FILE:/var/log/kadmind.log

[libdefaults]
default_realm = REALM
dns_lookup_realm = true
dns_lookup_kdc = true
ticket_lifetime = 24h
forwardable = yes
[realms]
EXAMPLE.COM = {
kdc = kerberos.example.com:88
admin_server = kerberos.example.com:749
default_domain = example.com
}
[domain_realm]
.example.com = EXAMPLE.COM
example.com = EXAMPLE.COM
[kdc]
profile = /var/kerberos/krb5kdc/kdc.conf
[appdefaults]
pam = {
debug = false
ticket_lifetime = 36000
renew_lifetime = 36000
forwardable = true
krb4_convert = false
}
Move the keytab file to some directory related to squid, like /etc/squid, and must change the permission like this:

chmod 400 /etc/squid/squid.domain.keytab
chown nobody /etc/squid/squid.domain.keytab

You can test is the squid box can use the keytab file using kinit command:
kinit -V -k -t squid.domain.keytab HTTP/squid.domain
klist


You need to put a line like this in squid startup script:

export KRB5_KTNAME=/etc/squid/squid.domain.keytab
In squid.conf you need set directives like this:
auth_param negotiate program /usr/local/squid/libexec/squid_kerb_auth -d
auth_param negotiate children 10
auth_param negotiate keep_alive on

acl AUTENTICADO proxy_auth REQUIRED
http_access allow AUTENTICADO
http_access deny all

Aditional reference about kerberos authentication for Squid:

• http://sourceforge.net/mailarchive/message.php?msg_id=11486457
• https://sourceforge.net/forum/message.php?msg_id=1853553

Check the /etc/hosts file to certify that the fqdn used on keytab file are associated to the correct address on hosts files, as well on DNS (A and PTR record). Dont associate the fqdn with loopback or other address.

Basic authentication

To make users not authenticated by a kerberos infra-structure keep using squid, you can use a Basic Authentication with many methods, I chose Ldap because the Samba/Winbind (that I already used) some times has problems of stability and a unknown timeouts of and refresh for password change and group membership.

To use this authentication, you can put this lines after the kerberos related lines (this is very important once that the order of method used to authenticated in browser is derived from this order, if you change this you’ll never get a negotiate authentication only basic):

auth_param basic program /usr/local/squid/libexec/squid_ldap_auth -b "CN=Users,dc=bc" -s sub -D "CN=user_to_bind_in_ldap,OU=Usuers,DC=bc" -w password -f "(&(objectClass=person)(sAMAccountName=%s))" -u sAMAccountName -P -v 3 -h ldap_server.domain
auth_param basic children 10
auth_param basic realm Proxy Authentication
auth_param basic credentialsttl 2 hours

Authorization

Sometimes you can control how access the Internet, sometimes you can control how can’t, or when they access, or yet where the peoples go. For achieve this you should use some directives from squid config file, like: ACL, External_ACL, http_access, http_reply_access, etc.

How we are doing the authentication with kerberos, the username (%LOGIN for squid external_acl) become like this: username@REALM. This is a problem for us once that we are trying to check the attribute sAMAccountName in Active Directory, that don’t include Realm, so we need to apply the patch bellow, and use the new parameter -K in squid_ldap_group to strip the Realm from username.


--- squid_ldap_group.c.original 2008-01-10 11:08:12.000000000 -0200
+++ squid_ldap_group.c 2008-01-04 19:35:09.000000000 -0200
@@ -215,6 +215,7 @@
int port = LDAP_PORT;
int use_extension_dn = 0;
int strip_nt_domain = 0;
+ int strip_kerberos_realm = 0;
int err = 0;

setbuf(stdout, NULL);
@@ -370,6 +371,9 @@
case 'S':
strip_nt_domain = 1;
break;
+ case 'K':
+ strip_kerberos_realm = 1;
+ break;
default:
fprintf(stderr, PROGRAM_NAME " ERROR: Unknown command line option '%c'\n", option);
exit(1);
@@ -424,6 +428,7 @@
#endif
fprintf(stderr, "\t-g\t\t\tfirst query parameter is base DN extension\n\t\t\t\tfor this query\n");
fprintf(stderr, "\t-S\t\t\tStrip NT domain from usernames\n");
+ fprintf(stderr, "\t-K\t\t\tStrip Kerberos realm from usernames\n");
fprintf(stderr, "\n");
fprintf(stderr, "\tIf you need to bind as a user to perform searches then use the\n\t-D binddn -w bindpasswd or -D binddn -W secretfile options\n\n");
exit(1);
@@ -470,6 +475,12 @@
if (u && u[1])
user = u + 1;
}
+ if (strip_kerberos_realm) {
+ char *u = strchr(user, '@');
+ if (u!=NULL) {
+ *u = '';
+ }
+ }
if (use_extension_dn) {
extension_dn = strtok(NULL, " \n");
if (!extension_dn) {

and


--- squid_ldap_group.8.original 2008-01-10 11:08:21.000000000 -0200
+++ squid_ldap_group.8 2008-01-07 11:36:45.000000000 -0200
@@ -152,6 +152,10 @@
Strip NT domain name component from user names (/ or \\ separated)
.
.TP
+.BI -K
+Strip Kerberos Realm component from user names (@ separated)
+.
+.TP
.BI -d
Debug mode where each step taken will get reported in detail.
Useful for understanding what goes wrong if the results is


To allow or deny some user with account in Active Directory based on they group membership, you can use Ldap to check the this using this lines on squid.conf:

external_acl_type ldapgroup %LOGIN /usr/local/squid/libexec/squid_ldap_group -b DC=bc -f
"(&(objectclass=person)(sAMAccountName=%v)(memberOf=cn=%a,OU=Security Groups,DC=REALM))" -D
CN=user_to_bind_in_ldap,CN=Users,DC=REALM -w password -h ldap-server.domain -v 3 -K

acl ldapgroup-access external ldapgroup @users_group

http_access allow all ldapgroup-access

Is possible to check other attributes as well, just change your Ldap query.