SSL is the only choice for protecting web content in transit and verifying the identity of the server. However, often SSL is used when the only data that must be protected are usernames and passwords. Public discussion forums, public web mail and blogs are examples:  the content is already public so encrypting that data with SSL is pointless. All that is really needed for those types of applications is protection against interception of username and passwords.

The encryption routines of SSL seriously tax a web server when under load. Yahoo Mail, for example, avoids this overhead by using SSL only to encrypt the login page then switches to unencrypted HTTP after that.

There are many times when just the login credentials need to be protected. Acquiring a costly SSL server certificate from a certificate authority then writing the web app to use it only for authentication is overkill. Even with SSL is in place, the web application typically stores usernames and passwords in a database in clear text. An attacker breaking into the server has access to the authentication credentials of every user.

HTTP digest authentication is a little-known standard built into all the major web servers and web browsers. With it, login credentials are never transmitted across the network, nor are they stored on the server in plain text. There is no need to purchase an expensive SSL host certificate each year, nor worry about CPU load due to crypto processing overhead.  The web server performs authentication, removing the need to develop that code in the web application itself. The application only has to verify that authentication has taken place.

Digest authentication is not the same as it’s older, dumber brother Basic authentication. Web servers and browsers have always supported Basic authentication, where username and password are transmitted in easily-decoded Base64 and must be stored on the server in plain text. Digest authentication looks similar from the user’s perspective… the browser pops up a dialog box asking for username and password, but under the hood the protocol uses hashes instead of plain text.

Likewise, Digest authentication is not the same as it’s evil cousin NTLM authentication. Microsoft’s proprietary LAN Manager authentication (now dubbed "Integrated Windows Authentication") doesn’t transmit credentials in plain text, but requires a continuous TCP connection between server and client, violating the stateless nature of the HTTP protocol. NTLM has a hard time traversing most web proxies and load balancers (ironically, the only reliable way to get NTLM to do that is encapsulate it in SSL). Although it’s now semi-supported now in non-Microsoft browsers such as Firefox, NTLM is proprietary and not an Internet standard.

How Digest authentication works

Digest authentication is a challenge-response mechanism:

• The browser sends an HTTP request (e.g. a GET) to a web server
• The server sees the URL being accessed has been configured to require Digest authentication and replies with a 401 "Authentication Required" status plus a "nonce": a unique hash of several data items, one of which is a secret key known only to the server.
• The browser pops up a dialog box requesting username and password. Once the user enters their information, an MD5 hash of the username, password, nonce and URL are computed and the browser re-sends the original request along with the hash.
• The web server compares that hash with it’s own computation of the same values. If they match, the original HTTP request is allowed to complete.

For a complete explanation of Digest authentication see RFC 2617, but the idea is that since hashes are used instead of the actual data, an eavesdropper intercepting the communications never sees the username or password.

Replay attacks where the interceptor sends the exact same data at a later time are prevented in several ways. In the Apache web server, a unique nonce is generated for each 401 message and incorporates time and other information to prevent replays.

How to use it

Digest authentication is applied to specific URLs or physical directories on your web site. One convenient aspect is that protecting one directory usually results in Digest authentication being automatically applied to all lower directories as well. The specifics of implementing Digest authentication depend on the particular web server:

• Apache Digest authentication
• IIS 6.0 Digest authentication

For Apache, a text file is used to store the username in plain text plus an MD5 hash of the password. Credentials can also be stored in a database using Apache modules such as mod_authn_dbi from the 2.1 Authentication Project.

Rolling your own

Many web developers try to create their own authentication mechanism rather than use standard mechanism, sometimes for no other reason than they dislike appearance of the browser’s password pop-up dialog box. Many prominent web sites don’t even bother to try preventing sending username and password in the clear.

Some developers are more vigilant and try to roll their own hash mechanism to perform authentication. However, this involves writing either Javascript, a Java applet, ActiveX or similar on the client side to compute the hashes, plus a matching component in the server app. Cryptography is tricky. Even when it’s only using a cryptographic hash to verify credentials it takes care, time and effort to get it right. For example, we show what’s required to to use HMAC digesting to protect web forms in our paper Preventing HTML form tampering.

It’s much easier and more reliable to use the Digest mechanism that already exists in popular web browsers and servers. The dialog presented to the user is not pretty, but Digest authentication it can be implemented in nearly zero development time, with the added benefit that the implementation is more likely to be free of serious errors.

One recommendation: when writing a web app that relies on the web server to perform authentication, it’s a very good idea for the application to verify authentication has taken place. The default state for web servers is to allow access to a URL… only a single configuration directive causes the server to prompt for a username.

In Apache when a user has been authenticated the environment variable "REMOTE_USER" is set with the username. The web app should test for that and display an error if it’s not present. It’s too easy for a web server configuration mistake to disable the authentication prompt and allow wide open access to data: an additional check that authentication has occurred should exist in the web application itself.

Other caveats and gotchas

Digest authentication is appropriate where you need to protect only the username and password from compromise. When the actual content must be protected (i.e. form data sent from the browser) or when the user needs strong assurance that they are connecting to the right server (e.g. a bank), SSL is required.

Also, Digest authentication is of course single factor and thus subject to all the weaknesses of traditional username / password authentication. If strong authentication of the client is needed, look into implementing SSL client certificates or another two-factor mechanism.

I should also mention (before everyone else does) that the MD5 has algorithm has been shown to be cryptographically weaker than previously thought. A "collision" (an identical hash from different data) can be found in less time than it should. MD5 hashes are still considered strong, but the crypto community sees this discovery as a sign that it’s time to start moving to other mechanisms like SHA-256. However, because of the way MD5 hashes are used (e.g. combining them with a short-lived nonce), the issue doesn’t really affect Digest authentication.

Protecting the login credentials is often all that many web applications need. In those situations, Digest authentication provides an good lightweight authentication mechanism. It is a published standard supported by all major web servers and recent browsers, provides good protection against interception, and removes both the need to code your own login mechanism and to store usernames and passwords in plain text on the server.

No related posts.

openssl s_client -connect www.example.com:443

The above lets you connect via SSL to a web server and manually type HTTP commands just like you can with non-SSL web servers using plain old netcat and telnet.

The OpenSSL command-line utility is a swiss army knife for encryption protocols. You can use it for far more than just generating keys and connecting to SSL servers. But there are also many other command-line tools that are incredibly useful for SSL testing and discovery purposes:

• Gnu wget can fetch HTTPS URLs, if it has been compiled with SSL support. To view the HTTP response headers, try wget -S
• Lynx and elinks text-based web browsers (again, assume they have been compiled with SSL support)
• Cadaver for SSL-enabled WebDAV servers
• stunnel for tunnelling any traffic through SSL
• ssldump for sniffing and decrypting SSL traffic

SSL tools like these are valuable for assessing more than just web servers. About a year ago we had a contract to do a security assessment of Cisco Security Agent (a host intrusion prevention tool originally developed by "Okena"). After a bit of network sniffing and poking around the server directories, it turned out that CSA client and server communicate using HTTP encrypted with SSL over a non-standard port (a huge number of commercial products use simple HTTP and SSL as their client-server protocol. They just run it on a port other than 443 to disguise the fact).

To evaluate how robust the client and server were to attack, we used ssldump to decrypt and the raw HTTP traffic (this was possible only because the CSA server held the SSL host key).

Once we could capture the raw HTTP and understood how requests were formatted, we used openssl, stunnel and other tools to capture and replay communications between server and client, spoof traffic, impersonate endpoints, and "fuzz" the communications looking for overflows and other obvious vulnerabilities. The product withstood attack fairly well.

Were it not for the availability of these command-line tools, especially ssldump, performing the vulnerability assessment would been much more limited and involved coding a lot of custom SSL clients using something like Perl and the LWP library.

It always bugs me when vendors use the word "secure" when they really mean "encrypted". Encryption is only one tool that can be used in security, and usually it’s not the encryption that’s the weak point. In other products we’ve evaluated, a little sniffing and investigation revealed that while SSL was the method of encryption, the key size was 48 bits or lower, making brute force decryption feasible. Worse, some products ignore the authentication features of the SSL protocol, allowing either the client or server to be impersonated by an attacker with ease. There’s not much point in encrypting traffic when there’s no authentication of the client or server.

Many vendors use encryption to hide how weak their client-server communications really are. It’s a good thing that excellent open source tools like the above make vulnerability assessments of SSL-encrypted communications almost the same level of difficulty as assessing non-encrypted ones.

Related posts:

• OWASP meetings are depressing

By default PGP and GPG use public key cryptography which provides a form of two factor authentication: to decrypt, you must know both a passphrase and have the correct private key. If your passphrase is compromised (e.g. by someone shoulder surfing or via a keylogger), the attacker still has to have your private key to decrypt any files.

However, once in a while you need to send a sensitive file to someone who doesn’t use PGP or GPG regularly and doesn’t have a public key. Without a public key, you can’t encrypt so that only that person can decrypt.

In that situation either you talk the person through the finicky process of creating and publishing a PGP/GPG key pair, or take the easy way out and use weak encryption such as the "password protect" feature of MS Word or WinZip. Word’s "encryption" is easily cracked. The original Zip encryption is also weak, though products like WinZip have added proprietary ciphers that are stronger. 

A little known feature of both PGP and GPG is that they can also do symmetric encryption. Just a passphrase is needed- no public or private keys are involved. It’s a quick and dirty way to get strong encryption that even a novice can use.

To encrypt a file with symmetric encryption, the syntax is:

pgp --symmetric filename

gpg --symmetric filename

The result is a binary file with the same name as the original and ".pgp" or ".gpg" appended.

If the encrypted file must be pasted into the body of an e-mail message (instead of added as an attachment), you’ll want to use the plain ASCII form of output:

pgp --symmetric --armor filename

gpg --symmetric --armor filename

The result is a plain text file ending in ".asc"

Decryption is performed using the usual "-d" switch:

pgp -d filename

gpg -d filename

(The documentation for PGP and GPG describe the symmetric encryption option and all the command-line switches available for output formats).

Symmetric encryption with PGP/GPG uses a strong cipher (GPG uses CAST5) so it’s much more resistant to attack than using application encryption or WinZip. However, the passphase is the weak point: the longer and more complex the passphrase, the more secure the file. A single dictionary word can be brute forced in only a few hours, so use a complex passphrase of mutiple words broken up with letters and symbols.

Getting the passphrase to the recipient is another issue. Key distribution is always tricky with symmetric encryption schemes. If the encrypted file is sent via email, don’t also use email to send the passphrase (sounds obvious, but you’d be amazed how often people do that). Use the phone, send a fax, strap it to a pigeon… whatever method works best for the situation and won’t let an attacker intercept both the encrypted file and the passphrase.

Related posts:

• Protecting laptop data with TrueCrypt