How to install Webhosting Guide

When dealing with mem-leaks in my mod_perl-apps I ran into a curious apache-problem. After a while apache could not be started but failed with strange errors like:
[emerg] (28)No space left on device: Couldn’t create accept lock
or
[crit] (28)No space left on device: mod_rewrite: could not create rewrite_log_lock Configuration Failed
or
[Wed Dec 07 00:00:09 2005] [error] (28)No space left on device: Cannot create SSLMutex
There was definitely enough space on the device where the locks are stored (default /usr/local/apache2/logs/). I tried to explicetely different Lockfiles using the LockFile-directive but this did not help. I also tried a non-default AcceptMutex (flock) which then solved the acceptlock-issue and ended in the rewrite_log_lock-issue.
Only reboot of the system helped out of my crisis.
Solution: There were myriads of semaphore-arrays left, owned by my apache-user.
ipcs -s | grep apache
Removing this semaphores immediately solved the problem.
ipcs -s | grep apache | perl -e 'while (<STDIN>) { @a=split(/\s+/); print `ipcrm sem $a[1]`}'

or with theese command

ipcs -s | grep apache | awk ‘ { print $2 } ‘ | xargs ipcrm sem

thank you for this helpful Article to Carlosrivero

No comments

This tutorial is designed to guide you through the initial steps of setting up Apache, MySQL, and PHP on Linux. The Linux distribution being utilized for this tutorial is Fedora Core 1, however the steps should be very similar across most distributions. This tutorial makes the assumption that you have the required development tools loaded for compiling programs from source, these tools are beyond the scope of this document and will not be covered here. Also, it assumes you can use the vi text editor for basic editing tasks.

 

Apache, MySQL, and PHP have become one of the most utilized combinations for developing content driven websites. They are robust, flexible, provide a decent level of security, and they are available for many different platforms. That being said, lets get to building a web server.

The first thing you need to do is obtain the sourceballs for each package, we will be compiling each package from scratch here, and, while there are also binary packages available for some distributions, I find your end results are usually better when building each package for your machine. Make sure you get the source files.

Here are the links and the package versions available at the time this tutorial was written

Apache
URL : http://httpd.apache.org/download.cgi
Current Version - 2.0.48

MySQL
URL : http://www.mysql.com/downloads/mysql-4.0.html
Current Version - 4.0.16

PHP
URL : http://www.php.net/downloads.php
Current Version - 4.3.4

Ok, so you’ve got the files now what ?, well now the fun begins..

Installation -

The first thing we need to do is extract the sourceballs so we can work with the files included in them. Beginning now we will be working as root, so open a terminal window, change to the directory in which you saved your downloaded files and become root by issuing the su command, enter the root password and you should be good to go.

To extract the sourceballs type the following commands;

#tar -zxf httpd-2.0.48.tar.gz (enter)

#tar -zxf mysql-4.0.16.tar.gz (enter)

#tar -zxf php-4.3.4.tar.gz (enter)

The commands above will extract the sourceballs into their own separate directories. Now lets move on to compiling the source into usable programs. We’ll start with Apache.

Compiling Apache -

Change into the directory created when you untarred the sourceball as follows;

#cd httpd-2.0.48 (enter)

Follow this command by typing;

#./configure –prefix=/usr/local/apache2 –enable-mods-shared=most (enter)

This tells Apache to install in the /usr/local/apache2 directory, and to build most of the available loadable modules. There are a ton of options with Apache, but these should work for the most part. Once the configure is done and the system returns the prompt to you, issue the following command;

#make

This will take a few minutes, once the prompt comes back again issue the following command;

#make install

Wait for a few minutes and viola !, Apache is installed with the exception of a few minor changes we still need to make. They are as follows..

Issue the following command;

#vi /usr/local/apache2/conf/httpd.conf

Check to make sure the following line is present in the file at the bottom of the LoadModule list, if it is not there add it;

LoadModule php4_module modules/libphp4.so

Find the DirectoryIndex line and edit it so it looks like the following;

DirectoryIndex index.html index.html.var index.php

Find the AddType application section and add the following line;

AddType application/x-httpd-php .php

Thats it, save the file and we are done with Apache. Now, on to MySQL !

Compiling MySQL -

Change into the MySQL source directory as follows;

#cd mysql-4.0.16 (enter)

Follow this command by typing;

#./configure –prefix=/usr/local/mysql –localstatedir=/usr/local/mysql/data –disable-maintainer-mode –with-mysqld-user=mysql –enable-large-files-without-debug (enter)

Sit back and wait for a while while configure does its thing, once the system returns the prompt to you issue the following command;

#make (enter)

Unless you have a very fast machine this will take some time, so spend time with your family, grab a beer, go for a walk, or whatever you’re into. When you get back, assuming the system has returned the prompt to you issue the following command;

#make install (enter)

Cool !, MySQL is installed, there are only a couple things left to do to get it working, first we need to create a group for MySQL as follows;

#/usr/sbin/groupadd mysql (enter)

Then we create a user called mysql which belongs to the mysql group;

#/usr/sbin/useradd -g mysql mysql (enter)

Now we install the database files as follows;

#./scripts/mysql_install_db (enter)

Then we make a couple minor ownership changes;

# chown -R root:mysql /usr/local/mysql (enter)

# chown -R mysql:mysql /usr/local/mysql/data (enter)

Last but not least, we use vi to add a line the ld.so.conf file as follows;

#vi /etc/ld.so.conf

And we add the following line;

/usr/local/mysql/lib/mysql

Thats it, MySQL is installed, you can run it by issuing the following command;

#/usr/local/mysql/bin/mysqld_safe –user=mysql &

And as long as we’re here we might as well set a root password for MySQL as follows;

#/usr/local/mysql/bin/mysqladmin -u root password new_password

Where new_password is the password you want to use.

Ok, so far so good, on to PHP !

Compiling PHP -

Change into the PHP source directory as follows;

#cd php-4.3.4 (enter)

Follow this command by typing;

#./configure –prefix=/usr/local/php –with-apxs2=/usr/local/apache2/bin/apxs –with-mysql=/usr/local/mysql (enter)

Once the prompt comes back to you issue the following command;

#make (enter)

Hang out for awhile, and then yep, you guessed it, once you have the prompt back;

#make install (enter)

Once the install finishes and you have the prompt back issue the following command;

#cp php.ini-recommended /usr/local/php/lib/php.ini (enter)

Then edit that file;

#vi /usr/local/php/lib/php.ini (enter)

And change the following;

Find the doc_root section and enter the correct path for the directory which serves your web content, such as;

doc_root= “/usr/local/apache2/htdocs/”

(this is default for apache2)

Then find the file_uploads section and change it to reflect the following;

file_uploads=Off

(for security reasons)

Thats if for PHP, now lets see if it all works..

Testing -

Assuming your MySQL process is still running from earlier, lets start Apache by issuing the following command;

#/usr/local/apache2/bin/apachectl start (enter)

This starts the Apache web server, now change into the following directory;

#cd /usr/local/apache2/htdocs (enter)

And using vi create a file called test.php;

#vi test.php

Add the following line to the file;

Save the file, then fire up your browser and point it to localhost/test.php. You should see a listing of all kinds of cool info about Apache, PHP, etc. If you do then your set !, if you don’t, then take a look at your logs for Apache and MySql, and remember Google is your friend. But hopefully you do, and now you have a fully functioning setup.

Ok, one last step and we’ll be done, you have everything running now, but you had to start Apache and MySql manually, that’s something you don’t want to have to remember to do everytime you reboot your machine, so lets fix it.

Starting Apache and MySQL Automatically -

Lets start with MySQL, as root make your working directory that of the MySQL source directory you worked with earlier, something similar to;

#cd /home/xxxx/mysql-4.0.16

Then, copy the file mysql.server to your /etc/init.d directory as follows;

#cp support-files/mysql.server /etc/init.d/mysql

Ok, lets create some links in the startup folders for run levels 3 and 5.

#cd /etc/rc3.d

#ln -s ../init.d/mysql S85mysql

#ln -s ../init.d/mysql K85mysql

#cd /etc/rc5.d

#ln -s ../init.d/mysql S85mysql

#ln -s ../init.d/mysql K85mysql

#cd ../init.d

#chmod 755 mysql

Thats it for MySQL, it should start automatically now when you reboot your machine. Now lets do the same for Apache, still as root make your working directory that of the Apache binaries as follows;

#cd /usr/local/apache2/bin

Then, copy the file called apachectl as follows;

#cp apachectl /etc/init.d/httpd

Now, for some more links;

#cd /etc/rc3.d

#ln -s ../init.d/httpd S85httpd

#ln -s ../init.d/httpd K85httpd

#cd /etc/rc5.d

#ln -s ../init.d/httpd S85httpd

#ln -s ../init.d/httpd K85httpd

And thats it for Apache !, it should start automatically along with MySQL the next time you boot your machine.

That brings us to the end of this tutorial, hopefully you found it helpful, and Good Luck !

No comments

Guide to .htaccess tutorial and tips

Introduction

In this tutorial you will find out about the .htaccess file and the power it has to improve your website. Although .htaccess is only a file, it can change settings on the servers and allow you to do many different things, the most popular being able to have your own custom 404 error pages. .htaccess isn’t difficult to use and is really just made up of a few simple instructions in a text file.

Will My Host Support It?

This is probably the hardest question to give a simple answer to. Many hosts support .htaccess but don’t actually publicise it and many other hosts have the capability but do not allow their users to have a .htaccess file. As a general rule, if your server runs Unix or Linux, or any version of the Apache web server it will support .htaccess, although your host may not allow you to use it.

A good sign of whether your host allows .htaccess files is if they support password protection of folders. To do this they will need to offer .htaccess (although in a few cases they will offer password protection but not let you use .htaccess). The best thing to do if you are unsure is to either upload your own .htaccess file and see if it works or e-mail your web host and ask them.

What Can I Do?

You may be wondering what .htaccess can do, or you may have read about some of its uses but don’t realise how many things you can actually do with it.

There is a huge range of things .htaccess can do including: password protecting folders, redirecting users automatically, custom error pages, changing your file extensions, banning users with certian IP addresses, only allowing users with certain IP addresses, stopping directory listings and using a different file as the index file.

Creating A .htaccess File

Creating a .htaccess file may cause you a few problems. Writing the file is easy, you just need enter the appropriate code into a text editor (like notepad). You may run into problems with saving the file. Because .htaccess is a strange file name (the file actually has no name but a 8 letter file extension) it may not be accepted on certain systems (e.g. Windows 3.1). With most operating systems, though, all you need to do is to save the file by entering the name as:

“.htaccess”

(including the quotes). If this doesn’t work, you will need to name it something else (e.g. htaccess.txt) and then upload it to the server. Once you have uploaded the file you can then rename it using an FTP program.

Warning

Before beginning using .htaccess, I should give you one warning. Although using .htaccess on your server is extremely unlikely to cause you any problems (if something is wrong it simply won’t work), you should be wary if you are using the Microsoft FrontPage Extensions. The FrontPage extensions use the .htaccess file so you should not really edit it to add your own information. If you do want to (this is not recommended, but possible) you should download the .htaccess file from your server first (if it exists) and then add your code to the beginning.

Custom Error Pages

The first use of the .htaccess file which I will cover is custom error pages. These will allow you to have your own, personal error pages (for example when a file is not found) instead of using your host’s error pages or having no page. This will make your site seem much more professional in the unlikely event of an error. It will also allow you to create scripts to notify you if there is an error (for example I use a PHP script on Free Webmaster Help to automatically e-mail me when a page is not found).

You can use custom error pages for any error as long as you know its number (like 404 for page not found) by adding the following to your .htaccess file:

ErrorDocument errornumber /file.html

For example if I had the file notfound.html in the root directory of my site and I wanted to use it for a 404 error I would use:

ErrorDocument 404 /notfound.html

If the file is not in the root directory of your site, you just need to put the path to it:

ErrorDocument 500 /errorpages/500.html

These are some of the most common errors:

401 - Authorization Required
400 - Bad request
403 - Forbidden
500 - Internal Server Error
404 - Wrong page

Then, all you need to do is to create a file to display when the error happens and upload it and the .htaccess file.

Part 2

In part 2 I will show you how to use some of the other .htaccess functions to improve your website.

 

Introduction

In the last part I introduced you to .htaccess and some of its useful features. In this part I will show you how to use the .htaccess file to implement some of these.

Stop A Directory Index From Being Shown

Sometimes, for one reason or another, you will have no index file in your directory. This will, of course, mean that if someone types the directory name into their browser, a full listing of all the files in that directory will be shown. This could be a security risk for your site.

To prevent against this (without creating lots of new ‘index’ files, you can enter a command into your .htaccess file to stop the directory list from being shown:

Options -Indexes

Deny/Allow Certian IP Addresses

In some situations, you may want to only allow people with specific IP addresses to access your site (for example, only allowing people using a particular ISP to get into a certian directory) or you may want to ban certian IP addresses (for example, keeping disruptive memembers out of your message boards). Of course, this will only work if you know the IP addresses you want to ban and, as most people on the internet now have a dynamic IP address, so this is not always the best way to limit usage.

You can block an IP address by using:

deny from 000.000.000.000

where 000.000.000.000 is the IP address. If you only specify 1 or 2 of the groups of numbers, you will block a whole range.

You can allow an IP address by using:

allow from 000.000.000.000

where 000.000.000.000 is the IP address. If you only specify 1 or 2 of the groups of numbers, you will allow a whole range.

If you want to deny everyone from accessing a directory, you can use:

deny from all

but this will still allow scripts to use the files in the directory.

Alternative Index Files

You may not always want to use index.htm or index.html as your index file for a directory, for example if you are using PHP files in your site, you may want index.php to be the index file for a directory. You are not limited to ‘index’ files though. Using .htaccess you can set foofoo.blah to be your index file if you want to!

Alternate index files are entered in a list. The server will work from left to right, checking to see if each file exists, if none of them exisit it will display a directory listing (unless, of course, you have turned this off).

DirectoryIndex index.php index.php3 messagebrd.pl index.html index.htm

Redirection

One of the most useful functions of the .htaccess file is to redirect requests to different files, either on the same server, or on a completely different web site. It can be extremely useful if you change the name of one of your files but allow users to still find it. Another use (which I find very useful) is to redirect to a longer URL, for example in my newsletters I can use a very short URL for my affiliate links. The following can be done to redirect a specific file:

Redirect /location/from/root/file.ext http://www.othersite.com/new/file/location.xyz

In this above example, a file in the root directory called oldfile.html would be entered as:

/oldfile.html

and a file in the old subdirectory would be entered as:

/old/oldfile.html

You can also redirect whole directoires of your site using the .htaccess file, for example if you had a directory called olddirectory on your site and you had set up the same files on a new site at: http://www.newsite.com/newdirectory/ you could redirect all the files in that directory without having to specify each one:

Redirect /olddirectory http://www.newsite.com/newdirectory

Then, any request to your site below /olddirectory will bee redirected to the new site, with the extra information in the URL added on, for example if someone typed in:

http://www.youroldsite.com/olddirecotry/oldfiles/images/image.gif

They would be redirected to:

http://www.newsite.com/newdirectory/oldfiles/images/image.gif

This can prove to be extremely powerful if used correctly.

Part 3

In part 3 I will cover a few other uses of the .htaccess file including password protection.

Introduction

Although there are many uses of the .htaccess file, by far the most popular, and probably most useful, is being able to relaibly password protect directories on websites. Although JavaScript etc. can also be used to do this, only .htaccess has total security (as someone must know the password to get into the directory, there are no ‘back doors’)

The .htaccess File

Adding password protection to a directory using .htaccess takes two stages. The first part is to add the appropriate lines to your .htaccess file in the directory you would like to protect. Everything below this directory will be password protected:

AuthName “Section Name”
AuthType Basic
AuthUserFile /full/path/to/.htpasswd
Require valid-user

There are a few parts of this which you will need to change for your site. You should replace “Section Name” with the name of the part of the site you are protecting e.g. “Members Area”.

The /full/parth/to/.htpasswd should be changed to reflect the full server path to the .htpasswd file (more on this later). If you do not know what the full path to your webspace is, contact your system administrator for details.

The .htpasswd File

Password protecting a directory takes a little more work than any of the other .htaccess functions because you must also create a file to contain the usernames and passwords which are allowed to access the site. These should be placed in a file which (by default) should be called .htpasswd. Like the .htaccess file, this is a file with no name and an 8 letter extension. This can be placed anywhere within you website (as the passwords are encrypted) but it is advisable to store it outside the web root so that it is impossible to access it from the web.

Entering Usernames And Passwords

Once you have created your .htpasswd file (you can do this in a standard text editor) you must enter the usernames and passwords to access the site. They should be entered as follows:

username:password

where the password is the encrypted format of the password. To encrypt the password you will either need to use one of the premade scripts available on the web or write your own. There is a good username/password service at the KxS site which will allow you to enter the user name and password and will output it in the correct format.

For multiple users, just add extra lines to your .htpasswd file in the same format as the first. There are even scripts available for free which will manage the .htpasswd file and will allow automatic adding/removing of users etc.

Accessing The Site

When you try to access a site which has been protected by .htaccess your browser will pop up a standard username/password dialog box. If you don’t like this, there are certain scripts available which allow you to embed a username/password box in a website to do the authentication. You can also send the username and password (unencrypted) in the URL as follows:

http://username:password@www.website.com/directory/

Summary

.htaccess is one of the most useful files a webmaster can use. There are a wide variety of different uses for it which can save time and increase security on your website.

Thanks to freewebmasterhelp.com for providing this article.

No comments

Apache Log Files Explained

Configure Web Logs in Apache
Author’s Note: While most of this piece discusses configuration options for any operating system Apache supports, some of the content will be Unix/Linux (*nix) specific, which now includes Macintosh OS X and its underlying Unix kernel.

One of the many pieces of the Website puzzle is Web logs. Traffic analysis is central to most Websites, and the key to getting the most out of your traffic analysis revolves around how you configure your Web logs. Apache is one of the most — if not the most — powerful open source solutions for Website operations. You will find that Apache’s Web logging features are flexible for the single Website or for managing numerous domains requiring Web log analysis.

For the single site, Apache is pretty much configured for logging in the default install. The initial httpd.conf file (found in /etc/httpd/conf/httpd.conf in most cases) should have a section on logs that looks similar to this (Apache 2.0.x), with descriptive comments for each item. Your default logs folder will be found in /etc/httpd/logs . This location can be changed when dealing with multiple Websites, as we’ll see later. For now, let’s review this section of log configuration.

ErrorLog logs/error_log

LogLevel warn

LogFormat “%h %l %u %t “%r” %>s %b “%{Referer}i” “%{User-Agent}i”" combined
LogFormat “%h %l %u %t “%r” %>s %b” common
LogFormat “%{Referer}i -> %U” referer
LogFormat “%{User-agent}i” agent

CustomLog logs/access_log combined

Error Logs
The error log contains messages sent from Apache for errors encountered during the course of operation. This log is very useful for troubleshooting Apache issues on the server side.

Apache Log Tip: If you are monitoring errors or testing your server, you can use the command line to interactively watch log entries. Open a shell session and type “tail –f /path/to/error_log” . This will show you the last few entries in the file and also continue to show new entries as they occur.

There are no real customization options available, other than telling Apache where to establish the file, and what level of error logging you seek to capture. First, let’s look at the error log configuration code from httpd.conf.

ErrorLog logs/error_log

You may wish to store all error-related information in one error log. If so, the above is fine, even for multiple domains. However, you can specify an error log file for each individual domain you have. This is done in the container with an entry like this:

DocumentRoot “/home/sites/domain1/html/”
ServerName domain1.com
ErrorLog /home/sites/domain1/logs/error.log

If you are responsible for reviewing error log files as a server administrator, it is recommended that you maintain a single error log. If you’re hosting for clients, and they are responsible for monitoring the error logs, it’s more convenient to specify individual error logs they can access at their own convenience.

The setting that controls the level of error logging to capture follows below.

LogLevel warn

Apache’s definitions for their error log levels are as follows:

 

Tracking Website Activity
Often by default, Apache will generate three activity logs: access, agent and referrer. These track the accesses to your Website, the browsers being used to access the site and referring urls that your site visitors have arrived from.

It is commonplace now to utilize Apache’s “combined” log format, which compiles all three of these logs into one logfile. This is very convenient when using traffic analysis software as a majority of these third-party programs are easiest to configure and schedule when only dealing with one log file per domain.

Let’s break down the code in the combined log format and see what it all means.

LogFormat “%h %l %u %t “%r” %>s %b “%{Referer}i” “%{User-Agent}i”" combined

LogFormat starts the line and simply tells Apache you are defining a log file type (or nickname), in this case, combined. Now let’s look at the cryptic symbols that make up this log file definition.

 

To review all of the available configuration codes for generating a custom log, see Apache’s [1] docs on the module_log_config , which powers log files in Apache.

Apache Log Tip: You could capture more from the HTTP header if you so desired. A full listing and definition of data in the header is found at the World Wide Web Consortium [2] .

For a single Website, the default entry would suffice:

CustomLog logs/access_log combined

However, for logging multiple sites, you have a few options. The most common is to identify individual log files for each domain. This is seen in the example below, again using the log directive within the container for each domain.

DocumentRoot “/home/sites/domain1/html/”
ServerName domain1.com
ErrorLog /home/sites/domain1/logs/error.log
CustomLog /home/sites/domain1/logs/web.log

DocumentRoot “/home/sites/domain2/html/”
ServerName domain2.com
ErrorLog /home/sites/domain2/logs/error.log
CustomLog /home/sites/domain2/logs/web.log

DocumentRoot “/home/sites/domain3/html/”
ServerName domain3.com
ErrorLog /home/sites/domain3/logs/error.log
CustomLog /home/sites/domain3/logs/web.log

In the above example, we have three domains with three unique Web logs (using the combined format we defined earlier). A traffic analysis package could then be scheduled to process these logs and generate reports for each domain independently.

This method works well for most hosts. However, there may be situations where this could become unmanageable. Apache recommends a special single log file for large virtual host environments and provides a tool for generating individual logs per individual domain.

We will call this log type the cvh format, standing for “common virtual host.” Simply by adding a %v (which stands for virtual host) to the beginning of the combined log format defined earlier and giving it a new nickname of cvh, we can compile all domains into one log file, then automatically split them into individual log files for processing by a traffic analysis package.

LogFormat “%v %h %l %u %t “%r” %>s %b “%{Referer}i” “%{User-Agent}i”" cvh

In this case, we do not make any CustomLog entries in the containers and simply have one log file generated by Apache. A program created by Apache called split_logfile is included in the src/support directory of your Apache sources. If you did not compile from source or do not have the sources, you can get the Perl script [3] .

The individual log files created from your master log file will be named for each domain (virtual host) and look like: virtualhost.log.

Log Rotation
Finally, we want to address log rotation. High traffic sites will generate very large log files, which will quickly swallow up valuable disk space on your server. You can use log rotation to manage this process.

There are many ways to handle log rotation, and various third-party tools are available as well. However, we’re focusing on configurations native to Apache, so we will look at a simple log rotation scheme here. I’ll include links to more flexible and sophisticated log rotation options in a moment.

This example uses a rudimentary shell script to move the current Web log to an archive log, compresses the old file and keeps an archive for as long as 12 months, then restarts Apache with a pause to allow the log files to be switched out.

mv web11.tgz web12.tgz
mv web10.tgz web11.tgz
mv web9.tgz web10.tgz
mv web8.tgz web9.tgz
mv web7.tgz web8.tgz
mv web6.tgz web7.tgz
mv web5.tgz web6.tgz
mv web5.tgz web6.tgz
mv web4.tgz web5.tgz
mv web3.tgz web4.tgz
mv web2.tgz web3.tgz
mv web1.tgz web2.tgz
mv web.tgz web1.tgz
mv web.log web.old
/usr/sbin/apachectl graceful
sleep 300
tar cvfz web.tgz web.old

This code can be copied into a file called logrotate.sh , and placed inside the folder where your web.log file is stored (or whatever you name your log file, e.g. access_log, etc.). Just be sure to modify for your log file names and also chmod (change permissions on the file) to 755 so it becomes an executable.

This works fine for a single busy site. If you have more complex requirements for log rotation, be sure to see some of the following sites. In addition, many Linux distributions now come with a log rotation included. For example, Red Hat 9 comes with logrotate.d , a log rotation daemon which is highly configurable. To find out more, on your Linux system with logrotate.d installed, type man logrotate .

 

Log Rotation Sites
For more information on log roation, see:

cronolog [4]
modperl [5]

No comments

Howto mod_rewrite with Apache

About mod_rewrite for Apache

This module uses a rule-based rewriting engine (based on a regular-expression parser) to rewrite requested URLs on the fly. It supports an unlimited number of rules and an unlimited number of attached rule conditions for each rule to provide a really flexible and powerful URL manipulation mechanism. The URL manipulations can depend on various tests, for instance server variables, environment variables, HTTP headers, time stamps and even external database lookups in various formats can be used to achieve a really granular URL matching.

This module operates on the full URLs (including the path-info part) both in per-server context (httpd.conf) and per-directory context (.htaccess) and can even generate query-string parts on result. The rewritten result can lead to internal sub-processing, external request redirection or even to an internal proxy throughput.

But all this functionality and flexibility has its drawback: complexity. So don’t expect to understand this entire module in just one day.

Using Apache’s mod_rewrite

The Apache module mod_rewrite can be used to perform various forms of URI acrobatic manipulation. A prerequisite concept before attempting to understand mod_rewrite are regular expressions.

When a URL is requested by a server, this does not necessarily map directly to the server’s filesystem. This request can be twisted and turned to (hopefully) present more sense to the browsing user.

Why should I care?

A clean URL is part of a good user experience. It works as a breadcrumb trail - allowing the user to see where they are located in the site, it doesn’t break in bookmarks, you can easily send it over email, and allows users to guess where they want to go next. Most importantly a easy to read URL will be indexed by search engines such as Google. Having all your pages indexed creates a huge advantage to getting visitors to your website. Many search engine robots cannot read a URL with symbols such as ? & or commas and therefor not indexing your website.

This is all possible by using human-readable URLs:

“In principle, users should not need to know about URLs which are a machine-level addressing scheme. In practice, users often go to websites or individual pages through mechanisms that involve exposure to raw URLs.” — Jakob Neilsen, Jakob Nielsen’s Alertbox, March 21, 1999: URL as UI

“a URL should contain human-readable directory and file names that reflect the nature of the information space.” — Jakob Nielsen, item #4 Top Ten Mistakes in Web Design

By choosing a well thought-out URL, you won’t have to change it during the next re-organization. URLs that remain the same tend to pick up more links over time.

Getting started:

First, Apache must be compiled with the mod_rewrite module for any of this to take place. Insert these lines into the vhost definition for the domain that you want to work with.

RewriteEngine on
RewriteLog /path/to/logs/server.rewrite.txt
RewriteLogLevel 1

The first line turns the RewriteEngine on. Otherwise, extra code doesn’t get processed by the Apache webserver. Next, we specify where the logfile that records the rewrite activity should be placed. This is mostly for debugging, as your CustomLog should be keeping track of traffic.

A beginning example:

One of the simplest uses of mod_rewrite is to re-direct a web request from one page to another. Many times this will be done if the first has expired, was spelled wrong, or the site has a new naming scheme. It’s nice to forward new users to the correct page in case they have the previous one bookmarked, or if a search engine has cached the old location.

RewriteRule ^/biogarphy.php3 /biography/ [R=301]

This forwards a browser request from one page to the other. because the [R=301] at the end. I’ve taken a file that was spelled wrong, and fixed it at the same time removing a an old filetype suffix. (php4 has replaced that suffix with .php) What if I were to dump php from my system, and go with *.html, *.jsp, or even *.willie? By rewriting my URI to look like a directory, it doesn’t matter what filetype I’m using, nor what my DirectoryIndex options are.

Compound Example:

What if you used the above example, but didn’t decide to create a “biography” directory at your Doc-root? Apache can still be told where the content resides by including another RewriteRule following the first. Rules will continue attempting to match until a “last” case is presented with the [L] modifier at the end. This is much like a switch programming structure, using break to prevent each option from being executed.

RewriteRule ^/biography/ /biogarphy.php3 [L]

This might seem a little redundant, since we just did the opposite. This line will tell requests to “biography” to read the content from /biogarphy.php3 instead of looking for a biography directory. Confusing? Well, I could do this instead:

RewriteRule ^/(.+)/?$ /content/$1.php [L]

I can search on anything that follows the beginning slash, and replace the file request to look for that file in the content directory through the use of the regular expression and the backreference.

I’ve also placed this inside another directory that I don’t necessarily want the browsing user to see, or know about, but it’s easier for the webmaster to keep track of the roles of each file on the site. Since I’ve upgraded from php3 to php4 the suffix has changed.

More Advanced - the Query String:

The query string is passed in separately from the URL. This means that a simple regex doesn’t necessarily do the trick, but a compound statement using RewriteCond (condition) is required.

RewriteCond %{QUERY_STRING} id=([^&;]*)
RewriteRule ^/$ http://%{SERVER_NAME}/%1/? [R]
RewriteRule ^/([^/]*)/?$ /index.php?id=$1 [L]

The RewriteCondition matches only when the following condition is true, and continues until a “last” [L] is stated. The Condition’s backreferences are different, using the % prefix, and their scope lasts beyond the Condition line.

This above example would translate “/?id=home” into “/home/”, and then re-assign the value of “home” to the id HTTP_GET_VAR. One more thing to notice here is that the the second line has a trailing ? - this is used to negate copying of the query string into the new, re-directed URI.

More Reference Links:

http://www.engelschall.com/pw/apache/rewriteguide/
http://httpd.apache.org/docs/mod/mod_rewrite.html
http://httpd.apache.org/docs/misc/rewriteguide.html

No comments