A Million Monkeys

If you don’t know what memcache is, the official site says the following:

“memcached is a high-performance, distributed memory object caching system, generic in nature, but intended for use in speeding up dynamic web applications by alleviating database load.”

Basically, what this means is that memcached is a daemon running on a server which allows you to save and retrieve variables in the server memory. This is primarily used to ease database load on websites and applications. The logic for doing something like this is…

- Before running a query, check if the result is available in memcache.
- If the result is in memcache, return the cached result.
- If not, run the query on the database, and store the result in memcache.

To install memcached, simply run the following command:

sudo apt-get install memcached

Once it’s installed, edit /etc/memcached.conf and change the line beginning ‘-m’ which is the amount of memory (in megabytes) to allocate to the server. You can also change the IP address that the server listens on  in the line beginning ‘-l’.

Now restart the daemon by running

/etc/init.d/memcached restart

Now you have memcache set up and running on your server. A great feature of memcached is that you can easily cluster servers. If you want to do this, simply install memcache on your other servers before continuing.

At this point, I’d recommend downloading memcache.php which is a php script showing you a lot of useful information about your memcached servers. Once downloaded, put it in a web facing directory on your server, and modify the $MEMCACHE_SERVERS array with your server addresses.

Using memcache in php scripts is very easy. There are either procedural, or object oriented functions already available. Here is an example of a script which will store a simple variable and then retrieve it and display it.

<?php
$memcache = new Memcache;

$memcache->addServer(’10.2.0.245′, 11211) or die (“Could not connect”);
$memcache->addServer(’10.2.0.249′, 11211) or die (“Could not connect”);

$memcache->set(‘mytestvariable’, “this is the data in my test variable”, false, 60) or die (“Unable to save the data to the server”);
echo “Data has been stored in the cache<br />”;

$result = $memcache->get(‘mytestvariable’);
echo “Retrieved data from the server:<br/>”;

var_dump($result);

?>

Obviously if you only have one server then only use one addServer line.

The script above will store the test variable in the cache for 60 seconds. If you drop the timeout down to something like 1 second, and then sleep for 3 seconds before attempting to get the data, you will find that the data has expired and nothing will be returned.

It’s also useful to know that you can store anything which can be serialized in memcache. This means it’s safe to store things like arrays in the cache without having to ‘flatten’ the data beforehand.

Obviously how and where you implement a cache is entirely dependant on how your system works. If you have a lot of intense database usage then you will find that even caching with a short timeout will vastly reduce the amount of database queries. You should find that memcache is considerably faster than normal database access, and even faster than the mysql query cache. You also have total control over what data is cached, and how long it is cached for before your queries fall back to checking a database.

Some further reading:
Facebook engineering blog – How facebook use memcache
Fotolog case study

Also, it’s worth checking out the PHP manual page.

Bookmark It

I’m posting this mainly because I forget the exact syntax, but it might be useful for others.

This applies to all services on all ports, but for this example I’ll use the standard IMAP port. For example, if I have a mail server (listening for IMAP connections on port 143) which isn’t accessible from outside it’s LAN, but I have another server on the LAN which is accessible from outside – I could forward the port on the open server to give myself access to the mail server.

Here’s the example setup:
Mail server: 10.1.0.2, listening on port 143 which isn’t accessible from outside.
Other server 10.1.0.3, accessible from outside.

By running these 2 commands, I can forward all requests to 10.1.0.3:143 to 10.1.0.2:143 and connect to the IMAP server from anywhere:

iptables -t nat -I PREROUTING 1 -i eth0 -p tcp –dport 143 -j DNAT –to-destination 10.1.0.2:143
iptables -t nat -I POSTROUTING 1 -o eth0 -p tcp -d 10.1.0.2 –dport 143 -j SNAT –to-source 10.1.0.3

For this to work, you will need to have IP forwarding enabled, which can be done temporarily (which will reset on reboot), or permanently:

echo 1 > /proc/sys/net/ipv4/ip_forward
(temporary)

vim /etc/sysctl.conf
// uncomment the line ‘net.ipv4.ip_forward=1′
(permanent)

Now if you telnet to 10.1.0.3:143, the connection should be forwarded to 10.1.0.2:143

Bookmark It

This guide explains how to turn standard debian or ubuntu mysql-server installs into a full ndb cluster. As we’re using the standard mysql-server package, you won’t need to download any non .deb binaries or do any compiling. If everything goes well, this should take under 10 minutes to get working. I don’t plan to cover much theory here, as it is just a guide to getting the server up and running.

Please note that clustering works differently in MySQL 5.1, so this guide may not be relevant for that.

I also recommend the book ‘MySQL Clustering’ by Alex Davies and Harrison Fisk (ISBN 0-672-32855-0) as it explains how the cluster works in a lot more detail than I plan to here.

I will explain how to set up 3 servers, with the following roles:

server-a = management node (IP 10.1.0.10)
server-b = storage and sql node (IP 10.1.0.11)
server-c = storage and sql node (IP 10.1.0.12)

It is easy to have storage and sql nodes on separate servers, but to make this guide easier to follow (and so we don’t need 5 machines), I’ll use 3 servers.

All servers will need to have mysql-server installed. If you need help doing this, have a look at this guide.

Setting up the management node (10.1.0.10)

Default debian/ubuntu installs look for /etc/mysql/ndb_mgmd.cnf for the management node, so we need to create this file, with the following contents:

[NDBD DEFAULT]
NoOfReplicas=2
DataDir= /var/lib/mysql-cluster

# Management Node
[NDB_MGMD]
HostName=10.1.0.10
DataDir=/var/lib/mysql-cluster

# Storage Nodes (one for each node)
[NDBD]
HostName=10.1.0.11
DataDir=/var/lib/mysql-cluster
[NDBD]
HostName=10.1.0.12
DataDir=/var/lib/mysql-cluster

# SQL Nodes (one for each node)
[MYSQLD]
HostName=10.1.0.11
[MYSQLD]
HostName=10.1.0.12

The line beginning NoOfReplicas tells the cluster how many copies of data should be kept.

Now we start the management node, and it will sit waiting for connections from the storage and SQL nodes:

/etc/init.d/mysql-ndb-mgm start

Setting up SQL and data nodes

All we need to do here is make a few changes to /etc/mysql/my.cnf (the mysql config file)

First, add the following 2 lines (using the IP of your management node) inside the [mysqld] section of the config file:

ndbcluster
ndb-connectstring=10.1.0.10

And near the bottom of the file there will be a section for [MYSQL_CLUSTER] which you will need to uncomment, and change the ndb-connectstring line to your management nodes IP.

If you are using separate data and sql nodes, the [mysqld] part is relevant to the sql nodes only, and the [MYSQL_CLUSTER] part is relevant to the data nodes only.

Before we start the services, we have to create the /var/lib/mysql-cluster directory and set it to be owned by the mysql user:

mkdir /var/lib/mysql-cluster
chown mysql:mysql /var/lib/mysql-cluster

Now we need to start the node services:

/etc/init.d/mysql restart
(which starts the sql node)
/etc/init.d/mysql-ndb restart
(which starts the data node)

By now, everything should be running, so we connect to the management node (by running ndb_mgm from the command line) and check that the other nodes have connected properly (using the show command):

ndb_mgm> show;
Cluster Configuration
———————
[ndbd(NDB)]     2 node(s)
id=2    @10.1.0.11  (Version: 5.0.51, Nodegroup: 0)
id=3    @10.1.0.12  (Version: 5.0.51, Nodegroup: 0, Master)

[ndb_mgmd(MGM)] 1 node(s)
id=1    @10.1.0.10  (Version: 5.0.51)

[mysqld(API)]   2 node(s)
id=4    @10.1.0.11  (Version: 5.0.51)
id=5    @10.1.0.12  (Version: 5.0.51)

This shows that everything has connected properly. If connections are missing, then it’s worth checking /var/log/syslog on the affected server to see if there are any error messages.

Using the cluster

It’s worth noting that any databases already on the servers will continue to work as before. Tables only become part of the cluster when their engine type is changed to ‘ndbcluster’ by issuing this command (from a mysql prompt):

alter table tablename engine=ndbcluster;

But for now we will create a new database and clustered table, and test that the data is clustered. The cluster setup applies to tables only, not databases, so we first need to create the database on both sql nodes:

create database cluster;

Now, when we create a table inside the cluster database, as long as the engine is ndbcluster, the data will be synced across data nodes, which we can test by doing the following (from a mysql prompt on either sql node):

create table clustertest (i int) engine=ndbcluster;
insert into clustertest () values (1);
select i from clustertest;

Which should return the single row with the value of 1. Now connect to the other SQL node and try…

insert into clustertest () values (2);
select i from clustertest;

Which should return both rows, which will happen whichever SQL node you connect to as the table is now stored in the cluster.

If you receive an error like:

ERROR 1015 (HY000): Can’t lock file (errno: 4009)

Then it is likely that some of your cluster nodes haven’t started correctly, so its worth checking the management interface again.

Shutting down and restarting the cluster

To shutdown the data nodes and management node, all you need to do is enter the command ‘shutdown’ in the management interface.

To restart, simply run

/etc/init.d/mysql-ndb-mgm start

On the management node, and

/etc/init.d/mysql-ndb start

On the data nodes. The SQL nodes continue running, and can be stopped/started using the standard mysql init script.

That’s it, you’ve now should have a working MySQL cluster. As you have NoOfReplicas=2 in the management config, you should be able to unplug either data node at any time and still have access to all of the clustered tables.

Bookmark It

If you install and uninstall a lot of packages using apt, aptitude, or dpkg, you may find that you end up with a lot of redundant packages, or ones which have been uninstalled but the config files are left behind.

You can find packages which have been removed but still have config files by running (basically the status field will be ‘rc’):

dpkg -l | grep “^rc”

Or, suppose you installed mysql 5.1 from my previous post using the dotdeb repositories and wanted to see a list of what was installed:

dpkg -l | grep dotdeb

Now, removing those packages is easy, all you need to do is pass the list to dpkg –purge using awk in a similar way to grep above, so the 2 examples above would become:

dpkg --purge `dpkg -l | awk '/^rc/{print $2}'`
dpkg --purge `dpkg -l | awk '/dotdeb/{print $2}'`

(note that you may need to delete and retype the backticks depending on how cut and paste is handled)

Using the methods above, it should be easy to clean up your system.

Bookmark It

For debian/ubuntu users who want to use MySQL 5.1, there aren’t really many options available apart from compiling from source.

Whilst this is probably the best solution, a far simpler and easier (no need to worry about things like the client) method is using the repositories from dotdeb.org. Although these are made for debian installs, I tried it earlier on ubuntu 8.10 and it worked without any problems. As usual, back up your data first!

Assuming you’ve got mysql-server-5.0 installed (although if you don’t have the server installed at all it should still work), here are the steps to get 5.1.

First, you need to edit /etc/apt/sources.list and add the dotdeb repository. This is the line you need to add for Lenny, but just change it to Etch if you haven’t upgraded yet (see my earlier post if you want to know how to do that)

deb http://packages.dotdeb.org lenny all

Then, update…

apt-get update

And then install 5.1

apt-get install mysql-server-5.1

This should also install the dependancies: libmysqlclient16, mysql-client-5.1, and update mysql-common. If you already have 5.0 installed, this will also remove the server and client for it.

If you get a message saying that packages can’t be authenticated, it isn’t anything to worry about. Just select ‘y’ and continue. After the install is complete, you should have MySQL 5.1 installed with the most useful engines:

Server version: 5.1.32-0.dotdeb.0 (Debian)

You can also use the dotdeb repositories to upgrade to the latest versions of apache, and php.

It’s probably worth reading this guide on apt pinning, which is the best way to maintain a system using repositories with different versions of the same packages.

Bookmark It

I finally decided to take the plunge and upgrade a few of my Etch servers to Lenny, and the process was a lot less painful than I was expecting.

These are basically the steps from the official manual, but I cant guarantee that your system will upgrade as easily as mine – as always when  upgrading you should backup your data and be prepared for the worst.

Usually I prefer apt-get over aptitude, but the official documentation recommends aptitude to do the upgrade, so heres what you need to do (as root)

First, edit /etc/apt/sources.list and change all mentions of etch to lenny. If you use vim, you can simply do the following:

vim /etc/apt/sources.list
(then, in vim)
:%s/etch/lenny/g
:wq

Then update…

aptitude update

At this point, you might receive the following error (I did on every server I upgraded)

W: There is no public key available for the following key IDs:
4D270D06F42584E6

W: You may want to run apt-get update to correct these problems

To fix this, you need to install the following:

aptitude install debian-archive-keyring
aptitude update

Now you’re ready to do the upgrade. The safest way to do this is in 3 parts

aptitude install aptitude
(updating apt first is a safe way of doing things)
aptitude upgrade
(when this finishes)
aptitude dist-upgrade

When this has finished, your system is upgraded to Lenny! All you need to do now is reboot, and hopefully the system will come back up and be running Lenny.

Bookmark It

My main reason for posting this is completeness (so I’ve got guides for everything you need for a basic hosting server here), not because it’s difficult to do.

Getting mysql server installed and running is simple to do:

apt-get install mysql-server

Once this is done, you will probably want to set the root password, or add an admin user:

mysql
grant all privileges on *.* to ‘username’@'host’ identified by ‘password’ with grant option;

Obviously use your own username, host, and password. To allow connections from any host, you can use ‘%’

Next, I usually change the config so MySQL is listening on all available IP addresses (instead of just localhost). This is done by commenting out the ‘bind-address’ line in /etc/mysql/my.cnf and restarting the server:

/etc/init.d/mysql restart

That’s it, you’ve now got a working MySQL server.

Bookmark It

This guide assumes that you’ve already got a server running either debian or ubuntu, and you want to make it serve web pages.

First, install apache2, php, and some other useful modules

apt-get install apache2 libapache2-mod-php5 php5-cli php5-curl php5-gd php5-imap php5-common php5-mysql

That’s it, you should now be able to browse to http://localhost and see the default apache page!

By default, the webroot is /var/www/ but if you’re planning to host a domain you’ll probably want to point it somewhere else. This is how you configure apache to listen for the domain www.example.com and set the webroot as /home/www/www.example.com/ (it’s assumed that you’ve already got an A record in the sites DNS pointing to this server)

/etc/init.d apache2 stop

Then create the apache zone file in /etc/apache2/sites-available/www.example.com/ with the following content:

<VirtualHost x.x.x.x:80>
ServerAdmin me@example.com
DocumentRoot /home/www/www.example.com
ServerName www.example.com
ErrorLog /var/log/apache2/error_log
CustomLog /var/log/apache2/access_log combined
EnableSendfile Off
EnableMMap Off
</VirtualHost>

Replacing x.x.x.x with your servers IP address.

Next, you need to enable the site, and start apache.

a2ensite www.example.com
/etc/init.d/apache2 start

That’s it!

Bookmark It

This is little bit later than I originally intended but I finally got around to setting up OpenVPN, and here’s how I did it.

This guide is pretty simple to follow and should have an OpenVPN server on debian or ubuntu working within half an hour. I’ll also explain how to connect to the VPN from a windows PC.

First, install OpenVPN on the server (you’ll need to be root for all of this guide)

apt-get install openvpn

Next, we need to configure the server. You need to make a decision here whether you want tun (routed) or tap (bridged) connections. The main difference is that tap will give the client a network address on the server network, whereas tun creates a private network managed by the server. In this guide I will use tap because I find that it works better with windows clients.

Now you need to create certificates for the server and client for authentication purposes (which is much more secure than the passwords used in pptp). This is done through a number of steps:

Preparing to generate the keys

mkdir /etc/openvpn/easy-rsa
cp /usr/share/doc/openvpn/examples/easy-rsa/2.0/* /etc/openvpn/easy-rsa

Now you need to edit /etc/openvpn/easy-rsa/vars with your required settings. You only really need to change the last section which is the default values for the fields in the certificates.

Generate the certificate authority (CA) which will be used to sign the server and client certificates.

cd /etc/openvpn/easy-rsa
source ./vars
./clean-all
./build-ca

Next, we need to create the server keys

./build-key-server servername

Answer ‘yes’ when asked to sign the certificate and commit to the database, and then you’ll need to generate the diffie-hellman parameters which are used for key exchange between the client and server.

./build-dh

And finally, create some client keys which will be used to allow clients to authenticate with the server. I prefer to use pkcs12 which stores the client public key and certificate in one passworded file.

./build-key-pkcs12 client1

As before, sign the key and commit to the database. You will be asked for a password which the client will use to connect to the server.

Now all the keys are created, we need to configure the server.

vim /etc/openvpn/server.conf
(add the following lines)
port 443
proto tcp
dev tap
ca /etc/openvpn/easy-rsa/keys/ca.crt
cert /etc/openvpn/easy-rsa/keys/servername.crt
key /etc/openvpn/easy-rsa/keys/servername.key
dh /etc/openvpn/easy-rsa/keys/dh1024.pem
ifconfig-pool-persist ipp.txt
server-bridge 10.1.0.1 255.255.255.0 10.1.0.236 10.1.0.245
push “route 10.0.0.0 255.0.0.0″
keepalive 10 120
comp-lzo
persist-key
persist-tun
status /var/log/openvpn-status.log
verb 3

The only lines which you will need to change are ‘server-bridge’, which is simply the default gateway, subnet mask, and the start and end IP’s to assign the clients, and the push route, which pushes specific routes to all clients.

Now we need to create an ethernet bridge. First, we need to install bridge-utils:

apt-get install bridge-utils

Rather than explain how to set up a network bridge, I found a shell script which will do it for you. This can be found here. Just edit this with your network settings and execute it. You will also need to set it to create the bridge at boot time:

update-rc.d bridge defaults

Now you can start the openvpn server

/etc/init.d/openvpn start

Now we need to set up the windows client. First, download the OpenVPN client from here (at the time of writing, select 2.1 RC15). Install it, and create a file ‘client.conf’ in the config directory with the following parameters

client
dev tap
proto tcp
remote x.x.x.x 443 # (replace with your server IP)
resolv-retry infinite
nobind
pkcs12 client1.p12 # (replace with the client name)
ns-cert-type server
comp-lzo
verb 3

You can also add ‘redirect-gateway’ to the client configuration to pass all traffic down the VPN tunnel (rather than just traffic intended for the VPN itself).

Now copy the client1.p12 certificate file to the config directory on the client, start the gui, and connect. Everything should now work.

If you need to create any clients in the future, do the following:

cd /etc/openvpn/easy-rsa
source ./vars
./build-key-pkcs12 clientx

If one of your certificates is compromised, you can revoke it using the guide here.

This guide has been written from my notes and what I remember, so there may be a couple of things which aren’t 100% right. If anything goes wrong then post a comment or contact me and I’ll update the guide.

Bookmark It

This article follows on from my hamachi based VPN tutorial. PPTP is much easier to set up on a debian server, and very easy to connect to from a Windows XP machine.

With the ease of use though, there are some downfalls. PPTP is known to be less secure than other VPN’s such as OpenVPN and IPSEC, but for most uses it should be fine. One advantage over my hamachi VPN is that you will end up with an IP on the remote LAN (instead of the 5.0.0.0/8 address you have from hamachi), meaning you don’t need to do any routing tweaks.

First, set up the server:

apt-get install pptpd

That’s it! Now, to configure, you just need to edit 2 files:

/etc/pptpd.conf
Just add 2 lines to the bottom of the file, for the internal IP address of the server, and a range of IP’s which the remote connections will use.
localip 10.1.0.50
remoteip 10.1.0.90-99

The comments at the bottom of the file show some other ways of assigning ranges of IP’s in the remoteip section.

Finally, to add a user, edit /etc/ppp/chap-secrets and add a line for a remote user in the format:

username pptpd password *

To limit connections from specific hosts, use them instead of the * at the end.

To set up the connection on a windows XP client, do the following (from Control Panel):

Network Connections
Create a New Connection
Next
Connect to the network at my workplace
Virtual Private Network connection
Enter your connection name
Do not dial the initial connection
Enter the IP of your server

Then run the connection with the username and password you entered into /etc/ppp/chap-secrets

By default this will route all your traffic (including normal web browsing) through the tunnel. If you don’t want this, go to the connection properties, then the networking tab. Choose TCP/IP properties, and click ‘Advanced’, then finally untick the ‘Use default gateway on remote network’ tickbox.

That should be all you need.

Soon, I’ll give OpenVPN a try, and try to write up a nice guide here.

Most of the information above came from other sites, and by googling. If anyone knows of a better or more secure way of using PPTP then please post comments below.

Bookmark It