Standby FreePBX server

Something a lot of people want to do is have a standby FreePBX server, as a backup should the primary one fail for any reason (network problem, hardware failure …). It’s actually quite tricky to implement an automated failover HA system that mirrors the realtime configuration of Asterisk/FreePBX, but what is possible using the backup module in FreePBX 2.8+ is to create a standby system that gets updated at regular intervals.

Firstly we need 2 FreePBX systems. Both should have the same version of FreePBX installed (at least v2.8) with the same modules installed. You may not need the exact same module versions on both systems, but I would recommend this. Also, going forward, you would need to remember to update the standby system when applying updates to the primary.

Below I’m going to do a walkthrough of creating a standby server for my demo system. Both systems are running the same version of FreePBX and have the same modules installed and enabled –

livedemo_system_status      

The backup job we are going to create will run on the standby server. This will connect to the primary server, backup the system, copy the backup to the standby server and restore the settings. To allow the standby server to connect to the primary we need to generate an SSH key.

To do this log on to the standby server as root and run the following commands. This assumes we are running Asterisk/Apache on both servers under the username ‘astersk’ –

su asterisk -c "ssh-keygen -t rsa"

Press enter to save the key in the default location and not to set a password.

Now we need to copy the public key to the primary server.  The first command ensures the folder permissions are correct and the second copies the file over. Obviously replace ‘livedemo.sysadminman.net’ with your primary server hostname or IP address. File locations may be different depending on your set up –

ssh [email protected] -C "chmod 700 /var/lib/asterisk"

su - asterisk -c "ssh-copy-id -i /var/lib/asterisk/.ssh/id_rsa.pub [email protected]"

You can now test that the keys are set up correctly by running –

ssh -i /var/lib/asterisk/.ssh/id_rsa [email protected]

this should give you a command prompt on the primary server, without asking for a password. If this is not the case you need to resolve this before continuing.

Now we have key based login working we can set up the backup job. To do this go to Tools, Backup (you may need to install the backup module in FreePBX) and give your backup job a name and select all of the options –

livedemo_backup_job

Now scroll down and set up the remote backup options. You need the primary server hostname, the asterisk username and the location of the key we created earlier. I’m also going to set this job to run ‘now’ so that I can test it –

remote_backup_options

This may take a little while to run but when it’s finished you should find that all of your settings are copied from the primary server to the standby server. You can now change the schedule to run automatically. It creates a reasonable load on the server when doing the backup so I would recommend doing it no more than once a day.

No settings will be applied to the standby server until you make a change and click the orange “Apply Changes” button in FreePBX.

There are quite a few things to consider when creating a standby. Here are some that I can think of –

DNS – changes to DNS records do not happen instantly, and can take around 24 hours to take effect. You must bear this in mind if you are using DNS names to connect your extensions to your VOIP server. Changing the A record may not automatically make them connect to the new server. Also if your inbound numbers forward to a DNS name these may not connect to the new server after a DNS change. You could look at a low TTL DNS setting (so that names don’t cache as long) but it still might not be a smooth changeover.

EXTENSIONS – you will have to change the IP address that your extensions register to. As mentioned above using DNS might not be straight forward.

INBOUND NUMBERS – you will probably need to change the way your inbound numbers are forwarded to your Asterisk server. If you forward the calls to a SIP URI you will need to change this with your provider so that calls go to the standby server. If calls are forwarded to the registered server this may work OK (but ensure you don’t end up with the primary and standby both trying to register with your provider)

EXTENSION PROVISIONING – if you use TFTP to provision your extensions then don’t forget to add the tftp folder to the backup schedule. You will also have to change the provisioning server for your extensions (maybe via DHCP?)

TESTING – as with any backup plan it’s always a good idea to test it. If you decide to run off the standby server for a while then at the end of the testing ensure you delete any trunks that have been configured (and Applied). You don’t want 2 servers trying to register with your call provider.

CDR and VOICEMAIL – obviously with a scheduled update, rather than realtime mirroring, the elements that are most likely to be out of date on the standby server are CDRs and Voicemail. Hopefully this is a trade-off worth making though to have a server back up and running.

 

Most of the problems above would be true even with a realtime HA system. It would be possible to mitigate some of them by having the IP addresses assigned to the primary server switch over to the standby on a failure. However, this relies on the primary and standby server being on the same network and you are then not covered for network failures  (probably one of the most common causes of downtime). So the pain involved with the IP switch does provide more resilience.

SysAdminMan has VPS servers on 2 totally separate networks in the UK so if you are looking to set up a standby server for you primary server please make this clear when ordering the standby server, and it will be created on a different network to your primary server.

If you have a SysAdminMan VPS and would like to discuss setting up a standby then please get in touch.