Optimize a Large High Traffic Wordpress Blog

December 9th, 2009

In this post, we shall go through the steps required to optimize a Wordpress blog. If you are a current customer of UNIXy, do know that we are happy to implement these changes for your blog free of charge. We are a fully managed provider so do take advantage of it.

There are times when hardware upgrades are not feasible due to either budgetary limits or time constraints. We understand that a Wordpress blog could be hosted on a shared platform or dedicated server (VPS included) so we have written this post with that in mind. The first 5 steps can be completed in a shared environment where access to global configuration files is not permitted by the provider. All 10 steps can be completed on a dedicated server or Virtual Private Server (VPS).

  1. WP Supercache

WP Supercache is a Wordpress plugin that once installed alleviates some of the burdens on the server. Essentially, it takes every visited PHP page, generates the HTML content, and stores it on disk for subsequent visits. This is called PHP caching. The performance gain is noticeable. Not only does it speed up requests and improves responsiveness but it also leads to less resource utilization on the server. To install WP Supercache, follow this link:

http://ocaoimh.ie/wp-super-cache/

2. Turn Autosave Off

The WP global variable AUTOSAVE_INTERVAL controls the how often Wordpress auto saves posts as they are edited. For example, while editing or writing a new posts, WP could be automatically saving tens of copies of your posts to the main Wordpress table. Each time such automatic save process kicks off the main posts table is locked in order to be updated, which prevents visitors from briefly accessing your blog. It also causes the load average to spike up. One could either increase the autosave interval or completely disable the functionality. To increase the interval to X number of seconds (the higher the better), do it as such inside the wp-config.php file:

define('AUTOSAVE_INTERVAL', 2200); // 2200 second autosave interval

To disable the autosave feature completely, create a simple plugin file and enable via wp-admin:


<?php
/*
Plugin Name: Disable Autosave
*/
function disable_autosave() {
wp_deregister_script('autosave');
}
add_action( 'wp_print_scripts', 'disable_autosave' );

?>

3. Disable Revisioning

This feature saves several revisions of the same post into the database. It clutters the posts table and ends up bloating indexes unecessarily. It’s best to disable this feature altogether if you don’t need it. Add this line to your wp-config.php to disable this feature:

define('WP_POST_REVISIONS', false);

No further action is needed.

4. Disable Non-Cacheable SQL Plugins

While it is a good idea to disable as many unneeded plugins as possible, it’s not always practical to do. But this step requires that you go through the code of each plugin and locate SQL queries that use the MySQL RAND() function. Because MySQL does not cache queries that use the random number generator function RAND(), there is a performance penalty associated with such plugins. For example, Random Post plugin uses RAND() in its select SQL, which forces the SQL to run against the database time and time again. You can locate the PHP scripts that use the RAND() function inside your plugin folder using this command:

grep -i "rand()" -r wp-content/plugins

Once located, one could either disable such plugin or replace MySQL RAND() with a PHP rand function. This way the PHP random function will be cached. The randomization will, however, not work as intended. The good news is that perceived functionality will not be much impacted.

5. Convert MyISAM Tables to InnoDB

The InnoDB engine is superior. There is no reason to run MyISAM anymore. Here is a Bash script that will convert all tables, that can be converted, to InnoDB. If you do not have access to a script shell, you can accomplish the same using phpMyAdmin or other MySQL management tools. Be sure to take a backup of the Wordpress database before running this script. Also, be sure to define a maintenance window for this change as the alter locks up tables temporarily making the blog inaccessible:

#!/bin/bash
DBNAME="database";
DBUSER="user";
DBPASS="pass";

for t in $(mysql -u$DBUSER -p$DBPASS --batch --column-names=false -e "show tables" $DBNAME);
do
echo "Converting table $t";
mysql -u$DBUSER -p$DBPASS -e "alter table ${t} type=InnoDB" $DBNAME;
echo "Repair table $t";
mysql -u$DBUSER -p$DBPASS -e "repair table ${t}" $DBNAME;
done

Remember that those who are hosting their Wordpress blog on a shared provider's server will not be able to proceed with the rest of the instructions. The following instructions require root-level access to the server and a technical understanding of systems.

6. Increase Query Cache

We have already covered MySQL's query cache thoroughly in a previous post. We will not cover this information here. Feel free to read our past post on our blog here:

http://blog.unixy.net/2009/06/improve-performance-with-mysql-query-caching/

7. Increase Sort Buffer Size

The MySQL sort_buffer_size variable controls the amount of memory that MySQL can allocate to dedicated to sorting. In other words, the more SQL queries that need to sort data the larger this value needs to be. But it shouldn't be that large as it can affect performance negatively. For example, you wouldn't fill up a large container of water only to find out you can't carry it for a mile. Instead, it's best to fill the container with only enough water. The same idea applies to sort_buffer_size. It is usually best to set it to slightly than the amount of L2 CPU cache. So for a Xeon x3430 with 8MB cache that would be around 6MB. The final my.cnf entry would then be:

sort_buffer_size = 6M

8. Increase Key Buffer Size

This configuration settings controls the amount of memory that MySQL can use to fit index blocks into memory while manipulating indexes. Increasing this value as much as possible (within reason) can improve performance greatly. It prevents MySQL from having to use the disk as temporary storage for its work area. Edit the main my.cnf file and put the following entry.

key_buffer_size = 128M

Restart MySQL for changes to apply.

9. Install and Configure Eaccelerator

Eaccelerator is a PHP cache module that ensures minimal CPU processing. It essentially renders PHP scripts once and saves the HTML output for subsequent calls. The benefits are important. Checkout the Eaccelerator Wiki on how to compile, configure, and run it here:

http://eaccelerator.net/wiki/InstallFromSource

10. Install Apache MPM Worker

Apache's Worker can have advantages over its default MPM Prefork. It is somewhat lightweight in the sense that it's threading based and can thus smoothout load average spikes when traffic increases. Worker requires a rebuild of Apache. Luckily, someone has already covered the steps requires to rebuild Apache with this MPM. Check it out here:

http://www.faqs.org/docs/Linux-HOWTO/Apache-Compile-HOWTO.html

With this step we conclude our crash course / guide to optimizing and improving Wordpress. If you are a customer and would like to implement any of these recommendations, don't hesitate to ping us. I hope you enjoyed this post.

That's all folks!

UNIXy Crash Course

Houston, we have snow!

December 4th, 2009

Houston does it for the first time on record. It is snowing like there’s no tomorrow!

Cheers!

Joe Interesting

VPS Failover Challenge w/ cPanel

December 2nd, 2009

We are well ahead of target to achieve a 99.99% uptime using our geographical failover solution. We have put our failover solution live online for the whole world to see. Do pay a visit to the website when you get a chance! It is located here:

http://www.cpanelfailover.net

Cheers!

UNIXy Challenge , , , , ,

Running vBulletin Cluster Using Varnish

November 29th, 2009

Varnish is an excellent Web accelerator that can be made to proxy requests in and out of a cluster of somewhat more fully fledged Web servers like Apache or Litespeed. It has some great features like its compiled language, called VCL, and C-like programming API.

Large vBulletin deployments tend to be heavy on CPU and memory due to PHP script processing. For a large vBulletin forum, we recommend a cluster of 5 physical servers with three of those running Xen virtualization. One of those servers will be dedicated to the MySQL master database. Three to be setup as “headless” PHP nodes and Varnish load balancing and failover. And finally one as the NFS file store. The three headless servers need to run Varnish in their own VM and Litespeed or Apache in their own VM similarly.

The varnish backend director functionality makes it ideal to balance incoming traffic across all PHP headless nodes. It makes the configuration scalable and plug and play especially when needing to scale out within hours. The challenge in this setup is in making Varnish work correctly with vBulletin. Otherwise, session problems will occur.

We have a lot to share on this implementation so keep checking this blog as we will post it all. In the next installment, we’ll go through our deployment of a large vBulletin forum for a customer. In the mean time, feel free to get in touch should you have a question or comment.

That’s all folks!

UNIXy Crash Course , , , , , , , , , ,

NUMA Impact On Web Hosting and the Cloud

November 27th, 2009

My interest in NUMA (Non-Uniform Memory Access) has picked up quite a bit as of late. This technology has a huge potential for application in large scale Web hosting and Internet solutions. In brief, NUMA is a technology that was conceived to get around bus contention in general. What this means is that, with NUMA, programs are no longer limited by memory size or disk IOPS on the same host node and can, therefore, harness the power of other nearby servers as if they were part of the same chassis. In fact, IO scales up as a result!

The problem with uniform memory access architectures is that in a single chassis multi-processor system with a large amount of memory the bus tends to saturate very quickly. Processor cores addressing the memory bus are thus contending for the same resource, which becomes a hot spot. Take a 32-core server with 128GB of DDR3 memory. This server will under perform should it run a parallel memory hungry application. The issue being that each task on a core will at some point need to address memory via the bus (DDR3 is a three-frequency or three-channel bus). So out of 32 tasks that are all attempting to access the bus, there will be (32 -3 = 29/3 ~ 9) 9 tasks that contend for the same channel. The end result is a saturated bus and poor performance.

NUMA solves this bus contention problem by creating a virtual network of processor nodes and giving each node a certain weight based on its locality. The most interesting aspect is its true scalability potential. Most importantly, NUMA is the only technology that can scale up. Amazon’s EC2 and most other technologies in the market nowadays can only scale out. In other words, one has to implement load balancing, replication, and data synchronization.

Here’s where it gets interesting. IBM has already comoditised NUMA with its x86 offering: the IBM x440. Although one can only chain two x440 chassis together to form one NUMA network, what was once a concept is now practical! The x440’s can be chained together via their SMP expansion module using remote IO cables. Now imagine having the ability to chain 5, 10, 100 16GB dual quad core boxes! Not only does one take advantage of of a few terrabytes of memory but also scale core compute and IO.

A challenge remains. The remote IO cables that allow formation of a NUMA node cluster is based on proprietary technology. The research and development effort required to deliver such technology is colossal for a smaller firm. Or maybe not!

Joe Future , , , , , ,