Archive

Archive for the ‘Crash Course’ Category

What is IOPS?

January 21st, 2010

IOPS stands for Input and Output Per Second. It is essentially a value that describes the raw capacity of a data storage system. It sets an expectation for performance. Some storage systems are said to be capable 120 IOPS and others 340 IOPS or much more. The number of IOPS, for most disk systems, represents a count of a mix of data reads (Output) and writes (Input) per second. By that number, one can judge how fast and responsive a disk system can be. The numbers can be obtained by performing sequential or random operations on the disk system. The best representation is random operations since it reflects a real-life usage pattern.

Disk system manufacturers obtain IOPS numbers via benchmarking. There are tools that measure disk throughput and capacity by performing several read and write tests. For example, Iometer is a tool that can measure disk system performance and produce the IOPS value for such system. One needs to be aware of the fact that some manufacturers provide the IOPS based on cached read and write tests. The latter is a best case scenario test and does not reflect real-life usage.

So what happens when a system reaches the IOPS threshold number? I did use the word “reach” because in theory the disk system will almost never exceed the advertised IOPS. Back to the question, when the system reaches the IOPS threshold, it starts queuing requests. A busy queue generally indicates a bottleneck.

How often have you found yourself with a bottlenecked 2TB disk choking after barely putting a few files? Disk size capacity has no bearing on its IOPS capacity. A 2TB disk could have the same IOPS capacity as a 250GB disk! So when does one need to upgrade the disk system or the configuration at least? We shall post a new entry on this blog with the instructions.

That’s all folks!

UNIXy Crash Course ,

Optimize a Large High Traffic Wordpress Blog

December 9th, 2009

In this post, we shall go through the steps required to optimize a Wordpress blog. If you are a current customer of UNIXy, do know that we are happy to implement these changes for your blog free of charge. We are a fully managed provider so do take advantage of it.

There are times when hardware upgrades are not feasible due to either budgetary limits or time constraints. We understand that a Wordpress blog could be hosted on a shared platform or dedicated server (VPS included) so we have written this post with that in mind. The first 5 steps can be completed in a shared environment where access to global configuration files is not permitted by the provider. All 10 steps can be completed on a dedicated server or Virtual Private Server (VPS).

  1. WP Supercache

WP Supercache is a Wordpress plugin that once installed alleviates some of the burdens on the server. Essentially, it takes every visited PHP page, generates the HTML content, and stores it on disk for subsequent visits. This is called PHP caching. The performance gain is noticeable. Not only does it speed up requests and improves responsiveness but it also leads to less resource utilization on the server. To install WP Supercache, follow this link:

http://ocaoimh.ie/wp-super-cache/

2. Turn Autosave Off

The WP global variable AUTOSAVE_INTERVAL controls the how often Wordpress auto saves posts as they are edited. For example, while editing or writing a new posts, WP could be automatically saving tens of copies of your posts to the main Wordpress table. Each time such automatic save process kicks off the main posts table is locked in order to be updated, which prevents visitors from briefly accessing your blog. It also causes the load average to spike up. One could either increase the autosave interval or completely disable the functionality. To increase the interval to X number of seconds (the higher the better), do it as such inside the wp-config.php file:

define('AUTOSAVE_INTERVAL', 2200); // 2200 second autosave interval

To disable the autosave feature completely, create a simple plugin file and enable via wp-admin:


<?php
/*
Plugin Name: Disable Autosave
*/
function disable_autosave() {
wp_deregister_script('autosave');
}
add_action( 'wp_print_scripts', 'disable_autosave' );

?>

3. Disable Revisioning

This feature saves several revisions of the same post into the database. It clutters the posts table and ends up bloating indexes unecessarily. It’s best to disable this feature altogether if you don’t need it. Add this line to your wp-config.php to disable this feature:

define('WP_POST_REVISIONS', false);

No further action is needed.

4. Disable Non-Cacheable SQL Plugins

While it is a good idea to disable as many unneeded plugins as possible, it’s not always practical to do. But this step requires that you go through the code of each plugin and locate SQL queries that use the MySQL RAND() function. Because MySQL does not cache queries that use the random number generator function RAND(), there is a performance penalty associated with such plugins. For example, Random Post plugin uses RAND() in its select SQL, which forces the SQL to run against the database time and time again. You can locate the PHP scripts that use the RAND() function inside your plugin folder using this command:

grep -i "rand()" -r wp-content/plugins

Once located, one could either disable such plugin or replace MySQL RAND() with a PHP rand function. This way the PHP random function will be cached. The randomization will, however, not work as intended. The good news is that perceived functionality will not be much impacted.

5. Convert MyISAM Tables to InnoDB

The InnoDB engine is superior. There is no reason to run MyISAM anymore. Here is a Bash script that will convert all tables, that can be converted, to InnoDB. If you do not have access to a script shell, you can accomplish the same using phpMyAdmin or other MySQL management tools. Be sure to take a backup of the Wordpress database before running this script. Also, be sure to define a maintenance window for this change as the alter locks up tables temporarily making the blog inaccessible:

#!/bin/bash
DBNAME="database";
DBUSER="user";
DBPASS="pass";

for t in $(mysql -u$DBUSER -p$DBPASS --batch --column-names=false -e "show tables" $DBNAME);
do
echo "Converting table $t";
mysql -u$DBUSER -p$DBPASS -e "alter table ${t} type=InnoDB" $DBNAME;
echo "Repair table $t";
mysql -u$DBUSER -p$DBPASS -e "repair table ${t}" $DBNAME;
done

Remember that those who are hosting their Wordpress blog on a shared provider's server will not be able to proceed with the rest of the instructions. The following instructions require root-level access to the server and a technical understanding of systems.

6. Increase Query Cache

We have already covered MySQL's query cache thoroughly in a previous post. We will not cover this information here. Feel free to read our past post on our blog here:

http://blog.unixy.net/2009/06/improve-performance-with-mysql-query-caching/

7. Increase Sort Buffer Size

The MySQL sort_buffer_size variable controls the amount of memory that MySQL can allocate to dedicated to sorting. In other words, the more SQL queries that need to sort data the larger this value needs to be. But it shouldn't be that large as it can affect performance negatively. For example, you wouldn't fill up a large container of water only to find out you can't carry it for a mile. Instead, it's best to fill the container with only enough water. The same idea applies to sort_buffer_size. It is usually best to set it to slightly than the amount of L2 CPU cache. So for a Xeon x3430 with 8MB cache that would be around 6MB. The final my.cnf entry would then be:

sort_buffer_size = 6M

8. Increase Key Buffer Size

This configuration settings controls the amount of memory that MySQL can use to fit index blocks into memory while manipulating indexes. Increasing this value as much as possible (within reason) can improve performance greatly. It prevents MySQL from having to use the disk as temporary storage for its work area. Edit the main my.cnf file and put the following entry.

key_buffer_size = 128M

Restart MySQL for changes to apply.

9. Install and Configure Eaccelerator

Eaccelerator is a PHP cache module that ensures minimal CPU processing. It essentially renders PHP scripts once and saves the HTML output for subsequent calls. The benefits are important. Checkout the Eaccelerator Wiki on how to compile, configure, and run it here:

http://eaccelerator.net/wiki/InstallFromSource

10. Install Apache MPM Worker

Apache's Worker can have advantages over its default MPM Prefork. It is somewhat lightweight in the sense that it's threading based and can thus smoothout load average spikes when traffic increases. Worker requires a rebuild of Apache. Luckily, someone has already covered the steps requires to rebuild Apache with this MPM. Check it out here:

http://www.faqs.org/docs/Linux-HOWTO/Apache-Compile-HOWTO.html

With this step we conclude our crash course / guide to optimizing and improving Wordpress. If you are a customer and would like to implement any of these recommendations, don't hesitate to ping us. I hope you enjoyed this post.

That's all folks!

UNIXy Crash Course

Running vBulletin Cluster Using Varnish

November 29th, 2009

Varnish is an excellent Web accelerator that can be made to proxy requests in and out of a cluster of somewhat more fully fledged Web servers like Apache or Litespeed. It has some great features like its compiled language, called VCL, and C-like programming API.

Large vBulletin deployments tend to be heavy on CPU and memory due to PHP script processing. For a large vBulletin forum, we recommend a cluster of 5 physical servers with three of those running Xen virtualization. One of those servers will be dedicated to the MySQL master database. Three to be setup as “headless” PHP nodes and Varnish load balancing and failover. And finally one as the NFS file store. The three headless servers need to run Varnish in their own VM and Litespeed or Apache in their own VM similarly.

The varnish backend director functionality makes it ideal to balance incoming traffic across all PHP headless nodes. It makes the configuration scalable and plug and play especially when needing to scale out within hours. The challenge in this setup is in making Varnish work correctly with vBulletin. Otherwise, session problems will occur.

We have a lot to share on this implementation so keep checking this blog as we will post it all. In the next installment, we’ll go through our deployment of a large vBulletin forum for a customer. In the mean time, feel free to get in touch should you have a question or comment.

That’s all folks!

UNIXy Crash Course , , , , , , , , , ,

Optimize Busy Mediawiki Portal Using Disk and Memory Caching

November 23rd, 2009

We have had great success with Mediawiki running behind Varnish, a proxy / web accelerator. The role of Varnish is to cache as many requests as possible. This helps alleviate resource consumption on the server such as CPU, memory, and disk, which in turn greatly improves performance and browser responsiveness. With Varnish in place, the load average went down from 30 to 4!

Configuring Varnish is a bit involved and we shall cover it on the next installment. In the mean time, here are some measures that can be put in place to gain a quick performance boost without much complexity.

  1. Enable Eaccelerator or Xcache OR
  2. Enable Mediawiki’s FileCache driver: $wgUseFileCache = true; AND
  3. Enable MySQL query caching by increasing query_cache_size in the configuration file
  4. Compile and enable Apache’s mod_mem_cache

Be sure to disable suPHP and set the correct permission on the cache directory so the Apache user is able to read/write to the cache. The cache directory is usually located directly under ${DocumentRoot}/.

As part of our fully managed service, we optimize and configure customers’ servers free of charge! Ask how we can improve your portal’s performance. That’s all folks!

UNIXy Crash Course

Dig initial domain registration or migration cookbook

July 25th, 2009

Dig, not to confuse with the social network Digg, is a tool that allows one to trouble shoot or view domain name DNS data. Dig is in the category of essential tools. Just like a voltage tester is an essential tool for an electrician, dig the most important tool for domain administrator. Dig tells the truth like nothing else.

Let us start with the basics first. A domain, to be accessible via a web browser, has to be associated with a name server that is tasked with responding to requests on its behalf and deliver the IP address associated with the domain in question. So every time someone decides to browse your Website, the browser has to go through a chain of requests and events to obtain the IP and then make a connection to the IP. The chain of events and requests are defined in protocols, standards, and best practices.

The good news is Dig can emulate the same browser actions and events without us having to understand the details behind them. One, however, needs to know how to interpret the results. The rest of this blog entry will be a list of essential Dig commands that will help trouble shoot a domain name. The following commands are done against the root domain name’s A record, although it could be against any valid DNS record like MX, SOA, TXT, etc.

First of all, if you are on a Windows computer, you will need to download Dig for Windows. Download it from here http://members.shaw.ca/nicholas.fong/dig/. You can install Dig on Linux directly from the repositories. Let’s get started!

Obtain the IP of a domain from your ISP’s cache

When running this command via your computer, you are essentially tapping into your ISP’s DNS server, which most often than not caches results to speed up lookups, improve performance, and save on bandwidth fees.

dig +nocmd vpslux.com A +short +norecurse

If the above command doesn’t return an IP address, chances are the domain’s IP is not in the cache. This could be because no one using your ISP has ever visited the domain or that the cache was flushed after the record expired.

Find out how much longer your ISP is going to cache your domain’s IP

dig +nocmd vpslux.com A +norecurse

;; ANSWER SECTION:
vpslux.com. 12884 IN A 74.52.123.139

That would be 12884 seconds until the ISP’s cache refreshes the data or removes the domain name from the cache altogether.

Obtain the IP of a domain bypassing the ISP’s cache

dig vpslux.com A +trace +short

That’s all folks. We hope this was useful.

UNIXy Crash Course , , ,