The Linux kernel is very smart and can leverage CPU features and instruction sets to bring down power consumption especially on a server. So one of the logical things to do when power savings are needed is to turn off CPU cores. To confirm that this is indeed what’s happening on your system, you could hot-plug these cores and bring them online in this fashion:

echo 1 > /sys/devices/system/cpu/cpu4/online
echo 1 > /sys/devices/system/cpu/cpu5/online

You can then verify that additional CPU cores have been brought up. You can also take them offline if needed:

echo 0 > /sys/devices/system/cpu/cpu4/online
echo 0 > /sys/devices/system/cpu/cpu5/online

The above will lead to this message being displayed in /var/log/messages (turned off core # 23):

[1132050.457980] CPU 23 is now offline
[1132065.819054] Booting Node 2 Processor 23 APIC 0x2b

That’s all folks!

We’ve compiled a list of six Unix/Linux tools that are cool and useful. From checking the weather via your terminal to having a sysadmin walk you through a difficult task. Interestingly, most of these tools were released decades ago! Read on…

kibitz

kibitz is a terminal tool that can be used by a mentor to assist his or her apprentice. It essentially makes what is normally one private terminal session visible to both parties providing bi-directional interactivity. The mentor can then walk the apprentice through various tasks and howtos. A typical session looks like the following to the mentor:

Whatever commands or text that the mentor or the apprentice types is seen on the other end. It looks like this to the apprentice:

tload

tload is a neat tool that displays the load average on the terminal in a form of character drawings. It gives an interesting visual of the system load (/proc/loadavg) over time – time being an adjustable parameter. Without further ado, here’s a screen shot of tload in action:

catchsegv

catchsegv helps one diagnose programs that segfault. If you’re dealing with an app that segfaults frequently, start it up with catchsegv and watch it produce a snapshot of CPU registers, memory maps, and backtrace at the time of the segfault! Here’s a snapshot of it in action:

dislocate

For those of you who are familiar with the program screen, dislocate is its grandfather.It’s neat tool that allows one to detach programs from the pseudo terminal and reattach to them whenever needed. This is tool for those with slow or unstable terminal sessions that are prone to disconnect.

tac

tac is a command line tool that reads a file in reverse order. It’s ideal for log sifting tasks where one’s interest is in grep’ing the most recent entries.

# cat /var/log/sip

Target 1… DONE
Target 2… DONE
Target 3… DONE
# tac /var/log/sip
Target 3… DONE
Target 2… DONE
Target 1… DONE

weather

weather is an expect-based program that runs from the terminal and grabs weather information from rainmaker.wunderground.com. Wunderground runs a public weather service on port 3000. weather leverages this service to pull weather data by city and displays it on the terminal. Here’s a sample run:

$ weather HOU

<snip>

Enter 3-letter city code: HOU

Weather Conditions at 12:53 PM CST on 24 Dec 2010 for Houston Intercontinental, TX.

Temp(F)    Humidity(%)    Wind(mph)    Pressure(in)    Weather

========================================================================

63          70%         ESE at 10       30.07      Overcast

Forecast for Houston, TX

406 am CST Fri Dec 24 2010

.Today…Mostly cloudy with a 30 percent chance of showers. Highs

in the upper 60s. Southeast winds 10 to 15 mph.

.Tonight…Showers likely and isolated thunderstorms. Lows in the

mid 40s. Northeast winds 5 to 10 mph in the evening becoming

north and increasing to 15 to 20 mph after midnight. Chance of

rain 70 percent in the evening decreasing to 60 percent after

<snip>

That’s all folks! I hope you enjoyed this entry.

In this article, we are going to cover the Linux utility strace. Before you read on, keep in mind that we (UNIXY) are a fully managed server provider and are always happy to assist our clients one way or the other. Should you happen to be a customer and need a hand with server management, please submit a request with us.

strace is a command line program that runs from one of the Linux shells. A shell is another program that allows a system administrator to interact with a Linux operating system – for example to run strace.

The strace utility is the crudest form of trouble shooting in Linux. It essentially hijacks the target program and traces all system function calls. Think of it as another program that attaches to the target program and snoops on its actions. The output of strace isn’t pretty either. One has to build a certain understanding and intuitiveness to make good use of it.

So how many times have you wondered what that Apache process is doing exactly pegging the CPU? Or how many times have you wondered why a certain PHP process dies unexpectedly? Well strace might help answer those questions. The strace output looks like this (don’t let this scare you. I’ll explain):


Process 6716 attached - interrupt to quit
restart_syscall(<... resuming interrupted call ...>) = 0
gettimeofday({1249790094, 207969}, NULL) = 0
open("/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor", O_RDONLY) = 18
read(18, "ondemand\n", 1024) = 9
read(18, "", 1024) = 0
close(18) = 0
open("/sys/devices/system/cpu/cpu0/cpufreq/scaling_cur_freq", O_RDONLY) = 18
read(18, "1200000\n", 1024) = 8
read(18, "", 1024) = 0
close(18) = 0
read(3, 0x80708a4, 4096) = -1 EAGAIN (Resource temporarily unavailable)
gettimeofday({1249790094, 209927}, NULL) = 0
poll([{fd=4, events=POLLIN}, {fd=3, events=POLLIN}, {fd=8, events=POLLIN|POLLPRI}, {fd=10, events=POLLIN|POLLPRI}, {fd=11, events=POLLIN|POLLPRI}, {fd=12, events=POLLIN|POLLPRI}, {fd=14, events=POLLIN|POLLPRI}, {fd=13, events=POLLIN|POLLPRI}, {fd=15, events=POLLIN|POLLPRI}, {fd=16, events=POLLIN|POLLPRI}], 10, 0) = 0
select(4, [3], [3], NULL, NULL) = 1 (out [3])
writev(3, [{"5\30\4\0\327W\300\2#\0\300\2S\0\27\0\233\4\5\0\330W\300"..., 520}], 1) = 520
read(3, 0x80708a4, 4096) = -1 EAGAIN (Resource temporarily unavailable)
read(3, 0x80708a4, 4096) = -1 EAGAIN (Resource temporarily unavailable)
gettimeofday({1249790094, 212466}, NULL) = 0
poll([{fd=4, events=POLLIN}, {fd=3, events=POLLIN}, {fd=8, events=POLLIN|POLLPRI}, {fd=10, events=POLLIN|POLLPRI}, {fd=11, events=POLLIN|POLLPRI}, {fd=12, events=POLLIN|POLLPRI}, {fd=14, events=POLLIN|POLLPRI}, {fd=13, events=POLLIN|POLLPRI}, {fd=15, events=POLLIN|POLLPRI}, {fd=16, events=POLLIN|POLLPRI}], 10, 996) = 0
gettimeofday({1249790095, 208261}, NULL) = 0
open("/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor", O_RDONLY) = 18

Each line in the above output represents an action made by the target program except the first two lines. The first one informs us that strace will begin tracing the program in question. The second one is made to recover from a half executed function system call.

The remaining lines are of importance and that’s what we need to study to figure what our target program is doing. Each line represents a function call the program is making and the result of the call. For example, this system call open

open("/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor", O_RDONLY) = 18

opens a file with certain attributes. The returned status is 18. What does the number 18 mean? What does O_RDONLY mean? RTFM! That is Read The Fine Manual! Linux comes with documentation tools that explain all these function calls and whatnot. In this example, open is the function. We can search for it and read its manual as such:

# apropos open
open (3posix) – open a file
# man 3 open

The manual page format can be cumbersome at first and it takes a bit of time to get used to it. Don’t let it discourage you! From reading manual we are able to understand more about the function call. Per the manual, return value 18 represents the “lowest numbered unused file descriptor.” We also understand what O_RDONLY means: Open file for reading only. Etcetera.

If you look back up at the strace output, you’ll note that shortly after the open() function call was made, close(18) followed it. We’ll need to go through the same process to read up on the system call close(). But the value 18 is deja vu! That’s the file descriptor number returned by open(). In other words, the program is opening a file /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor read-only then and then closing it.

We cannot go over each and every system call but you should be able to use the above process to understand pretty much any system call. But what would one be looking for when a program is not doing what it’s supposed to do and is failing? The Matrix sends agents after it to kill it! Just kidding! You have to go through each and every system call and check on the return status in the manual to understand if that is normal behavior or not. You can redirect the strace output to file so you can sift the (enormous) amount of data it produces:

strace -o /tmp/progra_strace.out -p

That’s all for today folks. I hope you enjoyed this one!

Many users tend to worry when the free memory on a Linux system is very low. Especially when the system is barely active. What many don’t know is that this is normal behavior for Linux. It simply takes your free memory and caches it. This caching behavior makes subsequent memory allocations much faster. So it improves performance overall.

However, if you really really need to flush the cache and buffers, you can run the following two commands on the shell as root:

sync; echo 3 > /proc/sys/vm/drop_caches

Here’s the output of free before and after running the above command.

Before

# free -m
             total       used       free     shared    buffers     cached
Mem:           502        487         15          0         15        170
-/+ buffers/cache:        301        201
Swap:         3153        375       2777

After

# free -m
             total       used       free     shared    buffers     cached
Mem:           502        345        157          0          1         44
-/+ buffers/cache:        300        202

As you can see the highlighted numbers, the buffers and cached values have dropped significantly after I ran the command. Again, this is not necessary. In fact, it’s not recommended at all but there it is if you’re curious to know.

That’s all folks. I hope you enjoyed this one.

We were able to install DirectAdmin (a Web hosting control panel) on Amazon’s Elastic Compute Cloud, branded as EC2. We would like to share the steps required to build a working instance of DirectAdmin on an Amazon server. First, I would like to personally thank Mark from DirectAdmin for being so accommodating and granting us a trial license. His help is very much appreciated! Keep in mind that UNIXY‘s truly fully managed server service goes above and beyond. We take business dreams and make them a reality.

Before we list the requirements, it is important to note that EC2 instances are Xen Virtual Machines. We encountered some issues running DA on the “Small” Amazon instances. The issues are related to 64-bit mode library mismatches. The only instances to be able to run DirectAdmin out of the box are of type “Large” and better. They are more expensive but ideal for this configuration.

Here are the requirements:

• Instance of type Large
• Linux CentOS 5 image
• One elastic (which is another word for static) IP address

Go ahead and bring up the node as you would normally do. Be sure to remove all extra software that comes pre-installed with the CentOS 5 image. Otherwise, it will break DirectAdmin. If in doubt, follow the instructions on the DirectAdmin Install page:

http://www.directadmin.com/install.html

Once logged in as root, go ahead and download the DirectAdmin install tarball:

# wget http://www.directadmin.com/setup.sh

By default, the licensing scheme of DirectAdmin makes it that setup.sh binds to the licensed IP address. Here’s how the license verification command looks like:

$BIN_DIR/wget $WGET_OPTION -O $DA_PATH/update.tar.gz –bind-address=$IP https://www.directadmin.com/cgi-bin/daupdate?uid=$CID\&lid=$LID

But as you know, EC2 instances are 1:1 private:public NAT. Which means that the license check step will fail and the DA install won’t start. To avoid this chicken-and-egg problem, we are going to tell setup.sh to not bind to any IP address but to simply connect to the DA licensing server using the elastic external IP address, which it will do by default. Go ahead and run this command against the setup.sh file:

# sed -i 's/--bind-address=\$IP//;' setup.sh

Before we run setup.sh, we need to trick the setup script to think that the external IP address is “attached” to the server. Here’s how:

# ifconfig eth0:0 inet 1.1.1.1 netmask 255.255.255.255 up

Be sure to replace the IP address 1.1.1.1 with your elastic IP. Then simply run setup.sh:

# cd /usr/local/directadmin/scripts; while [ true ]; do sed -i 's/--bind-address=\$IP//; s/--bind-address=\${3}//;' *.sh > /dev/null > 2&>1; sleep 10; done & sh setup.sh 11111 22222 vpslux.com eth0:0 1.1.1.1

Woaah! Wait a minute! What is that!? OK, let’s break it down. The DA scripts directory gets unpacked into the server after you run setup.sh. It dumps a list of scripts that are used by DA to setup accounts, domains, etc. Some scripts use the same –bind-address flag. Having that flag in those scripts breaks DA. the one-line script “while [ true ]; do sed -i ‘s/–bind-address=\$IP//; s/–bind-address=\${3}//;’ *.sh; sleep 10; done” simply goes to that directory and removes the flag.

The trick being DA attempts to run some of those scripts while setup.sh is running. So it’s important to have the sed script run in parallel during the installation. Once the setup.sh is done running, we’ll kill the job (or reboot per the final step). The arguments supplied to setup.sh are in this format

# setup.sh <ClientID> <LicenseID> <Hostname> <Interface> <ExternalIP>

Be sure to replace those values accordingly. Once the install finishes, simply reboot the VM:

# reboot

That’s all folks. Enjoy!

UNIXy has experience with migrating server applications in and out for customers. This is one of the value-add services we provide to our customers as they switch over to using our services. We are happy to share our ways of performing these tasks. Migrations are done differently in the industry and this is our proven and preferred method. We would like to hear from you about your ways so don’t hesitate to share them with us.

This is part one of a series of guides on how to perform a proper migration of a Web application. Proper because the goal is to seamlessly and preparedly switch over to a new server or provider. We’ll set the bar high enough to only allow room for a 5-10 minutes of cumulative downtime. Note that our experience is limited to Web applications residing on UNIX and/or Linux servers. Also, we are not covering the migration of Web applications that are part of a control panel such as cPanel, Plesk, or DirectAdmin. Control panels have their own migration tools. This guide, however, can assist in migrating a cPanel account over to a non-cPanel server and vice-versa.

While this proposed migration method might seem like a one-time process, it’s imperative to perform dry runs prior to the actual cut-over. Dry runs do not involve downtime at all and can thus help gauge the terrain for potential issues that may arise on go-live day. We call them exercises!

Web Application Migration Part I – Email suction

Email is one the most dreaded services to migrate because of the complexity involved. Not only is it our goal to have the new server send and receive email but also mirror the users’ email inboxes and folders as they were prior to the cutoff. Post-migration, users should wake up to their good old familiar inbox in the undergarment of the new server.

There are tools that one needs to equip herself to carry out an email migration; also commonly called suction in the industry. Our preferred tools are Imapcopy, the venerable Imapsync, and a multi-purpose Linux workstation with the GNU tools; but most importantly, a well seasoned systems administrator. As the saying goes: A cowl does not make a monk.

First things first: gather as much information on the current server as possible. Does the server allow direct IMAP access to users? Is it Webmail only? What about POP access? What is the mailbox / user count on the server? Is there a quota for disk space usage? What is the largest mailbox? Suction and IMAP go hand in hand. It’s only through the IMAP protocol that one can fully interact with a mailbox and its folders. This writeup will not over the nuts and bolts of IMAP or other protocols so be sure to read up on those before continuing.

Imapcopy allows one to copy a mailbox from one server to another using the IMAP protocol. This tool is useful as part of our drill exercises. Before we demo these tools, ensure that an IMAP daemon is listening on both the source and destination.Also, make sure that ports 143 on both the source and destination are open (telnet server 143). Then, download Imapcopy on your workstation. It’s also time to pick an existing mailbox (preferably one with multiple folders) for our trials.

The Imapcopy utility comes packaged with a sample configuration file. For the purpose of this demo, the changes are good enough for a successful run. We’re using our Linux workstation as the actual destination server. Here’s a diff of the changes we made (the symbols < and > and / or spaces were added to avoid getting spam on these inboxes once this page gets crawled):

— ImapCopy.cfg.orig 2008-07-12 23:39:12.000000000 -0500
+++ ImapCopy.cfg 2008-07-13 09:50:52.000000000 -0500
@@ -1,7 +1,7 @@
##############
# Sourceserver
##############
-SourceServer localhost
+SourceServer unixy.net
SourcePort 143

###################
@@ -51,6 +51,6 @@
# List of users and passwords
#############################
# SourceUser SourcePassword DestinationUser DestinationPassword
-Copy “foo” “foosrcpw” “foo” “foodestpw”
-Copy “bar” “barsrcpw” “bar” “test”
+Copy “null <@>unixy.net” “XXXXX” “null” “XXXXX”
+# Copy “bar” “barsrcpw” “bar” “test”

From the above sample configuration file, we are instructing Imapcopy to suction mailbox null <@>unixy.net. But before running the suction, we need to test connectivity and authentication between the “SourceServer” and “DestinationServer”. The flag “-i” of imapcopy achieves this connectivity test:

$ imapcopy -i
IMAPCopy 1.03 – 2005/04/20 [compiled with FreePascal] (c) 2001-2006 Armin Diehl <armin@freepascal.org>
Running on Linux

—————————————————————————
Login on sourceserver as null <@>unixy.net OK
Login on destinationserver as null OK
Sourceserver:
=============
Server-Info : [CAPABILITY IMAP4rev1 UIDPLUS
CHILDREN NAMESPACE THREAD=ORDEREDSUBJECT
THREAD=REFERENCES SORT QUOTA IDLE ACL ACL2=UNION
STARTTLS] Courier-IMAP ready. Copyright 1998-2005 Double
Precision, Inc. See COPYING for distribution information.
Capabilities : IMAP4REV1 UIDPLUS CHILDREN
NAMESPACE THREAD=ORDEREDSUBJECT THREAD=REFERENCES SORT QUOTA IDLE ACL ACL2=UNION
Personal Namespace : INBOX
Folder sperator : .
other Users Namespace:
Public Namespace : #shared
Folders to copy : ALL
Skip this folders : NONE

Destinationserver:
==================
Server-Info : [CAPABILITY IMAP4rev1 UIDPLUS CHILDREN NAMESPACE THREAD=ORDEREDSUBJECT
THREAD=REFERENCES SORT QUOTA IDLE ACL ACL2=UNION]
Courier-IMAP ready. Copyright 1998-2005 Double Precision,
Inc. See COPYING for distribution information.
Capabilities : IMAP4REV1 UIDPLUS CHILDREN
NAMESPACE THREAD=ORDEREDSUBJECT THREAD=REFERENCES SORT
QUOTA IDLE ACL ACL2=UNION
Personal Namespace : INBOX
Folder sperator : .
other Users Namespace:
Public Namespace : #shared

So far so good. Now let’s have it copy the null <@>unixy. net mailbox in its entirety. But before we do that, Imapcopy requires the default layout of a Maildir exists as such:

$ mkdir -p /home/null/Maildir/{cur,tmp,new}

And now we launch Imapcopy for the full copying of the mailbox:

$ imapcopy
IMAPCopy 1.03 – 2005/04/20 [compiled with FreePascal] (c) 2001-2006 Armin Diehl <armin@freepascal.org>
Running on Linux

Login on sourceserver as null <@>unixy.net OK
Login on destinationserver as null OK
Getting folderlist on sourceserver OK, found 1 folder
Getting List of messages in “INBOX” OK, 2 Messages found
Processing Folder INBOX
S:A0004 SELECT INBOX\r\n
R:* FLAGS (\Draft \Answered \Flagged \Deleted \Seen \Recent)\r\n
R:* OK [PERMANENTFLAGS (\* \Draft \Answered \Flagged \Deleted \Seen)] Limited\r\n
R:* 0 EXISTS\r\n
R:* 0 RECENT\r\n
R:* OK [UIDVALIDITY 1215995880] Ok\r\n
R:* OK [MYRIGHTS "acdilrsw"] ACL\r\n
R:A0004 OK [READ-WRITE] Ok\r\n
S:A0005 APPEND INBOX (\Seen) “14-Jul-2008 21:46:39 -0500″ {903}\r\n
R:+ OK\r\n
C: Command exit because of + (“+ OK”)
S: Message not shown (903 Bytes)
R:A0005 OK [APPENDUID 1215995880 9] APPEND Ok.\r\n
S:A0006 APPEND INBOX (\Seen) “14-Jul-2008 21:47:25 -0500″ {716}\r\n
R:+ OK\r\n
C: Command exit because of + (“+ OK”)
S: Message not shown (716 Bytes)
R:A0006 OK [APPENDUID 1215995880 10] APPEND Ok.\r\n
2 Messages copied, 0 Errors
S:A0007 LOGOUT\r\n
R:* BYE Courier-IMAP server shutting down\r\n
R:A0007 OK LOGOUT completed\r\n
S:A0008 LOGOUT\r\n
C: Command status 1

1 User processed, 2 Messages copied, 0 Error(s)
0 Folder(s) created, 0 Folder create errors, 0 Folder not copied

All good! As you can see from the above log, Imapcopy suctioned 2 messages for one user, null <@>unixy.net in this case. One extra step:

$ ls -al /home/null/Maildir/cur/
total 16
drwxr-xr-x 2 null null 4096 2008-07-14 21:48 .
drwxrwxrwx 6 null null 4096 2008-07-13 19:38 ..
-rw-r–r– 1 null null 695 2008-07-14 21:47 1216090136.M168971P8436V0000000000000305I00175CCF_1.MUX,S=695:2,S
-rw-r–r– 1 null null 874 2008-07-14 21:46 1216090136.M30165P8436V0000000000000305I00175CCD_0.MUX,S=874:2,S

Impeccable! While Imapcopy could potentially be used to carry out the migration, it is not the prefered method of suction. If it were to be used as part of a migration strategy, one would have to batch all the user mailboxes in one fell swoop. It makes the process time consuming and causes downtime especially when you have to transfer very large mailboxes. Imapcopy’s copying mechanism is also not incremental or retriable and that is a valid concern. Every time Imapcopy runs, it copies the mailbox in its entirety regardless. Here’s an example of a re-run. For brevity’s sake, we’re only showcasing the statistics of the run:

$ imapcopy
Login on sourceserver as null@unixy.net OK
Login on destinationserver as null OK
.
.
.
1 User processed, <b>2 Messages copied</b>, 0 Error(s)
0 Folder(s) created, 0 Folder create errors, 0 Folder not copied

Indeed, Imapcopy has no notion of incremental copying. You could see how it can get out of control with a large mailbox.

The goal here is to suction email messages per-user as they login into the new server for the first time. This is where Imapsync comes in. You supply the source and destination server, the mailbox to sync, and the password of the user’s mailbox for both the source and destination; imapsync does the rest.

First, let’s sync our IMAP mail box manually to test the waters:

$ imapsync –host1 unixy.net –user1 null <@>unixy.net –password1 XXXXX –host2 localhost –user2 null –password2 XXXXX
$RCSfile: imapsync,v $ $Revision: 1.219 $ $Date: 2007/04/04 09:32:20 $
Here is a linux system Linux MUX 2.6.20-16-server #2 SMP Tue Dec 18 05:52:19 UTC 2007 i686)
with perl 5.8.8
Mail::IMAPClient version used here is 2.2.9
Command line used :
/usr/bin/imapsync –host1 unixy.net –user1 null <@>unixy.net –password1 XXXXX –host2 localhost –user2 null –password2 XXXXX
will try to use CRAM-MD5 authentication on host1
will try to use CRAM-MD5 authentication on host2
From imap server [unixy.net] port [143] user [null <@>unixy.net]
To imap server [localhost] port [143] user [null]
Banner : * OK [CAPABILITY IMAP4rev1 UIDPLUS CHILDREN
NAMESPACE THREAD=ORDEREDSUBJECT THREAD=REFERENCES SORT
QUOTA IDLE ACL ACL2=UNION STARTTLS] Courier-IMAP ready.
Copyright 1998-2005 Double Precision, Inc. See COPYING
for distribution information.
Host unixy.net says it has NO CAPABILITY for AUTHENTICATE CRAM-MD5
Error login : [unixy.net] with user [null <@>unixy.net] auth [CRAM-MD5]: 3 NO Login failed.

Trying LOGIN Auth mechanism on [unixy.net] with user [null <@>unixy.net]
Success login on [unixy.net] with user [null <@>unixy.net] auth [CRAM-MD5]
Banner : * OK [CAPABILITY IMAP4rev1 UIDPLUS CHILDREN
NAMESPACE THREAD=ORDEREDSUBJECT THREAD=REFERENCES SORT
QUOTA IDLE ACL ACL2=UNION STARTTLS] Courier-IMAP ready.
Copyright 1998-2005 Double Precision, Inc. See COPYING
for distribution information.
Host localhost says it has NO CAPABILITY for AUTHENTICATE CRAM-MD5
Error login : [localhost] with user [null] auth [CRAM-MD5]: 3 NO Login failed.

Trying LOGIN Auth mechanism on [localhost] with user [null]
Success login on [localhost] with user [null] auth [CRAM-MD5]
From capability : QUOTA STARTTLS NAMESPACE IDLE
THREAD=ORDEREDSUBJECT ACL SORT UIDPLUS CHILDREN
ACL2=UNION IMAP4REV1 THREAD=REFERENCES
To capability : QUOTA STARTTLS NAMESPACE IDLE
THREAD=ORDEREDSUBJECT ACL SORT UIDPLUS CHILDREN
ACL2=UNION IMAP4REV1 THREAD=REFERENCES
From state Authenticated
To state Authenticated
From separator and prefix : [.][INBOX.]
To separator and prefix : [.][INBOX.]
++++ Calculating sizes ++++
From Folder [INBOX] Size: 1623 Messages: 2
Total size: 1623
Total messages: 2
Time : 12 s
++++ Calculating sizes ++++
To Folder [INBOX] Size: 1619 Messages: 2
Total size: 1619
Total messages: 2
Time : 0 s
++++ Listing folders ++++
From folders list : [INBOX]
To folders list : [INBOX]
++++ Looping on each folder ++++
From Folder [INBOX]
To Folder [INBOX]
++++ From [INBOX] Parse 1 ++++
++++ To [INBOX] Parse 1 ++++
++++ Verifying [INBOX] -> [INBOX] ++++
+ NO msg #5 [imWT88lfpdVUriyDdDcAOA:905] in INBOX
+ Copying msg #5:905 to folder INBOX
flags from : [\Seen][]
Copied msg id [5] to folder INBOX msg id [11]
+ NO msg #6 [lOzMkdoLOuFH7qheMK4NkQ:718] in INBOX
+ Copying msg #6:718 to folder INBOX
flags from : [\Seen][]
Copied msg id [6] to folder INBOX msg id [12]
Time : 1 s
++++ Statistics ++++
Time : 13 sec
Messages transferred : 2
Messages skipped : 0
Total bytes transferred: 1623
Total bytes skipped : 0
Total bytes error : 0
Detected 0 errors

At this point in the game, there’s no difference between imapcopy and imapsync. They both appear to fulfill the same role; copy the messages from one inbox to another. However, imapsync is much more capable than it appears on a first glimpse. As we’ll see in this next run, imapsync has more capabilities packed in.

++++ Statistics ++++
Time : 12 sec
<b>Messages transferred : 0 </b>
Messages skipped : 2
Total bytes transferred: 0
Total bytes skipped : 1623
Total bytes error : 0

Detected 0 errors

In the above snippet, we only display the compte rendu of the second run. As you can see, no messages were transfered over to the destination inbox. Impasync is an incremental tool. Big deal!

At this point, all that is needed is a source and destination host, a list of users and respective passwords. But our goal is not to batch the whole operation. The reason being we want to avoid long running jobs with higher latency that could affect end-user experience. Therefore we have to come up with a per-user at-logon solution.

It is common during migrations to ask users to visit a “migration” link in order to get the conversion or mailbox sync started. This is an OK approach and it works fine with Imapsync. The webmail page in question launches Imapsync on the server-side code of the page; quite simple. There’s an alternative, which is to hijack the IMAP software running on the legacy server. The latter method involves as creating a wrapper around the binary so that not only authentication is performed (as usual) but also an Imapsync session is started in the foreground or background. We won’t go over this here for now as we’ll stick with the migration link approach.

In the next installment, we’ll go over Web migration and then follow it up with database migration.