Download article as PDF

There are times when one needs to automate some of the mundate tasks associated with browsing a certain Web page or accomplishing a certain interactive task online. If the task is repetitive, then there has to be a way to automate it using a computer! Browser tasks are no exception. In this brief guide I’m going to show you how to automate some browsing actions on a website called http://www.vpslux.com. More specifically, I’m going to show you how to automate browsing a page called “explained.” Keep in mind that the action could be anything from authenticating to uploading a file!

Lynx is a text browser that runs from the command line, which is also known as “shell.” It runs on most operating systems notably Linux. It comes with a valuable feature that will help us automate actions that are normally performed interactively via a browser (Ex: Firefox, Safari, or Internet Explorer). The two most important features in our case are command logging and command scripting. Per the manual of Lynx:

-cmd_log=FILENAME
write keystroke commands and related information to the specified file.

-cmd_script=FILENAME
read keystroke commands from the specified file.  You can use the data written using the  -cmd_log  option.

Let’s record our actions by browsing http://www.vpslux.com using Lynx. From the command line or shell, launch Lynx with the following arguments and flags:

lynx -cmd_log=/tmp/www.vpslux.com http://www.vpslux.com

The above command will not only take me to the Website in question but also record every keystroke and action I perform and save it inside the file /tmp/www.vpslux.com. It is so simple and so powerful at the same time. Once I’m done with navigating to page “explained” by clicking on the “VPS Explained” located at the top of the first page (http://www.vpslux.com/?page=explained), I hit the letter ‘q’ on my keyboard to exit from Lynx. Inspect the file /tmp/www.vpslux.com to make sure all actions have been recorded. You should see something similar to this depending on the actions you have performed (I cut down quite a bit to keep it short):

# Command logfile created by Lynx 2.8.6rel.4 (15 Nov 2006)
# Arg0 = lynx
# Arg1 = -cmd_log=/tmp/vpslux.com
# Arg2 = http://vpslux.com
key Down Arrow
key Down Arrow
key Down Arrow
key Down Arrow
key Down Arrow
key ^J
key Down Arrow
key Down Arrow
key Down Arrow
key Down Arrow
key Down Arrow
key Down Arrow
key Down Arrow
key Down Arrow
key Down Arrow
key Down Arrow

The hard part is done! All we need to do now is create a small script that will use our www.vpslux.com file as the source or script. You could create a small Bash script called browseWeb.sh with the following content:

#!/bin/bash

lynx -cmd_script=/tmp/www.vpslux.com http://www.vpslux.com/

An extra step would to create a crontab entry to run the above script say every Monday morning at 4am:

crontab -e

0 4 * * 1 /home/user/browseWeb.sh

That’s all folks. I hope this is useful.