Bash Script – Script Based MySQL Table Replication on an Array of Slaves

This bash script only uses row count and MySQL checksum to determine table consistency. I created this script because I wanted a different option rather than using MySQL replication. I initially had replication set up and working between six sites but it would tend to break from time to time so I needed a different solution. In my situation, my data was only being updated once or twice a week. This script should only be used in cases where you’re not overly concerned about your data consistency and row count and checksum are sufficient enough to suit your needs. If you need a more accurate consistency check of your table data, look into the Percona Toolkit.

#!/bin/bash
#
# Sync DB Tables w/o Replication
# Author: Nathan Thomas 01/16/2014

#------#
# VARS #
#------#
# Server to sync DB from
MASTER='master01.fqdn.com'
MUSER='username'
MPASS='password'
DBNAME='mydbname'
TBLNAME='mytablename'
DUMP='/tmp/mydumpfile.sql'
LOGFILE='/var/log/mysql/mysql_repl_check.log'
MAILTO='my_email_addr@fqdn.com'

# Array of slave hostnames separated by a space
declare -a SLAVE=('slave01.fqdn.com' 'slave02.fqdn.com' 'slave03.fqdn.com')
# Array of slave usernames in same order
declare -a SUSER=('username1' 'username2' 'username3')
# Array of slave passwords in same order
declare -a SPASS=('password1' 'password2' 'password3')

#-----------#
# FUNCTIONS #
#-----------#
# Check table exists
# params: $1=username $2=password $3=hostname
function tableExists() {
        echo "`date "+%a %b%e %T"` - Called function 'tableExists' on $3." >> ${LOGFILE}
        echo "`date "+%a %b%e %T"` - Running query to verify table '${TBLNAME}' exists in database '${DBNAME}' on $3." >> ${LOGFILE}
        local QUERY="SELECT COUNT(1) FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_SCHEMA='${DBNAME}' AND TABLE_NAME='${TBLNAME}'"
        local RESULT=`mysql -u $1 -p$2 -h $3 -Bse "$QUERY" 2>&1`
        local RETVAL="$?"
        echo "`date "+%a %b%e %T"` - Query on $3 retured the following value: '${RESULT}'" >> ${LOGFILE}
        echo "`date "+%a %b%e %T"` - Exit status of last command on $3 returned: '${RETVAL}'" >> ${LOGFILE}
        if [ "${RETVAL}" -eq "0" -a "${RESULT}" -eq "0" 2>/dev/null ] ; then  # Retval 0, query returned 0, hide stderr, Table missing
                echo "`date "+%a %b%e %T"` - Table '${TBLNAME}' does not exist in database '${DBNAME}' on $3." >> ${LOGFILE}
                echo "`date "+%a %b%e %T"` - Function 'tableExists' completed successfully on $3." >> ${LOGFILE}
                return 1  # No dice
        elif [ "${RETVAL}" -eq "0" -a "${RESULT}" -eq "1" 2>/dev/null ] ; then  # Retval 0, query returned 1, hide stderr, Table exists
                echo "`date "+%a %b%e %T"` - Table '${TBLNAME}' exists in database '${DBNAME}' on $3." >> ${LOGFILE}
                echo "`date "+%a %b%e %T"` - Function 'tableExists' completed successfully on $3." >> ${LOGFILE}
                return 0  # All good
        else  # Some other error
                echo "`date "+%a %b%e %T"` - Unable to determine table '${TBLNAME}' exists in database '${DBNAME}' on $3." >> ${LOGFILE}
                echo "`date "+%a %b%e %T"` - We either encountered an error code in the SQL data, had a problem connecting to MySQL, or the last command exited with a nonzero status." >> ${LOGFILE}
                echo "`date "+%a %b%e %T"` - Function 'tableExists' did not finish successfully on $3." >> ${LOGFILE}
                return 2  # No dice
        fi
}

# Get row count
# params: $1=username $2=password $3=hostname
function getRowCount() {
        echo "`date "+%a %b%e %T"` - Called function 'getRowCount' on $3." >> ${LOGFILE}
        echo "`date "+%a %b%e %T"` - Verifying table '${TBLNAME}' exists in database '${DBNAME}' on $3." >> ${LOGFILE}
        tableExists $1 $2 $3
        if [ "$?" -eq "0" ] ; then  # Retval 0, table exists
                echo "`date "+%a %b%e %T"` - Running query to obtain row count on table '${TBLNAME}' in database '${DBNAME}' on $3." >> ${LOGFILE}
                local QUERY="SELECT COUNT(id) FROM ${TBLNAME}"
                local RESULT=`mysql -u $1 -p$2 -h $3 ${DBNAME} -Bse "${QUERY}" 2>&1`
                local RETVAL="$?"
                echo "`date "+%a %b%e %T"` - Query on $3 retured the following value: '${RESULT}'" >> ${LOGFILE}
                echo "`date "+%a %b%e %T"` - Exit status of last command on $3 returned: '${RETVAL}'" >> ${LOGFILE}
                if [ "${RETVAL}" -eq "0" -a -n "${RESULT}" -a "${RESULT}" -eq "${RESULT}" 2>/dev/null ] ; then  # Retval 0, check for nonzero string, is numeric, hide stderr
                        echo "${RESULT}"
                        echo "`date "+%a %b%e %T"` - The row count on $3 is: '${RESULT}'" >> ${LOGFILE}
                        echo "`date "+%a %b%e %T"` - Function 'getRowCount' completed successfully on $3." >> ${LOGFILE}
                        return 0  # All good
                else
                        echo "`date "+%a %b%e %T"` - Unable to retrieve row count on $3." >> ${LOGFILE}
                        echo "`date "+%a %b%e %T"` - We either encountered an error code in the SQL data, had a problem connecting to MySQL, or the last command exited with a nonzero status." >> ${LOGFILE}
                        echo "`date "+%a %b%e %T"` - Function 'tableExists' did not finish successfully on $3." >> ${LOGFILE}
                        return 1  # No dice
                fi
        fi
}

# Get table checksum
# params: $1=username $2=password $3=hostname
function getTableChecksum() {
        echo "`date "+%a %b%e %T"` - Called function 'getTableChecksum' on $3." >> ${LOGFILE}
 Continue reading "Bash Script – Script Based MySQL Table Replication on an Array of Slaves"

Install MySQL Activity Report from Source

MySQL Activity Report is a handy database reporting tool that uses RRD (Round Robin Database) to display hourly, daily, weekly, and monthly graphs and gives helpful performance tuning recommendations for your MySQL installation. Here are the steps to install it from source on Ubuntu Server 12.04.3 while logged in as root. This assumes that you have the build-essentials, header files, etc necessary to build software already installed.

1. Change into your local source directory

cd /usr/local/src

2. Download the source files for rrdtool and mysqlard

wget http://oss.oetiker.ch/rrdtool/pub/rrdtool-1.4.8.tar.gz
wget http://gert.sos.be/downloads/mysqlar/mysqlard-1.0.0.tar.gz

3. Unzip the files

tar -zxvf rrdtool-1.4.8.tar.gz
tar -zxvf mysqlard-1.0.0.tar.gz

4. Change into the rrd directory

cd rrdtool-1.4.8

5. Install dependencies

apt-get install libpango1.0-dev libxml2-dev

6. Build rrdtool (will install to the /opt/rrdtool-1.4.8 directory)
Continue reading “Install MySQL Activity Report from Source”

Bash Script – Move Asterisk Call Files into Spool Directory

After running into permission issues trying to get PHP’s shell_exec command to chown call files as the asterisk user(which only root can do), I decided to make this script.

As a side note, you should be able to get the shell_exec command to work using sudo by adding the www-data user to the sudoer’s file without a password but that wouldn’t work in my particular environment. The server I was working on was extremely outdated and didn’t even have sudo installed.

Add this to /etc/sudoers
www-data ALL=NOPASSWD: /path/to/script

The following code runs the script as a daemon. You will need to update rc to start this script at default run levels and also make sure to chmod +x this file to make it executable.

filename: /etc/init.d/mvcallfile

#!/bin/bash
# Move asterisk call file daemon startup script
# Author: Nathan Thomas

PROG=mvcallfile
 Continue reading "Bash Script – Move Asterisk Call Files into Spool Directory"