Move Proxmox Container to Different Storage

Move Proxmox Container to Different Storage

Get Social!

2015-03-05 00_18_04-Proxmox Virtual Environment storageA task often required when new storage is added or removed, and containers grow over time is to move a container onto another storage device.

The  Proxmox  Web GUI does not give us the ability to migrate a container from one storage device to another directly. To move a container onto different storage we have to take a backup of the container and restore it to the same ID with a different storage device specified. This can be time laborious when working with several containers.

The below script allows you to move an OpenVZ container from one storage device to another. The process requires that the container be stopped, which the script will handle.

Save the below script into a file called migrate.

vi migrate
#!/bin/bash
#
# Filename : migrate
# Description : Migrate Proxmox OpenVZ container from one storage to another
# Author : James Coyle
#
# Version:
# -Date       -Author      -Description
# 20-11-2013  James Coyle  Initial
#
#

# Variables
TMP=/tmp      #Location to use to create the backup for transferring to new storage. This needs to be big enough to store the backup archive for the container.

# Do not edit
usage() { 
	echo "Usage: $0" 
	echo "          [-c Required: Container ID to migrate <int>] "
	echo "          [-s Required: Target storage ID <string>]"
	echo "          [-d Optional: Delete the backup file after CT restoration <boolean>]"
	echo ""
	echo "Example: $0 -c 100 -s nasarray"
	echo ""
	exit 1; 
}

while getopts "c:s:d" o; do
  case "${o}" in
    c)
      CT=${OPTARG}
      ;;
    s)
      TARGET_STORAGE=${OPTARG}
      ;;
    d)
      DELETE=true
      ;;
    *)
      usage
      ;;
    esac
done
shift $((OPTIND-1))

# Check mandatory fields
if [ -z "${CT}" ] || [ -z "${TARGET_STORAGE}" ]; then
  usage
fi

RUNNING=false

set -e
set -o pipefail

echo "Moving $CT to $TARGET_STORAGE..."
if vzlist | fgrep -w -q " $CT "
then
    RUNNING=true
fi

if $RUNNING
then
    vzctl stop $CT
fi

vzdump --dumpdir $TMP $CT

ARCHIVE=$(ls -t $TMP/vzdump-openvz-$CT-*.tar | head -n 1)

vzrestore $ARCHIVE $CT -force -storage $TARGET_STORAGE

if $RUNNING
then
    vzctl start $CT
fi

if $DELETE
then
    LOG=$(ls -t $TMP/vzdump-openvz-$CT-*.log | head -n 1)
    echo "Deleting $LOG and $ARCHIVE"
    rm -rf $ARCHIVE $TMP/$LOG
fi

Set execution permissions on the script:

chmod + x migrate

The script has several parameters which are detailed below:

  • -d is specified if you would like the script to delete the temporary backup after the process has completed. Leave this out if you would like the backup tar file to be kept, just in case anything goes wrong.
  • -s is required to specify the name of the target storage. You can find this from the Proxmox Web GUI.
  • -c is required for the container ID to migrate.

In addition, the script contains the variable TMP. This will be the location of the backup tar created as part of the migration process and must contain enough space to store the content of the container being migrated. You can change this to suit your environment.

Example command:

./migrate -d -s newstorage -c 101

 


Export MySQL Database into Separate Files per Table

Category : How-to

Get Social!

mysql-logoI have recently been using git to check in an applications database. The database has many tables, some of which are populated with test data and created a fairly large file when exported. I noticed a few issues issues when checking these into git, namely that the large file was uploaded and saved in git as a single large file containing my changes and the other stuff which had not changed.

Instead of using this large file as one and checking it into git, breaking the file into several smaller files means that only the table which changed would be added to the git commit resulting in much smaller uploads.

The below code is a bash script which let’s you export, using mysqldump, all tables in a MySQL database to individual files. This will result in one file per MySQL table in the database. You will need to modify the following attributes:

  • [USER] – the username to use when connecting to the MySQL instance.
  • [PASSWORD] – the password for the above MySQL user.
  • [DATABASE] – the name of the MySQL database to export.
  • [BACKUP_LOCATION] – the location on the MySQL server where the SQL files will be created.
#!/bin/bash
GIT_MYSQL=/[BACKUP_LOCATION]
for T in `mysql -u [USER] -p[PASSWORD] -N -B -e 'show tables from [DATABASE]'`;
do
    echo "Backing up $T"
    mysqldump --skip-comments --compact -u [USER] -p[PASSWORD] [DATABASE] $T > $GIT_MYSQL/$T.sql
done;

Bash Script to Create an SSL Certificate Key and Request (CSR)

Category : How-to

Get Social!

padlockCreating multiple SSL certificates for web servers and application can be a repetitive task. Generally speaking, when creating these things manually you would follow the below steps:

  • Create a certificate key.
  • Create the certificate signing request (CSR) which contains details such as the domain name and address details.
  • Sign the certificate
  • Install the certificate and key in the application.

If nothing else, typing out the address and organisation for every certificate can be laborious.

The below script allows you to hard code many of the details to avoid the repetition and only specify the domain name as an argument. The script is dependent on openssl which can be installed using your distributions package manger or from their website. Use apt-get on Debian/ Ubuntu:

apt-get install openssl

Once you have openssl installed, copy the below script to a file called gen-cer.

vi gen-cer
#!/bin/bash

#Required
domain=$1
commonname=$domain

#Change to your company details
country=GB
state=Nottingham
locality=Nottinghamshire
organization=Jamescoyle.net
organizationalunit=IT
[email protected]

#Optional
password=dummypassword

if [ -z "$domain" ]
then
    echo "Argument not present."
    echo "Useage $0 [common name]"

    exit 99
fi

echo "Generating key request for $domain"

#Generate a key
openssl genrsa -des3 -passout pass:$password -out $domain.key 2048 -noout

#Remove passphrase from the key. Comment the line out to keep the passphrase
echo "Removing passphrase from key"
openssl rsa -in $domain.key -passin pass:$password -out $domain.key

#Create the request
echo "Creating CSR"
openssl req -new -key $domain.key -out $domain.csr -passin pass:$password \
    -subj "/C=$country/ST=$state/L=$locality/O=$organization/OU=$organizationalunit/CN=$commonname/emailAddress=$email"

echo "---------------------------"
echo "-----Below is your CSR-----"
echo "---------------------------"
echo
cat $domain.csr

echo
echo "---------------------------"
echo "-----Below is your Key-----"
echo "---------------------------"
echo
cat $domain.key

Make sure your script has execute permissions.

chmod +x gen-cer

You can then call the script with ./gen-cer and specify your domain name as an argument. For example:

./gen-cer mynewwebserver.jamescoyle.net

The script will then output the key as well as the CSR which you will need to submit to your certificate authority (CA).


Script to Automatically Detect and Restart Linux PPTP Client

Category : How-to

Get Social!

Linux penguinThe default PPTP client for Linux does not automatically start on boot, or restart on a failed or dropped connection. I have written a short script to ping your VPN server gateway IP address and start the PPTP client if a ping does not succeed.

See my other post if you have not yet set up your PPTP VPN client.

Create this script and make it executable:

vi /root/cron/pptp_cron.sh
chmod +x /root/cron/pptp_cron.sh

Add the below script to the file and change the following attributes for your own values:

  • your-vpn-host-or-ip-address
  • your-vpn-username
  • your-vpn-password
#!/bin/bash

HOST=your-vpn-host-or-ip-address
PPTPUSER=your-vpn-username
PPTPPASS=your-vpn-password

DATE=`date`
PINGRES=`ping -c 2 $HOST`
PLOSS=`echo $PINGRES : | grep -oP '\d+(?=% packet loss)'`
echo "$DATE : Loss Result : $PLOSS"

if [ "100" -eq "$PLOSS" ];
then
    echo "$DATE : Starting : $HOST"
    /usr/sbin/pptp pty file /etc/ppp/options.pptp user $PPTPUSER password $PPTPPASS
    echo "$DATE : Now running : $HOST"
else
    echo "$DATE : Already running : $HOST"
fi

Add the following entry to your cron to execute the script every minute.

crontab -e
 */1  * * * * /root/cron/pptp_cron.sh >> /var/log/pptp_pinger.log 2>&1

See my other post if you have not yet set up your PPTP VPN client.


Backup all Proxmox OpenVZ containers in one go

Category : How-to

Get Social!

proxmox logo gradThe below script is a bash script which works with Proxmox and the OpenVZ commands to backup all known containers to a specified folder.

You will need to set BACKUP_PATH to be the folder where you would like the backups to be stored and COMPRESS to a value which specifies the compression used. COMPRESS values can be:

  • 0 – no compression.
  • 1 – default compression (usually lzo).
  • gzip – gzip compression.
  • lzo – lzo compression.

Paste the below file into /bin/backup_all and make sure it’s executable.

vi /bin/backup-all
#!/bin/bash
#
# Filename : backup_all
# Description : Backup all OpenVZ containers in Proxmox
# Author : James Coyle
#
# Version:
# -Date      -Author     -Description
# 20-11-2013 James Coyle Initial
#
#

BACKUP_PATH=/var/lib/vz/dump
COMPRESS="lzo"

# Check dir exists
if [ ! -d $BACKUP_PATH ]; then
  echo "The directory $BACKUP_PATH does not exist."
  exit 99
fi

IFS=$'\n'
VMARRAY=($(vzlist -a -H))
VMIDS=""

for V in ${VMARRAY[@]}
do
  VMIDS="$VMIDS ${V:7:3}"
done

if [ -n $VMIDS ]; then
  vzdump $VMIDS --dumpdir $BACKUP_PATH --mode snapshot --compress $COMPRESS --remove 0
fi

echo "Backup of VMID(s) $VMIDS complete."

Restore a single Proxmox OpenVZ Container From The command Line

Get Social!

proxmox logo gradI mostly use Proxmox from the command line, or terminal, and I have created a few scripts to perform common and repetitive tasks.

The below script will restore a single OpenVZ container to the latest backup file available in the dump directory. The scripts takes a parameter for the container VMID to restore from backup. If the container exists, it will be stopped and removed before restoring the latest backup file available in the backup directory.

The script iterates through all of your backup files and only restores the latest based on the date in the file name, and not the date of the file creation or modified.

You will need to set the BACKUP_PATH variable to the location of your backup folder with no trailing slash, and BACKUP_EXT with the extension used for your chosen backup format.

If you save this script in the /bin then you can call the script from the terminal without having to move to the scripts directory. Create the file and paste the below script into it.

vi /bin/restore_one
#!/bin/bash
#
# Filename : restore_one
# Description : Restores a single OpenVZ Proxmox container to the latest backup file
#               available in the dump folder.
# Author : James Coyle
#
# Version:
# -Date -Author -Description
# 01-11-2013 James Coyle Initial
#
#

BACKUP_PATH=/var/lib/vz/dump
BACKUP_EXT=tar.lzo

# Do not change
SEARCH_PATH=$BACKUP_PATH/vzdump-openvz-$1-*.$BACKUP_EXT

function display-useage
{
  echo "Useage $0 [vmid to restore]"
  echo "Example: $0 999"

}

# Check dir exists
if [ ! -d $BACKUP_PATH ]; then
  echo "The directory $BACKUP_PATH does not exist"
  exit 99
fi

# Check if argument is present
if [ -z "$1" ]
then
  echo "Argument not present."
  display-useage
  exit 99
fi

# Check if vmid is available, on or off
VMON=$(vzlist | grep -P "[ ]+$1[ ]+")
VMOFF=$(vzlist --stopped | grep -P "[ ]+$1[ ]+")

if [ -n "$VMON" ]; then
  echo "Requesting stop of container."
  vzctl stop $1
  echo "Requesting deletion of container."
  vzctl delete $1
elif [ -n "$VMOFF" ]; then
  echo "Container is stopped."
  echo "Requesting deletion of container."
  vzctl delete $1
else
  echo "Container is not live."
fi

# Get unique VMIDs
for F in $SEARCH_PATH
do
  FILENAME=${F##*/}
  FILE_DATE=${FILENAME:18:19}
  FILE_DATE=${FILE_DATE//[_\-]/}

  if [ -z "$BACKUP_FILE" ]; then
    BACKUP_FILE=$F
  fi

  TEST_FILENAME=${BACKUP_FILE##*/}
  TEST_FILE_DATE=${TEST_FILENAME:18:19}
  TEST_FILE_DATE=${TEST_FILE_DATE//[_\-]/}
  if [ "$FILE_DATE" -gt "$TEST_FILE_DATE" ]; then
    BACKUP_FILE=$F
  fi
done

if [ -n $BACKUP_FILE ]; then
  # Restore VM
  echo "Restoring $1 with $BACKUP_FILE..."
  vzrestore $BACKUP_FILE $1
else
  echo "No backup file for VMID $1 exists."
fi

Make the script executable using chmod.

chmod +x /bin/restore_one

Use the below command, and substitute [VMID] with the container VMID to restore, to run the script.

restore_one [VMID]

See my other script on restoring multiple OpenVZ containers in Proxmox.


Visit our advertisers

Quick Poll

Do you use ZFS on Linux?

Visit our advertisers