Setup Headless Dropbox Sync Client on Linux

Setup Headless Dropbox Sync Client on Linux

Category : How-to

Get Social!

dropbox-logoDropbox is a cloud based file storage service which makes your files available from almost any internet connected device. You simply synchronize a folder with the service on each device and Dropbox keeps them in sync, automatically updating each folder as files are added and removed on each machine.

You can register for a free account which gives you a limited amount of storage to use – no strings attached. To register, visit Dropbox (Please note, that is my affiliate link which gives me a bonus if you sign up. If you’re not happy with this, you can simply visit dropbox.com and sign up. I’d be most grateful if you’d use the link :) ).

Dropbox offers a client to use on Windows, Mac OSX, Linux, iPhone and Android which you can download for free. These are all graphical interfaces and do not work for deployment on a headless server.

Thankfully, the Dropbox team have created an easily deployable Dropbox client which works without a desktop installed and can be managed by a Python script. To get started, download the Python script to your home directory on your headless server with wget.

cd ~
wget https://www.dropbox.com/download?dl=packages/dropbox.py -O dropbox.py

Give the script the permission to execute.

chmod +x dropbox.py

You can now use the Python script to download the Dropbox client. The client will be downloaded to your users home directory so make sure that you are logged in with the correct user.

./dropbox.py start -i

The client binaries will now be downloaded and installed. The next step is to register your account with the Dropbox client so that synchronization can begin. Start Dropbox for the first time with the Python script and you will be presented with a link. Paste the link into a web browser and login to your Dropbox account to grant access to the client. As soon as this process completes, your Dropbox client will begin synchronization.

./dropbox.py start
This client is not linked to any account... Please visit hhttps://www.dropbox.com/cli_link?host_id=10a5ce48e41d50f8135dd6fd55b70a91 to link this machine.

Once you have got all this working and the Dropbox client is synchronizing your first files, it’s time to add an init.d script so that the Dropbox client starts with your operating system. Things may differ here, depending on your Linux distribution. Add one of the below scripts to your init.d folder and substitute [USER] for the list of users who will use the client.

vi /etc/init.d/dropbox

Ubuntu/ Debian

#!/bin/sh
#dropbox service
DROPBOX_USERS="[USER]"

DAEMON=.dropbox-dist/dropboxd

start() {
   echo "Starting dropbox..."
   for dbuser in $DROPBOX_USERS; do
       HOMEDIR=`getent passwd $dbuser | cut -d: -f6`
       if [ -x $HOMEDIR/$DAEMON ]; then
           HOME="$HOMEDIR" start-stop-daemon -b -o -c $dbuser -S -u $dbuser -x $HOMEDIR/$DAEMON
       fi
   done
}

stop() {
   echo "Stopping dropbox..."
   for dbuser in $DROPBOX_USERS; do
       HOMEDIR=`getent passwd $dbuser | cut -d: -f6`
       if [ -x $HOMEDIR/$DAEMON ]; then
           start-stop-daemon -o -c $dbuser -K -u $dbuser -x $HOMEDIR/$DAEMON
       fi
   done
}

status() {
   for dbuser in $DROPBOX_USERS; do
       dbpid=`pgrep -u $dbuser dropbox`
       if [ -z $dbpid ] ; then
           echo "dropboxd for USER $dbuser: not running."
       else
           echo "dropboxd for USER $dbuser: running (pid $dbpid)"
       fi
   done
}

case "$1" in

   start)
       start
       ;;
   stop)
       stop
       ;;
   restart|reload|force-reload)
       stop
       start
       ;;
   status)
       status
       ;;
   *)
       echo "Usage: /etc/init.d/dropbox {start|stop|reload|force-reload|restart|status}"
       exit 1

esac

exit 0

Then add the execute permission and add it to the startup routine.

chmod +x /etc/init.d/dropbox 
update-rc.d dropbox defaults

RedHat/ Fedora/ CentOS

# chkconfig: 345 85 15
# description: Startup script for dropbox daemon
#
# processname: dropboxd
# pidfile: /var/run/dropbox.pid
# config: /etc/sysconfig/dropbox
#

### BEGIN INIT INFO
# Provides: dropboxd
# Required-Start: $local_fs $network $syslog
# Required-Stop: $local_fs $syslog
# Should-Start: $syslog
# Should-Stop: $network $syslog
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: Start up the Dropbox file syncing daemon
# Description: Dropbox is a filesyncing sevice provided by dropbox.com
# This service starts up the dropbox daemon.
### END INIT INFO

# Source function library.
. /etc/rc.d/init.d/functions

# To configure, add line with DROPBOX_USERS="[USERS]" to /etc/sysconfig/dropbox
# Probably should use a dropbox group in /etc/groups instead.

[ -f /etc/sysconfig/dropbox ] && . /etc/sysconfig/dropbox
prog=dropboxd
lockfile=${LOCKFILE-/var/lock/subsys/$prog}
config=${CONFIG-/etc/sysconfig/dropbox}
RETVAL=0

start() {
   echo -n $"Starting $prog"
   if [ -z $DROPBOX_USERS ] ; then
      echo -n ": unconfigured: $config"
      echo_failure
      echo
      rm -f ${lockfile} ${pidfile}
      RETURN=6
      return $RETVAL
   fi
   for dbuser in $DROPBOX_USERS; do
      daemon --user $dbuser /bin/sh -c "~$dbuser/.dropbox-dist/dropboxd&"
   done
   RETVAL=$?
   echo
   [ $RETVAL = 0 ] && touch ${lockfile}
   return $RETVAL
}

status() {
   for dbuser in $DROPBOX_USERS; do
      dbpid=`pgrep -u $dbuser dropbox | grep -v grep`
      if [ -z $dbpid ] ; then
         echo "dropboxd for USER $dbuser: not running."
      else
         echo "dropboxd for USER $dbuser: running (pid $dbpid)"
      fi
      done
}

stop() {
   echo -n $"Stopping $prog"
   for dbuser in $DROPBOX_USERS; do
      killproc ~$dbuser/.dropbox-dist/dropbox
   done
   RETVAL=$?
   echo
   [ $RETVAL = 0 ] && rm -f ${lockfile} ${pidfile}
}

# See how we were called.
case "$1" in
   start)
      start
      ;;
   status)
      status
      ;;
   stop)
      stop
      ;;
   restart)
      stop
      start
      ;;
   *)
      echo $"Usage: $prog {start|status|stop|restart}"
      RETVAL=3

esac

exit $RETVAL

Change permissions and add to the startup routine.

chmod 0755 /etc/init.d/dropbox 
chmod 0644 /etc/sysconfig/dropbox
chkconfig dropbox on

Mount a Windows/ Samba Share in Linux

Category : How-to

Get Social!

Linux penguinGiven the amount of trouble you can have getting Linux and Windows to play nicely together, you may be surprised to find out that it’s easy to mount either a Windows file share on Linux. You can also use the same method to mount a Samba share.

This article assumes you already have a Windows or Samba share set up which is protected by a username and password. The user could be part of a domain or local to the machine which presents the storage.

The first step is to create a credentials file which will contain the username and password for accessing the file share. This needs to be in a protected area so that the details cannot be viewed. For this example, I will put the file in the root home directory.

vi /root./smbcreds

Add the following details, substituting the below details

  • [DOMAIN] – this is the domain for the file share user. This may not be required in all environments.
  • [USER] – this is the user name for the file share.
  • [PASSWORD] – this is the password of the above user.
username=[DOMAIN]\[USER]
password=[PASSWORD]

For example:

username=JAMESCOYLE\james.coyle
password=mypassword

Before continuting with the configuration, we need to make sure that the required packages are installed. On Ubuntu, you can use the apt-get command. Use your distributions package manager for other Linux distributions.

apt-get install cifs-utils

The next step is to create a folder where you will mount the Windows or samba share to in Linux. I will use the mount point /mnt/winshare.

mkdir /mnt/winshare

Now open fstab and add a new row at the bottom of the file to mount the remote share to the local folder. Add the below line to your fstab and substitute the values for your environment.

  • [SHARE_LOCATION] – is the remote host name/ DNS and shared folder name.
  • [MOUNT_LOCATION] – is the local path which will be used as the mount point for the Windows or Samba share.
  • cifs – is the type of mount to use. For older Linux systems you may need to use smbfs.
  • [CREDENTIALS_FILE] – is the location of the local credentials file.
//[SHARE_LOCATION]  [MOUNT_LOCATION]  cifs  credentials=[CREDENTIALS_FILE],iocharset=utf8,uid=1001,_netdev,file_mode=0770,dir_mode=0770 0 0

For example:

//windowsbox/sharename  /mnt/winshare  cifs  credentials=/root/.smbcreds,iocharset=utf8,uid=1001,_netdev,file_mode=0770,dir_mode=0770 0 0

Finally we need to create the mount which will use the details in fstab. Change [MOUNT_POINT] to match the mount point you created.

mount [MOUNT_POINT]

For example:

mount /mnt/winshare

Benchmark MySQL server Performance with Sysbench

Get Social!

mysql-logoYou can spend hours tweaking the settings of a MySQL server instance to get the best possible performance for your hardware and environment. The hardest part is to ensure that the changes made are reflected with increased performance.

To ensure each change results in better performance of the MySQL server we need to measure the performance of the MySQL server before and after the change.

There are a verity of tools to automate MySQL benchmarking, one of which is Sysbench. I will be demonstrating the tests on a Debian 7 system however Sysbench will work on most common Linux distributions. Sysbench can be used to test both InnoDB or MyISAM database types in either a single server environment or a clustered environment with a single instance.

Installing Sysbench will differ on each Linux distribution; it can be downloaded and built from source from Sourceforge or installed with apt-get on Ubuntu or Debian.

apt-get install sysbench

Login to MySQL using the CLI or your favorite GUI tool and create a new database which will be used for the test. If you already have a database you can use for the test then you can skip this step. This example will use a database called dbtest for the tests.

create database dbtest;

The next step is to use the prepare statement with sysbench to generate a table in the specified database which will be used when performing tests.

From the command line, run the below command changing [USER] and [PASSWORD] to your MySQL access credentials.

sysbench --test=oltp --oltp-table-size=1000000 --mysql-db=dbtest --mysql-user=[USER] --mysql-password=[PASSWORD] prepare

sysbench 0.4.12:  multi-threaded system evaluation benchmark

No DB drivers specified, using mysql
Creating table 'sbtest'...
Creating 1000000 records in table 'sbtest'...

This has created a table called sbtest with 1000000 rows of data which will be used for testing. The below commands show the the created table and do not need to be executed.

mysql> use dbtest;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
mysql> show tables;
+------------------+
| Tables_in_dbtest |
+------------------+
| sbtest           |
+------------------+
1 row in set (0.00 sec)

mysql> SELECT COUNT(*) FROM sbtest;
+----------+
| COUNT(*) |
+----------+
|  1000000 |
+----------+
1 row in set (0.12 sec)

The next step is to being the performance tests. There are multiple parameters which can be changed to alter the test performed but we will do a simple read write test. Again you will need to change [USER] and [PASSWORD] to your MySQL access credentials.

sysbench --test=oltp --oltp-table-size=1000000 --oltp-test-mode=complex --oltp-read-only=off --num-threads=6 --max-time=60 --max-requests=0 --mysql-db=dbtest --mysql-user=[USER] --mysql-password=[PASSWORD] run

To perform a read only test, change the above parameter oltp-read-only=off to oltp-read-only=on.

The results will look similar to the below output. The main statistic to look for is transactions which shows the number of transactions the test managed to complete, and how many per second.

sysbench 0.4.12:  multi-threaded system evaluation benchmark

No DB drivers specified, using mysql
Running the test with following options:
Number of threads: 6

Doing OLTP test.
Running mixed OLTP test
Using Special distribution (12 iterations,  1 pct of values are returned in 75 pct cases)
Using "BEGIN" for starting transactions
Using auto_inc on the id column
Threads started!
Time limit exceeded, exiting...
(last message repeated 5 times)
Done.

OLTP test statistics:
    queries performed:
        read:                            456680
        write:                           163100
        other:                           65240
        total:                           685020
    transactions:                        32620  (543.63 per sec.)
    deadlocks:                           0      (0.00 per sec.)
    read/write requests:                 619780 (10329.05 per sec.)
    other operations:                    65240  (1087.27 per sec.)

Test execution summary:
    total time:                          60.0036s
    total number of events:              32620
    total time taken by event execution: 359.8823
    per-request statistics:
         min:                                  1.66ms
         avg:                                 11.03ms
         max:                                981.94ms
         approx.  95 percentile:              15.13ms

Threads fairness:
    events (avg/stddev):           5436.6667/31.44
    execution time (avg/stddev):   59.9804/0.00

Finally, you need to clean up your test area. If you can drop the entire database which was used for testing then login to MySQL and run the below command.

drop database dbtest;

If you are unable to drop the whole database then Sysbench comes with a cleanup command. Again you will need to change [USER] and [PASSWORD] to your MySQL access credentials.

sysbench --test=oltp --mysql-db=dbtest --mysql-user=[USER] --mysql-password=[PASSWORD] cleanup

 


Bash Script to Create an SSL Certificate Key and Request (CSR)

Category : How-to

Get Social!

padlockCreating multiple SSL certificates for web servers and application can be a repetitive task. Generally speaking, when creating these things manually you would follow the below steps:

  • Create a certificate key.
  • Create the certificate signing request (CSR) which contains details such as the domain name and address details.
  • Sign the certificate
  • Install the certificate and key in the application.

If nothing else, typing out the address and organisation for every certificate can be laborious.

The below script allows you to hard code many of the details to avoid the repetition and only specify the domain name as an argument. The script is dependent on openssl which can be installed using your distributions package manger or from their website. Use apt-get on Debian/ Ubuntu:

apt-get install openssl

Once you have openssl installed, copy the below script to a file called gen-cer.

vi gen-cer
#!/bin/bash

#Required
domain=$1
commonname=$domain

#Change to your company details
country=GB
state=Nottingham
locality=Nottinghamshire
organization=Jamescoyle.net
organizationalunit=IT
[email protected]

#Optional
password=dummypassword

if [ -z "$domain" ]
then
    echo "Argument not present."
    echo "Useage $0 [common name]"

    exit 99
fi

echo "Generating key request for $domain"

#Generate a key
openssl genrsa -des3 -passout pass:$password -out $domain.key 2048 -noout

#Remove passphrase from the key. Comment the line out to keep the passphrase
echo "Removing passphrase from key"
openssl rsa -in $domain.key -passin pass:$password -out $domain.key

#Create the request
echo "Creating CSR"
openssl req -new -key $domain.key -out $domain.csr -passin pass:$password \
    -subj "/C=$country/ST=$state/L=$locality/O=$organization/OU=$organizationalunit/CN=$commonname/emailAddress=$email"

echo "---------------------------"
echo "-----Below is your CSR-----"
echo "---------------------------"
echo
cat $domain.csr

echo
echo "---------------------------"
echo "-----Below is your Key-----"
echo "---------------------------"
echo
cat $domain.key

Make sure your script has execute permissions.

chmod +x gen-cer

You can then call the script with ./gen-cer and specify your domain name as an argument. For example:

./gen-cer mynewwebserver.jamescoyle.net

The script will then output the key as well as the CSR which you will need to submit to your certificate authority (CA).


View Available Exports on an NFS server

Tags :

Category : How-to

Get Social!

Linux penguinThere is a handy command called showmount which displays all the active folder exports on an NFS server. This can be handy when trying to connect to a new NFS export from a remote machine as you can see if the export is available in the NFS server.

Run the showmount command with the server name to check which NFS exports are available. In this example, localhost is the server name.

showmount -e localhost

The output shows the available exports and the IP which they are available from. The below example shows 3 exports available from the 10.10.10.0 IP range.

Export list for localhost:
/tmp                 10.10.10.0/24
/zfs/volume1         10.10.10.0/24
/zfs/volume2         10.10.10.0/24

Simple Bonnie++ Example

Category : How-to

Get Social!

Linux penguinBonnie++ is a disk and file system benchmarking tool for measuring I/O performance. With Bonnie++ you can quickly and easily produce a meaningful value to represent your current file system performance.

Before using Bonnie++ make sure that you have it installed on your system. In Ubuntu, use apt-get to install the bonnie++ package.

apt-get install bonnie++

Run the bonnie++ command  with the following attributes:

  • [TEST_LOCATION] – is where bonnie++ will create the benchmark operations.
  • [TEST_SIZE] – the size of the test file – this should be greater than double the RAM in your system.
  • [TEST_NAME] – this is simply a label which will be written out with the results.
  • [TEST_USER] – the user who should perform the test. This is not required if you are not running as root.
bonnie++ -d [TEST_LOCATION] -s [TEST_SIZE] -n 0 -m [TEST_NAME] -f -b -u [TEST_USER]

For example:

bonnie++ -d /tmp -s 4G -n 0 -m TEST -f -b -u james

Using uid:1000, gid:1000.
Writing intelligently...done
Rewriting...done
Reading intelligently...done
start 'em...done...done...done...done...done...
Version  1.96       ------Sequential Output------ --Sequential Input- --Random-
Concurrency   1     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
TEST             4G           374271  39 214541  19           392015  17 +++++ +++
Latency                         167ms   89125us             52047us    4766us

1.96,1.96,TEST,1,1387339401,4G,,,,374271,39,214541,19,,,392015,17,+++++,+++,,,,,,,,,,,,,,,,,,,167ms,89125us,,52047us,4766us,,,,,,

The easiest way to understand the results of a bonnie++ test is to run the output  through the bon_csv2html utility. This perl script uses the bonnie++ results and generates a HTML page which you can later open with your web browser.

Copy the last line of the bonnie++ output into the echo command to replace [RESULTS] and alter the [OUTPUT] path to point to where you would like to save your results.

echo "[RESULTS]" | bon_csv2html > [OUTPUT]

Example command:

echo "1.96,1.96,TEST,1,1387339401,4G,,,,374271,39,214541,19,,,392015,17,+++++,+++,,,,,,,,,,,,,,,,,,,167ms,89125us,,52047us,4766us,,,,,," | bon_csv2html > /tmp/test.html

Finally open the output file with your web browser.

bonnie-results-html

See my other post on using bonnie++ to benchmark your file system.


Visit our advertisers

Quick Poll

Do you use GlusterFS in your workplace?

Visit our advertisers