How to grow a mdadm RAID 5 array

How to grow a mdadm RAID 5 array

Tags :

Category : How-to

Get Social!

image

mdadm is a software RAID technology for creating local volumes with RAID functionality such as RAID 5. RAID 5 is used to provide redundancy in the even of a disk failure by using the equivalent space of one disk in the volume for parity data.
One advantage of using mdadm is that you can create volumes of disks to a given size and then expand them at a later date. For example, if you have a RAID 5 array with 3 disks you can expand it to four disks.
Before following these steps you should un-mount the file system using umount.
This example assumes your array is called md1 and the new disk is sdf.

mdadm --add /dev/md1 /dev/sdf1

Now we can grow /dev/md1 as follows.

mdadm --grow /dev/md1 --size=max

If this fails, you may need to remove the bitmap index before retrying the above command.

mdadm --grow --bitmap=none /dev/md1

The process will now start which can take a while. An Example, 5 x 2TB WD green disks took 3 days to complete. You can view the progress with:

watch cat /proc/mdstat

Once this completes, we run a file system check and resize the file system.

e2fsck -f /dev/md1
resize2fs /dev/md1
e2fsck -f /dev/md1
You can now re-mount your file system.
mount -a

Add a package to startup on Debian/ Ubuntu/ Red Hat/ CentOS

Tags :

Category : How-to

Get Social!

Linux penguin Most versions and distributions of Linux today have a start up manager application to easily set which programs are started when your Linux machine boots up. There are two distinct flavours of start up applications and a manual method for distributions without this feature installed.

Using a terminal and the command chkconfig on CentOS, Red Hat and Oracle Enterprise Linux; and update-rc.d on Debian and Ubuntu you can control which packages are available as services and which packages start with your computer.

Run Level

Before looking at the commands used to control startup services in Linux, it’s important to understand when a program should be running in relation to the current operating system state. For example, you probably don’t want your Apache service being started before you have networking.

Linux has the concept or a runlevel which dictates the state of the operating system as a number between 0 and 6 inclusive.

See my post on runlevels to understand when your application or service should be asked to start.

Start up with Red Hat, CentOS and Oracle Enterprise Linux

Controlling startup services in Linux distributions such as Red Hat Enterprise Linux (RHEL), CentOS and Oracle Enterprise Linux (OEL) is done using the chkconfig command.

See my chkconfig cheat sheet for more information on controlling startup services on RHEL based Linux.

Start up with Debian and Ubuntu

Debian and Ubuntu based Linux distributions use the command update-rc.d to control which services are started during machine boot.

For details of controlling such services, see my update-rc.d cheat sheet.

Start up for manually setting the start up services

For operating systems that are not managed by an application, such as upstart on Ubuntu, you will need to manually add the /etc/init.d/ start up script to the /etc/rc.local file.

The /etc/rc.local file contains a reference to all the services which are required on machine boot.

Run the below echo command to add a manual start up application. The application name must be the same as the /etc/init.d/ file which must exist.

echo "network" >> /etc/rc.local

To remove a service, edit the /etc/rc.local file and manually remove the service name.

vi /etc/rc.local

update-rc.d Cheat Sheet

Tags :

Category : Cheat Sheets

Get Social!

Linux penguinDebian and Ubuntu use the service command to control services and update-rc.d for adding and removing services from start up. Using the service command we can start, stop, restart and display all available services. With update-rc.d we can add and remove services and add them to the Ubuntu/ Debian start up scripts. As Linux operating systems have multiple states, or runlevels, you need to make sure you add any new services to the correct runlevels. For example, you would not want to start a web service application before starting networking.

See my post on runlevels for more information about runlevels in Linux..

What services are available for startup?

Use the status-all switch to list all services which are registered with the OS and issues them a status command. You will then get one of the following displayed next to each service:

  • [ + ] – Services with this sign are currently running.
  • [ – ] – Services with this sign are not currently running..
  • [ ? ] – Services that do not have a status switch.
service --status-all

Sample output:

service --status-all
 [ ? ] acpid
 [ - ] apparmor
 [ ? ] apport
 [ ? ] atd
 [ - ] bootlogd
 [ ? ] console-setup
 [ ? ] cron
 [ ? ] dbus
 [ ? ] dmesg
 [ ? ] dns-clean
 [ ? ] friendly-recovery
 [ ? ] glusterfs-server
 [ - ] grub-common
 [ ? ] gssd
 [ ? ] hostname
 [ ? ] hwclock
 [ ? ] hwclock-save
 [ ? ] idmapd
 [ ? ] irqbalance
 [ ? ] killprocs
 [ ? ] module-init-tools
 [ ? ] network-interface
 [ ? ] network-interface-container
 [ ? ] network-interface-security
 [ ? ] networking
 [ ? ] ondemand
 [ ? ] passwd
 [ ? ] plymouth
 [ ? ] plymouth-log
 [ ? ] plymouth-ready
 [ ? ] plymouth-splash
 [ ? ] plymouth-stop
 [ ? ] plymouth-upstart-bridge
 [ ? ] portmap
 [ ? ] portmap-wait
 [ ? ] pppd-dns
 [ ? ] procps
 [ ? ] rc.local
 [ ? ] resolvconf
 [ ? ] rpcbind-boot
 [ - ] rsync
 [ ? ] rsyslog
 [ ? ] screen-cleanup
 [ ? ] sendsigs
 [ ? ] setvtrgb
 [ + ] ssh
 [ ? ] statd
 [ ? ] statd-mounting
 [ - ] stop-bootlogd
 [ - ] stop-bootlogd-single
 [ ? ] sudo
 [ ? ] udev
 [ ? ] udev-fallback-graphics
 [ ? ] udev-finish
 [ ? ] udevmonitor
 [ ? ] udevtrigger
 [ ? ] ufw
 [ ? ] umountfs
 [ ? ] umountnfs.sh
 [ ? ] umountroot
 [ - ] unattended-upgrades
 [ - ] urandom
 [ ? ] whoopsie

Start a service

Starting a service is done using the command service followed by the service name and the command to start the service.

service apache2 start

Stop a service

Use the stop keyword with service to stop a service.

service apache2 stop

Check the status of a service

Each service has a status, usually running or not running. Some services, such as network, may have a different output and output more information on the service.

service apache2 status

Remove a service

Use the remove keyword with update-rc.d to remove the service start up command for an application. You will need to use the -f switch if the applications /etc/init.d start up file exists.

update-rc.d -f apache2 remove

 Add a service

Adding a service to Ubuntu or Debian is done with the update-rc.d command. You can specify which runlevels to start and stop the new service or accept the defaults. The init.d file will be added to the relevent rc.d startup folders.

 update-rc.d apache2 defaults

Setting Start and Kill priority

If you need more control over when your service is asked to start and stop, you may need to set the start and kill (S and K) values.

For a given runlevel, you may have several services starting. For example, you may have apache2 and mysql both starting on runlevel 3. Ideally, you’d want mysql to start before apache2 and shutdown after apache2. In this case we need to give mysql the priority in startup, but apache2 the priority in shutdown.

When starting, the lower the number, the earlier it will start. A start priority of 10 will start before a priority of 20. When killing, it’s the opposite. A higher number will be killed before a lower number.

To set the start and kill priority we simply add them to the above update-rc.d command with the start priority first, followed by the kill priority.

update-rc.d apache2 defaults [START] [KILL]

The below command will start mysql first, then apache2. On shutdown, the kill will be the reverse of the start with apache2 being killed first and mysql second.

update-rc.d apache2 defaults 90 90
update-rc.d mysql defaults 10 10

Because, in our example, both start and kill priorities are the same we can shorted the command to just

update-rc.d apache2 defaults 90
update-rc.d mysql defaults 10

Manually set the RunLevel to Start and Kill a service

You can manually specify the Linux RunLevel that the system must be in to Start and Kill your service. See my other blog post for more information on RunLevels.

To extend the above example, we can specify exactly which RunLevel apache2 will be started and stopped.

update-rc.d apache2 start 10 2 3 4 5 . stop 90 0 1 6 .

apache2 will be started (as long as it isn’t already) when the system enters RunLevel 2, 3, 4 or 5 with a priority of 10. It will then be asked to stop when the system enters RunLevel 0, 1 or 6 with a priority of 90.

Let me know in the comments if you think anything is missing.

logrotate Cheat Sheet

Tags :

Category : Cheat Sheets

Get Social!

logrotate Most Linux applications create log files that need to be managed by either archiving or deleting old log files. This process is called log file rotation. The most common log rotation utility for Linux is conveniently called logrotate. logrotate is configured using the main configuration file, or the logrotate configuration folder. The configuration file usually holds the global configuration and the pointer to the configuration folder. In common Linux distributions such as Ubuntu and Debian, the logrotate configuration file can be found:

/etc/logrotate.conf

And the configuration folder can be found:

/etc/logrotate.d/

The configuration folder is usually where you would add new log file configurations which are unique for a particular log file, or set of log files. For example, if you were to add a new log rotation action for the Ubuntu package manager apt, you may use something like below:

vi /etc/logrotate.d/apt
/var/log/apt/term.log {
  rotate 12
  monthly
  compress
  missingok
  notifempty
}
/var/log/apt/history.log {
  rotate 12
  monthly
  compress
  missingok
  notifempty
}

logrotate runs each day by default, and is invoked using the daily cron job. The below subject detail the common components of creating a logrotate configuration file.

Create a new empty template

To create a new logrotate configuration file, you need to create a new file in /etc/logrotate.d/. You will then need to add a reference to the log files you wish to rotate. This can be directly to a single file or use pattern matching to match a group of log files. The below example matches all log files in /var/log/myapp/ which have a .log extension.

/var/log/myapp/*.log {

}

You will need to add further commands to this template before it becomes useful. Further commands will be added inside the { and } tags.

Rotate based on log file size

Use the size keyword to rotate the log file when it exceeded a given file size. The below example rotates a file when it reaches 10 KB. 

/var/log/myapp/*.log {
  size 10k
}

Rotate based on time (Month, Week or Day)

You can rotate logs using the monthly, weekly or daily keyword to create a new log based on duration. The keywords explain them selves, and they can be used in conjunction with the size keyword to rotate on which ever criteria is met first.

/var/log/myapp/*.log {
  size 10k
  weekly
}

Limit how many log files are kept after rotation by number

The rotate keyword allows you to specify how many old, rotated, log files are kept before logrotate deletes them. The rotate keyword requires an integer to specify how many old log files are kept.

/var/log/myapp/*.log {
  size 10k
  weekly
  rotate 8
}

The above example will keep 8 old log files.

Limit how many files are kept after rotation by date

You can specify how long to keep rotated files using the maxage keyword. Any rotated log files which are older than maxage will be deleted. The below example will keep rotated log files for 56 days.

/var/log/myapp/*.log {
  size 10k
  weekly
  maxage 56
}

Compress rotated log files

Log files which have been rotated can be compressed to save disk space. Gzip is used by default.

/var/log/myapp/*.log {
  size 10k
  weekly
  rotate 8
  compress
}

You can change the default gzip compression to another format by specifying the compresscmd command and a different executable to use. The below example changes the compression format to bzip2 for better file compression.

/var/log/myapp/*.log {
  size 10k
  weekly
  rotate 8
  compress
  compresscmd /bin/bzip2
}

Ignore missing log files

If a log file does not exist when logrotate is running then an error will be thrown. You can use the keyword missingok to avoid this scenario and instruct logrotate to ignore the log file if it does not exist.

/var/log/myapp/*.log {
  size 10k
  weekly
  rotate 8
  missingok
}

Continue writing to the same log file after rotation

Usually when a log file is rotated the log file is moved to a new location. Some applications may throw an error, and others may continue to write to the relocated file. The copytruncate keyword copies all the log in the file to a new file and then truncates the original file. This keeps the original log file in place and also allows rotation to continue.

/var/log/myapp/*.log {
  size 10k
  weekly
  rotate 8
  copytruncate
}

 

Let me know in the comments if you think anything is missing.

chkconfig Cheat Sheet

Tags :

Category : Cheat Sheets

Get Social!

Linux penguinControlling startup services in Linux distributions such as Red Hat Enterprise Linux (RHEL), CentOS and Oracle Enterprise Linux (OEL) can be done using a Gnome GUI or a command line utility. The command line utility is called chkconfig and can list existing, add new or remove services from the operating systems startup list.

As Linux operating systems have multiple states, or runlevels, you need to make sure you add any new services to the correct runlevels. For example, you would not want to start a web service application before starting networking. See my post on runlevels.

What services are available for startup?

Use the –list switch to see your existing services and when they should be running.

chkconfig --list

An example output is below. This shows all of the machine runlevels and what the state of the service will be.

chkconfig --list

Note: This output shows SysV services only and does not include native systemd services. SysV configuration data might be overridden by native systemd configuration.

modules_dep 0:off 1:off 2:on 3:on 4:on 5:on 6:off
netconsole 0:off 1:off 2:off 3:off 4:off 5:off 6:off
network 0:off 1:off 2:off 3:on 4:off 5:off 6:off

In this example, there are three services available. netconsole is not available at any runlevels and is therefore never started automatically, network is available only in runlevel 3 and modules_dep is available in runlevel 2 and runlevel 3.

You could also use the example below to detail the runlevels of a single service only.

chkconfig network --list

Add a new service with chkconfig

Adding a new service is  easily done with the below command. The below example shows the service network being enabled to start at the next machine boot.

chkconfig network on

Use the –level switch to enable the service at specific runlevels. Use the below example to enable the service at runlevel 3.

chkconfig network on --level 3

Remove a service with chkconfig

Removing an existing service is done with the below command. The below example shows the service network being disabled from automatic start.

chkconfig network off

Use the –level switch to remove the service from specific runlevels. Use the below example to disable the service at runlevel 3.

chkconfig network off --level 3

Start a service

Starting a service is done using the command service followed by the service name and the command to start the service.

service network start

Stop a service

Use the stop keyword with service to stop a service.

service network stop

Check the status of a service

Each service has a status, usually running or not running. Some services, such as network, may have a different output and output more information on the service.

service network status

 

Let me know in the comments if you think anything is missing.

Linux Runlevels explained

Tags :

Category : Knowledge

Get Social!

Linux penguinA Linux runlevel dictates the state that the machine is currently operating, and what applications or services should be running at that time. The runlevel is specified as a number between 0 and 6 inclusive.

As your system starts up, it will move up through the runlevels until it reaches it’s desired state. Generally headless installs will be considered fully started at 2 or 3, and desktop GUIs usually at 4 or 5. The following details what each of the default runlevels represent. These are the standard definitions which may be slightly different depending on your Linux distribution.

  • 0 – is described as Halt. your machine will halt when the runlevel is set to 0.
  • 1 – is single-user Mode which is used for administrative tasks before the non-essential services are started.
  • 2 – is the first multi-user mode runlevel and will start some non-essential services. This may, or may not contain networking depending on your Linux distribution.
  • 3 – is a multi-user mode which headless servers usually run at. All essential and non-essential services such as Apache HTTP Server should be running.
  • 4 – is a multi-user mode runlevel which is not used by the default operating system and can be used for user defined purposes.
  • 5 – is when GUI desktops are loaded such as Gnome or KDE.
  • 6 – is the reboot runlevel. the operating system will reboot when runlevel 6 is issued.

You can check your current runlevel with the runlevel command. the below example shows a runlevel of 2.

runlevel
N 2

You can also change the current runlevel manually using the the init command. As an example, your machine will restart if you set the runlevel to 6. Use the following init command to set the runlevel to 6.

init 6

Each available runlevel has it’s own directory under /etc staring with rc. Inside the folder is a file, or symlink which controls the respective service. When you add a new service to your operating system startup you are simply adding the services control file to one or more of the runlevel folders.

/etc/rc0.d
/etc/rc1.d
/etc/rc2.d
/etc/rc3.d
/etc/rc4.d
/etc/rc5.d

 

« Previous Page — Next Page »

Visit our advertisers

Search

Quick Poll

How often do you change the password for the computer(s) you use?
Results

Recent Posts

Visit our advertisers

Site Links
Blog
Rated Sites

JamesCoyle.net Limited