Category Archives: Cheat Sheets

LXC 2.x/ LXD Cheat Sheet

Tags :

Category : Cheat Sheets

Get Social!

linux_containers_logoHere are some of the most used commands for creating and managing an LXC/ LXD host and containers. It’s assumed that you have a working environment and a privileged SSH connection to the LXC server for issuing the commands.

Basics

Start and Stop a LXC Container

Starting out with the basics here – starting and stopping an LXC container.

lxc start [CONTAINER]
lxc stop [CONTAINER]
List Containers

Display a list of container details for started and stopped containers. The name field is what’s usually used in other commands to reference the specific container.

lxc list
Create Container from Image

There are further details below on managing images and remote image repository, which you’ll need when creating a new container.

This example will create a new container and start it using the Ubuntu 1604 template. Change [CONTAINER] to be the name of the new container.

lxc launch ubuntu:16.04 [CONTAINER]
Delete Container

Removing a container cannot be undone – beware!

lxc delete [CONTAINER]

Images

Linux Containers are created from templates or images that are stored locally or downloaded from remote servers.

List Image Repositories

Local and Remote LXC servers and remote image servers can be added to your LXC installation and can be used to download images from when required. Run the below command to see what sources you have.

lxc remote list
List available images

Images that have been downloaded, imported or cached are stored locally in the image repository. The output will list the image name, size and various other details.

lxc image list

Remote images that reside on an image repository or remote LXC server can also be listed. This is great for seeing what images are available when creating new containers. Change [REMOTE_NAME] to be the name of the image repository from the image list command. Note: you’ll need to keep the : symbol at the end.

lxc image list [REMOTE_NAME]:
Get image details

Further details can be obtained from an image file than what’s displayed with image list. The below command will detail all information known about the image. Replace [IMAGE_NAME] with a valid image name displayed in the image list command, such as ubuntu-xenial.

lxc image info [IMAGE_NAME]
Add a new Image Repository

There are various public image repositories that can be added to your LXC installation. LinuxContainers.org is a common one and hosts several distribution types. Replace [NAME] with the text name you’d like to give to the repository (it’s just an alias) and [HOST] with the address of the repository.

lxc remote add [NAME] [HOST]

For example

lxc remote add lxc-org images.linuxcontainers.org
Delete a local image

Replace [IMAGE_NAME] with the the alias or fingerprint of the image.

lxc image delete [IMAGE_NAME]
Create new Image from Running Container

You can create a new image from an existing container with a simple command however it’s important to ensure that the created template will contain everything that the running container contained – such as SSH keys, data, etc. It’s therefore important to ensure you clean up anything which may be sensitve before running this command.

lxc publish [CONTAINER] --alias [ALIAS]

You’ll need to change [CONTAINER] to your Linux container name and [ALIAS] to the name you’d like to use for your new image.

Configuration

All the below instructions will assume you’re referring to a container alias called [CONTAINER]. You’ll need to replace this, wherever it’s seen, with the name of the Linux Container you’re acting on.

And config command using set can be altered to use get to retrieve what the current setting is. If the get returns nothing then it means it has not been manually set and the default value will be used.

Auto Start Container

Set the container to start automatically when the LXC service starts – usually at host boot time. Use 1 to enable and 0 to disable.

lxc config set [CONTAINER] boot.autostart 1

You can also use boot.autostart.delay to set a delay in seconds after starting this container, before starting the next.

lxc config set [CONTAINER] boot.autostart.delay 30

Start up can be ordered using lxc.autostart.order to prioritise which containers are started first. Higher numbers are started first.

lxc config set [CONTAINER1] boot.autostart.order 10
lxc config set [CONTAINER2] boot.autostart.order 8
CPU Limits

See CPU Resource Limits for more information on constraining CPU resources.

dd Cheat Sheet

Tags :

Category : Cheat Sheets

Get Social!

dd is one of the most versatile IO tools available for Linux. It’s used in a variety of ways ranging from Disk Benchmarking through to creating SWAP files and copying downloaded disk images to physical disks.

dd takes the following common switches:

  • if is the input file name and location.
  • of is the name and location of the output file.
  • bs is the block size that will be used to read and/ or write the file. Increasing this can help with performance  or dictate how much data will be read or written.
  • count is the number of blocks that will be used.
  • seek is the number of blocks on the output file that will be skipped before writing any data.
  • skip is the number of blocks that will be skipped on the input file before starting to read data.
  • conv is a comma separated list of additional parameters that can be used. See the man dd for more information.

The below headings will list a few example uses of dd in a typical Linux environment.

Backup disk partition with dd

You can use dd to copy an entire disk partition to a virtual disk file. This can be useful for creating a backup or to clone the disk to another machine.

dd if=/dev/sda1 of=~/localdisk_sda1.img

You can use this method to read a CD-ROM, USB drive or Flash disk to a file in the same way – just make sure the device is inserted and point the if= part of the dd command to the relevant /dev/ device.

You could also compress the image as part of the process with gzip.

dd if=/dev/sda1 | gzip -c > ~/localdisk_sda1.img.gz

Restore disk partition with dd

Similar to the above command, you can use dd to replace a disk’s partition with a virtual disk file.

dd if=~/localdisk_sda1.img of=/dev/sda1

If you compressed the image then you can decompress it first all in one go:

gunzip -c ~/localdisk_sda1.img.gz | dd of=/dev/sda1

Create a fixed size file with dd

You can create a fixed size file with DD that will be created in the location you specify.

dd if=/dev/zero of=/root/test bs=1024 count=1

This will create a file in /root/test of 1024 bytes in size. Increase either bs or count to change the size of the file. The resulting size will be bs * count. You can also use shorhand sizes such as K, M and G with bs, for example bs=1G,

dd if=/dev/zero of=upload_test bs=file_size count=1

Create a SWAP file with dd

dd can be used to create a SWAP file that can be used as a SWAP device by your computer. This is often needed with smaller instances on Cloud providers such as AWS.

The starting point is the same as the above command to create a file with the size that you’d like to use for swap. See my other blog post for more info.

Split a file with dd

dd can be used to read just part of a file, given offset and length coordinates. The below example will skip the first 100 bytes of the file and output the proceeding 10 bytes (byte 101 – 111).

dd if=filetosplit of=partfile bs=1 count=10 skip=100

You could repeat this process to split a large file into multiple smaller files, to be able to email it for example.

dd if=filetosplit of=partfile1 bs=1 count=100
dd if=filetosplit of=partfile2 bs=1 count=100 skip=100
dd if=filetosplit of=partfile3 bs=1 count=100 skip=200

Merge multiple files with dd

You can merge multiple files into a single file with dd. Following on from the above split example, the below will rejoin the 3 file parts into a single file.

dd if=partfile1 of=joinedfile bs=1 count=100
dd if=partfile2 of=joinedfile bs=1 count=100 seek=100
dd if=partfile3 of=joinedfile bs=1 count=100 seek=200

Convert text to lower case with dd

You can use the conv switch with dd to transform ascii text from upper case to lower case and visa-versa. Using lcase and ucase in the conv switch will instruct dd to convert the text as it’s written.

The below example will convert all characters in the filetoconvert.txt. file to lower case.

dd if=filetoconvert.txt of=convertedfile.txt conv=lcase

 

Linux User Management Cheat Sheet

Tags :

Category : Cheat Sheets

Get Social!

This is my cheat sheet on Linux user administration covering functions such as adding, and removing users and assigning them to groups.

Add a new Linux user

Use the useradd command to add a new user.

useradd [USERNAME]

Change a users password

The user account is locked until you set a password with the passwd command.

passwd [USERNAME]

Add user to group – new user

If you are adding a new user, you can add it to a group in the same command. This command will create a user of [USERNAME] in group [GROUPNAME].

useradd -G [GROUPNAME] [USERNAME]

Add user to group – existing user

If the user already exists, you can add it to an existing group with the usermod command.

usermod -a -G [GROUPNAME] [USERNAME]

Delete a user

Run the userdel command to remove an existing Linux user.

userdel [USERNAME]

View existing users and groups

Run the below cat command to view existing Linux users. You sill see the user names and user IDs of all users on your server.

cat /etc/passwd | sort

Use this cat command to view the existing Linux groups.

cat /etc/group | sort

Change a users home directory location

You can change the users home directory with the usermod command.

usermod -d -m [NEW_DIRECTORY] [USERNAME]

Change a users UID

Use the usermod -u command to change the user ID of a user.

usermod -u [UID] [USER_NAME]

 

OpenSSL Certificate Cheat Sheet

Tags :

Category : Cheat Sheets

Get Social!

openssl-logoThese commands cover the basics of OpenSSL and are valid for either Windows or Linux with the exception that paths may need to be corrected for the respective platform.

Install OpenSSL

For windows http://www.openssl.org/related/binaries.html

For Ubuntu

sudo apt-get install openssl

Create Private Key

The last argument in the below line is the key length. This can be changed to 2048 or 4096 if required for better encryption.

openssl genrsa -des3 -out server.key 1024

Generate a CSR (Certificate Signing Request)

You will be asked for the details of the certificate such as domain name and address when running this command.

openssl req -new -key server.key -out server.csr

Remove Passphrase from Key

Some applications do not allow for the private key to have a passphrase. The below commands will remove the passphrase – be careful as it will mean the key is no longer protected and can be viewed by anyone with read access to the file.

openssl rsa -in server-with-passphrase.key -out server.key

Generating a Self-Signed Certificate

Once you have generated a key and CSR you will need to sign the request and generate the public certificate. If you do not have a certificate authority you can sign the certificate yourself. The below will generate a certificate which is valid for one year.

openssl x509 -req -days 365 -in server.csr -signkey server.key -out server.crt

Convert x509 to pem

openssl x509 -inform der -in server.crt -out server.pem

pkcs12 to pem – key only

Use the below command to extract only the key from a pkcs12 certificate.

openssl pkcs12 -nocerts -in c:\server.pfx -out c:\server-key.key

pkcs12 to pem – certificate only

Use the below command to extract only the public certificate from a pkcs12 certificate.

openssl pkcs12 -nokeys -in server.pfx -out server-cert.cer

Check a private key

You can check a private key with the below command.

openssl rsa -in privateKey.key -check

Check a certificate

Use the below command to check a certificate.

openssl x509 -in certificate.crt -text -noout

 

update-rc.d Cheat Sheet

Tags :

Category : Cheat Sheets

Get Social!

Linux penguinDebian and Ubuntu use the service command to control services and update-rc.d for adding and removing services from start up. Using the service command we can start, stop, restart and display all available services. With update-rc.d we can add and remove services and add them to the Ubuntu/ Debian start up scripts. As Linux operating systems have multiple states, or runlevels, you need to make sure you add any new services to the correct runlevels. For example, you would not want to start a web service application before starting networking.

See my post on runlevels for more information about runlevels in Linux..

What services are available for startup?

Use the status-all switch to list all services which are registered with the OS and issues them a status command. You will then get one of the following displayed next to each service:

  • [ + ] – Services with this sign are currently running.
  • [ – ] – Services with this sign are not currently running..
  • [ ? ] – Services that do not have a status switch.
service --status-all

Sample output:

service --status-all
 [ ? ] acpid
 [ - ] apparmor
 [ ? ] apport
 [ ? ] atd
 [ - ] bootlogd
 [ ? ] console-setup
 [ ? ] cron
 [ ? ] dbus
 [ ? ] dmesg
 [ ? ] dns-clean
 [ ? ] friendly-recovery
 [ ? ] glusterfs-server
 [ - ] grub-common
 [ ? ] gssd
 [ ? ] hostname
 [ ? ] hwclock
 [ ? ] hwclock-save
 [ ? ] idmapd
 [ ? ] irqbalance
 [ ? ] killprocs
 [ ? ] module-init-tools
 [ ? ] network-interface
 [ ? ] network-interface-container
 [ ? ] network-interface-security
 [ ? ] networking
 [ ? ] ondemand
 [ ? ] passwd
 [ ? ] plymouth
 [ ? ] plymouth-log
 [ ? ] plymouth-ready
 [ ? ] plymouth-splash
 [ ? ] plymouth-stop
 [ ? ] plymouth-upstart-bridge
 [ ? ] portmap
 [ ? ] portmap-wait
 [ ? ] pppd-dns
 [ ? ] procps
 [ ? ] rc.local
 [ ? ] resolvconf
 [ ? ] rpcbind-boot
 [ - ] rsync
 [ ? ] rsyslog
 [ ? ] screen-cleanup
 [ ? ] sendsigs
 [ ? ] setvtrgb
 [ + ] ssh
 [ ? ] statd
 [ ? ] statd-mounting
 [ - ] stop-bootlogd
 [ - ] stop-bootlogd-single
 [ ? ] sudo
 [ ? ] udev
 [ ? ] udev-fallback-graphics
 [ ? ] udev-finish
 [ ? ] udevmonitor
 [ ? ] udevtrigger
 [ ? ] ufw
 [ ? ] umountfs
 [ ? ] umountnfs.sh
 [ ? ] umountroot
 [ - ] unattended-upgrades
 [ - ] urandom
 [ ? ] whoopsie

Start a service

Starting a service is done using the command service followed by the service name and the command to start the service.

service apache2 start

Stop a service

Use the stop keyword with service to stop a service.

service apache2 stop

Check the status of a service

Each service has a status, usually running or not running. Some services, such as network, may have a different output and output more information on the service.

service apache2 status

Remove a service

Use the remove keyword with update-rc.d to remove the service start up command for an application. You will need to use the -f switch if the applications /etc/init.d start up file exists.

update-rc.d -f apache2 remove

 Add a service

Adding a service to Ubuntu or Debian is done with the update-rc.d command. You can specify which runlevels to start and stop the new service or accept the defaults. The init.d file will be added to the relevent rc.d startup folders.

 update-rc.d apache2 defaults

Setting Start and Kill priority

If you need more control over when your service is asked to start and stop, you may need to set the start and kill (S and K) values.

For a given runlevel, you may have several services starting. For example, you may have apache2 and mysql both starting on runlevel 3. Ideally, you’d want mysql to start before apache2 and shutdown after apache2. In this case we need to give mysql the priority in startup, but apache2 the priority in shutdown.

When starting, the lower the number, the earlier it will start. A start priority of 10 will start before a priority of 20. When killing, it’s the opposite. A higher number will be killed before a lower number.

To set the start and kill priority we simply add them to the above update-rc.d command with the start priority first, followed by the kill priority.

update-rc.d apache2 defaults [START] [KILL]

The below command will start mysql first, then apache2. On shutdown, the kill will be the reverse of the start with apache2 being killed first and mysql second.

update-rc.d apache2 defaults 90 90
update-rc.d mysql defaults 10 10

Because, in our example, both start and kill priorities are the same we can shorted the command to just

update-rc.d apache2 defaults 90
update-rc.d mysql defaults 10

Manually set the RunLevel to Start and Kill a service

You can manually specify the Linux RunLevel that the system must be in to Start and Kill your service. See my other blog post for more information on RunLevels.

To extend the above example, we can specify exactly which RunLevel apache2 will be started and stopped.

update-rc.d apache2 start 10 2 3 4 5 . stop 90 0 1 6 .

apache2 will be started (as long as it isn’t already) when the system enters RunLevel 2, 3, 4 or 5 with a priority of 10. It will then be asked to stop when the system enters RunLevel 0, 1 or 6 with a priority of 90.

Let me know in the comments if you think anything is missing.

logrotate Cheat Sheet

Tags :

Category : Cheat Sheets

Get Social!

logrotate Most Linux applications create log files that need to be managed by either archiving or deleting old log files. This process is called log file rotation. The most common log rotation utility for Linux is conveniently called logrotate. logrotate is configured using the main configuration file, or the logrotate configuration folder. The configuration file usually holds the global configuration and the pointer to the configuration folder. In common Linux distributions such as Ubuntu and Debian, the logrotate configuration file can be found:

/etc/logrotate.conf

And the configuration folder can be found:

/etc/logrotate.d/

The configuration folder is usually where you would add new log file configurations which are unique for a particular log file, or set of log files. For example, if you were to add a new log rotation action for the Ubuntu package manager apt, you may use something like below:

vi /etc/logrotate.d/apt
/var/log/apt/term.log {
  rotate 12
  monthly
  compress
  missingok
  notifempty
}
/var/log/apt/history.log {
  rotate 12
  monthly
  compress
  missingok
  notifempty
}

logrotate runs each day by default, and is invoked using the daily cron job. The below subject detail the common components of creating a logrotate configuration file.

Create a new empty template

To create a new logrotate configuration file, you need to create a new file in /etc/logrotate.d/. You will then need to add a reference to the log files you wish to rotate. This can be directly to a single file or use pattern matching to match a group of log files. The below example matches all log files in /var/log/myapp/ which have a .log extension.

/var/log/myapp/*.log {

}

You will need to add further commands to this template before it becomes useful. Further commands will be added inside the { and } tags.

Rotate based on log file size

Use the size keyword to rotate the log file when it exceeded a given file size. The below example rotates a file when it reaches 10 KB. 

/var/log/myapp/*.log {
  size 10k
}

Rotate based on time (Month, Week or Day)

You can rotate logs using the monthly, weekly or daily keyword to create a new log based on duration. The keywords explain them selves, and they can be used in conjunction with the size keyword to rotate on which ever criteria is met first.

/var/log/myapp/*.log {
  size 10k
  weekly
}

Limit how many log files are kept after rotation by number

The rotate keyword allows you to specify how many old, rotated, log files are kept before logrotate deletes them. The rotate keyword requires an integer to specify how many old log files are kept.

/var/log/myapp/*.log {
  size 10k
  weekly
  rotate 8
}

The above example will keep 8 old log files.

Limit how many files are kept after rotation by date

You can specify how long to keep rotated files using the maxage keyword. Any rotated log files which are older than maxage will be deleted. The below example will keep rotated log files for 56 days.

/var/log/myapp/*.log {
  size 10k
  weekly
  maxage 56
}

Compress rotated log files

Log files which have been rotated can be compressed to save disk space. Gzip is used by default.

/var/log/myapp/*.log {
  size 10k
  weekly
  rotate 8
  compress
}

You can change the default gzip compression to another format by specifying the compresscmd command and a different executable to use. The below example changes the compression format to bzip2 for better file compression.

/var/log/myapp/*.log {
  size 10k
  weekly
  rotate 8
  compress
  compresscmd /bin/bzip2
}

Ignore missing log files

If a log file does not exist when logrotate is running then an error will be thrown. You can use the keyword missingok to avoid this scenario and instruct logrotate to ignore the log file if it does not exist.

/var/log/myapp/*.log {
  size 10k
  weekly
  rotate 8
  missingok
}

Continue writing to the same log file after rotation

Usually when a log file is rotated the log file is moved to a new location. Some applications may throw an error, and others may continue to write to the relocated file. The copytruncate keyword copies all the log in the file to a new file and then truncates the original file. This keeps the original log file in place and also allows rotation to continue.

/var/log/myapp/*.log {
  size 10k
  weekly
  rotate 8
  copytruncate
}

 

Let me know in the comments if you think anything is missing.

Next Page »

Visit our advertisers

Search

Quick Poll

Are you using Docker.io?
Results

Recent Posts

Visit our advertisers

Site Links
Blog
Rated Sites

JamesCoyle.net Limited