GlusterFS storage mount in Proxmox

Proxmox 3.1 brings a new storage plugin; GlusterFS. Thanks to this storage technology we can use distributed and redundant network storage to drive OpenVZ containers, qemu disk images, backups, templates and iso’s – basically all the Proxmox storage types.

Proxmox 3.1 uses version 3.4 of the GlusterFS client tools and therefore a compatible GlusterFS server version is required. For the current version, please see this post for the latest PPA Ubuntu repository, and this post for setting up a 2 node GlusterFS server.

Adding a single GlusterFS share to Proxmox 3.1 is one of the easiest things you will do, providing the server is already set up. The trouble comes in when you are using GlusterFS in a redundant/ failover scenario as Proxmox only allows you to enter one GlusterFS server IP meaning that you are loosing all the benefits of having a redundant file system.

At this point it’s worth understanding something about the GlusterFS server setup. Let’s say you have two physical servers which replicate a single GlusterFS share. This gives you a level of redundancy as one server can fail without causing any issues. It also gives you load balancing but that is a separate point altogether. The client can then connect to one of these servers as it mounts the filesystem however because of the way GlusterFS works it needs access to both the GlusterFS servers. This is because the first connection the client makes is to one of the servers to get a list of servers available for the storage share it is going to mount. In our example here, there are two storage servers available and it is this list which is sent to the client. Then, as the mount point is used by the client, it can communicate with any server in the known list. So then, although using a single IP for a client is a single point of failure, it’s only a single point of failure on the initial communication when obtaining the list of servers available.

Add using Proxmox web GUI

To set up a single server mount point, click on the Storage tab which can be found on DataCenter. Click Add and then GlusterFS volume. You will then see the Add: GlusterFS Volume dialogue box.

Enter your GlusterFS server information into the dialogue box along with the other required fields described below.

• ID – the name to use for the storage mount point. This will be visible in your storage list.
• Server – the GlusterFS server IP or hostname.
• Volume name – the share name on the GlusterFS server.

Using the GlusterFS client you are able to specify multiple GlusterFS servers for a single volume. With the Proxmox web GUI, you can only add one IP address. To use multiple server IP addresses, create a mount point manually.