Deploy a CentOS Server Customized with Cloud-Init

Table of Contents

Introduction

Cloud-init allows you to customize a new server installation during its deployment using data supplied in YAML configuration files. This tutorial will briefly introduce one method of utilizing cloud-init when provisioning servers at ProfitBricks.

A quick rundown of the procedure we will follow is:

  1. Create the customized meta-data and user-data files.
  2. Use genisoimage to generate a config.iso containing those files.
  3. Upload config.iso and a CentOS 7 GenericCloud OS image to ProfitBricks.
  4. Provision a new server from those images.

This process is based around the NoCloud data source.

Prepare a Config ISO

For this tutorial, you should have access to a server where you can install the genisoimage package. On an existing CentOS 7 server this can be done by running:

sudo yum install genisoimage

Which should result in output similar to:

$ sudo yum install genisoimage

Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * base: mirrors.sonic.net
 * epel: mirror.chpc.utah.edu
 * extras: repos.lax.quadranet.com
 * updates: centos.mirror.constant.com
Resolving Dependencies
--> Running transaction check
---> Package genisoimage.x86_64 0:1.1.11-23.el7 will be installed
--> Processing Dependency: libusal = 1.1.11-23.el7 for package: genisoimage-1.1.11-23.el7.x86_64
--> Processing Dependency: libusal.so.0()(64bit) for package: genisoimage-1.1.11-23.el7.x86_64
--> Processing Dependency: librols.so.0()(64bit) for package: genisoimage-1.1.11-23.el7.x86_64
--> Running transaction check
---> Package libusal.x86_64 0:1.1.11-23.el7 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

====
Package                          Arch                        Version                             Repository                 Size
====
Installing:
 genisoimage                      x86_64                      1.1.11-23.el7                       base                      298 k
Installing for dependencies:
 libusal                          x86_64                      1.1.11-23.el7                       base                      135 k

Transaction Summary

====
Install  1 Package (+1 Dependent package)

Total download size: 433 k
Installed size: 1.5 M
Is this ok [y/d/N]: y
Downloading packages:
(1/2): libusal-1.1.11-23.el7.x86_64.rpm                                                                    | 135 kB  00:00:00
(2/2): genisoimage-1.1.11-23.el7.x86_64.rpm                                                                | 298 kB  00:00:00
----------------------------------------------------------------------------------------------------------------------------------
Total                                                                                             1.0 MB/s | 433 kB  00:00:00
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Installing : libusal-1.1.11-23.el7.x86_64                                                                                   1/2
  Installing : genisoimage-1.1.11-23.el7.x86_64                                                                               2/2
  Verifying  : libusal-1.1.11-23.el7.x86_64                                                                                   1/2
  Verifying  : genisoimage-1.1.11-23.el7.x86_64                                                                               2/2

Installed:
  genisoimage.x86_64 0:1.1.11-23.el7

Dependency Installed:
  libusal.x86_64 0:1.1.11-23.el7

Complete!

With that completed successfully, we can move into creating the two configuration files.

Create YAML Configuration Files

There are two configuration files that we need to create. I'm going to create a directory called cloudinit and place the meta-data and user-data files in there.

mkdir cloudinit
cd cloudinit

Create meta-data

The meta-data file should contain information specific to the cloud provider. In this case we will create a very basic meta-data file containing just two lines.

$ cat <<EOF > meta-data
instance-id: iid-local01;
local-hostname: node01
EOF

The value set for local-hostname will be used to set the hostname on the newly created server.

Create user-data

The user-data file can contain a number of different directives to help you customize your cloud server. For this tutorial, we will just introduce a couple of them. First we will create the file and then we can discuss the contents a bit.

$ cat <<EOF > user-data
#cloud-config

package_upgrade: true

packages:
  - git

users:
  - name: yourusername
    gecos: Your Username
    lock-passwd: false
    sudo: ALL=(ALL) NOPASSWD:ALL
    ssh_authorized_keys:
      - ssh-rsa AAAAB3NzaC1y...
EOF

The first line #cloud-config is required to indicate that this is a configuration file for cloud-init.

The package_upgrade: true line instructs the server to immediately update the OS and installed packages when the server is created.

The packages: section can be used to indicate which additional packages you would like to have installed. In this case, we've asked for just one, git, to be installed.

The users: section allows us to add some users to the system. You will want to customize the information here using your favorite text editor. Be sure to provide a valid SSH public key so that you can access the server once it is deployed.

Build the ISO

Now that the meta-data and user-data files are ready, lets create the config.iso using genisoimage.

$ genisoimage -output config.iso -volid cidata -joliet -rock user-data meta-data

Note: Do NOT change the -volid cidata as the filesystem volume label must be set to cidata.

You should end up with a small (~400k) config.iso file.

Upload the ISO

The config.iso needs to be uploaded to the ProfitBricks regional FTP server under iso-images in order to make it accessible in the DCD and via the Cloud API. If you are going to use this config.iso image in the Las Vegas, NV (USA) location, you could upload it with this curl command:

$ curl --ftp-ssl -u user@domain.com --upload-file config.iso ftp://ftp-las.profitbricks.com/iso-images/

Note: You'll need to replace user@domain.com with your valid ProfitBricks username. You will be prompted to enter your password.

The other FTP regional locations currently available are:

Frankfurt (Germany): ftp-fra.profitbricks.com/iso-images
Karlsruhe (Germany): ftp-fkb.profitbricks.com/iso-images
Newark, NJ (USA): ftp-ewr.profitbricks.com/iso-images

The upload should complete quickly, as the file is not very large. You should receive an e-mail indicating that your image has been uploaded, processed, and is ready for use.

Upload a CentOS 7 GenericCloud Image

Now we need to get a cloud-init enabled CentOS 7 "GenericCloud" image and upload it to ProfitBricks. One place you can find a list of these is on cloud.centos.org. If you want the newest one available, then grab:

CentOS-7-x86_64-GenericCloud.qcow2

There are lots of ways to download and then upload this file into your ProfitBricks account. You might pull it down to your workstation, uncompress it if necessary, and then upload it using software of your choice.

If you'd like to complete the process from the server with genisoimage on it, then a fairly efficient method from the command line using curl would be:

$ curl -OL http://cloud.centos.org/centos/7/images/CentOS-7-x86_64-GenericCloud.qcow2.xz
% Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                             Dload  Upload   Total   Spent    Left  Speed
100  239M  100  239M    0     0  23.1M      0  0:00:10  0:00:10 --:--:-- 25.6M

Now uncompress the xz archive using unxz:

unxz CentOS-7-x86_64-GenericCloud.qcow2.xz

Upload the CentOS-7-x86_64-GenericCloud.qcow2 file to the same ProfitBricks regional location you uploaded config.iso to. Please Note: The difference this time is that it needs to be placed in hdd-images.

$ curl --ftp-ssl -u user@domain.com --upload-file CentOS-7-x86_64-GenericCloud.qcow2 ftp://ftp-las.profitbricks.com/hdd-images/

Note: Just like earlier with the config.iso upload you'll need to replace user@domain.com with your valid ProfitBricks username. You will be prompted to enter your password.

The other FTP regional locations currently available are:

Frankfurt (Germany): ftp-fra.profitbricks.com/hdd-images
Karlsruhe (Germany): ftp-fkb.profitbricks.com/hdd-images
Newark, NJ (USA): ftp-ewr.profitbricks.com/hdd-images

This upload will take longer, as the uncompressed file is around 850MB. You should receive an e-mail indicating that your image has been uploaded, processed, and is ready for use.

Create a Server Using the Uploaded Images

Okay - now that we have the ISO and HDD images uploaded, we can go ahead and use them to provision a server.

First we should adjust the properties of our newly uploaded images using the Image Manager in the DCD.

Notice that I set the OS Type to Data / Other OS and provided a Description.

Cloud-Init config.iso image properties

For this image I set the OS Type to Linux, checked the box to indicate the image is Bootable, and provided a Description. Don't forget to press the Save button!

Cloud-Init GenericCloud image properties

Create a new "Composite Instance" and give your server a Name. Adjust the CPU, RAM, LAN, and Storage properties to match your preferences.

Cloud-Init Composite Instance

Select the CentOS-7-x86_64-GenericCloud.qcow2 image from the Image -> Own Images menu.

On the Storage tab of the server details, add a CD-ROM and select the config.iso image from the Image -> Own Images menu. It should NOT be the boot device. We want the server to boot from the HDD (or SSD).

Cloud-Init Add CD-ROM

Go ahead and provision the server.

Test

Because we created a new user and provided a public SSH key for that user, we should be able to SSH into the newly created server. Verify the IP address that was assigned to your new server and then test the connection.

Cloud-Init Verify Network

You should be able to connect by providing the private key that corresponds to the public key that was entered in user-data.

$ ssh -i your_private_key.key yourusername@the.server.ip.address
The authenticity of host 'x.x.x.x (x.x.x.x)' can't be established.
RSA key fingerprint is x.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'x.x.x.x' (RSA) to the list of known hosts.

Once you are connected, you should be able to use sudo as your user and set a root password or perform other administrative functions. We can verify that the updates were applied and the git package was installed by checking the contents of /var/log/yum.log. Here is some truncated example output:

$ sudo cat /var/log/yum.log
Oct 05 20:20:58 Updated: systemd-libs-219-42.el7_4.1.x86_64
Oct 05 20:20:59 Updated: bash-4.2.46-29.el7_4.x86_64
Oct 05 20:20:59 Updated: openssh-7.4p1-12.el7_4.x86_64
Oct 05 20:21:09 Updated: selinux-policy-3.13.1-166.el7_4.4.noarch
...
Oct 05 20:21:25 Installed: kernel-3.10.0-693.2.2.el7.x86_64
Oct 05 20:21:25 Updated: nss-tools-3.28.4-12.el7_4.x86_64
Oct 05 20:21:38 Updated: selinux-policy-targeted-3.13.1-166.el7_4.4.noarch
Oct 05 20:21:38 Updated: openssh-clients-7.4p1-12.el7_4.x86_64
Oct 05 20:21:38 Updated: sudo-1.8.19p2-11.el7_4.x86_64
Oct 05 20:21:38 Updated: ncurses-5.9-14.20130511.el7_4.x86_64
Oct 05 20:21:38 Updated: python-perf-3.10.0-693.2.2.el7.x86_64
...
Oct 05 20:22:38 Installed: perl-Git-1.8.3.1-12.el7_4.noarch
Oct 05 20:22:39 Installed: git-1.8.3.1-12.el7_4.x86_64

There is also a considerable amount of debug information contained in /var/log/cloud-init.log.

Conclusion

This tutorial briefly covered one very specific method for deploying new CentOS 7 server and customizing it with cloud-init at ProfitBricks. Please let me know if you have questions about the process or run into any problems getting this to work properly.

Additional information about Cloud-init is available at Read the Docs. You may be especially interested in reviewing some of the YAML configuration examples.