Setup CoreOS Cluster on ProfitBricks

Table of Contents

Introduction

CoreOS is beginning to catch on as a great host environment for Docker containers. It allows you to run your containers at scale, either within a local cluster or distributed across multiple datacenters. This tutorial will walk you through how to setup a 3-node CoreOS cluster at ProfitBricks. We will take you through your DCD setup and the few commands you need to run to spin up your first CoreOS environment.

Setup your Datacenter for CoreOS

You will want to log into your Data Center Designer and create a new Data Center. In our example we created a datacenter named Coreos.

1. From the palette add three Composite Instances into the diagramming area. This represents the CoreOS cluster your going to build.

Since we're simply doing a proof of concept we'll keep our initial spec low; however, define these to fit your end goal with this tutorial. Also, remember that ProfitBricks allows you to independently scale core and memory so you can start small and grow the cluster resources on an as needed basis.

In our test environment we're configuring the values to be:

| Setting  | Value  |
|:--|---|
| Cores  | 1  |
| RAM  | 2  |
| OS Image | Ubuntu-14.04-LTS-Server |

We'll go ahead and name each of the instances core001, core002, and core003. The Ubuntu image is used temporarily and will be removed once we have successfully bootstrapped CoreOS.

2. Add three additional HDD Storage objects to your datacenter.

You will be installing CoreOs on these second volumes so leave their image unassigned. They will show up as additional volumes within the instance.

3. Configure your networking.

In our test environment we create only two LANs:

LAN1 is public facing and should be assigned to NIC1 on all instances.

LAN2 is private and should be connected to NIC2 on all instances. This allows CoreOS to communicate across a private network. You will use the private IPs you assign in the CoreOS cloud-config.yaml file.

You should assign values to the following properties:

| Property | Value |
|:--|---|
| Name | *your choice, alphanumeric* |
| Primary IP | *use a private space here, e.g. 192.0.2.10* |
| DHCP | *unchecked* |

By getting into the habit of defining these properties at build time you make it easier to code against your datacenter in the future. You could go the extra mile and name your volumes, diagram connections, etc.

4. Go ahead and provision the datacenter.

Creating CoreOS Configuration

You will be authenticating with CoreOS using SSH keys. To do this you will need to generate a new key or use your existing one. You can follow our tutorial on how to create and setup SSH keys. For now, let's assume you have your keys generated. To create the configuration file that will be used to setup each CoreOS instance you will need to get the ssh-rsa string from your id_rsa.pub file. You can get this by doing the following (your path might change depending on where you keep this file):

cat .ssh/id_rsa.pub

Copy the entire string starting at ssh-rsa.

ssh-rsa AAAAB3NzaC... user@domain.com

You will now need to log into what will become your first CoreOS node.

ssh root@[public-ip]

Install CoreOS

You will need to download the following bash script.

curl https://raw.githubusercontent.com/coreos/init/master/bin/coreos-install > coreos-install.sh

You will use this in conjunction with the cloud-config.yaml file covered in the next section.

Create your cloud-config.yaml

You will first need to generate an etcd discovery URL. This is done by doing:

curl -w "\n" https://discovery.etcd.io/new

NOTE: Whenever building a new cluster you will need to generate an etcd token.

The return value will be your discovery URL.

Now, let's create your cloud-config.yaml file.

nano cloud-config.yaml

The first section of your configuration file should look like this. You will need to replace the ssh_authorized_keys value with the one from your id_rsa.pub file.

#cloud-config

ssh_authorized_keys:
  - ssh-rsa AAAAB3NzaC1yc2... user@domain.com

You will then need to set your hostname value:

hostname: core001

Finally, add in your configuration:

coreos:
 etcd:
  name: core001
  discovery: https://discovery.etcd.io/fe4e5d291f892f9f7dc9ce9edd9d61aa
  addr: 192.0.2.12:4001
  peer-addr: 192.0.2.12:7001
 fleet:
  public-ip: 162.254.26.142
 units:
  - name: etcd.service
    command: start
  - name: fleet.service
    command: start

write_files:
    -  path: /etc/systemd/network/eth1.network
       permissions: 0644
       owner: root
       content: |
         [Match]
         Name=eth1

         [Network]
         Address=162.254.26.142/24
         Gateway=162.254.26.1

         DNS=8.8.8.8

    -  path: /etc/systemd/network/eth0.network
       permissions: 0644
       owner: root
       content: |
         [Match]
         Name=eth0

         [Network]
         Address=192.0.2.12/24

Putting this all together we get a file that looks similar to this:

#cloud-config

ssh_authorized_keys:
  - ssh-rsa AAAAB3NzaC1yc2... user@domain.com


hostname: core001

coreos:
 etcd:
  name: core001
  discovery: https://discovery.etcd.io/fe4e5d291f892f9f7dc9ce9edd9d61aa
  addr: 192.0.2.12:4001
  peer-addr: 192.0.2.12:7001
 fleet:
  public-ip: 192.0.2.12
 units:
  - name: etcd.service
    command: start
  - name: fleet.service
    command: start

write_files:
    -  path: /etc/systemd/network/eth1.network
       permissions: 0644
       owner: root
       content: |
         [Match]
         Name=eth1

         [Network]
         Address=162.254.26.142/24
         Gateway=162.254.26.1

         DNS=8.8.8.8

    -  path: /etc/systemd/network/eth0.network
       permissions: 0644
       owner: root
       content: |
         [Match]
         Name=eth0

         [Network]
         Address=192.0.2.12/24

Run coreos-install.sh

Your next step is to run coreos-install.sh using the second volume you attached when creating the datacenter.

Get a list of your ProfitBricks volumes:

lsblk

You should see entry like this:

vdb    253:16   0  50G  0 disk

Change the permissions on the shell script and then run it:

chmod 700 coreos-install.sh
./coreos-install.sh -d /dev/vdb -C stable -c cloud-config.yaml

You should see the following something like this:

Downloading the signature for http://beta.release.core-os.net/amd64-usr/current/coreos_production_image.bin.bz2...
2014-11-03 23:44:25 URL:http://beta.release.core-os.net/amd64-usr/current/coreos_production_image.bin.bz2.sig [543/543] -> "/tmp/coreos-install.UDCp99urYm/coreos_production_image.bin.bz2.sig" [1]
Downloading, writing and verifying coreos_production_image.bin.bz2...
2014-11-03 23:44:59 URL:http://beta.release.core-os.net/amd64-usr/current/coreos_production_image.bin.bz2 [181793608/181793608] -> "-" [1]
gpg: Signature made Thu 16 Oct 2014 01:29:51 AM UTC using RSA key ID E5676EFC
gpg: key 93D2DCB4 marked as ultimately trusted
gpg: checking the trustdb
gpg: 3 marginal(s) needed, 1 complete(s) needed, PGP trust model
gpg: depth: 0  valid:   1  signed:   0  trust: 0-, 0q, 0n, 0m, 0f, 1u
gpg: Good signature from "CoreOS Buildbot (Offical Builds) <buildbot@coreos.com>"
Success! CoreOS beta current is installed on /dev/vdb

You don't need to reboot the server since that will happen in the next steps. At this point you should have a volume that has CoreOS installed on it. You will need to repeat the above steps taking care with your cloud-config.yaml values on your remaining nodes.

Update ProfitBricks Datacenter

Before you can begin to use CoreOS you will need to set the CoreOS volume as the boot volume and remove the other volume from being attached to the instance. This is all done through the DCD.

1. Log back into your Datacenter Designer.

2. Click the storage connection for the Ubuntu volume and delete it.

3. Click the server instance and then in Inspect click the Storages tab. Ensure the remaining volume is set as the boot device. This is the volume where you installed CoreOS.

4. Click Provision Data Center

5. Once the designer reports that all changes have been committed reboot the server via the DCD.

If you bring up the console from within the DCD you should see that the CoreOS node is listening on the private interface you defined in the configuration file.

Connect to your CoreOS Server

Once you can ping your public IP again you should be able to connect using the RSA key you created in the previous example.

You should now be able to simply connect via:

ssh -A core@[public_ip]

NOTE: You need to use the username 'core' versus 'root'.

You should now be able to spin up a container without any issue:

docker run -i -t ubuntu /bin/bash

Now that you have validated your CoreOS cluster you can return to the DCD and remove the Ubuntu volumes from your datacenter. You will not need those anymore.

Conclusion

You can read about the various values found in the cloud-config.yaml file at the CoreOS site.