This is a hands on introduction to habitat.sh. It is a relatively new tool in the world of config management and aims to simplify many aspects of packaging / deploying and operating any kind of distributed systems regardless of the intended deploy target.

Some of the features that I have found particularly useful are:

  • Simple / sane and reproducible approach to packaging.
  • Supervision and handling lifecycles of processes.
  • Service discovery including propagation of configuration to consuming services.

In this brief tour we will see some basic commands needed to build and run services packaged with habitat, without going into a lot of detail.

Setup

Lets start by cloning Stark & Wayne's habitat-plans repo.

$ git clone https://github.com/starkandwayne/habitat-plans.git
$ cd habitat-plans
$ ls
LICENSE         README.md       ci          mysql           postgresql      redis           scripts         shield          shield-agent    wordpress       wordpress-proxy  

Here we can see the list of software that S&W engineers are working on to create 'Enterprise Ready' habitat plans.

We will use the hab cli to build and run these services so now would be a good time to install habitat.

$ hab --version
hab 0.21.0/20170421000356  

Since we are building the plans under the starkandwayne origin we will need to generate an origin key for starkandwayne. Every package in Habitat belongs to an origin, and is cryptographically signed with that origin's private key.

$ hab origin key generate starkandwayne
» Generating origin key for starkandwayne
★ Generated origin key pair starkandwayne-20170424121434.

Since I will be demonstrating the workflow in a Mac OS environment you will also need Docker for Mac installed.

Building a package

Once your environment is set up you can enter the habitat studio (a shell environment setup to quickly iterate on your habitat plans). On a Mac this will download and start a docker container. On Linux this will simply chroot

$ HAB_ORIGIN=starkandwayne hab studio enter
(...)
[0][default:/src:0]#

A habitat package is defined via a plan.sh file. Lets take a look at the one for postgresql.

[0][default:/src:0]# ls postgresql
config  default.toml  hooks  plan.sh  tests  
[1][default:/src:0]# less postgresql/plan.sh
(...)

You will see a bunch of variables:

pkg_name=postgresql  
pkg_version=9.6.1  
pkg_origin=starkandwayne  
pkg_description="PostgreSQL is a powerful, open source object-relational database system."  
pkg_upstream_url="https://www.postgresql.org/"  
pkg_license=('PostgreSQL')  
pkg_source=https://ftp.postgresql.org/pub/source/v${pkg_version}/${pkg_name}-${pkg_version}.tar.bz2  
(...)

And further down some hooks that start with do_*

do_build() {  
        # ld manpage: "If -rpath is not used when linking an ELF
        # executable, the contents of the environment variable LD_RUN_PATH
        # will be used if it is defined"
        ./configure --disable-rpath \
              --with-openssl \
              --prefix="$pkg_prefix" \
              --with-uuid=ossp \
              --with-includes="$LD_INCLUDE_PATH" \
              --with-libraries="$LD_LIBRARY_PATH" \
              --sysconfdir="$pkg_svc_config_path" \
              --localstatedir="$pkg_svc_var_path"
        make world
}

do_install() {  
  make install-world
}

These functions and variables instruct habitat on how to create the starkandwayne/postgresql package. Let's build it now:

[2][default:/src:0]# build postgresql

This will take a while. The Postgresql source is downloaded and compiled according to the plan.sh. Eventually, you will see in the output that the build has completed:

(...)
   postgresql: hab-plan-build cleanup
   postgresql:
   postgresql: Source Path: /hab/cache/src/postgresql-9.6.1
   postgresql: Installed Path: /hab/pkgs/starkandwayne/postgresql/9.6.1/20170424121553
   postgresql: Artifact: /src/results/starkandwayne-postgresql-9.6.1-20170424121553-x86_64-linux.hart
   postgresql: Build Report: /src/results/last_build.env
   postgresql: SHA256 Checksum: 6187c711bd1f4471722053754b068f8f6de048612ca2b9a00aaec08f2ce6eb17
   postgresql: Blake2b Checksum: ae297ffaa007478eea6fba9676c2170e0af6dbb025dfdb61cf9512178a636fb2
   postgresql:
   postgresql: I love it when a plan.sh comes together.
   postgresql:
   postgresql: Build time: 5m23s

The resulting *.hart file (which is the resulting artifact) will reside under the results/ directory. This directory will also be present on your host machine because the directory you were in got mounted under /src when you typed hab studio enter.

[3][default:/src:0]# ls results
last_build.env  starkandwayne-postgresql-9.6.1-20170424121553-x86_64-linux.hart  

Habitat has also already installed your built plan into the studio under /hab/pkgs/...

[4][default:/src:0]# find /hab/pkgs/starkandwayne -type d -maxdepth 4
/hab/pkgs/starkandwayne
/hab/pkgs/starkandwayne/postgresql
/hab/pkgs/starkandwayne/postgresql/9.6.1
/hab/pkgs/starkandwayne/postgresql/9.6.1/20170424121553
/hab/pkgs/starkandwayne/postgresql/9.6.1/20170424121553/share
/hab/pkgs/starkandwayne/postgresql/9.6.1/20170424121553/lib
/hab/pkgs/starkandwayne/postgresql/9.6.1/20170424121553/config
/hab/pkgs/starkandwayne/postgresql/9.6.1/20170424121553/bin
/hab/pkgs/starkandwayne/postgresql/9.6.1/20170424121553/include
/hab/pkgs/starkandwayne/postgresql/9.6.1/20170424121553/hooks

Running your built service

To start your built package in the habitat studio you can simply run:

[5][default:/src:0]# hab start starkandwayne/postgresql
hab-sup(MR): Butterfly Member ID 67642576ef154e76b8160c83c605b475  
hab-sup(SR): Adding starkandwayne/postgresql/9.6.1/20170424122104  
hab-sup(MR): Starting butterfly on 0.0.0.0:9638  
hab-sup(MR): Starting http-gateway on 0.0.0.0:9631  
postgresql.default(SR): Initializing  
postgresql.default(SV): Starting process as user=hab, group=hab  
postgresql.default(O): Starting PostgreSQL  
postgresql.default(O): 2017-04-24 12:45:35.587 GMT: FATAL:  could not open shared memory segment "/PostgreSQL.1733059837": Permission denied  
postgresql.default(O): 2017-04-24 12:45:35.597 GMT: LOG:  database system is shut down  
hab-sup(SV): postgresql.default - process 15914 died with exit code 1  
hab-sup(SV): postgresql.default - Service exited  

Due to the constraints of the studio context this package actually won't start the way we want it (Permission denied). But that doesn't really matter! Since I am introducing you to a possible mac workflow you will need to export your packages to docker anyway.

[6][default:/src:0]#  hab pkg export docker starkandwayne/postgresql
(...)

Once we have exported the package we can exit the studio and run it in docker on our host machine:

[7][default:/src:0]# exit
$ docker images
REPOSITORY                                  TAG                    IMAGE ID            CREATED              SIZE  
starkandwayne/postgresql                    9.6.1-20170424122104   938e89b8286b        About a minute ago   429 MB  
starkandwayne/postgresql                    latest                 938e89b8286b        About a minute ago   429 MB  
$
$ docker run -p 5432:5432 starkandwayne/postgresql
hab-sup(MR): Butterfly Member ID eb68d2c1489a4c8084ac824311702750  
hab-sup(SR): Adding starkandwayne/postgresql/9.6.1/20170424122104  
hab-sup(MR): Starting butterfly on 0.0.0.0:9638  
hab-sup(MR): Starting http-gateway on 0.0.0.0:9631  
hab-sup(SC): Updated pwfile 30087c89f3dfc2b2179a2b567a67f094cf1cd8b1c6e3cc1473aef95fe65de533  
hab-sup(SC): Updated postgresql.conf 3ad4f841f942d1ae606ad89a54cb0ebe8a4313a1af4f1a8958c50b3aedaeb6ef  
hab-sup(SC): Updated pg_hba.conf cc1489fff1c05e7b547fac8018dbe22d0327a8ea6a288a6604a1b6c8232619e1  
postgresql.default(SR): Initializing  
(...)

You can test connecting to postgres from another terminal:

$ cat postgresql/default.toml | egrep '(name|password)'
name = 'admin'  
password = 'admin'  
$ PGPASSWORD=admin psql -h localhost -U admin postgres
psql (9.6.2, server 9.6.1)  
Type "help" for help.

postgres=# \q  

Conclusion

This was a very quick introduction of how to build, export and run a PostgreSQL service packaged via habitat.

To learn more about habitat please go through the tutorial on the habitat.sh website. It does a good job of explaining the basic concepts while walking you through the creation of your own plan.
The docs are also an excellent point of reference.

Also since Stark & Wayne have begun contributing plans to the habitat community you can keep following our blog for more habitat related content including workflow tips / best practices and other topics.