UDPATED 2016-2-29: cf cli v6.16.0+ includes easy to use commands. Blog post updated to use them.

Service brokers are a plug'n'play extension for any Cloud Foundry - hosted or private - to bring in unlimited external services - data, messaging, microservices, and more. Until now, only an administrator could register and update service brokers. This meant a normal user could not manage their own service brokers for their own team. This is no longer a limitation!

Private service brokers are available to all! Self-service is a good thing.

The cf CLI v6.16.0+ has extended its cf create-service-broker (and ilk) commands to register brokers for a Space. Any user can now register service brokers for a single Space.

The steps involved are:

  1. Run a simple service broker
  2. Register access to the broker in each space where it is required
  3. Provision and bind to it!

Running a broker

A Cloud Foundry Service Broker is any HTTP API that supports the 5+ API endpoints https://docs.cloudfoundry.org/services/api.html

Matt Stine has a simple demo broker that is a Spring app https://github.com/cloudfoundry-community/haash-broker

In your Cloud Foundry, create a space to run your broker:

cf create-space demo-broker  
cf target -s demo-broker  

Deploy the broker app:

git clone https://github.com/cloudfoundry-community/haash-broker  
cd haash-broker  

Matt's app needs some configuration to give it random service IDs:

rm src/main/resources/import.sql  
serviceid=$RANDOM  
planid=$RANDOM  
cat > src/main/resources/import.sql << EOF  
insert into services (id, name, description, bindable) values ('$serviceid', 'haash-broker-$RANDOM', 'HaaSh - HashMap as a Service', true);  
insert into plans (id, name, description, service_id) values ('$planid', 'basic', 'Basic Plan', '$serviceid');  
EOF  

Now we can build and deploy the broker API:

./gradlew assemble
cf push --random-route  

Our broker API application is now running. To get its randomly created URL:

$ cf apps
Getting apps in org starkandwayne / space demo-broker as drnic@starkandwayne.com...  
OK

name           requested state   instances   memory   disk   urls  
haash-broker   started           1/1         512M     1G     haash-broker-tenurial-fibreglass.cfapps.io  

Store it in a variable for safe keeping (UPDATE 25 May 2017: grep routes: instead of grep urls:)

broker_url=$(cf app haash-broker | grep routes: | awk '{print $2}')  

The hard-coded demo credentials are: warreng & natedogg.

To test out our broker:

curl warreng:natedogg@${broker_url}/v2/catalog  

Returns a JSON catalog of services available:

{
  "services": [
    {
      "id": "28716",
      "name": "drnic",
      "plans": [
        {
          "id": "24709",
          "name": "basic",
          "description": "Basic Plan"
        }
      ],

Register and enable access

The broker is now running. In real life, anyone could have deployed the broker, to anywhere. It could be deployed via Cloud Foundry, it could be deployed via BOSH, it could be running locally on your laptop and made public via https://ngrok.com/

It is the app developers who register the broker for their private access.

They can do this if they are given the basic auth credentials for the broker.

The app developers will now create a space where they want to access the broker:

cf create-space demo-apps  
cf target -s demo-apps  

To register the private broker into this space we use the new --space-scoped flag:

$ cf create-service-broker haash-broker  \
  warreng natedogg https://${broker_url} \
  --space-scoped

The service broker is now registered and usable.

Displaying the marketplace will show all the shared brokers, plus your new haash-broker:

$ cf marketplace
...
haash-broker-18079  basic  HaaSh - HashMap as a Service  

You can now create and bind service instances, and service keys:

cf create-service haash-broker basic my-haash-db  
cf create-service-key my-haash-db my-keys  
cf service-key my-haash-db my-keys  

This confirms that the app developer is now able to create service instances and bind to them/create service keys. Lovely!

The output will look like:

{
 "password": "natedogg",
 "uri": "http://haash-broker-tenurial-fibreglass.cfapps.io/HaaSh/4fa2b2d6-733e-425a-a878-d1034a2fc45f",
 "username": "warreng"
}

Updating service broker catalog

If the backend service broker you are using ever updates its catalog of services and plans then each developer space will need to ask Cloud Foundry to update the catalog/marketplace.

Once a broker as been registered as a private (space-scoped) broker it can be updated by similarly authorized users:

$ cf update-service-broker haash-broker warreng natedogg https://${broker_url}
Updating service broker haash-broker as drnic@starkandwayne.com...  
OK  

Multiple registrations are not possible

The current implementation of private brokers only allows a single registration of each unique broker URL.

I'm not sure why; but to help people find this behaviour via Google here's the error you get if you try to register the broker in multiple spaces:

{
   "code": 270003,
   "description": "The service broker url is taken: https://haash-broker-tenurial-fibreglass.cfapps.io",
   "error_code": "CF-ServiceBrokerUrlTaken"
}

I tried offering multiple routes, but then I get validation errors for: reuse of same broker name, reuse of service name, and reuse of same service ID/plan IDs. No work around.

I believe it is desirable to reuse the same broker across multiple spaces - for example your application code moving through a CI pipeline will be deployed into multiple spaces, but will want to use the same backend private broker.

There is an unscheduled/icebox ticket to allow private brokers across an entire organization. This story is not scheduled to be worked on as a part of the "private brokers" epic.

Summary

Private service broker registration and access is now a thing!! Upgrade to v228+ and enjoy the happiness of not having to "ask an admin" to do it.

Platforms are for self-service. Private broker registration is self-service. It is lovely.

Thanks to the CF CAPI team for working on this feature set!