This topic describes how to build a CPI.

Distribution

CPIs are distributed as regular releases, typically with a release job called cpi and a few packages that provide compilation/runtime environment for that job if necessary (e.g. bosh-aws-cpi-release includes Ruby and bosh-warden-cpi-release includes golang). To qualify to be a CPI release, it must include a release job that has bin/cpi executable.

Both bosh-init and the Director expect to be configured with a CPI release to function properly. In the case of bosh-init, specified CPI release is unpacked and installed on the machine running bosh-init. For the Director, CPI release job is colocated on the same VM, so that the director release job can access it.

CPIs written in Ruby:

CPIs written in golang:


RPC

Since CPI is just an executable, following takes place for each CPI method call:

  1. Callee starts a new CPI executable OS process (shells out)
  2. Callee sends a single JSON request over STDIN
    • CPI optionally starts logging debugging information to STDERR
  3. CPI responds with a single JSON response over STDOUT
  4. CPI executable exits
  5. Callee parses and interprets JSON response ignoring process exit code

As a reference the Director’s implementation is in bosh_cpi’s external_cpi.rb and bosh-init implements it in cpi_cmd_runner.go.

Request

  • method [String]: Name of the CPI method. Example: create_vm.
  • arguments [Array]: Array of arguments that are specific to the CPI method.
  • context [Hash]: Additional information provided as a context of this execution. Specified for backwards compatibility and should be ignored.

Example:

{
    "method": "delete_disk",
    "arguments": ["vol-1b7fb8fd"],
    "context": { "director_uuid":"fefb87c8-38d1-46a5-4552-9749d6b1195c" }
}

Response

  • result [Null or simple values]: Single return value. It must be null if error is returned.
  • error [Null or hash]: Occurred error. It must be null if result is returned.
    • type [String]: Type of the error.
    • message [String]: Description of the error.
    • ok_to_retry [Boolean]: Indicates whether callee should try calling the method again without changing any of the arguments.
  • log [String]: Additional information that may be useful for auditing, debugging and understanding what actions CPI took while executing a method. Typically includes info and debug logs, error backtraces.

Example successful response from create_vm CPI call:

{
    "result": "i-384959",
    "error": null,
    "log": ""
}

Example error response from create_vm CPI call:

{
    "result": null,
    "error": {
        "type": "Bosh::Clouds::CloudError",
        "message": "Flavor `m1.2xlarge' not found",
        "ok_to_retry": false
    },
    "log": "Rescued error: 'Flavor `m1.2xlarge' not found'. Backtrace: ~/.bosh_init/ins..."
}

BOSH team maintains several CPIs written in Ruby. We currently take advantage of the bosh_cpi gem which provides Bosh::Cpi::Cli class that deserialized requests and serializes responses. If you are planning to write a CPI in Ruby, we recommend to take advantage of this gem.


Testing

There are two test suites each CPI is expected to pass before it’s considered to be production-ready:


Concurrency

The CPI is expected to handle multiple method calls concurrently (and in parallel) with a promise that arguments represent different IaaS resources. For example, multiple create_vm CPI method calls may be issued that all use the same stemcell cloud ID; however, attach_disk CPI method will never be called with the same VM cloud ID concurrently.

Note: Since each CPI method call is a separate OS process, simple locking techniques (Ruby’s Mutex.new for example) will not work.


Rate Limiting

Most CPIs have to deal with IaaS APIs that rate limit (e.g. OpenStack, AWS APIs). Currently it’s the responsibility of the CPI to handle rate-limiting errors, properly catch them, wait and retry actions that were interrupted. Given that there is no timeout around how long a CPI method can run, it’s all right to wait as long as necessary to resume making API calls. Though it’s suggested to log such information.


Debugging

It usually useful to get a detailed log of CPI requests and responses from the callee. To get a full debug log from bosh-init set BOSH_INIT_LOG_LEVEL=debug environment variable. When working with the Director you can find similar debug logs via bosh task X --debug command.


Next: CPI API v1


Contribute changes to this page