Note: Applies to CLI v2.

It’s usually necessary to apply an opinitated set of structural changes to a YAML document (manifest, cloud config, etc.) before submitting it to the CLI commands (bosh create-env, bosh deploy, etc.) for processing. Such changes could be an addition or removal of certain job properties, instance groups, changes to property values.

Note: Replacing values such as passwords and certificates is not considered a structural change. Refer to [CLI variable interpolation](cli-int.html) for details.

To get a final YAML document one can apply desired changes once and save the result; however, over time it may become harder or just tedious to reapply these changes if base document changes. Additionally if it’s necessary to have multiple slightly different changes on top of the base document for different teams existing editing tools may not be enough. To make such workflows easier you can encode a set of changes into one or more operations file.

A single operation represents a single change. An operations file is a YAML document that contains multiple operations that are to be applied serially to a different YAML document. Instead of storing all operations in a single file, they can be grouped logically into many operations files.

Several CLI commands such as create-env, deploy and interpolate allow to provide operations files via --ops-file flag to be applied before processing the document.


Example

Following is an operations file (replace-name.yml) with a single operation that replaces value of top level key name with a string other-cf:

- type: replace
  path: /name
  value: other-cf

Given base YAML document (base.yml):

name: my-cf

Result of applying above operations file to the base YAML document would be:

name: other-cf

That could be demonstrated with the help of bosh interpolate command whose purpose is to simply apply operations files to base document and print the result:

$ bosh interpolate base.yml --ops-file replace-name.yml

name: other-cf

Path syntax

Each operation acts on a location within a YAML document. Path represents a location. It’s important to note that path (location) does not represent what operation will be performed, just like lat & long do not represent what happens at a physical location.

Here are some path examples:

  • /: matches document root
  • /0: matches 0th item in the array at the root
  • /instance_groups/0: matches 0th instance group within instance_groups array
  • /instance_groups/name=zookeeper: matches instance group (hash) with a name key that has value zookeeper

All paths follow these rules:

  • Paths can have multiple components separated by a /

  • Paths always start at the root of the document with a /

  • String components typically refer to hash keys (ex: /key1)

    • Strings ending with ? refer to hash keys that may or may not exist
    • “optionality” carries over to the items to the right
  • Integer components refer to array indices (ex: /0, /-1)

  • - component refers to an imaginary index after last array index (ex: /-)

    • If there is an array of length 3 ([0,1,2]), then - would refer to 4th non-existent position
  • key=val component matches hashes within an array (ex: /key=val)

    • Values ending with ? refer to array items that may or may not exist

Path components without “optional” (?) annotation imply that referenced location must exist within a document. Operation will fail to be performed if that location is not found. “Optional” annotation can be used to signify indifference to the presense of referenced location, making it possible for operation either to ignore it (while removal) or create it lazily (while replacement). If a component in a path is annotated as optional, components following it will be considered optional implicitly.


Operations

There are currently two types of operations: replace and remove.

Replace operation can be used to append an item to an array of any length if last component of a path is a - (see above for details).

Following base YAML document is used with operations below:

key: 1

key2:
  nested:
    super_nested: 2
  other: 3

array: [4,5,6]

items:
- name: item7
- name: item8
- name: item8

Hash

- type: replace
  path: /key
  value: 10
  • sets key to 10
- type: replace
  path: /key_not_there
  value: 10
  • errors because key_not_there is expected (does not have ?)
- type: replace
  path: /new_key?
  value: 10
  • creates new_key because it ends with ? and sets it to 10
- type: replace
  path: /key2/nested/super_nested
  value: 10
  • requires that key2 and nested hashes exist
  • sets super_nested to 10
- type: replace
  path: /key2/nested?/another_nested/super_nested
  value: 10
  • requires that key2 hash exists
  • allows nested, another_nested and super_nested not to exist because ? carries over to nested keys
  • creates another_nested and super_nested before setting super_nested to 10, resulting in:
  ...
  key2:
    nested:
      another_nested:
        super_nested: 10
      super_nested: 2
    other: 3

Array

- type: replace
  path: /array/0
  value: 10
  • requires array to exist and be an array
  • replaces 0th item in array array with 10
- type: replace
  path: /array/-
  value: 10
  • requires array to exist and be an array
  • appends 10 to the end of array
- type: replace
  path: /array2?/-
  value: 10
  • creates array2 array since it does not exist
  • appends 10 to the end of array2

Arrays of hashes

- type: replace
  path: /items/name=item7/count
  value: 10
  • finds array item with matching key name with value item7
  • adds count key as a sibling of name, resulting in:
  ...
  items:
  - name: item7
    count: 10
  - name: item8
- type: replace
  path: /items/name=item8/count
  value: 10
  • errors because there are two values that have item8 as their name
- type: replace
  path: /items/name=item9?/count
  value: 10
  • appends array item with matching key name with value item9 because values ends with ? and item does not exist
  • creates count and sets it to 10 within created array item, resulting in:
  ...
  items:
  - name: item7
  - name: item8
  - name: item8
  - name: item9
    count: 10

Next: CLI Variable Interpolation

Previous: CLI Environments


Contribute changes to this page