cf/239

The cf-release v239 was released on July 13, 2016.

IMPORTANT - Known issue: The WebDAV blobstore and Cloud Controller API / Clock / Worker jobs are unable to start after a VM restart because creation of the run directory for each process, /var/vcap/data/sys/run/*, was moved to the bosh pre-start script. The jobs are unable to start because /var/vcap/data/sys/run is mounted on a temporary file-system and the bosh pre-start script is not executed on VM restart, only deployment. A fix is in the pipeline for CF-240. To workaround this issue, operators can do a bosh deploy, which will recognize the failing jobs and properly create the run directory. - In an effort to not run processes as a privileged user, the WebDAV blobstore must now run on unprivileged ports. By default, internal access has been moved to port 4443 and the external access to port 8080. As the WebDAV blobstore is a SPOF, internal components using the blobstore will not be able to reach the blobstore until all processes have been reconfigured to use the new internal port. This will cause limited downtime in the sense that anything needing to reach the blobstore will fail until the deployment is complete, including app pushes, app restaging, and app restarting. This will happen automatically for users of CF manifest generation scripts. See Job Spec Changes. - Cloud Foundry now defaults to run containers on Diego in unprivileged mode. One known incompatibility is running applications that use FUSE file system support. See Job Spec Changes for instructions on how to continue running containers in privileged mode. - The noaa library for connecting to the firehose has a number of reliability improvements. If you use it, it is recommended that you upgrade to the latest version.

Contents: - CC and Service Broker APIs - DEA-Warden-HM9000 Runtime - Buildpacks and Stacks - Identity - Routing - Loggregator - Internal Components - Job Spec Changes - Recommended BOSH Stemcell Versions - Recommended Versions of Additional Releases

CC and Service Broker APIs

CC API Version: 2.58.0

Service Broker API Version: 2.9

CAPI Release

• As an operator, I expect all CF processes to run as least privileged user details
• As a CF operator, I would like to be able to configure whether or not Diego / Garden creates privileged containers for LRPs and Tasks details
• Monit hangs when nfs is not available details
• As a CF user, I would like Diego to validate the SHA checksum of my droplet before running it details

Cloud Controller

• operator should be able to use a manifest property to seed a shared domain and associate it with a TCP router group on deploy of cf-release details
• operator should be able to use manifest property to configure reservable route port quota for initial deploys details
• As an admin, I expect to be able to disable access to /v2/apps/:guid/env and /v3/apps/:guid/env for all users with a feature flag details
• As an Operator, I would like files in my S3 blobstore to be encrypted at rest using SSE-S3 details
• app_stop, app_start are potentially not locking properly, especially when iterating over processes details
• Users can remove themselves from Orgs and Spaces details
• Authentication failures should not show as api errors in new relic details
• Docker apps should not be pushable by admin when diego_docker is turned off. details

TPS

• As a CF user, I expect to see an uptime of 0 and no container metrics for crashed app instances on Diego details

Pull Requests and Issues

• cloudfoundry/cloud_controller_ng#598: CloudController still chooses the DEA which is almost full to do app staging if the DEA number is less than 5 details
• cloudfoundry/cloud_controller_ng#602: Listing of user-provided service instances by organization GUID does not work with IN operator details

DEA-Warden-HM9000 Runtime

• dmesg has been locked down details

DEA

• Fix order of magnitude in CPU consumption reported by cf app details
• DEA is guaranteed to heartbeat during evacuation details
• Update ruby to version 2.3.1

Warden

• Warden containers’ dns_servers can be specified details
• Update ruby version to 2.3.1

HM9000

• when an evacuating heartbeat is received send a start message for each app instance details
• Start messages to Cloud Controller are over HTTPS
• Evacuator now sends start messages after creation details
• Sender now sends stop messages over http details
• All messages from HM9000 to Cloud Controller are over HTTPS

Buildpacks and Stacks

stacks

updated to 1.69.0 (from 1.67.0)

1.69.0

1.68.0

java-buildpack

updated to v3.8.1 (from v3.7.1)

v3.8.1

v3.8

python-buildpack

updated to v1.5.7 (from v1.5.6)

v1.5.7

Identity

No Changes

Routing

• Update router manifest properties, see below details
• Manifest generation will now set the property uaa.zones.internal.hostnames to ["uaa.service.cf.internal"] if no stub overrides that value. This is in support of routing components contacting UAA over its internal TLS port. details
• Fix issue where GoRouter was not sending logs to syslog. details
• Thanks to Jonty Wareing from the UK Government Digital Service, Gorouter now supports the PROXY protocol details
  • Warning: An issue was found with PROXY protocol support where, when enabled, the Gorouter is unable to accept concurrent connections. PROXY protocol support is disabled by default.

Loggregator

• The noaa library for connecting to the firehose has a number of reliability improvements. If you use it, it is recommended that you upgrade to the latest version.
• Expose Metron URL Through Bosh 2.0 Links details
• Fix the race condition in NOAA details
• Manage logs endpoint auth token lifecycle (was:cloudfoundry/noaa #14: Reconnection token failed) details
• Remove AZ property from Traffic Controller details

Internal Components

consul

No changes.

etcd and etcd_metrics_server

• etcd-release was bumped from v57 to v58. Significant changes:
  • Improved how operators configure etcd_metrics_server to work with a secure TLS etcd cluster. details

postgres

• All long-running processes in postgres job now run as vcap instead of root. details

nats and nats_stream_forwarder

No changes.

Job Spec Changes

• Added etcd_metrics_server.etcd.dns_suffix property to etcd_metrics_server job to support configuring the job to talk to the secure etcd server. details.
• blobstore.tls.port now defaults to 4443 and must be above 1024. When using WebDAV blobstore, the Cloud Controller must now be configured with the same port by adding :4443 to cc.buildpacks.webdav_config.private_endpoint, cc.droplets.webdav_config.private_endpoint, cc.packages.webdav_config.private_endpoint, and cc.resource_pool.webdav_config.private_endpoint.

• blobstore.port now defaults to 8080 and must be above 1024. This port should also be configured for the route_registrar running alongside the blobstore:

    route_registrar:
      routes:
      - name: blobstore
        port: 8080
  

• Added capi.nsync.diego_privileged_containers (default false) to run containers on Diego in unprivileged mode. Must be changed to true for FUSE file system support. details

• Added capi.stager.diego_privileged_containers (default false) to run containers on Diego in unprivileged mode. Must be changed to true for FUSE file system support. details

• Added router.enable_proxy (default false) to enable PROXY protocol support.

• Renamed router.debug_addr to router.debug_address.

• Added dea_next.dns_servers property to allow containers to have a defined set of dns servers different than the host.

Recommended BOSH Stemcell Versions

• AWS: light-bosh-stemcell-3262.2-aws-xen-hvm-ubuntu-trusty-go_agent
• vSphere: bosh-stemcell-3262.2-vsphere-esxi-ubuntu-trusty-go_agent
• OpenStack: N/A
• BOSH-Lite: bosh-stemcell-3262.2-warden-boshlite-ubuntu-trusty-go_agent

These are soft recommendations; several different versions of the stemcells are likely to work fine with this version of cf-release and the corresponding versions of the additional releases listed below.

Recommended Versions of Additional Releases

These versions are soft recommendations, as several different versions of these releases may work correctly with this version of cf-release. - Diego release v0.1480.0. Release notes for v0.1480.0 · v0.1479.0 · v0.1478.0 · v0.1477.0. - Garden-Linux release v0.339.0. Release notes for v0.339.0. - etcd release v60. Release notes for v60 · v59 · v58. - cflinuxfs2-rootfs release v1.18.0. Release notes for v1.18.0 · v1.17.0.

- name: "cf"
  version: "239"
  url: "https://bosh.io/d/github.com/cloudfoundry/cf-release?v=239"
  sha1: "4d1d000ccdf34918738420a710f295977790df28"

bosh upload-release --sha1 4d1d000ccdf34918738420a710f295977790df28 \
  "https://bosh.io/d/github.com/cloudfoundry/cf-release?v=239"