Skip to content

log_parser job from logsearch/14

Github source: 35eb45ac or master branch

Properties

elasticsearch

flush_size

Redis queue flush size

Default
100

host

IP / DNS of elasticsearch http endpoint

port

Port of elasticsearch http endpoint

Default
9200

logstash_parser

debug

Debug level logging

Default
false

filters

A list of URL downloads (tar, tar.gz, zip)

Default
[]

idle_flush_time

How frequently to flush events if the output queue is not full.

message_max_size

Maximum log message length. Anything larger is truncated (TODO: move this to ingestor?)

Default
1.048576e+06

workers

The number of worker threads that logstash should use (default: auto = one per CPU)

Default
auto

redis

host

Redis host of queue

key

Name of queue to pull messages from

Default
logstash

port

Redis port of queue

Default
6379

Templates

Templates are rendered and placed onto corresponding instances during the deployment process. This job's templates will be placed into /var/vcap/jobs/log_parser/ directory (learn more).

  • bin/log_parser_ctl (from bin/log_parser_ctl)
  • bin/monit_debugger (from bin/monit_debugger)
  • config/filters_post.conf (from config/filters_post.conf.erb)
  • config/filters_pre.conf (from config/filters_pre.conf.erb)
  • config/input_redis_and_output_elasticsearch.conf (from config/input_redis_and_output_elasticsearch.conf.erb)
  • data/properties.sh (from data/properties.sh.erb)
  • helpers/ctl_setup.sh (from helpers/ctl_setup.sh)
  • helpers/ctl_utils.sh (from helpers/ctl_utils.sh)

Packages

Packages are compiled and placed onto corresponding instances during the deployment process. Packages will be placed into /var/vcap/packages/ directory.