|  | 
|  | 1 | +--- | 
|  | 2 | +mapped_pages: | 
|  | 3 | +  - https://www.elastic.co/guide/en/ecs-logging/ruby/current/setup.html | 
|  | 4 | +navigation_title: Get started | 
|  | 5 | +--- | 
|  | 6 | + | 
|  | 7 | +# Get started with ECS Logging Ruby [setup] | 
|  | 8 | + | 
|  | 9 | + | 
|  | 10 | +## Step 1: Set up application logging [setup-step-1] | 
|  | 11 | + | 
|  | 12 | + | 
|  | 13 | +### Add the dependency [_add_the_dependency] | 
|  | 14 | + | 
|  | 15 | +Add this line to your application’s Gemfile: | 
|  | 16 | + | 
|  | 17 | +```ruby | 
|  | 18 | +gem 'ecs-logging' | 
|  | 19 | +``` | 
|  | 20 | + | 
|  | 21 | +Execute with: | 
|  | 22 | + | 
|  | 23 | +```cmd | 
|  | 24 | +bundle install | 
|  | 25 | +``` | 
|  | 26 | + | 
|  | 27 | +Alternatively, you can install the package yourself with: | 
|  | 28 | + | 
|  | 29 | +```cmd | 
|  | 30 | +gem install ecs-logging | 
|  | 31 | +``` | 
|  | 32 | + | 
|  | 33 | + | 
|  | 34 | +### Configure [_configure] | 
|  | 35 | + | 
|  | 36 | +`Ecs::Logger` is a subclass of Ruby’s own [`Logger`](https://ruby-doc.org/stdlib/libdoc/logger/rdoc/Logger.md) and responds to the same methods. | 
|  | 37 | + | 
|  | 38 | +For example: | 
|  | 39 | + | 
|  | 40 | +```ruby | 
|  | 41 | +require 'ecs_logging/logger' | 
|  | 42 | + | 
|  | 43 | +logger = EcsLogging::Logger.new($stdout) | 
|  | 44 | +logger.info('my informative message') | 
|  | 45 | +logger.warn { 'be aware that…' } | 
|  | 46 | +logger.error('a_progname') { 'oh no!' } | 
|  | 47 | +``` | 
|  | 48 | + | 
|  | 49 | +Logs the following JSON to `$stdout`: | 
|  | 50 | + | 
|  | 51 | +```json | 
|  | 52 | +{"@timestamp":"2020-11-24T13:32:21.329Z","log.level":"INFO","message":"very informative","ecs.version":"1.4.0"} | 
|  | 53 | + {"@timestamp":"2020-11-24T13:32:21.330Z","log.level":"WARN","message":"be aware that…","ecs.version":"1.4.0"} | 
|  | 54 | + {"@timestamp":"2020-11-24T13:32:21.331Z","log.level":"ERROR","message":"oh no!","ecs.version":"1.4.0","process.title":"a_progname"} | 
|  | 55 | +``` | 
|  | 56 | + | 
|  | 57 | +Additionally, it allows for adding additional keys to messages. | 
|  | 58 | + | 
|  | 59 | +For example: | 
|  | 60 | + | 
|  | 61 | +```ruby | 
|  | 62 | +logger.info('ok', labels: { my_label: 'value' }, 'trace.id': 'abc-xyz') | 
|  | 63 | +``` | 
|  | 64 | + | 
|  | 65 | +Logs the following: | 
|  | 66 | + | 
|  | 67 | +```json | 
|  | 68 | +{ | 
|  | 69 | +  "@timestamp":"2020-11-24T13:32:21.331Z", | 
|  | 70 | +  "log.level":"INFO", | 
|  | 71 | +  "message":"oh no!", | 
|  | 72 | +  "ecs.version":"1.4.0", | 
|  | 73 | +  "labels":{"my_label":"value"}, | 
|  | 74 | +  "trace.id":"abc-xyz" | 
|  | 75 | +} | 
|  | 76 | +``` | 
|  | 77 | + | 
|  | 78 | +To include info about where the log was called, call the methods with `include_origin: true`, like `logger.warn('Hello!', include_origin: true)`. This logs: | 
|  | 79 | + | 
|  | 80 | +```json | 
|  | 81 | +{ | 
|  | 82 | +  "@timestamp":"2020-11-24T13:32:21.331Z", | 
|  | 83 | +  "log.level":"WARN", | 
|  | 84 | +  "message":"Hello!", | 
|  | 85 | +  "ecs.version":"1.4.0", | 
|  | 86 | +  "log.origin": { | 
|  | 87 | +    "file.line": 123, | 
|  | 88 | +    "file.name": "my_file.rb", | 
|  | 89 | +    "function": "call" | 
|  | 90 | +  } | 
|  | 91 | +} | 
|  | 92 | +``` | 
|  | 93 | + | 
|  | 94 | + | 
|  | 95 | +### Rack configuration [_rack_configuration] | 
|  | 96 | + | 
|  | 97 | +```ruby | 
|  | 98 | +use EcsLogging::Middleware, $stdout | 
|  | 99 | +``` | 
|  | 100 | + | 
|  | 101 | +Example output: | 
|  | 102 | + | 
|  | 103 | +```json | 
|  | 104 | +{ | 
|  | 105 | +  "@timestamp":"2020-12-07T13:44:04.568Z", | 
|  | 106 | +  "log.level":"INFO", | 
|  | 107 | +  "message":"GET /", | 
|  | 108 | +  "ecs.version":"1.4.0", | 
|  | 109 | +  "client":{ | 
|  | 110 | +    "address":"127.0.0.1" | 
|  | 111 | +  }, | 
|  | 112 | +  "http":{ | 
|  | 113 | +    "request":{ | 
|  | 114 | +      "method":"GET", | 
|  | 115 | +      "body.bytes":"0" | 
|  | 116 | +    } | 
|  | 117 | +  }, | 
|  | 118 | +  "url":{ | 
|  | 119 | +    "domain":"example.org", | 
|  | 120 | +    "path":"/", | 
|  | 121 | +    "port":"80", | 
|  | 122 | +    "scheme":"http" | 
|  | 123 | +  } | 
|  | 124 | +} | 
|  | 125 | +``` | 
|  | 126 | + | 
|  | 127 | + | 
|  | 128 | +## Step 2: Enable APM log correlation (optional) [setup-step-2] | 
|  | 129 | + | 
|  | 130 | +If you are using the Elastic APM Ruby agent, [enable log correlation](apm-agent-ruby://docs/reference/logs.md). | 
|  | 131 | + | 
|  | 132 | + | 
|  | 133 | +## Step 3: Configure Filebeat [setup-step-3] | 
|  | 134 | + | 
|  | 135 | +:::::::{tab-set} | 
|  | 136 | + | 
|  | 137 | +::::::{tab-item} Log file | 
|  | 138 | +1. Follow the [Filebeat quick start](beats://docs/reference/filebeat/filebeat-installation-configuration.md) | 
|  | 139 | +2. Add the following configuration to your `filebeat.yaml` file. | 
|  | 140 | + | 
|  | 141 | +For Filebeat 7.16+ | 
|  | 142 | + | 
|  | 143 | +```yaml | 
|  | 144 | +filebeat.inputs: | 
|  | 145 | +- type: filestream <1> | 
|  | 146 | +  paths: /path/to/logs.json | 
|  | 147 | +  parsers: | 
|  | 148 | +    - ndjson: | 
|  | 149 | +      overwrite_keys: true <2> | 
|  | 150 | +      add_error_key: true <3> | 
|  | 151 | +      expand_keys: true <4> | 
|  | 152 | + | 
|  | 153 | +processors: <5> | 
|  | 154 | +  - add_host_metadata: ~ | 
|  | 155 | +  - add_cloud_metadata: ~ | 
|  | 156 | +  - add_docker_metadata: ~ | 
|  | 157 | +  - add_kubernetes_metadata: ~ | 
|  | 158 | +``` | 
|  | 159 | +
 | 
|  | 160 | +1. Use the filestream input to read lines from active log files. | 
|  | 161 | +2. Values from the decoded JSON object overwrite the fields that {{filebeat}} normally adds (type, source, offset, etc.) in case of conflicts. | 
|  | 162 | +3. {{filebeat}} adds an "error.message" and "error.type: json" key in case of JSON unmarshalling errors. | 
|  | 163 | +4. {{filebeat}} will recursively de-dot keys in the decoded JSON, and expand them into a hierarchical object structure. | 
|  | 164 | +5. Processors enhance your data. See [processors](beats://docs/reference/filebeat/filtering-enhancing-data.md) to learn more. | 
|  | 165 | +
 | 
|  | 166 | +
 | 
|  | 167 | +For Filebeat < 7.16 | 
|  | 168 | +
 | 
|  | 169 | +```yaml | 
|  | 170 | +filebeat.inputs: | 
|  | 171 | +- type: log | 
|  | 172 | +  paths: /path/to/logs.json | 
|  | 173 | +  json.keys_under_root: true | 
|  | 174 | +  json.overwrite_keys: true | 
|  | 175 | +  json.add_error_key: true | 
|  | 176 | +  json.expand_keys: true | 
|  | 177 | + | 
|  | 178 | +processors: | 
|  | 179 | +- add_host_metadata: ~ | 
|  | 180 | +- add_cloud_metadata: ~ | 
|  | 181 | +- add_docker_metadata: ~ | 
|  | 182 | +- add_kubernetes_metadata: ~ | 
|  | 183 | +``` | 
|  | 184 | +:::::: | 
|  | 185 | +
 | 
|  | 186 | +::::::{tab-item} Kubernetes | 
|  | 187 | +1. Make sure your application logs to stdout/stderr. | 
|  | 188 | +2. Follow the [Run Filebeat on Kubernetes](beats://docs/reference/filebeat/running-on-kubernetes.md) guide. | 
|  | 189 | +3. Enable [hints-based autodiscover](beats://docs/reference/filebeat/configuration-autodiscover-hints.md) (uncomment the corresponding section in `filebeat-kubernetes.yaml`). | 
|  | 190 | +4. Add these annotations to your pods that log using ECS loggers. This will make sure the logs are parsed appropriately. | 
|  | 191 | + | 
|  | 192 | +```yaml | 
|  | 193 | +annotations: | 
|  | 194 | +  co.elastic.logs/json.overwrite_keys: true <1> | 
|  | 195 | +  co.elastic.logs/json.add_error_key: true <2> | 
|  | 196 | +  co.elastic.logs/json.expand_keys: true <3> | 
|  | 197 | +``` | 
|  | 198 | + | 
|  | 199 | +1. Values from the decoded JSON object overwrite the fields that {{filebeat}} normally adds (type, source, offset, etc.) in case of conflicts. | 
|  | 200 | +2. {{filebeat}} adds an "error.message" and "error.type: json" key in case of JSON unmarshalling errors. | 
|  | 201 | +3. {{filebeat}} will recursively de-dot keys in the decoded JSON, and expand them into a hierarchical object structure. | 
|  | 202 | +:::::: | 
|  | 203 | + | 
|  | 204 | +::::::{tab-item} Docker | 
|  | 205 | +1. Make sure your application logs to stdout/stderr. | 
|  | 206 | +2. Follow the [Run Filebeat on Docker](beats://docs/reference/filebeat/running-on-docker.md) guide. | 
|  | 207 | +3. Enable [hints-based autodiscover](beats://docs/reference/filebeat/configuration-autodiscover-hints.md). | 
|  | 208 | +4. Add these labels to your containers that log using ECS loggers. This will make sure the logs are parsed appropriately. | 
|  | 209 | + | 
|  | 210 | +```yaml | 
|  | 211 | +labels: | 
|  | 212 | +  co.elastic.logs/json.overwrite_keys: true <1> | 
|  | 213 | +  co.elastic.logs/json.add_error_key: true <2> | 
|  | 214 | +  co.elastic.logs/json.expand_keys: true <3> | 
|  | 215 | +``` | 
|  | 216 | + | 
|  | 217 | +1. Values from the decoded JSON object overwrite the fields that {{filebeat}} normally adds (type, source, offset, etc.) in case of conflicts. | 
|  | 218 | +2. {{filebeat}} adds an "error.message" and "error.type: json" key in case of JSON unmarshalling errors. | 
|  | 219 | +3. {{filebeat}} will recursively de-dot keys in the decoded JSON, and expand them into a hierarchical object structure. | 
|  | 220 | +:::::: | 
|  | 221 | + | 
|  | 222 | +::::::: | 
|  | 223 | +For more information, see the [Filebeat reference](beats://docs/reference/filebeat/configuring-howto-filebeat.md). | 
|  | 224 | + | 
0 commit comments