You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: _data-prepper/pipelines/configuration/processors/split-string.md
+83-16Lines changed: 83 additions & 16 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -29,36 +29,103 @@ source | N/A | N/A | The key to split.
29
29
delimiter | No | N/A | The separator character responsible for the split. Cannot be defined at the same time as `delimiter_regex`. At least `delimiter` or `delimiter_regex` must be defined.
30
30
delimiter_regex | No | N/A | The regex string responsible for the split. Cannot be defined at the same time as `delimiter`. At least `delimiter` or `delimiter_regex` must be defined.
31
31
32
-
### Usage
32
+
### Example
33
33
34
34
To get started, create the following `pipeline.yaml` file:
35
35
36
36
```yaml
37
-
pipeline:
37
+
split-string-all-configs-pipeline:
38
38
source:
39
-
file:
40
-
path: "/full/path/to/logs_json.log"
41
-
record_type: "event"
42
-
format: "json"
39
+
http:
40
+
path: /logs
41
+
ssl: false
42
+
43
43
processor:
44
44
- split_string:
45
+
# 1) The top-level list of split "entries"
45
46
entries:
46
-
- source: "message"
47
+
# 2) Use `source` + `delimiter` (comma)
48
+
- source: csv_line
47
49
delimiter: ","
50
+
51
+
# 3) Another `source` + `delimiter` (pipe)
52
+
- source: tags
53
+
delimiter: "|"
54
+
55
+
# 4) `source` + `delimiter` (slash) to split a path
Next, create a log file named `logs_json.log`. After that, replace the `path` in the file source of your `pipeline.yaml` file with your file path. For more detailed information, see [Configuring OpenSearch Data Prepper]({{site.url}}{{site.baseurl}}/data-prepper/getting-started/#2-configuring-data-prepper).
75
+
You can test the pipeline using the following command:
54
76
55
-
Before you run Data Prepper, the source appears in the following format:
56
-
57
-
```json
58
-
{"message": "hello,world"}
77
+
```bash
78
+
curl -sS -X POST "http://localhost:2021/logs" \
79
+
-H "Content-Type: application/json" \
80
+
-d '[
81
+
{
82
+
"csv_line": "x,y",
83
+
"tags": "beta|test",
84
+
"path": "usr/local/bin",
85
+
"semicolons": "alpha;beta ; gamma"
86
+
}
87
+
]'
59
88
```
60
-
After you run Data Prepper, the source is converted to the following format:
89
+
{% include copy.html %}
90
+
91
+
The document stored in OpenSearch contains the following information:
0 commit comments