Skip to content

Commit 33dc514

Browse files
committed
Refactor ClpKeyValuePairStreamHandler docs.
1 parent 9ece534 commit 33dc514

File tree

1 file changed

+49
-43
lines changed

1 file changed

+49
-43
lines changed

README.md

Lines changed: 49 additions & 43 deletions
Original file line numberDiff line numberDiff line change
@@ -30,27 +30,31 @@ The package is hosted with pypi (https://pypi.org/project/clp-logging/), so it
3030
can be installed with `pip`:
3131

3232
`python3 -m pip install --upgrade clp-logging`
33+
34+
## Logging handlers
35+
36+
### ClpKeyValuePairStreamHandler
3337

34-
## Logging key-value pairs with `ClpKeyValuePairStreamHandler`
38+
*New in v0.0.14*
3539

36-
Introduced in version 0.0.14, `ClpKeyValuePairStreamHandler` enables applications to log and
37-
serialize key-value pair objects directly into the CLP key-value pair IR stream format using
38-
Python's standard logging APIs. It operates similarly to Python's standard `logging.StreamHandler`,
39-
with the following differences:
40-
- Log events (`logging.LogRecord`) should contain the key-value pairs that a user wants to log
41-
as a Python dictionary.
42-
- These key-value pairs are written directly, without being formatted as a string.
43-
- The key-value pairs will be serialized into the CLP key-value pair IR format before writing to
44-
the stream.
40+
This handler enables applications to write structured log events directly into CLP's key-value pair
41+
(kv-pair) IR stream format. The handler accepts structured log events in the form of Python
42+
dictionaries, where each dictionary entry must abide by the requirements detailed
43+
[below](#key-value-pair-requirements). The handler will also automatically include certain
44+
[metadata](#automatically-generated-kv-pairs), like the log event's level, with each log event.
4545

46-
> [!NOTE]
47-
> The current `ClpKeyValuePairStreamHandler` does not support `CLPLogLevelTimeout`. This feature
48-
> will be added in a future release.
46+
> [!NOTE]
47+
> Since this handler accepts structured log events, it doesn't support setting a
48+
> [Formatter][py-logging-formatter] (because the log events don't need formatting into a string).
49+
50+
> [!WARNING]
51+
> `ClpKeyValuePairStreamHandler` currently doesn't support
52+
> [CLPLogLevelTimeout](#log-level-timeout-feature-clplogleveltimeout). This feature will be added in
53+
> a future release.
4954
50-
### Key-value pairs as Python dictionary
55+
#### Key-value pair requirements
5156

52-
Key-value pairs in the log event, presented as Python dictionaries, must abide by the following
53-
rules:
57+
`ClpKeyValuePairStreamHandler` requires kv-pairs abide by the following rules:
5458

5559
- Keys must be of type `str`.
5660
- Values must be one of the following types:
@@ -62,29 +66,31 @@ rules:
6266
- must adhere to the aforementioned rules for keys and values.
6367
- can be empty.
6468

65-
### Auto-generated kv-pairs vs. User-generated kv-pairs
69+
#### Automatically generated kv-pairs
6670

67-
As detailed [here][clp-ffi-py-kv-pair-ir-stream], each log event contains both auto-generated and
68-
user-generated kv-pairs.
71+
In addition to the kv-pairs explicitly logged by the application, the handler will add kv-pairs,
72+
like the log event's level, to each log event. We refer to the former as *user-generated* kv-pairs
73+
and the latter as *auto-generated* kv-pairs.
6974

70-
In this logging handler, we differentiate auto/user-generated kv-pairs as follows:
71-
- **Auto-generated kv-pairs**: The log event metadata generated by the Python logging infrastructure
72-
or logging handler itself, structured with the following schema:
73-
- "timestamp" (`dict`):
74-
- "unix_millisecs" (`int`): Unix timestamp in milliseconds since epoch.
75-
- "utc_offset_secs" (`int`): The number of seconds the timezone is ahead of (positive) or
76-
behind (negative) UTC.
77-
- "level" (`dict`):
78-
- "name" (`str`): Log level name.
79-
- "num" (`int`): The numeric value associated with the log level.
80-
- "source_location" (`dict`):
81-
- "path" (`str`): Source file path of the logging statement.
82-
- "line" (`int`): Line number of the logging statement.
83-
- **User-generated kv-pairs**: Key-value pairs provided by the application via the Python logging
84-
API. These are passed through without modification and serialized as-is.
75+
> [!NOTE]
76+
> The kv-pair IR stream format stores auto-generated kv-pairs separately from user-generated
77+
> kv-pairs, so users don't need to worry about key collisions with the auto-generated keys.
8578
79+
The handler adds the following auto-generated kv-pairs to each log event:
8680

87-
### Example: Use `ClpKeyValuePairStreamHandler` to log Python dictionary
81+
| Key | Value type | Description |
82+
|---------------------|------------|---------------------------------------------------|
83+
| `timestamp` | `dict` | The log event's timestamp |
84+
| - `unix_millisecs` | `int` | The timestamp as a Unix timestamp in milliseconds |
85+
| - `utc_offset_secs` | `int` | The timestamp's UTC offset in seconds |
86+
| `level` | `dict` | The log event's level |
87+
| - `name` | `str` | The level's name |
88+
| - `num` | `int` | The level's numeric value |
89+
| `source_location` | `dict` | The source location of the logging statement |
90+
| - `path` | `str` | The source location's path |
91+
| - `line` | `int` | The source location's line number |
92+
93+
### Example: `ClpKeyValuePairStreamHandler`
8894

8995
```python
9096
import logging
@@ -104,16 +110,15 @@ logger.info({
104110
})
105111
```
106112

107-
### Read key-value pair IR streams
113+
### Reading kv-pair IR streams
108114

109115
The following options are available for reading and deserializing kv-pair IR streams generated by
110116
this handler:
111-
- [clp-ffi-py][clp-ffi-py-pypi]: This library provides [Deserializer][clp-ffi-py-deserializer-doc]
112-
to access a kv-pair IR stream in Python. See [this example][clp-ffi-py-deserializer-example] for
113-
usage details.
114-
- clp-ffi-js: TODO
115-
116-
## Logging handlers
117+
118+
- [clp-ffi-py][clp-ffi-py-pypi]: This library provides a [Deserializer][clp-ffi-py-deserializer-doc]
119+
to access a kv-pair IR stream in Python. [This example][clp-ffi-py-deserializer-example]
120+
illustrates its usage.
121+
- [YScope Log Viewer][2]: This UI can be used to view kv-pair IR streams.
117122

118123
### CLPStreamHandler
119124

@@ -353,4 +358,5 @@ word][7].
353358
[clp-ffi-py-deserializer-doc]: https://github.com/y-scope/clp-ffi-py?tab=readme-ov-file#example-code-using-deserializer-to-read-keyvaluepairlogevents-from-an-ir-stream
354359
[clp-ffi-py-deserializer-example]: https://github.com/y-scope/clp-ffi-py?tab=readme-ov-file#example-code-using-deserializer-to-read-keyvaluepairlogevents-from-an-ir-stream
355360
[clp-ffi-py-kv-pair-ir-stream]: https://github.com/y-scope/clp-ffi-py?tab=readme-ov-file#using-key-value-pair-ir-streams
356-
[clp-ffi-py-pypi]: https://pypi.org/project/clp-ffi-py/
361+
[clp-ffi-py-pypi]: https://pypi.org/project/clp-ffi-py/
362+
[py-logging-formatter]: https://docs.python.org/3/library/logging.html#logging.Formatter

0 commit comments

Comments
 (0)