@@ -30,27 +30,31 @@ The package is hosted with pypi (https://pypi.org/project/clp-logging/), so it
3030can be installed with ` pip ` :
3131
3232` python3 -m pip install --upgrade clp-logging `
33+
34+ ## Logging handlers
35+
36+ ### ClpKeyValuePairStreamHandler
3337
34- ## Logging key-value pairs with ` ClpKeyValuePairStreamHandler `
38+ ⭐ * New in v0.0.14 *
3539
36- Introduced in version 0.0.14, ` ClpKeyValuePairStreamHandler ` enables applications to log and
37- serialize key-value pair objects directly into the CLP key-value pair IR stream format using
38- Python's standard logging APIs. It operates similarly to Python's standard ` logging.StreamHandler ` ,
39- with the following differences:
40- - Log events (` logging.LogRecord ` ) should contain the key-value pairs that a user wants to log
41- as a Python dictionary.
42- - These key-value pairs are written directly, without being formatted as a string.
43- - The key-value pairs will be serialized into the CLP key-value pair IR format before writing to
44- the stream.
40+ This handler enables applications to write structured log events directly into CLP's key-value pair
41+ (kv-pair) IR stream format. The handler accepts structured log events in the form of Python
42+ dictionaries, where each dictionary entry must abide by the requirements detailed
43+ [ below] ( #key-value-pair-requirements ) . The handler will also automatically include certain
44+ [ metadata] ( #automatically-generated-kv-pairs ) , like the log event's level, with each log event.
4545
46- > [ !NOTE]
47- > The current ` ClpKeyValuePairStreamHandler ` does not support ` CLPLogLevelTimeout ` . This feature
48- > will be added in a future release.
46+ > [ !NOTE]
47+ > Since this handler accepts structured log events, it doesn't support setting a
48+ > [ Formatter] [ py-logging-formatter ] (because the log events don't need formatting into a string).
49+
50+ > [ !WARNING]
51+ > ` ClpKeyValuePairStreamHandler ` currently doesn't support
52+ > [ CLPLogLevelTimeout] ( #log-level-timeout-feature-clplogleveltimeout ) . This feature will be added in
53+ > a future release.
4954
50- ### Key-value pairs as Python dictionary
55+ #### Key-value pair requirements
5156
52- Key-value pairs in the log event, presented as Python dictionaries, must abide by the following
53- rules:
57+ ` ClpKeyValuePairStreamHandler ` requires kv-pairs abide by the following rules:
5458
5559- Keys must be of type ` str ` .
5660- Values must be one of the following types:
@@ -62,29 +66,31 @@ rules:
6266 - must adhere to the aforementioned rules for keys and values.
6367 - can be empty.
6468
65- ### Auto-generated kv-pairs vs. User- generated kv-pairs
69+ #### Automatically generated kv-pairs
6670
67- As detailed [ here] [ clp-ffi-py-kv-pair-ir-stream ] , each log event contains both auto-generated and
68- user-generated kv-pairs.
71+ In addition to the kv-pairs explicitly logged by the application, the handler will add kv-pairs,
72+ like the log event's level, to each log event. We refer to the former as * user-generated* kv-pairs
73+ and the latter as * auto-generated* kv-pairs.
6974
70- In this logging handler, we differentiate auto/user-generated kv-pairs as follows:
71- - ** Auto-generated kv-pairs** : The log event metadata generated by the Python logging infrastructure
72- or logging handler itself, structured with the following schema:
73- - "timestamp" (` dict ` ):
74- - "unix_millisecs" (` int ` ): Unix timestamp in milliseconds since epoch.
75- - "utc_offset_secs" (` int ` ): The number of seconds the timezone is ahead of (positive) or
76- behind (negative) UTC.
77- - "level" (` dict ` ):
78- - "name" (` str ` ): Log level name.
79- - "num" (` int ` ): The numeric value associated with the log level.
80- - "source_location" (` dict ` ):
81- - "path" (` str ` ): Source file path of the logging statement.
82- - "line" (` int ` ): Line number of the logging statement.
83- - ** User-generated kv-pairs** : Key-value pairs provided by the application via the Python logging
84- API. These are passed through without modification and serialized as-is.
75+ > [ !NOTE]
76+ > The kv-pair IR stream format stores auto-generated kv-pairs separately from user-generated
77+ > kv-pairs, so users don't need to worry about key collisions with the auto-generated keys.
8578
79+ The handler adds the following auto-generated kv-pairs to each log event:
8680
87- ### Example: Use ` ClpKeyValuePairStreamHandler ` to log Python dictionary
81+ | Key | Value type | Description |
82+ | ---------------------| ------------| ---------------------------------------------------|
83+ | ` timestamp ` | ` dict ` | The log event's timestamp |
84+ | - ` unix_millisecs ` | ` int ` | The timestamp as a Unix timestamp in milliseconds |
85+ | - ` utc_offset_secs ` | ` int ` | The timestamp's UTC offset in seconds |
86+ | ` level ` | ` dict ` | The log event's level |
87+ | - ` name ` | ` str ` | The level's name |
88+ | - ` num ` | ` int ` | The level's numeric value |
89+ | ` source_location ` | ` dict ` | The source location of the logging statement |
90+ | - ` path ` | ` str ` | The source location's path |
91+ | - ` line ` | ` int ` | The source location's line number |
92+
93+ ### Example: ` ClpKeyValuePairStreamHandler `
8894
8995``` python
9096import logging
@@ -104,16 +110,15 @@ logger.info({
104110})
105111```
106112
107- ### Read key-value pair IR streams
113+ ### Reading kv- pair IR streams
108114
109115The following options are available for reading and deserializing kv-pair IR streams generated by
110116this handler:
111- - [ clp-ffi-py] [ clp-ffi-py-pypi ] : This library provides [ Deserializer] [ clp-ffi-py-deserializer-doc ]
112- to access a kv-pair IR stream in Python. See [ this example] [ clp-ffi-py-deserializer-example ] for
113- usage details.
114- - clp-ffi-js: TODO
115-
116- ## Logging handlers
117+
118+ - [ clp-ffi-py] [ clp-ffi-py-pypi ] : This library provides a [ Deserializer] [ clp-ffi-py-deserializer-doc ]
119+ to access a kv-pair IR stream in Python. [ This example] [ clp-ffi-py-deserializer-example ]
120+ illustrates its usage.
121+ - [ YScope Log Viewer] [ 2 ] : This UI can be used to view kv-pair IR streams.
117122
118123### CLPStreamHandler
119124
@@ -353,4 +358,5 @@ word][7].
353358[ clp-ffi-py-deserializer-doc ] : https://github.com/y-scope/clp-ffi-py?tab=readme-ov-file#example-code-using-deserializer-to-read-keyvaluepairlogevents-from-an-ir-stream
354359[ clp-ffi-py-deserializer-example ] : https://github.com/y-scope/clp-ffi-py?tab=readme-ov-file#example-code-using-deserializer-to-read-keyvaluepairlogevents-from-an-ir-stream
355360[ clp-ffi-py-kv-pair-ir-stream ] : https://github.com/y-scope/clp-ffi-py?tab=readme-ov-file#using-key-value-pair-ir-streams
356- [ clp-ffi-py-pypi ] : https://pypi.org/project/clp-ffi-py/
361+ [ clp-ffi-py-pypi ] : https://pypi.org/project/clp-ffi-py/
362+ [ py-logging-formatter ] : https://docs.python.org/3/library/logging.html#logging.Formatter
0 commit comments