Skip to content

Add note on telemetry and call-home functionality #740

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
May 8, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion modules/ROOT/partials/release-notes/release-23.1.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ These options are described in more detail xref:contributor:adr/ADR023-product-i
Component activity within the platform is logged in a way that makes it difficult to find, persist and consolidate this information.
Log configuration is also a challenge.
To address these two issues a logging framework has been added to the platform, offering a consistent custom resource configuration and a separate, persisted sink (the current implementation support OpenSearch).
This is discussed in more detail xref:contributor:adr/ADR025-logging_architecture.adoc[in this ADR] and xref:concepts:logging.adoc[on this concepts page].
This is discussed in more detail xref:contributor:adr/ADR025-logging_architecture.adoc[in this ADR] and xref:concepts:observability/logging.adoc[on this concepts page].

In this release this has been added to the following components:

Expand Down
2 changes: 1 addition & 1 deletion modules/ROOT/partials/release-notes/release-23.4.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ See this xref:contributor:adr/ADR026-affinities.adoc[ADR] for more information.

The logging framework (added to the platform in Release 23.1) offers a consistent custom resource configuration and a separate, persisted sink (defaulting to OpenSearch).
This has now been rolled out across all products.
See this xref:contributor:adr/adr025-logging_architecture[ADR] and this xref:concepts:logging.adoc[concepts page] for more information.
See this xref:contributor:adr/adr025-logging_architecture[ADR] and this xref:concepts:observability/logging.adoc[concepts page] for more information.

===== Service Type

Expand Down
2 changes: 1 addition & 1 deletion modules/ROOT/partials/release-notes/release-25.3.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ Released on 2025-03-21.

===== Observability

Products now log information about their xref:concepts:containerdebug.adoc[container environment], on startup and on an interval after that.
Products now log information about their xref:concepts:observability/containerdebug.adoc[container environment], on startup and on an interval after that.

===== NiFi

Expand Down
6 changes: 3 additions & 3 deletions modules/concepts/nav.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -22,8 +22,8 @@
*** xref:operations/graceful_shutdown.adoc[]
*** xref:operations/temporary_credentials_lifetime.adoc[]
** Observability
*** xref:labels.adoc[Labels]
*** xref:logging.adoc[Logging]
*** xref:containerdebug.adoc[Container environment]
*** xref:observability/labels.adoc[Labels]
*** xref:observability/logging.adoc[Logging]
*** xref:observability/containerdebug.adoc[Container environment]
*** xref:observability/telemetry.adoc[Telemetry]
** xref:container-images.adoc[]
4 changes: 2 additions & 2 deletions modules/concepts/pages/index.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -34,5 +34,5 @@ It covers xref:operations/cluster_operations.adoc[starting, stopping and restart

== Observability

Learn about which xref:labels.adoc[labels] are attached to the operators and the stacklets and the resources that get created by the operator.
Also learn about xref:logging.adoc[].
Learn about which xref:observability/labels.adoc[labels] are attached to the operators and the stacklets and the resources that get created by the operator.
Also learn about xref:observability/logging.adoc[].
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ main product container.
== Output

The collected information is logged to the Kubernetes pod logs (readable via `kubectl logs <pod>`)
and to the xref:logging.adoc#architecture[shared Stackable log dir]
and to the xref:observability/logging.adoc#architecture[shared Stackable log dir]
(for pickup by the Vector log agent).

Additionally, it is written as JSON to `/stackable/log/containerdebug-state.json`.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
= Logging
:page-aliases: logging.adoc
:description: A conceptual explanation of the logging architecture of the Stackable Data Platform, and how it is configured.
:keywords: logging, observability, log aggregation, Kubernetes, k8s, Vector, Elasticsearch, OpenSearch
:vector: https://vector.dev/
Expand Down
7 changes: 7 additions & 0 deletions modules/concepts/pages/observability/telemetry.adoc
Original file line number Diff line number Diff line change
@@ -1,5 +1,12 @@
= Telemetry signals in operators

[NOTE]
====
Telemetry data is useful for your own observability and monitoring systems.
There is no "call-home" functionality.
Data is not exported out of the container unless explicitly configured.
====

Since SDP 25.7.0, all Stackable operators emit telemetry data in the following ways:

* Console logs in plain and JSON format
Expand Down
4 changes: 2 additions & 2 deletions modules/contributor/pages/guidelines/logging.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

= Implementing log aggregation

For a conceptual overview of logging aggregation, consult the xref:concepts:logging.adoc[logging concept page].
For a conceptual overview of logging aggregation, consult the xref:concepts:observability/logging.adoc[logging concept page].

== Overview

Expand Down Expand Up @@ -188,4 +188,4 @@ if let Some(ContainerLogConfig {
log_config,
));
}
----
----
4 changes: 2 additions & 2 deletions modules/tutorials/pages/logging-vector-aggregator.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ and apply it:
[source,bash]
include::example$logging-aggregator/main.sh[tag=zk]

TIP: You can learn more about how to configure logging in a product at the xref:concepts:logging.adoc[logging concept documentation].
TIP: You can learn more about how to configure logging in a product at the xref:concepts:observability/logging.adoc[logging concept documentation].

== Watch the logs

Expand Down Expand Up @@ -112,4 +112,4 @@ Congratulations, this concludes the tutorial!

== What's next?

Look into different sink configurations which are more suited to production use in the https://vector.dev/docs/reference/configuration/sinks/[sinks overview documetation] or learn more about how logging works on the platform in the xref:concepts:logging.adoc[concepts documentation].
Look into different sink configurations which are more suited to production use in the https://vector.dev/docs/reference/configuration/sinks/[sinks overview documetation] or learn more about how logging works on the platform in the xref:concepts:observability/logging.adoc[concepts documentation].
Loading