feat(telemetry): adding initial telemetry functionality to the cli #956

billxinli · 2025-12-01T16:40:13Z

Summary

This PR adds telemetry functionality to the Socket CLI to track usage patterns, performance metrics, and errors. The implementation includes instrumentation across CLI commands, subprocess executions, and API interactions.

Telemetry Infrastructure

Organization-scoped tracking: All telemetry requires org context - cannot track without organization
Event batching: Configurable batch sizes with periodic flushing (500ms intervals)
Graceful degradation: Telemetry failures never block CLI execution
Session tracking: Unique session IDs per CLI invocation
Privacy-first: Comprehensive PII sanitization (tokens, file paths, package names)
Queue size limiting: Max 1,000 events to prevent memory leaks during API outages
Timeout protection: 2-second max flush time prevents hanging on exit

Event Types Tracked

CLI lifecycle: cli_start, cli_complete, cli_error
Subprocess execution: subprocess_start, subprocess_complete, subprocess_error
API interactions: api_request, api_response, api_error
Custom events: Generic event tracking with metadata support

PII Sanitization

API tokens: Redacts sktsec_* tokens and hex tokens
File paths: Replaces home directory with ~
Package names: Strips package arguments after wrapper CLIs
Sensitive flags: Redacts values after --api-token, --token, -t

Example Sanitization

Input:  ['node', 'socket', 'npm', 'install', '@my/private-pkg', '--token', 'sktsec_abc123']
Output: ['npm', 'install']  // Package name and token removed

Telemetry Configuration

  const TELEMETRY_SERVICE_CONFIG = {
    batch_size: 10,           // Events per batch
    flush_interval: 500,      // 0.5 second periodic flush
    flush_timeout: 2_000,     // 2 second max flush duration
    max_queue_size: 1_000,    // Memory leak protection
  }

Breaking Changes

None. Telemetry is opt-in via organization configuration and fails gracefully.

Note

Introduces org-scoped telemetry across the CLI, SDK, and package manager wrappers with sanitization, batching/flush, global error handling, and comprehensive tests; also updates ecosystems and bumps the SDK.

Telemetry Infrastructure:
- Add utils/telemetry/* (integration, service, types) for org-scoped events, argv/error sanitization, session IDs, batching, periodic flush, and timeouts.
CLI:
- Instrument src/cli.mts to track cli_start, cli_complete, cli_error; ensure finalizeTelemetry(); add handlers for uncaught exceptions and unhandled rejections.
Package Manager Wrappers (npm, npx, pnpm, yarn):
- Track subprocess start/exit in cmd-*.mts and flush telemetry before process exit.
SDK Integration:
- In utils/sdk.mts, add request/response hooks to emit api_request, api_response, api_error (skipping telemetry endpoints) with optional debug logging.
Ecosystem Updates:
- Extend ALL_ECOSYSTEMS (e.g., alpm, qpkg, vscode) and comment out strict type check due to temporary registry/SDK mismatch.
Tests:
- Add unit tests for CLI, SDK hooks, telemetry integration, and service (src/test/*.mts, utils/telemetry/*.test.mts).
Dependencies:
- Bump @socketsecurity/sdk to 1.4.95 (lockfile updated).

^{Written by Cursor Bugbot for commit 1f673f5. Configure here.}

package.json

billxinli · 2025-12-01T16:47:08Z

src/utils/ecosystem.mts

+// Temporarily commented out due to dependency version mismatch.
+// SDK has "alpm" but registry's EcosystemString doesn't yet.
+// type MissingInEcosystemString = Exclude<PURL_Type, EcosystemString>


The sdk synced the latest version of OAS spec.

billxinli · 2025-12-01T17:06:36Z

src/utils/sdk.mts

+    hooks: {
+      onRequest: (info: RequestInfo) => {
+        // Skip tracking for telemetry submission endpoints to prevent infinite loop.
+        const isTelemetryEndpoint = info.url.includes('/telemetry')


The sdk is calling the hooks on some endpoint and not others. I am not sure if this is the intended pattern. (Should only some endpoints and functions gets hooks and not others? If so, that could simplify this awkward logic.)

socket-security-staging · 2025-12-09T20:14:34Z

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff	Package	Supply Chain Security	Vulnerability	Quality	Maintenance	License
	npm/@socketsecurity/sdk@1.4.94 ⏵ 1.4.95

View full report

socket-security · 2025-12-09T20:17:06Z

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff	Package	Supply Chain Security	Vulnerability	Quality	Maintenance	License
	npm/@socketsecurity/sdk@1.4.94 ⏵ 1.4.95

View full report

cursor

Comment @cursor review or bugbot run to trigger another review on this PR

src/commands/npm/cmd-npm.mts

src/cli.mts

billxinli · 2025-12-09T21:36:44Z

@cursor review

cursor

Comment @cursor review or bugbot run to trigger another review on this PR

src/utils/telemetry/integration.mts

cursor · 2025-12-09T21:44:16Z

src/utils/telemetry/integration.mts

+    await trackSubprocessError(command, startTime, error, exitCode)
+  } else if (exitCode === 0) {
+    await trackSubprocessComplete(command, startTime, exitCode)
+  }


Bug: Signal-terminated subprocesses not tracked in telemetry

The trackSubprocessExit function only tracks events when exitCode !== null && exitCode !== 0 (error) or exitCode === 0 (complete). When a subprocess is killed by a signal, Node.js sets code to null and signalName to the signal. This case falls through without tracking any telemetry event, leaving a gap in subprocess tracking for signal-terminated processes.

cursor · 2025-12-09T21:44:16Z

src/cli.mts

+
+  // eslint-disable-next-line n/no-process-exit
+  process.exit(1)
+})


Bug: Async handlers for process events may not complete

Using async handlers with process.on('uncaughtException') and process.on('unhandledRejection') is problematic because Node.js doesn't wait for async operations to complete before the handler returns. If the async telemetry calls (trackCliError, finalizeTelemetry) fail or take time, the process may exit before they complete. Additionally, any errors thrown within these async handlers won't be caught, potentially causing secondary unhandled rejections.

Additional Locations (1)

src/cli.mts#L158-L171

This is where the telemetry will await to be flushed before being exited with 1

billxinli commented Dec 1, 2025

View reviewed changes

package.json Outdated Show resolved Hide resolved

billxinli commented Dec 1, 2025

View reviewed changes

billxinli requested a review from jdalton December 1, 2025 16:59

billxinli commented Dec 1, 2025

View reviewed changes

feat(telemetry): adding initial telemetry functionality to the cli

3082a6e

billxinli force-pushed the 1.x-telemetry branch from bcc83ee to 3082a6e Compare December 9, 2025 20:12

billxinli marked this pull request as ready for review December 9, 2025 20:13

cursor bot reviewed Dec 9, 2025

View reviewed changes

src/commands/npm/cmd-npm.mts Outdated Show resolved Hide resolved

src/cli.mts Outdated Show resolved Hide resolved

cursor bot reviewed Dec 9, 2025

View reviewed changes

feat(cr): cr

da523cf

billxinli force-pushed the 1.x-telemetry branch from 1f673f5 to da523cf Compare December 9, 2025 23:41

feat(telemetry): adding initial telemetry functionality to the cli #956

Are you sure you want to change the base?

feat(telemetry): adding initial telemetry functionality to the cli #956

Uh oh!

Conversation

billxinli commented Dec 1, 2025 • edited by cursor bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Telemetry Infrastructure

Event Types Tracked

PII Sanitization

Example Sanitization

Telemetry Configuration

Breaking Changes

Uh oh!

Uh oh!

billxinli Dec 1, 2025

Choose a reason for hiding this comment

Uh oh!

billxinli Dec 1, 2025

Choose a reason for hiding this comment

Uh oh!

socket-security-staging bot commented Dec 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

socket-security bot commented Dec 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cursor bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

billxinli commented Dec 9, 2025

Uh oh!

cursor bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

cursor bot Dec 9, 2025

Choose a reason for hiding this comment

Bug: Signal-terminated subprocesses not tracked in telemetry

Uh oh!

cursor bot Dec 9, 2025

Choose a reason for hiding this comment

Bug: Async handlers for process events may not complete

Uh oh!

billxinli Dec 9, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

billxinli commented Dec 1, 2025 •

edited by cursor bot

Loading

socket-security-staging bot commented Dec 9, 2025 •

edited

Loading

socket-security bot commented Dec 9, 2025 •

edited

Loading