feat(aws)!: Remove deprecated PrivateLink env variable (#1002)

Signed-off-by: Vincent Boutour <[email protected]>

Author: ViBiOh

aws/logs_monitoring/README.md

@@ -310,18 +310,6 @@ You can run the Forwarder in a VPC private subnet and send data to Datadog over
     2. Set `VPCSecurityGroupIds` and `VPCSubnetIds` based on your VPC settings.
     3. Set `DdFetchLambdaTags`, `DdFetchStepFunctionsTags` and `DdFetchS3Tags` to `false`, because AWS Resource Groups Tagging API doesn't support PrivateLink.
 
-#### DdUsePrivateLink is deprecated
-
-The `DdUsePrivateLink` option has been deprecated since [v3.41.0][16]. This option was previously used to instruct the Forwarder to use a special set of PrivateLink endpoints for data intake: `pvtlink.api.{{< region-param key="dd_site" code="true" >}}`, `api-pvtlink.logs.{{< region-param key="dd_site" code="true" >}}`, and `trace-pvtlink.agent.{{< region-param key="dd_site" code="true" >}}`. Since v3.41.0, the Forwarder can send data over PrivateLink to Datadog using the regular DNS names of intake endpoints: `api.{{< region-param key="dd_site" code="true" >}}`, `http-intake.logs.{{< region-param key="dd_site" code="true" >}}`, and `trace.agent.{{< region-param key="dd_site" code="true" >}}`. Therefore, the `DdUsePrivateLink` option is no longer needed.
-
-If you have an older deployment of the Forwarder with `DdUsePrivateLink` set to `true`, then you may find mismatches between your configured PrivateLink endpoints and the [ones documented in Datadog][14], which is expected. Although the older PrivateLink endpoints were removed from that doc, they remain to function. When upgrading the Forwarder, there is no change required, that is, you can keep `DdUsePrivateLink` enabled and continue to use the older endpoints.
-
-However, if you are interested in switching to the new endpoints, you need to follow the updated instructions above to:
-
-1. Set up the new endpoints to `api.{{< region-param key="dd_site" code="true" >}}`, `http-intake.logs.{{< region-param key="dd_site" code="true" >}}`, and `trace.agent.{{< region-param key="dd_site" code="true" >}}`.
-2. Set `DdUseVPC` to `true`.
-3. Set `DdUsePrivateLink` to `false`.
-
 ### AWS VPC and proxy support
 
 If you must deploy the Forwarder to a VPC without direct public internet access, and you cannot use AWS PrivateLink to connect to Datadog (for example, if your organization is hosted on the Datadog EU site: `datadoghq.eu`), then you can send data through a proxy.
@@ -467,9 +455,6 @@ To test different patterns against your logs, turn on [debug logs](#troubleshoot
 `PermissionsBoundaryArn`
 : ARN for the Permissions Boundary Policy.
 
-`DdUsePrivateLink` (DEPRECATED)
-: Set to true to enable sending logs and metrics through AWS PrivateLink. See [Connect to Datadog over AWS PrivateLink][2].
-
 `DdHttpProxyURL`
 : Sets the standard web proxy environment variables HTTP_PROXY and HTTPS_PROXY. These are the URL endpoints your proxy server exposes. Do not use this in combination with AWS Private Link. Make sure to also set `DdSkipSslValidation` to true.
 
@@ -613,9 +598,6 @@ To test different patterns against your logs, turn on [debug logs](#troubleshoot
 `PERMISSIONS_BOUNDARY_ARN`
 : ARN for the Permissions Boundary Policy.
 
-`DD_USE_PRIVATE_LINK` (DEPRECATED)
-: Set to true to enable sending logs and metrics through AWS PrivateLink. See [Connect to Datadog over AWS PrivateLink][2].
-
 `DD_HTTP_PROXY_URL`
 : Sets the standard web proxy environment variables HTTP_PROXY and HTTPS_PROXY. These are the URL endpoints your proxy server exposes. Do not use this in combination with AWS Private Link. Make sure to also set `DD_SKIP_SSL_VALIDATION` to true.
 

aws/logs_monitoring/settings.py

@@ -121,24 +121,6 @@ def get_env_var(envvar, default, boolean=False):
 ## @param DD_CUSTOM_SOURCE
 DD_CUSTOM_SOURCE = get_env_var("DD_SOURCE", "")
 
-# DEPRECATED. No longer need to use special endpoints, as you can now expose
-# regular Datadog API endpoints `api`, `http-intake.logs` and `trace.agent`
-# via PrivateLink. See https://docs.datadoghq.com/agent/guide/private-link/.
-# @param DD_USE_PRIVATE_LINK - whether to forward logs via PrivateLink
-# Overrides incompatible settings
-#
-DD_USE_PRIVATE_LINK = get_env_var("DD_USE_PRIVATE_LINK", "false", boolean=True)
-if DD_USE_PRIVATE_LINK:
-    logger.debug("Private link enabled, overriding configuration settings")
-    # Only the US Datadog site is supported when PrivateLink is enabled
-    DD_SITE = "datadoghq.com"
-    DD_NO_SSL = False
-    DD_PORT = 443
-    # Override URLs
-    DD_URL = "api-pvtlink.logs.datadoghq.com"
-    DD_API_URL = "https://pvtlink.api.datadoghq.com"
-    DD_TRACE_INTAKE_URL = "https://trace-pvtlink.agent.datadoghq.com"
-
 
 class ScrubbingRuleConfig(object):
     def __init__(self, name, pattern, placeholder, enabled=True):

aws/logs_monitoring/template.yaml

@@ -180,13 +180,6 @@ Parameters:
       - true
       - false
     Description: Set to false to disable log compression. Only valid when sending logs over HTTP.
-  DdUsePrivateLink:
-    Type: String
-    Default: false
-    AllowedValues:
-      - true
-      - false
-    Description: DEPRECATED, DO NOT CHANGE. See README.md for details. Set to true to deploy the Forwarder to a VPC and send logs, metrics, and traces via AWS PrivateLink. When set to true, must also set VPCSecurityGroupIds and VPCSubnetIds.
   DdUseVPC:
     Type: String
     Default: false
@@ -205,11 +198,11 @@ Parameters:
   VPCSecurityGroupIds:
     Type: CommaDelimitedList
     Default: ""
-    Description: Comma separated list of VPC Security Group Ids. Used when DdUsePrivateLink or DdUseVPC is enabled.
+    Description: Comma separated list of VPC Security Group Ids. Used when DdUseVPC is enabled.
   VPCSubnetIds:
     Type: CommaDelimitedList
     Default: ""
-    Description: Comma separated list of VPC Subnet Ids. Used when DdUsePrivateLink or DdUseVPC is enabled.
+    Description: Comma separated list of VPC Subnet Ids. Used when DdUseVPC is enabled.
   DdCompressionLevel:
     Type: Number
     Default: 6
@@ -329,19 +322,16 @@ Conditions:
       - !Equals [!Ref DdFetchLogGroupTags, true]
       - !Equals [!Ref DdFetchLambdaTags, true]
     - !Equals [!Ref DdForwarderExistingBucketName, ""]
-  SetDdUsePrivateLink: !Equals [!Ref DdUsePrivateLink, true]
   SetDdUseVPC: !Equals [!Ref DdUseVPC, true]
   SetDdHttpProxyURL: !Not
     - !Equals [!Ref DdHttpProxyURL, ""]
   SetDdNoProxy: !Not
     - !Equals [!Ref DdNoProxy, ""]
   SetLayerARN: !Not
     - !Equals [!Ref LayerARN, ""]
-  UseVPC: !Or
-    - !Condition SetDdUsePrivateLink
-    - !Condition SetDdUseVPC
   SetDdForwardLog: !Equals [!Ref DdForwardLog, false]
-  SetDdStepFunctionsTraceEnabled: !Equals [!Ref DdStepFunctionsTraceEnabled, true]
+  SetDdStepFunctionsTraceEnabled:
+    !Equals [!Ref DdStepFunctionsTraceEnabled, true]
   SetDdUseCompression: !Equals [!Ref DdUseCompression, false]
   SetDdCompressionLevel: !Not
     - !Equals [!Ref DdCompressionLevel, 6]
@@ -422,7 +412,10 @@ Resources:
             - !Ref DdForwarderExistingBucketName
           S3Key: !Sub
             - "aws-dd-forwarder-${DdForwarderVersion}.zip"
-            - {DdForwarderVersion: !FindInMap [Constants, DdForwarder, Version]}
+            - {
+                DdForwarderVersion:
+                  !FindInMap [Constants, DdForwarder, Version],
+              }
         - ZipFile: " "
       MemorySize: !Ref MemorySize
       Runtime: python3.13
@@ -536,12 +529,8 @@ Resources:
             - SetDdMaxWorkers
             - !Ref DdMaxWorkers
             - !Ref AWS::NoValue
-          DD_USE_PRIVATE_LINK: !If
-            - SetDdUsePrivateLink
-            - true
-            - false
           DD_USE_VPC: !If
-            - UseVPC
+            - SetDdUseVPC
             - true
             - false
           HTTP_PROXY: !If
@@ -581,7 +570,7 @@ Resources:
         - !Ref ReservedConcurrency
         - !Ref AWS::NoValue
       VpcConfig: !If
-        - UseVPC
+        - SetDdUseVPC
         - SecurityGroupIds: !If
             - SetVpcSecurityGroupIds
             - !Ref VPCSecurityGroupIds
@@ -684,7 +673,7 @@ Resources:
                   Effect: Allow
                 - !Ref AWS::NoValue
               - !If
-                - UseVPC # Required for Lambda deployed in VPC
+                - SetDdUseVPC # Required for Lambda deployed in VPC
                 - Action:
                     - ec2:CreateNetworkInterface
                     - ec2:DescribeNetworkInterfaces
@@ -821,7 +810,7 @@ Resources:
         - !Ref SourceZipUrl
         - !Sub
           - "https://github.com/DataDog/datadog-serverless-functions/releases/download/aws-dd-forwarder-${DdForwarderVersion}/aws-dd-forwarder-${DdForwarderVersion}.zip"
-          - {DdForwarderVersion: !FindInMap [Constants, DdForwarder, Version]}
+          - { DdForwarderVersion: !FindInMap [Constants, DdForwarder, Version] }
   # The Forwarder's source code is too big to fit the inline code size limit for CloudFormation. In most of AWS
   # partitions and regions, the Forwarder is able to load its source code from a Lambda layer attached to it.
   # In places where Datadog can't/doesn't yet publish Lambda layers, use another Lambda to copy the source code
@@ -1040,7 +1029,6 @@ Metadata:
           - InstallAsLayer
           - LayerARN
           - PermissionsBoundaryArn
-          - DdUsePrivateLink
           - DdUseVPC
           - DdHttpProxyURL
           - DdNoProxy