Subtle issue representing `null` as unset value, resulting in deprecation warning

[lorengordon]

Terraform Version

❯ terraform version
Terraform v1.2.8
on linux_amd64
+ provider registry.terraform.io/hashicorp/aws v4.28.0

### Terraform Configuration Files

```terraform
variable "enable_classiclink" {
  type    = bool
  default = null
}

resource "aws_vpc" "main" {
  cidr_block         = "10.0.0.0/16"
  enable_classiclink = var.enable_classiclink
}

Debug Output

not comfortable posting this

Expected Behavior

No deprecation warning should be displayed when the input is null, since null should be treated the same as unset.

Actual Behavior

Received the deprecation warning:

│ Warning: Argument is deprecated
│
│   with aws_vpc.main,
│   on main.tf line 8, in resource "aws_vpc" "main":
│    8:   enable_classiclink = var.enable_classiclink
│
│ With the retirement of EC2-Classic the enable_classiclink attribute has been deprecated and will be removed in a future version.

Steps to Reproduce

1. terraform init
2. terraform plan

Additional Context

Slack convo in #hangops: https://hangops.slack.com/archives/C0Z93TPFX/p1662060793011949?thread_ts=1662056311.323019&cid=C0Z93TPFX

References

No response

[lorengordon]

If you set the option to null directly on the resource attribute, then there is no deprecation warning:

resource "aws_vpc" "main" {
  cidr_block         = "10.0.0.0/16"
  enable_classiclink = null
}

[lorengordon]

Attempting to conditionally set enable_classiclink fails to work around it, and still generates the deprecation warning:

resource "aws_vpc" "main" {
  cidr_block         = "10.0.0.0/16"
  enable_classiclink = var.enable_classiclink == true ? true : null
}

[lorengordon]

This also fails to avoid the deprecation warning:

variable "enable_classiclink" {
  type     = bool
  default  = false
  nullable = false
}

resource "aws_vpc" "main" {
  cidr_block         = "10.0.0.0/16"
  enable_classiclink = var.enable_classiclink ? true : null
}

[jbardin]

Hi @lorengordon,

Thanks for reporting the issue. As you can see from the example where null is assigned directly, it's not likely a problem with the fact there is an assignment at all, rather the issue is that the value is coming from a variable which may not be known at the point of validation. This means the problem is on the provider side, rather than within Terraform core. The fact that there is a deprecation message with the warning also indicates the provider is creating the diagnostic output, as that text is not available within Terraform.

To be more specific, the problem lies in the legacy SDK, which is treating the fact that an unknown value is assigned to the attribute as a sign that the attribute is intended to be used. This is likely a fair assumption in most cases, since the warning is there to indicate that the attribute should not be used in the configuration at all. Do you have an example where assigning something to a deprecated field should not issue a warning, or where that warning is causing a problem?

[lorengordon]

I hate to at people, but I was mostly opening the issue at the prompting of @apparentlymart, and the discussion from the slack thread I linked.

I'm rather confused with the explanation, as there are no dynamic or unknown values here. The null assignment is entirely known. Why would a known null value be different when coming from a variable default or user input, vs when assigned directly to a resource attribute in the config?

It seems like these two things should behave exactly the same, when no variables are passed to the first config:

variable "enable_classiclink" {
  type    = bool
  default = null
}

resource "aws_vpc" "main" {
  cidr_block         = "10.0.0.0/16"
  enable_classiclink = var.enable_classiclink
}

resource "aws_vpc" "main" {
  cidr_block         = "10.0.0.0/16"
  enable_classiclink = null
}

[jbardin]

OK, I see the confusion now. Config validation is done statically, which means variables are inherently unknown at that point since nothing has yet been evaluated.

Most validation is done within Terraform core, but deprecated attributes are not handled within Terraform, those are determined by the provider, so in this case the provider can only see that the attribute has been set in the config when the unknown variable is assigned. I would say the irregularity here is that we don't currently warn about the deprecated field assignment when null is assigned directly.

[lorengordon]

Gotcha, that makes more sense. It would certainly be rather nice to be able to avoid the deprecation warning just by passing null to the argument, but having consistent behavior would help reduce confusion either way, and make it more clear what users need to do here.

[apparentlymart]

Thanks for reporting this @lorengordon, and thanks @jbardin for looking into the details.

When we were discussing it (in HangOps Slack) I knew something wasn't right here but I wasn't sure what, so I asked Loren to open this issue so we would have a chance to look into it. Unfortunately I've been away from code most of the afternoon so I wasn't able to look deeper immediately, but I'm glad to see the confirmation that the deprecation warning logic does indeed still live inside the provider, since I thought that was true but wasn't sure if I'd forgotten a change of responsibilities at some point.

With this explanation, it seems like what we have here is really a bug in the Terraform SDK, rather than in Terraform Core: the usual rule for validation is that you skip validation if you receive an unknown value, and assume that you'll get an opportunity to validate some more in a later phase when Terraform Core has more information. The SDK here seems to be checking the deprecation warning even if the value is unknown, which causes it to report a false positive. If it had instead skipped that validation as expected, then the provider would've had an opportunity to catch it during the planning phase instead, once that input variable becomes a known null.

Unfortunately at first glance the SDK code seems to disprove that theory: https://github.com/hashicorp/terraform-plugin-sdk/blob/fa8d313665945816f6eb6c79182e4abdc1540504/helper/schema/schema.go#L1737-L1747

We can see it there first checking whether the value is wholly known, exactly as I would've expected. Seems like something more subtle is going on. Hopefully we can dig a little deeper to confirm it's the SDK doing this and then, if so, understand a bit more about why before forwarding this bug on to the SDK team. 😖