[SPARK-33786][SQL][3.0] The storage level for a cache should be respected when a table name is altered.

[imback82]

What changes were proposed in this pull request?

This is a back port of #30774.

This PR proposes to retain the cache's storage level when a table name is altered by ALTER TABLE ... RENAME TO ....

Why are the changes needed?

Currently, when a table name is altered, the table's cache is refreshed (if exists), but the storage level is not retained. For example:

        def getStorageLevel(tableName: String): StorageLevel = {
          val table = spark.table(tableName)
          val cachedData = spark.sharedState.cacheManager.lookupCachedData(table).get
          cachedData.cachedRepresentation.cacheBuilder.storageLevel
        }

        Seq(1 -> "a").toDF("i", "j").write.parquet(path.getCanonicalPath)
        sql(s"CREATE TABLE old USING parquet LOCATION '${path.toURI}'")
        sql("CACHE TABLE old OPTIONS('storageLevel' 'MEMORY_ONLY')")
        val oldStorageLevel = getStorageLevel("old")

        sql("ALTER TABLE old RENAME TO new")
        val newStorageLevel = getStorageLevel("new")

oldStorageLevel will be StorageLevel(memory, deserialized, 1 replicas) whereas newStorageLevel will be StorageLevel(disk, memory, deserialized, 1 replicas), which is the default storage level.

Does this PR introduce any user-facing change?

Yes, now the storage level for the cache will be retained.

How was this patch tested?

Added a unit test.

[imback82]

cc @cloud-fan

[SparkQA]

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37465/

[SparkQA]

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37465/

[SparkQA]

Test build #132863 has finished for PR 30793 at commit 01aee31.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[AmplabJenkins]

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/132863/

[cloud-fan]

thanks, merging to 3.0!