[cxx-interop] Fix metadata mismatch regarding fields of structs #81035

susmonteiro · 2025-04-23T17:14:51Z

In #78467 and #78961, we stopped emitting metadata for private C++ fields. However, this created a mismatch between the fields emitted and the number of fields + their offsets in the StructDescriptor.

rdar://147263490

susmonteiro · 2025-04-23T17:15:06Z

@swift-ci please smoke test

susmonteiro · 2025-04-24T11:00:40Z

@swift-ci please smoke test

Xazax-hun

This is great, thanks for tracking this down! I am not super confident about some details, asked some questions inline. Maybe we should get someone more familiar with type metadata reviewed this PR.

Xazax-hun · 2025-04-24T13:41:49Z

lib/IRGen/GenMeta.cpp

@@ -1845,7 +1845,7 @@ namespace {

    void addLayoutInfo() {
      // uint32_t NumFields;
-      B.addInt32(getNumFields(getType()));


I wonder if we need to do something similar for ClassContextDescriptorBuilder too.

I am less sure about emitInitializeFieldOffsetVector and co. Maybe those are good as is but maybe we want to ask someone familiar with type metadata.

I wonder if we need to do something similar for ClassContextDescriptorBuilder too.

Yes I think you're right. I intend to check if we need a similar fix for classes and add those changes in a future PR

Regarding emitInitializeFieldOffsetVector, I'm also not sure. I'll investigate this further

emitInitializeFieldOffsetVector should include all fields. We are using that information to handle values in value witness functions.

@drexin would it be a problem to have a discrepancy between the fields in emitInitializeFieldOffsetVector and the fields in StructContextDescriptorBuilder/FieldTypeMetadataBuilder?

Hm, I am not sure. cc: @mikeash

Reflection relies on the indexes in the field offset vector and the indexes of the descriptor's fields lining up. In theory you could get away with having additional entries at the end of the field offset vector, but not in the middle, and not more field records than field offsets.

Xazax-hun · 2025-04-24T13:45:37Z

lib/IRGen/StructMetadataVisitor.h

@@ -64,7 +65,8 @@ template <class Impl> class StructMetadataVisitor
    // Struct field offsets.
    asImpl().noteStartOfFieldOffsets();
    for (VarDecl *prop : Target->getStoredProperties())
-      asImpl().addFieldOffset(prop);
+      if (!isPrivateField(prop))


I wonder if we can get out of sync with emitInitializeFieldOffsetVector here.

susmonteiro · 2025-04-24T14:49:47Z

@swift-ci please smoke test

susmonteiro · 2025-04-25T11:52:50Z

@swift-ci please smoke test

Xazax-hun · 2025-04-25T13:06:40Z

test/Interop/Cxx/stdlib/print-stdlib-types.swift

+    // CHECK-linux-gnu: 42
+    // CHECK-linux-gnu: shared_ptr<CInt>()
+printStdSharedPtrOfString()
+// CHECK-macosx: basic_string<CChar, std.__1.char_traits<CChar>, std.__1.allocator<CChar>>()


Alternatively, you could add regexes that match on all platforms. I don't have a preference.

Thanks for the suggestion!
Hopefully we will soon fix the conformance of std::string to CustomStringConvertible. Then we can remove all these os options, since the value of the string will be printed instead of the type name.
That said, I don't mind switching to regex

Good point, the string one is temporary (maybe not the shared_ptr one though). Let's leave it as is.

I think using regexes would be a bit more reliable here, since that would ensure the correct behavior on platforms that aren't listed here, e.g. Android or FreeBSD. The regex could be really permissive: something like {{.*}}.basic_string<CChar, {{.*}}> should do the trick.

Sure, I'll make this change.

susmonteiro · 2025-04-25T15:26:20Z

@swift-ci please smoke test

Xazax-hun · 2025-04-25T15:31:35Z

test/Interop/Cxx/stdlib/print-stdlib-types.swift

+    // CHECK-linux-gnu: 42
+    // CHECK-linux-gnu: shared_ptr<CInt>()
+printStdSharedPtrOfString()
+// CHECK-macosx: basic_string<CChar, std.__1.char_traits<CChar>, std.__1.allocator<CChar>>()


Good point, the string one is temporary (maybe not the shared_ptr one though). Let's leave it as is.

Misclicked.

susmonteiro · 2025-04-25T15:43:48Z

@swift-ci please smoke test

susmonteiro · 2025-04-28T09:12:25Z

@swift-ci Please test macOS platform

egorzhdan

This looks great, thanks!
I only have a couple of comments on testing.

test/Interop/Cxx/stdlib/Inputs/stdlib-types.h

egorzhdan · 2025-05-06T12:21:31Z

test/Interop/Cxx/stdlib/print-stdlib-types.swift

+    // CHECK-linux-gnu: 42
+    // CHECK-linux-gnu: shared_ptr<CInt>()
+printStdSharedPtrOfString()
+// CHECK-macosx: basic_string<CChar, std.__1.char_traits<CChar>, std.__1.allocator<CChar>>()


I think using regexes would be a bit more reliable here, since that would ensure the correct behavior on platforms that aren't listed here, e.g. Android or FreeBSD. The regex could be really permissive: something like {{.*}}.basic_string<CChar, {{.*}}> should do the trick.

susmonteiro · 2025-05-07T15:19:16Z

@swift-ci please smoke test

susmonteiro · 2025-05-14T15:46:00Z

@swift-ci please smoke test macOS platform

susmonteiro · 2025-05-15T15:28:37Z

@swift-ci please smoke test

drexin

With the changes this looks good to me.

lib/IRGen/StructLayout.cpp

lib/IRGen/StructMetadataVisitor.h

j-hui · 2025-05-15T17:23:24Z

test/Interop/Cxx/class/Inputs/simple-structs.h

+
+public:
+  HasPrivateFieldsOnly(int i, std::string s) : privInt(i), privStr(s) {}
+};


I recommend also testing:

some records with protected access (just to be thorough; these are also imported as private)

some structs with inherited fields, covering public/protected/private inheritance + using BaseClass::protected_field;

A computed property synthesized from SWIFT_COMPUTED_PROPERTY

You might be able to reuse some of the structs I defined in /Users/j-hui/Documents/open/swift/test/Interop/Cxx/class/access/Inputs/{non-public.h,non-public-inheritance.h,using-non-public.h} for testing, for the latter two I primarily focused on methods. If you throw in some fields you should be able to reuse the inheritance patterns I cover there.

I added some tests for records with protected access. Regarding the rest of the tests you suggested:

some structs with inherited fields, covering public/protected/private inheritance + using BaseClass::protected_field;

Currently we don't print inherited fields. It's something we want to fix, but for now I decided not to add tests for this.

A computed property synthesized from SWIFT_COMPUTED_PROPERTY

These are also not getting printed, just like computed properties in Swift code.

j-hui · 2025-05-15T17:25:53Z

test/Interop/Cxx/class/Inputs/simple-structs.h

+#ifndef TEST_INTEROP_CXX_CLASS_INPUTS_SIMPLE_STRUCTS_H
+#define TEST_INTEROP_CXX_CLASS_INPUTS_SIMPLE_STRUCTS_H
+
+#include <string>


nit: If we're trying to keep these structs simple in this test, maybe we should stick to primitive data types, and not have this test depend on the implementation of std::string? We should still absolutely test that but I think that should happen in a different test.

That said, do test nested printing, i.e., use a struct Inner that has a mix of public and private fields as the fields of another struct Outer, and try printing Outer.

nit: since you're no longer using std::string in this header file, you can probably delete this #include statement now 🙂

j-hui · 2025-05-15T17:27:35Z

test/Interop/Cxx/class/print.swift

+
+func printCxxStructPrivateAndPublicFields() {
+    let s = HasPrivateAndPublicFields(24, 42, "Hello", "World")
+    print(s)


Are there any other operations other than printing that relies on metadata? Perhaps we should test those too.

Not sure, I'm not very familiar with metadata. Do you have any operations in mind?

print covers it pretty well if the type doesn't conform to any of the pretty-printing protocols. You might also test _forEachFieldWithKeyPath.

I'll take a look at that, thanks!

lib/IRGen/StructLayout.cpp

susmonteiro · 2025-05-19T13:07:26Z

@swift-ci please smoke test

susmonteiro · 2025-05-19T17:46:16Z

@swift-ci please smoke test

j-hui

This looks great, thanks for addressing my comments @susmonteiro ! There's just a few nits concerning the test setup, but those can be fixed in a follow-up if you'd like.

j-hui · 2025-05-19T21:45:42Z

test/Interop/Cxx/class/Inputs/simple-structs.h

+#ifndef TEST_INTEROP_CXX_CLASS_INPUTS_SIMPLE_STRUCTS_H
+#define TEST_INTEROP_CXX_CLASS_INPUTS_SIMPLE_STRUCTS_H
+
+#include <string>


nit: since you're no longer using std::string in this header file, you can probably delete this #include statement now 🙂

j-hui · 2025-05-19T21:47:24Z

test/Interop/Cxx/class/print-simple-structs.swift

@@ -0,0 +1,40 @@
+// RUN: %empty-directory(%t)
+// RUN: %target-build-swift %s -o %t/a.out -Xfrontend -enable-experimental-cxx-interop -I %S/Inputs


Suggested change

// RUN: %target-build-swift %s -o %t/a.out -Xfrontend -enable-experimental-cxx-interop -I %S/Inputs

// RUN: %target-build-swift %s -o %t/a.out -Xfrontend -cxx-interoperability-mode=default -I %S/Inputs

j-hui · 2025-05-19T21:48:32Z

test/Interop/Cxx/class/print-simple-structs.swift

@@ -0,0 +1,40 @@
+// RUN: %empty-directory(%t)


nit: is there any reason not to simplify this to something like?

// RUN: %target-run-simple-swift(-cxx-interoperability-mode=default -Xfrontend -disable-availability-checking -I %S/Inputs) | %FileCheck %s

No, there isn't. Thanks for the suggestion!

j-hui · 2025-05-19T21:50:07Z

test/Interop/Cxx/stdlib/print-stdlib-types.swift

@@ -0,0 +1,90 @@
+// RUN: %empty-directory(%t)


same question about using %target-run-simple-swift here

j-hui · 2025-05-19T21:51:14Z

test/Interop/Cxx/stdlib/print-stdlib-types.swift

@@ -0,0 +1,90 @@
+// RUN: %empty-directory(%t)
+// RUN: %target-build-swift %s -o %t/a.out -Xfrontend -enable-experimental-cxx-interop -I %S/Inputs


nit:

Suggested change

// RUN: %target-build-swift %s -o %t/a.out -Xfrontend -enable-experimental-cxx-interop -I %S/Inputs

// RUN: %target-build-swift %s -o %t/a.out -Xfrontend -cxx-interoperability-mode=default -I %S/Inputs

susmonteiro · 2025-05-20T09:17:31Z

@swift-ci please smoke test

susmonteiro · 2025-05-21T09:48:40Z

@swift-ci please smoke test

susmonteiro · 2025-05-21T13:55:27Z

@swift-ci please smoke test windows platform

…/metadata-private-fields" This reverts commit e805d93.

Adds support for printing a C++ foreign reference in Swift. Also skips metadata of private fields in C++ records imported as Swift classes, following up on #81035 (cherry-picked from 848fad0)

susmonteiro added the c++ interop Feature: Interoperability with C++ label Apr 23, 2025

susmonteiro force-pushed the susmonteiro/metadata-private-fields branch from f36ff56 to a39010b Compare April 24, 2025 10:59

susmonteiro marked this pull request as ready for review April 24, 2025 11:03

susmonteiro requested review from zoecarver, egorzhdan, Xazax-hun, j-hui, fahadnayyar, hnrklssn and rjmccall as code owners April 24, 2025 11:03

Xazax-hun reviewed Apr 24, 2025

View reviewed changes

susmonteiro force-pushed the susmonteiro/metadata-private-fields branch from a39010b to a49738e Compare April 24, 2025 14:49

susmonteiro force-pushed the susmonteiro/metadata-private-fields branch from a49738e to abd2549 Compare April 25, 2025 11:52

Xazax-hun reviewed Apr 25, 2025

View reviewed changes

susmonteiro force-pushed the susmonteiro/metadata-private-fields branch from abd2549 to 2a99a7d Compare April 25, 2025 15:25

Xazax-hun previously approved these changes Apr 25, 2025

View reviewed changes

susmonteiro force-pushed the susmonteiro/metadata-private-fields branch from 2a99a7d to bdacdd0 Compare April 25, 2025 15:42

egorzhdan requested a review from tbkka May 1, 2025 14:37

egorzhdan reviewed May 6, 2025

View reviewed changes

susmonteiro force-pushed the susmonteiro/metadata-private-fields branch from bdacdd0 to 3efa921 Compare May 7, 2025 15:18

susmonteiro force-pushed the susmonteiro/metadata-private-fields branch from 3efa921 to f21499c Compare May 14, 2025 15:44

drexin approved these changes May 15, 2025

View reviewed changes

j-hui reviewed May 15, 2025

View reviewed changes

susmonteiro force-pushed the susmonteiro/metadata-private-fields branch from f21499c to 2b59c48 Compare May 19, 2025 13:06

egorzhdan approved these changes May 19, 2025

View reviewed changes

susmonteiro force-pushed the susmonteiro/metadata-private-fields branch from 2b59c48 to eca94bf Compare May 19, 2025 17:41

j-hui approved these changes May 19, 2025

View reviewed changes

[cxx-interop] Fix metadata mismatch regarding fields of structs

72b13b3

susmonteiro force-pushed the susmonteiro/metadata-private-fields branch from eca94bf to 72b13b3 Compare May 20, 2025 09:15

susmonteiro enabled auto-merge May 20, 2025 09:19

susmonteiro disabled auto-merge May 20, 2025 09:19

susmonteiro enabled auto-merge May 20, 2025 09:57

susmonteiro merged commit e805d93 into main May 21, 2025
3 checks passed

susmonteiro deleted the susmonteiro/metadata-private-fields branch May 21, 2025 18:25

hamishknight added a commit to hamishknight/swift that referenced this pull request May 22, 2025

Revert "Merge pull request swiftlang#81035 from swiftlang/susmonteiro…

3fb8e2a

…/metadata-private-fields" This reverts commit e805d93.

susmonteiro mentioned this pull request May 30, 2025

[cxx-interop] Passing std.vector to print causes EXC_BAD_ACCESS #81862

Closed

		@@ -0,0 +1,40 @@
		// RUN: %empty-directory(%t)
		// RUN: %target-build-swift %s -o %t/a.out -Xfrontend -enable-experimental-cxx-interop -I %S/Inputs

		@@ -0,0 +1,90 @@
		// RUN: %empty-directory(%t)
		// RUN: %target-build-swift %s -o %t/a.out -Xfrontend -enable-experimental-cxx-interop -I %S/Inputs

[cxx-interop] Fix metadata mismatch regarding fields of structs #81035

[cxx-interop] Fix metadata mismatch regarding fields of structs #81035

Uh oh!

Conversation

susmonteiro commented Apr 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

susmonteiro commented Apr 23, 2025

Uh oh!

susmonteiro commented Apr 24, 2025

Uh oh!

Xazax-hun left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

susmonteiro commented Apr 24, 2025

Uh oh!

susmonteiro commented Apr 25, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

susmonteiro commented Apr 25, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

susmonteiro commented Apr 25, 2025

Uh oh!

susmonteiro commented Apr 28, 2025

Uh oh!

egorzhdan left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

susmonteiro commented May 7, 2025

Uh oh!

susmonteiro commented May 14, 2025

Uh oh!

susmonteiro commented May 15, 2025

Uh oh!

drexin left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

susmonteiro commented Apr 23, 2025 •

edited

Loading