Skip to content

Reapply "[Clang][Sema] placement new initializes typedef array with correct size (#83124)" #89036

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Apr 23, 2024

Conversation

mahtohappy
Copy link
Contributor

When in-place new-ing a local variable of an array of trivial type, the generated code calls 'memset' with the correct size of the array, earlier it was generating size (squared of the typedef array + size).

The cause: typedef TYPE TArray[8]; TArray x; The type of declarator is Tarray[8] and in SemaExprCXX.cpp::BuildCXXNew we check if it's of typedef and of constant size then we get the original type and it works fine for non-dependent cases.
But in case of template we do TreeTransform.h:TransformCXXNEWExpr and there we again check the allocated type which is TArray[8] and it stays that way, so ArraySize=(Tarray[8] type, alloc Tarray[8*type]) so the squared size allocation.

ArraySize gets calculated earlier in TreeTransform.h so that if(!ArraySize) condition was failing.
fix: I changed that condition to if(ArraySize).
fixes #41441

mahtohappy and others added 3 commits April 17, 2024 00:48
…ze (llvm#83124)

When in-place new-ing a local variable of an array of trivial type, the
generated code calls 'memset' with the correct size of the array,
earlier it was generating size (squared of the typedef array + size).

The cause: `typedef TYPE TArray[8]; TArray x;` The type of declarator is
Tarray[8] and in `SemaExprCXX.cpp::BuildCXXNew` we check if it's of
typedef and of constant size then we get the original type and it works
fine for non-dependent cases.
But in case of template we do `TreeTransform.h:TransformCXXNEWExpr` and
there we again check the allocated type which is TArray[8] and it stays
that way, so ArraySize=(Tarray[8] type, alloc Tarray[8*type]) so the
squared size allocation.

ArraySize gets calculated earlier in `TreeTransform.h` so that
`if(!ArraySize)` condition was failing.
fix: I changed that condition to `if(ArraySize)`. 


Fixes llvm#41441
This just replaces an '#include<new>' with a declaration of array
placement new.
@llvmbot llvmbot added clang Clang issues not falling into any other category clang:frontend Language frontend issues, e.g. anything involving "Sema" labels Apr 17, 2024
@llvmbot
Copy link
Member

llvmbot commented Apr 17, 2024

@llvm/pr-subscribers-clang-modules

@llvm/pr-subscribers-clang

Author: None (mahtohappy)

Changes

When in-place new-ing a local variable of an array of trivial type, the generated code calls 'memset' with the correct size of the array, earlier it was generating size (squared of the typedef array + size).

The cause: typedef TYPE TArray[8]; TArray x; The type of declarator is Tarray[8] and in SemaExprCXX.cpp::BuildCXXNew we check if it's of typedef and of constant size then we get the original type and it works fine for non-dependent cases.
But in case of template we do TreeTransform.h:TransformCXXNEWExpr and there we again check the allocated type which is TArray[8] and it stays that way, so ArraySize=(Tarray[8] type, alloc Tarray[8*type]) so the squared size allocation.

ArraySize gets calculated earlier in TreeTransform.h so that if(!ArraySize) condition was failing.
fix: I changed that condition to if(ArraySize).
fixes #41441


Full diff: https://github.com/llvm/llvm-project/pull/89036.diff

3 Files Affected:

  • (modified) clang/docs/ReleaseNotes.rst (+2)
  • (modified) clang/lib/Sema/TreeTransform.h (+13-1)
  • (added) clang/test/SemaCXX/PR41441.cpp (+23)
diff --git a/clang/docs/ReleaseNotes.rst b/clang/docs/ReleaseNotes.rst
index 96ad92b540b47f..636d76f1836759 100644
--- a/clang/docs/ReleaseNotes.rst
+++ b/clang/docs/ReleaseNotes.rst
@@ -534,6 +534,8 @@ Bug Fixes to C++ Support
   Fixes (#GH70604), (#GH79754), (#GH84163), (#GH84425), (#GH86054), (#GH86398), and (#GH86399).
 - Fix a crash when deducing ``auto`` from an invalid dereference (#GH88329).
 - Fix a crash in requires expression with templated base class member function. Fixes (#GH84020).
+- placement new initializes typedef array with correct size
+  (`#GH41441 <https://github.com/llvm/llvm-project/issues/41441>`_)
 
 Bug Fixes to AST Handling
 ^^^^^^^^^^^^^^^^^^^^^^^^^
diff --git a/clang/lib/Sema/TreeTransform.h b/clang/lib/Sema/TreeTransform.h
index eb05783a6219dc..0c7fdb357235e1 100644
--- a/clang/lib/Sema/TreeTransform.h
+++ b/clang/lib/Sema/TreeTransform.h
@@ -12864,6 +12864,19 @@ TreeTransform<Derived>::TransformCXXNewExpr(CXXNewExpr *E) {
     ArraySize = NewArraySize.get();
   }
 
+  // Per C++0x [expr.new]p5, the type being constructed may be a
+  // typedef of an array type.
+  QualType AllocType = AllocTypeInfo->getType();
+  if (ArraySize) {
+    if (const ConstantArrayType *Array =
+            SemaRef.Context.getAsConstantArrayType(AllocType)) {
+      ArraySize = IntegerLiteral::Create(SemaRef.Context, Array->getSize(),
+                                         SemaRef.Context.getSizeType(),
+                                         E->getBeginLoc());
+      AllocType = Array->getElementType();
+    }
+  }
+
   // Transform the placement arguments (if any).
   bool ArgumentChanged = false;
   SmallVector<Expr*, 8> PlacementArgs;
@@ -12925,7 +12938,6 @@ TreeTransform<Derived>::TransformCXXNewExpr(CXXNewExpr *E) {
     return E;
   }
 
-  QualType AllocType = AllocTypeInfo->getType();
   if (!ArraySize) {
     // If no array size was specified, but the new expression was
     // instantiated with an array type (e.g., "new T" where T is
diff --git a/clang/test/SemaCXX/PR41441.cpp b/clang/test/SemaCXX/PR41441.cpp
new file mode 100644
index 00000000000000..d0f2917e52f211
--- /dev/null
+++ b/clang/test/SemaCXX/PR41441.cpp
@@ -0,0 +1,23 @@
+// RUN: %clang --target=x86_64-pc-linux -S -fno-discard-value-names -emit-llvm -o - %s | FileCheck %s
+
+namespace std {
+  using size_t = decltype(sizeof(int));
+};
+void* operator new[](std::size_t, void*) noexcept;
+
+// CHECK: call void @llvm.memset.p0.i64(ptr align 1 %x, i8 0, i64 8, i1 false)
+// CHECK: call void @llvm.memset.p0.i64(ptr align 16 %x, i8 0, i64 32, i1 false)
+template <typename TYPE>
+void f()
+{
+    typedef TYPE TArray[8];
+
+    TArray x;
+    new(&x) TArray();
+}
+
+int main()
+{
+    f<char>();
+    f<int>();
+}

@Sirraide Sirraide changed the title [Clang][Sema] placement new initializes typedef array with correct size Reapply "[Clang][Sema] placement new initializes typedef array with correct size (#83124)" Apr 17, 2024
Copy link
Collaborator

@erichkeane erichkeane left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The original patch pointed out a regression:

#83124 (comment)

Please contact that person and get a reproducer and make sure you aren't breaking them.

@llvmbot llvmbot added the clang:modules C++20 modules and Clang Header Modules label Apr 19, 2024
@dwblaikie
Copy link
Collaborator

The original patch pointed out a regression:

#83124 (comment)

Please contact that person and get a reproducer and make sure you aren't breaking them.

Repro was provided further down: #83124 (comment)

Please include test coverage and a fix before approval/recommit.

@mahtohappy
Copy link
Contributor Author

Yes it's fixed now.

@dwblaikie
Copy link
Collaborator

Yes it's fixed now.

Please include details of the fix I the patch description

@mahtohappy
Copy link
Contributor Author

Check is for dependent types and earlier I was not checking that with the condition being only
if (ArraySize) so normal arrays with new initializer also were passing the check.
Now added the dependent type check as well
if (ArraySize && E->isTypeDependent())

Copy link
Collaborator

@erichkeane erichkeane left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where is the repro from the original author? What did they share, and what ended up being the solution/test here for it?

// Per C++0x [expr.new]p5, the type being constructed may be a
// typedef of an array type.
QualType AllocType = AllocTypeInfo->getType();
if (ArraySize && E->isTypeDependent()) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does Value dependence matter?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it does. Both the original testcase from @slackito and simpler version of @dwblaikie are working now.
normal arrays with new initializer also were passing the check and their type was correct earlier so need for modification for them. typedef TYPE TArray[8]; TArray x; here type of array was Tarray which should be Type* so the check only for type dependent case.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note I asked about VALUE dependence, not TYPE dependence. I see why type dependence would matter, but expressions can be both type and value dependent.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, it doesn't have value dependence.

@@ -0,0 +1,23 @@
// RUN: %clang --target=x86_64-pc-linux -S -fno-discard-value-names -emit-llvm -o - %s | FileCheck %s

namespace std {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tests STILL haven't changed from the original patch. Please add the regression to this test as well.

@mahtohappy
Copy link
Contributor Author

Hi, the build is failing for windows but there's not test failures and no errors in the log. What should I do from here?

@mahtohappy
Copy link
Contributor Author

Hi @cor3ntin Please merge this.

@mahtohappy
Copy link
Contributor Author

Hi @erichkeane Please merge this change.

@erichkeane erichkeane merged commit 74cab54 into llvm:main Apr 23, 2024
5 checks passed
@ilovepi
Copy link
Contributor

ilovepi commented Apr 23, 2024

I think we're seeing some build failures after this patch, and it isn't clear to me that this is a bug in the source, so I'd appreciate it if you could take a look.

FAILED: obj/src/media/audio/tools/signal_generator/signal_generator.signal_generator.cc.o 
../../prebuilt/third_party/clang/custom/bin/clang++ -MD -MF obj/src/media/audio/tools/signal_generator/signal_generator.signal_generator.cc.o.d -D_LIBCPP_DISABLE_DEPRECATION_WARNINGS -D_LIBCPP_DISABLE_VISIBILITY_ANNOTATIONS -D_LIBCPP_REMOVE_TRANSITIVE_INCLUDES -D_LIBCPP_ENABLE_THREAD_SAFETY_ANNOTATIONS=1 -DZX_ASSERT_LEVEL=2 -I../.. -Igen -Ifidling/gen/sdk/fidl/fuchsia.media/fuchsia.media/hlcpp -Ifidling/gen/sdk/fidl/fuchsia.hardware.audio/fuchsia.hardware.audio/hlcpp -Ifidling/gen/sdk/fidl/fuchsia.hardware.audio.signalprocessing/fuchsia.hardware.audio.signalprocessing/hlcpp -I../../sdk -Igen/sdk -I../../sdk/lib/fidl_base/include -I../../src/zircon/lib/zircon/include -Ifidling/gen/zircon/vdso/zx/zither/legacy_syscall_cdecl -I../../sdk/lib/fit/include -I../../sdk/lib/stdcompat/include -I../../sdk/lib/fit-promise/include -I../../sdk/lib/fidl/include -I../../zircon/system/ulib/zx/include -I../../zircon/system/ulib/async/include -I../../zircon/system/ulib/async-default/include -Ifidling/gen/sdk/fidl/fuchsia.images/fuchsia.images/hlcpp -Ifidling/gen/sdk/fidl/fuchsia.sysmem/fuchsia.sysmem/hlcpp -Ifidling/gen/sdk/fidl/fuchsia.io/fuchsia.io/hlcpp -Ifidling/gen/sdk/fidl/fuchsia.unknown/fuchsia.unknown/hlcpp -Ifidling/gen/sdk/fidl/fuchsia.sysmem2/fuchsia.sysmem2/hlcpp -Ifidling/gen/sdk/fidl/fuchsia.images2/fuchsia.images2/hlcpp -Ifidling/gen/sdk/fidl/fuchsia.math/fuchsia.math/hlcpp -Ifidling/gen/sdk/fidl/fuchsia.media.audio/fuchsia.media.audio/hlcpp -Ifidling/gen/sdk/fidl/fuchsia.ultrasound/fuchsia.ultrasound/hlcpp -I../../sdk/lib/fidl/cpp/wire/include -I../../zircon/system/ulib/sync/include -I../../sdk/lib/fdio/include -I../../zircon/system/public -I../../zircon/system/ulib/fbl/include -I../../third_party/googletest/src/googletest/include -I../../zircon/system/ulib/affine/include -I../../zircon/system/ulib/zircon-internal/include -I../../third_party/re2/src -I../../zircon/system/ulib/async-loop/include -I../../zircon/system/ulib/fzl/include -fcolor-diagnostics -fcrash-diagnostics-dir=clang-crashreports -fcrash-diagnostics=all -gen-reproducer=error -ffp-contract=off --sysroot=gen/zircon/public/sysroot/cpp --target=x86_64-unknown-fuchsia -ffuchsia-api-level=4294967295 -march=x86-64-v2 -mtune=generic -mbranches-within-32B-boundaries -ffile-compilation-dir=. -no-canonical-prefixes -fno-omit-frame-pointer -momit-leaf-frame-pointer -fdata-sections -ffunction-sections -O0 -Xclang -debug-info-kind=constructor -g3 -grecord-gcc-switches -gdwarf-5 -gz=zstd -Wall -Wextra -Wconversion -Wextra-semi -Wimplicit-fallthrough -Wnewline-eof -Wstrict-prototypes -Wwrite-strings -Wno-sign-conversion -Wno-unused-parameter -Wnonportable-system-include-path -Wno-missing-field-initializers -Wno-extra-qualification -Wno-cast-function-type-strict -Wno-cast-function-type-mismatch -Wno-unknown-warning-option -Wno-deprecated-pragma -fvisibility=hidden -Werror -Wa,--fatal-warnings -ftrivial-auto-var-init=pattern -Wthread-safety -Wno-unknown-warning-option -Wno-thread-safety-reference-return -Wno-undef -fvisibility-inlines-hidden -std=c++17 -fno-exceptions -fno-rtti -ftemplate-backtrace-limit=0 -c ../../src/media/audio/tools/signal_generator/signal_generator.cc -o obj/src/media/audio/tools/signal_generator/signal_generator.signal_generator.cc.o
In file included from ../../src/media/audio/tools/signal_generator/signal_generator.cc:4:
In file included from ../../src/media/audio/tools/signal_generator/signal_generator.h:7:
In file included from fidling/gen/sdk/fidl/fuchsia.media/fuchsia.media/hlcpp/fuchsia/media/cpp/fidl.h:6:
In file included from ../../sdk/lib/fidl/cpp/internal/header.h:11:
In file included from ../../sdk/lib/fit/include/lib/fit/function.h:9:
In file included from ../../prebuilt/third_party/clang/custom/bin/../include/c++/v1/memory:937:
In file included from ../../prebuilt/third_party/clang/custom/bin/../include/c++/v1/__memory/shared_ptr.h:32:
../../prebuilt/third_party/clang/custom/bin/../include/c++/v1/__memory/unique_ptr.h:627:10: error: no matching conversion for functional-style cast from 'double *' to 'unique_ptr<double[][4]>'
  627 |   return unique_ptr<_Tp>(new _Up[__n]());
      |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../../src/media/audio/tools/signal_generator/signal_generator.cc:803:25: note: in instantiation of function template specialization 'std::make_unique<double[][4]>' requested here
  803 |   input_history_ = std::make_unique<HistoryBuffer[]>(num_channels_);
      |                         ^
../../prebuilt/third_party/clang/custom/bin/../include/c++/v1/__memory/unique_ptr.h:125:59: note: candidate constructor (the implicit copy constructor) not viable: no known conversion from 'double *' to 'const unique_ptr<double[][4]>' for 1st argument
  125 | class _LIBCPP_UNIQUE_PTR_TRIVIAL_ABI _LIBCPP_TEMPLATE_VIS unique_ptr {
      |                                                           ^~~~~~~~~~
../../prebuilt/third_party/clang/custom/bin/../include/c++/v1/__memory/unique_ptr.h:355:43: note: candidate constructor template not viable: no known conversion from 'double *' to 'nullptr_t' (aka 'std::nullptr_t') for 1st argument
  355 |   _LIBCPP_HIDE_FROM_ABI _LIBCPP_CONSTEXPR unique_ptr(nullptr_t) _NOEXCEPT
      |                                           ^          ~~~~~~~~~
../../prebuilt/third_party/clang/custom/bin/../include/c++/v1/__memory/unique_ptr.h:397:55: note: candidate constructor not viable: no known conversion from 'double *' to 'unique_ptr<double[][4]>' for 1st argument
  397 |   _LIBCPP_HIDE_FROM_ABI _LIBCPP_CONSTEXPR_SINCE_CXX23 unique_ptr(unique_ptr&& __u) _NOEXCEPT
      |                                                       ^          ~~~~~~~~~~~~~~~~
../../prebuilt/third_party/clang/custom/bin/../include/c++/v1/__memory/unique_ptr.h:362:64: note: candidate template ignored: requirement '_CheckArrayPointerConversion<double *>::value' was not satisfied [with _Pp = double *, _Dummy = true, $2 = _EnableIfDeleterDefaultConstructible<true>]
  362 |   _LIBCPP_HIDE_FROM_ABI _LIBCPP_CONSTEXPR_SINCE_CXX23 explicit unique_ptr(_Pp __p) _NOEXCEPT
      |                                                                ^
../../prebuilt/third_party/clang/custom/bin/../include/c++/v1/__memory/unique_ptr.h:410:55: note: candidate template ignored: could not match 'unique_ptr<_Up, _Ep>' against 'double *'
  410 |   _LIBCPP_HIDE_FROM_ABI _LIBCPP_CONSTEXPR_SINCE_CXX23 unique_ptr(unique_ptr<_Up, _Ep>&& __u) _NOEXCEPT
      |                                                       ^
../../prebuilt/third_party/clang/custom/bin/../include/c++/v1/__memory/unique_ptr.h:352:43: note: candidate constructor template not viable: requires 0 arguments, but 1 was provided
  352 |   _LIBCPP_HIDE_FROM_ABI _LIBCPP_CONSTEXPR unique_ptr() _NOEXCEPT : __ptr_(__value_init_tag(), __value_init_tag()) {}
      |                                           ^
../../prebuilt/third_party/clang/custom/bin/../include/c++/v1/__memory/unique_ptr.h:369:55: note: candidate constructor template not viable: requires 2 arguments, but 1 was provided
  369 |   _LIBCPP_HIDE_FROM_ABI _LIBCPP_CONSTEXPR_SINCE_CXX23 unique_ptr(_Pp __p, _LValRefType<_Dummy> __d) _NOEXCEPT
      |                                                       ^          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../../prebuilt/third_party/clang/custom/bin/../include/c++/v1/__memory/unique_ptr.h:373:55: note: candidate constructor template not viable: requires 2 arguments, but 1 was provided
  373 |   _LIBCPP_HIDE_FROM_ABI _LIBCPP_CONSTEXPR_SINCE_CXX23 unique_ptr(nullptr_t, _LValRefType<_Dummy> __d) _NOEXCEPT
      |                                                       ^          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../../prebuilt/third_party/clang/custom/bin/../include/c++/v1/__memory/unique_ptr.h:380:55: note: candidate constructor template not viable: requires 2 arguments, but 1 was provided
  380 |   _LIBCPP_HIDE_FROM_ABI _LIBCPP_CONSTEXPR_SINCE_CXX23 unique_ptr(_Pp __p, _GoodRValRefType<_Dummy> __d) _NOEXCEPT
      |                                                       ^          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../../prebuilt/third_party/clang/custom/bin/../include/c++/v1/__memory/unique_ptr.h:386:55: note: candidate constructor template not viable: requires 2 arguments, but 1 was provided
  386 |   _LIBCPP_HIDE_FROM_ABI _LIBCPP_CONSTEXPR_SINCE_CXX23 unique_ptr(nullptr_t, _GoodRValRefType<_Dummy> __d) _NOEXCEPT
      |                                                       ^          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../../prebuilt/third_party/clang/custom/bin/../include/c++/v1/__memory/unique_ptr.h:395:25: note: candidate constructor template not viable: requires 2 arguments, but 1 was provided
  395 |   _LIBCPP_HIDE_FROM_ABI unique_ptr(_Pp __p, _BadRValRefType<_Dummy> __d) = delete;
      |                         ^          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.

The type in question is defined as
typedef double HistoryBuffer[4]. This code has been working correctly for almost 4 years.

Here's a reproducer from -gen-reproducer=always`, though it isn't reduced.
repro.zip

Failing bot: https://ci.chromium.org/ui/p/fuchsia/builders/ci/clang_toolchain.ci.core.x64-debug-tot-build_only/b8749851620110598913/overview

Would you mind taking a look, since this doesn't appear to be a source issue (at least not from my quick reading).

@mahtohappy
Copy link
Contributor Author

Hi @ilovepi Sure. Looking at this.

@pranavk
Copy link
Contributor

pranavk commented Apr 23, 2024

@mahtohappy can we please revert this in the meantime while you look for a fix here. Feel free to land it again with the fix.

@erichkeane
Copy link
Collaborator

@mahtohappy can we please revert this in the meantime while you look for a fix here. Feel free to land it again with the fix.

He's likely outside of work hours, but feel free to submit a revert.

pranavk added a commit that referenced this pull request Apr 23, 2024
@awson
Copy link
Contributor

awson commented May 29, 2024

It looks like the author of the patch have either diverted to other activities or is unable to fix it.

Could we, perhaps, just diagnose the situation and gracefully fail with something like "typedefed array blah-blah is not supported in blah-blah" instead of generating segfaulting code?

cor3ntin pushed a commit that referenced this pull request Oct 15, 2024
The [last attempt](#89036) to
fix #41441 has been reverted
immediately.

Here I'm trying the simplest idea I've been able to come with: skip
handling dependent case in `BuildCXXNew`.

The original test (borrowed form
#89036) passes.

Also I've created and added to the tests a minimal repro of the code
#89036 fails on. This
(obviously) also passes.
DanielCChen pushed a commit to DanielCChen/llvm-project that referenced this pull request Oct 16, 2024
The [last attempt](llvm#89036) to
fix llvm#41441 has been reverted
immediately.

Here I'm trying the simplest idea I've been able to come with: skip
handling dependent case in `BuildCXXNew`.

The original test (borrowed form
llvm#89036) passes.

Also I've created and added to the tests a minimal repro of the code
llvm#89036 fails on. This
(obviously) also passes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
clang:frontend Language frontend issues, e.g. anything involving "Sema" clang:modules C++20 modules and Clang Header Modules clang Clang issues not falling into any other category
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Inplace new-ing an array overwrites the square of the memory
7 participants