Skip to content

[clang][Interp] Implement dynamic memory allocation handling #70306

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jul 14, 2024

Conversation

tbaederr
Copy link
Contributor

Implement handling for new/delete/new[]/delete[] expressions via a new DynamicAllocator class.

This introduces four new opcodes:

  • Alloc - Allocates one element (new int(14))
  • AllocN - Allocates N elements of the given primitive (new int[100])
  • AllocCN - Allocates N elements of the given (composite) descriptor (new S[100])
  • Free - de-allocates memory allocates using any of the above.

This patch currently has a few NFC changes (like adding const in some places) that I will push to main later today. Some other changes I just noticed are in tests/, which I will just remove.

@llvmbot llvmbot added clang Clang issues not falling into any other category clang:frontend Language frontend issues, e.g. anything involving "Sema" labels Oct 26, 2023
@llvmbot
Copy link
Member

llvmbot commented Oct 26, 2023

@llvm/pr-subscribers-clang-analysis

@llvm/pr-subscribers-clang

Author: Timm Baeder (tbaederr)

Changes

Implement handling for new/delete/new[]/delete[] expressions via a new DynamicAllocator class.

This introduces four new opcodes:

  • Alloc - Allocates one element (new int(14))
  • AllocN - Allocates N elements of the given primitive (new int[100])
  • AllocCN - Allocates N elements of the given (composite) descriptor (new S[100])
  • Free - de-allocates memory allocates using any of the above.

This patch currently has a few NFC changes (like adding const in some places) that I will push to main later today. Some other changes I just noticed are in tests/, which I will just remove.


Patch is 41.19 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/70306.diff

20 Files Affected:

  • (modified) clang/lib/AST/CMakeLists.txt (+1)
  • (modified) clang/lib/AST/ExprConstant.cpp (+1-1)
  • (modified) clang/lib/AST/Interp/ByteCodeExprGen.cpp (+76)
  • (modified) clang/lib/AST/Interp/ByteCodeExprGen.h (+2)
  • (modified) clang/lib/AST/Interp/Context.cpp (+2-1)
  • (modified) clang/lib/AST/Interp/Descriptor.cpp (+1-1)
  • (modified) clang/lib/AST/Interp/Descriptor.h (+3-3)
  • (added) clang/lib/AST/Interp/DynamicAllocator.cpp (+91)
  • (added) clang/lib/AST/Interp/DynamicAllocator.h (+90)
  • (modified) clang/lib/AST/Interp/EvalEmitter.cpp (+3-2)
  • (modified) clang/lib/AST/Interp/Interp.cpp (+37)
  • (modified) clang/lib/AST/Interp/Interp.h (+94)
  • (modified) clang/lib/AST/Interp/InterpBlock.h (+5-5)
  • (modified) clang/lib/AST/Interp/InterpState.cpp (+17)
  • (modified) clang/lib/AST/Interp/InterpState.h (+10)
  • (modified) clang/lib/AST/Interp/Opcodes.td (+25)
  • (modified) clang/lib/AST/Interp/Pointer.cpp (+1-1)
  • (modified) clang/lib/AST/Interp/Pointer.h (+4-3)
  • (added) clang/test/AST/Interp/new-delete.cpp (+247)
  • (modified) clang/test/SemaCXX/cxx2a-consteval.cpp (+2-41)
diff --git a/clang/lib/AST/CMakeLists.txt b/clang/lib/AST/CMakeLists.txt
index fe3f8c485ec1c56..1423623fb038cab 100644
--- a/clang/lib/AST/CMakeLists.txt
+++ b/clang/lib/AST/CMakeLists.txt
@@ -75,6 +75,7 @@ add_clang_library(clangAST
   Interp/Function.cpp
   Interp/InterpBuiltin.cpp
   Interp/Floating.cpp
+  Interp/DynamicAllocator.cpp
   Interp/Interp.cpp
   Interp/InterpBlock.cpp
   Interp/InterpFrame.cpp
diff --git a/clang/lib/AST/ExprConstant.cpp b/clang/lib/AST/ExprConstant.cpp
index 6b47b8a1256477d..0a95259821466e2 100644
--- a/clang/lib/AST/ExprConstant.cpp
+++ b/clang/lib/AST/ExprConstant.cpp
@@ -6860,8 +6860,8 @@ static std::optional<DynAlloc *> CheckDeleteKind(EvalInfo &Info, const Expr *E,
     return std::nullopt;
   }
 
-  QualType AllocType = Pointer.Base.getDynamicAllocType();
   if (DeallocKind != (*Alloc)->getKind()) {
+    QualType AllocType = Pointer.Base.getDynamicAllocType();
     Info.FFDiag(E, diag::note_constexpr_new_delete_mismatch)
         << DeallocKind << (*Alloc)->getKind() << AllocType;
     NoteLValueLocation(Info, Pointer.Base);
diff --git a/clang/lib/AST/Interp/ByteCodeExprGen.cpp b/clang/lib/AST/Interp/ByteCodeExprGen.cpp
index 1b33c69b93aa4b9..c42fcabe3b075ba 100644
--- a/clang/lib/AST/Interp/ByteCodeExprGen.cpp
+++ b/clang/lib/AST/Interp/ByteCodeExprGen.cpp
@@ -1617,6 +1617,82 @@ bool ByteCodeExprGen<Emitter>::VisitCXXScalarValueInitExpr(
   return this->visitZeroInitializer(classifyPrim(Ty), Ty, E);
 }
 
+template <class Emitter>
+bool ByteCodeExprGen<Emitter>::VisitCXXNewExpr(const CXXNewExpr *E) {
+  assert(classifyPrim(E->getType()) == PT_Ptr);
+  const Expr *Init = E->getInitializer();
+  QualType ElementType = E->getAllocatedType();
+  std::optional<PrimType> ElemT = classify(ElementType);
+
+  const Descriptor *Desc;
+  if (ElemT) {
+    if (E->isArray())
+      Desc = nullptr; // We're not going to use it in this case.
+    else
+      Desc = P.createDescriptor(E, *ElemT, Descriptor::InlineDescMD,
+                                /*IsConst=*/false, /*IsTemporary=*/false,
+                                /*IsMutable=*/false);
+  } else {
+    Desc = P.createDescriptor(
+        E, ElementType.getTypePtr(),
+        E->isArray() ? std::nullopt : Descriptor::InlineDescMD,
+        /*IsConst=*/false, /*IsTemporary=*/false, /*IsMutable=*/false, Init);
+  }
+
+  if (E->isArray()) {
+    assert(E->getArraySize());
+    PrimType SizeT = classifyPrim((*E->getArraySize())->getType());
+
+    if (!this->visit(*E->getArraySize()))
+      return false;
+
+    if (ElemT) {
+      // N primitive elements.
+      if (!this->emitAllocN(SizeT, *ElemT, E, E))
+        return false;
+    } else {
+      // N Composite elements.
+      if (!this->emitAllocCN(SizeT, Desc, E))
+        return false;
+    }
+
+  } else {
+    // Allocate just one element.
+    if (!this->emitAlloc(Desc, E))
+      return false;
+
+    if (Init) {
+      if (ElemT) {
+        if (!this->visit(Init))
+          return false;
+
+        if (!this->emitInit(*ElemT, E))
+          return false;
+      } else {
+        // Composite.
+        if (!this->visitInitializer(Init))
+          return false;
+      }
+    }
+  }
+
+  if (DiscardResult)
+    return this->emitPopPtr(E);
+
+  return true;
+}
+
+template <class Emitter>
+bool ByteCodeExprGen<Emitter>::VisitCXXDeleteExpr(const CXXDeleteExpr *E) {
+  const Expr *Arg = E->getArgument();
+
+  // Arg must be an lvalue.
+  if (!this->visit(Arg))
+    return false;
+
+  return this->emitFree(E->isArrayForm(), E);
+}
+
 template <class Emitter> bool ByteCodeExprGen<Emitter>::discard(const Expr *E) {
   if (E->containsErrors())
     return false;
diff --git a/clang/lib/AST/Interp/ByteCodeExprGen.h b/clang/lib/AST/Interp/ByteCodeExprGen.h
index 83986d3dd579ed6..f65dc9494db36ef 100644
--- a/clang/lib/AST/Interp/ByteCodeExprGen.h
+++ b/clang/lib/AST/Interp/ByteCodeExprGen.h
@@ -107,6 +107,8 @@ class ByteCodeExprGen : public ConstStmtVisitor<ByteCodeExprGen<Emitter>, bool>,
   bool VisitSourceLocExpr(const SourceLocExpr *E);
   bool VisitOffsetOfExpr(const OffsetOfExpr *E);
   bool VisitCXXScalarValueInitExpr(const CXXScalarValueInitExpr *E);
+  bool VisitCXXNewExpr(const CXXNewExpr *E);
+  bool VisitCXXDeleteExpr(const CXXDeleteExpr *E);
 
 protected:
   bool visitExpr(const Expr *E) override;
diff --git a/clang/lib/AST/Interp/Context.cpp b/clang/lib/AST/Interp/Context.cpp
index cb96e56fb5e1ad8..93da5267bc1e6f4 100644
--- a/clang/lib/AST/Interp/Context.cpp
+++ b/clang/lib/AST/Interp/Context.cpp
@@ -164,7 +164,7 @@ bool Context::Run(State &Parent, const Function *Func, APValue &Result) {
     State.Current = new InterpFrame(State, Func, /*Caller=*/nullptr, {});
     if (Interpret(State, Result)) {
       assert(Stk.empty());
-      return true;
+      return !State.maybeDiagnoseDanglingAllocations();
     }
 
     // State gets destroyed here, so the Stk.clear() below doesn't accidentally
@@ -177,6 +177,7 @@ bool Context::Run(State &Parent, const Function *Func, APValue &Result) {
 
 bool Context::Check(State &Parent, llvm::Expected<bool> &&Flag) {
   if (Flag)
+    // return !Parent.maybeDiagnoseDanglingAllocations();
     return *Flag;
   handleAllErrors(Flag.takeError(), [&Parent](ByteCodeGenError &Err) {
     Parent.FFDiag(Err.getRange().getBegin(),
diff --git a/clang/lib/AST/Interp/Descriptor.cpp b/clang/lib/AST/Interp/Descriptor.cpp
index abd75a8d67bbd3f..2a21f60588d46cd 100644
--- a/clang/lib/AST/Interp/Descriptor.cpp
+++ b/clang/lib/AST/Interp/Descriptor.cpp
@@ -262,7 +262,7 @@ Descriptor::Descriptor(const DeclTy &D, PrimType Type, bool IsTemporary,
 }
 
 /// Arrays of composite elements.
-Descriptor::Descriptor(const DeclTy &D, Descriptor *Elem, MetadataSize MD,
+Descriptor::Descriptor(const DeclTy &D, const Descriptor *Elem, MetadataSize MD,
                        unsigned NumElems, bool IsConst, bool IsTemporary,
                        bool IsMutable)
     : Source(D), ElemSize(Elem->getAllocSize() + sizeof(InlineDescriptor)),
diff --git a/clang/lib/AST/Interp/Descriptor.h b/clang/lib/AST/Interp/Descriptor.h
index be9a380138a7b11..2fd4e9208264525 100644
--- a/clang/lib/AST/Interp/Descriptor.h
+++ b/clang/lib/AST/Interp/Descriptor.h
@@ -72,7 +72,7 @@ struct InlineDescriptor {
   /// Flag indicating if the field is mutable (if in a record).
   unsigned IsFieldMutable : 1;
 
-  Descriptor *Desc;
+  const Descriptor *Desc;
 };
 
 /// Describes a memory block created by an allocation site.
@@ -102,7 +102,7 @@ struct Descriptor final {
   /// Pointer to the record, if block contains records.
   Record *const ElemRecord = nullptr;
   /// Descriptor of the array element.
-  Descriptor *const ElemDesc = nullptr;
+  const Descriptor *const ElemDesc = nullptr;
   /// Flag indicating if the block is mutable.
   const bool IsConst = false;
   /// Flag indicating if a field is mutable.
@@ -129,7 +129,7 @@ struct Descriptor final {
   Descriptor(const DeclTy &D, PrimType Type, bool IsTemporary, UnknownSize);
 
   /// Allocates a descriptor for an array of composites.
-  Descriptor(const DeclTy &D, Descriptor *Elem, MetadataSize MD,
+  Descriptor(const DeclTy &D, const Descriptor *Elem, MetadataSize MD,
              unsigned NumElems, bool IsConst, bool IsTemporary, bool IsMutable);
 
   /// Allocates a descriptor for an array of composites of unknown size.
diff --git a/clang/lib/AST/Interp/DynamicAllocator.cpp b/clang/lib/AST/Interp/DynamicAllocator.cpp
new file mode 100644
index 000000000000000..a353d09d1f960ea
--- /dev/null
+++ b/clang/lib/AST/Interp/DynamicAllocator.cpp
@@ -0,0 +1,91 @@
+//==---- DynamicAllocator.cpp - Types for the constexpr VM -------*- C++ -*-==//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "DynamicAllocator.h"
+#include "InterpBlock.h"
+#include "InterpState.h"
+
+using namespace clang;
+using namespace clang::interp;
+
+Block *DynamicAllocator::allocate(const Expr *Source, PrimType T,
+                                  size_t NumElements) {
+  assert(NumElements > 0);
+  // Create a new descriptor for an array of the specified size and
+  // element type.
+  const Descriptor *D = allocateDescriptor(
+      Source, T, Descriptor::InlineDescMD, NumElements, /*IsConst=*/false,
+      /*IsTemporary=*/false, /*IsMutable=*/false);
+  return allocate(D);
+}
+
+Block *DynamicAllocator::allocate(const Descriptor *ElementDesc,
+                                  size_t NumElements) {
+  assert(NumElements > 0);
+  // Create a new descriptor for an array of the specified size and
+  // element type.
+  const Descriptor *D = allocateDescriptor(
+      ElementDesc->asExpr(), ElementDesc, Descriptor::InlineDescMD, NumElements,
+      /*IsConst=*/false, /*IsTemporary=*/false, /*IsMutable=*/false);
+  return allocate(D);
+}
+
+Block *DynamicAllocator::allocate(const Descriptor *D) {
+  assert(D->asExpr());
+
+  auto Memory =
+      std::make_unique<std::byte[]>(sizeof(Block) + D->getAllocSize());
+  auto *B = new (Memory.get()) Block(D, /*isStatic=*/false);
+  B->invokeCtor();
+
+  InlineDescriptor *ID = reinterpret_cast<InlineDescriptor *>(B->rawData());
+  ID->Desc = const_cast<Descriptor *>(D);
+  ID->IsActive = true;
+  ID->Offset = sizeof(InlineDescriptor);
+  ID->IsBase = false;
+  ID->IsFieldMutable = false;
+  ID->IsConst = false;
+  ID->IsInitialized = false;
+  assert(ID->Desc);
+
+  if (auto It = AllocationSites.find(D->asExpr()); It != AllocationSites.end())
+    It->second.Allocations.emplace_back(std::move(Memory));
+  else
+    AllocationSites.insert(
+        {D->asExpr(), AllocationSite(std::move(Memory), D->isArray())});
+  return B;
+}
+
+void DynamicAllocator::deallocate(const Expr *Source,
+                                  const Block *BlockToDelete, InterpState &S) {
+  assert(AllocationSites.contains(Source));
+
+  auto It = AllocationSites.find(Source);
+  assert(It != AllocationSites.end());
+
+  auto &Site = It->second;
+  assert(Site.size() > 0);
+
+  // Find the Block to delete.
+  size_t I = 0;
+  [[maybe_unused]] bool Found = false;
+  for (auto &A : Site.Allocations) {
+    Block *B = reinterpret_cast<Block *>(A.Memory.get());
+    if (B == BlockToDelete) {
+      S.deallocate(B);
+      B->invokeDtor();
+      Site.Allocations.erase(Site.Allocations.begin() + I);
+      Found = true;
+      break;
+    }
+  }
+  assert(Found);
+
+  if (Site.size() == 0)
+    AllocationSites.erase(It);
+}
diff --git a/clang/lib/AST/Interp/DynamicAllocator.h b/clang/lib/AST/Interp/DynamicAllocator.h
new file mode 100644
index 000000000000000..d2a1d96232b9730
--- /dev/null
+++ b/clang/lib/AST/Interp/DynamicAllocator.h
@@ -0,0 +1,90 @@
+//==- DynamicAllocator.h - Bytecode allocator for the constexpr VM -*- C++
+//-*-=//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_CLANG_AST_INTERP_DYNAMIC_ALLOCATOR_H
+#define LLVM_CLANG_AST_INTERP_DYNAMIC_ALLOCATOR_H
+
+#include "Descriptor.h"
+#include "llvm/ADT/SmallVector.h"
+#include "llvm/ADT/iterator_range.h"
+#include "llvm/Support/Allocator.h"
+
+namespace clang {
+class Expr;
+namespace interp {
+class Block;
+class InterpState;
+
+/// Manages dynamic memory allocations done during bytecode interpretation.
+///
+/// We manage allocations as a map from their new-expression to a list
+/// of allocations. This is called an AllocationSite. For each site, we
+/// record whether it was allocated using new or new[], the
+/// IsArrayAllocation flag.
+///
+/// For all array allocations, we need to allocat new Descriptor instances,
+/// so the DynamicAllocator has a llvm::BumpPtrAllocator similar to Program.
+class DynamicAllocator final {
+  struct Allocation {
+    std::unique_ptr<std::byte[]> Memory;
+    Allocation(std::unique_ptr<std::byte[]> Memory)
+        : Memory(std::move(Memory)) {}
+  };
+
+  struct AllocationSite {
+    llvm::SmallVector<Allocation> Allocations;
+    bool IsArrayAllocation = false;
+
+    AllocationSite(std::unique_ptr<std::byte[]> Memory, bool Array)
+        : IsArrayAllocation(Array) {
+      Allocations.push_back({std::move(Memory)});
+    }
+
+    size_t size() const { return Allocations.size(); }
+  };
+
+public:
+  DynamicAllocator() = default;
+
+  unsigned getNumAllocations() const { return AllocationSites.size(); }
+
+  /// Allocate ONE element of the given descriptor.
+  Block *allocate(const Descriptor *D);
+  /// Allocate \p NumElements primitive elements of the given type.
+  Block *allocate(const Expr *Source, PrimType T, size_t NumElements);
+  /// Allocate \p NumElements elements of the given descriptor.
+  Block *allocate(const Descriptor *D, size_t NumElements);
+
+  /// Deallocate the given source+block combination.
+  void deallocate(const Expr *Source, const Block *BlockToDelete,
+                  InterpState &S);
+
+  /// Checks whether the allocation done at the given source is an array
+  /// allocation.
+  bool isArrayAllocation(const Expr *Source) const {
+    assert(AllocationSites.contains(Source));
+    return AllocationSites.at(Source).IsArrayAllocation;
+  }
+
+  // FIXME: Public because I'm not sure how to expose an iterator to it.
+  llvm::DenseMap<const Expr *, AllocationSite> AllocationSites;
+
+private:
+  using PoolAllocTy = llvm::BumpPtrAllocatorImpl<llvm::MallocAllocator>;
+  PoolAllocTy DescAllocator;
+
+  /// Allocates a new descriptor.
+  template <typename... Ts> Descriptor *allocateDescriptor(Ts &&...Args) {
+    return new (DescAllocator) Descriptor(std::forward<Ts>(Args)...);
+  }
+};
+
+} // namespace interp
+} // namespace clang
+#endif
diff --git a/clang/lib/AST/Interp/EvalEmitter.cpp b/clang/lib/AST/Interp/EvalEmitter.cpp
index f8942291b3b162d..14d5446f64324ee 100644
--- a/clang/lib/AST/Interp/EvalEmitter.cpp
+++ b/clang/lib/AST/Interp/EvalEmitter.cpp
@@ -35,7 +35,7 @@ EvalEmitter::~EvalEmitter() {
 
 llvm::Expected<bool> EvalEmitter::interpretExpr(const Expr *E) {
   if (this->visitExpr(E))
-    return true;
+    return S.maybeDiagnoseDanglingAllocations();
   if (BailLocation)
     return llvm::make_error<ByteCodeGenError>(*BailLocation);
   return false;
@@ -43,7 +43,7 @@ llvm::Expected<bool> EvalEmitter::interpretExpr(const Expr *E) {
 
 llvm::Expected<bool> EvalEmitter::interpretDecl(const VarDecl *VD) {
   if (this->visitDecl(VD))
-    return true;
+    return S.maybeDiagnoseDanglingAllocations();
   if (BailLocation)
     return llvm::make_error<ByteCodeGenError>(*BailLocation);
   return false;
@@ -122,6 +122,7 @@ template <PrimType OpType> bool EvalEmitter::emitRet(const SourceInfo &Info) {
 bool EvalEmitter::emitRetVoid(const SourceInfo &Info) { return true; }
 
 bool EvalEmitter::emitRetValue(const SourceInfo &Info) {
+  llvm::errs() << __PRETTY_FUNCTION__ << "\n";
   // Method to recursively traverse composites.
   std::function<bool(QualType, const Pointer &, APValue &)> Composite;
   Composite = [this, &Composite](QualType Ty, const Pointer &Ptr, APValue &R) {
diff --git a/clang/lib/AST/Interp/Interp.cpp b/clang/lib/AST/Interp/Interp.cpp
index 8b0e7beb4a1acc1..3ec093b16c8f9c2 100644
--- a/clang/lib/AST/Interp/Interp.cpp
+++ b/clang/lib/AST/Interp/Interp.cpp
@@ -548,6 +548,43 @@ bool CheckFloatResult(InterpState &S, CodePtr OpPC, const Floating &Result,
   return true;
 }
 
+bool CheckDynamicMemoryAllocation(InterpState &S, CodePtr OpPC) {
+  if (S.getLangOpts().CPlusPlus20)
+    return true;
+
+  const SourceInfo &E = S.Current->getSource(OpPC);
+  S.FFDiag(E, diag::note_constexpr_new);
+  return false;
+}
+
+bool CheckNewDeleteForms(InterpState &S, CodePtr OpPC, bool NewWasArray,
+                         bool DeleteIsArray, const Block *B,
+                         const Expr *NewExpr) {
+  if (NewWasArray == DeleteIsArray)
+    return true;
+
+  const Descriptor *D = B->getDescriptor();
+
+  QualType TypeToDiagnose;
+  // We need to shuffle things around a bit here to get a better diagnostic,
+  // because the expression we allocated the block for was of type int*,
+  // but we want to get the array size right.
+  if (D->isArray()) {
+    QualType ElemQT = D->getType()->getPointeeType();
+    TypeToDiagnose = S.getCtx().getConstantArrayType(
+        ElemQT, APInt(64, D->getNumElems(), false), nullptr, ArrayType::Normal,
+        0);
+  } else
+    TypeToDiagnose = D->getType()->getPointeeType();
+
+  const SourceInfo &E = S.Current->getSource(OpPC);
+  S.FFDiag(E, diag::note_constexpr_new_delete_mismatch)
+      << DeleteIsArray << 0 << TypeToDiagnose;
+  S.Note(NewExpr->getExprLoc(), diag::note_constexpr_dynamic_alloc_here)
+      << NewExpr->getSourceRange();
+  return false;
+}
+
 /// We aleady know the given DeclRefExpr is invalid for some reason,
 /// now figure out why and print appropriate diagnostics.
 bool CheckDeclRef(InterpState &S, CodePtr OpPC, const DeclRefExpr *DR) {
diff --git a/clang/lib/AST/Interp/Interp.h b/clang/lib/AST/Interp/Interp.h
index 7ef1e344224a3c3..9062e4a81772888 100644
--- a/clang/lib/AST/Interp/Interp.h
+++ b/clang/lib/AST/Interp/Interp.h
@@ -14,6 +14,7 @@
 #define LLVM_CLANG_AST_INTERP_INTERP_H
 
 #include "Boolean.h"
+#include "DynamicAllocator.h"
 #include "Floating.h"
 #include "Function.h"
 #include "FunctionPointer.h"
@@ -112,6 +113,15 @@ bool CheckCtorCall(InterpState &S, CodePtr OpPC, const Pointer &This);
 bool CheckPotentialReinterpretCast(InterpState &S, CodePtr OpPC,
                                    const Pointer &Ptr);
 
+/// Checks if dynamic memory allocation is available in the current
+/// language mode.
+bool CheckDynamicMemoryAllocation(InterpState &S, CodePtr OpPC);
+
+/// Diagnose mismatched new[]/delete or new/delete[] pairs.
+bool CheckNewDeleteForms(InterpState &S, CodePtr OpPC, bool NewWasArray,
+                         bool DeleteIsArray, const Block *B,
+                         const Expr *NewExpr);
+
 /// Sets the given integral value to the pointer, which is of
 /// a std::{weak,partial,strong}_ordering type.
 bool SetThreeWayComparisonField(InterpState &S, CodePtr OpPC,
@@ -1369,6 +1379,17 @@ bool InitPop(InterpState &S, CodePtr OpPC) {
   return true;
 }
 
+template <PrimType Name, class T = typename PrimConv<Name>::T>
+bool Init(InterpState &S, CodePtr OpPC) {
+  const T &Value = S.Stk.pop<T>();
+  const Pointer &Ptr = S.Stk.peek<Pointer>();
+  if (!CheckInit(S, OpPC, Ptr))
+    return false;
+  Ptr.initialize();
+  new (&Ptr.deref<T>()) T(Value);
+  return true;
+}
+
 /// 1) Pops the value from the stack
 /// 2) Peeks a pointer and gets its index \Idx
 /// 3) Sets the value on the pointer, leaving the pointer on the stack.
@@ -1990,6 +2011,79 @@ inline bool OffsetOf(InterpState &S, CodePtr OpPC, const OffsetOfExpr *E) {
   return true;
 }
 
+inline bool Alloc(InterpState &S, CodePtr OpPC, const Descriptor *Desc) {
+  assert(Desc);
+
+  if (!CheckDynamicMemoryAllocation(S, OpPC))
+    return false;
+
+  DynamicAllocator &Allocator = S.getAllocator();
+  Block *B = Allocator.allocate(Desc);
+
+  S.Stk.push<Pointer>(B, sizeof(InlineDescriptor));
+
+  return true;
+}
+
+template <PrimType Name, class SizeT = typename PrimConv<Name>::T>
+inline bool AllocN(InterpState &S, CodePtr OpPC, PrimType T,
+                   const Expr *Source) {
+  if (!CheckDynamicMemoryAllocation(S, OpPC))
+    return false;
+
+  SizeT NumElements = S.Stk.pop<SizeT>();
+
+  DynamicAllocator &Allocator = S.getAllocator();
+  Block *B = Allocator.allocate(Source, T, static_cast<size_t>(NumElements));
+
+  S.Stk.push<Pointer>(B, sizeof(InlineDescriptor));
+
+  return true;
+}
+
+template <PrimType Name, class SizeT = typename PrimConv<Name>::T>
+inline bool AllocCN(InterpState &S, CodePtr OpPC,
+                    const Descriptor *ElementDesc) {
+  if (!CheckDynamicMemoryAllocation(S, OpPC))
+    return false;
+
+  SizeT NumElements = S.Stk.pop<SizeT>();
+
+  DynamicAllocator &Allocator = S.getAllocator();
+  Block *B = Allocator.allocate(ElementDesc, static_cast<size_t>(NumElements));
+
+  S.Stk.push<Pointer>(B, sizeof(InlineDescriptor));
+
+  return true;
+}
+
+static inline bool Free(InterpState &S, CodePtr OpPC, bool DeleteIsArr...
[truncated]

@tbaederr tbaederr force-pushed the new-delete branch 5 times, most recently from e7974a7 to 23aff34 Compare October 29, 2023 13:17
@tbaederr
Copy link
Contributor Author

tbaederr commented Oct 31, 2023

FYI, this needs #70763, #70663 and #70772 to make a large chunk of the added test cases acually do something (especially because of #70763). And then some of them fail.

@tbaederr
Copy link
Contributor Author

tbaederr commented Jun 5, 2024

Rebased and updated with a lot more tests.

@tbaederr
Copy link
Contributor Author

Ping

Copy link
Collaborator

@AaronBallman AaronBallman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Comment on lines +2787 to +2788
// FIXME: There is no restriction on this, but it's not clear that any
// other form makes any sense. We get here for cases such as:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FWIW: https://wg21.link/p2747r2 was just adopted for C++26 (doesn't need to be handled in this PR, just a comment on "any other form")

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, I saw. I'll wait for @cor3ntin to come up with the test cases though :)

@tbaederr tbaederr merged commit fa133d3 into llvm:main Jul 14, 2024
7 checks passed
@llvm-ci
Copy link
Collaborator

llvm-ci commented Jul 14, 2024

LLVM Buildbot has detected a new failure on builder lldb-aarch64-windows running on linaro-armv8-windows-msvc-05 while building clang at step 4 "build".

Full details are available at: https://lab.llvm.org/buildbot/#/builders/141/builds/764

Here is the relevant piece of the build log for the reference:

Step 4 (build) failure: build (failure)
...
367.241 [2480/10/3839] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\Interp\Function.cpp.obj
371.519 [2479/10/3840] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\Interp\Context.cpp.obj
372.145 [2478/10/3841] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\Interp\ByteCodeEmitter.cpp.obj
372.883 [2477/10/3842] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\Interp\InterpBuiltin.cpp.obj
381.067 [2476/10/3843] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\Interp\EvaluationResult.cpp.obj
381.398 [2475/10/3844] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\Interp\DynamicAllocator.cpp.obj
381.518 [2474/10/3845] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\Interp\InterpBlock.cpp.obj
383.165 [2473/10/3846] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\Interp\Compiler.cpp.obj
387.951 [2472/10/3847] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\Interp\InterpFrame.cpp.obj
388.579 [2471/10/3848] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\Interp\Interp.cpp.obj
FAILED: tools/clang/lib/AST/CMakeFiles/obj.clangAST.dir/Interp/Interp.cpp.obj 
ccache C:\PROGRA~1\LLVM\bin\clang-cl.exe  /nologo -TP -DGTEST_HAS_RTTI=0 -DUNICODE -D_CRT_NONSTDC_NO_DEPRECATE -D_CRT_NONSTDC_NO_WARNINGS -D_CRT_SECURE_NO_DEPRECATE -D_CRT_SECURE_NO_WARNINGS -D_HAS_EXCEPTIONS=0 -D_SCL_SECURE_NO_DEPRECATE -D_SCL_SECURE_NO_WARNINGS -D_UNICODE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -IC:\Users\tcwg\llvm-worker\lldb-aarch64-windows\build\tools\clang\lib\AST -IC:\Users\tcwg\llvm-worker\lldb-aarch64-windows\llvm-project\clang\lib\AST -IC:\Users\tcwg\llvm-worker\lldb-aarch64-windows\llvm-project\clang\include -IC:\Users\tcwg\llvm-worker\lldb-aarch64-windows\build\tools\clang\include -IC:\Users\tcwg\llvm-worker\lldb-aarch64-windows\build\include -IC:\Users\tcwg\llvm-worker\lldb-aarch64-windows\llvm-project\llvm\include /DWIN32 /D_WINDOWS   /Zc:inline /Zc:__cplusplus /Oi /Brepro /bigobj /permissive- -Werror=unguarded-availability-new /W4  -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -Wimplicit-fallthrough -Wcovered-switch-default -Wno-noexcept-type -Wnon-virtual-dtor -Wdelete-non-virtual-dtor -Wsuggest-override -Wstring-conversion -Wmisleading-indentation -Wctad-maybe-unsupported /Gw /O2 /Ob2 /DNDEBUG -MD  /EHs-c- /GR- -std:c++17 /showIncludes /Fotools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\Interp\Interp.cpp.obj /Fdtools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\ -c -- C:\Users\tcwg\llvm-worker\lldb-aarch64-windows\llvm-project\clang\lib\AST\Interp\Interp.cpp
C:\Users\tcwg\llvm-worker\lldb-aarch64-windows\llvm-project\clang\lib\AST\Interp\Interp.cpp(718,17): error: call to constructor of 'APInt' (aka 'llvm::APInt') is ambiguous
  718 |         ElemQT, APInt(64, D->getNumElems(), false), nullptr,
      |                 ^     ~~~~~~~~~~~~~~~~~~~~~~~~~~~
C:\Users\tcwg\llvm-worker\lldb-aarch64-windows\llvm-project\llvm\include\llvm/ADT/APInt.h(111,3): note: candidate constructor
  111 |   APInt(unsigned numBits, uint64_t val, bool isSigned = false)
      |   ^
C:\Users\tcwg\llvm-worker\lldb-aarch64-windows\llvm-project\llvm\include\llvm/ADT/APInt.h(137,3): note: candidate constructor
  137 |   APInt(unsigned numBits, unsigned numWords, const uint64_t bigVal[]);
      |   ^
1 error generated.
394.070 [2471/9/3849] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\Interp\InterpStack.cpp.obj
394.183 [2471/8/3850] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\Interp\InterpState.cpp.obj
395.085 [2471/7/3851] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\Interp\PrimType.cpp.obj
396.192 [2471/6/3852] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\Interp\Pointer.cpp.obj
400.254 [2471/5/3853] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\Interp\Program.cpp.obj
403.797 [2471/4/3854] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\Interp\Disasm.cpp.obj
404.875 [2471/3/3855] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\Interp\EvalEmitter.cpp.obj
406.878 [2471/2/3856] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\ASTContext.cpp.obj
409.010 [2471/1/3857] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\ExprConstant.cpp.obj
ninja: build stopped: subcommand failed.

@llvm-ci
Copy link
Collaborator

llvm-ci commented Jul 14, 2024

LLVM Buildbot has detected a new failure on builder clang-x64-windows-msvc running on windows-gcebot2 while building clang at step 4 "annotate".

Full details are available at: https://lab.llvm.org/buildbot/#/builders/63/builds/509

Here is the relevant piece of the build log for the reference:

Step 4 (annotate) failure: 'python ../llvm-zorg/zorg/buildbot/builders/annotated/clang-windows.py ...' (failure)
...
[3902/6006] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\Interp\InterpFrame.cpp.obj
[3903/6006] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\ASTDumper.cpp.obj
[3904/6006] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\Interp\Source.cpp.obj
[3905/6006] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\Interp\Record.cpp.obj
[3906/6006] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\Interp\InterpState.cpp.obj
[3907/6006] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\Interp\InterpShared.cpp.obj
[3908/6006] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\Interp\DynamicAllocator.cpp.obj
[3909/6006] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\Interp\Pointer.cpp.obj
[3910/6006] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\Decl.cpp.obj
[3911/6006] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\Interp\Interp.cpp.obj
FAILED: tools/clang/lib/AST/CMakeFiles/obj.clangAST.dir/Interp/Interp.cpp.obj 
C:\b\slave\clang-x64-windows-msvc\build\stage1\bin\clang-cl.exe  /nologo -TP -DGTEST_HAS_RTTI=0 -DUNICODE -D_CRT_NONSTDC_NO_DEPRECATE -D_CRT_NONSTDC_NO_WARNINGS -D_CRT_SECURE_NO_DEPRECATE -D_CRT_SECURE_NO_WARNINGS -D_GLIBCXX_ASSERTIONS -D_HAS_EXCEPTIONS=0 -D_SCL_SECURE_NO_DEPRECATE -D_SCL_SECURE_NO_WARNINGS -D_UNICODE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -IC:\b\slave\clang-x64-windows-msvc\build\stage2\tools\clang\lib\AST -IC:\b\slave\clang-x64-windows-msvc\llvm-project\clang\lib\AST -IC:\b\slave\clang-x64-windows-msvc\llvm-project\clang\include -IC:\b\slave\clang-x64-windows-msvc\build\stage2\tools\clang\include -IC:\b\slave\clang-x64-windows-msvc\build\stage2\include -IC:\b\slave\clang-x64-windows-msvc\llvm-project\llvm\include /DWIN32 /D_WINDOWS   /Zc:inline /Zc:__cplusplus /Zi -gcodeview-ghash /Oi /Brepro /bigobj /permissive- -Werror=unguarded-availability-new /W4  -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -Wimplicit-fallthrough -Wcovered-switch-default -Wno-noexcept-type -Wnon-virtual-dtor -Wdelete-non-virtual-dtor -Wsuggest-override -Wstring-conversion -Wmisleading-indentation -Wctad-maybe-unsupported /Gw /O2 /Ob2  -std:c++17 -MD  /EHs-c- /GR- -UNDEBUG /showIncludes /Fotools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\Interp\Interp.cpp.obj /Fdtools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\ -c -- C:\b\slave\clang-x64-windows-msvc\llvm-project\clang\lib\AST\Interp\Interp.cpp
C:\b\slave\clang-x64-windows-msvc\llvm-project\clang\lib\AST\Interp\Interp.cpp(718,17): error: call to constructor of 'APInt' (aka 'llvm::APInt') is ambiguous
  718 |         ElemQT, APInt(64, D->getNumElems(), false), nullptr,
      |                 ^     ~~~~~~~~~~~~~~~~~~~~~~~~~~~
C:\b\slave\clang-x64-windows-msvc\llvm-project\llvm\include\llvm/ADT/APInt.h(137,3): note: candidate constructor
  137 |   APInt(unsigned numBits, unsigned numWords, const uint64_t bigVal[]);
      |   ^
C:\b\slave\clang-x64-windows-msvc\llvm-project\llvm\include\llvm/ADT/APInt.h(111,3): note: candidate constructor
  111 |   APInt(unsigned numBits, uint64_t val, bool isSigned = false)
      |   ^
1 error generated.
[3912/6006] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\Interp\State.cpp.obj
[3913/6006] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\Interp\InterpBuiltin.cpp.obj
[3914/6006] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\Interp\PrimType.cpp.obj
[3915/6006] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\Interp\MemberPointer.cpp.obj
[3916/6006] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\Expr.cpp.obj
[3917/6006] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\Interp\Program.cpp.obj
[3918/6006] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\NSAPI.cpp.obj
[3919/6006] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\NestedNameSpecifier.cpp.obj
[3920/6006] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\ASTImporterLookupTable.cpp.obj
[3921/6006] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\OpenACCClause.cpp.obj
[3922/6006] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\ParentMap.cpp.obj
[3923/6006] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\MicrosoftCXXABI.cpp.obj
[3924/6006] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\OSLog.cpp.obj
[3925/6006] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\Mangle.cpp.obj
[3926/6006] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\ItaniumCXXABI.cpp.obj
[3927/6006] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\PrintfFormatString.cpp.obj
[3928/6006] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\QualTypeNames.cpp.obj
[3929/6006] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\ODRHash.cpp.obj
[3930/6006] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\ODRDiagsEmitter.cpp.obj
[3931/6006] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\OpenMPClause.cpp.obj
[3932/6006] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\Interp\Compiler.cpp.obj
[3933/6006] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\MicrosoftMangle.cpp.obj
[3934/6006] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\JSONNodeDumper.cpp.obj
[3935/6006] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\ParentMapContext.cpp.obj
[3936/6006] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\ASTImporter.cpp.obj
[3937/6006] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\Interp\Disasm.cpp.obj
[3938/6006] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\ItaniumMangle.cpp.obj
Step 11 (stage 2 build) failure: stage 2 build (failure)
...
[3902/6006] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\Interp\InterpFrame.cpp.obj
[3903/6006] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\ASTDumper.cpp.obj
[3904/6006] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\Interp\Source.cpp.obj
[3905/6006] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\Interp\Record.cpp.obj
[3906/6006] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\Interp\InterpState.cpp.obj
[3907/6006] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\Interp\InterpShared.cpp.obj
[3908/6006] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\Interp\DynamicAllocator.cpp.obj
[3909/6006] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\Interp\Pointer.cpp.obj
[3910/6006] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\Decl.cpp.obj
[3911/6006] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\Interp\Interp.cpp.obj
FAILED: tools/clang/lib/AST/CMakeFiles/obj.clangAST.dir/Interp/Interp.cpp.obj 
C:\b\slave\clang-x64-windows-msvc\build\stage1\bin\clang-cl.exe  /nologo -TP -DGTEST_HAS_RTTI=0 -DUNICODE -D_CRT_NONSTDC_NO_DEPRECATE -D_CRT_NONSTDC_NO_WARNINGS -D_CRT_SECURE_NO_DEPRECATE -D_CRT_SECURE_NO_WARNINGS -D_GLIBCXX_ASSERTIONS -D_HAS_EXCEPTIONS=0 -D_SCL_SECURE_NO_DEPRECATE -D_SCL_SECURE_NO_WARNINGS -D_UNICODE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -IC:\b\slave\clang-x64-windows-msvc\build\stage2\tools\clang\lib\AST -IC:\b\slave\clang-x64-windows-msvc\llvm-project\clang\lib\AST -IC:\b\slave\clang-x64-windows-msvc\llvm-project\clang\include -IC:\b\slave\clang-x64-windows-msvc\build\stage2\tools\clang\include -IC:\b\slave\clang-x64-windows-msvc\build\stage2\include -IC:\b\slave\clang-x64-windows-msvc\llvm-project\llvm\include /DWIN32 /D_WINDOWS   /Zc:inline /Zc:__cplusplus /Zi -gcodeview-ghash /Oi /Brepro /bigobj /permissive- -Werror=unguarded-availability-new /W4  -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -Wimplicit-fallthrough -Wcovered-switch-default -Wno-noexcept-type -Wnon-virtual-dtor -Wdelete-non-virtual-dtor -Wsuggest-override -Wstring-conversion -Wmisleading-indentation -Wctad-maybe-unsupported /Gw /O2 /Ob2  -std:c++17 -MD  /EHs-c- /GR- -UNDEBUG /showIncludes /Fotools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\Interp\Interp.cpp.obj /Fdtools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\ -c -- C:\b\slave\clang-x64-windows-msvc\llvm-project\clang\lib\AST\Interp\Interp.cpp
C:\b\slave\clang-x64-windows-msvc\llvm-project\clang\lib\AST\Interp\Interp.cpp(718,17): error: call to constructor of 'APInt' (aka 'llvm::APInt') is ambiguous
  718 |         ElemQT, APInt(64, D->getNumElems(), false), nullptr,
      |                 ^     ~~~~~~~~~~~~~~~~~~~~~~~~~~~
C:\b\slave\clang-x64-windows-msvc\llvm-project\llvm\include\llvm/ADT/APInt.h(137,3): note: candidate constructor
  137 |   APInt(unsigned numBits, unsigned numWords, const uint64_t bigVal[]);
      |   ^
C:\b\slave\clang-x64-windows-msvc\llvm-project\llvm\include\llvm/ADT/APInt.h(111,3): note: candidate constructor
  111 |   APInt(unsigned numBits, uint64_t val, bool isSigned = false)
      |   ^
1 error generated.
[3912/6006] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\Interp\State.cpp.obj
[3913/6006] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\Interp\InterpBuiltin.cpp.obj
[3914/6006] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\Interp\PrimType.cpp.obj
[3915/6006] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\Interp\MemberPointer.cpp.obj
[3916/6006] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\Expr.cpp.obj
[3917/6006] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\Interp\Program.cpp.obj
[3918/6006] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\NSAPI.cpp.obj
[3919/6006] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\NestedNameSpecifier.cpp.obj
[3920/6006] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\ASTImporterLookupTable.cpp.obj
[3921/6006] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\OpenACCClause.cpp.obj
[3922/6006] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\ParentMap.cpp.obj
[3923/6006] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\MicrosoftCXXABI.cpp.obj
[3924/6006] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\OSLog.cpp.obj
[3925/6006] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\Mangle.cpp.obj
[3926/6006] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\ItaniumCXXABI.cpp.obj
[3927/6006] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\PrintfFormatString.cpp.obj
[3928/6006] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\QualTypeNames.cpp.obj
[3929/6006] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\ODRHash.cpp.obj
[3930/6006] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\ODRDiagsEmitter.cpp.obj
[3931/6006] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\OpenMPClause.cpp.obj
[3932/6006] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\Interp\Compiler.cpp.obj
[3933/6006] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\MicrosoftMangle.cpp.obj
[3934/6006] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\JSONNodeDumper.cpp.obj
[3935/6006] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\ParentMapContext.cpp.obj
[3936/6006] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\ASTImporter.cpp.obj
[3937/6006] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\Interp\Disasm.cpp.obj
[3938/6006] Building CXX object tools\clang\lib\AST\CMakeFiles\obj.clangAST.dir\ItaniumMangle.cpp.obj

tbaederr added a commit that referenced this pull request Jul 14, 2024
…70306)"

This reverts commit fa133d3.

It looks like this has some more serious problems:
https://lab.llvm.org/buildbot/#/builders/39/builds/528

As well as build failures on MacOS.
@DavidSpickett
Copy link
Collaborator

@tbaederr
Copy link
Contributor Author

tbaederr commented Jul 16, 2024

You can also try it under qemu - https://linaro.atlassian.net/wiki/spaces/TCWGPUB/pages/29360783374/How+to+Reproduce+32+bit+Arm+Builds+Without+Hardware.

Using those instructions, I get lots of errors, even on things that should be working, like the tools/clang/unittests/AST/Interp/InterpTests.

@DavidSpickett
Copy link
Collaborator

I know there is UB in there (#94741) so this is not unsurprising. Though so far none of that has been the cause of failures natively.

Let's see what Leandro finds.

@tbaederr
Copy link
Contributor Author

I know there is UB in there (#94741) so this is not unsurprising.

Ah, interesting. Didn't know about that issue

@luporl
Copy link
Contributor

luporl commented Jul 16, 2024

I have been able to reproduce the issue on ARM hardware and collected the info below.
This was with a release build. I'll check if I can get more details with a debug build.

Program received signal SIGSEGV, Segmentation fault.
memset () at ../sysdeps/arm/memset.S:51
51	../sysdeps/arm/memset.S: No such file or directory.
(gdb) bt
#0  memset () at ../sysdeps/arm/memset.S:51
#1  0x05ad8eec in clang::interp::DynamicAllocator::allocate(clang::interp::Descriptor const*, unsigned int) [clone .part.0] ()
#2  0x05b2027a in clang::interp::Interpret(clang::interp::InterpState&, clang::APValue&) ()
#3  0x05ac96fe in clang::interp::Call(clang::interp::InterpState&, clang::interp::CodePtr, clang::interp::Function const*, unsigned int) ()
#4  0x05cd1dcc in clang::interp::Compiler<clang::interp::EvalEmitter>::VisitCallExpr(clang::CallExpr const*) ()
#5  0x05cd0670 in clang::interp::Compiler<clang::interp::EvalEmitter>::visit(clang::Expr const*) ()
#6  0x05cdbbda in clang::interp::Compiler<clang::interp::EvalEmitter>::VisitCastExpr(clang::CastExpr const*) ()
#7  0x05cd0670 in clang::interp::Compiler<clang::interp::EvalEmitter>::visit(clang::Expr const*) ()
#8  0x05cd541a in clang::interp::Compiler<clang::interp::EvalEmitter>::visitExpr(clang::Expr const*) ()
#9  0x05a26b64 in clang::interp::EvalEmitter::interpretExpr(clang::Expr const*, bool) ()
#10 0x05a20c58 in clang::interp::Context::evaluateAsRValue(clang::interp::State&, clang::Expr const*, clang::APValue&) ()
#11 0x059ca096 in EvaluateAsRValue((anonymous namespace)::EvalInfo&, clang::Expr const*, clang::APValue&) ()
#12 0x059cac2c in clang::Expr::isCXX11ConstantExpr(clang::ASTContext const&, clang::APValue*, clang::SourceLocation*) const ()
#13 0x059caeb2 in EvaluateCPlusPlus11IntegralConstantExpr(clang::ASTContext const&, clang::Expr const*, llvm::APSInt*, clang::SourceLocation*) ()
#14 0x059cceb8 in clang::Expr::getIntegerConstantExpr(clang::ASTContext const&, clang::SourceLocation*) const ()
#15 0x04d93f24 in AnalyzeImplicitConversions(clang::Sema&, clang::Expr*, clang::SourceLocation, bool) [clone .constprop.0] ()
#16 0x04d94d3c in clang::Sema::CheckCompletedExpr(clang::Expr*, clang::SourceLocation, bool) ()
#17 0x050d7ec4 in clang::Sema::ActOnFinishFullExpr(clang::Expr*, clang::SourceLocation, bool, bool, bool) ()
#18 0x04f20652 in clang::Sema::BuildStaticAssertDeclaration(clang::SourceLocation, clang::Expr*, clang::Expr*, clang::SourceLocation, bool) ()
#19 0x04f20b78 in clang::Sema::ActOnStaticAssertDeclaration(clang::SourceLocation, clang::Expr*, clang::Expr*, clang::SourceLocation) ()
#20 0x04c37dba in clang::Parser::ParseStaticAssertDeclaration(clang::SourceLocation&) ()
#21 0x04c2d022 in clang::Parser::ParseDeclaration(clang::DeclaratorContext, clang::SourceLocation&, clang::ParsedAttributes&, clang::ParsedAttributes&, clang::SourceLocation*) ()
#22 0x04c00bb0 in clang::Parser::ParseExternalDeclaration(clang::ParsedAttributes&, clang::ParsedAttributes&, clang::ParsingDeclSpec*) ()
#23 0x04c3ddae in clang::Parser::ParseInnerNamespace(llvm::SmallVector<clang::Parser::InnerNamespaceInfo, 4u> const&, unsigned int, clang::SourceLocation&, clang::ParsedAttributes&, clang::BalancedDelimiterTracker&) ()
#24 0x04c403d0 in clang::Parser::ParseNamespace(clang::DeclaratorContext, clang::SourceLocation&, clang::SourceLocation) ()
#25 0x04c2d0d0 in clang::Parser::ParseDeclaration(clang::DeclaratorContext, clang::SourceLocation&, clang::ParsedAttributes&, clang::ParsedAttributes&, clang::SourceLocation*) ()
#26 0x04c00bb0 in clang::Parser::ParseExternalDeclaration(clang::ParsedAttributes&, clang::ParsedAttributes&, clang::ParsingDeclSpec*) ()
#27 0x04c01594 in clang::Parser::ParseTopLevelDecl(clang::OpaquePtr<clang::DeclGroupRef>&, clang::Sema::ModuleImportState&) ()
#28 0x04bf63e0 in clang::ParseAST(clang::Sema&, bool, bool) ()
#29 0x0396e5ea in clang::FrontendAction::Execute() ()
#30 0x03916b90 in clang::CompilerInstance::ExecuteAction(clang::FrontendAction&) ()
#31 0x03a0b2b6 in clang::ExecuteCompilerInvocation(clang::CompilerInstance*) ()
#32 0x00e14120 in cc1_main(llvm::ArrayRef<char const*>, char const*, void*) ()
#33 0x00e0f402 in ExecuteCC1Tool(llvm::SmallVectorImpl<char const*>&, llvm::ToolContext const&) ()
#34 0x00e117e6 in clang_main(int, char**, llvm::ToolContext const&) ()
#35 0x00d68a5e in main ()
   0xf7cac25c <+60>:	subscs	r2, r2, #8
   0xf7cac260 <+64>:	stmiacs	r3!, {r1, r12}
   0xf7cac264 <+68>:	subscs	r2, r2, #8
=> 0xf7cac268 <+72>:	stmiacs	r3!, {r1, r12}
   0xf7cac26c <+76>:	bcs	0xf7cac24c <memset+44>
   0xf7cac270 <+80>:	and	r2, r2, #7
   0xf7cac274 <+84>:	subs	r2, r2, #1
End of assembler dump.
(gdb) p/x $r3
$1 = 0x9fda000
(gdb) p/x $r1
$2 = 0x0
(gdb) p/x $r12
$3 = 0x0

@luporl
Copy link
Contributor

luporl commented Jul 16, 2024

In my debug build, clang is receiving a SIGSEGV in frame 7 below (that is actually frame 0, as the frames below it are from a gdb call that received a signal). In it, this holds an invalid address.
In frame 8, NE is 0x3fffffff, which probably caused the invalid access.
In frame 12, NumElements.V is 0xffffffff, which may be the cause of the huge NE value later.

I stopped there, as I lack experience with clang's sources.
Please let me know if you need more information or help with reproducing the issue with qemu.

#7  0xe55f55cc in clang::interp::Integral<32u, true>::Integral (this=0x57f000) at /home/leandro.lupori/git/ci/llvm/clang/lib/AST/Interp/Integral.h:69
#8  0xe55f3e4e in ctorArrayTy<clang::interp::Integral<32, true> > (Ptr=0x575a20, D=0x577240) at /home/leandro.lupori/git/ci/llvm/clang/lib/AST/Interp/Descriptor.cpp:48
#9  0xe5639e06 in clang::interp::Block::invokeCtor (this=0x5759e8) at /home/leandro.lupori/git/ci/llvm/clang/lib/AST/Interp/InterpBlock.h:120
#10 0xe56e5f52 in clang::interp::DynamicAllocator::allocate (this=0xfffec530, D=0x577240, EvalID=88) at /home/leandro.lupori/git/ci/llvm/clang/lib/AST/Interp/DynamicAllocator.cpp:69
#11 0xe56e5df6 in clang::interp::DynamicAllocator::allocate (this=0xfffec530, Source=0x575220, T=clang::interp::PT_Sint32, NumElements=4294967295, EvalID=88)
    at /home/leandro.lupori/git/ci/llvm/clang/lib/AST/Interp/DynamicAllocator.cpp:50
#12 0xe5644402 in clang::interp::AllocN<(clang::interp::PrimType)5, clang::interp::Integral<32u, false> > (S=..., OpPC=..., T=clang::interp::PT_Sint32, Source=0x575220, IsNoThrow=true)
    at /home/leandro.lupori/git/ci/llvm/clang/lib/AST/Interp/Interp.h:2841
#13 0xe56ee890 in clang::interp::Interpret (S=..., Result=...) at /home/leandro.lupori/git/ci/build/tools/clang/lib/AST/Opcodes.inc:901
#14 0xe563f126 in clang::interp::Call (S=..., OpPC=..., Func=0x5760e0, VarArgSize=0) at /home/leandro.lupori/git/ci/llvm/clang/lib/AST/Interp/Interp.h:2554
#15 0xe561e956 in clang::interp::EvalEmitter::emitCall (this=0xfffec428, A0=0x5760e0, A1=0, L=...) at /home/leandro.lupori/git/ci/build/tools/clang/lib/AST/Opcodes.inc:3495
#16 0xe55cb414 in clang::interp::Compiler<clang::interp::EvalEmitter>::VisitCallExpr (this=0xfffec428, E=0x576270)
    at /home/leandro.lupori/git/ci/llvm/clang/lib/AST/Interp/Compiler.cpp:4031
#17 0xe55e2814 in clang::StmtVisitorBase<llvm::make_const_ptr, clang::interp::Compiler<clang::interp::EvalEmitter>, bool>::Visit (this=0xfffec428, S=0x576270)
    at /home/leandro.lupori/git/ci/build/tools/clang/include/clang/AST/StmtNodes.inc:602

tbaederr added a commit that referenced this pull request Jul 17, 2024
@tbaederr
Copy link
Contributor Author

Thanks for all the help with this issue. I think I figured it out - there was an unsigned wraparound some time before doing the actual allocation, which caused us to write over the bounds of the allocation. I've pushed 72b3d7b to ensure this doesn't happen and e94e72a to implement the dynamic allocation management.

yuxuanchen1997 pushed a commit that referenced this pull request Jul 25, 2024
…#70306)"

Summary: This reverts commit 48d703e.

Test Plan: 

Reviewers: 

Subscribers: 

Tasks: 

Tags: 


Differential Revision: https://phabricator.intern.facebook.com/D60251015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
clang:frontend Language frontend issues, e.g. anything involving "Sema" clang Clang issues not falling into any other category
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants