ORC-1858: [C++] Add support to get StripeStatistics without row index

[wushap]

What changes were proposed in this pull request?

add a C++ API in reader to get stripe level statistics without reading row group index.

Why are the changes needed?

To #2137

How was this patch tested?

UT PASS

Was this patch authored or co-authored using generative AI tooling?

NO

[dongjoon-hyun]

Thank you for making a PR, @wushap .

cc @wgtmac

[ffacs]

@wushap Gently ping, could you make the CI happy?

[ffacs]

std::make_unique would be better

[wgtmac]

Thanks for adding this! Could you please create a JIRA issue for this? If you don't have one, please request one from https://selfserve.apache.org/jira-account.html.

[wushap]

Thanks for adding this! Could you please create a JIRA issue for this? If you don't have one, please request one from https://selfserve.apache.org/jira-account.html.

https://issues.apache.org/jira/browse/ORC-1858
here

[wgtmac]

I'd prefer not to introduce a new API, especially when the name of getStripeStatisticsOnly is a little bit misleading. This is not your fault, getStripeStatistics does not clearly state it carries row index statistics as well :(

What about changing the existing API like below:

virtual std::unique_ptr<StripeStatistics> getStripeStatistics(uint64_t stripeIndex, bool includeRowIndex = true) const = 0;

Then return different StripeStatistics subclasses depending on the value of includeRowIndex.

  class StripeStatisticsImpl : public StripeStatistics {
   private:
    std::unique_ptr<StatisticsImpl> columnStats_;

    // DELIBERATELY NOT IMPLEMENTED
    StripeStatisticsImpl(const StripeStatisticsImpl&);
    StripeStatisticsImpl& operator=(const StripeStatisticsImpl&);

   public:
    StripeStatisticsImpl(const proto::StripeStatistics& stripeStats,
                         const StatContext& statContext);

    virtual const ColumnStatistics* getColumnStatistics(uint32_t columnId) const override {
      return columnStats_->getColumnStatistics(columnId);
    }

    uint32_t getNumberOfColumns() const override {
      return columnStats_->getNumberOfColumns();
    }

    virtual const ColumnStatistics* getRowIndexStatistics(uint32_t columnId,
                                                          uint32_t rowIndex) const override {
      throw NotImplementedYet("getRowIndexStatistics");
    }

    virtual ~StripeStatisticsImpl() override;

    uint32_t getNumberOfRowIndexStats(uint32_t columnId) const override {
      throw NotImplementedYet("getNumberOfRowIndexStats");
    }
  };

  class StripeStatisticsWithRowIndexImpl : public StripeStatisticsImpl {
   private:
    std::vector<std::vector<std::shared_ptr<const ColumnStatistics> > > rowIndexStats_;

   public:
    StripeStatisticsWithRowIndexImpl(const proto::StripeStatistics& stripeStats,
                         std::vector<std::vector<proto::ColumnStatistics> >& indexStats,
                         const StatContext& statContext);

    virtual const ColumnStatistics* getRowIndexStatistics(uint32_t columnId,
                                                          uint32_t rowIndex) const override {
      // check id indices are valid
      return rowIndexStats_[columnId][rowIndex].get();
    }

    virtual ~StripeStatisticsWithRowIndexImpl() override;

    uint32_t getNumberOfRowIndexStats(uint32_t columnId) const override {
      return static_cast<uint32_t>(rowIndexStats_[columnId].size());
    }
  };

[wushap]

This looks better

[wgtmac]

+1

Thanks @wushap!

[dongjoon-hyun]

+1, LGTM. Thank you, @wushap and @wgtmac .
Merged to main for Apache ORC 2.2.0.

[dongjoon-hyun]

Welcome to the Apache ORC community and congratulation for your first commit, @wushap .

I added you to the Apache ORC contributor group and assigned ORC-1858 to you.