BigTableInstance.getRows sometimes returns 0 values silently

[arbesfeld]

I noticed when running getRows that I would sometimes get 0 results. No error was thrown so this took a while to uncover, and eventually I had to do it in batches:

  let rows = [];
  const MAX_KEYS = 1000;

  for (const i = 0; i * MAX_KEYS < rowIds.length; i++) {
    const [newRows] = await eventsTable.getRows({
      decode: false,
      keys: rowIds.slice(i * MAX_KEYS, (i + 1) * MAX_KEYS),
      filter: [{
        column: {
          cellLimit: 1,
        },
      }],
    });

    rows = rows.concat(newRows);
  };

I'm not sure if this is even the right approach, since 1000 seems pretty arbitrary.

Potentially this is related to #1755? However I was only using ~9k keys which where ~100 characters each which should be under the 4mb limit.

[callmehiphop]

So I believe in v1 you weren't allowed to send more than 1mb worth of keys (after serialization). I'm not sure if such a limit exists in v2 however.

/cc @lesv

[stephenplusplus]

@lesv any insight on this one? Thanks!

[stephenplusplus]

@callmehiphop anyone else from Bigtable you can ping for us?

[callmehiphop]

/cc @garye @sduskis

[sduskis]

I don't know enough about this case to give a clear answer. Is the table populated with data for each of the keys? Has someone tried to reproduce this case? There isn't a known issue for the service failing, but I can to reproduce this with java and see if it succeeds.

[stephenplusplus]

@callmehiphop wdyt?

[callmehiphop]

@arbesfeld are you still seeing this?

[arbesfeld]

I'm not sure - I did a workaround by making requests to Bigtable with smaller chunks of keys and that seemed to fix the issue.

[stephenplusplus]

Cool, I think we can close this out. We have a note in our docs advising to chunk up requests as well.