processRecords failure - correct handling ?

[walshe-boylesoftware]

if processRecords() fails for any one record, how should one handle it ? @sahilpalvia

[sahilpalvia]

That depends on the application/downstream processing logic. If processRecords method throws an exception the KCL will move on to the next set of records.

You could have a try catch block around the record or batch processing and retry if it fails.

[walshe-boylesoftware]

yes we have a retry strategy, but just wondering what should actually happen if some exception happens there too (causing both checkpoint and completeCallback to not happen ).

I thought that the next batch of records cannot be delivered to the processor until completeCallback() was called ? @sahilpalvia

[sahilpalvia]

The NodeJS KCL will send the completed status to the MultiLangDaemon only when callback is invoked. If the control exists the function without invoking the callback it is possible that the KCL will get stuck.

[sahilpalvia]

Closing this issue out. Please feel free to reopen if problem persists.