KETTLE-68: Implement policy-driven automatic retry behaviour in DataSources

Metadata

Source
KETTLE-68
Type
Improvement
Priority
Major
Status
Open
Resolution
N/A
Assignee
Antranig Basman
Reporter
Antranig Basman
Created
2018-06-13T11:11:17.839-0400
Updated
2018-06-13T11:11:17.839-0400
Versions
N/A
Fixed Versions
N/A
Component
N/A

Description

As part of the discussions surrounding the GPII's UX for failed user logins, we decided that it would be useful to have a configurable retry behaviour built into Kettle's DataSources. This would operate an exponential backoff policy -
i) After some configurable period of not receiving a valid response through any cause (including slow/dropped connection, explicit error), the datasource would schedule a retry
ii) The retry period would rise exponentially (e.g. through doubling) up to some maximum configurable threshold
iii) After some further configurable period of operating this policy without response, an error would finally be returned to the upstream user - this would consist of the most recent error payload received from downstream, or a synthetic one in the case of repeated timeouts

This would be packaged as a standard request wrapper that could be applied to any DataSource. Much as with the queuing/throttling work in https://issues.fluidproject.org/browse/FLUID-5542 and https://issues.fluidproject.org/browse/FLUID-6209 , it looks like this can't be accommodated within the existing reuse model of "transforming promise chain" which might need to be rethought.