Inference and serving#
The serving code is isolated from the rest of the framework (i.e., training, evaluation) and has minimal dependencies.
Changes to serving-related code are guaranteed to be backward compatible.
Model inference is deterministic: the same example is guaranteed to yield the same prediction.
Learners and models are extensively tested, including integration testing on real datasets; and, there exists no execution path in the serving code that crashes as a result of an error; Instead, in case of failure (e.g., malformed input example), the inference code returns a util::Status.
Hyper-parameters’ semantic is never modified.
The default value of hyper-parameters is never modified.
The default value of a newly-introduced hyper-parameter is set in such a way that the hyper-parameter is effectively disabled.
The following mechanisms will be put in place to ensure the quality of the library:
Training benchmarks with ranges of acceptable evaluation metrics.