กำหนดบริการและ API ของบริการ

In BentoML a Service is the unit you deploy. It wraps your model and exposes one or more API endpoints clients can call. 🛎️

A runner is BentoML's optimized wrapper around a saved model. It handles inference in its own worker so the API stays responsive.

You build a service by naming it and passing its runners. The Service object is what BentoML serves and scales.

You expose a function as an API with the @svc.api decorator. That function becomes a callable HTTP route.

You set typed IO descriptors so BentoML can validate and serialize. NumpyNdarray tells it to expect and return arrays.

Within the API function you invoke the model through the runner. Using run sends the input to the model worker and returns the result.

BentoML looks for your code in a file by convention. You place the service in service.py at your project root.

One command starts a dev server. bentoml serve points at the file and the svc object, then watches for changes.

Not every API speaks arrays. Swap the descriptor to JSON when your clients send and expect dictionaries instead.

Once it runs, BentoML serves interactive Swagger docs at the root URL so you can test each endpoint in the browser. 🎉

You can hit the live endpoint from your terminal. A quick curl POST sends sample input and shows the prediction it returns.

You want to call your model from inside an API function. What is the right wrapper to use?

You turned a runner into a Service, exposed an @svc.api endpoint with typed IO, dropped it in service.py, and served it with built-in docs. Live API done! 🙌