Skip to content

Latest commit

 

History

History
140 lines (120 loc) · 7.72 KB

README.md

File metadata and controls

140 lines (120 loc) · 7.72 KB

KFServing Python SDK

Python SDK for KFServing Server and Client.

Installation

KFServing Python SDK can be installed by pip or Setuptools.

pip install

pip install kfserving

Setuptools

Install via Setuptools.

python setup.py install --user

(or sudo python setup.py install to install the package for all users)

KFServing Server

KFServing's python server libraries implement a standardized KFServing library that is extended by model serving frameworks such as Scikit Learn, XGBoost and PyTorch. It encapsulates data plane API definitions and storage retrieval for models.

KFServing provides many functionalities, including among others:

  • Registering a model and starting the server
  • Prediction Handler
  • Liveness Handler
  • Readiness Handlers

KFServing supports the following storage providers:

  • Google Cloud Storage with a prefix: "gs://"
    • By default, it uses GOOGLE_APPLICATION_CREDENTIALS environment variable for user authentication.
    • If GOOGLE_APPLICATION_CREDENTIALS is not provided, anonymous client will be used to download the artifacts.
  • S3 Compatible Object Storage with a prefix "s3://"
    • By default, it uses S3_ENDPOINT, AWS_ACCESS_KEY_ID, and AWS_SECRET_ACCESS_KEY environment variables for user authentication.
  • Azure Blob Storage with the format: "https://{$STORAGE_ACCOUNT_NAME}.blob.core.windows.net/{$CONTAINER}/{$PATH}"
  • Local filesystem either without any prefix or with a prefix "file://". For example:
    • Absolute path: /absolute/path or file:///absolute/path
    • Relative path: relative/path or file://relative/path
    • For local filesystem, we recommended to use relative path without any prefix.
  • Persistent Volume Claim (PVC) with the format "pvc://{$pvcname}/[path]".
    • The pvcname is the name of the PVC that contains the model.
    • The [path] is the relative path to the model on the PVC.
    • For e.g. pvc://mypvcname/model/path/on/pvc
  • Generic URI, over either HTTP, prefixed with http:// or HTTPS, prefixed with https://. For example:
    • https://<some_url>.com/model.joblib
    • http://<some_url>.com/model.joblib

KFServing Client

Getting Started

KFServing's python client interacts with KFServing APIs for executing operations on a remote KFServing cluster, such as creating, patching and deleting of a InferenceService instance. See the Sample for KFServing Python SDK Client to get started.

Documentation for Client API

Class Method Description
KFServingClient set_credentials Set Credentials
KFServingClient create Create InferenceService
KFServingClient get Get or watch the specified InferenceService or all InferenceServices in the namespace
KFServingClient patch Patch the specified InferenceService
KFServingClient replace Replace the specified InferenceService
KFServingClient rollout_canary Rollout the traffic on canary version for specified InferenceService
KFServingClient promote Promote the canary version of the InferenceService to default
KFServingClient delete Delete the specified InferenceService
KFServingClient wait_isvc_ready Wait for the InferenceService to be ready
KFServingClient is_isvc_ready Check if the InferenceService is ready

Documentation For Models