Tensorfuse Howto

Connect your cloud account: Connect your cloud account (AWS, GCP or Azure) to Tensorfuse. Tensorfuse will automatically provision the resources to manage your infrastructure.

Describe your environment: Use Python to describe your container images and hardware specifications. No YAML required. For example, use tensorkube.Image to specify the base image, Python version, apt packages, pip packages, environment variables, etc.

Define your model loading function: Use the @tensorkube.entrypoint decorator to define a function that loads your model onto the GPU. Specify the image and GPU type to use.

Define your inference function: Use the @tensorkube.function decorator to define your inference function. This function will handle incoming requests and return predictions.

Deploy your model: Deploy your ML model to your own cloud via the Tensorfuse SDK. Your model and data will remain within your private cloud.

Start using the API: Begin using your deployment through an OpenAI-compatible API endpoint provided by Tensorfuse.

Monitor and scale: Tensorfuse will automatically scale your deployment in response to incoming traffic, from zero to hundreds of GPU workers in seconds.