News
AWS Partners with Facebook on TorchServe Model Server
To help PyTorch users deploy their machine learning models quickly, Amazon Web Services (AWS) has partnered with PyTorch developer Facebook to create the TorchServe open source model server.
Announced Tuesday, TorchServe is a purpose-built framework for deploying PyTorch machine learning models at scale. PyTorch is a Facebook-backed deep learning library for Python, by many accounts the programming language of choice for machine learning developers. PyTorch enjoys similar popularity in the field of deep learning frameworks.
In turn, AWS is a favored platform among PyTorch developers: More than eight of every 10 PyTorch projects are hosted on the AWS cloud, according to an AWS blog post. "With over 83% of the cloud-based PyTorch projects happening on AWS, we are excited to launch TorchServe to address the difficulty of deploying PyTorch models," AWS said.
TorchServe, available on GitHub here, lets developers run their PyTorch models without having to write custom code. It already comes with commonly used handlers, such as those for object detection and text and image classification. It also includes a prediction API. "No more writing lengthy service handlers for initialization, preprocessing, and post-processing," AWS said.
Other features include "multi-model serving, model versioning for A/B testing, monitoring metrics, and RESTful endpoints for application integration." TorchServe also supports the Amazon SageMaker machine learning platform and the Amazon Elastic Container Service for Kubernetes.
"With TorchServe, you can deploy PyTorch models in either eager or graph mode using TorchScript, serve multiple models simultaneously, version production models for A/B testing, load and unload models dynamically, and monitor detailed logs and customizable metrics," AWS said.
Among the early adopters of TorchServe are Toyota, whose research division is using the service to deploy PyTorch models to fleets of automated vehicles, and Matroid, a maker of "computer vision" software that's using TorchServe in its efforts around object detection.