Union.ai Introduction

Union.ai works by providing a Python-driven experience where developers and data scientists can define, manage, and execute complex workflows using familiar constructs and libraries. It handles infrastructure provisioning and scaling automatically, allowing users to focus on their core work. Union.ai supports local development with seamless transition to remote execution. It offers declarative infrastructure, where users express their requirements and Union.ai takes care of provisioning and configuring the necessary resources. The platform supports various compute frameworks like Ray, Spark, and Dask, as well as distributed training. Union.ai also provides enhanced performance through workflow caching, optimized file reads, and a fine-tuned execution engine. It allows work with different GPU types and offers secure multi-cloud capabilities with full data lineage, versioning, and reproducibility.