This article explores what actually happens when Apache Airflow runs on spot instances, using real experiments to simulate node preemption across both control plane and worker nodes. It walks through how tasks recover using retries, how S3 enables checkpointing without rerunning previous steps, and how to handle partial outputs through validation strategies like success markers. It also highlights the limitations of this approach, particularly around the Airflow metadata database, and outlines the architectural patterns required to build a fault-tolerant Airflow system on interruptible infrastructure.

This article investigates why some Kubernetes storage classes on Rackspace Spot attach volumes faster than others. Users have occasionally reported Pods stuck in Pending due to PVC attachment delays. By examining the architecture of Cinder CSI and Ceph CSI, we show how multiple control plane layers in Cinder can introduce lifecycle latency, and how Ceph’s architecture reduces these delays.

Running Apache Airflow entirely on Spot instances isn’t the standard approach most teams take, but we decided to try something different on Kubernetes. Using Rackspace Spot’s auction-based market with lower preemption risk, this guide shows how an all-Spot setup can be both cost-efficient and practical to run.

We're excited to announce availability of Rook/Ceph as our next-generation Container Storage Interface (CSI) provider in Spot. A new StorageClass, ‘spot-ceph’ is now available in Spot’s IAD and ORD data-centers. Specifying this storage class when using PVCs will use the ceph-csi driver to mount your persistent volumes on managed Ceph clusters in Spot.

An in-depth exploration of Spot Instance history, examining how pricing models evolved and why those changes matter for reliability, cost, and cloud strategy.

Discover Rackspace Spot, the auction-based cloud marketplace that democratizes compute resources. Bid transparently and save up to 90% on costs!