Reduce Cloud Costs with Auction Driven Pricing: Rackspace Spot

This article explores what actually happens when Apache Airflow runs on spot instances, using real experiments to simulate node preemption across both control plane and worker nodes. It walks through how tasks recover using retries, how S3 enables checkpointing without rerunning previous steps, and how to handle partial outputs through validation strategies like success markers. It also highlights the limitations of this approach, particularly around the Airflow metadata database, and outlines the architectural patterns required to build a fault-tolerant Airflow system on interruptible infrastructure.

Running Apache Airflow entirely on Spot instances isn’t the standard approach most teams take, but we decided to try something different on Kubernetes. Using Rackspace Spot’s auction-based market with lower preemption risk, this guide shows how an all-Spot setup can be both cost-efficient and practical to run.

This article investigates why some Kubernetes storage classes on Rackspace Spot attach volumes faster than others. Users have occasionally reported Pods stuck in Pending due to PVC attachment delays. By examining the architecture of Cinder CSI and Ceph CSI, we show how multiple control plane layers in Cinder can introduce lifecycle latency, and how Ceph’s architecture reduces these delays.

We're excited to announce availability of Rook/Ceph as our next-generation Container Storage Interface (CSI) provider in Spot. A new StorageClass, ‘spot-ceph’ is now available in Spot’s IAD and ORD data-centers. Specifying this storage class when using PVCs will use the ceph-csi driver to mount your persistent volumes on managed Ceph clusters in Spot.

An in-depth exploration of Spot Instance history, examining how pricing models evolved and why those changes matter for reliability, cost, and cloud strategy.

Discover Rackspace Spot's low-cost managed database service using PostgreSQL. Enjoy high availability, automatic provisioning, and cloud-native architecture.

Discover the hidden expenses of deploying a simple web app on Kubernetes. Learn about the infrastructure costs and resource consumption involved.

Discover Rackspace Spot, the auction-based cloud marketplace that democratizes compute resources. Bid transparently and save up to 90% on costs!