Release Notes

August 2025

NEW Spot CLI, Gen-1 Load Balancer enhancements, Rapid Billing rollout, API validation and reliability fixes.

Features and Enhancements

  • Spot CLI (spotctl) BETA :

    • New command-line interface (spotctl) is now available in BETA, allowing users to automate Kubernetes cluster creation and management. The CLI supports full CRUD operations for Cloudspaces, Node Pools (both Spot and On-Demand), and includes helpful discovery commands for regions, organizations, server classes, and pricing. For detailed usage instructions, see the Spot CLI Documentation. If you encounter issues, please report them on GitHub Discussions.

  • Gen-1 Load Balancers: HTTP/UDP, source IP preservation and access lists:

    • Gen-1 Load Balancers now support both HTTP and UDP protocols. They also support source IP preservation and allow you to restrict access using source ranges (access lists). See Source IP preservation and Load Balancer source ranges for details.

  • Rapid Billing for all users:

    • All organizations have been migrated from monthly billing to Rapid Billing. Invoices are generated automatically when usage reaches a small threshold (typically $25), helping reduce fraud and ensure fair access to capacity. For organizations that prefer to maintain monthly billing - Email Us

  • Other improvements:

    • API validation: Only one Spot node pool per Cloudspace can have autoscaling enabled.
    • Fixed an issue where some new accounts showed "subscription not active" immediately after creation.
    • General backend reliability improvements and bug fixes.

July 2025

Database as a Service (BETA), Multi-Factor Authentication, Events Dashboard, Enhanced API capabilities and more.

Features and Enhancements

  • Database as a Service (BETA) - PostgreSQL Support:

    • Rackspace Spot now offers PostgreSQL Database as a Service in beta, extending the platform's capabilities beyond Kubernetes infrastructure. This managed database service is currently available for Cloudspaces deployed in DFW-1 or DFW-2 regions. PostgreSQL instances are provisioned with publicly accessible endpoints, enabling seamless integration with applications both inside and outside your Cloudspace.

  • Multi-Factor Authentication for Organizations:

    • Enhanced security is now available through organization-wide Multi-Factor Authentication. Organization owners can enforce MFA requirements for all users within their organization, providing an additional layer of protection for sensitive Kubernetes workloads and infrastructure management.

  • Events Dashboard and Monitoring:

    • Spot now provides comprehensive event visibility through a new Events dashboard. Users can monitor two distinct event streams: Cloudspace events generated from within the Kubernetes cluster itself, and Organization events generated by the Spot backend based on organizational actions and changes. The Events dashboard currently displays events from the past 24 hours, giving users real-time insight into cluster activity and organizational changes.

  • Enhanced API Capabilities - Kubeconfig Generation:

    • Building upon the Public API, Spot now supports kubeconfig generation directly through API token authentication. This capability allows developers and automation tools to programmatically obtain valid kubeconfig files for their Cloudspaces without manual intervention, enabling seamless integration with CI/CD pipelines and infrastructure automation workflows. (POST - Generate kubeconfig (unauthenticated API))

  • Python SDK for Rackspace Spot:

    • A comprehensive Python SDK is now available for Rackspace Spot, enabling developers to integrate Spot capabilities directly into their Python applications and automation scripts. The SDK provides programmatic access to core Spot functionality and is open-source and available on GitHub.

  • Other enhancements:

    • Enhanced Billing and Cost Explorer Precision: Improvements to billing accuracy and Cost Explorer calculations provide more precise cost tracking and reporting.
    • Feature Availability Matrix: A comprehensive feature availability matrix is now available in the documentation, providing clear visibility into which features are supported in each region. (Feature Availability by Region)

Known Issues & Limitations

  • Database as a Service (BETA) Regional Availability: PostgreSQL Database as a Service is currently available only in DFW-1 and DFW-2 regions during the beta phase.
  • Events Dashboard Time Range: The Events dashboard currently shows events for the past 24 hours only. Extended historical event data will be available in upcoming releases.

May 2025

Public API, Fractional GPU allocation, Control Plane Health metrics, Improved Fraud Prevention and more.

Features and Enhancements

  • Public API for Rackspace Spot, Kubeconfig management:

    • Rackspace Spot now supports a public API, addressing a long-standing roadmap item. The API provides all key functionality required to use and customize Spot.
    • In addition, the API will (in the very near future, pending some end user feedback) publish a new unauthenticated API to help users manage Cloudspace kubeconfigs. API clients can now use a refresh token (same token used with the Terraform provider) downloaded from the user interface to call this authenticated API, enabling them to get a valid kubeconfig for their desired Kubernetes Cloudspace.

  • Fractional GPU Allocation (Mutli-Instance GPU):

    • Cloudspaces using NVIDIA A30 and NVIDIA H100 GPUs can now leverage fractional GPU allocation to optimize resource allocation. Multi-instance GPUs (MIG) can be enabled from the Cloudspace overview page by clicking on 'Fractional GPU', and then choosing the MIG profile to partition the physical GPU into multiple isolated GPU profiles, each with dedicated CPU and memory resources.

  • Control Plane Health Metrics:

    • Spot has always provided a free, non-production Kubernetes control plane that is currently centrally hosted in our US data-centers. To provide users with better visibility into control plane's performance and available capacity, Spot now publishes the API server response latency in the Cloudspace 'Capacity & Health' panel.

  • Improved Fraud Prevention:

    • To prevent market price changes from bad actors,Spot now places a hold for a small $ amount before allowing a payment card to be used. The amount held is refunded back within 7 days (and is not used to pay any invoices generated).
    • New users or new organizations will be billed as soon as they accrue $1 worth of usage. Upon successfully paying that amount, their invoice amount is increased incrementally.
    • Automatic suspension: Organizations will be automatically suspended after 5 successive invoice payment failures. Once suspended, any bids from that organization are withdrawn from the auction, which means any Cloudspaces in the organization will lose access to their worker nodes. Any PVCs and load balancers are held for an additional grace period. If the suspended account doesn't pay the outstanding amount, the organization will be deleted.

  • Minimum bid prices for Gen-1 servers:

    • To ensure Spot can continue to offer sufficient capacity, we are adjusting the minimum bid prices for Large, Extra Large and Double Extra Large Gen-1 servers. Revised minimum bid prices for these server classes enable us to pay for the power and operating costs of these servers, allowing us to offer them in sufficient quantity for a healthy, liquid market auction.
    • All existing customers' bids remain valid and will continue to be honored; therefore, the market prices for these servers will remain unchanged in the near term and will continue to be determined by the Spot market auction.

  • Other enhancements:

    • Delete pending invitation: users can now delete a pending invitation to add a new user.
    • Billing accuracy: use of 3rd party load balancers or persistent volumes will be distinguished from use of Rackspace native solutions, and users are only billed when using the Rackspace provided load balancer or persistent storage.
    • Billing accuracy: improved accuracy due to increased precision.
    • Validations to allow changes for custom labels, custom annotations and custom taints in Cloudspace NodePools.
    • Node availability: fixed a race condition where some nodes were getting cordoned wrongly if they were pre-empted and won back in quick succession.
    • Node availability: improved reliability of node recycling.

Known Issues & Limitations

  • The NVIDIA GPU Operator is currently compatible with Kubernetes v1.30.10; and is not yet supported with Kubernetes v1.31.1.

April 2025

New high performance hardware, 1-click GPU operator, user management and more.

Features and Enhancements

  • New Gen-2 Data-centers, Performance Optimized Infrastructure:

    • Enterprise class shared storage, blazing fast CPUs, bigger and faster pipes: Spot now offers a full selection of state of the art, shiny new infrastructure. US-West-SJC-1 (San Jose, CA) and US-Central-DFW-2 (Dallas, TX) are new generation 2 data-centers, with new performance optimized hardware.

  • 1-click NVIDIA GPU Operator installation:

    • When deploying a Cloudspace with GPUs, Spot now makes it easy to configure the NVIDIA Operator, out of the box, as the Cloudspace is provisioned. To enable the GPU operator, simply click on "Customize Kubernetes Configuration" in Cloudspace options, and enable the NVIDIA GPU Operator switch.

  • Add Users:

    • It is now easy and self-service to invite users to collaborate within your Organizations. Simply navigate to 'Users' under 'Account' and invite users using their email address.

  • Minimum Bid price increment to pre-empt existing workloads:

    • Spot now requires a minimum bid price increment of either 0.5 cent per hr (when market price is less than 5 cents per hour) or 1 cent per hour (when market price is greater than 5 cents per hour). To help make this process more intuitive for new and existing users, the user interface now shows the first 10 available bid prices via a visual bid slider.

Known Issues & Limitations

  • The NVIDIA GPU Operator is currently compatible with Kubernetes v1.30.10; and is not yet supported with Kubernetes v1.31.1.

March 2025

New Kubernetes Versions, Upgrades, Cost Explorer and Customizable CNI

Features and Enhancements

  • New Kubernetes Versions:

    • Spot now supports K8s v1.31.1 as the default version for new Cloudspaces. In addition, Kubernetes v1.30.10 and v1.29.6 are also supported.

  • Kubernetes Upgrades:

    • Cloudspaces running older versions of Kubernetes can now be upgraded to newer versions. When an Upgrade is available, users can navigate to the Cloudspace Overview page and initiate an upgrade to a new version. Upgrades are orchestrated with the control plane upgraded first, then worker nodes upgraded one node at a time.

  • Cost Explorer:

    • Rackspace Spot now provides a 'Cost Explorer' to allow users to better track their spending. Cost Explorer currently shows daily costs for each Cloudspace for the previous 30 days. Daily cost estimates are accurate effective March 10, 2025; numbers shown for dates before then will be inaccurate.

  • Customizable CNIs:

    • When provisioning new Cloudspaces, users now can choose to use a different CNI configuration. Calico with IPTables will continue to be the default CNI for now; but users have two new choices: Cilium with Kubeproxy, and "Bring Your Own CNI". Using Bring Your Own CNI allows advanced users the flexibility to run custom CNI configurations without being "auto-corrected" by Spot health monitoring probes.

  • Other improvements:

    • Bug fixes in cluster auto-scaler should result in more consistent auto-scaling performance.
    • Enhanced cluster health monitoring checking for Kubernetes API server liveness and response latencies.

Known Issues & Limitations

  • Known issue with Persistent Volume Attachment: Users have reported problems with PVCs that get stuck while attaching. We are developing a new solution for user volumes that avoids this issue. In the interim, users who run into this issue can try to "recycle" the nodes that are hosting the Pods that get stuck due to volume dismount failures.
  • Pre-emption notification for Slack webhooks only: Pre-emption notices are currently configured to support Slack webhooks. We expect to support other notification mechanisms in future releases.

February 2025

Features and Enhancements

  • Cloudspace Uptime:

    • A number of improvements over the past several months and weeks resulted in aggregate Cloudspace control plane uptime exceeding 99.9% for the first time in January 2025.

  • New Control Plane DB infrastructure:

    • New product_short_name control planes are now using a new, Cloud-Native-Postgres based DB architecture running on brand new hardware in the San Jose (US-West) datacenter. We've seen vastly improved performance and reliability with this new design.

  • Status Page:

  • Custom Node labels and Annotations:

    • When provisioning or editing a pool of servers (either Spot Servers or On-demand Servers), you can now apply custom labels and annotations that will be applied to all servers in that pool. For instance, you can label the type of server (Spot vs On-demand) or the capacity (16 vCPU vs 8 vCPU) or the amount of memory or any other infrastructure attributes; which can then be used to ensure pods are being run by servers with the right labels.

  • Billing Changes for Market fairness:

    • To preserve fairness in market pricing, new customers will be billed every time they accrue a certain threshold worth of charges (currently, $25). This is being done to ensure that only paying customers are getting access to servers and able to affect market prices for servers.

  • Other improvements:

    • Worker node disk space garbage collection: this parameter is being configured to trigger timely disk space cleanup.
    • Fixed a bug that would caused the Kubernetes endpoint port to drift over time.
    • Fixed a bug that would sometimes show incorrect server cost information in the Capacity UI.
    • Fixed a bug that would prevented users from querying price history for servers.
    • Added the ability to delete a Cloudspace while it is deploying.

Known Issues & Limitations

  • Known issue with Persistent Volume Attachment: Users have reported problems with PVCs that get stuck while attaching. We are developing a new solution for user volumes that avoids this issue. In the interim, users who run into this issue can try to "recycle" the nodes that are hosting the Pods that get stuck due to volume dismount failures.
  • Pre-emption notification for Slack webhooks only: Pre-emption notices are currently configured to support Slack webhooks. We expect to support other notification mechanisms in future releases.

December 2024

Features and Enhancements

  • Cloudspace Provisioning:

    • Gen-1 K8s control planes have been deprecated via both API and UI. Users that are using Gen-1 control planes should deploy new cloudspaces (thereby using the Gen-2 control plane), which offers increased uptime and performance. Gen-1 control planes will be terminated and retired by Jan 31, 2025.
    • Increased concurrency in Cloudspace provisioning, improving provisioning performance.
    • Users can now delete Cloudspaces that are not fully provisioned or stuck in an error state.
    • Fixed a bug that would cause Cloudspace provisioning to fail if the pre-emption notification webhook was invalid.
    • Reconfigured Gen-2 control plane DB layer to improve reliability.

  • Capacity limits:

    • New users are limited to the amount of capacity they can access pending additional verification. While this introduces an additional step, this ensures a fair market for all of our users by ensuring prices are only being set due to bids from validated users.

  • Load Balancer Availability:

    • Several fixes that will greatly improve Load Balancer capacity management, that will greatly reduce the likelihood of a Load Balancer not being available or a Load Balancer re-assignment.

  • Other improvements:

    • Kubernetes service endpoint port: Fixed a bug that would cause the service endpoint port to be sometimes reconfigured incorrectly.
    • Improve Cloudspace resource cleanup on termination or down-sizing.

Known Issues & Limitations

  • Known issue with Persistent Volume Attachment: Users have reported problems with PVCs that get stuck while attaching. Gen-2 Cloudspaces control planes avoid this infrastructure. We are developing a new solution for user volumes that avoids this issue.
  • Bare metal BETA ended: Bare metal capacity, which was previously available in BETA with Gen-1 control planes, is not compatible with Gen-2 control planes. We will track this on our backlog as a possible feature we could reintroduce.
  • Pre-emption notification for Slack webhooks only: Pre-emption notices are currently configured to support Slack webhooks. We expect to support other notification mechanisms in future releases.

October 2024

NVIDIA H100 and A30 GPUs, New 2nd gen data-center (San Jose, US West)

Features and Enhancements

  • New region w/ 2nd generation infrastructure: US-West, San Jose

    • Rackspace Spot now offers next generation server infrastructure in a new data-center location - San Jose, CA (us-west-sjc-1). Gen-2 data-centers run newer infrastructure from the metal up, offering improved performance and reliability.

  • NVIDIA H100 and A30 GPU server classes:

    • San Jose data-center now enables support for two new server classes featuring GPUs!

    • GPU A30 Virtual Server v2. + + Extra Large features:

      • 1x NVIDIA A30 GPU
      • Accelerator Optimized (INTEL 6526Y) CPU with 24 hyperthreaded cores
      • 128 GB Memory
      • Multipath enabled NVMe encrypted storage with 25GiB Networking

    • GPU H100 Virtual Server v2.Mega Extra Large features:

      • 1x NVIDIA H100 GPU
      • Accelerator Optimized (INTEL 8568Y) CPU with 48 hyperthreaded cores
      • 128 GB Memory
      • Multipath enabled NVMe encrypted storage with 25GiB Networking

  • Gen-2 provisioning:

    • Recommended as the default: Gen-2 is now the recommended and default method for provisioning Cloudspaces. Gen-1 remains available for now, but we intend to deprecate Gen-1 to focus development efforts on Gen-2 (subject to user community feedback).
    • Networking: Early users of Gen-2 provisioning experienced problems with networking instability and high CPU usage. This release updates to a newer version of Calico that addresses this problem.
    • Idempotency: Gen-2 provisioning is now idempotent, improving reliability in certain scenarios.

Known Issues & Limitations

  • Known issue with Persistent Volume Attachment: Users have reported problems with PVCs that get stuck while attaching. Gen-1 Cloudspace control planes can be affected by this problem. Gen-2 Cloudspaces control planes avoid this infrastructure.
  • Bare metal limitations: Bare metal capacity has been a challenge with the amount of actual usable capacity often being less than "available" capacity. In addition, Bare metal cloudspaces are not (yet) compatible with Gen-2 provisioning due to infrastructure limitations.
  • Pre-emption notification for Slack webhooks only: Pre-emption notices are currently configured to support Slack webhooks. We expect to support other notification mechanisms in future releases.

September 2024

Features and Enhancements

  • New K8s Provisioning (Gen-2) (Experimental):

    • This release provides a brand new K8s provisioning experience with a completely rewritten K8s control plane. Gen-2 enables a number of advantages, including:

      • A new, Cluster API based K8s control plane
      • Faster, more reliable provisioning of the control plane since it avoids use of certain infrastructure that have negatively affected Gen-1 control planes
      • Faster, more reliable provisioning of worker nodes
      • More scalable organizations with a larger number of Cloudspaces in each organization and larger number of nodes in each Cloudspace
      • More modular and extensible method to manage K8s cluster add-ons

    • Gen-2 provisioning is Experimental as of this release, but has performed well in early testing and will be our focus going forward.

  • Bare Metal Servers (BETA):

    • Bare Metal Servers continue to be in BETA while we monitor the product experience and user feedback.
    • Capacity availability for Bare metal has been a challenge, and in certain situations, the amount of capacity available to Spot has been below expected levels. This has been especially true in smaller sites such as UK (London).
    • In addition, the underlying Bare Metal Server provisioning APIs do not enable some of the optimizations used for Gen-2 Cloudspaces, so Bare Metal Servers are currently only available with Gen-1 provisioning.

  • Long lived tokens for easier automation:

    • Spot now allows you to manage long lived tokens for use with automation frameworks, such as Terraform. You can now create new long lived tokens and revoke existing tokens as necessary for your workflow.

  • Terraform provider: new release v0.1.0:

    • The Spot Terraform provider can use the new long lived API tokens, eliminating the need for frequent token updates. Please ensure your Terraform provider is updated.
    • The provider also allows you to to create Gen-2 cloudspaces by using the optional attribute deployment_type with the value gen2 in the cloudspace resource. If left undefined, or if set to 'gen1', the Cloudspace will be provisioned using the traditional gen-1 provisioning architecture.

  • Other improvements:

    • Verify Cloudspace Health: If you suspect a problem with your Cloudspace, you can request a rescan of the Cloudspace health by a new Cloudspace action: "Verify Cloudspace Health". This will rescan the health of the Cloudspace control plane and worker nodes.

Known Issues & Limitations

  • Known issue with Persistent Volume Attachment: Users have reported problems with PVCs that get stuck while attaching. Gen-1 Cloudspace control planes can be affected by this problem. Gen-2 Cloudspaces control planes avoid this infrastructure.
  • Recommendation to limit Cloudspaces to 50 servers: As a short term measure to improve your provisioning experience, we recommend limiting each Cloudspace to a total of 50 servers, especially for Gen-1 Cloudspaces.
  • Bare metal limitations: Bare metal capacity has been a challenge with the amount of actual usable capacity often being less than "available" capacity. In addition, Bare metal cloudspaces are not (yet) compatible with Gen-2 provisioning due to infrastructure limitations.
  • Time to provision new Cloudspaces and new servers: Gen-1 Cloudspace provisioning typically takes about 30 minutes, and adding new servers as worker nodes typically takes about 20 minutes. Bare metal servers can take up to 40 minutes to be reallocated. Gen-2 Cloudspaces will generally perform at least twice as fast as these.
  • Pre-emption notification for Slack webhooks only: Pre-emption notices are currently configured to support Slack webhooks. We expect to support other notification mechanisms in future releases.

July 2024

Features and Enhancements

  • New region: United Kingdom (London):

    • Access to capacity in London enables closer proximity for our users in greater Europe.

  • Bare Metal Servers (BETA):

    • Rackspace Spot now includes BETA access to bare metal servers in select regions. Bare metal servers can offer higher performance than similarly configured virtual servers. Bare metal servers are available in select regions.

  • Terraform provider: new release v0.0.12:

    • New optional attribute, wait_until_ready in the Cloudspace resource. This optional attribute allows users to specify whether the resource creation should wait until the cloudspace is ready before returning. It defaults to false. This attribute can be used to simplify user automation to block until a Cloudspace is fully ready.
    • Fixed a bug that was causing an invalid kubeconfig.

  • Node Controls:

    • Reboot node: Users can now select a group of nodes and reboot them via the User interface.
    • Recycle node: Users can now select a group of nodes that may be in a bad state and "recycle" them. Recycling a node causes the node to terminated and replaced by a newly provisioned node from the underlying cloud infrastructure.

  • User experience:

    • Price history: Where information is available, users can now lookup spot market price history for their selected server class before placing a bid.

  • Other improvements:

    • Tuned CSI layer timeouts to improve the reliability of persistent volume detach + attach operations that occur when a pod is rescheduled onto another worker node.
    • Disabled "automatic stack reconciliation" in certain situations, such as when the virtual control plane is unreachable by the Kubernetes components on a worker node. This reduces the incidence of unexpected pod termination on worker nodes.
    • Fixed an issue that was causing node metrics to not show up correctly in the Capacity and Health UI.

Known Issues & Limitations

  • Known issue with Persistent Volume Attachment: Users have reported problems with PVCs that get stuck while attaching. This update includes a re-tuned CSI driver that we expect will reduce the incidence rate of the problem.
  • Recommendation to limit Cloudspaces to 50 servers: As a short term measure to improve your provisioning experience, we are recommending limiting each Cloudspace to a total of 50 servers.
  • No persistent token for non-interactive use: Several users have requested a long lived token for non-interactive use. Currently, this token expires every 72 hours.
  • Time to provision new Cloudspaces and new servers: Cloudspace provisioning typically takes about 30 minutes, and adding new servers as worker nodes typically takes about 20 minutes. Bare metal servers can take up to 40 minutes to be reallocated.
  • Pre-emption notification for Slack webhooks only: Pre-emption notices are currently configured to support Slack webhooks. We expect to support other notification mechanisms in future releases.

June 2024

Big news! Spot now includes support for "On-demand" servers, aka servers that will not be pre-empted due to a market auction. This has been a much requested feature that enables users to run a small amount of "stable" capacity in conjunction with use of Spot servers.

Features and Enhancements

  • On-demand Servers (Non pre-emptible):

    • Rackspace Spot now includes support for "On-demand" servers. "On-demand" servers are carved out as a dedicated capacity pool with high availability because they will not be preempted due to fluctuations in Spot market prices

  • Billing:

    • On-demand servers are billed at a stable price that doesn't vary much, if at all. These prices match the average SPOT price of comparable servers from other leading Hyperscalers. You read that correctly: Rackspace Spot is providing on-demand servers at the price that other cloud providers provide Spot servers
    • To minimize any billing surprises, prices for On-demand servers do not vary dynamically, and will be reviewed over time to ensure we are delivering these servers at a very attractive price

  • Use via Terraform: new release v0.0.11:

    • Added support for "On-demand nodes"

  • User experience:

    • Added support for "On-demand" nodes
    • Improved usability of Spot bids based on user feedback
    • Refactored the bid insights tools into a separate section to declutter the bid table, and to enable the addition of new bid insights tools in future (such as price history)

  • Cloudspace provisioning:

    • Improved cleanup of Cloudspace nodes that are terminated or pre-empted. This improves performance and reliability of Cloudspaces that are running at larger scale or experiencing a large amount of node churn due to pre-emption

Known Issues & Limitations

  • Known issue with Persistent Volume Attachment: Users have reported problems with PVCs that get stuck while attaching. We have traced this to a problem with how the CSI provider used by Spot is interacting with the underlying Hypervisor layer and are working on this as a priority
  • Recommendation to limit Cloudspaces to 50 servers: As a short term measure to improve your provisioning experience, we are recommending limiting each Cloudspace to a total of 50 servers.
  • No persistent token for non-interactive use: Several users have requested a long lived token for non-interactive use. Currently, this token expires every 72 hours.
  • Time to provision new Cloudspaces and new servers: Cloudspace provisioning typically takes about 30 minutes, and adding new servers as worker nodes typically takes about 20 minutes.
  • Pre-emption notification for Slack webhooks only: Pre-emption notices are currently configured to support Slack webhooks. We expect to support other notification mechanisms in future releases.

May 2024

Our focus for the May release was to improve stability, reliability and billing accuracy. We also put in place some important building blocks for key new features coming down the pipeline!

Features and Enhancements

  • Kubernetes Control Plane:

    • Fixed a rare bug that could cause the Kubernetes Control Plane to use an incorrect version of the etcd volume when recovering from a failure.

  • Network performance:

    • New Cloudspaces are deployed with Calico/VXLAN to improve pod<>pod networking performance. Existing Cloudspaces will continue to use Calico/IPIP, which has been reported to have inconsistent performance when routed through certain portions of the underlying cloud network infrastructure.

  • Worker Node Configuration:

    • Changed the way CSI and CNI components are deployed onto worker nodes to improve system reliability and security, and to provide more flexibility for future changes to Cloudspace provisioning.
    • New Cloudspaces are deployed with an updated build of Kubernetes components that fixes occasional bugs in mountaing a persistent volume with the message "MountVolume.MountDevice failed for volume 'pvc-...'".

  • Use via Terraform: new release v0.0.9:

    • Users can now provision their first Cloudspace in an organization via Terraform. The Spot User Interface now provides access to the Terraform provider access token once there is a valid payment method on file. Previously, access to the Terraform provider was only provided once the first Cloudspace had been provisioned via the UI.
    • Users can now leverage advanced filtering capabilities on the serverclasses and regions data sources. This update allows for more granular control and precise resource management, making infrastructure deployment more efficient. Refer documentation for example usage.

  • User experience:

    • Refined the Cloudspace bidding UX to more clearly distinguish the use of Spot capacity (vs other options that are coming soon). Server Pools are now more strongly typed in the UX.

  • Cloudspace provisioning:

    • Added checks and fixes to improve provisioning resiliency, especially for larger Cloudspaces.
    • Added guidelines in the UI to limit individual Cloudspaces to 50 servers, as a temporary best practice.
    • Improved cleanup of servers from deleted bids.

  • Billing:

    • Fixed bugs to improve billing accuracy and correctness. Additional validation is performed that a server was ready and fully usable by Kubernetes before billing for that server begins.
    • Added billing for Load Balancers.
    • Detect and prevent use of Prepaid cards, which has been highly correlated with failed payments. To ensure fairness for all users of Spot (and prevent fraudulent use), we are no longer accepting Prepaid cards as a payment method. Spot will not perform ongoing health checks or reconciliation vs bid specification for Cloudspaces associated with a Prepaid card.

Known Issues & Limitations

  • Recommendation to limit Cloudspaces to 50 servers: As a short term measure to improve your provisioning experience, we are recommending limiting each Cloudspace to a total of 50 servers.
  • No persistent token for non-interactive use: Several users have requested a long lived token for non-interactive use. Currently, this token expires every 72 hours.
  • Time to provision new Cloudspaces and new servers: Cloudspace provisioning typically takes about 30 minutes, and adding new servers as worker nodes typically takes about 20 minutes.
  • Only 1 Autoscaling node pool: Cloudspaces currently support up to 1 autoscaling node pool.
  • Pre-emption notification for Slack webhooks only: Pre-emption notices are currently configured to support Slack webhooks. We expect to support other notification mechanisms in future releases.

April 2024

In April, we made several enhancements that make it easier for programmatic use of Spot, including several enhancements to the Terraform provider and the ability to programmatically manage bids using price and capacity insights.

Features and Enhancements

  • Regions & Capacity:

    • Tuned capacity in Hong Kong and Sydney to improve provisioning reliability

  • Terraform provider update: new release v0.0.6:

    • The provider now provides kubeconfig as an output resource for a cloudspace
    • Enhanced the spot_spotnodepool resource to return the outcome of the bid as an output variable. This allows a Terraform user to test whether a pool bid was accepted
    • Importing resources: provider now imports resources using the organization name & cloudspace name, without requiring the user to provide the underlying object identifiers
    • Provider now accepts the api_token as a variable without requiring an environment variable
    • Terraform provider enables the use of OIDC based tokens in the Cloudspace Kubeconfig, allowing for long lived access to the Kubernetes cluster

  • Cloudspace provisioning:

    • Improved reliability and speed in provisioning of large Cloudspaces: Servers are now provisioned in batches, rather than all at once. As a result, Cloudspaces get usable capacity much faster than before, and are provisioned more consistently without getting stuck on an errant worker node

  • Billing:

    • Improved billing based on user feedback. Newly provisioned servers now begin billing from the time when they are first Ready in the Kubernetes cluster. Previously, servers would start billing from the time when they were powered on
    • Server pricing is now available on the Spot pricing page

  • New Docs:

Known Issues & Limitations

  • Only 1 Autoscaling node pool: Cloudspaces currently support up to 1 autoscaling node pool.
  • Pre-emption notification for Slack webhooks only: Pre-emption notices are currently configured to support Slack webhooks. We expect to support other notification mechanisms in future releases

March 2024

In its first month of general availability, Spot saw a ~3000% increase in server demand and bids! It's safe to say we're excited with the traction, but saw a number of opportunities to improve and were in a race against time to keep up with the growth.

Features and Enhancements

  • Regions & Capacity:

    • Added 3 new regions in Chicago (ORD), Hong Kong (HKG) and Sydney (SYD)
    • Released additional capacity in existing regions of Virginia (IAD) and Dallas (DFW)

  • Terraform provider update:

  • Server Classes:

    • Deprecated "General Purpose Virtual Server - Small", the 1vCPU, 1GB memory ServerClass due to poor performance as a Kubernetes worker node. Users who were using this server class should edit their Cloudspace to use other server classes that offer significantly better performance and reliability

  • Kubernetes authentication:

    • User interface alerts the user when kubeconfig access will be denied due to an unverified email address
    • Increased window of kubeconfig access token to 72 hours from 10 hours previously
    • Documented a mechanism for longer lived access via kubelogin

  • Cloudspace provisioning:

    • Improved reliability in Cloudspace node provisioning when encountering provisioning failures in backend "Undercloud"
    • Reduced false alarms in Cloudspace status when a new Cloudspace is being provisioned
    • Improved monitoring for Cloudspace health
    • Improved monitoring for health of K8s control plane components
    • Improved monitoring for persistent volumes that are unable to attach or detach

  • Billing:

    • Aligned billing cycle for new users to the end of each calendar month

  • User experience:

    • Improved, faster signup and login experience

Known Issues & Limitations

  • Only 1 Autoscaling node pool: Cloudspaces currently support up to 1 autoscaling node pool. Based on user feedback, we are happy to relax this constraint and support the ability to autoscale multiple bid pools
  • Pre-emption notification for Slack webhooks only: Pre-emption notices are currently configured to support Slack webhooks. We expect to support other notification mechanisms in future releases
  • Support for refresh tokens: Refresh tokens are not yet supported (stay tuned!)

January 2024

With the January 2024 release, Rackspace Spot is now generally available!

Features and Enhancements

  • Kubernetes Cluster Autoscaling: Cloudspaces now support the ability to autoscale up to one bid for capacity. Bids for capacity create node pools, and one of these node pools can be selected to autoscale
  • Kubernetes authentication with OIDC: Spot Cloudspaces are now configured to enable Kubernetes API authentication with OIDC. This resolves previous limitations such as a lack of Service Account tokens, or inability to support stream connections
  • Access to the Kubernetes Dashboard: You can now directly navigate to the Kubernetes dashboard from the Cloudspaces user interface
  • Spot Price Insights: Based on user feedback, we are exposing additional visibility and insights into the market conditions, via insights into Capacity Availability and comparison vs Hyperscaler prices
  • Terraform provider: Spot now has its own Terraform provider, making it easier to automate and manage Spot Cloudspaces using infrastructure-as-code
  • Multiple Spot Bids: Cloudspaces can now be composed of multiple bids, of any available server classes. This is useful as a hedge against a temporary market spike in any one server class, or for workloads that require a mix of different node sizes

Known Issues & Limitations

  • Only 1 Autoscaling node pool: Cloudspaces currently support up to 1 autoscaling node pool. Based on user feedback, we are happy to relax this constraint and support the ability to autoscale multiple bid pools
  • Pre-emption notification for Slack webhooks only: Pre-emption notices are currently configured to support Slack webhooks. We expect to support other notification mechanisms in future releases
  • Support for refresh tokens: Refresh tokens are not yet supported (stay tuned!)
Type to search, ESC to discard
Type to search, ESC to discard
Type to search, ESC to discard

© 2023 RACKSPACE TECHNOLOGY

On This Page
Release Notes August 2025 July 2025 May 2025 April 2025 March 2025 February 2025 December 2024 October 2024 September 2024 July 2024 June 2024 May 2024 April 2024 March 2024 January 2024