Variable Cloud Billing is Killing Your CFO

Ok, we admit, that headline might be a little dramatic. However, CFOs, department leaders, data scientists and deep learning researchers are all experiencing the shock and horror of getting their monthly bill from AWS, GCP or Azure. It brings back those distant memories of that time you got your cell phone bill when data plans first became the norm with smartphones. Pure shock! 

Recently, The Information posted an article on how companies are continuing to be surprised by their cloud bills. As you can see from their graph below, AWS bills have been climbing significantly. Many companies are affected but the Pinterest story caught our eye because of the really nasty $20M surprise executives got during last year’s holiday season. Consumers were spending so much time using the online scrapbook site that Pinterest’s computing bills on AWS shot past their expectations.

The Information's article stated, “Pinterest, which had paid in advance for AWS’s services, had to buy additional capacity at a higher price. That contributed to Pinterest spending roughly $190 million on AWS last year, $20 million more than it had initially expected, according to a person close to the company and figures reportedly seen by The Information.”

As you can also see from the graph, Pinterest wasn’t alone in receiving a larger than expected AWS bill. So how do you prevent getting a similar, even if far smaller nasty surprise?

Flat-Rate Billing vs. Hourly Billing for AI Development
For those companies that do use AWS, Azure, Google Cloud, and other cloud services suppliers that have variable billing cycles, they face the heartache of not knowing what they are going to pay until they get their monthly bill; hence the potential for nasty surprises. Eliminating these billing surprises is a three-step process:

1. Choose an AI / deep learning multi-GPU cloud services provider that offers flat-rate billing rather than hourly billing. Only then can you accurately budget for your cloud service costs.
   
   
2. In your due diligence, ensure companies that claim to be flat-rate truly are. Ask potential cloud service providers if there are any other charges that could be levied on you. If they mention something like ‘just data transfer egress charges’, ‘training’ or ‘setup’ or ‘holding’ costs, or even 'support' costs, choose a supplier that doesn’t charge extra for any of those services. Choose a supplier that doesn’t add data transfer charges either for ingress or egress of data. For example, one healthcare provider did data processing in a public cloud but synced the data back to customer records kept in their private cloud – every transfer back (egress) of that data attracted another data transfer cost. The result was another nasty surprise.
   
   
3. Make sure that other options that can be provided by your cloud services provider, such as high-speed networking options (e.g., InfiniBand, 100GbE) or high-speed storage options, such as WEKA.io, are also done so at a flat-rate. You want to make sure it is one price you're paying and will receive no supplemental bills.

When Does Hourly Billing Make Sense for Cloud Customers?
Hourly billing works well for those early-stage developers or students who only need to work on small models that take a mere hour or two to train. Although most scientists and developers can use their desktop systems for these items, sometimes spinning up a quick V100 instance on a provider for an hour or two makes sense. Just be sure to create it, do what you need to and then delete it so you only pay for what you used.

At Cirrascale, we have identified the threshold of when customers should begin thinking about utilizing our flat-rate, multi-GPU cloud services as one of the following:

1. You develop or train your models in the cloud for more than 12-15 hours a day on a fairly regular cycle.
2. You want to have a bare-metal instance running at all times and not have to worry about environment changes or container reboots if your training jobs takes longer than a week.
3. Your CFO has had multiple conversations with you about your growing and wildly variable cloud services bill.
4. Your ingress/egress charges are more than you expected and need to control them.
5. You want your servers managed by someone other than the people you're paying to do data science work.

Please contact us if you’d like to discuss how to achieve your AI development cloud services with flat-rate billing and absolutely NO SURPRISES.