Mixpanel Engineering

Real-time scaling

We’re moving. Goodbye Rackspace.

with 81 comments

If you're interested in what we work on, please apply - we're hiring: http://mixpanel.com/jobs/

At Mixpanel, where our hardware is and the platform we use to help us scale has become increasingly important. Unfortunately (or fortunately) our data processing doesn’t always scale linearly. When we get a brand new customer sometimes we have to scale by a step function; this has been a problem in the past but we’ve gotten better at this.

So what’s the short of it? We’re unhappy with the Rackspace Cloud and love what we’re seeing at Amazon.

Over the history we’ve used quite a few “cloud” offerings. First was Slicehost back when everything was on a single 256MB instance (yeah, that didn’t scale). Second was Linode because it was cheaper (money mattered to me at that point). Lastly, we moved over to the Rackspace Cloud because they cut a deal with YCombinator (one of the many benefits of being part of YC). Even with all the lock in we have with Rackspace (we have 50+ boxes and hiring if you want to help us move them!), it’s really not about the money but about the features and the product offering, here’s why we’re moving:

EBS

IO is a huge scaling problem that we have to think about very carefully. We’ve since deprecated Cassandra from our stack but Rackspace is a terrible provider if you’re using Cassandra in general. Your commit log and data directory should be on two different volumes–Rackspace does not make this easy or affordable. EBS is a godsend.

What happens when you need more disk space? You’re screwed -> resize your box and go down. Need more than 620G of space? You can’t do it.

EBS lets you mount volumes on to any node. This is awesome if you ever need to move your data to a bad node instead of having to scp it over.

Edit: Nobody is saying you get better IO performance on Amazon simply that EBS solves different IO challenges that Rackspace does not. IO is basically terrible everywhere on the cloud.

Instances

We’re super excited about the variety of instances that Amazon offers. The biggest money savers for us we foresee are going to be Amazon’s standard XL as well as the high CPU ones. Rackspace offers a more granular variety which is a benefit if you need to be thrifty but it bottlenecks fast as you begin to scale and realize what kind of hardware you need.

Uptime

Rackspace Cloud has had pretty atrocious uptime over the year there has been two major outages where half the internet broke. Everyone has their problems but the main issue is we see really bad node degradation all the time. We’ve had months where a node in our system went down every single week. Fortunately, we’ve always built in the proper redundancy to handle this. We know this will happen Amazon too from time to time but we feel more confident about Amazon’s ability to manage this since they also rely on AWS.

Control Panel

Rackspace’s control panel is the biggest pain in the ass thing to use. Their interface is clunky, bloated, and slow. In my experience, I can’t count how many times I’ve seen their Java exceptions while frantically trying to provision a new node to help scale Mixpanel.

Amazon has awesome and very well vetted command line tools that blow Rackspace out the water. I can’t wait to write a script and start up a node. I believe Rackspace has an SDK / command line tools now though–very early and beta however.

Quota limits

Probably the most frustrating thing on Rackspace is their insane requirement to post a ticket to get a higher memory quota. We’ve had fires where we needed to add extra capacity only to get an error when creating a new node that we can’t. Once you post a ticket, you have to wait for their people to answer your ticket in a 24-hour period. Now we just ask Rackspace for +100G increments way before we ever need it. I know Amazon doesn’t impose these limits to the same (annoying) extent.

Globalization

Amazon has a CDN and servers distributed globally. This is important to Mixpanel as websites all over the world are sending us data. There’s nothing like this on Rackspace. We have lots of Asian customers and speed matters.

Backups

Rackspace has a limit on their automatic backups: 2G. Our databases aren’t weak 2G memory bound machines–nobody’s is at scale. S3 is a store for everything and EBS is just useful for this kind of thing. Cloudfront on Rackspace is still in its infancy.

Backups on Amazon will be so much cleaner and straight forward.

Pricing

We’ve done a very methodical pricing comparison for our own hardware and have determined that the pricing is actually about the same across both services. We don’t know how well the hardware will scale on Amazon so we over-estimated to make up for crazy issues. Amazon came to about 5-10% cheaper but take that with a grain of salt. It’s probably closer to equal.

Here’s one huge thing though, Amazon in the long-run for our business will be drastically cheaper with the concept of Reserved instances and bidded instances. That’s extremely sexy to us.

Also? Amazon constantly reduces its prices. I’ve never seen Rackspace do that.

What’s the main reason for us?

Amazon just iterates on their product faster than anyone else and has the best one. We expect people to use us because of that in the long-run and we’ve taken note. Rackspace is extremely slow and as the person in charge of infrastructure and scalability we’re going to use the platform that knows how to keep guys like me happy by running fast and anticipating my needs.

Amazon’s products:

Rackspace cloud’s products:

If you have an opinion, express it. Tell us what you hate about Amazon and problems you’ve seen. We haven’t moved yet.

If you're interested in what we work on, please apply - we're hiring: http://mixpanel.com/jobs/

Written by Suhail

November 8th, 2010 at 4:03 pm

Posted in Uncategorized

81 Responses to 'We’re moving. Goodbye Rackspace.'

Subscribe to comments with RSS or TrackBack to 'We’re moving. Goodbye Rackspace.'.

  1. Welcome aboard!

    Jeff Barr

    8 Nov 10 at 6:47 pm

  2. [redacted]

    zane

    8 Nov 10 at 7:40 pm

  3. I see that gogrid is trolling this post, but we had gogrid cloud and managed colo services and they are terrible. here are some reasons:

    -the “hardware” load balancers don’t work half the time when you set it up. just because they configure an f5 load balancer (that you can replicate with haproxy for free and faster) doesn’t mean you actually get a dedicated f5 load balancer for your “cloud”
    -the costs are much higher, they try to make inbound bandwidth seem free, but ratchet up outbound costs
    -you can’t clone or backup servers, you have to setup templates up front and modify them
    -takes a very long time to start up a new server, rackspace and amazon are much quicker
    -they are consistently behind os snapshots. rackspace and amazon are already offering centos 64bit 5.5+, gogrid is still stuck at 5.3
    -rackspace + amazon use limelight cdn and amazon has it’s own great cdn, gogrid has 18 “pops”

    about rackspace:
    -i hate their control panel it constantly crashes
    -i hate the fact that you can’t assign multiple ip’s to your servers
    -i hate that unlike gigenet.net there are no standard mrtg/rrdtool/nagios/cacti style real time performance graphs

    generally i think rackspace just bought slicehost and rebranded it without making any real improvements.

    the company’s support is generally garbage. one of our managed products took them weeks to find a remote KVM to connect to the server and charged us extra every time we connected to it.

    we switched completely to rackspace because when we switched, amazon was still too early and undeveloped, but have improved considerably and we are looking at moving there again. everything about amazon makes more and more sense, particularly how quickly they develop.

    i’ve also been to gogrids datacenter and saw when they were first putting up their “cloud” system. at that point in 2008 it was one rack of chinese servers. they probably grew since then. their datacenter was extremely packed with lots of colo and managed hardware, which was so densely packed and hot that it was unbearable.

    how can you compare a multi-billion dollar company’s offering to gogrid (or even rackspace) which is is owned by a small private company?

    dan

    8 Nov 10 at 7:57 pm

  4. You only mentioned Linode once… Would you consider going back to Linode? If not, why? Is Amazon really that much cheaper? Thanks.

    lucas rockwell

    8 Nov 10 at 8:19 pm

  5. I’m curious, can you tell me whether the Amazon IO limits referred to in this post (from another real-time analytics provider, incidentally) are something you’re concerned with?

    http://bit.ly/96DtVv

    FWIW, I can’t find other mention of such limits.

    I’m not an engineer, but I’m interested in the challenges and opportunities around real-time web data. It seems like a giant, and potentially super-expensive, firehose of IO activity.

    Thanks and best of luck.

    Ryan

    8 Nov 10 at 8:21 pm

  6. Hey Ryan – Amazon may not have IO limits anymore, but when we looked at them several years ago they did. We didn’t do additional research before writing that article so we assumed they still existed, but they may not. And if that is the case, that’s a very good thing!

    And MixPanel, for what it’s worth, the comments in our blog post linked by Ryan are probably worth reading. Good luck!

    Sean

    8 Nov 10 at 9:46 pm

  7. Rackspace cloud servers do go down sometimes and their interface sure is clunky – but if you are not yet at the mixpanel size they are a great option if you just have a big number of 1-2gb machines.

    James Sullivan

    8 Nov 10 at 10:13 pm

  8. Amazon has been almost great for our business. It scales and it’s fun to configure. Although there is one big problem: you cannot add ANAME records to your software if you’re using the elastic load balancer. Only CNAME records can be made. This is one of the most annoying deficiencies of EC2 by far. I hope they fix that soon.

    Julius de Bruijn

    8 Nov 10 at 11:02 pm

  9. suhail,

    were you able to use the Rackspace server image tarchives to re-create your servers at Amazon, or did you just start with fresh Amazon instances, configure them, then load your data ? or something like that ?
    and i would agree about the Rackspace control panel. it seems to crash or oops at the most inconvenient times, which is almost every time i use it these days.
    and the backup image size limit based on the memory footprint of the VM a no go.

    thanks for this post!

    zerowolfgang

    9 Nov 10 at 2:59 am

  10. Hi
    I’ve been on AWS for about two months now and I’m thinking of moving off it. The reason is simply because of the lack of transparency regarding the data transfer numbers (and hence cost) that they report.

    I’ve had a micro EC2 server running for the whole of Oct to see how much it would cost “just doing nothing”. I was testing the calling of a webservice on a regular basis (but that was only a few kb in size) but for some reason it was reported that I’d used up over 300GB. Then in the first 4 days of this month, my account also reported usage over over 250Gb – this simply isn’t the case (and can’t be the case).

    Two emails to the support desk have alas led to being told politely but firmly that they can’t help me. They are also not prepared to suspend the charges to my account even for 1 day while I try to investigate (or get help from anyone else)

    Result is that I have an instance which I now dare not start up because I have no control over the data transfer rates which I consider highly suspect and AWS can’t/won’t help. (It seems there is no way to reconcile the data transfer rates reported with what you are charged)

    So be very careful !!

    Yes I’m looking elsewhere too…

    Alex

    Alex

    9 Nov 10 at 3:20 am

  11. My experience in our testing has been the same. It is a pain to create several hundred instances quickly in rackspace and to terminate them cleanly and quickly (either through Control Panel or through command-line). Amazon is way better on this front.
    Also some obvious things that should be on their Control Panel requires ticket to their support (example, changing data center for instance creation). Cloning a saved instance is also expensive on rackspace (data transfer is being charged for copy of instances. It is all in their DCs, does not make sense). They send way more maintenance emails than AWS.
    I agree IO sucks everywhere in cloud (be it Amazon or something else). Once you mature, you should move critical stuff (databases) to metal.

    AJ

    9 Nov 10 at 5:43 am

  12. Interesting thoughts, although some of the issues likely could have been dealt with by going to a Hybrid solution which Amazon simply can’t provide at this point. That was the route that GitHub took when they moved off EngineYard, which was running on Amazon’s platform. Move your IO intensive workloads to physical hardware, cut out the virtualization overhead. By moving to Amazon you may only be sidestepping the issue for now. For large scale application deployments a all-cloud-all-the-time deployment is likely to be naive. Sometimes you really do need physical hardware.

    James Thompson

    9 Nov 10 at 7:19 am

  13. We’ve been very happy with AWS for the last 2.5 years. Our scaling needs have been light, but I feel confident that their ecosystem can get us anywhere we need to go. We’ve gone from 1 small instance to 2 high-cpu mediums loadbalanced in two availability zones with an RDS instance using multi-az. We also use simpledb for configuration management and use S3 for all sorts of stuff.

    Things are not perfect, especially with RDS right now (in beta), and I/O does suck, but the cost that really matters to us is development time, and we’ve saved a lot there. All of the problems we’ve had have been solvable, and that’s important. With EBS for example, if I/O is very important, you can always run RAID across multiple volumes (at the cost of complexity).

    Wes Winham

    9 Nov 10 at 7:27 am

  14. [...] “The folks at Mixpanel are leaving Rackspace’s server cloud for Amazon and have left a little note about their reasons. There’s been some talk that Rackspace’s offering has not been up [...]

  15. Interesting comparison. Just out of interest. What database did you go for, instead of Cassandra? A custom-build solution, another NoSQL, or back to old-fashioned relational DBs?

    Wladimir

    9 Nov 10 at 7:29 am

  16. Welcome Aboard! We love AWS at Yipit. I get frustrated by one thing or another every few weeks. Support and documentation are a bit weak (unless you pay extra), but the actual stack is amazing. We use S3 heavily and have become huge fans of RDS over the past few months.

    On RDS, we haven’t had IO problems to the same degree that we did with MySQL installed on ec2 instances. The lesson there is that because backups are handled by Amazon’s software, it can happen at the block level and doesn’t consume IO in the same way as something like mysqldump would.

    Cost is extremely competitive with anything else out there.

    @alex, I understand your issue but I think you’ll notice that even with obscene transfer quantities, you’ll have trouble going over $20/month. The biggest cost is ec2 hours in pretty much every scenario.

    Adam Nelson

    9 Nov 10 at 7:30 am

  17. I’m planning on writing a little web service in the coming months, so this is a really interesting conversation. Does nobody serious use Google App Engine? I see a lot of the comments are comparing Amazon and RackSpace.

    How “big” did Mixpanel get before outgrowing RackSpace? Would you still have started out with them, knowing what you know now?

    Rob

    9 Nov 10 at 7:44 am

  18. Have you checked out SpiderOak’s DIY-API? We offer a ‘Amazon S3 for backup/storage’ with similar structure and service offering at a much lower cost.

    Take a peek if you feel like it: https://spideroak.com/diy/

  19. I’m also curious, what caused you to move away from Linode? Was it the lack of a cloud-storage offering? Linode does kind of have a reputation for having very fast but very small storage allocations, and while you can pretty easily transfer disk images around between instances, they’re still ultimately local.

    Adam

    9 Nov 10 at 8:27 am

  20. While I don’t feel like starting a debate, this comment by Zane makes me laugh:

    > Amazon main business is selling books… their cloud services are a by product of extra hardware.

    So, we have written thousands of pages of documentation, spun up a world-wide sales, support, and marketing effort, and are currently hiring hundreds of people (see http://aws.amazon.com/jobs) all because we had some extra hardware laying around? I don’t think so.

    Jeff Barr

    9 Nov 10 at 8:41 am

  21. [...] Interesting Rackspace cloud review Posted on November 9, 2010 by admins Ref: http://code.mixpanel.com/amazon-vs-rackspace/ [...]

  22. And why do we care? Rackspace is going after a different market than Amazon is. If anything its your fault for picking a cloud service that never suited your needs to begin with.

    If you designed your app correctly the “move” should happen without your users noticing and without the need to write some dramatic article about it to get attention for yourself.

    John Mith

    9 Nov 10 at 9:21 am

  23. I’m curious, are you running your entire operation on Amazon, or are you supplementing your own hardware?

    I’ve looked at many scenarios for using AWS, and for our operation Amazon beats buying and installing our own hardware in only two areas – ease of deployment, and as a cheaper way to run machines that are not always on (e.g., our cluster of web servers see very little traffic outside of North American business hours).

    You may be able to peer with Amazon for very little cost. In Seattle we have a not-for-profit peering group that you can connect to for a one-time fee of less than $500, and that gets you a gigabit connection to Amazon’s US West availability zone. No Amazon charges for I/O over that connection, either!

    Brandon Adams

    9 Nov 10 at 9:30 am

  24. [...] We’re moving. Goodbye Rackspace. at Mixpanel Engineering They say Amazon, 50ish machine user. [...]

  25. Can you get CentOS at amazon??? We skipeed Amazon because they did not mention CentOS on the list of supported linux distributions.

    Martin

    9 Nov 10 at 9:41 am

  26. @James Thompson: Github was never using AWS through EY. Github was on EY’s own infrastructure. EY’s new AppCloud is what is based on AWS.

    I’ve been beginning to believe that scaling anything on EC2 is going to be a pain. We have servers that see frequent spikes in IO wait to 50% or more.

    On top of that, our account manager basically has nothing to offer. Disk latency problems on a MySQL box? Use RDS! Even though it is the same EC2 + RAID EBS we’re already using… somehow it avoids the latency spikes on a shared network.

    Network latency issues? Sorry, it is a shared resource. Get bigger instances for better priority and more instances in different zones!

    And their premium support… add 20% on your bill to get confirmation of an issue in 1hr. Nothing for quicker resolution because its all one big shared resource.

    Ken Robertson

    9 Nov 10 at 10:04 am

  27. Amazon has a reduced price for S3 only if you use Reduced Redundancy Storage, which basically means you pay less because you less frequently duplicate the files. You are not getting the same level of safety as Standard Storage for a lesser price.

    pseudonymous coward

    9 Nov 10 at 10:16 am

  28. Interesting conversation and it really seems to highlight the overall lack of maturity of that vague bit of marketingspeak referred to as “the cloud.” Anyone who knows infrastructure knows that ultimately “the cloud” still requires properly tuned and scaled bare metal hardware sitting in a data center and the expertise to build and scale the network, storage, memory, and Internet bandwidth. Even though Rackspace has certainly been around for a while traditional hosting companies – who arguably pioneered dirt-cheap “cloud” servers (i.e., VPSes) – are still feeling around in the dark about how to properly scale the kind of infrastructure required to have decent IO and CPU performance on a “cloud” instance, be it AWS, Rackspace, GoGrid, Linode, Gigenet, etc. etc. etc. I can tell you from a limited perspective (e.g., about 40 metal hypervisors globally) properly scaling CPU, storage, memory, and bandwidth for thousands of VMs (ok, “instances”) it is indeed non-trivial to properly build out and scale virtual / cloud infrastructure.

    boethius

    9 Nov 10 at 10:20 am

  29. I’ve done a bunch of work recently with AWS, after having been on GoGrid. I still use slicehost for some small stuff for personal use too.

    Overall, I frigging love Amazon. They just finished setting it up so you can put your SSL certs on the load balancers so you don’t have to do funky port forwarding or anything like that for multiple sites, etc. They lower prices regularly. They upgrade features very regularly.

    A couple concerns to keep in mind during migration:

    1. EC2 is nice, S3 storage sucks. I wouldn’t rely on S3 for much of anything because it’s so damn slow.

    2. You CANNOT have multiple IPs per instance.

    3. If you use the load balancers, it is CNAME only for domain mapping to them since the load balancers don’t have a dedicated / guaranteed IP.

    Good luck tho, beyond that, I love the service!

    Justin DeMaris

    9 Nov 10 at 11:24 am

  30. I have one comment for this post. Customer Service!!

    If I have a problem, I like to call and talk to a real person. To Amazon, I’m just a number and especially a number they don’t want to talk to.

    Rackspace usually picks up immediately and often times I’m actually talking to a tech guy. That’s invaluable and even if they are a little behind, they’ll catch up and then they will be that much better. And no, I have no affiliation with Rackspace.

    Jonathan Pfaff

    9 Nov 10 at 12:00 pm

  31. Full disclosure: I’m a former Rackspace employee on the Cloud Servers product, and a current customer of Rackspace Cloud Servers.

    It sounds like that the products weren’t explained very well to you, as Rackspace Cloud has a lot of the offerings you’re looking for. The CDN functionality can be found on the Cloud Files product, and for a large site like this, I’m very surprised that it wasn’t mentioned to you, as it’s awesome for serving your static content worldwide through the Limelight CDN.

    The API for Cloud Servers and Files is actually really robust, and it’s far easier and more efficient to manage instances and files with one of the third party API tools like Cloudkick or a Python script than use the Control Panel. It’s very possible to have your usage monitoring suite/scripts notice that your nodes are rising in traffic, and have them fire off a python script which builds another node from your base build image backup, grabs the IP, plugs it into your HA setup, and starts taking load right away.

    You’ll also find Amazon’s support offering lacking without paying out of the ears, and as has been stated earlier, Amazon’s extra storage is nice, but if you think I/O is an issue now, wait until you rely on that for your I/O needs.

    The inability to add storage outside of an increase in overall server resources is a pain point, and one I hope they implement a solution for. But really, if you’re doing really heavy I/O work for a product like analytics, you’re probably better off cutting out the virtualization layer completely since you’re always going to be competing for I/O, and think about getting some dedicated hardware for your compute power and using fiber-channel SAN for storage.

    Also, whether you’re on Amazon, Rackspace Cloud, or any other provider, databases don’t scale horizontally well unless you’re designing a distributed architecture from the start.

    If you’re really at the point where money isn’t a huge issue and performance is what matters, it’s time to go dedicated with some big iron. If you’re not at the point to where you can drop that kind of cash, you need to find a way to code around the strengths and limitations of cloud hosting so that you can extract value out of it, no matter which provider you end up choosing to go with.

    Jeff Pierce

    9 Nov 10 at 12:28 pm

  32. [...] מחשוב ענן בחו"ל ולאחרים שגם מתעניינים במחשוב ענן לקרוא את הפוסט [...]

  33. I’m curious, did you ever try Joyent?

    We host our larger apps at Joyent and smaller at Rackspace.

    We briefly considered ditching Joyent in favor of Rackspace. But during load testing of the Rackspace instance, we found that a 4096 Rackspace instance didn’t even achieve the same performance as a 1024 Joyent instance. So we stayed with Joyent.

    Mike

    9 Nov 10 at 12:49 pm

  34. [...] “The folks at Mixpanel are leaving Rackspace’s server cloud for Amazon and have left a little note about their reasons. There’s been some talk that Rackspace’s offering has not been up to snuff once you [...]

  35. @Dan: On RSCloud, you can have additional public IPs. We do this with a load balancer today.

    Jeremy

    9 Nov 10 at 1:35 pm

  36. [...] We’re Moving. Goodbye Rackspace. (From the Mixpanel Engineering blog) This is a pretty damning account of one company’s decision to leave its Rackspace cloud for AWS. It’s not the first such tale of Rackspace woe, which underscores the importance of its OpenStack effort to create a better cloud platform. [...]

  37. We’ve been very happy with Amazon; we do tests with others every few months and Amazon keeps winning. To be honest; Rackspace comes out as one of the worst. They are just not ready for this. And tests prove it.

    frank

    9 Nov 10 at 2:36 pm

  38. I’ve been using Hexagrid’s cloud. The support has been great. I’ve tested it for HPC capabilities and it rocks. There’s been some minor glitches, but overall I’m very impressed.

    Steve

    9 Nov 10 at 3:09 pm

  39. [...] We’re moving. Goodbye Rackspace. at Mixpanel Engineering (tags: amazon cloudcomputing engineering rackspace startup host mixpanel sysadmin hosting college amazon.com aws cdn cloud comparison infrastructure webhosting linode ec2 computing) [...]

  40. @Adam Nelson : My AWS bill for the first *4 days* this month shows I’m going to get charged $42 and I’ve used 340Gb !! That’s when I shut it down. (remember this is a micro instance doing next to nothing)

    @Jonathan Pfaff : Spot on, I’m finding the hard way “customer service” and “AWS” don’t really go together. Its really amazing that they don’t want to help a new customer fix any initial problems or at least add clarity to how they calculate what they charge.

    Alex

    9 Nov 10 at 10:24 pm

  41. Amazon pricing is middle of the road to high depending on the instance. Rackspace is generally more expensive not the same.

    See http://cloudpricecalculator.com for the comparisons.

    The is a larger issue in the lack of a compute metric to compare different cloud offers. Amazon has not reduced the price of its standard small instance in four years. They benefit from Moore’s Law, but they are not inclined to pass it along.

    Daniel Berninger

    10 Nov 10 at 1:28 am

  42. If you need IO speed then you can stripe (RAID0) across a number of EBS devices at the risk of decreasing your data reliability. I bet you can’t do that with Rackspace! Also, EBS snapshots rock – consider combining them with XFS for consistent snapshotting https://launchpad.net/ec2-consistent-snapshot

    The only problems that I’ve had with EC2 (other than occasional short downtime, which really happens very rarely, and is well commented in their status feeds which they don’t delete after incidents unlike some other providers) have been due to poor OS images. One, for instance, became inaccessible after 100+ days uptime because it refused to accept an IP address from EC2′s DHCP server. Rebooting it was fast and fixed it, but it happened again a few months later. Fortunately the OS in question now has official builds for EC2 and they have worked brilliantly so far!

    Good luck with the move!

    Benjie Gillam

    10 Nov 10 at 3:06 am

  43. [...] Why Mixpanel migrated from Rackspace to Amazon (11/2010) – EBS more powerful and flexible than disks, variety of instance types, likely less outages, (much) better control tools, easier backups thanks to no 2G limit, pricing (reserved and bidded instances, A. constuntly  reduces its prices); main reason: “Amazon just iterates on their product faster than anyone else and has the best one.”. [...]

  44. [...] We’re moving. Goodbye Rackspace. Line Break [...]

  45. interesting to know about it

    neo

    11 Nov 10 at 11:46 pm

  46. Although I certainly wouldn’t want to hijack this thread, but having posted a previous comment above saying that I was having trouble with AWS data transfer rates/prices; AWS have since reached out to me and help me identify a configuration problem which was leading to huge transfer volumes. So I think it’s only fair to ‘update’ my comments above by saying that they have helped me to resolve the problems I had.

    Alex

    12 Nov 10 at 11:12 am

  47. [...] We’re moving. Goodbye Rackspace. – Why MixPanel are moving from Rackspace Cloud to Amazon EC2, and a good Hacker News discussion thread. [...]

  48. Honestly, I’ve worked with box Rackspace and EC2 extensively.

    Amazon nodes suck balls for performance. You get really craptastic CPU allocation that is prone to change on you quite drastically which has caused our property to become unresponsive because we went from and effect 4 CPUs to 3.2 effective CPUs week over week. 4 CPUs on a doublewide instance, that translates to about $720/month for a 4 CPU box before you factor in storage. I have that much compute power on my desk for goodness sake with over 3TB of storage, and apparently IO that’s not too far from what I’m getting on an EBS.

    For that money, you could have a dedicated server with Rackspace, not even a cloud instance, and get better performance and more bang for your buck by a lot. I think we were paying around $800/month about three years ago for a dedicated instance that had 4 drive RAID 10 and 4 CPUs. Every year we would get a new server and bump up the config for less money.

    Just looking at server beach, I can get 4x146GB SAS in RAID 10, 24GB RAM, 8 Cores at 2.26Ghz for $938/mo.

    This will destroy a double wide instance at amazon. You could afford to allocate 50% more machines for the same money to allow for scaling and still come in about the same price even with reserved instances. Sure, you can’t spin something up in 5 minutes, but if you have 50% headroom, who cares?

    If you go with someone like rackspace for dedicated instances, you can get a private network space and just mount network drives with big honking 1TB drives for storage volumes if you need them. You’ll still get better IO over a Gb network that you would for an EBS. Heck, if you can find a provider willing to give you a large blade chassis, you can get all of the above over internal channels that make Gb ethernet look like a Yugo being passed by a Ferrari.

    I for one am very very not sold on EC2 for large deployments.

    I use Rackspace because I need lots of small nodes doing very simple jobs, but I need a lot of them. Amazon doesn’t offer small nodes that are as cost effective. I like EBS, but like you said, the IO is horrible. Want to take about some other services like S3 or RDS? S3 is the most awful thing ever. It’s slow, clunky and a pain in the arse to access. RDS machines have demonstrated fractional performance over manually configured server instances of the same size.

    Alex

    14 Nov 10 at 6:38 pm

  49. Wow, reading this article and then I see my own face randomly on the Rackspace screen grab above. Threw me off there for a second :)

    Chad Keck

    17 Nov 10 at 11:54 am

  50. Alex, you are a freaking idiot. Of course the problem was you, not amazon.

    Jd

    20 Nov 10 at 12:16 am

  51. [...] will flourish, while software built for boxes will fall along the wayside. Rackspace gets it. Dinged for scalability issues, it launched OpenStack to remedy its software situation. Are you ready for this brave new [...]

  52. [...] Rackspace Cloud to AWS – The story of Mixpanel – http://code.mixpanel.com/amazon-vs-rackspace/ Uncategorizedtwiiter ← Google searches – world goes w… Get into the Game! [...]

  53. File backups shouldn’t be a task. A backup plan will run forever

    Dawne Boomer

    23 Nov 10 at 3:10 am

  54. We’re doing a cloud provider review right now, and have been looking hard at Amazon, Rackspace and Joyent. Joyent seems to be trailing badly adoption-wise (http://www.jackofallclouds.com/2010/11/state-of-the-cloud-november-2010/), but I’m not sure if this is due to marketing, pricing or performance reasons. Does anyone have any experience w/ Joyent, esp whether their claim to a much more efficient/resilient platform? Efficiency = $ savings as you scale in my book.

    Also, are any of you running NoSQL solutions on these providers? If so, which DB/cloud combo, and what has been your experience?

    Eric

    3 Dec 10 at 9:46 am

  55. I’ve been using both for the Amazon and Rackspace for the past couple of months. Personally, I like Rackspace more, but really it’s only because of personal preferences. I’ve had a lot of problems with Amazon having networking issues and our server becoming unreachable. This has happened multiple times. However, I would say that Rackspace has been better about this, but as I write this, my Rackspace server is experience problems out of my control.

    Brad Proctor

    11 Dec 10 at 3:04 pm

  56. [...] We’re Moving. Goodbye Rackspace. (From the Mixpanel Engineering blog) This is a pretty damning account of one company’s decision to leave its Rackspace cloud for AWS. It’s not the first such tale of Rackspace woe, which underscores the importance of its OpenStack effort to create a better cloud platform. [...]

  57. I think its a good move. Amazon just gives you so many more options, there isn’t really much you can’t do. Not to mention they have multiple data centers you can choose from and seem to be constantly pushing out new features (VMware import tool and the S3 object size increase to name a couple recent ones).

    Adam

    2 Jan 11 at 2:05 am

  58. When we sold our Data Centre business, we were going to go into IaaS, building Cloud hardware and putting a utility billing engine on the top of it … but looking at Amazon, and seeing how users gravitate towards them, we made the decision to build a ‘Cloud Management Platform’ instead. Wow are we glad we did that.

    Amazon innovate at an incredible pace, and way out distance any rivals. Reading this post just shows that anyone building an app really should be looking at AWS over any of the VPS or low-tier Cloud providers.

    The thing from our experience is that most business users and IT managers know what Cloud Computing is now, and are aware of Amazon, but don’t quite know how to take advantage of it – because it’s not easy. That’s where Rackspace win – their CP is really simple to use – the barrier to entry is low. That’s the problem we’re trying to solve – building a CP on Amazon that makes it really easy to use so businesses can get on-board.

    Ed Byrne

    12 Jan 11 at 7:18 am

  59. There are many newer ‘next generation’ IaaS cloud providers coming online (http://dediserve.com who I work with being just one) who are focussed on high IO, low latency, ‘enterprise’ kit and credentials, with cloud benefits.

    It is these providers who will replace the ‘first gen’ hosting companies and vps providers out there…

    Stephen McCarron

    12 Jan 11 at 7:50 am

  60. What is the captcha code? Please provide me captcha code codes or plugin. Thanks in advance.

    Edmund Speagle

    1 Mar 11 at 2:45 am

  61. Thanks very much for your great text. I have been looking for such information for a really long time. Not everything is completely clear to me, but it is definitely interesting and worth reading.

    okna wrocław

    3 Mar 11 at 8:37 am

  62. We pretty much appreciate your site post. There are a multitude of approaches we could put it to decent use by means of minimum effort in time and hard earned cash. Thank you really regarding helping have the post answer many issues we have experienced before now.

  63. Where can one get a cheap and secure Virtual private server?…

    If you do not need a lot of RAM, then just launch of micro instance on EC2, its like .085 cents a day with block storage attached to it[1]. If you need additional RAM, then you might want to look into using Slicehost or Linode, but I can’t seem to fig…

    Quora

    11 Mar 11 at 12:12 pm

  64. [...] tu ce ai face pentru cinci euro? [...]

    cinci euro

    15 Mar 11 at 7:43 pm

  65. [...] is Mixpanel moving to?In November, Mixpanel wrote a blog post about moving from Rackspace to EC2:http://code.mixpanel.com/amazon-…Now, they're showing messages like the following:http://f.cl.ly/items/1c3m1J1f1H0…If not [...]

  66. For a lot of ventures it will be interesting to see how DotCloud develops – they want to be your sysadmin running on AWS/EC2. They raised $10m recently.
    http://www.dotcloud.com/

    johans

    25 Mar 11 at 2:29 pm

  67. [...] http://code.mixpanel.com/amazon-vs-rackspace/ Share and Enjoy: [...]

  68. We went to Amazon as well until we’ve had constant issues with their EBS and have lost several hundred million unique hits a day because of an EBS issue which spanned 4 days and made our volumes get stuck in detaching.

    That’s when we went back to Rackspace and whenever we have an issue we can get someone live on the telephone to help. Amazon? Only forum support.

    Jon

    21 Apr 11 at 11:04 pm

  69. I am very happy to read thisThis is the type of manual that needs to be given and not the accidental misinformation that’s at the other blogsAppreciate your sharing this greatest doc.

  70. Has mixpanel been affected by the recent EC2/EBS outage?

    eik3

    28 Apr 11 at 9:51 am

  71. I’d have to test with you here. Which isn’t one thing I usually do! I enjoy studying a submit that can make people think. Additionally, thanks for permitting me to comment!

  72. I am writing a research paper and html and pdf documents have different citations. I can’t remember which I used when I printed my resources. It gave me both options..

    Wade Rambeau

    23 Jun 11 at 7:04 am

  73. We have aggressively looked at all of them but looked at benchmarks and found Rackspace is at the bottom of the list with GoGrid and such. We’re currently in talks with Savvis but haven’t pulled the trigger yet.

    Steven

    23 Jun 11 at 11:32 am

  74. It’s hard to find knowledgeable people on this matter, however you sound like you understand what you’re speaking about! Thanks

  75. [...] This post is a follow up to We’re moving. Goodbye Rackspace. [...]

  76. [...] 去年差不多的時間,Mixpanel 從 Rackspace Cloud 搬到 AWS:「We’re moving. Goodbye Rackspace.」。 [...]

  77. Interesting reading all the comments – although
    I’m still a tad confused.

    Amazon more features – barely any support.
    Rackspace less features – better support.

    Phil

    19 Feb 12 at 8:45 pm

  78. good choice you move, Rackspace FUCKING SUCKS

    oey asdf

    14 Apr 12 at 6:42 pm

  79. [...] [2] http://code.mixpanel.com/2010/11/08/amazon-vs-rackspace/ Share this:TwitterFacebookLinkedInLike this:LikeBe the first to like this post. [...]

  80. [...] source: Amazon vs Rackspace [...]

  81. [...] MixPanel Chooses AWS – a real life example from 2010 documenting why MixPanel migrated to Amazon AWS. [...]

Leave a Reply

Safe place to purchase clomid and buy weight loss pills