The Briefing Room
General Category => Science, Technology and Knowledge => Computers => Topic started by: kevindavis007 on September 07, 2016, 01:04:25 am
-
Which cloud services do you prefer? Amazon Web Services, Azure, or Google Cloud? Personally I prefer Amazon Web Services..
-
AWS is king, hands down. Better class of offerings, although I might give the API Gateway to Azure.
AWS Lambda is very cool.
-
Which cloud services do you prefer? Amazon Web Services, Azure, or Google Cloud? Personally I prefer Amazon Web Services..
My own. All it takes is a Linux box sitting in the DMZ (or port-forwarded if you like) running the services you need... and a domain service to get around dynamic DNS (which most routers will take care of anymore, once you buy the service).
-
My own. All it takes is a Linux box sitting in the DMZ (or port-forwarded if you like) running the services you need... and a domain service to get around dynamic DNS (which most routers will take care of anymore, once you buy the service).
Can "your own" autoscale when you have a major web site running a promotion that causes traffic to jump by 200%? And then scale back automatically when traffic dies down?
No offense, but your answer is patently ridiculous. Cloud services exist for a reason. If you are trying to run "your own," that just means you are not running a major web-enabled enterprise at scale.
Moreover, cloud services are entirely disposable. Do you have a dozen hardcore, multi-tier new product development projects going at the same time, each requiring separate QA, UAT, and/or staging environments? Try managing that on physical hardware. I've been in an enterprise that DID try to manage it that way. You would wait *months* to get new environments spun up. On AWS, you can have the environments in days or even hours.
Finally, if you've ever deployed something to production that performs... differently... than in QA/load testing, but you can't roll back for business reasons, cloud computing gives you a massive advantage of throwing a sh!t ton of servers at the problem until you can track down the performance issue. The cost in these instances is often a pittance compared to the loss in business revenue associated with a rollback.
Sure, AWS is probably NOT the right solution if you are hosting some dentist's website, but if you are trying to run an enterprise at scale, and have not embraced cloud computing, get with the times. It's a tremendous leap forward.
-
No offense, but your answer is patently ridiculous. Cloud services exist for a reason. If you are trying to run "your own," that just means you are not running a major web-enabled enterprise at scale.
No one said anything about enterprise, or I would not have said a thing.
-
No one said anything about enterprise, or I would not have said a thing.
Well, in that case, no one said anything about physical servers on your own network, either. The question was about CLOUD services.
-
When speaking of cloud, the vendor list varies by what service you are looking to purchase. If you are looking for Infrastructure, Azure is top dog. Of course, there are others in this market space as well. In the services space, it gets a tad more confusing.
If you are looking at storage, there are lots of options. Here is a review that might be helpful - http://www.toptenreviews.com/services/web-hosting/best-cloud-services/
If you are looking for a development environment in the cloud, I would say that there is no real market leader yet, though IBM"s offering "Bluemix" is cloud development on top of a cloud infrastructure. In some ways, it is THE environment to go to for dev-ops type of environment. Other services such as DR or BCP in cloud are even more of a startup environment.
I guess it all comes down to what you are looking for from your provider and evaluate against your needs. It is still a very highly customized environment - at least from the offering perspective.
-
@roamer_1
@bolobaby
Let's not fight too much about clouds and roll-your-own solutions.
Just for two cents' worth, my take on the comment was that for one's own needs one can often set up a system you have on your home network and then access it from elsewhere as if it as your own personal cloud.
Now, clearly, that isn't comparable to the commercial "clouds" that are available, but if you just need a little for yourself, and you trust your home ISP to not throttle in-coming traffic, then a roll-your-own probably works.
-
@roamer_1
@bolobaby
Let's not fight too much about clouds and roll-your-own solutions.
Just for two cents' worth, my take on the comment was that for one's own needs one can often set up a system you have on your home network and then access it from elsewhere as if it as your own personal cloud.
Now, clearly, that isn't comparable to the commercial "clouds" that are available, but if you just need a little for yourself, and you trust your home ISP to not throttle in-coming traffic, then a roll-your-own probably works.
Agreed..
-
@roamer_1
@bolobaby
Let's not fight too much about clouds and roll-your-own solutions.
Just for two cents' worth, my take on the comment was that for one's own needs one can often set up a system you have on your home network and then access it from elsewhere as if it as your own personal cloud.
Now, clearly, that isn't comparable to the commercial "clouds" that are available, but if you just need a little for yourself, and you trust your home ISP to not throttle in-coming traffic, then a roll-your-own probably works.
The majority of the public aren't using cloud services for commercial enterprise solutions. I know I just need the ability to back-up household computers to make sure I don't lose family photos, Quicken data files, favorite links, etc. Designating two networked computers as back-up servers works fine for me.
Or maybe I'm the rare exception rather than the majority. Hell, I don't know....
-
Never put anything "in the Cloud" that you don't want the world to see. Once it gets on the Cloud, you have lost control of it. It belongs to the box where it resides. Granted, you may not care that people see your ZIP, GPG or PGP encrypted file because they can't see what is in it. But remember, Truecrypt used to be safe.
I put things on the Cloud that I want to retrieve from another computer. If someone wants to see my Emmaus talk or my Sunday School lesson, have at it.
-
Never put anything "in the Cloud" that you don't want the world to see. Once it gets on the Cloud, you have lost control of it. It belongs to the box where it resides. Granted, you may not care that people see your ZIP, GPG or PGP encrypted file because they can't see what is in it. But remember, Truecrypt used to be safe.
I put things on the Cloud that I want to retrieve from another computer. If someone wants to see my Emmaus talk or my Sunday School lesson, have at it.
So, I think you are mistaking the term "cloud services" in this context. We aren't talking dropbox here. We're talking virtual servers and services inside a VPN hosted by a major provider.
To be clear, the days of [one server]=[one physical box] are essentially over. Almost all companies now - even those not on the cloud - are managing virtual servers through some sort of hypervisor. As an IT professional who has been in the industry a long, long time, it's been fun to watch the progression from mainframe to client-server to distributed web apps to cloud services, with a few minor stops along the way.
Computing power is becoming increasingly commoditized. In may ways, this is actually a good thing. Amazon Lambda, for example, takes it even a step further, creating "server-less" computing solutions.
Cloud services are essential for any company that needs to control costs and scale in a flexible manner. Cloud services can be much more fault tolerant with redundant zones, which takes hardware and major network failures out of the question. Cloud services allow companies to offer SaaS solutions to customers with ease, and without lengthy ramp-up times.
Simply bear in mind that "cloud" in this context is not storage per se (although you could use storage solutions, especially around big data), but it's about computing power - exposing things like APIs to third parties, or providing a messaging backbone for your mobile app, or crunching massive data as part of a machine learning system on a periodic basis.
-
@Restored
@bolobaby
Seems pretty clear to me that y'all are talking about two different things.
-
The Cloud refers to storage outside your machine, not web services. I work for a major tech university. We don't allow things to be stored outside the wire. That includes "the Cloud". The Cloud doesn't exist inside the wire. That's a NAS.
-
The Cloud refers to storage outside your machine, not web services. I work for a major tech university. We don't allow things to be stored outside the wire. That includes "the Cloud". The Cloud doesn't exist inside the wire. That's a NAS.
Incorrect. I'm an IT executive and I can tell you with 100% certainty that when someone asks your opinion of cloud services, as above, and even provides the additional context of AWS vs Azure, for example, they are not limiting themselves to storage, but virtual servers and services. If, for example, you do not know what an API gateway is, or how to use in-memory caching solutions to improve application performance, or how SNS can be used to manage and broadcast events, or even how you can use cloud-based HSMs to enforce security, you should take the time to educate yourself on these very important topics.
Start digging in here: https://aws.amazon.com/
Storage is just one part of cloud services. All tiers of a multi-tier application can exist in the cloud. And, once again, I'll bring up AWS Lambda just to underscore the point. Lambda allows you to fire off compute activity based on specific events. So, you could set up a text-based ordering system that monitors an SNS queue. When a new text order comes in, Lambda could fire off a series of cloud based web services that checks inventory, places the order, and puts an outbound "order complete" on another queue. That outbound message could be picked up be an EC2 instance for handling, which - if you understand cloud servers - may or may not have been instantiated 10 minutes ago, and may be terminated 10 minutes from now, never to be seen again.
If your major tech university is not teaching these concepts, they are failing their students. No offense, but they are.
-
AWS is good, but it's also a ripoff. Check out vps.net, where you can get a VPS for $5 a month in some of the cheaper suites.
-
AWS is good, but it's also a ripoff. Check out vps.net, where you can get a VPS for $5 a month in some of the cheaper suites.
That's definitely a better price, but only for the VPS. AWS has a variety of services that are available OOTB, like RedShift for high volume data capture and parsing, which we used to store Segment.io web analytic events. You can't get that with just a VPS.
-
That's definitely a better price, but only for the VPS. AWS has a variety of services that are available OOTB, like RedShift for high volume data capture and parsing, which we used to store Segment.io web analytic events. You can't get that with just a VPS.
Right but then you're tied to some proprietary BS. I'm sure it works for you and that's fine.
-
Right but then you're tied to some proprietary BS. I'm sure it works for you and that's fine.
Out of curiosity, what's the largest number of events you've had to handle at any given time, either service calls, analytic events, or page requests?
-
Out of curiosity, what's the largest number of events you've had to handle at any given time, either service calls, analytic events, or page requests?
I do system administration for a company that performs research contracts for pharmacy companies. By "events" I'm not sure what you mean? I administer about 50 servers who each are constantly logging "events'.
-
I do system administration for a company that performs research contracts for pharmacy companies. By "events" I'm not sure what you mean? I administer about 50 servers who each are constantly logging "events'.
So, for example, let's say you worked for Amazon, the retail site side of the house.
1. Amazon obviously gets millions of people looking at their site each day, shopping for various items. Each rendering of a product page, search results, checkout page, writing reviews, help pages, etc would be a page request event.
2. Amazon is a data-driven company, so they log lots and lots of information about your visit. If you click an image to see the expanded view, they want to know about it. If you use the "calculate shipping" widget, which doesn't render a new page but handles the request through AJAX, they want to know. If you even scroll down to a certain point on a page, they may log an event (I don't know for sure, but I could check easily enough). All these events are web analytic events.
3. Amazon exposes product and shopping APIs to third parties in order to improve their funnel and affiliate sales. Each service call against these APIs would be an event.
That's the kind of thing I'm talking about. High volume, concurrent events, of which you may have little control over the rate of requests. So, if some new affiliate creates a site that hits your API waaaay more than expected, you need to be able to either scale quickly or throttle the affiliate requests. Or, if you come under DOS attack (or even just an unknown crawler), you need to be able to handle that. Or if someone runs a Superbowl ad for something that can be bought on your site, you want to be able to handle the sudden increase in traffic.
-
Just as an FYI: the forum is, in fact, considering a move to a cloud-based service like AWS after the "little" snafu today that crashed the forum for several hours.
-
Just drop logs in an S3 bucket or log to a db instance. If you rely on traditional app server logs stored to the machine, you will lose them when an instance is terminated.
It's the most common mistake people make when first moving to ephemeral servers.
-
Just drop logs in an S3 bucket or log to a db instance. If you rely on traditional app server logs stored to the machine, you will lose them when an instance is terminated.
It's the most common mistake people make when first moving to ephemeral servers.
I noticed that with AWS (I opened a free account to do some experimentation). How robust are the instances? If I have one running and it shuts down, how much do I lose?
-
Just as an FYI: the forum is, in fact, considering a move to a cloud-based service like AWS after the "little" snafu today that crashed the forum for several hours.
Oceander, just as a point of contention on cloud services, IaaS does NOT provide resiliency. If you are trying to go to cloud for the purpose of resiliency, understand that often what is sold is rapid recovery of the OS only! Your data, your configs, many times are up to you to detect and repair.
-
Oceander, just as a point of contention on cloud services, IaaS does NOT provide resiliency. If you are trying to go to cloud for the purpose of resiliency, understand that often what is sold is rapid recovery of the OS only! Your data, your configs, many times are up to you to detect and repair.
Part of what I want to find is the ability to scale resources up and down with a high degree of granularity. I'd also like a better ability to fine-tune the resources that are getting the most use, without having to acquire large amounts of resources that aren't needed. For example, the forum doesn't actually need that much disk space, but it does need cpu time, RAM, and bandwidth.
-
I think that AWS is probably the best overall.
For one thing (the last time I checked in 2012), roughly 5% of all internet operations (worldwide) are handled by AWS. They are the single largest commercial provider of servers and processing for the internet, period. You can't beat that in terms of scalability.
-
I think that AWS is probably the best overall.
For one thing (the last time I checked in 2012), roughly 5% of all internet operations (worldwide) are handled by AWS. They are the single largest commercial provider of servers and processing for the internet, period. You can't beat that in terms of scalability.
True, but the question then becomes cost; the problem is finding the mini-max solution.
-
I noticed that with AWS (I opened a free account to do some experimentation). How robust are the instances? If I have one running and it shuts down, how much do I lose?
(I'll talk in terms of AWS, as it is my preferred provider.)
The quick answer is "everything that you would normally expect stored on a server's local disk."
These are ephemeral instances. When they crash, they get terminated and removed, along with all the data that was on their local disk. Your EC2 cluster realizes the need for a new server and starts spinning one up to replace the one it just terminated. This new server is based off your machine image (AMI), which is a point-in-time, clean configuration for the box. Savvy shops will build custom deployment managers that will grab the latest version of deployed software so that AMIs don't have to change constantly.
You have to make a paradigm shift about where you store data. Local things that you want saved must be moved to cloud-based storage solutions - either cloud DB, or cloud disk. That's why I mentioned S3 earlier.
I don't know how this forum works under the hood, but you may need to rejigger some of the code to properly work with multiple ephemeral instances and centralized storage - I don't know. A google search reveals that a couple of providers have already put SMF solutions on AWS - Bitnami, Turnkey, and Webuzo. Start there and see if it prices in the range y'all are willing to pay. Bear in mind that with autoscaling, you may have fewer servers running during non-peak hours depending on your min/max server counts set in your autoscaling group.
-
(I'll talk in terms of AWS, as it is my preferred provider.)
The quick answer is "everything that you would normally expect stored on a server's local disk."
These are ephemeral instances. When they crash, they get terminated and removed, along with all the data that was on their local disk. Your EC2 cluster realizes the need for a new server and starts spinning one up to replace the one it just terminated. This new server is based off your machine image (AMI), which is a point-in-time, clean configuration for the box. Savvy shops will build custom deployment managers that will grab the latest version of deployed software so that AMIs don't have to change constantly.
You have to make a paradigm shift about where you store data. Local things that you want saved must be moved to cloud-based storage solutions - either cloud DB, or cloud disk. That's why I mentioned S3 earlier.
I don't know how this forum works under the hood, but you may need to rejigger some of the code to properly work with multiple ephemeral instances and centralized storage - I don't know. A google search reveals that a couple of providers have already put SMF solutions on AWS - Bitnami, Turnkey, and Webuzo. Start there and see if it prices in the range y'all are willing to pay. Bear in mind that with autoscaling, you may have fewer servers running during non-peak hours depending on your min/max server counts set in your autoscaling group.
It certainly will require that I reconceive how I think about things. For one thing, I am used to thinking of disk storage as something persistent, so that even if the virtual server and/or container (as the plesk parallels system calls it) crashes.
Is it fair to say that what you end up with is something like the RAM drives we used to use to overcome the deadly slow pace of actual disk drives back in the 1980s? So that if the RAM "goes away" so does everything that was stored on the virtual disk?
I may set up a dummy version of the forum to test what happens when it's running on AWS.
-
It certainly will require that I reconceive how I think about things. For one thing, I am used to thinking of disk storage as something persistent, so that even if the virtual server and/or container (as the plesk parallels system calls it) crashes.
Is it fair to say that what you end up with is something like the RAM drives we used to use to overcome the deadly slow pace of actual disk drives back in the 1980s? So that if the RAM "goes away" so does everything that was stored on the virtual disk?
I may set up a dummy version of the forum to test what happens when it's running on AWS.
One of the nicest things about AWS is that they have a really helpful crew to help you with all of it.
-
One of the nicest things about AWS is that they have a really helpful crew to help you with all of it.
Good to know. Still, I like to learn things myself through trial and error - which is why I want to play around with a dummy clone of the forum first. Sometimes I "code" without a net, but that's not very smart when it comes to messing with the whole forum.
-
It certainly will require that I reconceive how I think about things. For one thing, I am used to thinking of disk storage as something persistent, so that even if the virtual server and/or container (as the plesk parallels system calls it) crashes.
Is it fair to say that what you end up with is something like the RAM drives we used to use to overcome the deadly slow pace of actual disk drives back in the 1980s? So that if the RAM "goes away" so does everything that was stored on the virtual disk?
I may set up a dummy version of the forum to test what happens when it's running on AWS.
They have persistent storage. Set up the instance, then save it to an image. Then if the forum has a problem, just launch the image. I run my own personal website from AWS (though I'm meaning to change it).
-
They have persistent storage. Set up the instance, then save it to an image. Then if the forum has a problem, just launch the image. I run my own personal website from AWS (though I'm meaning to change it).
@Oceander
Right - what you would do is set up an external Elastic File System, which you can think of as a NAS. You would mount the EFS to your EC2 instances as part of your AMI so that every new server that instantiates connects to the EFS.
If you don't do this, you lose everything that is stored on the machine itself when it terminates. So, when you are looking at EC2 instance specifications (M3 or C1, for example) and they list storage, that storage is ephemeral. Use EFS to have non-ephemeral storage that gets reused between ephemeral instances.
(Or, if you are archiving data, or persisting logs that you are unlikely to access after written, you can drop them into an S3 bucket like I mentioned before. That's what we would do with server logs.)
-
They have persistent storage. Set up the instance, then save it to an image. Then if the forum has a problem, just launch the image. I run my own personal website from AWS (though I'm meaning to change it).
@Oceander
Right - what you would do is set up an external Elastic File System, which you can think of as a NAS. You would mount the EFS to your EC2 instances as part of your AMI so that every new server that instantiates connects to the EFS.
If you don't do this, you lose everything that is stored on the machine itself when it terminates. So, when you are looking at EC2 instance specifications (M3 or C1, for example) and they list storage, that storage is ephemeral. Use EFS to have non-ephemeral storage that gets reused between ephemeral instances.
(Or, if you are archiving data, or persisting logs that you are unlikely to access after written, you can drop them into an S3 bucket like I mentioned before. That's what we would do with server logs.)
@Weird Tolkienish Figure
@bolobaby
That's useful to know. In terms of the forum, it consists of a large number of php scripts which are static and the database, which is where all of the dynamic data is kept (along with a lot of other mostly static data that is specific to the particular forum, like member records).
Since I tend to keep backups of the files, and since they're static, a loss of the files themselves would not be catastrophic and would probably not even be noticed. A loss of the database would be pretty bad since everything that changed since the last backup would be lost.