Amazon Web Services is looking the goods
I have to admit I am a fan of the work that Amazon has done putting together what is now a compelling collection of infrastructure services. Together these facilities provide a fantastic vehicle for hosting highly scalable and reliable systems. And the pricing model where you pay for what you use, with prices constantly being reviewed is very enticing. Elasticity is taken to an entirely new level – machines can be purchased by the hour for as little as eleven cents. Some of the services charge in micro cents – more on that later. This is the first of three posts examining Amazon Web Services. this post introduces the key concepts, the next post will talk about some of the scaling techniques provided, and the final post will focus on how Altium is currently leveraging Amazon offerings.
The key services Amazon offer include storage, compute power, and a range of auxiliary services designed to enhance these. Here is a very brief overview:
- Amazon S3 (Simple Structured Storage) provides storage facilities. You can store files of all kinds including video, audio, software and anything else. You pay for the storage and the bandwidth to access them. Costs are very low. Altium uses S3 to store many things including training videos and software builds. When Altium releases a new build, tens of thousands of customers need to be able to get the 1.8GB file very quickly and this works well for that. Storage reliability comes in two levels – the highest provides a 99.999999999% (11 nines) probability of not being lost.
- Amazon Cloudfront provides a perimeter caching facility – files stored in S3 are distributed to local nodes around the world so that people can get access to the files quickly. There is an option to provide files as delivered via streaming.
- Amazon EC2 (Elastic Compute Cloud) provides access to virtual computers you can buy by the hour. The computers come in a number of hardware configurations ranging from low-end single processor machines through to big boxes with lots of processors and 68GB RAM. Machines come in Linux and Windows flavors. You can also get the machines preconfigured with certain hardware packages, and the price of those (if they are chargeable) is built into the rental. For example you can get a machine MS SQL Server in different flavours ranging from free to the Enterprise level. Machines can be imaged so that you can take them off line or replicate them very quickly. EC2 instances can be tied to Elastic Block Storage, effectively S3 storage.
- Amazon RDS is a special case of EC2 that comes with an embedded MySQL database and a range of value added facilities, including automatic backups and the ability to achieve failover replication into an alternative hardware partition in case the main server goes down.
- Amazon SimpleDB is a lightning fast schema-less string-based database consisting of items with many named attribute value pairs. You can, for instance, store a customer record with values for Name, Address, Phone etc, but you can store anything you like. If you want to store Favorite Color for one customer, you can. You can even store multiple values for the one attribute.
There are a range of other facilities, but I will discuss these in a later post where I talk about facilities to help you achieve elastic scaling (up and down).
What Fast Internet to the Home will Mean
In the recent Australian election campaign, one of the key campaign focal points was the policy concerning a high-capacity broadband backbone for the majority of the country. There are many aspects of this that are interesting.
The fact that it is an election issue interesting of itself – it means that there is now recognition that Australia is falling behind other developed nations with regard to the Internet and that keeping pace with this particular form of infrastructure is important.
Then there is the fact that there are such divergent views as to how this should be implemented. One party says they want to put in an optic fibre cable running to 93% of the homes and deliver at least 100Mbps, possibly 1Gbps. This will cost an estimated $43Billion. The other party proposes a network based on a major rollout of wireless towers delivering 12Gbps to 97% of homes at a cost of a little over $6Billion.
And the other thing I found interesting about all of this is that no-one seemed to have any imagine about what would be done with all that bandwidth. I have no problem with spending on a major infrastructural scheme, a scheme that brings back memories of the Snowy Mountain Hydroelectric scheme in the scale of its cost, but I question why people are proposing such major projects when they don’t seem to be able come out with illustrative examples other than entertainment and medicine. Apparently we are all going to have our lives saved.
I noticed some people asking why would we need 1Gbps, and my response is to remind people that when Bill Gates and Paul Allen were discussing how much RAM a computer could ever possibly have, they thought that 640KB would be ample.
There are plenty of things that 1Gbps bandwidth could allow, and I will attempt to explore some of them, but first there is one other variable in this that I haven’t seen mentioned in the media and that is the bandwidth consumption that this fast connectivity will allow. How are the consumers going to be charged? If the speed encourages people to pull down heaps of stuff, or upload huge amounts of data, it will mean that costs will blow out if the data price does not come down considerably.
One of our staff based in Japan has 1Gbps to his home and an upload limit of 50GB per day, no download limit. I wish.
Now, what kind of things could you do if you had that much bandwidth?
The obvious ones, like entertainment, with games, movies don’t really bear mentioning, except that once the time taken to download files of a given size passes some threshold, it ceases to be a barrier to use. For different people that point will differ.
Torrents will become incredibly convenient – files will be downloaded more often and therefore available from more locations, all of them high speed. I wonder what that will do for the movie, music and software industries.
One of the big changes that will come about is the interconnectivity via the Internet of all manner of devices. The speed and ubiquity of the net will increase the use of these devices in the so-called Internet of Things. Devices will maintain state in the cloud and check in to report their current situation as well as see if there is something the device should know about. Agent software will interact with this information and perhaps make changes to the values for other devices.
Here is an example: a car’s GPS is sending to the cloud information about where it is – XYZ coordinates that enable some agent software to determine velocity, direction etc. The agent software knows where home is and can make a determination that the car is heading to the house. As it passes some predetermined proximity boundary, the agent can check to see what the current situation is with the home air conditioner or heater and based on the ambient temperature and preconfigured preferred temperature, it can turn the air conditioner on.
Opportunities to exploit this information will grow as the bandwidth to support them grows. So with the GPS example, the data could be sent back to a central agent that can determine how many cars are not moving quickly enough in one location, and tell the GPS that there is a traffic problem up ahead. Or the data could be anonymised and fed to a government town planning service that can use the data to determine future bridges, tunnels, toll gates etc.
Metcalfe’s law states that the value of a communications network is equal to the square of the number of nodes. When devices are connected like this, the potential is unimaginable.
Like Facebook or Twitter, where individuals decide what they wish to publish, and other individuals decide what they wish to subscribe to, the idea of devices publishing a range of data for other permitted devices to subscribe to, either directly or through some intermediate agent software, is very compelling. Printers can publish their toner levels and how many drum cartridges they have, while some device designed to dispatch a printer repair agent can keep an eye on things. Not only can devices publish information about their current status, they can also have built in a range of metadata – information that they seek out from elsewhere on the intranet and republish. For example, while a printer may publish information about its toner levels, it may also have a Google-style search that publishes the results of a search on its own model number. This would be useful in the case of factory recalls, announcements of new firmware, or perhaps sales of peripherals, accessories and consumables.
To bastardize a quote from the movie Field of Dreams, build it and they will come.
Pride in Equipment
CIOs and IT Managers of the past took pride in how many servers they managed. CIOs and IT Managers of the future will pride themselves in how few.
Recently one of my team members went to one of our regional offices to help rollout some new equipment and redo the IT infrastructure. What they discovered was chaos:
Notice the mess?
Here is the result after fixing it, with all cables color-coded, fiber optics built to connect everything etc:
Mess? What mess?
The point I am illustrating is that the server room, like everything else, needs maintenance and attention. It needs to be managed by professionals. Why would anyone want to do this when their core competency lies in making people laugh, making widgets, or fixing them?
I have been at conferences where people talk about the tribulations of moving premises with a server room and how proud they are of their new server room and their team who set it up. But seriously, why should anyone be proud of their server equipment for its own sake? Yes, we should take pride in our work, yes we should take pride in being professional, and we should be embarrassed if we see entropy bring a rack down to the example shown here. But our true source of pride should come from enabling our business to be the best it can be in delivering its core competency.
Unless servers are part of our core competency, More servers, less fun I say.
What excited me about Salesforce when I first saw it
In thinking about writing for this blog, I was musing about the journey I have been on with using the cloud and was thinking about my early days with Salesforce.
When I first used Salesforce, there were two things that stood out for me as being incredibly powerful about the product, and enabled me to see it as a potential business platform. These were the ability to define custom objects, with clearly defined relationships between them, and a customisable user interface – a user interface that was highly structured, yet flexible enough to drag and drop fields and sections around the page. Bear in mind these were nearly five years ago, and a lot of progress has been made since then.
I figured that with these features, I would be able to use Salesforce as a complete business platform, but I ran into the first of many of the limits Salesforce has imposed on the platform use. I will be writing more about these limits in a detail post at a later point, but for now I want to talk about a limit that made me realise Salesforce really didn’t understand just how much potential they had to break the shackles of their CRM roots: you couldn’t have more than 25 custom tabs. I told them that 25 was nothing and they asked me how many could I possible want – 40? 50? I sent them a reply listing more than 90 specific tabs, which really got their attention.
Now I have more than 300 custom objects in our installation of Salesforce being used in all sorts of interesting areas. I intend to share some of these use cases over the coming weeks along with other stories using other cloud applications and platforms.
For now, I just want to plant a seed that the initial limits born out of the need to protect all clients in a multi-tenanted architecture place unnecessary restraints on people’s perceptions of what is possible. I intend to talk more about this concept as well.
The idea of runtime metacustomisation is such a powerful concept though. Born out of the idea that a single platform can be leveraged by not hundreds, not thousands, but by millions of users, it really excites me to think about how much leverage can be gained by abstracting more of the development into the background. Just like what happened when we went from DOS to Windows, and suddenly all of the work required to write drivers to support different monitors, printers and input devices became a thing of the past.
Plenty has excited me since, but it is hard to recapture that initial excitement when you know you are onto something special.
Hello World!
Hello World!
I am Alan Perkins, CIO at Altium Limited, a company that lives and breathes the philosophy that innovation is the key to a better world. Our company reflects it, our people reflect it and our products reflect it.
For the past four years we have been moving our entire business into the cloud and I am often asked to speak about our achievements and experiences. IDC recognised this last year by awarding Altium with an Enterprise Innovation award and naming me one of the top ten finalists for Asia Pacific CIO of the Year.
I intend to share my thoughts about various aspects of the cloud here. I welcome comments, and am keen to see others learn from my experiences, both the good and the not so good.
Cloud81 