Tom Brokaw at the Microsoft 2008 Launch Event Celebrates Technology Heroes

It's not often that you hear people who are involved in technology referred to as "heroes".   Most of us are just techno-geeks trying to do the best we can.   And while I don't necessarily drink the kool-aid as far as those of us who are using the newer Microsoft technologies being called heroes, I have to say I was struck by Tom Brokaw's opinion of what technology has meant to our world and our society.

I went to Microsoft's Los Angeles Launch Event 2008 this past week in celebration of the release of their three new products, Windows Server 2008, Sql Server 2008, and Visual Studio 2008.   Lots of great advances, lots of awesome stuff, and their theme was "Heroes Happen Here".   By that, they mean that the people who are using these new technologies are the heroes of technology.

But Tom's keynote speech (I was actually surprised when he came out on the stage at the Nokia Theater) revolved around what technology has meant to people in other parts of the world.   He admitted to not being very computer savvy...and admitted he would probably never write a line of code with Visual Studio, or manage a Hyper-V Virtual Server Farm...but he did recall the importance of technology in making our world smaller, and hopefully better.   He talked about the people who went to Pakistan to help during their last devastating earthquake, and how that when they came down from hiking into the deep mountains they were able to put fingers to keyboard and let the world know what had happened.   He talked about how technology was helping to improve farming, and irrigation, and what that meant to the lives of people living in Africa.  He talked about surgeries being led remotely by doctors via videoconferencing.  I can't recall all the stories he told, but they all held essentially the same meaning:  that the people, the programmers, the administrators, all of us who help make technology what it is, and make it available to the true heroes of the world, we all have a stake and a helping hand in that heroism, and he wanted to thank us.

Steve Ballmer, Microsoft's CEO, gave the rest of the keynote address as the event moved from true heroism into our own personal versions of "code heroes"...but what Tom Brokaw said does ring true.   Technology helps in ways that sometimes we never know about, and our role in that is crucial, and we shouldn't take for granted what we do with it.   We're all heroes.

An Intro to Amazon Web Services

Amazon Web Services, or more simply AWS, provides a wide range of web services for building technical infrastructure. It's not a replacement for having an ISP (or a data center, if your firm is that large), but it's a great way to avoid spending too much money on infrastructure before you even have a proven business model. It's also a brilliant way to scale up resources on demand, even if – and sometimes, especially when – you already have a successful business model. In this article, let's introduce the web services available from Amazon, discuss a bit about use cases, then talk about business strategy for leveraging AWS.

Have you ever had a great idea for a new tech business, but weren't sure what to do next? Write a business plan? Set up a meeting with a venture capital firm? Launch a web site? Getting from A to B can become enormously expensive, when you start to think about buying servers, hiring sysadmins, leasing facilities, etc., before you can even demonstrate your idea running on the Internet.

Amazon considered that problem. For a few years at the start of this decade – post Bubble-bursting – they analyzed what it cost for new businesses to launch on the Internet, where the risks seemed to hit worst. They took their internal technology infrastructure and re-examined what parts could be rolled out as public services. Associates (ECS) was launched in 2002, then much of what people recognize now as AWS really began to catch on in 2006. More services roll out just about every quarter, it seems.

So about your great idea for a new tech business... Do it. If you have a web app, then go prototype it using Ruby on Rails, or Django, or any number of PHP frameworks. Build it on your laptop, upload code to EC2, save an AMI, replicate as needed. (Don't worry, we'll catch up on these acronyms in a bit here.) Or, if your business idea involves a lot of data crunching, maybe something like Hadoop, again you can build on your personal computer, then upload and run it at scale on EC2. The bottom line is: pay only for what you use, and scale up to enterprise size on demand.

One minor thing you want need to know beforehand: Amazon.com staff abbreviate terms and even acronyms using numbers. Have you seen how the term internationalization gets abbreviated as "I18N" sometimes? They do that. A lot. Apparently, it comes from the top; it's a geek thing, and at Amazon.com even the business types are dyed-in-the-wool tech geeks, all the way up to the CEO. So the AWS web service names follow similar spelling conventions: instead of saying Simple Storage Solution, they use the term S3. Having a good decoder ring might help make reading their documentation a wee bit simpler. Ahem.

Let's take a tour through their current set of web services...

Associates

Amazon Associates, formerly called ECS, was the first web service offering from Amazon. Use it in conjunction with Amazon Associates Program to create web storefronts. In other words, your site acts as a reseller that refers products at Amazon and its third-party vendors. Earn up to 8.5% in referral fees – which can be quite good money, overall.

EC2

Elastic Computing Cloud provides resizable compute capacity. Servers on demand, rented per hour. Grid computing, clouds, or – as they say on Sand Hill Road – "Google scale". You can configure a Linux server, then store it as an image (Amazon Machine Image, or AMI) and launch as many as you need. It takes only a few minutes to launch a new server, and you can stop it anytime you like. You pay by the hour, paying only for capacity that you actually use.

Small Instance (default)

• $0.10/machine/hour, plus data transfer
• 1.7 Gb memory, 1 32-bit virtual core with 1 compute unit each, 160 Gb disk

Large Instance

• $0.40/machine/hour, plus data transfer
• 7.5 Gb memory, 2 64-bit virtual cores with 2 compute units each, 850 Gb disk

Extra Large Instance

• $0.80/machine/hour, plus data transfer
• 15 Gb memory, 4 64-bit virtual cores with 2 compute units each, 1690 Gb disk

Are there any downsides? Sure, EC2 is not a direct replacement for running your own data center – if you need that. For starters, there are no dedicated IP addresses (they're working on it). You'll need arrange for dynamic DNS. That happens to be a very good way to combine the strengths of a local ISP and EC2, by resolving your DNS from the local ISP and pointing into the cloud.

Another downside is that while you do get root on your virtual server, you never really know what the actual hardware lives, and you face some restrictions when it comes time to make kernel mods. Amazon documentation keeps repeating the refrain about a hypothetical "circa 2007 1.7 GHz Xeon processor", but that may be the most you'll ever learn about their data centers. (I hear they're stacked in shipping containers near hydroelectric plants, but that's just rumor – albeit a rather "cleantech" rumor, at that). My dev team ran into problems with some special uses of MySQL that required kernel changes. Hadoop won't run on the standard AMI, but there are special Hadoop AMI which work well. Generally there are work-arounds, but be forewarned that you're not going to walk into any data center and push a reboot button on an EC2 instance. Ever.

Still, I find that EC2 runs better than other Linux virtualization systems that I've tried. Tough to beat that price-point, too.

S3

Simple Storage Service allows you to write, read, and delete objects in "buckets". Each object can range from 1 byte to 5 Gb in size, with an unlimited number of objects per bucket. Think of S3 as a very large hashtable – think of key/value pairs – persisted to disk and replicated across several different data centers. Use REST or SOAP operations to read and write objects. You can also use BitTorrent for streaming media out of S3.

One application is to store LOBs in S3 instead of in your database. Or serialize large objects directly out of your middleware. A good approach is to use some transport language like XML or JSON to encode data objects, so they can be used directly in an AJAX call by a web client. An even better idea is to put a Distributed Hash Table (DHS) as a kind of middle-tier cache for objects persisted out to S3. Read more about that in Amazon's paper about their Dynamo project.

How much you ask? $0.15/Gb/month, plus data transfer. That's almost getting cheaper than buying a disk upgrade for your laptop, byte by byte.

One thing to consider: how do you manage your business requirements for off-site backups? I know that our business insurance certainly requires that kind of practice in place. Frankly, with S3 you must manage it yourself, and that implies costs for network transfer. One suggestion to Amazon would be to provide alternate means to bring S3 data out at a reduced speed, trickled out for backups.

SQS

Simple Queue Service provides a way to manage message queues – up to 8 Kb of text data per message – which can scale arbitrarily large. It's inexpensive and highly reliable. This is my favorite part of AWS, and potentially the most valuable to Internet entrepreneurs.

If you're familiar with MQSeries from IBM, you understand what this provides. For example, when I was working in banking software, MQSeries could harden gigabytes of transaction data reliably, while we waited to have new mainframes booted on the other side of the queue. That may be only 25 words of description, but in practice it was a nightmare made simple through an amazing IBM technology.

SQS follows that pattern. Integrating SQS into a web app is a little bit of a cognitive stretch for many developers; you really must embrace a different mindset. Once you do, you probably won't be thinking in terms of MVC design patterns much longer.

Costs? $0.000001/message, plus data transfer. In other words, $1 per million messages processed.

SimpleDB

SimpleDB might be described as a cross between a relational database and a spreadsheet, except that each cell may have different attributes. The whole shebang gets indexed automagically, then you can perform queries, joins, intersections, etc. – in very large quantities. Think of it as a good place to store pointers and metadata for those large objects stored in S3. You can hit the SimpleDB part of an object first, run its metadata through your business logic, then determine whether or not you want to stream out gigabytes from, say, some video you stored in an S3 bucket.

How well does that compare with Oracle licenses? At a mere $0.14/machine/hour, plus $1.50/Gb/month, plus data transfer, and with less headache about scaling and performance issues, it does look rather compelling.

FPS

Having worked around e-commerce for 15 years, in my opinion Flexible Payment Services has got to be one of more ingenious parts of AWS. It's a web service that simplifies the process of taking money reliably (for you, the seller) and conveniently (for your customer). It handles payment the same as how people do checkout on Amazon purchases – which is pretty much guaranteed to be familiar to consumers. You can program rules about billing, create recurring fees, add a Pay Now widget to any web site, or set up to handle micro-payments which would be prohibitively expensive through most financial processors.

You may be able to negotiate a better rate with your bank – once you have sales established – if you're not concerned about little nuances like chargebacks, fraud, etc.

MTurk

Mechanical Turk is described as "a marketplace for work that requires human intelligence" or an "elastic workforce". See this Wikipedia entry for more background about the name. You describe a kind of task, called a "HIT", then people sign up to perform your HITs. There is a $0.005 per HIT minimum commission, and Amazon takes 10% of commissions.

This is the kind of service which has been used to coordinate massively collaborative search for downed planes and lost ships, such as the search for Steve Fossett.

Alexa

There are also four Alexa web services, primarily for web analytics: Site Thumbnail, Top Sites, Web Information Service, and Web Search. I've used them a little commercially, and they are quite different from the other AWS offerings. Probably best to reserve that discussion for a follow-up article.

Strategy...

Let's repeat this point again, because it's important: the strategy for leveraging AWS is to build out your engineering so that operations can scale from Day One. Get running quickly, only pay for what you use. Get early feedback from testers and customers about your business ideas. Most importantly: save your seed equity for expenditures which are more important than technology infrastructure.

What could be more important in a tech start-up than technology costs? The following items top my list:

• Legal fees for NDA, HR, contracts, financing, etc.
• Health insurance and family benefits for employees
• Patent and trademark filings, which grow increasingly complex internationally
• SG&A to establish initial sales

If there is any way possible, avoid going for VC money until the last possible second. VCs appreciate that, probably even more than entrepreneurs. Wait on a cash infusion from venture capital sources until you have initial sales established and need to build sales channel, need to spend on marketing to establish your brand, etc.

AWS has been crafted by some of the most successful people working in e-commerce to give you exactly that kind of quick-start advantage. And why not? Ask yourself, would you rather earn a few hundred dollars each month from multitudes of new tech start-ups, early in their growth curve, assisting them to grow larger (and spend more)... or would you prefer to wait until a start-up has proven itself in the marketplace, then try to sell e-commerce and fulfillment services? Amazon has been active in both areas, but I have a hunch that the former strategy (AWS) earns more revenue over time.

We can discuss more later about system architecture, design patterns, and how to build infrastructure that scales up and scales down on-demand. Meanwhile, keep in mind that AWS can provide you with enabling services so that your new business idea has a much better path toward becoming a success.

So you are looking to connect with people like you...

Maybe the best way is to start your own group.    And no, I don't mean therapy group!

When I relocated to the Central Coast, I wanted to see how many .NET developers there were here because that was my specialty and something I was interested in.   In Los Angeles where I had lived before, it was not uncommon to find several "User Groups"...groups of people who shared a common interest that got together once a month to talk about what they are interested in.   So I started hunting around for a User Group here.

I couldn't find one, although certainly there is SLOCAMA, and SLOBytes, there was no .NET or even a general "programming" User Group.   So I started asking around, and it was hard for me to explain to people what exactly it was I was trying to accomplish.

So I started my own.   In November of 2006, I founded the San Luis Obispo .NET User Group.  Now we are 40+ members strong and meet once a month.   A typical meeting will generally consist of pizza or pasta or panda express, a 90 minute to 2 hour presentation by a member or an outside speaker on a topic we are interested in, and then everyone hangs around to chat.   It's a great way for all of us to expand our knowledge and share our ideas.

And the reality is, it wasn't that hard to get it started.  You basically need two things:  a conference room, and a projector.   In most cases, you can find conference rooms at banks, accounting firms, there are many local places that you can get one.   In fact, the San Luis Business Center will rent one to you, and KCBX.NET will rent you one with a projector!   Granted, you might spend $80 a meeting, but the benefits far outweigh the cost.   If you're lucky like we are, you have someone in the group with access to a room and then you don't have that cost either.

Then you need a vision, a purpose.  In our meetings, it's .NET programming.   Yours might be databases, or design, or even electrical engineering.  The topic and vision don't matter as long as you have one.  And if you have one, I guarantee that there are others out there who share it.

Finally, you need speakers.  Our group has only had two or three people from outside come in and speak.  For the most part, our own members have been willing to step up and investigate something and come back with their findings.   None of us are professional speakers, and it's very laid back and low key and about the sharing of ideas.

I placed a single comment in the Tribune with the weekly Central Coast Technology article written by Dan Logan and set up my first meeting.   Fourteen people attended, and we've just grown from there.  Dan continues to be supportive, and groups like Softec have stepped up to offer their support as well in the interest of "Community Education".

It's really that easy to start a group.   Get a room, a topic, a projector if you need it, a speaker (even if it's you), some food, and set up shop.  Even if only two people show up the first time, word will spread and your group will grow.

It's definitely worth your time and effort.

 

Robert Hope, Founder

San Luis Obispo .NET User Group