Category Archives: Cloud Computing

Cloud Computing Technology

Clouds As Business Services (CABS)

Snow is again falling in Munich. The sky is covered with real clouds. One cannot see the sun.

snow in munich

Today, cloud computing is one of the most discussed IT agendas. (Along with social, mobile, and big data.) Many of my friends are playing important roles, i.e. directors, evangelists, program managers, and helping to define new directions, … Everywhere I turn, I see and hear about it: one company offering SAAS solution, another with PAAS, another offering fusion cloud, yet another with private cloud, …

Is it spring already? If not, cloud is certainly breeding like rabbits!

I just came back from a trip to Bonn, Germany. I flew into the Köln-Bonn airport this past Monday. Because my hotel could not beforehand confirm room availability during my entire time there, I decided to rent a car at the airport, in case I needed to move. However, when I got into the hotel, the front desk told me that there is room availability for the entire week. Hence, my rental car sat in the basement garage for the entire week. All together, it accumulated 30 KMs. On my way back, I thought how nice it would have been, if I just took a cab from/back to the airport. It would have been cheaper and certainly a lot less hassle.

Pure Product Strategy is heading toward a dead end

Similar situation applies to enterprise computing. Despite all the fluffs around utility computing, what companies really need are solutions that address their needs. The problems with cloud computing nowadays are:

  1. Every IT vendor wants to be the cool kid and offers one or more cloud PRODUCT(s).
  2. Hence, there are too many cloud PRODUCTs out there.
  3. A lot of efforts are required to adapt existing business processes to fit PRODUCTs.
  4. Many times, those PRODUCTs overpromise and under deliver.
  5. Hard to measure if one PRODUCT really produces better ROI than another.
  6. No formal PRODUCT standards & portability. Vendors prepares nice labels; but, they are apples and oranges apart.
  7. Since your data lives in the vendor cloud, it’s pretty hard to swap PRODUCTs.
  8. …

Yes, I have capitalized the word PRODUCT in every sentence. Vendors want to appear savvy. Hence, all their marketing departments are marching in the same direction. Maybe, the font is different, the marketing logo is bigger, and the punch line is sexier. Yet, they all want a piece of your IT budget.

Debunking Gossips

When all rushing to the door, funny things happen. I remember people telling me following myths:

  1. Freemium product is the best business model.
  2. Being Lean will reshape the product innovation process.
  3. Product companies have higher valuation than service companies.

Let’s check the facts.

  1. Oracle bought Eloqua last year for $871 million. Both offer pretty expensive softwares and are doing quite well without having freemium. SAP, Microsoft, Apple. Somehow, big names live pretty well without freemium.
  2. Apollo program took the man to the moon. Was it cheap? Definitely not. It was one of the most expensive endeavors in the human history, costing $23.9 billions (http://en.wikipedia.org/wiki/Apollo_program#Program_cost). Even the SpaceX Dragon costs a cool $800 millions – $1 billion. Being fat works pretty well.
  3. Google won because they provided one of the best searching experience / services. Even their website reads it. (About Google Product & Services)

Obviously, grapevines have been debunked.

  1. The markets are willing to pay for premium quality – iPhone, iPod, iMac, …
  2. Complex engineering requires extensive planning & design – BMWs, airlines, yachts, …
  3. Winning services company really pays attention to user experience – Lexus, Mövenpick hotels, Tiffany, …

Better Services Excel over Lame Products

When companies take a pure product orientation, they become really very inward looking. (Just read Steve Blank’s blogs – http://steveblank.com ) People sitting in black rooms would spend a lot of efforts (money, time, and resources) guessing what the market will look like in 1 – 5 years from now. Then, the marketing department would go on blitzkrieg advertising campaigns. 9 out of 10 times, those efforts would fail and products would be shelved. Bunch of $$$s needed to be written off.

I was in LA a few years ago and saw lots of impersonators on the Hollywood Boulevard. It’s hilarious to have photos taken with the fake Bruce Willis, who never died in any action films & only getting senile slower and slower (Stirb Langsam). There are lots of starving movie-star-wannabes who are getting nowhere. Meanwhile, on NYC streets, the demand for cab drivers remains high. Some of those drivers may one day become famous, as in the case of Chevy Chase. Cab drivers were talkative & seemed pretty content on my trips.

stirb langsam

The point is: The market always demands better solutions. When IT companies think in term of providing superior services, they will look at one customer’s problem at a time and provides services people need; in contrast to product companies, who maybe too ambitious and forgot about the details.

Successful companies share common DNAs.

  1. They really know what people want, down to SERVING the need of a single customer. (Detail oriented)
  2. They are extremely focused on providing SERVICEs where the customer will use once, love, and come back for more – customer loyalty. (Focused)
  3. It’s better to SERVE one client really well than making garbage products where none of the 100 customers like. (Process-driven)
  4. Value creation through understanding and SERVING customers’ decision-making process. To SERVE, one must know how something makes sense to the customer. (Know-how)
  5. A good SERVICE is useful and provided only at the right moment. Otherwise, it would be a luxury. (Timing.)

Closing Thoughts

While cloud computing is already here and will becoming increasing important. The winners will be the ones who understand the value creation process and deliver solutions from customers’ point of view. This is what I call CABS (Clouds As Business Services).

Next time, I go back to Bonn, I will definitely hail a cab than driving in the snow on AutoBahn!

My Presentation @ the University of Burgundy

OK, today, I gave my 2nd presentation of the year at the University of Burgundy, France, on following topics.

  1. Lean Start-up Methodology
  2. General Project Management techniques, i.e. Waterfall, Agile, Lean, SCRUM
  3. State of Cloud Computing industry, latest news, i.e. mergers and acquisitions, the evolution of cloud computing platforms, coolest technologies, i.e. NoSQL, Functional Language, …
  4. Microblogging &  Liberalization of communication mediums
  5. Methodology to manage risk

Yes, it was done in French again to a stadium of ~100 students. Since yesterday, I have shared with students my experience of starting company, and providing cool technologies to IT giants, i.e. Salesforce, Amazon, Oracle.

So, here is my presentation on SlideShare.

http://www.slideshare.net/rgao2009/lean-start-cloud-computing-methodology-in-french

The History of the Salesforce Ruby Adapters

T Minus One – the Accidental Discovery

“Kevin, are you guys still interested in that FB CMS app?” – I asked.

“Ray, Sure, we do. The community has real needs for it”. – Kevin Answered.

That was the start of my love affair of Ruby. It was the winter of 2008/2009. One morning, I went out for the usual walk with my dog.  The night before, there was a heavy snowstorm, almost 3 inches. Texas, being a southern state, was ill equipped for snow. There was no truck on the road, spraying salt or sand. Naturally, there was nobody on the street, shoveling snow. The pavement was encased in ice and was as slick as the skating ring.

I decided walking on the grass instead of sliding on the sidewalk. Suddenly, I mis-stepped into an uncovered sprinkler hole. Apparently, my neighbor had a broken sprinkler head, and decided to replace it. With the sudden overnight snowfall, that job was unfinished; and, there was a big hole under the snow.

I limped back home with bruised knees. There were two big holes in my jeans, where my knees would have been. My doctor told me to keep my legs straight and knees unbent for the next three weeks. Thus, I walked like C3PO for the majority of February.

What should I do during that time? Lists of activities floated through my head, watching TV 24 hours a day, playing computer games, listening to music, Facebooking, … Yet, all those seemed to be fads to me, because somebody has either done it or is doing it.

Suddenly, an interesting idea appeared. I remembered that my church’s single group leader asked me to help them write a little Craiglist app for our group back in November 2008, which I silently nodded.

That app had to run from both inside and outside of Facebook, and, would do:

  1. Group managers could approve and reject new listings, since repetitive listings posted 20,000 times would be annoying. (Role based administration)
  2. A listing would stay valid for a maximum of 31 days. After expiring, the listing’s owner could reactivate it with group admin’s approval. A job server would take care of the listing’s expiration. (Job server / State Machine)
  3. Only group members could create and edit listings. Non-member must join our Facebook group prior to posting. (Security / Privacy).

For the next three months, I toiled away at this volunteer project, accumulating over 600+ hours. (Those were not Bunga Bunga nights; rather, late nights.) Next, at our group meeting, an important topic rose, where to host the app?

First, I volunteered to host that app on my Mac Mini Server. But, with 800+ group members, the 4GB memory was stretched to the limit, in light of running a Mongrel cluster, MySQL DB, JIRA, etc.

Furthermore, being a community project, the application data should not be owned by me, e.g. no hosting live / private data on my machine. It turned out to be a smart decision; as 8 months later, my Mac Mini’s boot disk would fail.

Next, we decided to go with a low-cost / dedicated hosting plan with a Florida company. But, their network connection was quite flaky, as it would lock up sometimes, causing 5+ seconds latency in loading a simple element.

We urgently needed an alternative solution! What other choices were available? I decided to explore cloud computing. Being thrift is critical for a community project such as ours. Moreover, paying 30 cents an hour is much more flexible than paying $250 a month.

Our safeguard was to regularly freeze and restart working VMs.

T Zero – The Cloud User Group

While it was fun that we could make snapshots and restart VMs at will in early 2009, I felt the need to work with other people rather than being the lone wolf. Thus, I organized the first cloud computing user group in Dallas/Forth-Worth area in 2009.

The first session was held at Cohabitat, in Uptown, Dallas. It covered topics and offerings by various players, e.g. RightScale, Heroku, Engine Yard, Google Engine, Azure.

At the seminar, I ran into Nick, who was the partner enablement manager @ Salesforce based in Dallas, and fellow ex-BEAer. He convinced me to give their platform a serious look. Running my app on Cloud2: that sounded like an interesting propositions.

With SF’s tutorial PDFs in hand, I experimented with the Force.com platform. Suddenly, I realized that fact: I was repeating other people’s effort. In effect, I was reinventing the wheel. The Force.com already provided a rich & well-defined data-model. Out of the box, there were already Accounts, Assets, Cases, Groups, Products, Users objects.

This led to a natural conclusion. The age of Community Data Model (CDM) has begun. It is faster to build an app by extending the CDM model than write the DB schema from the scratch.

As a programmer, I thought: with a little elbow grease, I could leverage the data model for my own purpose. (Since I was the lone developer for the church project, system admin and db admin tasks were eating up my valuable time. I was working literally 16+ hours a day, developing and maintaining the system. Yeap, it was unpaid work @ finest. So, I needed a way to offload those tasks.)

As the light bulb went off in my head, I realized the need for building a bridge between the SF platform and my app. After that, I can concentrate on my app logic, while benefiting from the CDM. Perhaps, I could quickly prototype multiple applications.

T Plus One – Searching for the Perfect Adapter

Plan is a plan. Now, it is time to turn it into reality. But, locating a Ruby/Salesforce bridge was the hard part.

Since 2008, Github has already become my main CVS repository. (During that time, there were only single digit Salesforce-related projects hosted on GitHub.)

Through Googling, I found a forked copy of the original ‘activesalesforce’ adapter, which rested on the RubyForge. But, that GEM’s API version was still hard-coded to version 11. Also, there was hardly any documentation. Even the in-lined API documentation was bare bone. I had a hard time understanding it, much less use the adapter.

After further Googling, I saw an old tutorial by the “Old Fart Developer”. Further struggling with setting up the GEM took me another week and half. Finally, I got lucky one day and made a successful call to the Salesforce and got back an XML representation of the Account object.

Chatter went into beta in late 2009. The Chatter Developer challenge was on. The competition was building your cool Chatter app and having the chance of winning a MacBook Pro laptop. A few months earlier, I had just purchased my 17” and didn’t need another laptop. But, thoughts of building new tools and working on a new paradigm – utilizing and extending a common object model attracted me. So, I read thoroughly all-available Chatter documentations, although I decided not to enter the competition.

T Plus Two – Friend of a Friend

My other interest was language, as in real human languages. It is just my personal opinion that given the long history, human language is far richer than the computer language. Among them, I liked to revisit French, which I studied in the university.

At Dijon, I became interested in marrying Embedded Devices with Cloud Computing. Several months earlier, at F8 meeting in April 2010, I thought about building an answering machine for social network feeds. Still, I wanted to push the boundary a bit further; I had already devised a plan to broadcast news feeds information onto game consoles, e.g. Xbox and PS3. For that reason, I would buy an Xbox 360. (Although I didn’t game much, I am interested in exploring non-traditional ways for visualization and expressing business data.) Of course, the technical challenge remained immense, as Xbox has a reputation for being a closed system. It is tightly locked-in, guarding against any attempts for sending and receiving information outside of the Live network. Still I had a detailed design for building a software bridge, crossing between the Game Console and the Windows Operating System boundary.

My church project still needed improvements. Building the standalone front-end website was a top agenda. I investigated the Hobo GEM, which is a nifty web app builder for RoR. I became very familiar at writing Hobo tag libraries. From there, I modified my app to extract and modify data in the Salesforce System.

I am a strong believer in the community-driven development approach. Eric Raymond’s book – The Cathedral and the Bazaar, has left a profound impression on my philosophy. So, I am more than happy to share my projects source code publicly, e.g. SFRWatcher, Facebook CMS app, WCI Portal Investigator, …

Now, I remembered Nick and decided to send him some screen shots. Some of those pictures eventually floated to Quinton. Those pictures showed that it was possible to retrieve both Chatter Feed (including attachments) and perform CRUD operations on Salesforce objects, e.g. Account, User, Product2, etc.

One day at FNAC (French version of Best Buy), I ran into an old acquaintance, from several years ago at another conference. We talked a little bit; and, he mentioned to me about the University of Burgundy and how their computer science faculty is open-minded. Perhaps, they would like some people from the industry making presentations over newest technologies. Since my most recent projects involved building extensions to cloud computing, it was likely that they could benefit from having my lectures/presentations on that topic. Additionally, I lived 35 minutes, by bus ride, from the campus.

I came home and wrote the head of the department a nice letter, who passed my contact information onto Christopher, their leading professor on semantic web and contextual search technologies. Now, I have formed a good contact with the French Academia.

Going back to the other thread, to my surprise, Quinton, Salesforce’s chief Developer Evangelist, wrote me back. With a few e-mails exchanges, he asked me to build a new adapter on top of the ActiveSalesforce GEM. Appropriately, I named my new GEM to ‘asf-soap-adapter’ in remembrance of all the contribution by previous developers.

Even though I have played with RoR technologies for around three years, building an industrial strength adapter was still a new experience for me. As a community would spring up around it, many useful suggestions from users would profoundly affect its development.

I multi-tasked. In the morning, I had my French studies. After lunch, I went back to my apartment; either worked on my presentation at the university and/or developed the ‘asf-soap-adapter’. As there were 9 hours time difference between France and California, So, I would often work long hours, until 1 or 2 a.m., answering e-mails and having conference calls.

In a matter of two weeks, I released the first GEM version, 1.0.1, at the end of September. It was my bundle of joy! Because I had a really difficult start-up curve with scanty documentation in the old GEM, this time, I used YARD tool and wrote extensive docs for my GEM.

T Plus Three – More Hacking

Next, Salesforce requested me to build a demo application for the DreamForce 2010 conference. Based on the conversation, I knew Marc Benioff would show this application for his KeyNote presentation. Unknown to me, the GEM would be the core technology for connecting RubyOnRails platform with Salesforce’s Database.com announcement, cementing the $212 million Heroku acquisition.

For the next three weeks, we had a sprint plan for developing the demo application (http://dbzillademo.heroku.com/) ‘dbzillademo’ using my adapter. The data came from ShopZilla / BizRate. At the second conference, they told me that they had uploaded a whooping 15 gigabytes. Now, we have more technical challenges ahead. First, the Salesforce SOAP API has a hard-cap, retrieval limits to maximum 2,000 rows per HTTP request. Furthermore, Salesforce SOAP API did not offer an offset feature, i.e. you cannot randomly jump from page 1 to page 25. If your query returned >50,000 rows, it meant having to use the “queryLocator” for ~25 times. At 25 x 2.5 seconds per call, it meant that the app would be locked up for 62.5 seconds. The second challenge was restricted by Heroku’s environment. Calls > 30 seconds response time would cause a generic time-out page being shown. While that made sense for standard applications, it was a real point for us finding solutions, enhance improving the adapter (ruby platform).

For the next week, Quinton and I worked on the solution for this problem. After some hacking, we refined our SOQL query to produce response time to less than 20 seconds. (I give much kudos to him, as he knows the Salesforce platform inside and out. Without him, the platform & demo would not have progress so rapidly.)

Still, I would like to see the response time be less than 2 seconds. Thus, I experimented with MemCache, using server memory to minimize remote calls. The knowledge gained from developing the demo has been incorporated into developing a better REST adapter. In the ‘asf-rest-adapter’, ‘memcache’ client has become a prerequisite.

After Thought

~ 30,000 people attended December’s DreamForce 2010 conference. It was an eye-widening experience. I saw many cool technologies. Many groups and companies brought out their best products and ingenious solutions. It was a great learning experience. My demo, along with others, was considered as important topics and was exhibited in the Moscone Center West’s Developer Pavilion.

VMForce, a system for building Java web application using Spring Technology and Force data, was announced much earlier in March 2010. They have set a very high bar for us, given their history and experience. There are many areas where that combo hits the sweet spot. But, our technology, RoR + Force.com platform, held up very well, particularly it was incepted and developed much later.

In the future, I would like to work with our peers to incorporate and co-develop the “Cloud-surfin” concept: community data model, many variants of programming languages & deployment platforms.

I am a firm believer that we are at a junction in the evolution of technologies. Whereas the old model spawned a wealth of individual browser-based web-apps, collaborative development and community effort will be keys for success for the future, e.g. reusable objects, classes, and the community data schemas. Borrowing from the “Aspect-Oriented Programming” concept, the “cross-cutting” technique could be also applied to the community data model / database design.

As a student of human languages, French has a flexible position in the use of adjectives, placed either before or after the noun, which it decorates. Very obvious things for users are an application’s feature-sets and behaviors, e.g. the response time, the accuracy of search, etc. Where the data lives, whether it is in Oracle, MySQL, or in a cloud-based data source, should be delegated to the best performing system.

Over the course of this project, I learned a lot. Here are particularly important tips. Test-Driven Development helps to reduce defects. Breaking large jobs into small chunks and rapid/frequent deployment (Running LEAN) ensures the project stay on course. User feedback is your best source of information. Yes, a community is the real genius, even it were only a group of three people.

My second presentation at L’U-B.

Today, I gave my second presentation at the University of Burgundy. I presented following major topics:

  1. The social network
  2. World Languages vs. languages of IT.
  3. New Developments in the world of IT.
  4. Cloud Computing
  5. Questions and Answer.

While my French still needs improvements, I spoke for one and half hours. During the presentation, I particularly stressed the importance of developing programming language unique for each culture, country, and human language. The world of IT as we known is becoming more and Americanized by English as the dominant programming language. While it is good to have a language everyone agrees on, it also poses limitation on human thoughts. During the written process, certain amount of information is lost via the selection process of:

  1. Deciding what is most important
  2. What to store and save; hence, what to omit,
  3. The sequence of storage
  4. Primary actors and actions
  5. The structure of the language.

The limitations of the written language thus pose a barrier to achieve perfect information storage. (See my slides on details.) Of course, the perfect information maybe wholly unnecessary. How to use and analyze the information perhaps carry greater importances.

Anyway, while I am not proposing to fragment the languages of computation, I do think it is necessary to introduce additional thought patterns and schools of thoughts to enrich our lives.

Additional things  I discussed included:

  1. Cloud Surfing – Ability to use the best feature and services from each of the cloud computing providers
  2. Database Cross-Cutting – The ability to leverage community schema. 80% of the common functions, e.g. logging, e-mail, group and policy management are nearly identical across multiple applications. It is much more efficient to extend a community database model than reinventing the wheel again. The community knowledge about the structure of applications is much more intelligent than one person, as well as better acceptance to help the evolution of the data model.

Of course, those ideas are still relatively new. Further refinement is still needed.

2nd presentation with professors of the University of Burgundy.

Christophe, me, Julien, and Nader Mbarek

The presentation is attached.

2nd_IUT_presentation

A Case of Imperfect Information (La folie de l’information Parfaite)

Prologue

I am in France right now. From time to time, I see new words that either I don’t understand or misinterpret the real meaning. Yes, there are words with multiple meanings. For example the word ‘le sens’ could either translate into ‘the direction of traffic’ or ‘the meaning of a word’. Even though, the English word ‘advantage’ translates into ‘l’avantage’, the French prefers using ‘l’inconvénient’ for ‘disadvantage’. Moreover, the romance languages (French, Italian, Spanish, and Romanian) have no superlative words, e.g. “good, better, and best”. They use “bon, meilleur, et le meilleur”.

As the world becomes more and more globalized, the demand for language localization grows accordingly. Translating technical words is a truly interesting experience. See the following table.

Languages for IT

As the above table shows, the German and the English logics are pretty similar, aside from capitalizing every noun and building long words. But, the romance languages (French, Spanish, etc) employ reverse logic, emphasizing the noun and placing it before the adjectives. Of course, Chinese, aside from using hieroglyphs, the logic is surprising similar to English.

Could we say that other cultures and civilizations have made poor judgments and erroneously translations? Is that really the cause for the superiority of English language, which is currently the dominant language for the field of information technology? Can someone please show me a dominant programming language of which the syntax is based on a non-English language, e.g. Arabic, Russian, Chinese, Hindu, etc?

Now, these are interesting questions, some of which are beyond the scope of this article, as I have yet to learn Arabic, Hindu, and Sanskrit. Let’s look back at the history; Chinese are quite proud for having documented evidences that they invented the gunpowder, the compass, the paper on which is used for printing, and the printing process. Yet, during the last several hundred years, those are not the domains which the Chinese civilization dominated. Furthermore, the Arabs excelled in the astronomy, around the time when Europe was still under the Dark Ages. Yet, it was Christopher Columbus who saw the future by crossing the Atlantic Ocean and led the path for colonizing the new world. Moreover, it was the people of the Hindus valley who had developed the initial concept of Zero as a number. Yet, it was Isaac Newton, a man born in an island nation far away from India, who developed Calculus.

Ironic, isn’t it!

Now, let’s go back to the 1st topic, information technology, and focusing particularly on the data & the storage. The database is the foundation of modern day software. One can hardly imagine a situation where a sophisticated application runs without using it, whether it is in a giant server-farm (virtual or physical), in mobile devices, e.g. smart-phones & iPad, or on smart credit cards, e.g. “la carte plus” which contains a chip for storing data. Yes, data is everywhere. Tools for manipulating information are everywhere, whether it is called Business Intelligence (BI), or Enterprise Application Integration (EAI), or the contextual web (intent-based searching using user profile information).

It has become clear that the means for analyzing data is as important, if not more, than the storage. You can give a Stradivari violin to a football player. But, if he doesn’t know how to fiddle, it would be quite useless; and, the noise can hurt your ears.

The Database Management function has been historical a distinct role in IT organizations. The DBAs are experts who are responsible for creating schemas, backing up data, and maintaining DB’s integrity. It is hard work. With the database growing evermore sophisticated, the complexity grows accordingly. It’s not hard to observe that the spiraling cost of hiring more proficient DBAs is denting the IT budget. Having many chefs in the kitchen is another interesting byproduct.

A better approach is needed. Cloud computing virtualizes vital computation resources, e.g. the storage, the processor, the bandwidth, etc. Those resources are managed by a team of experts. Spreading the cost of knowledge and responsibilities for managing data across multiple organizations is clearly the very first advantage. Furthermore, the collective intelligence of the community or the communities is clearly greater than an individual. Perhaps, it was Isaac Newton who discovered the gravity. But, it was a team of experts, scientists, engineers, visionaries, and workers, who built the rocket, which landed the first man on the moon, making that “a small step for man, {but}, a giant leap for mankind.”

Analysis

Rather than saying the translations were incorrect, it would be better taking advantage of the situation and consider it as an opportunity for improvements. After all, an application, ‘un logiciel’, is really a means for interacting with the end-user. The focus should be on developing something that is really useful rather than spending all the energy and time troubling over where to put the stuff. Furthermore, it is probably true that 80% of the database schemas in common applications are the same. If there is a cross-cutting (behavior based data management tool), the cloud DB is that.

I remember seeing the following lines somewhere, although cannot exactly remember in which language. “Perfect information did not exist in the past. Building flawless plans is my current goal. And, when we reach absolute perfection in the future, we will be in heaven.”