Release It - Part I

I am half way through the book Release It by Michael Nygard and thought I would scribble down my first take on this. The book aims to prepare developers for the uncertainties in a production environment. Essentially, the code you write behaves differently in different environments and most of the time what developers see (even forsee) is the behavior in their dev/qa/load-test environment. By the time you are done with load + functionality testing, the assumption is that things are production ready and so it's pushed to production. The kudos mails are in and the team heads out for the project party :-).

Once in the production environment, given the right push, your code starts to get stressed and strained. This manifests itself as behavioral differences compared to what was seen in other environments. As the computation involved in the application grows, the resources available to it get strained until finally it breaks down, bringing down the whole application. I am sure this is nothing new for a lot of people out there. I have heard organizations resorting to crazy (but practical given the circumstances they are in) procedures like restarting the applications once every 4 hours to get around these limitations. If true, there are even more surprising things out there like 400 restarts per day.

The book encourages developers to see things from a pessimistic point of view. Anything and everything outside your computational memory can fail, so how do you plan to work around this? The memory itself can fail but I guess you can't really do anything there. If you access the network, it could fail so have you planned for it? What is your plan? If you store lots of data into a database and try to read it, how does the sheer volume of data affect your application? If you integrate with third party services like credit card payment service or address verfication service, how does their SLA affect yours? If a part of your system is in trouble, does it bring down your whole application? How can your degrade your application gracefully when parts of your system is down? Have you planned for capacity difference between your application and the applications you integrate with? These are just some of the issues that this book is trying to address. Some of the proposed solutions like :-

1. Timeout (even for thread waits)
2. Fail Fast
3. Circuit Breaker

etc. are not so difficult to implement. There are more solutions available in this book if you are interested. In today's application where integration with auxillary services is the key to improving the core value of your application, I think the book has a lot to offer. I hope to provide more input once I am done reading the book.

Do you see the magic!!!

I develop REST services for my client and any typical REST service can be broken down into the following pieces:-

1. The request has to be converted to a domain object. The request contains the domain object in a serialized/marshalled state. This serialized state could be in many forms like request parameters (not the best of ideas), xml, json etc. The request url path along with the http method used uniquely identifies the service required by the domain object.
2. Now that the domain object is ready, we then proceed to business validations and persistence. This is a territory that people avoid screwing up. Of course, some still get it wrong but there is a concerted effort to get things right here.
3. Now the clients need to know about the domain object we just persisted, so it has to be serialized and sent back.

There is nothing unique about this picture. Any web application that we see essentially does the same thing. Let us take a look at these steps a little closer. Step 2 is well guarded and given a lot of importance since it is presumed to be close to the business. Step 3 is easy because all that is left is streaming out the domain object (a tad trivialized). To me, Step 1 is the most worrying because developers are left to deal with inputs from someone or something. These inputs are plain strings whose final target is a typed java object.

Without a framework like Struts2 to convert string representations to typed object, developers will have to deal with some really messy code. The work involved is repetitive and time consuming. Essentially these are glue codes that tie two representations together. I consistently see developers make mistakes and write messy code here. So I wrote a little reflection based code that copies over the data from the request to the java object after proper type conversion and field level validations (based on hibernate validation). The code was properly unit tested with copious amount of debug logs and proved to be very useful in all our use cases. So I decided to present this idea to my client. To my surprise, they were not so keen on a reflection based code. Some of the objections raised were maintenance, debugging and "magic".

For maintenance, I explained how the code was structured to be very clear and extended, if required. A slew of unit test cases and copious amount of logging should provide enough tool for somebody who wants to debug a problem. In fact, I employed the same idea used in Struts2 to create the java objects from the request and the best debugging tool I have used there is their awesome logs. Its magical and possibly dangerous (as in the conversion happens magically), so can't be trusted(?) is something I can't really defend because it essentially points to a difference in the wave lengths at which we think. Most of the popular frameworks out there do some form of magic to attain their goals. Hibernation uses antlr to generate sql from their hql. Struts2 uses ognl (which internally uses reflection) to convert request parameters to java object and the list goes on. We do not see any of these code but yet we use them without any doubt.

So may be the reason is that reflection is seen as something that is difficult to get right by the everyday developer. Yes, reflection can be dangerous, when used incorrectly. But that's true of any powerful feature. You can do dangerous things using Aspects in Java. But it's a bad argument to say that powerful language features are so difficult that only masters can attain them.

India Banao

I came across http://indiabanao.org/ and I like their idea. I am not sure how successful this is going to be but I would like to help the cause. I would urge others to also join in. Some of the articles written by the founder are really good.

The top two issues with the Indian system are :-

1) Corruption and

2) Slow....extremely slow justice system. I am not saying that it is unfair but the whole process is so slow that for the victims it seems unfair.  

I hope initiatives like http://www.parivartan.com/ can help with the corruption but I don't see anything to fix the judicial system.

"Monkey" Incident is back?

Is the infamous Monkey incident back?. I am not sure what the truth is but there seems to be a lot of smoke so maybe there is/was some fire somewhere.

http://ibnlive.in.com/news/symonds-reopens-monkeygate-row-blames-bhajji/77733-5.html?fads=ads

Hoping to be back..

It has been sometime since I have written anything. I start many posts but just don't get around to finish them because most of them tend to be very long. I guess I am too lazy :-)

I moved to California(US) about a year ago. I have been working in a new environment with new people. I am hoping to start writing again...so lets see how long I can carry this forward.

Smart Marketing...Funny too!!!

Found this on the web.... :). Smart...right?

BSNL to the rescue :-)

Its has been nearly 3 months since I had net connection at home. The location of my new home wasn't favorable to Airtel. Reliance didn't respond to my queries and Tata wasn't reliable. This left me no other option but BSNL. I wasn't happy about this but BSNL knew how to surprise me.

1. The application process was very fairly straight forward. The employees taking the application could have been better but I will take that.
2. I applied for a landline on Wednesday and got my connection by the following Tuesday. That is extremely good by BSNL standards.
3. I am awaiting my broadband connection from BSNL and in the meantime using their Netone dial-up internet. The speed may not be great but it is decent.

So over-all I am happy(i.e. as of now) and hope that these guys don't live up to their reputation.

Red Australia!!!

The red machine came to Australia. The red machine conquered Australia. The Finn started with a bang. His Ferrari was a clear 1 second ahead of the next car (the Spaniard). Is this a sign of things to come?. I hope so :) but we will get a better picture in a month or two.

As always Ferrari is in with a radically different idea. The current car has a longer wheel base than other cars and people are scratching their heads trying to figure out why. Though the Finn is blisteringly fast, he is also a known car wrecker. Once he learns how to conserve his car then he is there.

Pay me more?

I believe that a developer (hacker in Paul Graham's terminology) is more like a bound artist. I mean..think about it....no two software applications are the same. Getting something working 'successfully' needs some real sorcery. You have to carve more here...less there...some magic portion and volla..you have a great application. Of course the term great application is relative. When I was talking about a bound artist, I was referring to those artists who start off with a goal/idea in mind and try to figure out how to convey that idea. Of course there are those who start with a clean canvas, hoping that gold is going to ooze out :) and harden into an idea.

My favorite example of a bound artist is the designer of a Ferrari car :). The car has to be out in 2 years. It should be drop dead gorgeous with maybe some 600-700 horses under the hood. It should behave exceptionally well when whipped...basically an artistic + engineering master piece. Watch National Geographic for more details on how artists make this dream/goal a reality.

Coming back to software, not everybody might agree with me and not everybody can be this artist. Even in mundane tasks like maintenance the real artist will find ways to wow the audience. Anyway my real intention is to understand what is the best way to reward these artist. This concern is not related to any specific geographic location. I can think of the following ways:-

1. Pay a high salary - potential based pay if the work is not complete
   
2. Pay a decent salary but share with them profits derived from their work
   
3. Pay decent salary with recurring themes like bonus, free trips etc

Let me try to explain the context in which I made my choice. I work in a team where almost all application requirements come from our Sales team. A lot of these requirements are hard but I can see how a user will appreciate it. To achieve these requirements we made some hard choices which was initially painful but is slowly paying us dividends. The Sales teams are pinning on the success of this application because they are going to share profits. So I believe that 2 is the right way in my context. Even though its the Sales team's idea, that idea won't generate money if you stick it in the soil. Making it work was hard and still is. What are you experiences or opinion on this?.

Formula One is Back

The wait is almost over!!! The post Schumacher era of F1 racing will start in exactly 7 days from today. Will the mantle rest on the Spaniard's shoulder(Alonso), or will Kimi finally deliver on his promise?. Your thoughts.....if any :)