Continue deployment is very popular around world at the moment. It has been seen as one corner stone of agile practices. But I do believe it takes very long time for organization to be able deploy often.
My view about continue deployment is that you should NOT do continuous deployment unless you have a safe net.
Safe net means differently from business to business. But there are several things that are essential across different business types.
Monitoring becomes more and more important for IT businesses. Without any data supporting your theory, you may make wrong decision. The decision may cost your company as highly competition from anywhere around the world with low entry requirement for an IT business.
There are tons of article talking about monitoring. But I do believe that every business should encourage monitoring across all layers:
- Operation/System Services including databases
- Application Layer error
- User Interactions
Everything has its important in helping us making right decision and fix problems as quick as possible. I believe that nothing can be 100% right unless we put them in production. With fast growing amount of data, it is not possible for teams to test most systems with scale which is same as production. Expecting fail and have a safe net to catch failure as early as possible and design a system to make problem to be fixed as soon as possible.
If you don't have monitoring in place, do NOT do continuous deployment.
Environments always should be first problem you solve if you want team to be agile. I have been experience in lots of companies that they are experience as following order:
1. Use product as development sandbox
2. Have staging environment and staging environment db box is used for development
3. Have continuous build and automatic acceptance testing environments
4. Then have dedicated dev, testing and staging environment
5. Then have dedicated preview sites
6. Have gateway function to select users for preview features (Not done in any company yet, only read in paper now)
Without achieving (4), there are not enough testing before going to production. In that case, please do NOT use continuous deployment unless your system in VERY small. If you have done continuous deployment without reach (4), please let me know.I would love to know that how can you minimize risk while not having such environments.
Testing and Quality Control
Unit testing, functional testing/integration testing and acceptance tests are essential for your continuous deployment.
Some people out there are fans of unit testing or functional testing. I don't believe one type of test is enough for continuous deployment.
Functional testing and Acceptance Tests will tell us whether system did what we expect. As every software engineer knows, there are many cases that you can't put into functional test cases. It tests from user point of view. But it never tests from software engineer point of view. There are lots of place that we can miss with functional testing. I have see that some one promote functional testing and they promote technical stories in the same time. It is kind of making sense because they want to cover in more granular level.
Unit testing is essential as most agree. It give us confident in a granular level so that we will be very confident as software engineer. And another advantage is that we have static about how much coverage of testing is at the moment. It is very important thing for me. Someone says 80% is ideal line that you can have enough confidence. I would say sometimes 70% maybe enough too. And progressively working towards 80% is best way to do. But you will find that after 65%, it normally becomes very hard to do. And it is worth to write test coverage as requirement in stories from that time.
With unit testing only, we can assure that system does what user or product team wants. And we haven't tested in an integrated environment yet. With those reasons, functional and acceptance tests are very important too.
I have not done any feature flipping in any organization yet. But I believe that feature flipping is important for continuous deployment. There are frequent requests about hide certain features for different kinds of reasons. Normally reaction at the moment, we just delay deployments. But it will prevent other important features to be released too. When team grows, the problems will occur more and more. And I am pushing feature flipping at the moment. But it may result in messy code and it needs frequent clean up are always concerns.