Programiści .NET od zamierzchłych czasów używali podmiany wersji w pliku AssemblyInfo.cs do uaktualnienia wersji tworzonej aplikacji. To było oczywiste jak stawianie średnika na końcu linii. Jednak czasy się zmieniają, mamy nowe CLI i warto wiedzieć, że można łatwo zmienić numer wersji aplikacji bez dodatkowych narzędzi ani skryptów.
Czasem robiąc code review rodzą się wątpliwości, czy dany kawałek kodu w ogóle działa albo czy przypadkiem nie psuje reszty systemu. Najlepiej w takiej sytuacji przed zamknięciem pull requesta po prostu to przetestować, ale do tego trzeba ściągnąć najnowszą wersję kodu, zbudować i uruchomić projekt a wcześniej odłożyć swoje zmiany na bok. Sam wiem po sobie, że często z tego rezygnowałem. Można jednak rozszerzyć swój proces CI/CD aby build server sam robił deploy każdego pull request’u na odrębne środowisko.
Wiele razy słyszałem jak ludzie mylą Continuous Delivery z Continuous Deployment dlatego zdecydowałem się tutaj pokazać różnice, zaczynając przy okazji od Continuous Integration. Bez obaw, postaram się unikać słownikowych definicji.
Wyobraźmy sobie, że chcemy ocenić poziom zaawansowania procesu wytwarzania oprogramowania jaki obowiązuje w naszym projekcie. Możemy tu wydzielić następujące etapy:
Did you see source code with VS scrollbar full of different colors and warning sign in every file? For a long time that was my case. It did not help that we agreed on some coding guidelines within our team. If some rules are not enforced, then code issues will get into your source code – doesn’t matter if it’s rush, manual refactoring, merging conflicts or less careful teammates. To avoid that the best way is to make code inspection part of your bulid pipeline.
We live in the time of TDD and unit testing. We already know that it’s one of the best practices so we do it in hope that it solves most of our problems. But is unit testing really enough? If you feel it’s not, and have heard of other types of test automation, but you’re not sure if they are for you, let us think it over together.
What is test automation
The most common argument for test automation is that it reduces the cost of testing. It’s how people perceive this kind of practice. And indeed, hours of our manual clicking through the system (especially if you do not have dedicated tester) costs a lot, and in the same time you could do something considerably more valuable. Test automation gives you faster feedback after finishing some feature – automated tests should run in a few minutes and tell if it’s done or by doing one improvement you broke three other parts of the system.
As long this is pretty common understanding, it’s sometimes not enough for people who are not struggling with such problems because they are not doing regression at all (very common problem which is even worse than long and expensive manual regression).
Among this we forget about few more important arguments for test automation, especially from developer’s perspective.
Most important benefits
1. Test automation enables high level refactoring – the most basic rule behind TDD is: red, green, refactor. And you might have felt that because of unit testing you were able to refactor the code. But unit testing is about a good design of project, not about the best refactoring. Low level testing allows low level refactoring and high level testing allows high level and exciting refactoring.
My teammate and I have a list of expensive refactorings that we want to do if we have time – we call it “wishlist”. Several items from the top: upgrade NHibernate, upgrade jquery, upgrade jquery ui, css cleanup, clean up db objects, remove legacy ORM etc.
Does unit testing allow us to realize all of these wishes? Of course not. But thank to UI testing we can refactor the UI part – jQuery, CSS etc., due to acceptance tests we can fix business logic, due to integration tests – upgrade external libraries and so on. This is the most exciting ability that test automation gives to developers.
2. Test automation enables continuous delivery – continuous delivery is amazing because we can get feedback from the user in a very short time. It means that the team needs to deliver frequently and it’s obvious that every deploy to production needs to be preceded by a lot of testing. Without test automation it’s even impossible to proceed with such a practice.
3. Test automation stabilize regression – the first question is – do you have a proper regression scenario? I’d bet in most cases it’s like that: for the next “X” amount of time everyone is testing the system. Do we know what to test? Are we sure that we do the same regression as for the last time? No, because manual work – deployment or testing – is error-prone. We do not like going step-by-step through the manual. In contrary – computer likes it. What is possible to automate should be automated.
As you can see there are many pros to test automation. Why isn’t it as common as unit testing? The answer is simple – it’s hard to work out proper test automation solution, and in opposite it’s considerably easy to set up very bad and impossible to maintain test automation that discourages managers from further investment in such practice. But of course it’s worth trying.
In future I’m planning to write more about rules for writing good and stable automated tests.
This quiz is to let you evaluate your team/project in terms of good practices. Below you will find a list of different development practices/techniques. Mark the practices you use in your project and sum the points assigned to each technique. Then you can place yourself on the right place on the awesomeness scale. Be frank with yourself – don’t mark “UI testing” if you have one UI test that produces unknown result or “Friday (20% of time)” if you once stayed longer at work to do some refactoring.
|Source Code Versioning||Using SVN, Git, etc. (instead of shared disk space)||2|
|Using build server||Separated machine compiling the solution, running tests and publishing the results||2|
|Automatic versioning||Assigning application version by buildserver||1|
|Clean build||No warnings while building the solution||1|
|Code quality rules||Having and following a list of code quality rules checked by buildserver. Build fails if not passed||1|
|Code quality metrics||Measuring “code coverage”, number of lines of code, number of unit tests, cyclomatic complexity||1|
|Separated tester role||Having a tester whose only responsibility is testing the quality, doing regression tests etc||1|
|“One click” deployment||Using WebDeploy or any other script that lets to deploy the software in less than few clicks||1|
|Continuous delivery||Continuous integration + automated testing + continuous deployment||2|
|Database upgrade script||Database migration is done automatically as part of deployment||1|
|Database refactoring||Database is refactored, there is no left stored procedure/table, no “obsolete” fields||1|
|Pair programming||Two developers + one computer||1|
|Code review||Each commit/functionality is reviewed by another developer||2|
|External code audits||Code review done by developers from the outside of the team||1|
|Friday (20% of time)||Every month/week/”on Friday” stop doing productive staff and hold on to refactor something||1|
|Maintain central documentation, wiki||Describe important information/workflows in one place and keep updating||1|
|Work agile||Contact with client, work in iterations (not only Scrum)||1|
|Unit testing||Non-production code testing small parts of production code||2|
|TDD||Test-driven development: write test, write code, refactor||1|
|ATDD||Specify requirements as automated tests||1|
|BDD||Test whole behaviour not just parts of code||1|
|Discuss BDD scenarios with customer||Customer utilise BDD scenarios, they can be written with specific language (i.e. Gherkin)||1|
|Automated UI testing||Automated tests of scenarios using user interface, using tools like Selenium, WatiN/WatiR||1|
|Performance testing||Load tests, stress tests, analysing reports||1|
|Integration tests||Automated testing of integration with other systems (i.e. exposed service endpoints)||1|
1. [0 – 5 points] Ok, this is probably one-man academic project or a simple website. But at least you have found this list of ideas. Each month peek one and try to experiment. The benefits are invaluable! The great pleasure of seeing the build server full of green signs of passed tests… Can you imagine the feeling of safety when refactoring your code or UI? The instant feedback from users and the awesome feeling that you and your project are awesome and can be mentioned as an example to follow. This all is a bread and butter for mature teams.
2. [6 – 11 points] Is it that you are willing, but you don’t have time? No one does. Is your project too small? Notepad.exe has more that 200 UI tests. Think, that the next person that will take over your project in the next 6 months is a serial killer that knows your address.
3. [12 – 18 points] This is great that you came to this point. You passed the test. But be aware of the fact how many things you can improve. Remember that there are better projects, easier to maintain and refactor.
4. [19 – 25 points] Excellent work. This level is sometimes the maximum for some projects. Your team/project rules. People can actually mimic your way of working, I’m not telling that you must go now ask for a rise, but of course you can try :]
5. [25 – 31 points] You are simply awesome. You’ve reached the top. You are like Chuck Norris in development Texas and you turned “GodMode” in your project. Keep it up. PS: Do you still remember about earning money?
This was just my proposal of practices used by developers. It’d be fabulous if you add your ideas in the comment. I promise to add best ideas to the list!
You can also share your score if you want.