Open Source- We don't have to hide from it.

"By 2011, at least 80% of commercial software will contain significant amounts of open source code" Gartner Analysis.

"Open source in the application quality market is emerging, and we expects a slow but steady increase. The current tools are well-suited for technically adept agile development teams. " Thomas Murph

“Organizations are saving millions of dollars on IT by using open source software. In 2004, open source software saved large companies (with annual revenue of over $1 billion) an average of $3.3 million. Medium-sized companies (between $50 million and $1 billion in annual revenue) saved an average $1.1 million. Firms with revenues under $50 million saved an average $520,000.” Walli, S., Gynn, D., Rotz, B. V. The Growth of Open Source Software in Organizations: A Report.

Software testing community seems to be curious about Open source tools but still there is large scope to increase their use to reduction in Costs. Open source is collaboratively developed software that has no license fees. A community of professionals come together, work on common cause and produce something. The source code is made available to all, so that anyone can can make changes to suit his own needs. By doing so, the community is able to provide free solution to industry which goes evolving with contributions from experts.
People are reluctant to adapt to open source tools due to various doubts like:
What if the community stops existing or what if no support is available in future?
This is equally true with proprietary softwares. In case of open source tools, since the source code is available, it is as good as you are owning the software. Hence, even if support from community ceases, you can use your expertise to support it.
Can I depend on current version for stability?
Since large community of experts is supporting these tools, bugs are found and fixed in short time. The releases are made available more frequently hence you are with stable software in short period of time.
Are these softwares secure? Are the vulnerable to attacks?
Sometimes proprietary softwares have flaws which become evident as you start using them. But you can not do anything about them as source code is not available. With open source softwares, you can modify the code to remove these flaws.
With proprietary softwares, we sometimes end up with paying high price for which are not necessary in our context. Open source development is oriented at providing customization so that customer can modify and use the software to suit his own needs. Since the after installation support is available through open source community, you are left with costs to host software, train users and upgrade if needed.
We are evaluating various open source Test Management tools currently. We have taken into account tools like TestLink, Fitnesse, Testopia, Multi-Tester and few more. We are comparing them based on their capabilities to Capture requirements, Design Test Cases, Interfacing with different bug trackers, tracability, reports & metrices etc. During our study we have found these tools more flexible and easy to use. For example TestLink can be integrated with number of bug tracking softwares like JIRA, Mantis etc. Similaryly, you can use it with different databases like MS-SQL, MySQL and Postgres. It is hard to find such capabilities in propritery softwares.
A servey result on current open source tools can be viewed at www.opensourcetesting.org.

Test Metrics in Iterative Developement

Using analysis and measurement as drivers of the enhancement process is one major difference between iterative development and other models of development. It provides support for determining the effectiveness of the processes and the quality of product. It allows one to study, and therefore improve the processes for the particular environment. Data collected by Test Metrics in previous iteration can be used to improve future work estimates, redefine the focus areas and to improve efficiency. While the need of metrics is well recognized, their implementation is area where large variation is required according to nature of project and testing environment.
Basic Process
Appropriate process for use of test metrics is vital to achieve adequate quality. Basic steps in process of implementation of metrics are:
Identify the pain area or goals: Identification of pain area or problem area is an important step. In this step, risk analysis is done for the future iteration. The objective of Risk Analysis is to identify potential problems that could affect cost or outcome of the project.
Define metrics for each goal: Based on identified pain areas appropriate metrics should be defined. Base metrics will be used for data that can be captured directly. Base metrics like Number of Test cases executed/passed/failed/deferred etc. are used. Similar metrics for time & defects are also used. Calculated Metrics or Derived Metrics convert the base metrics data into more useful information. Test metrics are meaningful if they provide objective feedback to the project team regarding any of the development processes from analysis, to coding, to testing. Hence defining metrics those will provide objective result is important.
Next step is to setup a procedure to collect data. In this step, means of collecting data are decided. The collected data should be relevant, valid, and measurable, which can be used for analysis to derive conclusions about process effectiveness. Collecting irrelevant data leads to wastage of effort, time and team starts loosing faith in need of implementation of metrics.
Communicating metrics implementation plan to all stakeholders is very important. Unless the team members (test team as well as other entities) keep deliverable updated, data collected by metrics will not be valid. Many times, members of a project team intuitively understand that changes in the development life cycle could improve the quality and reduce the cost of a project, but they are unwilling to implement changes without objective proof of where the changes should occur. Hence it is important to spread awareness amongst stakeholders about objectives of metrics implementation.
Collecting Data & Analyzing it can be carried out using number of tools starting from Microsoft Excel to more specialized tools like McCabe QA, LDRA Test bed, Functional Point Analysis tools & Rational Suit. This stage should be carried out with minimum manual intervention as manual recording of data is subject to bias (deliberate or unconscious), error, omission, and delay. Therefore automatic data capture is desirable, and sometimes also essential. Derived/Calculated metrics are calculated in this stage.
Communicating results of metrics implementation should be communicated to all stakeholders periodically. Results should be in the format that is easily readable and consistent in its format. Analysis reports should include graphs highlighting the information based on which further decisions are expected.
Applying Results to achieve improvement should be done in diplomatic fashion. The results should not be used as measure of performance of individuals. Instead they should be used in constructive manner to achieve continuous improvement. Before taking decision, feedback should be taken from concerned members of team. In this stage it is necessary to revisit effectiveness of defined metrics. Metrics should be evaluated and restructured to get effective results in next iteration of development.
Test Metrics In Iterative Model
Implementation of test metrics in Iterative model has became comparatively difficult due to time constraints, variation in focus areas & variation in client requirements. Effective implementation of test metrics requires quick actions, accurate selection of metrics and collective commitment. Further the time and efforts spent on metrics collection should be optimum. If more time and effort is spent on this activity, it is likely that metrics will be perceived as futile activity. This generally leads in lost faith of project management and finally abandoning of metrics collection.
Test metrics are important to iterative development as they provide quick glimpse of the incremental quality. Problems surfacing in early iterations can be fixed and problems in future iterations can be guessed by applying techniques like risk analysis and past experience.
Test metrics also help in making decisions about test and development related estimation for upcoming iterations. Implementation of test metrics should not be responsibility of a small team. Instead all stakeholders should be aware of importance of implementation and need of doing that. Data collection and analysis activity should be automated to avoid biased decisions and manual mistakes.
Conclusions
Test Metrics implementation in Iterative development should be based on following:
1.Invest less time and efforts for metrics implementation by measures.
2.Automate data collection and analysis so as to avoid manual mistakes and biased decisions.
3.Accurate selection of metrics and data point to avoid collection of data that is of no use.
4.As requirements change frequently in iterative development and new risks are identified, introduce new metrics or reform the existing metrics whenever required.
5.Quick decisions and their implementation in order to get proper results.
6.Total involvement of project management and other stakeholders.
7.Timely and frequent communication with stakeholders.
8.Avoid using metrics for evaluation of individual performance.
9.Challenges are never ending but measurements and preventive actions will always help getting proper results.
10.Do not make metrics collection as responsibility of a single individual or small team. Involve all stake holders so that data collection is effective and efficient.
This post is based on my paper 'Test Metrics in Iterative Development' published in STC2008. Copyright of this paper are with QAI and use for profit is not allowed.

Challenges in having Dispersed Testing Teams

In current business scenario, teams are frequently located at different locations separated by miles. With profits earned from Outsourcing, this trend is bound to increase logarithmically with time. Managing teams in this situation where face to face communication is rarest possibility, is a difficult task. Frequent misunderstandings and communication gap, if not taken care of, lead to increased difficulties for managers. We are currently working in similar situation. Although our teams are located in same time zone, still we are facing challenges in executing projects with the other team. We have development team and part of testing team located at one location and other testing team working at second location. Challenges faced in this situation can be classified as those related to Operational, Technical and Human factors.
Operational issues like Network connectivity affect project execution seriously if project repositories at both locations are not synchronized. Repository synchronization should be automated so that updating one repository will lead to immediate updating at other location. Also if test environments are not maintained separately at these locations, connectivity failure leads to complete breakdown of testing activity at the remote location. Alternative connectivity solutions should be made available to minimize the effect.
All stakeholder should be made aware of importance of reporting so that status of work carried out by one team will be always know to other team. This problem can be eliminated by using test management tools for carrying out test activities. These tools can be shared by both teams. Since most tools are web based, managers are able to get reports from anywhere using internet. Open source tools like TestLink, Fitnesse etc. can be used for this purpose. Vital and updated information should be made available to new joiners so that learning period can be minimized. Customized Handover templates/ checklists can be utilized for this purpose.
Since members of both team do not get chance to meet face to face, there is always a feeling of separation and mistrust. In order to minimize this, periodic meetings and video conferences should be conducted. Messenger used for communication between teams should support conferencing and should be able to maintain history so that information exchanged during conferences can be easily tracked. Initiatives like common picnics and peer exchange may help in improving trust between teams. Role of managers and leads becomes vital to maintain co-ordination between teams. Steps should be taken to increase sense of belonging and responsibility in both teams.
This blog is featured on http://www.testrepublic.com/