TestLink

Hi

I've been using TestLink for a few years now, it was my choice of test manager when I started up a test group for a company who had no previous systematic testing experience. The way we organise our Software products at Pico Technology has caused me some problems with organising our tests in TestLink.

We currently have 2 products in active development:

• PicoScope - our main product, along with PicoScope Automotive which is the same product tweaked for the Automotive industry
• PicoDiagnostic - used for electrical fault finding on vehicles, more aimed at mechanics rather than electrical/electronic engineers as the first product is

So far, so easy. My intial approach was to keep these 2 products as 2 seperate TestLink Projects and that works well for the most part.

The problem comes from the fact that our two main products are sold both directly as Pico products and as rebranded OEM products to various companies all around the world. Each OEM product is largely the same as the 2 main products but with a different set of supported features, language sets, hardware devices and in some cases menu structures etc.

The way I have been dealing with this so far is to have all the tests for the seperate PicoScope and PicoDiagnostics products in their own TestLink projects and to try to select the right subset of requirement and test cases for each OEM test plan, through a cunning use of keywords and I've managed to keep on top of this about as well as can be expected but it takes up alot of time to administer it.

The main problems I have with this approach are:

• Requirements reports don't work since each OEM only uses a small subset of the requirements.
• Test Plans list becomes very long, with ~15 OEM each with 2-3 releases a year, after 12 months I've got a list of 30-45 test plans in various states of use.
• some OEMs use PicoScope and PicoDiagnostics as a sort of suite of software requiring the reports to be drawn from 2 seperate projects which have different version numbers making it hard to keep track of and with no good way to link 2 seperate test plans into a single release of software.

I have considered a second approach to laying out the tests, rather than a project per software product, have a project per OEM product. This would allow the requirement for that OEM and only that OEM to be stored together. Test plan lists would only be adding 2-3 new test plans per year which would be very easy to manage. The 2 seperate software products can be handled as test platforms.

I see only 1 flaw with the OEM project layout, although the exact subset of test requirements and cases to be used for each OEM differs, they are for the most part identical test cases. I would have to copy and paste test requirments and cases between each of the seperate OEM projects, make sure that changes I make in the test cases in the main version of each software product gets copied into all the other OEM Projects at the appropriate time, the different OEMs tend to lag a anything between 6 months to a couple of years behind the standard version of the software so it's not as simple as copy and update the test plans as soon as I make a change to a test case version. I've been trying to get the OEMs to be used closer to the standard version of the software but the customers themselves are reluctant to do this.

Is there anything I've missed in TestLink which would work in this situation? Test Cases shared and synced between Projects for example?

How do you handle multiple releases of software suites which is similar but not identical to each other?

To some extent, I suspect the crazy matrix of Software (OS platforms, languages, customisation) and Hardware we have is never going to sit very well in any test management software and I'm just going to have to accept that more of my time if going to be taken on the administration of the tests rather than the running of them.

Thanks
Trevor

P.S. Sorry for the insanely long post.