List of Failed cases based on their last execution

[aostad]

Just wondering how we can get the list of Failed test cases, based on their last execution result (not based on the build). For example, a test case is failed in v1.01 but passed in v1.02. If you click on "Failed Test Cases" it shows up as a failed test case in the list. This does not help much as we can get that kind of result through "Query Metrics".

That would be nice to see the list of failed cases based on their last execution, regardless of what build they have been executed in.

So, is there any way to see the list of failed test cases based on their last execution only (regardless of the build)?

[Amaradana]

Refer Test Result Metrix report, You can filter the status to pass /fai l/not run/blocked. This is available in TL 1.9

[LeeRenewData]

I think this is a bug. I filed http://mantis.testlink.org/view.php?id=5263 but it was closed as "no change required" (its supposed to work that way).
Well, 1.8.5 didn't work that way, and its pretty useless the way it is. I found what I think is the correct fix but mind you I'm on 1.9.3 so it may not be exactly the same on 1.9.4. Note that last_execution is being set to true for the Failed Test Cases report, but I think get_linked_tcversions is using the wrong option to determine to group by build_id or not (its using execution_details but I think its suppose to use last_execution). I made this change and it appears to do what I want.

On about line# 697 of lib/functions/testplan.class.php

Code: Select all

                // 2012-10-11 - Lee Drengenberg - I think this line below is wrong.  execution_details may mean
                // whether to SHOW the build or not, not whether to group by it or not
                //$groupByBuild=($my['options']['execution_details'] == 'add_build') ? ',build_id' : '';
                // I think what we really want is to based this on the last_execution option.
                // If we only want the Failed Test Case report to show tests which failed on the last_execution, then
                $groupByBuild=($my['options']['last_execution'] == true) ? '' : ',build_id';

[fman]

As we have already stated on Mantis, this is not a bug, and this is work as designed and expected.
In addition: we are free to change any behaivour that was present on previous TestLink Version as far as we provide clear
and reasonable explanation of these changes.

Then posting here saying it is a bug is IMHO not a good attitude/approach, and make as spend more time to re-explain our choices.
If you read clearly this post, you can see what the original user said.
If you read also clearly our answer on mantis you will understand that:

List of Failed cases based on their last execution IS DIFFERENT , to have a DETAIL OF FAILED TEST CASES by build.

Then if you need a feature, you are free to request it, and we are also free to consider if we will add or not it to stable code.
But when we said that we are not going to change an existent feature (that may be in previous versions work as you like, but from a conceptual point of view was WRONG, i.e. me created a feature through a bug), we do not appreciate this kind of comment on forum.

If you give a look to first answer to this post:

Refer Test Result Metrix report, You can filter the status to pass /fai l/not run/blocked. This is available in TL 1.9


You see that feature exists => we are right, this is NOT A BUG.
If you want a new option aske for it, instead of saying we are saying wrong things.

[LeeRenewData]

You've made your position clear in the bug. That's why I didn't try to re-open it or file a new bug. But our QA team needs a report that shows the details of tests that failed on the last execution. The Test Result Matrix report is VERY useful and I use it very frequently, but it doesn't show all the details of the tests which failed only on the last execution. I could filter the Test Result Matrix report to show only Failed tests (I love the new grouping and filtering btw), but that doesn't give me any of the details of the failure. For each failure I'd have to navigate to either the Failed Test Cases report or to Executions to see the details.

Also, we've agreed to provide the details on failed test cases to our business partner when we release our product and provide it to them. The Failed Test Cases report is what we used for that when we were on TL 1.8.5. So we NEEDED a report that does that same thing now. We have absolutely no use for the Failed Test Cases (or Blocked or Not Run) when they are 'per build'.

I *think* there may be other TestLink users who need the same thing we do. That's why I bothered to file the original bug and to now post my solution on the forum. I'm trying to help my fellow TestLink users.

[aostad]

I'm going to give a try to your solution. Thanks for sharing.