TestLink

I had to add a new test-result status, called "Conditional Pass" in the list of possible test results (Pass, Fail, Not Run, Blocked). We assigne the status of a test case to "Conditional Pass", when it passes or fails under certain condition. So it's hard to say the test either 100% Fails or Pass.

I know modifying code locally will be hard to maintain later, especially when it comes to upgrade to a new version of TestLink. But to have that option is very imporant for the company that I'm working for. So I had to add the feature and take the responsibility of maintaing the code in the future.

Let me know if you are interested so I can share it with you and show you how to maintain the code even after upgrading to a new version of TestLink.

what version are you using? Check in the manual installation whether this was given or not?

I'm using 1.8.1.
I could not find anything about adding/configuring a new test result other than Pass,Fail, and Blocked.

Adding a new result status is very easy and can be localized to your custom config file. There was no need for code changes.

Well, it's not that much easy. custom_conf.inc file is one of the files that need to be modified. You need to make it work for reporting, chart, metric, background/forground color, and defining some new definition in string file too.

If somebody start doing that she/he will notice there are more than just a config file that need to be modified.

Anyway, it was just a kind suggestion.

Cheers,
Ali

Yes, you're right. You do need to change the colour in the style sheet and add the displayable text in the strings file.

I've added a new status that works for reports, charts, etc and did not need to modify any other code but the custom config. Here is what I added for a new status called ready_to_run

$tlCfg->results['status_code'] = array (
"failed" => 'f',
"blocked" => 'b',
"passed" => 'p',
"not_run" => 'n',
"not_available" => 'x',
"unknown" => 'u',
"ready_to_run" => 'k',
"all" => 'a'
);

$tlCfg->results['status_label'] = array(
"not_run" => "test_status_not_run",
"passed" => "test_status_passed",
"failed" => "test_status_failed",
"blocked" => "test_status_blocked",
// "all" => "test_status_all_status",
// "not_available" => "test_status_not_available",
"ready_to_run" => "test_status_ready_to_run"
// "unknown" => "test_status_unknown"
);
//
$tlCfg->results['status_label_for_exec_ui'] = array(
"not_run" => "test_status_not_run",
"passed" => "test_status_passed",
"failed" => "test_status_failed",
"blocked" => "test_status_blocked",
"ready_to_run" => "test_status_ready_to_run"
);
$tlCfg->results['charts']['status_colour']=array(
"not_run" => "000000",
"passed" => "00FF00",
"failed" => "FF0000",
"blocked" => "0000FF",
"ready_to_run" => "C08040"
);
$tlCfg->reportsCfg->exec_status = $tlCfg->results['status_label_for_exec_ui'];
/** Revered list of Test Case execution results */
$tlCfg->results['code_status'] = array_flip($tlCfg->results['status_code']);

I'm sorry you do not find easy to make a couple of changes in two files.
may be you can develop a better solution, or write down detailed instructions that can help other users

I also would like to add new status to the test cases and test results/reports.

Did the option of adding to custom_config worked for anyone?

When I made changes to custom_config.inc file,I am seeing some errors .

Is this post is not for adding a custom status to execution and results? PLease follow the above given and change accordingly. It worked for me.

I'm not sure what kind of error you get, but here is what I've done for adding a new test-status called "Conditional Pass" to execution, report, metrics, and charts. You can get the idea and use for adding your desired test-status.


NOTE: when you make any change in the code you have to maintain the code and merge it after updating to a new release of testlink. I will explain it at the end.

So, here is the procedure:
Assumption:
-testlink is installed in "testlink" directory.
-The new test status is called "Conditional Pass".
-Current version of testlink is 1.8.1

1. First make a backup of files that you are going to change. Be careful! Do not change the type of file when you make a backup of them. In Windows, I do copy/paste the file in the same directory. So Windows will rename it to "filename - Copy" by default.

Locate following files and make a backup of them:
custom_config.inc.php,
strings, //The file that is for your language
const.inc.php,
reports.cfg.php,
resultsByStatus.php,
testlink.php,

2. Apply following changes in "custom_config.inc" file:
find " $tlCfg->results['status_code'] = array " and insert the new status like this:
$tlCfg->results['status_code'] = array (

"failed" => 'f',

"conditional_pass" => 'q',

"blocked" => 'b',

"passed" => 'p',

"not_run" => 'n',

// "not_available" => 'x',

// "unknown" => 'u',

// "all" => 'a'
);


Locate " $tlCfg->results['status_label'] = array " and insert the new status to the code:
$tlCfg->results['status_label'] = array(

"passed" => "test_status_passed",

"conditional_pass" => "test_status_conditional_pass",

"failed" => "test_status_failed",

"blocked" => "test_status_blocked",

"not_run" => "test_status_not_run",

//"not_available" => "test_status_not_available",

//"all" => "test_status_all_status",

//"unknown" => "test_status_unknown"
);

Locate " $tlCfg->results['status_label_for_exec_ui'] = array " and apply following change again:
$tlCfg->results['status_label_for_exec_ui'] = array(

"passed" => "test_status_passed",

"conditional_pass" => "test_status_conditional_pass",

"failed" => "test_status_failed",

"blocked" => "test_status_blocked",

"not_run" => "test_status_not_run",

// "not_available" => "test_status_not_available"
);


3. Add following lines to "Strings" file. If you're using English version you need to modify \testlink\locale\en_GB\Strings file:

$TLS_test_status_conditional_pass = "Cond. Pass";
$TLS_trep_conditional_pass = $TLS_test_status_conditional_pass;
$TLS_last_status_conditional_pass = $TLS_test_status_conditional_pass;
$TLS_th_total_conditional_pass = "#" . $TLS_test_status_conditional_pass;
$TLS_req_title_conditional_pass = "Conditionall_Pass Requirements";
$TLS_number_conditional_pass = "# " . $TLS_test_status_conditional_pass;
$TLS_link_report_conditional_pass_tcs = "Conditional Pass Test Cases";



4. Open \testlink\cfg\const.inc.php file and look for this line:
$tlCfg->results['status_code'] = array (

then add following line in the body of function:
"conditional_pass" => 'q',

locate this line:
$tlCfg->results['status_label'] = array(

then add following line to the body of function:
"conditional_pass" => "test_status_conditional_pass",


locate this line:
$tlCfg->results['status_label_for_exec_ui'] = array(

then add following line to the function:
"conditional_pass" => "test_status_conditional_pass"


Locate this line:
$tlCfg->results['charts']['status_colour']=array(

then add following line to the function:
"conditional_pass" => "808000",



5. Open \testlink\cfg\reports.cfg.php file and add following to it:
$tlCfg->reports_list['list_tc_conditional_pass'] = array(
'title' => 'link_report_conditional_pass_tcs',
'url' => 'lib/results/resultsByStatus.php?type=' . $tlCfg->results['status_code']['conditional_pass'],

'enabled' => 'all',

'format' => 'format_html,format_ods,format_xls,format_mail_html'
);



6. Open \testlink\lib\result\resultsByStatus.php file in an editor and look for following line:
foreach( array('failed','blocked','not_run') as $verbose_status )

Then change it to this:
foreach( array('conditional_pass','failed','blocked','not_run') as $verbose_status )


7. Open \testlink\gui\themes\default\css\testlink.php file in an editor and apply following changes:
In " Execution & Results " section add following:
div.conditional_pass {
background: #808000; /* hanah */
}


In " execution tree coloring " section look for:
span.light_passed, span.light_failed, span.light_blocked,
span.light_not_run, span.light_not_available, span.light_unknown
{


and change it to:
span.light_passed, span.light_failed, span.light_blocked, span.light_not_run, span.light_not_available, span.light_unknown, span.light_conditional_pass{

After above change add following block below it:
span.light_conditional_pass {
background: #FFEECC;
}


In " render test result status " section add following lines:
.conditional_pass {

color: white;

background: #808000; /* hanah */

}


By now, if you have not missed any characters (, ; .) during cut/paste, you should have your new added test-status in execution, reports, metrics, and charts.


How to maintain the changes in the future releases?
You need a tool that can compare two file and merge them. You can download and install "winmerge", which is free.
Using the tool compare the files that you have changed with the same files in the new release of test link and apply/merge changes one by one. Do not forget to make a backup of all original file before proceeding.

Cheers,
Ali

Thanks Ali.

I could add the extra status to testlink but it shows up as
"LOCALIZE: test_status_conditional pass" in all places I added.


I tried removing "test_status_" text from custom_config.inc file then the new status shows up as "LOCALIZE: conditional pass" not as "conditional pass".

Any idea as how to get rid of extra text in the status?

Please double check the "Strings" file and make sure you have defined this:
$TLS_test_status_conditional_pass = "Cond. Pass";

Don't forget the ";" at the end.

Hi Ali,

I checked the strings file and the semicolon exists at the end.

I modified the string file under
.....\testlink\locale\en_US\strings.txt.

Line from the file:
$TLS_test_status_exemption = "Exemption";

When you get "LOCALIZED:..." that means the variable is not defind for the program.

I can see you missed "_" in "test_status_conditional pass"
Look for it and changed it to "test_status_conditional_pass"

If you want to name the new test-status be called "Exemption" instead of conditional_pass, replace "conditional_pass" with "Exemption" in all places. In that case, you should not have "conditional_pass" in any file.

thank you so much for this... It was exactly what I needed. I had one question. Is there a way to make the color of the "Exception" Status Text change on the Test Result Matrix? Currently it is black, but I would like to make it orange.