Performance issue when there are many sub-suites

[gzapryanov]

Hello,

I am using TestLink 1.8.2 and I have created around 6000 test cases organized in several layers of sub-suites like:
Project
|-> sub-project
|->Functional area
|->Story ID

The problem is that as the project becomes bigger and bigger we are creating a lot of sub-suites of type functional area and Story ID. So at some point loading the tree took more than 30sec which is the default PHP execution timeout (I am using JSTREE).

I made some research and it seems that most of the time is spent on the function "count_testcases" which invokes "get_all_testcases_id" which invokes recursively SQL queries to count all the test cases in all the sub-suites. The goal of this counting is to display the number of test cases for each sub-suite in the tree next to the node name. I was looking into the DB schema and couldn't figure out better approach for counting test cases from particular node ID.

Since 30 sec is not acceptable time for loading the tree I have commented out TC counting and now it loads 6000 test cases for 2-3 sec but I liked the fact that I knew exactly how many test cases there were in each sub-suite.

Any ideas how to improve counting of test cases from a given node ID and to avoid recursive invocations of get_all_testcases_id?

[gzapryanov]

I made the change in the code. Now:

- When I load the test case tree there is no counting and only the names of the sub-suites are displayed. The tree with more than 6000 test cases and many sub-suites is loaded for 2 sec
- If you want to get the number of test cases in particular sub-suite then you have to click on the sub-suite name and it is displayed before details section and under test suite name
- I have also removed count_testcases from the Home page. Previously prior to displaying TestLink index page, there was counting procedure to check if there are any test cases in the project, this cause huge latency in loading the home page of a project with many test case. Now the index is displayed for 2 sec no matter how huge is the project

If anyone is interested in these improvements, I can supply some sort of a patch.

[fman]

1. are you using EXT-JS tree ? because with this you can have lazy loading => improve performance at least for test spec tree, while with other tree components this can not be done -> on 1.9 at up ONLY EXT-JS tree component is available.
please open an issue on Mantis with CONTRIBUTION on title and post your code (no patches but the whole files you have changed).

[cookie]

where can i find that piece of code, we are trying to use testlink with huge test case number 50 000 we are having big performance issue.

thx for any tip we are using 1.9 testlink

[fman]

Can you send an export of your DB zipped to allow us to do some performance tests ?

francisco.mancardi@gmail.com

[cookie]

Can you send an export of your DB zipped to allow us to do some performance tests ?

francisco.mancardi@gmail.com

sorry i can not this is company policy. You should be able to generate that many test cases using script btw.
do you know where can i find that code from gzapryanov, should this improve performance of 1.9 tree node menu?

very good work by the way we very like testlink 1.9.

thx

[fman]

I'm sorry very much but without a sample DB I will can do nothing.

[gzapryanov]

Guys, appologies that I don't have the time to make this the proper way and I will try to do it by the end of the week. For now:

* Cookie, please consider the fact that the big number of test cases is not an issue by itself. The real problem comes when you have too many (cannot define too many) sub-suites/folders in the tree. Or when you have multiple levels of sub-suites.
* Performance degradation occurs because TestLink tries to count all the test cases from particular node. Lazy loading doesn't help in this case because you still have to count the test cases in all the sub-suites and this is done in recursive manner. This root cause is the DB schema - every test case, folder (suite) or version is presented as a node and every node has a parent node. So in order to count all the test cases from hundreds of suites one function is invoked to count the children of the root node, then the chidren of the children until it reaches to the leaves of the tree. Then it makes 1 request which counts all the nodes (test cases) which have parent id like the id of the all suites/folders.

The change is nothing special but just commenting out the procedures for countiung test cases (of course you will no be able to see how many test cases there are in a folder which is diplayed next to the folder name). Here are the files and the lins for TestLink 1.8.2:

----------------------------------------------------------
lib/ajax/gettprojectnodes.php
----------------------------------------------------------
// $tcase_qty = 0;
//$tproject_mgr->count_testcases($row['id']);


/*Changed by Georgi Zapryanov
if(!is_null($tcase_qty))
{
$path['text'] .= "({$tcase_qty})";
}*/

----------------------------------------------------------
lib/testcases/listTestCases.php
----------------------------------------------------------
//Changed by Georgi Zapryanov
//$tprojectMgr->count_testcases($argsObj->tproject_id);
$gui->ajaxTree->root_node->name = $argsObj->tproject_name;
// Change by Georgi Zapryanov . " ($tcase_qty)";
----------------------------------------------------------
lib/general/mainPage.php
----------------------------------------------------------
//Commented out by Georgi Zapryanov
//$tproject_mgr->count_testcases($testprojectID) > 0 ? 1 : 0;
----------------------------------------------------------

Again, this is not the proper way of making changes and I will submit it in the right way once I have some time for this.

[fman]

>> This root cause is the DB schema - every test case, folder
We will accept any suggestion on what type of DB schema create that have good performance when having to return number of children on each level of tree.

[cookie]

My issue is explained better here on this thread:
http://www.teamst.org/forum/viewtopic.php?f=11&t=3722

Thank you for that code.

cheers