Page 1 of 1

Incorrect Execution Results reported in Testlink from Jenkin

PostPosted: Mon Jul 27, 2015 8:16 am
by vossqa
Good day

Is there perhaps a known issue with incorrect results passed from Jenkins to Testlink.
I have a configured the Testlink plugin on Jenkins. After I run my TestNG test cases, I proceed to update the results in Testlink.
The test cases are displayed correctly, but all of them are updated as "Passed"

TestLink build ID: 282

TestLink build name: build_2015-07-27_09-40-21

Total of 4 tests. Where 4 (+4) passed, 0 failed, 0 were blocked and 0 were not executed.

List of test cases and execution result status

Test case ID Test case external ID Version Name Test project ID Execution status
87555 SOL-10503 1 Add Provider Credential Policy 67109 Passed
87557 SOL-10504 1 Assign Prov Credential Policy to Provider Admin 67109 Passed
87559 SOL-10505 1 Add Customer Credential Policy 67109 Passed
87561 SOL-10506 1 Assign Cust Credential Policy to Customer Admin 67109 Passed


In my test cases TestNG result, 2 of the test cases have failed.

TestNG Results
Total Tests: 4 (+4)
Failed Tests: 2 (+2)
com.company.abc.def.auto.applicationtest.test.GenericTest.testAdd
com.company.abc.def.auto.applicationtest.test.GenericTest.testAdd
Skipped Tests: 0 (±0)
Failed Configurations: 0 (±0)
Skipped Configurations: 0 (±0)

I have a suspicion it might be related to my Test result seeking strategy definition :
I use :
TestNG class name
Include Pattern -> output/testng-results.xml
Key Custom Field -> TL-Jenkins

I will try to attach my testng-results and testlink testcase export file to this forum post

I am using : TestLink 1.9.13 (Stormbringer)
Jenkins ver. 1.585
Testlink Plugin 3.10

Re: Incorrect Execution Results reported in Testlink from Je

PostPosted: Mon Jul 27, 2015 8:45 am
by vossqa
I am battling to attach the results file, is there perhaps another way to supply the logs?

Re: Incorrect Execution Results reported in Testlink from Je

PostPosted: Mon Jul 27, 2015 9:17 am
by devonk
You could post the xml on Pastebin or try adding it by creating an issue on the TestLink Mantis Issue Tracker.

Re: Incorrect Execution Results reported in Testlink from Je

PostPosted: Mon Jul 27, 2015 3:17 pm
by vossqa
Thank you Devon

I have attached the log via Mantis :

http://mantis.testlink.org/view.php?id=7204

Re: Incorrect Execution Results reported in Testlink from Je

PostPosted: Thu Jul 30, 2015 5:21 am
by vossqa
I have logged the following issue on Jenkins as per kinow's request :

https://issues.jenkins-ci.org/browse/JENKINS-29716