Incorrect Execution Results reported in Testlink from Jenkin

Discussion and information for XML-RPC interface.

Moderators: Amaradana, TurboPT, TL Developers

Post Reply
vossqa
TestLink user
Posts: 9
Joined: Sat Aug 02, 2014 4:32 pm

Incorrect Execution Results reported in Testlink from Jenkin

Post by vossqa »

Good day

Is there perhaps a known issue with incorrect results passed from Jenkins to Testlink.
I have a configured the Testlink plugin on Jenkins. After I run my TestNG test cases, I proceed to update the results in Testlink.
The test cases are displayed correctly, but all of them are updated as "Passed"

TestLink build ID: 282

TestLink build name: build_2015-07-27_09-40-21

Total of 4 tests. Where 4 (+4) passed, 0 failed, 0 were blocked and 0 were not executed.

List of test cases and execution result status

Test case ID Test case external ID Version Name Test project ID Execution status
87555 SOL-10503 1 Add Provider Credential Policy 67109 Passed
87557 SOL-10504 1 Assign Prov Credential Policy to Provider Admin 67109 Passed
87559 SOL-10505 1 Add Customer Credential Policy 67109 Passed
87561 SOL-10506 1 Assign Cust Credential Policy to Customer Admin 67109 Passed


In my test cases TestNG result, 2 of the test cases have failed.

TestNG Results
Total Tests: 4 (+4)
Failed Tests: 2 (+2)
com.company.abc.def.auto.applicationtest.test.GenericTest.testAdd
com.company.abc.def.auto.applicationtest.test.GenericTest.testAdd
Skipped Tests: 0 (±0)
Failed Configurations: 0 (±0)
Skipped Configurations: 0 (±0)

I have a suspicion it might be related to my Test result seeking strategy definition :
I use :
TestNG class name
Include Pattern -> output/testng-results.xml
Key Custom Field -> TL-Jenkins

I will try to attach my testng-results and testlink testcase export file to this forum post

I am using : TestLink 1.9.13 (Stormbringer)
Jenkins ver. 1.585
Testlink Plugin 3.10
vossqa
TestLink user
Posts: 9
Joined: Sat Aug 02, 2014 4:32 pm

Re: Incorrect Execution Results reported in Testlink from Je

Post by vossqa »

I am battling to attach the results file, is there perhaps another way to supply the logs?
devonk
Advanced user
Posts: 16
Joined: Sun Mar 15, 2015 9:43 am

Re: Incorrect Execution Results reported in Testlink from Je

Post by devonk »

You could post the xml on Pastebin or try adding it by creating an issue on the TestLink Mantis Issue Tracker.
vossqa
TestLink user
Posts: 9
Joined: Sat Aug 02, 2014 4:32 pm

Re: Incorrect Execution Results reported in Testlink from Je

Post by vossqa »

Thank you Devon

I have attached the log via Mantis :

http://mantis.testlink.org/view.php?id=7204
vossqa
TestLink user
Posts: 9
Joined: Sat Aug 02, 2014 4:32 pm

Re: Incorrect Execution Results reported in Testlink from Je

Post by vossqa »

I have logged the following issue on Jenkins as per kinow's request :

https://issues.jenkins-ci.org/browse/JENKINS-29716
Post Reply