Using the API to update test execution results from NUnit

[mkbutler]

New user to TestLink doing some research.

Can someone in the know let me know if this is possible?

From what I can tell I can programmatically update test execution results for a test case via the TestLink API in this latest release. By that I mean:
- Create a test case in TestLink and get its ID.
- I can then create a test in NUnit and Selenium and put the ID of a TestLink test case in that test.
- I can then write some C# code and use the TestLink API and have the NUnit test update the test execution results of that Selenium test into TestLink.

So that I can open up TestLink and see a report with the combined latest results of all manual and automated test executions.

In my dream, I can see a report where I can trace from a test exectuion result to the test case itself and then to the requirement that the test case is validating.

Thanks,

MArk B.

[rinmar]

Hello Mark,

i search a solution like this.
We have a GUI autom. Testtool which should dump the
results in TestLink and created own executions there with the status.
I will try the API Java Client for first Tests.

The difference between autom. and manual Testruns we could handle
over Keywords.

Would be great to get some news if you found a solution.

Thanks
Marko

[mkbutler]

bump

Looks like it's you and me for now Marko. Not sure when I can get working on it though. I've got a drop coming soon and that will keep me busy for several days.

[rinmar]

Hi Mark,

we solved it with Java Client API.
We changed some code, but now it works well.

Our Testautom. is running and create result.
The Java Code create executions in TestLink based on TestcaseID.

two things todo
1. Testautom. has to set some variables & notes for testlink
2. Testautom. has to start java api after run

Next issues:
3. Automatic start of reporting in nightly build ( i will create own topic for this)

[mkbutler]

rinmar,

Congratulations on success. Could you post your code or an example? I'm going to be doing this in C# but an example in any language would help.

I haven't been able to work on this much. Still working on the logistics and politics of getting the application adopted in the workplace.

MArk B.

[havlatm]

Guys,
I will share your work for other users if you send it to me or attach via tracker. Thanks.

[rinmar]

package org.testlink.api.client.sample;

import java.net.MalformedURLException;
import java.net.URL;
import java.util.ArrayList;
import java.util.Date;
import java.util.Hashtable;
import java.util.Map;

import org.apache.xmlrpc.XmlRpcException;
import org.apache.xmlrpc.client.XmlRpcClient;
import org.apache.xmlrpc.client.XmlRpcClientConfigImpl;

public class TestlinkAPIXMLRPCClient
{
// Substitute your Dev Key Here
public static final String DEV_KEY = "xxxxxxxx";
// Substitute your Server URL Here
public static final String SERVER_URL = "http://xxxxx/testlink/lib/api/xmlrpc.php";

/**
* report test execution to TestLink API
*
* @param int tcid
* @param int tpid
* @param String status
*/
public static void testLinkReport(int tcid, String status)
{
try
{
XmlRpcClient rpcClient;
XmlRpcClientConfigImpl config;

config = new XmlRpcClientConfigImpl();
config.setServerURL(new URL(SERVER_URL));
rpcClient = new XmlRpcClient();
rpcClient.setConfig(config);

//////////////////////////
// Get available projects.
//////////////////////////

ArrayList<Object> params = new ArrayList<Object>();
Hashtable<String, Object> executionData = new Hashtable<String, Object>();
executionData.put("devKey", DEV_KEY);
params.add(executionData);

Object[] result = (Object[]) rpcClient.execute("tl.getProjects", params);

int idProject = -1;

// Typically you'd want to validate the result here and probably do something more useful with it
System.out.println("Result was:\n");
for (int i = 0; i < result.length; i++)
{
Map item = (Map)result;
System.out.println("Keys: " + item.keySet().toString() + " values: " + item.values().toString());
if (item.get("name").equals("xxxx")) {
idProject = Integer.parseInt((String)item.get("id"));
}
}

if (idProject == -1)
return;

/////////////////////////////////////////////
// Get available test plans in xxx project.
// DOES ONLY WORK WHEN ALL TEST PLAN NAMES DO
// NOT CONTAIN ANY SPECIAL CHARS LIKE UMLAUTS
/////////////////////////////////////////////

params = new ArrayList<Object>();
executionData = new Hashtable<String, Object>();
executionData.put("devKey", DEV_KEY);
executionData.put("testprojectid", idProject);
params.add(executionData);

result = (Object[]) rpcClient.execute("tl.getProjectTestPlans", params);

int idTestPlan = -1;

// Typically you'd want to validate the result here and probably do something more useful with it
System.out.println("Result was:\n");
for (int i = 0; i < result.length; i++)
{
Map item = (Map)result;
System.out.println("Keys: " + item.keySet().toString() + " values: " + item.values().toString());

for (Object key : item.keySet()) {
Object element = item.get(key);
if (element instanceof Map == false)
continue;

if (((Map)element).get("name").equals("Szenario")) {
idTestPlan = Integer.parseInt((String)((Map)element).get("id"));
}
}
}

if (idTestPlan == -1)
return;

////////////////////
// Create new build.
////////////////////

params = new ArrayList<Object>();
executionData = new Hashtable<String, Object>();
executionData.put("devKey", DEV_KEY);
executionData.put("testplanid", idTestPlan);
executionData.put("buildname", "Automatischer Test am " + (new Date()).toString());
executionData.put("buildnotes", "Notizen...");
params.add(executionData);

result = (Object[]) rpcClient.execute("tl.createBuild", params);

int idBuild = -1;

// Typically you'd want to validate the result here and probably do something more useful with it
System.out.println("Result was:\n");
for (int i=0; i< result.length; i++)
{
Map item = (Map)result;
System.out.println("Keys: " + item.keySet().toString() + " values: " + item.values().toString());

if (((Boolean)item.get("status")) == true) {
idBuild = Integer.parseInt((String)item.get("id"));
}
}

if (idBuild == -1)
return;

// Report test case result in new build.
params = new ArrayList<Object>();
executionData = new Hashtable<String, Object>();
executionData.put("devKey", DEV_KEY);
executionData.put("testcaseid", tcid);
executionData.put("testplanid", idTestPlan);
executionData.put("buildid", idBuild);
executionData.put("status", status);
executionData.put("notes", "testtest");
params.add(executionData);

result = (Object[]) rpcClient.execute("tl.reportTCResult", params);

// Typically you'd want to validate the result here and probably do something more useful with it
System.out.println("Result was:\n");
for (int i=0; i< result.length; i++)
{
Map item = (Map)result;
System.out.println("Keys: " + item.keySet().toString() + " values: " + item.values().toString());
}
}
catch (MalformedURLException e)
{
e.printStackTrace();
}
catch (XmlRpcException e)
{
e.printStackTrace();
}
}

public static void main(String[] args)
{
// Substitute this for a valid tcid and tpid within your project
TestlinkAPIXMLRPCClient.testLinkReport(4, "b");
}
}

[rinmar]

The next idea is to create a new status like "in work"
which we want to set from automatic. testrun is it still running..

I hope in future we will have some workflow engine behind testlink, because i miss the Start + Stop of a execution to get needed time information. With the new status we can calculate the time between passed - in work to get some feeling how long a test takes... (forecast for next builds...)

[naveentiru]

Hi rinmar,

Could you please help me in using XMLRPC in TL with java example.

Please give me the details from basics.

Thanks,
Naveen

[manzat]

hello !
when I run your code I get an error message:

Result was:

Exception in thread "main" java.lang.NullPointerException
Keys: [message, code] values: [Can not authenticate client: invalid developer key, 2000]
at org.testlink.api.client.sample.TestlinkAPIXMLRPCClient.testLinkReport(TestlinkAPIXMLRPCClient.java:61)
at org.testlink.api.client.sample.TestlinkAPIXMLRPCClient.main(TestlinkAPIXMLRPCClient.java:172)

[andredecotia]

I got a problem (a Java exception) executing that procedure above...

Error:
org.apache.xmlrpc.XmlRpcException: Failed to create input stream: http://10.50.18.188/lib/api/xmlrpc.php
at org.apache.xmlrpc.client.XmlRpcSunHttpTransport.getInputStream(XmlRpcSunHttpTransport.java:65)
at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:141)
at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:94)
at org.apache.xmlrpc.client.XmlRpcSunHttpTransport.sendRequest(XmlRpcSunHttpTransport.java:44)
at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:53)
at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:166)
at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:157)
at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:146)
at teste.TestlinkAPIXMLRPCClient.testLinkReport(TestlinkAPIXMLRPCClient.java:48)
at teste.TestlinkAPIXMLRPCClient.main(TestlinkAPIXMLRPCClient.java:170)
Caused by: java.io.FileNotFoundException: http://10.50.18.188/lib/api/xmlrpc.php
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)
at org.apache.xmlrpc.client.XmlRpcSunHttpTransport.getInputStream(XmlRpcSunHttpTransport.java:63)
... 9 more
Caused by:
java.io.FileNotFoundException: http://10.50.18.188/lib/api/xmlrpc.php
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)
at org.apache.xmlrpc.client.XmlRpcSunHttpTransport.getInputStream(XmlRpcSunHttpTransport.java:63)
at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:141)
at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:94)
at org.apache.xmlrpc.client.XmlRpcSunHttpTransport.sendRequest(XmlRpcSunHttpTransport.java:44)
at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:53)
at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:166)
at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:157)
at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:146)
at teste.TestlinkAPIXMLRPCClient.testLinkReport(TestlinkAPIXMLRPCClient.java:48)
at teste.TestlinkAPIXMLRPCClient.main(TestlinkAPIXMLRPCClient.java:170)