TestLink Automation Configuration

[Matt]

Hi All,

Apologies if this question has been asked before, I've looked around the forum and couldn't find the answer to my question.

I am running TestLink 1.9, and I have several Test Suites within the database. We are undertaking a process of automating some of these tests, using QTP. I know that within TestLink there is an option to run each test case individually using automation, utilising the XML-RPC interface, but this does not meet our automation requirements.

What we would like is to be able to initiate an automation test sweep from a top-level Test Suite, for the results to be fed back into TestLink and then stored in the database, with everything visible in the reports as if it were a manual sweep.

The path I think we would be looking at is:- TestLink --> XML-RPC --> QTP --> XML-RPC --> TestLink.

Has anybody implemented a similar system before? Within the department we have knowledge of VB.net, C#, VbS, Python, etc, as well as PhP.

Regards,
Matt

[jack090]

Hey Matt, I am currently working on a similar in house project for my company using selenium, sahi, and Windows UI Automation to be kicked off from TestLink. It's all written in C# (and basically uses command line exec's) and I made a small edit to \lib\execute\execSetResults.php to basically insert a record into a custom DB I setup that manages our selenium grid etc.. We have successfully kicked off around 20 automated tests simultaneously. I'm working on a big blog post that encompasses a lot of the work. I'm working on cloud integration so we could potentially run 1000's of tests simultaneously and/or just upping the amount of clients to our grid. I'd say about a week more and I will be done with the framework. If QTP can take command line arguments then it should probably work from my framework. Only a small piece of my edits touch TL, but after any of my tests complete I am using XMLRPC of Testlink to record the notes etc... Feel free to message me to remind me to post details once I finish this week. (Always busy here). Cheers~ Jack

[mtalame]

Hi Matt,

I am also interested in your solution. Any updates?

Regards,
Toufic

[Matt]

None as yet Toufic. I messaged Jack and didn't hear back. Jack?

[Amaradana]

long time back i also done for couple of people who are using this, but i havent heard back from them. Matt, please share the information if you have any

[jack090]

Hey guys sorry it's been so long. Our project for "simple" automation has morphed into a gigantic testing suite at my office. I currently have TL as our testing repository where all of our build quality results are stored and used for reporting/ test execution. Here's the simple version of what I have done:

A user executes a test and a insert record is put into a custom SQL DB. We have a dropdown that allows the user to specify which environment it will run against on the TestCase in TL. This is great because we can have developers run the test on their local box before it gets checked into our source control. We use Bamboo as our source repository so we are currently working on automatic deployment (and automated testing) which will run against the build outside of TL (although results are stored in TL). I have cannibalized Selenium Grid technology and ramped up 5 machines able to run 10 "threads" each. (we have the ability run against cloud space such as saucelabs/amazon) This is all handled by a controller which load balances the tests based on each servers available threads. Threads can be set aside for certain users, uses, debugging, and etc... The controller will spawn the runner will email the results of the test to whoever kicked the test off... In some of the emails I have screenshots being sent as attachments (or files if needed). I have a parent/child table in SQL that allows you to have a parent Test Case that will spawn all of it's children. Once the children all run and complete the Parent will update with a pass/fail status in TestLink.

We are currently under a big push to start automating more of our test cases so I've been busy helping our QA team write the tests. I've made a custom toolbox which wraps all of the selenium methods (nobody calls straight selenium code) however the testers can record in selenium. All they have to do is run a find and replace to remove "selenium." from their tests. They then drop that into the source control and build the .dll(s). Once the .dll is compiled we can drop it on the controller server and the new test is executed. I even have where they can write simple sql commands to find values and use them for dynamic tests. I've got about 2-3 months of full-time work on this and in the process of documenting it for my team. The really neat thing is that it isn't tied to selenium, as I have automation using selenium 2.0 beta, Sahi, and Microsoft's UI Automation (which allows control over any program that runs on windows). Depending on the test we can run any web browser we want (selenium) and check for compatability. The project has had a lot of success so now our VP is pushing our dev team, BA's and testing team to using it full time. To think we almost bought QTP/Quality Center when we now have a custom solution that can automate just about anything (including windows apps). Once the documentation process is in a semi-good state on my side I will try and get a simple version of the documentation/source code out there for people to use. Luckily I get paid to work on this project and we are not entirely opposed to turning this over to the open source community. Feel free to message me.

[jack090]

http://www.4shared.com/dir/3l5Muo4J/_online.html This is a pdf of our architecture. I'll post more when I get it.

[Matt]

Jack, that sounds rather good. We've gone a different route, we're using QTP and WinRunner (a holdover from previous 'regimes' if you like) and we're not having a great deal of success. We can get tests queued for Automation from TestLink, and the results passed back again, using an API which we wrote in-house, but our actual automation isn't where we want it to be. Sadly I don't know too much about the technical side of it (I did the integration into TestLink: Someone else wrote the API's and did all the fiddly bits), but I might be able to find out if anybody thinks my method would be preferable to Jacks....?

[Mohammad]

Can you send us a sample of the in-house api please?

we have been successfully using a product called TestShell that has an integration with QC, but we would like to build such an integration with TL. seems like what you've been doing is pretty much what we need...

Mohammad

[jack090]

Can you send us a sample of the in-house api please?

we have been successfully using a product called TestShell that has an integration with QC, but we would like to build such an integration with TL. seems like what you've been doing is pretty much what we need...

Mohammad

Do you mean me?

[aostad]

Jack, any update on this topic?

[SThayalan]

I need guidance and notes to integrate Test Link with Test Studio automation tool on visual studio using C# code. Can someone guide me how to set it up so that if I invoke from Test Link it will run the test and update the results as required or the viceversa