satis egitimisatis egitimitengda.pro

Open@Blog

Discussion on the state of cloud computing and open source software that helps build, manage, and deliver everything-as-a-service.

  • Home
    Home This is where you can find all the blog posts throughout the site.
  • Categories
    Categories Displays a list of categories from this blog.
  • Tags
    Tags Displays a list of tags that has been used in the blog.
  • Bloggers
    Bloggers Search for your favorite blogger from this site.
  • Login
Posted by on in CloudStack Tips
  • Font size: Larger Smaller
  • Hits: 13411
  • Print
  • Report this post

Data Driven Tests in Marvin

Data driven tests (DDT) is useful for generating more tests with less code. If you have lot of assertions to be repeated with related data sets or parameters, Data driven tests are a way to go - saves a lot of redundant code. The data sets provided as input will be used iteratively in the test scripts in effect, running different tests cases. I’ve recently tried to implement this with Marvin, CloudStack’s Integration Test framework which is written in Python. 

Before beginning to code test suites in data driven fashion, it’s best to understand the entities which drive the tests.  For example, we may want to test the same set of VM life cycle tests for different network offerings such as a simple isolated offering, an offering with persistent network feature and an offering with Netscaler as a service provider for LB. These offerings then become the data sets which are supplied as input to the test script. And the assertions for VM life cycle are repeated for each of these network offerings.

DDT in Python

Python provides a powerful and efficient way to achieve this with the ddt library. It comes with a class decorator @ddt and method decorators @data and @file_data. Use the class decorator with the TestCase class and specify the data sets with which the tests need to be run with the @data or @file_data method decorator. While @data takes in the arguments directly to be passed to the test, @file_data will load the data set from a JSON file.

CloudStack Test Case Example

Here’s an example:

Consider a case where we need to verify various network operations in a VPC with 2 different VPC offerings: Default VPC Offering and Default VPC Offering with NS as LB.

First, Install DDT (pip install ddt) if not already done.

More info here: http://ddt.readthedocs.org/en/latest/example.html

Then, start by defining the class decorator with the test class:

@ddt

class TestVPCNetwork(cloudstackTestCase):

Then define the data sets using the method decorator: 

@data("network_offering", "network_offering_vpcNS")

def test_01_create_network(self, value):

(where the network offerings are already defined prior to starting the tests)

 Result: test_01_create_network will be iterated for both the network offerings.

 

The complete test suite can be found here:

https://github.com/ksowmya/cloudstack-1/commit/67e8b3e43a8720d7887639498d609aebce8e2935

 

There’s another way of writing data driven tests in python viz. Generative tests. Nose (we use nose as the test runner with marvin) supports test generators. Detailed here: https://nose.readthedocs.org/en/latest/writing_tests.html#test-generators Using this, we can write test functions which are generators. But however, it doesn’t work if we are using unittest.TestCase subclasses, which is the case with marvin.

So, for now, I am sticking with ddt and hoping to implement this for all VPC tests and explore the possibility of using this with other test suites as well.

 

Rate this blog entry:
  • No comments made yet. Be the first to submit a comment

Leave your comment

Guest Monday, 01 September 2014

Open@Citrix

Citrix supports the open source community via developer support and evangeslism. We have a number of developers and evangelists that participate actively in the open source community in Apache Cloudstack, OpenDaylight, Xen Project and XenServer. We also conduct educational activities via the Build A Cloud events held all over the world. 

Connect