Smart Unit Testing With Visual Studio 2015

Smart Testing (former pex & moles ) has a new name now – IntelliTest and explores your .NET code to generate test data and a suite of unit tests. For every statement in the code, a test input is generated that will execute that statement. A case analysis is performed for every conditional branch in the code. For example, if statements, assertions, and all operations that can throw exceptions are analyzed. This analysis is used to generate test data for a parameterized unit test for each of your methods, creating unit tests with maximum code coverage. Then you bring your domain knowledge to improve these unit tests.

When you run IntelliTest, you can easily see which tests are failing and add any necessary code to fix them. You can select which of the generated tests to save into a test project to provide a regression suite. As you change your code, rerun IntelliTest to keep the generated tests in sync with your code changes.

Get started with IntelliTest

You must use Visual Studio Enterprise. Sad smile

Explore: Use IntelliTest to explore your code paths and generate test data
  1. Open your solution in Visual Studio. Then open the class file that has methods you want to test.
  2. Right-click in a method in your code and choose Run IntelliTest to generate unit tests for all the code paths in your method.

    Right-click in your method to generate unit tests

    A parameterized unit test is generated for this method. The test data is created to exercise the code paths in the method. IntelliTest runs your code many times with different inputs. Each run is represented in the table showing the input test data and the resulting output or exception.

    Exploration Results window is displayed with tests

    To generate unit tests for all the public methods in a class, simply right-click in the class rather than a specific method. Then choose Run IntelliTest. Use the drop-down list in the Exploration Results window to display the unit tests and the input data for each method in the class.

    Select the test results to view from the list

    For tests that pass, check that the reported results in the result column match your expectations for your code. For tests that fail, fix your code and add exception handling if necessary. Then rerun IntelliTest to see if your fixes generated more test data from different code paths.

Persist: Save test data and unit tests as a regression suite
  • Select the data rows that you want to save with the parameterized unit test into a test project.

    Select tests; right-click and choose Save

    You can view the test project and the parameterized unit test that has been created with a PexMethod attribute. (The individual unit tests, corresponding to each of the rows, are saved in the .g.cs file in the test project.) The unit tests are created using the Visual Studio test framework, so you can run them and view the results from Test Explorer just as you would for any unit tests that you created manually.

    Open class file in test method to view unit test

    Any necessary references are also added to the test project.

    If the method code changes, rerun IntelliTest to keep the unit tests in sync with the changes.

Assist: Use IntelliTest to find issues in your code
  1. If you have more complex code, IntelliTest can help you discover any issues for unit testing. For example, if you have a method that has an interface as a parameter and there is more than one class that implements that interface. After you run IntelliTest, warnings are displayed for this issue. View the warnings to decide what you want to do.

    Right-click method and choose Smart Unit Tests

     

  2. After you investigate the code and understand what you want to test, you can fix a warning to choose which classes to use to test the interface.

    Right-click the warning and choose Fix

    This choice is added into the PexAssemblyInfo.cs file.

    [assembly: PexUseType(typeof(Camera))]

     

  3. Now you can rerun IntelliTest to generate a parameterized unit test and test data just using the class that you fixed.

    Rerun Smart Unit Tests to generate the test data

Q & A

Q: Can you use IntelliTest for unmanaged code?

A: No, IntelliTest only works with managed code, because it analyzes the code by instrumenting the MSIL instructions.

Q: When does a generated test pass or fail?

A: It passes like any other unit test if no exceptions occur. It fails if any assertion fails, or if the code under test throws an unhandled exception.

If you have a test that can pass if certain exceptions are thrown, you can set one of the following attributes based on your requirements at the test method, test class or assembly level:

  • PexAllowedExceptionAttribute
  • PexAllowedExceptionFromTypeAttribute
  • PexAllowedExceptionFromTypeUnderTestAttribute
  • PexAllowedExceptionFromAssemblyAttribute

Q: Can I add assumptions to the parameterized unit test?

A: Yes, use assumptions to specify which test data is not required for the unit test for a specific method. Use the PexAssume class to add assumptions. For example, you can add an assumption that the lengths variable is not null like this.

PexAssume.IsNotNull(lengths);

If you add an assumption and rerun IntelliTest, the test data that is no longer relevant will be removed.

Q: Can I add assertions to the parameterized unit test?

A: Yes, IntelliTest will check that what you are asserting in your statement is in fact correct when it runs the unit tests. Use the PexAssert class to add assertions. For example, you can add an assertion that two variables are equal.

PexAssert.AreEqual(a, b);

If you add an assertion and rerun IntelliTest, it will check that your assertion is valid and the test fails if it is not.

Q: What testing frameworks does IntelliTest support?

A: Currently only mstest is supported.

Q: Can I learn more about how the tests are generated?

A: Yes, read this blog post about the model and process.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s