How to make a basic test framework in C#
Unit testing can be an excellent approach to building applications. In C#, there is a fairly standard convention that you often see. You'll often see a test structure like this:
[TestClass]
public class UnitTest1
{
[TestInitialize]
public void Setup()
{
}
[TestMethod]
public void TestMethod1()
{
}
}
I want to simplify it a bit.
public class DemoUnitTests
{
public DemoUnitTests()
{
}
public void TestMethod1()
{
}
}
There are a few things to note here. Instead of a test class attribute, I changed the class name. The test framework will look for all classes that end with Tests. All methods in test classes that start with public void and do not have parameters will be treated as test methods. I don't have an explicit test set up method but do allow test set up to be run in the constructor. A new class instance will be used for every test.
A test will fail if an exception is thrown. Many of the existing Assert methods in test frameworks throw an exception. In order to keep this demo simple, I suggest using the Should Assertion Library.
I'm going to walk through the main parts of the test method. The entire class will be listed at the end.
To start, the assembly is passed in. Using reflection and the assembly, we can filter the matching classes. This just finds the types. We'll create instances later.
The return type of the method is IEnumerable of string. For simplicity sake, we will return a string whenever something interesting happens during the test run. That could be when there's a class error, a failed test or the test run has completed. Since we are yielding the string, we can be updated on problems as they happen.
public IEnumerable<string> RunAssemblyTests(Assembly assembly)
{
IOrderedEnumerable<Type> testClassTypes = assembly.GetExportedTypes()
.Where(t => t.IsClass
&& (t.Name.EndsWith("Test") ||
t.Name.EndsWith("Tests")))
.OrderBy(x => x.FullName);
...
We will go over each test class, make sure it is valid. Having a test class constructor with parameters is beyond the scope of this demo. Having it wired up to dependency injection would be an interesting variant.
foreach (var testClassType in testClassTypes)
{
if (testClassType.GetConstructor(Type.EmptyTypes) == null)
{
yield return
string.Format("Error: {0} does not have an empty constructor.",
testClassType.FullName);
continue;
}
...
Once we have the test class type, we can look for the methods that meet our criteria. When you get methods from a type, you can filter it. The DeclaredOnly flag will make sure you only return methods defined on that type. We don't want to test the inherited ToString(). Limiting the tests to public instance classes should make it easier to manage. You can create private & static methods that won't be run as a test.
IEnumerable<MethodInfo> testMethodInfoList =
testClassType
.GetMethods(BindingFlags.DeclaredOnly
| BindingFlags.Public
| BindingFlags.Instance)
.Where(m => m.ReturnType == typeof (void)
&& !m.GetParameters().Any());
Here the test is actually run. Since we use reflection to look through the assembly to find the test class types and methods, they have to be instantiated. Activator.CreateInstance is easiest for the class. MethodInfo.Invoke can call a class by the instance and parameters (if any). Both of them are put inside the try block so that an exception in the test or constructor is marked as a failed test. The yield return shows the problem right away. I'd rather not wait until the test run is completed.
foreach (MethodInfo methodInfo in testMethodInfoList)
{
...
try
{
var classInstance = Activator.CreateInstance(testClassType);
methodInfo.Invoke(classInstance, new object[] {});
}
catch (Exception ex)
{
...
}
if (!successful)
{
yield return
string.Format("----{0}Test failed: {1}.{2}{0}{3}{0}----",
Environment.NewLine,
testClassType.FullName,
methodInfo.Name,
methodExceptionMessage);
}
}
When it is completed with all the tests, we can just return the result summary.
yield return string.Format("{0} tests run. {1} tests failed.",
testsRun, testsFailed);
As an example, here is the test framework called by a console app. I just have to pass it what assembly to test. For demo purposes, I referenced the assembly of the current program. All the tests were included in there. Each test assembly would be manually referenced.
internal class Program
{
private static void Main(string[] args)
{
var stopWatch = new Stopwatch();
stopWatch.Start();
var tests = new TestConvention();
var assembly = Assembly.GetAssembly(typeof (Program));
var results = tests.RunAssemblyTests(assembly);
foreach (var result in results)
{
Console.WriteLine(result);
}
stopWatch.Stop();
Console.WriteLine(stopWatch.ElapsedMilliseconds*0.001 + " seconds");
}
}
Now to try it out. Here are a few tests.
public class DemoTests
{
public DemoTests()
{
}
public void EmptyTest()
{
}
public void SuccessfulAdditionTest()
{
(1 + 1).ShouldEqual(2);
}
public void FailingAdditionTest()
{
(1 + 1).ShouldEqual(5);
}
public void FailingContainsTest()
{
"TestString".ShouldNotContain("Test");
}
}
Now that there is a working example, how could this be changed? The most obvious example would be to add special method names for setting up and cleaning up the test cases. Then you could switch the constructor to be include per-class setup logic. I avoided that for demo simplicity.
Another area for improvement would be integration with Visual Studio. Visual Studio 2012 has unit test hooks that might be useful here. If you don't want that, you could wrap the entire test run in a single MSTest or NUnit test method.
Full TestConvention class:
public class TestConvention
{
public IEnumerable<string> RunAssemblyTests(Assembly assembly)
{
IOrderedEnumerable<Type> testClassTypes = assembly.GetExportedTypes()
.Where(t => t.IsClass
&& (t.Name.EndsWith("Test") ||
t.Name.EndsWith("Tests")))
.OrderBy(x => x.FullName);
int testsRun = 0;
int testsFailed = 0;
foreach (var testClassType in testClassTypes)
{
if (testClassType.GetConstructor(Type.EmptyTypes) == null)
{
yield return
string.Format("Error: {0} does not have an empty constructor.",
testClassType.FullName);
continue;
}
IEnumerable<MethodInfo> testMethodInfoList =
testClassType
.GetMethods(BindingFlags.DeclaredOnly
| BindingFlags.Public
| BindingFlags.Instance)
.Where(m => m.ReturnType == typeof (void)
&& !m.GetParameters().Any());
foreach (MethodInfo methodInfo in testMethodInfoList)
{
testsRun++;
string methodExceptionMessage = "";
bool successful = true;
try
{
var classInstance = Activator.CreateInstance(testClassType);
methodInfo.Invoke(classInstance, new object[] {});
}
catch (Exception ex)
{
methodExceptionMessage = ex.InnerException.Message;
successful = false;
testsFailed++;
}
if (!successful)
{
yield return
string.Format("----{0}Test failed: {1}.{2}{0}{3}{0}----",
Environment.NewLine,
testClassType.FullName,
methodInfo.Name,
methodExceptionMessage);
}
}
}
yield return string.Format("{0} tests run. {1} tests failed.",
testsRun, testsFailed);
}
}