Tuesday, December 15, 2009

Simple C# Database Application Unit Test

I promised a way to unit test Database driven applications.  I've read that it is so difficult to do this and it is so hard to maintain.  The reason is that those people try to couple unit tests and the database.  Separate them!  Don't keep a unit testing database around because it will be out of date sooner than you think, tests will begin to fail and you will not continue to create / update those unit tests.

How do you separate the database from the application?  Capture the data.  Code is worth more than words, so here's my sample database app:


public void DoStuff()
{
    DataSet myData = GetMyData();

    foreach (DataRow dr in myData.Tables[0].Rows)
    {
        Console.WriteLine("Data: " + dr[0] + "," + dr[1]);
    }
}
It's too coupled with the database, so it's way too complicated to unit test reliably.  Step 1, capture the data and get out. Step 2, use the data you provide in the unit test or get it from the database.  Here's what your application should look like:

public bool captureData = true;
public DataSet myData = null;
public void DoStuff2()
{
    if( myData == null )
        myData = GetMyData();

    if (captureData)
    {
        myData.WriteXmlSchema("c:/myData.DataSet.schema.xml");
        myData.WriteXml("c:/myData.DataSet.xml");
        return;
    }

    foreach (DataRow dr in myData.Tables[0].Rows)
    {
        Console.WriteLine("Data: " + dr[0] + "," + dr[1]);
    }
}
And here's what our unit test will look like:


[TestMethod()]
public void DoStuff2Test()
{
    DataSet unitTestMyData = new DataSet();
    unitTestMyData.ReadXmlSchema("c:/myData.DataSet.schema.xml");
    unitTestMyData.ReadXml("c:/myData.DataSet.xml");

    MyDatabaseApp.Program p = new MyDatabaseApp.Program();
    p.myData = unitTestMyData;
    p.DoStuff2();

    Assert.AreEqual(2, p.myData.Tables["thetable"].Rows.Count, "Need 2 rows of this data for this reason");
}


Make sure you read the schema before the xml data otherwise all your columns will be strings.  I told you this was a lot easier than people make it out to be.  Make sure you begin writing unit tests for your all your Database driven appliactions, C#, VB.Net, doesn't matter... do it!

Monday, October 5, 2009

Visual Studio First Unit Test

Well, I decided to go with the built in unit test that Visual Studio 2008 provides simply because of the integration and everyone has it available without any installs.  The whole point is to just working with unit tests.  The lessons learned can be applied to any unit test software like NUnit.  So I started out with a simple method that does the same thing Math.Pow() does, raises a decimal to a power, returning a decimal.  (MyMath.cs)
public static class MyMath
{
  public static decimal MyPower(decimal num, int power)
  {
    int i = 0;
    decimal retval = 1;

    while (i < power)
    {
      retval *= num;
      i++;
    }

    return retval;
  }
}
After creating the method, i select the Test menu in Visual Studio and New Test....  In the Add New Test dialog box, I select the Unit Test template and name it TestPower.cs, and select "Create a new Visual C# test project...".  I click OK and name the new project TestProjectSimpleUnitTest and click Create.

I am presented with a new project TestProjectSimpleUnitTest that contains TestPower.cs.  Scrolling to the end of that file, I see:
[TestMethod]
public void TestMethod1()
{
      //
      // TODO: Add test logic    here
      //
}
I rename that method to TestMyPower() and delete the comments.  I right click the TestProjectSimpleUnitTest project and set it as the Startup Project.  I then press F5 to start the unit test and see that it passed.

Now, lets actually test MyPower.  First thing we have to do is add a reference to the SimpleUnitTest project in the TestProjectSimpleUnitTest project.  Add a using clause at the top of the TestPower.cs file:
using SimpleUnitTest;
and fill in my test method like this:
[TestMethod]
public void TestMethodMyPower()
{
  Assert.AreEqual(9m, MyMath.MyPower(3, 2), "3 to power of 2");
  Assert.AreEqual(1m, MyMath.MyPower(3, 0), "3 to power of 0");
  Assert.AreEqual(27m, MyMath.MyPower(3, 3), "3 to power of 3");
}
All my tests pass in this case.  But, supposing they didn't it's easy to set a breakpoint in the MyPower code or in the unit test code and select debug in the Test Results pane and debug your code in a controlled fashion that does not impact your project using MyPower!

Next time I'm going to get into the real good stuff... creating unit test for a database driven application!

Thursday, October 1, 2009

Discovering the importance of Unit Tests

I used to never create unit tests for the code I'd write.  Why?  Deadlines.  Get it done, test it, send it to QA, have defects reported, rinse and repeat.  I became tired of that so I pressed my boss to give me a little more time in development so that I can create formal unit tests for my code and send stronger code to QA.  It took a littler longer to do than I'd hoped, so that I wasn't too thrilled about.  However, the results were astounding!  All programmers out there, you have to start creating unit tests for your code!  NOW!

The application I'm working on takes data in from multiple databases, processes it and writes data back out to the databases.  I had to overcome database dependency for my unit tests.  Nothing worse than getting a new database instance for development and discover 60 unit tests break.

Second obstacle probably the biggest benefit for development and that was what happens when QA finds a defect?  You need to recreate the defect.  How long does that take?  Minimum a half hour?  Sometimes days?  To recreate the defect, you need the data they were using.  Sure, you can setup your database the way QA says it was, but was it really setup exactly the way they say?  Come on, we all have that doubt in the back of our heads because it's bitten us before.  What can you possibly do to alleviate this headache?  Aren't your unit tests database independent?  So, capture the data and make a unit test out of it!

I've had a few occasions when I had a defect that had a simple one liner solution.  Out of 230+ unit tests, the fix broke about 40 or more unit test!  Would QA catch it?  During regression testing, sure!  If there were moving on like usual, no, they would have not caught those scenarios.

I'll post my experiences with unit testing and integration into Visual Studio 2008 using Resharper soon.

Here is how to create a simple Database Application Unit Test [CodingCramp.blogspot.com].

Sunday, June 28, 2009

C# BackgroundWorker

From time to time, we all create GUIs for our applications. One of the most annoying things about a GUI is when it locks up, or gives the appearance of locking up. The quickest solution for this problem is Application.DoEvents().

That's fine for something that is looping quickly, but there is a better solution. It's easy to fix this problem with the C# BackgroundWorker Class. What it does is create another thread for you to do long running tasks on that would slow down your GUI.

In Visual Studio, create a new C# Windows Forms project. On the form, select from the Toolbox, under the Components Section, the BackgroundWorker object and drop it on the form. In the grey area below the form, you will see backgroundWorker1. Double click it to create the DoWork() method. This is where you will put your long running task.

Now, when you need to update, or touch, any UI component like a ProgressBar or TextBox, you cannot do that from a BackgroudWorker thread. Let's say we need to append a line of text to a textbox from our background thread. Let's create a method to do this:


private void AppendTextToDebugWindow(string textToAppend)
        {
            textBox1.AppendText(textToAppend + Environment.NewLine);
        }
Now, if you call this from your backgroundworker, you will get an InvalidOperationException - Cross-thread operation not valid: Control 'textBox1' accessed from a thread other than the thread it was created on.

This is pretty easy to fix. Create a delegate with the same signature as the above method. In our case:
delegate void AppendTextToDebugWindowHandler(string textToAppend);
Then, in the AppendTextToDebugWindow, change it to look like this:

private void AppendTextToDebugWindow(string textToAppend)
        {
            if (textBox1.InvokeRequired)
            {
                this.Invoke(new AppendTextToDebugWindowHandler(this.AppendTextToDebugWindow), new object[] { textToAppend });
                return;
            }
            textBox1.AppendText(textToAppend + Environment.NewLine);
        }
See! I told you this was easy! Here's is the sample Form1 that I used:

using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Linq;
using System.Text;
using System.Windows.Forms;
using System.Threading;

namespace TestBackgroundWorker
{
    delegate void AppendTextToDebugWindowHandler(string textToAppend);

    public partial class Form1 : Form
    {
        public Form1()
        {
            InitializeComponent();
        }

        private void button1_Click(object sender, EventArgs e)
        {
            // Don't forget to check if the background worker is busy
            if (!backgroundWorker1.IsBusy)
            {
                backgroundWorker1.RunWorkerAsync();
                return;
            }
        }

        private void backgroundWorker1_DoWork(object sender, DoWorkEventArgs e)
        {
            AppendTextToDebugWindow("backgroundWorker1 started");
            Thread.Sleep(5000);
            AppendTextToDebugWindow("backgroundWorker1 done");
        }

        private void AppendTextToDebugWindow(string textToAppend)
        {
            if (textBox1.InvokeRequired)
            {
                this.Invoke(new AppendTextToDebugWindowHandler(this.AppendTextToDebugWindow), new object[] { textToAppend });
                return;
            }
            textBox1.AppendText(textToAppend + Environment.NewLine);
        }
    }
}

Thursday, June 18, 2009

C#: How to setup and use TransactionScope

Many of us developers have used SqlTransaction to ensure data integrity. But, SqlTransaction works on only one database. You can get it to work across multiple databases buy using the Save() method, but it gets ugly. There is a much better solution, TransactionScope. TransactionScope uses DTC (aka MSDTC) to manage the Sql Transactions and promoting transactions from Lightweight Transaction Manager (LTM) to full DTC.

Let’s see what we need to do to get SqlTransaction to work across multiple databases:


public void DoWorkSqlTransactions()
{
using (SqlConnection con1 = new SqlConnection("my connection string"))
using (SqlConnection con2 = new SqlConnection("my second connection string"))
{
try
{
con1.Open();
con2.Open();
SqlTransaction tran1 = con1.BeginTransaction();
SqlTransaction tran2 = con2.BeginTransaction();

try
{
SqlCommand cmd1 = new SqlCommand("update ...", con1, tran1);
cmd1.ExecuteNonQuery();

// Don't want select in transaction
cmd1 = new SqlCommand("select ...", con1);

SqlCommand cmd2 = new SqlCommand("insert ...", con2, tran2);
cmd2.ExecuteNonQuery();
tran1.Save("savepoint");
tran2.Save("savepoint");
tran1.Commit();
tran2.Commit();
}
catch (Exception)
{
tran1.Rollback();
tran2.Rollback();
}
}
catch (Exception)
{
// error handling for connection failure
}
finally
{
con1.Close();
con2.Close();
}
}
}

Now, try to do that with 5 or more databases at the same time! And try passing those SqlTransaction objects around! Really, really ugly code. Let’s look at the same code, but with TransactionScope:


public void DoWorkSqlTransactionScope()
{
try
{
using (TransactionScope ts = new TransactionScope(TransactionScopeOption.RequiresNew))
using (SqlConnection con1 = new SqlConnection("my connection string"))
using (SqlConnection con2 = new SqlConnection("my second connection string"))
{
con1.Open();
con2.Open();

SqlCommand cmd1 = new SqlCommand("update ...", con1);
cmd1.ExecuteNonQuery();

// Suppress the select statement from the transaction:
using (TransactionScope ts2 = new TransactionScope(TransactionScopeOption.Suppress))
{
cmd1 = new SqlCommand("select ...", con1);
cmd1.ExecuteReader(); // and so on...
}

SqlCommand cmd2 = new SqlCommand("insert ...", con2);
cmd2.ExecuteNonQuery();

ts.Complete();
}
}
catch (Exception)
{
// whole transaction was rolled back automatically
}
}
As you can see, any call to the db made within the TransactionScope is automatically enrolled in a transaction. This is true even if the call is made in another class, or external dll! Think of what that can do for you…. If you don’t want to include something, suppress it like I did in the above example.

To use TransactionScope, you need to configure the machine you are developing on and the machine the application will be deployed on for DTC.

Configuring DTC is pretty simple to get working, and a little more work to secure. Follow this guide to configure it for Vista and Server 2008: http://technet.microsoft.com/en-us/library/cc753510(WS.10).aspx

For most other Windows Operating Systems, the short of it is run dcomcnfg for Component Services. Expand Component Services, Computers. Right click My Computer in the left pane, select Properties. Open the MSDTC tab. Select Security Configuration button. Check all the check boxes and select the “No Authentication Required” radio button. It may suggest to restart the DTC service, which is fine, do it. DTC may need to be configured on your Sql Server, too.

Now that the OS is configured for DTC, now we have to setup our project. Just add a reference for System.Transactions to the project, and a using clause for System.Transactions to the file where the class wrapping all the sql calls is, even if it is the ui or business layer. At the end of the using clause, call Complete() on the TransactionScope variable to commit the all the transactions.

Wednesday, June 17, 2009

C#: Simple Threading Example Using ThreadPool

With machines now being sold with multiple cores, why should your application be using only one? Harness the full potential of the processor(s) and make your app shine!

Threading seems so complicated, it really isn’t. Today, I’ll provide an example of the simplest thread code you’ve ever seen. In fact, it’s so simple, here it is:

class TheTest
{
public void Go()
{
for (int i = 0; i < 100; i++)
{
WaitCallback wcb = new WaitCallback(this.DoWork);
ThreadPool.QueueUserWorkItem(wcb, i);
}
Console.ReadLine();
}

public void DoWork(object state)
{
int i = (int)state;
Console.WriteLine(string.Format("{0:d3} started",i));
Thread.Sleep(500);
Console.WriteLine(string.Format("{0:d3} done", i));
}
}

Create a new instance of the TheTest class, call go. You’ll see 100 threads get created and do work. You can pass in anything you want for the state. That’s a very good place to put your “unit of work” that the thread must work on. Just make sure you add a using clause for System.Threading.

Of course, this post doesn’t touch on any locking or blocking issues, I’ll do that later. But for now, why not multithread?

Linq Performance

I needed to clear all the textboxes in a windows form without specifying each and every textbox. Easy enough to do in C#:



foreach (Control c in Controls)
{
if (c is TextBox)
((TextBox)c).Clear();

if (c is ListBox)
((ListBox)c).Items.Clear();

if (c is Panel)
{
foreach (Control c2 in c.Controls)
{
if (c2 is TextBox)
{
((TextBox)c2).Clear();
}
}
}
}

To me, that just seemed expensive. I haven't had a chance to play with Linq, until now. So I came up with the Linq equivilant:



foreach (TextBox tb in from Control control in Controls
where control is TextBox
select (TextBox)control)
{
tb.Clear();
}

foreach (ListBox lb in from Control control in Controls
where control is ListBox
select (ListBox)control)
{
lb.Items.Clear();
}

foreach (TextBox tb in from Control control in panel1.Controls
where control is TextBox
select (TextBox)control)
{
tb.Clear();
}

For me, this is actually easier to read! OK, but performance must be horrible, but I'm not one to judge without an actual test. I used the System.Diagnostics.Stopwatch class like this:



System.Diagnostics.Stopwatch stopwatch = new System.Diagnostics.Stopwatch();
stopwatch.Start();
// Do stuffstopwatch.Stop();
MessageBox.Show(string.Format("linq: {0:#,0}", stopwatch.ElapsedTicks));


I had some very interesting numbers! For a form with about 25 controls, I got these numbers:
Linq averaged 3,700,000 ticks on first run
Foreach averaged 4,400,000 ticks on first run


Clicking the Clear button a second time resulted in faster times for both Linq and Foreach, probably due to a form of caching:
Linq averaged 333,000 ticks on subsequent runs
Foreach averaged 1,010,000 ticks on subsequent runs

So, Linq must be optimized for ControlCollection, which is derived from ArrangedElementCollection, which implements IList, ICollection, IEnumerable, ICloneable. I'll have to do more investigation to see where foreach is faster than Linq and post that result later.

Tuesday, June 16, 2009

C# String.Format("format", arguments)

This is my personal reference for String.Format().

Here are the quick reference tables.  Below are the explanations.


Format

Int 1234

Decimal

1234.567

Decimal

0.1234567

Decimal
1234.5

0:c

$1,234.00

$1,234.57

$0.12

$1,234.50

0:d

1234

-

-

-

0:e

1.234000e
+003


1.234567e+003

1.234567e-001

1.234500e+003

0:f

1234.00

1234.57

0.12

1234.50

0:g

1234

1234.567

0.1234567

1234.5

0:n

1234.00

1,234.57

0.12

1,234.50

0:p

123,400.00
%


123,456.70
%


12.35 %

123,450.00
%


0:r

-

-

-

-

0:x

4d2

-

-

-

0:X

4D2

-

-

-

Padding/Precision:







0:d5

01234

-

-

-

0:e5

1.23400e+003

1.23457e+003

1.23457e-001

1.23450e+003

0:n5

1,234.00000

1,234.56700

0.12346

1,234.50000

0:c5

$1,234.00000

$1,234.56700

$0.12346

$1,234.50000

0:x5

004d2

-

-

-


Using: DateTime.Parse("December
13, 2008 5:01:30.7770 pm"
);



Format

DateTime (above date)

Jan
5 (rest same)


Description

0:D

Saturday, December
13, 2008


Wednesday, January
09, 2008


Long Date

0:F

Saturday, December
13, 2008 5:01:30 PM


Wednesday, January
09, 2008 5:01:30 PM


Full Date/Time
(long time)


0:G

12/13/2008 5:01:30
PM


1/9/2008 5:01:30 PM

General Date/Time
(long time)


0:M

December 13

January 09

Month Day

0:O

2008-12-13T17:01:30.7770000

2008-01-09T17:01:30.7770000

Round-Trip
Date/Time


0:R

Sat, 13 Dec 2008
17:01:30 GMT


Wed, 09 Jan 2008
17:01:30 GMT


RFC1123

0:T

5:01:30
PM


5:01:30
PM


Long
Time


0:U

Saturday, December
13, 2008 10:01:30 PM


Wednesday, January
09, 2008 10:01:30 PM


Universal Full
Date/Time (long time)


0:Y

December, 2008

January, 2008

Year Month

0:d

12/13/2008

1/9/2008

Short Date

0:f

Saturday,
December 13, 2008 5:01 PM


Wednesday,
January 09, 2008 5:01 PM


Full Date/Time
(short time)


0:g

12/13/2008 5:01 PM

1/9/2008 5:01 PM

General Date/Time (short time)

0:m

December
13


January
09


Month
Day


0:o

2008-12-13T17:01:30.7770000

2008-01-09T17:01:30.7770000

Round-Trip Date/Time

0:r

Sat, 13
Dec 2008 17:01:30 GMT


Wed, 09
Jan 2008 17:01:30 GMT


RFC1123

0:s

2008-12-13T17:01:30

2008-01-09T17:01:30

Sortable Date/Time

0:t

5:01 PM

5:01 PM

Short
Time


0:u

2008-12-13 17:01:30Z

2008-01-09 17:01:30Z

Universal Sortable
Date/Time


0:y

December,
2008


January,
2008


Year
Month




Useful Examples


Format

Use

Output

0:000-00-0000

SSN

012-34-5678

0:(###)###-####

US Phone Number

(555)555-5555

0:#-(###)###-####

US Phone
Number


1-(555)555-5555

0:1-(###)###-####

US Phone Number

1-(555)555-5555



Formatting DateTime objects with DateTime.ToString("Format Here"):


Format

Description

d

Non-padded
day


dd

Zero padded day

ddd

Short day
name (Tue)


dddd

Full day name (Tuesday)

h

Non-padded hour 12
hour format


hh

Zero padded hour 12
hour format


H

Non-padded hour 24
hour format


HH

Zero padded hour 24
hour format


m

Non-padded minute

mm

Zero padded minute

M

Non-padded Month

MM

Zero padded Month

MMM

Short month name
(Jun)


MMMM

Full month name
(June)


s

Non-padded seconds

ss

Zero-padded seconds

tt

AM/PM

yy

2 digit year

yyyy

4 digit year

:

As is

/

As is

MM/DD/yyyy

Sample: 06/14/2008

You can customize the above with Number Customizations:

# is a digit place holder.

0 is a zero place holder.

. is a printed decimal point.

, is a printed comma.

% means use the number as a percentage where 1 is 100%.

If you want to have the actual number be the percentage, prefix the % with a '.
Example:

String.Format("{0:##.00'%",
12.34); // 12.34%


To do left and right padding:

String.Format("{0,-9}",
1234); // “1234


String.Format("{0,9}", 1234); // “     1234“


The Basics

How to call String.Format(): String.Format("{ArgumentPosition,Padding}",
arguments[] or argument,argument…);
 

String.Format("{0} {1}",
"Hello", "World");

Prints: Hello World

{0} and {1} are the positions of the arguments “Hello” and “World” specifying
where in the string they are top be used.

String.Format("First
Number: {0}, Third Number: {2}, Second Number: {1}."
, 1,2,3);

Prints:
First Number: 1, Third Number: 3, Second Number: 2.


You do not have to call ToString() on your arguments, String.Format()
is smart enough to do that for us. Suppose you created your own Class called
Currency, which overrides ToString(). When your instance of Currency has a value of $19.95,
you can use:

String.Format("MyCurrency value:
{0}"
, myCurrency);


Prints: MyCurrency value: $19.95

Continue that thought just a little further... if you print DataRow information:

String.Format("Column
'ID': {0}"
, dr["ID"]);