C#: Chunking out work for worker threads and take advantage of multiple cores

The hardest part of adding threading to your application is deciding what to make your threads to.  You need to look for a unit of work that can be done independent of other processing.  In this case, we need to process every row of a DataTable and do some stuff.  I'll leave out the database calls, but I could be calling a stored procedure for every row and inserting records.  A lot of time would be wasted for each call to do it's round trip to the database, so a bunch of threads doing the calls and waiting will give us a huge burst of performance.

Say you have a DataTable, and you need to do some stuff with each row.  You can greatly improve the performance of your operation by creating worker threads.  It’s simple and easy to do!   Check out this simple example:

        private void button1_Click(object sender, EventArgs e)

        {

            // Setup of data for worker threads

            DataTable dt = new DataTable();

            dt.Columns.Add("id",typeof(int));

            dt.Columns.Add("value",typeof(string));

            for( int i = 0; i < 5000; i++ )

            {

                DataRow newRow = dt.NewRow();

                newRow["id"] = i;

                newRow["value"] = i + " text";

                dt.Rows.Add(newRow);

            }

            // Chunk out the work

            const int numberOfThreads = 4;

            int chunkSize = dt.Rows.Count / numberOfThreads;

            int currentRow = 0;

           

            for (int i = 0; i < numberOfThreads; i++)

            {

                List<DataRow> rowsToWorkOn = new List<DataRow>();

                int maxRowNumber = (currentRow + chunkSize);

                if (i == numberOfThreads)

                {

                    // Last chunk, don't run over the end of the datatable

                    maxRowNumber = dt.Rows.Count;

                }

                for (int j = currentRow; j < maxRowNumber; j++)

                {

                    rowsToWorkOn.Add(dt.Rows[j]);

                }

                currentRow += chunkSize;

                ThreadPool.QueueUserWorkItem(DoWork, rowsToWorkOn);

            }

        }

        public void DoWork(object state)

        {

            List<DataRow> rowsToWorkOn = (List<DataRow>)state;

            foreach (DataRow dr in rowsToWorkOn)

            {

                System.Diagnostics.Trace.WriteLine(dr["id"].ToString());

            }

        }